# SI + AI Recognized Subtitles

1. Quick implementation🚀
2. How does it work?
3. Step by step guide
4. Events
5. Live Demo
6. 🔹 React
7. 🔹 Vue
8. 🔹 Vanilla JavaSript
9. Properties

# Quick implementation 🚀

GO!

# How does it work?

The SI + AI Recognized Subtitles SDK is an slight variation of the Simultaneous Interpretation (SI) SDK:

• While the professional interpreters do the intepretation, we will get the voice recognition of their audio.
• The audience will be able to hear the interpretation and read the subtitles at the same time.

The SI + AI Recognized Subtitles SDK should be integrated in the audience page of your event allowing your audience to switch between the different languages of interpretation and AI Recognized Subtitles.

You will have to send the original stream to the interpreters using our dashboard or the REST API. The audio of the interpreters will automatically play in the SDK you integrated in your audience page.

Once your audience select a language the SDK can automatically mute your original stream or, you can catch the events that we send to decide when to mute the original stream.

🚨 Important:

• RsiInterpretationPlayer must be imported before rsiSubtitles.
  • RsiSubtitles cannot work without the widget RsiInterpretationPlayer
• In order to use the SDK with subtitles, you must set the parameter hasSubtitles to be true.

# Step by step guide

Note: The arrows below are dropdowns, click to open them and see the image.

# 1. Go to your dahsboard

# 2. Generate an API Key

⬇️ For the next steps you can use the Dashboard or the REST API. We will be using the dashboard:

# 3. Create an Event

⚠️ For original stream please use iframe, YouTube or Vimeo. If you want to use flv, HLS and RTMP we recommend you use our RTMP approach, if not your video and interpreted audios won't be synchronized.

# 4. Assign Languages

# 5. Assign Interpreters

⚠️ The interpreters accounts must have already been registered in www.akkadu.com

# 6. Select Output Type

# 7. Integrate the SDK in your audience event page

Choose between any of our repos: React, Vue or Vanilla JavaScript

# 8. Insert your API Key and room name

Once you insert your API Key and room name you are connecting the SDK with the interpreters pages.

• 🚨 Set hasSubtitles to true

# 9. Test

• The interpreter is in the right room:

  Check the name of the event is the same as the one you created in rsi.akkadu.com/dashboard

  

  

  Check the roomName is the same as the one showing in rsi.akkadu.com/dashboard

  

  

• The original stream has started to stream

# Events

Here is the list of the events for the SI + AI Recognized Subtitles SDK that can be passed as props when using our React and Vue verison. For VanillJS you can register this listeners using addEventListener. Examples: React, Vue, VanillaJS.

# Demo

You can check our live demo here or you can check below demo videos.

In above video:

• On the left size you can see a Vanilla JavaScript "audience page" with the SDK already integrated.
• On the right size you can see the interpreter page who is receiving the original stream via iframe although you can also use YouTube, Vimeo, FLV, m3u8 and RTMP.
• Notice how every time the audience selects a language the original stream is automatically muted.

In above video you can see how everytime the audience selects a language, besides listening to the interpreted audio, also can see the live AI subtitles.

⚠️ If you are a video conferencing platform using WebRTC we recommend you use this SDK approach but, if you are a video streaming platform using HLS, you can consider our RTMP approach which will give you synchronized video with the audios and the subtitles.

# 🔹 React

• 🚨 RsiInterpretationPlayer must be imported before rsiSubtitles.

  • RsiSubtitles cannot work without the widget RsinterpretationPlayer

• 🚨 In order to use the widget subtitles you must add as parameter of RsiInterpretationPlayer hasSubtitles to be true.

import React from 'react';
import { RsiInterpretationPlayer, RsiSubtitles } from '@akkadu/rsi-react'

export default class ComponentInterpretationPlayer {
  const handleOnSubtitlesReady = (e) => {
    console.info('receive onReady event for the subtitles widget: ',e );
  }

  render() {
    return (
      <div>
        <RsiInterpretationPlayer
          sdkKey="sdk_key_XXXX"
          roomName="your_room_name"
          hasSubtitles={true}
          onReady={handleOnReady}
          onLanguageSelected={handleOnLanguageSelected}
          onConnectionStatusUpdated={handleOnConnectionStatusUpdated}
        />
        <RsiSubtitles
          sdkKey="sdk_key_XXXX"
          roomName="your_room_name"
          onReady={handleOnSubtitlesReady}
        />
      </div>
    );
  }
}

# 🔹 Vue

• 🚨 RsiInterpretationPlayer must be imported before rsiSubtitles.

  • RsiSubtitles cannot work without the widget RsinterpretationPlayer

• 🚨 In order to use the widget subtitles you must add as parameter of RsiInterpretationPlayer hasSubtitles to be true.

<template>
  <RsiInterpretationPlayer
    sdkKey="sdk_key_XXXX"
    roomName="your_room_name"
    :hasSubtitles="true"
    v-on:onLanguageSelected="handleOnLanguageSelected"
    v-on:onReady="handleOnReady"
    v-on:onConnectionStatusUpdated="handleOnConnectionStatusUpdated"
  />
  <RsiSubtitles
    sdkKey="sdk_key_XXXX"
    roomName="your_room_name"
    v-on:onReady="handleOnSubtitlesReady"
  />
</template>

<script lang="js">
  import { RsiInterpretationPlayer, RsiSubtitles } from '@akkadu/rsi-vue'
  export default {
    components: {
      RsiInterpretationPlayer,
      RsiSubtitles
    },
    methods: {
      handleOnSubtitlesReady(e) {
        console.info('handleOnReady event for the subtitles widget:', e);
      },
    },
  }
</script> 

# 🔹 Vanilla JavaScript

• 🚨 RsiInterpretationPlayer must be imported before rsiSubtitles.

  • RsiSubtitles cannot work without the widget RsinterpretationPlayer

• 🚨 In order to use the widget subtitles you must add as parameter of RsiInterpretationPlayer data-has-subtitles to be true.

<!-- stylesheet -->
<link rel="stylesheet" href="https://akkadu-assets.s3.amazonaws.com/akkadu-rsi-widget/rsi-vanilla/1.0.62/interpretation-player.min.css" />

<link rel="stylesheet" href="https://akkadu-assets.s3.amazonaws.com/akkadu-rsi-widget/rsi-vanilla/1.0.62/rsi-subtitles.min.css" />

<!-- Component -->
<div
  id="akkadu-interpretation-player"
  data-sdk-key="your_sdk_key"
  data-room-name="your_room_name"
  data-position-menu="bottom"
  data-is-box-shadow="true"
  data-is-player-controlled="true"
  data-has-subtitles="true">
</div>
<div
  id="akkadu-rsi-subtitles"
  data-sdk-key="your_sdk_key"
  data-room-name="your_room_name">
</div> 
<script>
  const component = document.querySelector('#akkadu-interpretation-player')
  component.addEventListener('onReady', function (e) { 
    console.info('received onReady event for subtitles:', e.detail.isReady);
  });
</script>

<!-- 🚨 Script must be loaded after the component -->
<script src="https://akkadu-assets.s3.amazonaws.com/akkadu-rsi-widget/rsi-vanilla/1.0.62/interpretation-player.min.js" ></script>

<script src="https://akkadu-assets.s3.amazonaws.com/akkadu-rsi-widget/rsi-vanilla/1.0.62/rsi-subtitles.min.js" ></script>

# Properties

The properties for the SI + AI Recognized Subtitles SDK

Additional properties To further customize the RSI Subtitles Widget you can use the following props.

⚠️ For VanillaJS version all the props should be prefixed with data-* and instead of camelcase kebab-case should be used e.g. data-display-flag

# roomName

When creating an event in your Dashboard or using the REST API you will generate (dashboard) or get as a response (REST API) the variable roomName which is unique for every event.

You need to add this roomName inside the SDK. This is one of the main properties of the SDK which will connect the SDK with the interpreters room.

Note: You can also add a query parameter on the audience page URL of the event you want to get the Simultaneous Interpretation. For example:

• URL path : wvw.your-application/your-audience-page?rsi-roomname=abcd

# The refresh button

When the stream is stuck, some browser/devices requires an action from the user to play the stream again.

This is what we are solving with the refresh button.

When the interpreted stream is stuck and need to be unmuted a refresh button will appear near the widget.

# Error Messages

Sometimes the SDK may return below error messages in the browser console:

1. "Your sdkKey doesn't have an event with the roomName: abcd"

This means the roomName added in the SDK you integrated in your audience page doesn't match the roomName associated to the event of that sdkKey.

When creating an event with the REST API you should get in response a roomName, when creating an event in your dashboard you will automatically create a room name for that event, regardless of the method you use, please make sure you insert the same roomName in the properties of the SDK or you used the same roomName as a query parameter on the page.

More info about the roomName here

2. "The origin: www.your-website.com doesn't match the origin accepted by your sdkKey: xx_your_sdk_key"

Each sdkKey can only have one origin. You can create a new sdkKey in your dashboard and associate an origin (website) to it. If you are using our widget on localhost, you should not get any origin issues.

3. "The sdkKey or roomName is not defined"

This means the property sdkKey or/and the roomName is undefined. Make sure you didn't left them empty.