SI + AI Recognized Subtitles
- Quick implementationπ
- How does it work?
- Step by step guide
- Events
- Live Demo
- πΉ React
- πΉ Vue
- πΉ Vanilla JavaSript
- Properties
Quick implementation π
How does it work?
The SI + AI Recognized Subtitles
SDK is an slight variation of the Simultaneous Interpretation (SI)
SDK:
- While the professional interpreters do the intepretation, we will get the voice recognition of their audio.
- The audience will be able to hear the interpretation and read the subtitles at the same time.
The SI + AI Recognized Subtitles
SDK should be integrated in the audience page of your event allowing your audience to switch between the different languages of interpretation and AI Recognized Subtitles.
You will have to send the original stream to the interpreters using our dashboard or the REST API. The audio of the interpreters will automatically play in the SDK you integrated in your audience page.
Once your audience select a language the SDK can automatically mute your original stream or, you can catch the events that we send to decide when to mute the original stream.
π¨ Important:
RsiInterpretationPlayer
must be imported beforersiSubtitles
.RsiSubtitles
cannot work without the widgetRsiInterpretationPlayer
- In order to use the
SDK with subtitles
, you must set the parameterhasSubtitles
to betrue
.
Step by step guide
Note: The arrows below are dropdowns, click to open them and see the image.
1. Go to your dahsboard
You can type the URL rsi.akkadu.com/dashboard or click Login
2. Generate an API Key
Click API Keys
Click + Create New
Insert the App Name and Domain, then click Save
You will see your API Key in your dashboard
β¬οΈ For the next steps you can use the Dashboard or the REST API. We will be using the dashboard:
3. Create an Event
Click + Create Event
Insert all the relevant information
β οΈ For original stream please use iframe
, YouTube
or Vimeo
. If you want to use flv
, HLS
and RTMP
we recommend you use our RTMP approach, if not your video and interpreted audios won't be synchronized.
4. Assign Languages
Click on Languages
Click + Add Languages
5. Assign Interpreters
β οΈ The interpreters accounts must have already been registered in www.akkadu.com
Insert the interpreters accounts
6. Select Output Type
Select: None
7. Integrate the SDK in your audience event page
Choose between any of our repos: React, Vue or Vanilla JavaScript
You should be able to see the languages you created in the dahsboard
8. Insert your API Key and room name
Once you insert your API Key and room name you are connecting the SDK with the interpreters pages.
- π¨ Set
hasSubtitles
totrue
9. Test
0. Test your API Key (optional)
In this fake event audience page, without doing any integration, by inserting your API Key
and room name
you can check:
- if the SDK is showing the languages you created in the dashboard
- if the interpreters accounts are correctly linked to the
room name
. You will have to interpret from the interpreter account then select the language channel to confirm you can hear the interpreter audio
As you can see in the below image the SDK is showing the languages I created in the dashboard: English
, Spanish
and Korean
.
If I enter to the Spanish interpreter account and start interpreting I should be able to hear the audio after selecting the Spanish
channel.
1. Go to www.akkadu.com and enter with your interpreter account
2. Once inside the interpreter dashboard find the event you just created in rsi.akkadu.com/dahsboard:
3. Click interpret
4. You should be able to see the original stream
The interpreter is in the right room:
Check the name of the event is the same as the one you created in rsi.akkadu.com/dashboard
Check the roomName is the same as the one showing in rsi.akkadu.com/dashboard
The original stream has started to stream
5. Click Start Turn and start interpreting
6. Check you can hear the interpreted audio in the SDK
For this case you should hear audio in the Spanish channel.Events
Here is the list of the events for the SI + AI Recognized Subtitles
SDK that can be passed as props when using our React and Vue verison. For VanillJS you can register this listeners using addEventListener
. Examples: React, Vue, VanillaJS.
Events | Description |
---|---|
onReady | Called when the event is ready |
Demo
You can check our live demo here or you can check below demo videos.
In above video:
- On the left size you can see a Vanilla JavaScript "audience page" with the SDK already integrated.
- On the right size you can see the interpreter page who is receiving the original stream via
iframe
although you can also useYouTube
,Vimeo
,FLV
,m3u8
andRTMP
. - Notice how every time the audience selects a language the original stream is automatically muted.
In above video you can see how everytime the audience selects a language, besides listening to the interpreted audio, also can see the live AI subtitles.
β οΈ If you are a video conferencing platform using WebRTC we recommend you use this SDK approach but, if you are a video streaming platform using HLS, you can consider our RTMP approach which will give you synchronized video with the audios and the subtitles.
πΉ React
π¨
RsiInterpretationPlayer
must be imported beforersiSubtitles
.RsiSubtitles
cannot work without the widgetRsinterpretationPlayer
π¨ In order to use the
widget subtitles
you must add as parameter ofRsiInterpretationPlayer
hasSubtitles
to betrue
.
import React from 'react';
import { RsiInterpretationPlayer, RsiSubtitles } from '@akkadu/rsi-react'
export default class ComponentInterpretationPlayer {
const handleOnSubtitlesReady = (e) => {
console.info('receive onReady event for the subtitles widget: ',e );
}
render() {
return (
<div>
<RsiInterpretationPlayer
sdkKey="sdk_key_XXXX"
roomName="your_room_name"
hasSubtitles={true}
onReady={handleOnReady}
onLanguageSelected={handleOnLanguageSelected}
onConnectionStatusUpdated={handleOnConnectionStatusUpdated}
/>
<RsiSubtitles
sdkKey="sdk_key_XXXX"
roomName="your_room_name"
onReady={handleOnSubtitlesReady}
/>
</div>
);
}
}
πΉ Vue
π¨
RsiInterpretationPlayer
must be imported beforersiSubtitles
.RsiSubtitles
cannot work without the widgetRsinterpretationPlayer
π¨ In order to use the
widget subtitles
you must add as parameter ofRsiInterpretationPlayer
hasSubtitles
to betrue
.
<template>
<RsiInterpretationPlayer
sdkKey="sdk_key_XXXX"
roomName="your_room_name"
:hasSubtitles="true"
v-on:onLanguageSelected="handleOnLanguageSelected"
v-on:onReady="handleOnReady"
v-on:onConnectionStatusUpdated="handleOnConnectionStatusUpdated"
/>
<RsiSubtitles
sdkKey="sdk_key_XXXX"
roomName="your_room_name"
v-on:onReady="handleOnSubtitlesReady"
/>
</template>
<script lang="js">
import { RsiInterpretationPlayer, RsiSubtitles } from '@akkadu/rsi-vue'
export default {
components: {
RsiInterpretationPlayer,
RsiSubtitles
},
methods: {
handleOnSubtitlesReady(e) {
console.info('handleOnReady event for the subtitles widget:', e);
},
},
}
</script>
πΉ Vanilla JavaScript
π¨
RsiInterpretationPlayer
must be imported beforersiSubtitles
.RsiSubtitles
cannot work without the widgetRsinterpretationPlayer
π¨ In order to use the
widget subtitles
you must add as parameter ofRsiInterpretationPlayer
data-has-subtitles
to betrue
.
<!-- stylesheet -->
<link rel="stylesheet" href="https://akkadu-assets.s3.amazonaws.com/akkadu-rsi-widget/rsi-vanilla/1.0.62/interpretation-player.min.css" />
<link rel="stylesheet" href="https://akkadu-assets.s3.amazonaws.com/akkadu-rsi-widget/rsi-vanilla/1.0.62/rsi-subtitles.min.css" />
<!-- Component -->
<div
id="akkadu-interpretation-player"
data-sdk-key="your_sdk_key"
data-room-name="your_room_name"
data-position-menu="bottom"
data-is-box-shadow="true"
data-is-player-controlled="true"
data-has-subtitles="true">
</div>
<div
id="akkadu-rsi-subtitles"
data-sdk-key="your_sdk_key"
data-room-name="your_room_name">
</div>
<script>
const component = document.querySelector('#akkadu-interpretation-player')
component.addEventListener('onReady', function (e) {
console.info('received onReady event for subtitles:', e.detail.isReady);
});
</script>
<!-- π¨ Script must be loaded after the component -->
<script src="https://akkadu-assets.s3.amazonaws.com/akkadu-rsi-widget/rsi-vanilla/1.0.62/interpretation-player.min.js" ></script>
<script src="https://akkadu-assets.s3.amazonaws.com/akkadu-rsi-widget/rsi-vanilla/1.0.62/rsi-subtitles.min.js" ></script>
Properties
The properties for the SI + AI Recognized Subtitles
SDK
Properties | Description | Required | Choices | Default |
---|---|---|---|---|
sdkKey | Your API Key | true | string: sdk_key_XXXX | null |
roomName | You can either add your roomName as a query parameter or as a propertie more | false | string: "abcd" | "abcd" |
Additional properties To further customize the RSI Subtitles Widget you can use the following props.
Properties | Description | Required | Type | Default |
---|---|---|---|---|
subtitlesWrapperClass | To set a custom class for RSI Subtitles widget wrapper | false | string | none |
subtitlesTextClass | To set a custom class for the text of the RSI Subtitles'swidget. | false | string | none |
β οΈ For VanillaJS version all the props should be prefixed with data-*
and instead of camelcase kebab-case should be used e.g. data-display-flag
roomName
When creating an event in your Dashboard or using the REST API you will generate (dashboard) or get as a response (REST API) the variable roomName
which is unique for every event.
You need to add this roomName
inside the SDK. This is one of the main properties of the SDK which will connect the SDK with the interpreters room.
Note: You can also add a query parameter on the audience page URL of the event you want to get the Simultaneous Interpretation. For example:
- URL path : wvw.your-application/your-audience-page?
rsi-roomname=abcd
The refresh button
When the stream is stuck, some browser/devices requires an action from the user to play the stream again.
This is what we are solving with the refresh button
.
When the interpreted stream is stuck and need to be unmuted a refresh button
will appear near the widget.
Error Messages
Sometimes the SDK may return below error messages in the browser console:
1. "Your sdkKey doesn't have an event with the roomName: abcd"
This means the roomName
added in the SDK you integrated in your audience page doesn't match the roomName
associated to the event of that sdkKey
.
When creating an event with the REST API you should get in response a roomName
, when creating an event in your dashboard you will automatically create a room name for that event, regardless of the method you use, please make sure you insert the same roomName
in the properties of the SDK or you used the same roomName as a query parameter on the page.
More info about the roomName here
2. "The origin: www.your-website.com doesn't match the origin accepted by your sdkKey: xx_your_sdk_key"
Each sdkKey can only have one origin. You can create a new sdkKey in your dashboard and associate an origin (website) to it. If you are using our widget on localhost, you should not get any origin issues.
3. "The sdkKey or roomName is not defined"
This means the property sdkKey
or/and the roomName
is undefined. Make sure you didn't left them empty.