Speeding up subtitle creation at VIKI
Role
Working team
Problem
Rakuten Viki is a global streaming platform specialising in Asian dramas. Uniquely, many shows on Viki are subtitled in over 200 languages by fan subtitlers.
Fan subtitlers self-organise into teams for each drama. Each team’s speed is different, however, which meant viewers had to wait anywhere between 4 hours to 4 days to watch a new episode with English subtitles.
We wanted to investigate ways to improve the subtitle creation speed.
The drama Mars on Viki, featuring subtitles created by the team “The Rebels”
I interviewed 21 fans to learn how fans currently create subtitles.
I found that while each team’s approach differs slightly, most teams work in sequential order, starting with segmenting and ending with editing.Through earlier user interviews and an internal audit, we identified some problem areas in the flow:
Subtitling process
1.
Segment video
Each line of dialog is cut into a “segment” to be filled with subtitles.
2.
Subtitle in English
Translate each segment of dialog into English subtitles.
3.
Edit English subtitles
Editors review subtitles for grammar, tone and consistency.
4.
Done!
Once English subtitles are done, other languages begin work.
The problem? Many times when a team is done with a video, they don’t inform the next team to start working on the video. This causes as much as 25 hours of delay.
Challenge
We knew we had to automate the subtitling process. Once a team is done, the next team should automatically be notified. By automating, we can also start tracking teams’ progress and identify bottlenecks.
The challenge is that doing so is expensive, with uncertain returns on investment. That’s because it requires us to tweak our legacy subtitle tools and build a notification system on top of it.
Approach
What we needed as a proof of concept — an experiment to show that this idea would really work.
So I set out to run a low-cost, low-tech experiment to verify if automating the process helps with subtitling speed and works well with fan teams. This is where the fun begins!
We wanted to test this on live Viki shows, but with low engineering effort and minimal impact on viewers. Working with UX Research, Content, Community, and Engineering teams, I crafted an experiment plan for a Wizard of Oz test.
We tested on 3 live dramas of different genres, with control and test episodes per drama. Only team leaders were aware of the test, as we didn’t want to influence how others behaved. We measured how quickly fans responded to messages, time taken to complete subtitles, and overall user reception (via a survey).
Outcome
Turnaround speed increased 61%
The experiment showed that automating the subtitling process can help speed up subtitle creation.
We observed that the turnaround speed between each team (e.g. between the segmenting and subtitling team) increased by 61% with the automated notifications.
Test
Control
+61%
Test
Control
+14%
Positive reception
When asked if they would prefer to continue receive the hacked-together messages they got during the test, majority of the team leaders said yes.
Yes
No
74%
Thanks to this experiment, we were able to validate that the idea of automating notifications would help increase subtitle creation speed and reduce the risk of building this feature.
Other case studies
