Varshul Gupta, co-founders of Dubverse and Anuja Dawan saw a major problem with video-based educational content only being available in English during the peak of the pandemic. Non-native English speakers were hindered by the lack of local language support. Gupta & Dhawan came up with an AI-based platform for dubbing that would allow creators the ability to create a video in up to 30 languages simultaneously.
“AI was initially limited to labs or institutes two-three year ago, but we thought about what AI could do to solve real-life issues,” Dhawan (30) explains how Dubverse was created. Although it was originally designed for educational purposes, Dubverse now serves all visual creators who want to dub a video using the language they prefer.
For the longest time, dubbing has been associated with movies. Dubbing has been used by movie studios and production companies to make their films available in other languages. This allows them to reach a wider audience. Voice-over, or dubbing, is the process in which the script sound is replaced by words from another language. The challenge is to make each dialogue sound fluid and have the right emotions so that it blends well with a native tongue.
Dubbing content has always been complex and involved many layers. The process requires a narrator and scriptwriter as well as a studio and video editor. It is expensive, time-consuming and costly. The fee for the narrator can range from Rs 20,000 to Rs 5,00,000. This depends on their experience and what kind of work they have done in the past.
However, as high-quality video production becomes more popular, the task of dubbing videos in multiple languages is becoming more difficult, particularly for content creators who have limited budgets.
Gurgaon-based startup Dhawan wants to offer dubbing services to filmmakers. It also offers news outlets and independent creators the opportunity to do their own dubbing. Dhawan states that the language barrier is real. She adds that “what we are trying is to make it so that any video, from a TikTok of 10 seconds to a YouTube content of over a minute length, can be easily dubbed in multiple languages with one click.”
AI-generated voices are not a new concept. AI voices have been used for years in e-books and voice assistants. Dhawan’s Dubverse platform makes digital voices sound like they are being narrated from a person. She says, “We realized that when an AI speaker speaks it sounds monotonous, flat and boring, so consumers don’t consume the content.” “We went back to the drawing boards and asked, “How can we improve it?” We decided to improve these voices so they sound more human-like.
Dubverse is a major shift in content creators’ ability to record their videos in languages that sound more human-like and less robotic without the use of a narrator. Dhawan’s company uses a mix of artificial intelligence and humans to create voices that sound real. When Dhawan is asked how machine learning algorithms and software can create audio that sounds more human-like than synthetically generated, Dhawan replies, “We take a person’s voice, we clone and create an AI avatar from it.” It is possible to keep the original voice but speak a different language and accent.
Dubverse, an AI-based dubbing platform, promises a seamless method to dub a video using any language. The user simply needs to insert a link to a YouTube video into the platform. Next, choose a language to convert the video to and select a speaker. The video will then be available in a new language. To reach the same level of content creators, the company hopes to make the dubbing of videos in multiple languages “accessible”. Imagine a YouTube creator with multiple channels. He or she wants the content available in Hindi, Bengali, Spanish and German.
Dhawan claims that her company has 30 human voices that have been trained to imitate an accent or voice. Dhawan claims that AI-powered voice-over software can be used to dub voices, but it is important to remember the importance of human beings as the software is trained with real people. Dhawan claims that artificial intelligence-powered software can still produce a voice similar to a human voice, but she has seen results ranging from 90 to 95%. However, this varies from one language to another.
