Microsoft Edge's latest video translation capability will use AI-generated voice dub

There is no doubt that Microsoft Edge has some good translation capabilities: from the live translation of its PDF reader to the more advanced video translation enhancements that effectively allow users to translate audio or subtitles while watching videos in the browser, Microsoft wants to make sure no one feels left behind.

This is most likely why the Redmond-based tech giant is bringing AI-generated voice dub translations to videos, according to the latest post from software enthusiast @Leopeva64.

In other words, you’ll be able to listen to video translations rather than reading them on Microsoft Edge, all due to a new enhancement that uses AI to generate that audio translation.

After many attempts and multiple crashes I was finally able to make a short recording of Edge's video translation feature, in the first video you can hear the original audio and in the second you can hear Edge's AI-powered translation:https://t.co/FIcnSbSZjT pic.twitter.com/ozilW5YMOc

— Leopeva64 (@Leopeva64) May 6, 2024

According to the enthusiast, Microsoft Edge will have AI generate different voices for dubbing the translation based on who’s speaking. For instance, if a man is speaking in the video, AI will generate a male voice, and if a woman is speaking, then a female voice will be generated.

Here’s another example of Edge’s video translation feature, this time with a female voice (the AI uses female voices if the voice in the video is female), the first video has the original audio and the second the dubbed audio:

Leopeva64

Here's another example of Edge's video translation feature, this time with a female voice (the AI uses female voices if the voice in the video is female), the first video has the original audio and the second the dubbed audio:https://t.co/Y6lHH3UUaH pic.twitter.com/nYemTsdXln

— Leopeva64 (@Leopeva64) May 6, 2024

The capability is currently live in Edge Canary, and it supports voice translation from English, Spanish, and Korean to German, Spanish, Hindi, Italian, and Russian.

For now, however, the capability is laggy, and it crashes constantly: Currently the translation is constantly interrupted (you can hear it in the video) and the browser crashes and lags a lot when enabling this feature, says the software enthusiast, but there is no doubt Microsoft will remedy this when the capability reaches its general availability.

However, the AI-generated video translation will be a game changer for Microsoft Edge users, as it broadens accessibility to visually and hearing-impaired individuals.

The capability should make it to the stable channel in the following weeks.