Isa Kabakci

October 20, 2023, 10:01 am | Read time: 4 minutes

The “HeyGen” AI has been around for quite some time. However, users have only recently started paying attention to the tool. “HeyGen” is a program that allows people to speak in another language. Specifically, you record any video, for example, in your native language, speak for about 30 seconds, and then let the AI convert the spoken content into the target language. TECHBOOK editor Isa Kabakci took a closer look at the tool and was surprised by the results.

Also interesting: How the new Beatles song was created using AI 

What Exactly Is “HeyGen”?

“HeyGen” is a startup from the U.S., founded in 2020. The main business is relatively simple to explain: You can create videos with text using AI-generated avatars and voices. This is especially useful for content creators, influencers, explainer videos, and people in marketing. Essentially, you don’t need a camera, actors, or a recording studio.

Additionally, it’s possible to create your own videos in another language. Not only is the spoken language converted, but the lip movements are synchronized as well. It’s also remarkable that your own voice’s intonation is adjusted to the target language.

The currently available languages are Spanish, French, Hindi, Italian, English, German, Polish, Portuguese, Turkish, Mandarin, Korean, Japanese, and Dutch (some languages were integrated later). However, it’s important to mention that the AI can only translate the video if the source language is supported by the tool. More on this “problem” below.

Also interesting: Tech founder: “AI offers the greatest opportunity humanity has ever had”

More on the topic

Other apps have had that for a long time. WhatsApp Tests Two Practical Features for Video Calls and Chats

On Instagram and TikTok Instagram Turns Your Posts Into Ads

Suddenly, We’re Speaking Japanese

The “HeyGen” AI went viral on X (formerly Twitter) some time ago. The well-known tech influencer Jon Finger uploaded a video in which he suddenly speaks German, even though he doesn’t actually know the language. This piqued our curiosity, so we tested it as well.

TECHBOOK editor Isa Kabakci grew up bilingual (German and Turkish) and tested the AI in both languages. The first test via “HeyGen Labs” was initially conducted in Turkish and was supposed to be translated into Portuguese. However, this failed because the Turkish language was not available on “HeyGen” at that time–Turkish was integrated later. Therefore, another video was recorded in German, which the tool then translated into Japanese. First the original video in German, then the AI-based version in Japanese:

Advantages and Disadvantages of the Translation AI

There are many advantages, but also some disadvantages to “HeyGen.” Let’s start with the positive aspects. As mentioned earlier, this tool is quite suitable for content creators and media professionals. You don’t necessarily have to buy expensive equipment (camera, microphone, etc.) but can create videos with text relatively easily using the AI by having avatars speak the content. “HeyGen” is also quite practical if you want to record a video in another language, provided it is supported by the tool. This means you can quickly reach a foreign audience.

Don’t want to miss important (or quirky) news on tech topics anymore? Then follow us on WhatsApp!

The disadvantages, on the other hand, are varied. Among other things, deepfakes can be easily created with the AI, allowing false information to be spread deliberately. Especially when false statements are attributed to a well-known person. The pricing could also deter potential users. Although you can translate such an AI-based video for free, you may have to wait several days for it to be completed. If you subscribe, starting at about 22 euros per month, you can edit the videos immediately–provided you have enough credits.