February 10, 2024, 8:12 am | Read time: 5 minutes

Generative AIs can not only produce text but also generate images, videos, and audio “recordings.” Overall, material created or edited by AI–one could also say manipulated–is referred to as a deepfake. Many generative AIs are now so powerful that their deepfakes appear deceptively real. This gives them great potential for both entertaining antics and fraudulent and unscrupulous purposes.

How Deceptively Real Images Are Created

Some of the best-known AI image generators include Dalle-E, Midjourney, Leonardo.ai, Adobe Firefly, and Canva. Initially, the image generators are trained with image data, such as portrait photos of real people. Then, two neural networks compete against each other to generate a portrait of a fictional person in real time. One neural network calculates a new face from all the possible learned human features–facial shapes, skin properties, and eye colors to hairstyles.

Subsequently, the second neural network evaluates the result based on its own experience from the training data. If the fake face passes the “reality check,” it is considered complete. Otherwise, the first network must rework it. The result is usually convincingly real “photos.” Depending on the prompt (verbal command), the AI could also produce paintings in the style of Frida Kahlo, create a new logo for TECHBOOK, or design a movie poster for a fictional space western.

Read also: Abuse of AI-Generated Nude Images Is Increasing–Lawyer Explains Rights 

Deepfakes and Disinformation

More than 90 percent of deepfakes are so realistic that they can easily be mistaken for authentic. For example, in 2023, an image of Pope Francis in a white puffer jacket went viral, as did a “photo” of Donald Trump’s alleged arrest. In these contexts, deepfakes are not just amusing antics but could be deliberately used for disinformation and incitement.

Previously, good media literacy involved recognizing collages and placing images in their context–especially when they had been taken out of context. Now, one must be even more critical of images. While it was already difficult to verify certain photos, such as those from war or crisis zones, this will become even harder in the future.

However, even though AIs are now quite good, the results are often not perfect. Sometimes fake images of people can be identified by faulty facial symmetry, texture errors on the skin, or strange transitions in the background. Occasionally, deepfakes show errors in complex clothing, such as a baseball cap brim growing out of a person’s forehead. The AI also sometimes struggles with the correct number of fingers and toes. To provide users with clarity about the authenticity of an image or video, the EU has enacted the AI Act, which, among other things, requires that deepfakes be labeled as such.

Yet even with labeling, deepfakes will significantly impact many industries, especially in the creative sector. In the art, culture, and music scenes, there are debates about the threat to artists from AI-generated works, which can be produced in just a few seconds. Additionally, artists and cultural creators accuse AI companies like OpenAI of unlawfully using their copyrighted texts, images, etc., as training data.

Also interesting: Netflix Distorts Original Image Material in Documentary

More on the topic

AI-Generated Images Taylor Swift Targeted by Pornographic Deepfakes–Here’s How X Responded

Growing Danger How to Check if Your Photo Is Being Used for Deepfakes

Using Deepfakes Against Language Barriers

However, there are also numerous positive applications for deepfakes. For example, Spotify is testing the Voice Translation feature, where an AI convincingly imitates the voices of podcasters and translates them into another language. This feature is currently only available for a few major podcasts but could theoretically help break down global language barriers. Apps like HeyGen make a similar contribution by not only translating what a speaker says into the camera but also adjusting voice and facial expressions.

Another, more playful application is found in genealogy. On X, users share countless AI animations originating from a genealogy portal. The company offers its customers the ability to animate uploaded portraits with head and eye movements and occasionally a smile. This is called Deep Nostalgia and works with faces from paintings or sculptures. So, if you want to bring an old family photo to life, you can do so with the help of AI.

Read also: 12 Important Terms in the Field of AI and What They Mean

Increasing Misuse of Deepfakes

While deepfakes may be a technical achievement, they are also increasingly used for fraud, defamation, and pornography. For example, phishing notifications can be made more convincing with AI. Phone calls with fake voices of relatives are no longer a magic trick. Additionally, awareness of the vast possibilities of deepfakes is only slowly developing. Particularly well-made video deepfakes can easily convince people who are not aware of the dangers.

A thriving business field is also deepfake pornography. This ranges from software providers offering specific AI generators to pornographic distribution platforms, poorly regulated social networks, and Google searches. The user share of so-called nudify apps has recently increased significantly, and cases of non-consensually created and distributed deepfake pornography frequently make headlines. Not only stars like Taylor Swift are affected, but also “ordinary” girls and women–with devastating personal consequences.

As great as the potential of deepfakes and the excitement about the capabilities of AIs are, clear regulations are needed to effectively prevent abuse and fraud. However, the technology still seems to be developing much faster than the legal framework.