Doctored images have been around for decades. The term "Photoshopped" is part of everyday language. But in recent years, it has seemingly been replaced by a new word: deepfake.
It's almost everywhere online, but you likely won't find it in your dictionary at home. What exactly is a deepfake, and how does the technology work?
RELATED STORY | Scripps News Reports: Sex, Lies, and Deepfakes
A deepfake is an image or video that has been generated by artificial intelligence to look real. Most deepfakes use a type of AI called a "diffusion model." In a nutshell, a diffusion model creates content by stripping away noise.
"With diffusion models, they found a very clever way of taking an image and then constructing that procedure to go from here to there," said Lucas Hansen said. He and Siddharth Hiregowdara are cofounders of CivAI, a nonprofit educating the public on the potential — and dangers — of AI.
How diffusion models work
It can get complicated, so imagine the AI – or diffusion model – as a detective trying to catch a suspect. Like a detective, it relies on its experience and training.
It recalls a previous case -– a sneaky cat on the run. Every day it added more and more disguises. On Monday, no disguise. Tuesday, it put on a little wig. Wednesday, it added some jewelry. By Sunday, it's unrecognizable and wearing a cheeseburger mask.
The detective learned these changes can tell you what it wore and on what day.
AI diffusion models do something similar with noise, learning what something looks like at each step.
"The job of the diffusion model is to remove noise," Hiregowdara said. "You would give the model this picture, and then it will give you a slightly de-noised version of this picture."
RELATED STORY | Scripps News got deepfaked to see how AI could impact elections
When it's time to solve the case and generate a suspect, we give it a clue: the prompts we give when we create an AI-generated image.
"We have been given the hint that this is supposed to look like a cat. So what catlike things can we see in here? Okay, we see this curve, maybe that's an ear," Hiregowdara said.
The "detective" works backward, recalling its training.
It sees a noisy image. Thanks to the clue, it is looking for a suspect — a cat. It subtracts disguises (noise) until it finds the new suspect. Case closed.
Now imagine the "detective" living and solving crimes for years and years. It learns and studies everything — landscapes, objects, animals, people, anything at all. So when it needs to generate a suspect or an image, it remembers its training and creates an image.
Deepfakes and faceswaps
Many deepfake images and videos employ some type of face swapping technology.
You've probably experienced this kind of technology already — faceswapping filters like on Snapchat, Instagram or Tiktok use technology similar to diffusion models, recognizing faces and replacing things in real time.
"It will find the face in the image and then cut that out kind of, then take the face and convert it to its internal representation," Hansen said.
The results are refined then repeated frame by frame.
The future and becoming our own detectives
As deepfakes become more and more realistic and tougher to detect, understanding how the technology works at a basic level can help us prepare for any dangers or misuse. Deepfakes have already been used to spread election disinformation, create fake explicit imagesof a teenager, even frame a principal with AI-created racist audio.
"All the netizens on social media also have a role to play," Siwei Lyu said. Lyu is a SUNY Empire Innovation Professor at the University of Buffalo's Department of Computer Science and Engineering, and the director of the Media Forensics Lab. His team has created a tool to help spot deepfakes called "DeepFake-o-meter."
"We do not know how to handle, how to deal, with these kinds of problems. It's very new. And also requires technical knowledge to understand some of the subtleties there," Lyu said. "The media, the government, can play a very active role to improve user awareness and education. Especially for vulnerable groups like seniors, the kids, who will start to understand the social media world and start to become exposed to AI technologies. They can easily fall for AI magic or start using AI without knowing the limits."
RELATED STORY | AI voice cloning: How programs are learning to pick up on pitch and tone
Both Lyu and CivAI believe in exposure and education to help combat any potential misuse of deepfake technology.
"Our overall goal is that we think AI is going t impact pretty much everyone in a lot of different ways," Hansen said. "And we think that everyone should be aware of the ways that it's going to change them because it's going to impact everyone."
"More than just general education — just knowing the facts and having heard what's going to happen," he added. "We want to give people a really intuitive experience of what's going on."
Hansen goes on to explain CivAI's role in educating the public.
"We try and make all of our demonstrations personalized as much as possible. What we're working on is making it so people can see it themselves. So they know it's real, and they feel that it's real," Hansen said. "And they can have a deep gut level feel for tthe impact that it's going to have."
"A big part of the solution is essentially just going to be education and sort of cultural changes," he added. "A lot of this synthetic content is sort of like a new virus that is attacking society right now, and people need to become immune to it in some ways. They need to be more suspicious about what's real and what's not, and I think that will help a lot as well."
