These signing schemes are a very long way off right now. Even if they get widely adopted and if they aren't subverted immediately, it's going to take a decade or more for widespread adoption. Most of the "crazy shit" footage you see online is from an old cellphone that's not getting replaced any time soon.
Do we imagine that the keys from a DSLR or phone can’t be extracted by a sufficiently motivated actor? Or that the sensor on such a device couldn’t be replaced with a substitute image generator?
iPhones do cryptographic handshakes and attestation internally between the SoC and the camera chips. You can swap out the camera but the OS will know it, and it won't be signing the result.
The way you've phrased it makes your question unfalsifiable anyway. We can imagine a "sufficiently motivated actor" who can do literally anything. But for most cases this kind of tech will work well enough. It's got a lot harder to root phones over the years and big companies are in a great position to use AI to shake out lots of other bugs that could be used to do it in future.
> Credentials that follow the C2PA standard, for example, attach a cryptographically signed manifest to a photo or video at the moment of capture; this metadata records the device, the time and location of the recording
This claim sounds like a tidy solution, but my gut feeling is that it doesn't really solve the problem. I struggle to articulate exactly why I think it's wrong, but I'll list some ideas:
* The camera presumably holds an asymmetric private key. The camera is in the user's hands, and chips could be removed and examined to extract the key with immense difficulty. Extremely few people in this world have the knowledge and equipment to do this, but it only takes one person to break the system for everyone.
* The camera maker might leak the key through negligence. Look at other examples where companies leaked TLS/SSL private keys or other cryptographic secrets.
* This scheme seems to give a lot of power to a small number of companies to define image authenticity in a proprietary way. This gatekeeping doesn't seem like a good direction for society.
* If someone is determined to break the system, eventually someone does. Look at the history of DRM and how countless schemes have been broken. Look at how Intel SGX was broken (it claims to allow a CPU to execute secret computations on behalf of someone else without the physical hardware owner being able to inspect those computations).
* Some image authentication schemes were cracked, such as ElcomSoft breaking an older version of Canon's scheme in the year 2010.
* What if a genuine camera takes a real photo of a real film slide or electronic monitor displaying utterly ridiculous imagery? (Analogous to the analog hole for DRM.)
* Does the private key have an expiration date? At any given point in time, is there an assumption that all photos older than a certain time have unreliable signatures? (Think of how the TLS certificate authority infrastructure comes with expiration dates.)
I don't have the answers to this problem. One thing I want to suggest, though, is that images coming straight off a camera have a lot of detail, and I think that is hard for AI to fake. Faking an image at Instagram resolution with relatively high JPEG compression seems plausible. Faking 6000×4000 pixels (24 MP) without leaving telltale seams and artifacts is a lot harder. I say this as someone who does digital photography and spends a lot of time pixel-peeping.
> The types of forensic analyses that I routinely perform do not currently operate at internet scale and speed.
Fair point. But I don't think you can tech-shortcut your way out of this problem.
> These provenance signals are often criticized as being only a partial solution or vulnerable to attack. This is true if we think of them as mechanisms for catching bad actors. But that’s not what they’re supposed to do. Their purpose is to let good actors prove that their audiovisual recording is what it claims to be.
You're being too charitable. It only takes one bad actor, one successful cracker to destroy the foundations of authenticity. One credibly signed fake image is enough to cast doubt on all the signed real images out there.
> * If someone is determined to break the system, eventually someone does.
State actors will be determined and find a way - either through extortion, breaking into a vendor or doing key extraction from a device. So while it may offer some sense of security for funny videos, anything politically relevant can‘t be trusted that way.
Yeah, just go all in on implementing another nightmare dystopian technology. Not only you will get unfairly disadvantaged when filming on older hardware, consider also misinformation capabilities granted to big corps and state actors: look, this video evidence is totally authentic, it has all the right signatures! ...nevermind that we got the signing keys through NSL from camera manufacturer.
> [...] the deepfake-video-call software has to take shortcuts, such as simplifying faces and how they interact with their surroundings, and allowing some drift between audio and video or between video frames. The human eye doesn’t notice the shortcuts, but we can forensically expose them.
The way you've phrased it makes your question unfalsifiable anyway. We can imagine a "sufficiently motivated actor" who can do literally anything. But for most cases this kind of tech will work well enough. It's got a lot harder to root phones over the years and big companies are in a great position to use AI to shake out lots of other bugs that could be used to do it in future.
So I think this tech can work well.
This claim sounds like a tidy solution, but my gut feeling is that it doesn't really solve the problem. I struggle to articulate exactly why I think it's wrong, but I'll list some ideas:
* The camera presumably holds an asymmetric private key. The camera is in the user's hands, and chips could be removed and examined to extract the key with immense difficulty. Extremely few people in this world have the knowledge and equipment to do this, but it only takes one person to break the system for everyone.
* The camera maker might leak the key through negligence. Look at other examples where companies leaked TLS/SSL private keys or other cryptographic secrets.
* This scheme seems to give a lot of power to a small number of companies to define image authenticity in a proprietary way. This gatekeeping doesn't seem like a good direction for society.
* If someone is determined to break the system, eventually someone does. Look at the history of DRM and how countless schemes have been broken. Look at how Intel SGX was broken (it claims to allow a CPU to execute secret computations on behalf of someone else without the physical hardware owner being able to inspect those computations).
* Some image authentication schemes were cracked, such as ElcomSoft breaking an older version of Canon's scheme in the year 2010.
* What if a genuine camera takes a real photo of a real film slide or electronic monitor displaying utterly ridiculous imagery? (Analogous to the analog hole for DRM.)
* Does the private key have an expiration date? At any given point in time, is there an assumption that all photos older than a certain time have unreliable signatures? (Think of how the TLS certificate authority infrastructure comes with expiration dates.)
I don't have the answers to this problem. One thing I want to suggest, though, is that images coming straight off a camera have a lot of detail, and I think that is hard for AI to fake. Faking an image at Instagram resolution with relatively high JPEG compression seems plausible. Faking 6000×4000 pixels (24 MP) without leaving telltale seams and artifacts is a lot harder. I say this as someone who does digital photography and spends a lot of time pixel-peeping.
> The types of forensic analyses that I routinely perform do not currently operate at internet scale and speed.
Fair point. But I don't think you can tech-shortcut your way out of this problem.
> These provenance signals are often criticized as being only a partial solution or vulnerable to attack. This is true if we think of them as mechanisms for catching bad actors. But that’s not what they’re supposed to do. Their purpose is to let good actors prove that their audiovisual recording is what it claims to be.
You're being too charitable. It only takes one bad actor, one successful cracker to destroy the foundations of authenticity. One credibly signed fake image is enough to cast doubt on all the signed real images out there.
State actors will be determined and find a way - either through extortion, breaking into a vendor or doing key extraction from a device. So while it may offer some sense of security for funny videos, anything politically relevant can‘t be trusted that way.
I really don't understand these approaches. As if what the poor little girl Nayirah saw was deepfaked? How can a chip in a camera make it more true?
...for now.