4. Photo Forensics: As Seen on CSI
Veröffentlicht: 27.11.2015
in der Serie Photo Forensics
zurück vor

A digital camera contains a vast array of sensor cells, each with a photo detector and an amplifier. The photo detectors measure incoming light and transform it into an electrical signal. The electrical signals are then converted into pixel values. In an ideal camera, there would be a perfect correlation between the amount of light striking the sensor cells and the pixel values of the digital image. Real devices have imperfections, however, and these imperfections introduce noise in the image.

One source of noise is caused by stray electrons within the sensor cell. These stray electrons combine with the electrons generated by the photo detector as it responds to light. The resulting noise pattern is random, fluctuating from image to image. A second source of noise has very different characteristics. This noise arises from slight variations in the size and material properties of the sensor cells themselves. Physical inconsistencies across the sensor cells lead to differences in the efficiency with which the cells convert light into digital pixel values. These variations, termed photo-response non-uniformity (PRNU), lead to a stable noise pattern that is distinctive to the device.

Although the PRNU will leave a trace in any image, it is easiest to see in a blank image like that of a cloudless sky. If the sky is perfectly uniform, then an ideal camera with an array of identical sensor cells would produce an image that is perfectly homogeneous.  In contrast, a real camera will produce an image that has a very faint speckle pattern. The speckle occurs because some sensor cells over-report the amount of incoming light, while others under-report it. If the average sensor cell multiplies the amount of light by a factor of 1, the over-reporting sensor cells multiply it by a factor slightly greater than 1, and the under-reporting cells multiply it by a factor slightly smaller than 1. Unlike sensor noise, which modulates the pixel regardless of its value, PRNU modulates the pixel proportional to its value. Also unlike sensor noise, PRNU is a fixed property of the sensor and so it does not vary from image to image.

The PRNU associated with a particular device is not only stable, it is also distinctive. Even devices of the same make and model have different PRNUs. The stable and distinctive properties of the PRNU allow it to serve two forensic functions. The PRNU can be used to determine whether a particular image is likely to have originated from a given device. The PRNU can also be used to detect localized tampering in an image that was taken from a known device. This second use allows us to confirm the authenticity of an image taken by a photographer who has already produced a body of trusted work.

The estimation of the PRNU relies in part on the distinctive statistical properties of noise, which are unlike the statistical properties of normal image content. (For the details of the computation see 1Jessica Fridrich. Sensor Defects in Digital Image Forensics. In Husrev Taha Sencar and Nasir Memon (eds.), Digital Image Forensics: There is More to a Picture than Meets the Eye (New York: Springer, 2012), 179–218. Jan Lukas, Jessica Fridrich, and Miroslav Goljan. Digital Camera Identification from Sensor Noise. IEEE Transactions on Information Security and Forensics, 1(2):205–214, 2006..) It is possible to get a crude estimate of a device’s PRNU from a single image, but a reliable estimate requires 10 to 20 images (the exact number depends on the quality of the camera, as well as the quality and content of the images).

Example

I estimated the PRNU of my iPhone using 20 images of largely uniform texture-less scenes such as the sky or a blank wall. A small 64 x 64 pixel region of the PRNU is shown below. From the scale at the bottom of this image, you can see that the PRNU factor typically lies between -0.01 to 0.01 (meaning that it only modulates the recorded pixel value by 1%).

I then used my iPhone to photograph a group of students on the Dartmouth College campus. To measure the similarity between this image’s PRNU and the device’s PRNU, I performed a simple pixel correlation. In this case, the resulting correlation value was 0.073. Given that a perfect match would yield a value of 1.0, this correlation may seem to provide insufficient evidence of a match.

(29044)

To test whether a value of 0.073 is sufficient to link the image to my iPhone, I used the photo-sharing website Flickr to randomly select 100,000 images that were the same size as the iPhone image. I then compared the PRNU extracted from each image with the PRNU of the iPhone. The median similarity for these images was 0.0056, more than an order of magnitude smaller than that of the image recorded by the iPhone. Of the 100,000 images I tested, 98.4% had a correlation value less than 0.02, 99.9% had a value less than 0.03, and 99.99% had a value less than 0.04. (The maximum similarity value was 0.0428.) The seemingly low correlation between the iPhone image PRNU and iPhone PRNU was enough to link the image to the device.

Summary

Imperfections in the sensor array introduce stable and distinctive patterns in the images recorded by that device. This photo-response non-uniformity (PRNU) causes some sensor cells to consistently over-report or under-report the amount of measured light. The PRNU pattern for a device can be estimated from a single image or a collection of images. Because the PRNU is both constant and distinctive, it can be used to link an image to a specific device and it can reveal when images from that device have been manipulated.

Further Reading

Jessica Fridrich. Sensor Defects in Digital Image Forensics. In Husrev Taha Sencar and Nasir Memon (eds.), Digital Image Forensics: There is More to a Picture than Meets the Eye (New York: Springer, 2012), 179–218.

Jan Lukas, Jessica Fridrich, and Miroslav Goljan. Digital Camera Identification from Sensor Noise. IEEE Transactions on Information Security and Forensics, 1(2):205–214, 2006.

Portions of this entry are adapted from my upcoming book, Photo Forensics (MIT Press).

 

5 Kommentar(e)
Claus Gunti
Abgesendet am 07.12.2015 um 12:42

Dear colleague,

Thank you very much for the challenging insight about these technical aspects of photographic imaging all too often neglected in the art historical or photo-theoretical discourse. I was wondering if it would be possible to extrapolate the correlation of PRNU values and specific devices at a larger scale. If one were to discard computing power issues, would it be realistic to imagine that a social media company would link your phone's PNRU based on your images in order to extent data mining outside the infrastructure of its own network, e.g. finding images on the web that you took.

Sorry if the question seems naive, my background is rather art historical and focussing on photographic "retouching" in art. But as surveillance also is one of my research interest, it'd be interesting to know.

Kind regards,
Claus Gunti

Antworten
Hany Farid
Abgesendet am 14.12.2015 um 21:44

This is possible with one caveat. Images that are uploaded to social media are often modified at the time of upload. The most standard modification is to resize the image from its original resolution (typically on the order of several megapixels) to something more manageable and appropriate for social media (typically on the order of a fraction of a megapixel). This resizing will either destroy or greatly diminish the PRNU and make it difficult to track images across platforms. If, however, an image is uploaded and retained at its original resolution, this technology could be used to link an image to other images and eventually to you and your device.

Despite its forensic usefulness, this technology does have some potential for misuse — not unlike most other technologies.

Antworten
Joerg Scheller
Abgesendet am 24.12.2015 um 12:36

People say that humanity is constantly transforming into machines, but can’t it be that machines also become more human, or that they adopt some human characteristics?
The PRNU is not perfect, neither are humans. The PRNU is both constant and distinctive, are fingerprints not the same? They are definitely unique and if you don't acid-treat them, they remain constantly the same.
Is there another way to use the imperfections of the PRNU, other than for forensic reasons?
Nowadays almost everybody in our western world owns a device that can be used to take digital images. There's digital cameras, smartphones, laptops and webcams (and these are just the most common ones). The number of people possessing such a device will increase and so will the amount of images these devices create.
Is here the hidden potential of a new safety measure coming in the future? Can each and every person be identified by their images? Will big brother not only be watching us, but also our photographs?
I for sure don't know anybody who shares their iPhone, hence to use the distinctive PRNU information to identify someone besides by their teeth oder fingerprints isn't that far from reality than it seems. In the future, the PRNU could be noted in addition to fingerprint records in the passport, for example. Also what would happen to individuals who still use analogue cameras? Are they the new terrorists? Those who create better quality photos proportional to the amount of photos (assumingly), are they the new public enemies?

Amélie Korzil, BA student in photography, ZHdK

Antworten
Joerg Scheller
Abgesendet am 24.12.2015 um 12:41

Mia Spindler, "Stray Electrons", 2015
Mia Spindler is a BA student in Media Arts at ZHdK

Antworten
Hany Farid
Abgesendet am 04.01.2016 um 22:33

As with most technologies, this technology leads to potentially complex privacy issues. There are some simple things that you can do to protect your photographic privacy: (1) reduce the resolution of images that you upload (this will significantly minimize the likelihood that the PRNU will stay intact); and (2) remove the image metadata (most online services do this, and it is easy to do yourself). In addition, unlike fingerprints or DNA, the average consumer updates their phone/camera every few years, so it will be difficult to track users over a long period of time.

Antworten
Antwort verwerfen