In recent decades, we have witnessed the advancement of information technologies leading to the ubiquitous availability of multimedia devices and digital content. This path of technological evolution has naturally led to a critical issue that must be addressed next, namely, to ensure that content, devices, and intellectual property are being used by authorized users for legitimate purposes, and to be able to forensically prove with high confidence when otherwise. When security is compromised or authenticity is forged, forensic methodologies and tools are employed to reconstruct what has happened to digital content in order to answer who has done what, when, where, and how.

In order to verify the integrity of a digital document, security practice often creates a piece of digital reference known as a hash.  A hash is generated based on the document content using cryptographically sound operations so that only an authorized user (such as document owner or sender) can develop a valid hash, attach with the document, and later the hash can be used by others as a reference. When the document content is tampered, the result from a verifying operation would show inconsistency with the attached hash, thus revealing the content tampering.  Authenticating multimedia content, however, is more sophisticated than text data, because not only can multimedia content can be stored in substantially different format, they are also transcoded in various ways, for example, to reduce the bitrate when transmission bandwidth is limited, to a different size when the end-user has a display device of different size and shape than the senders. An example of size change combined with content tampering is shown in the attached Figure 1.

This NSF supported project tackles the verification of digital data through a novel framework of forensic hash, which is a small-size reference of about a few hundred of bytes that can help reveal more information about the processing history of the data under question and facilitate answering questions about what has been done on the data.  Forensic hash constructions developed through our research provides robust geometric transform estimation, accurate tampering localization, excellent discrimination capability and a novel capability to estimate seam carving – the state-of-the-art resizing tools for images and video. With such capabilities in answering a broad scope of forensic questions in an accurate and efficient way, our proposed forensic hash can be a valuable tool to help evaluate the trustworthiness of electronic media and signals.

We have further leverage the new forensic hash framework and extend it to address an important problem beyond security. Increasingly, images transmitted and rendered at receivers’ end may have a different size and aspect ratio from those of the original, in order to meet the heterogeneous situations and needs of transmission bandwidth and display capabilities.  This calls for automatically monitoring and evaluation of image quality for Quality-of-Service (QoS) assurance in broadband communications and online/mobile media applications.  We use “retargeted images” having undergone content-adaptive resizing as an example and made novel use of the forensic hash framework from this research project to tackle the new challenge of quality assessment for retargeted images. For the U.S. wireless and communication industry to address the market’s needs and global competition, the research outcome from this project contributes to technology innovations. Furthermore, the mentoring and engagement of the undergraduate and graduate students who participated in this research (including female and under-represented minorities) have contributed to the work force development in STEM areas that is critical for sustained technological advancement and prosperity.

Please report errors in award information by writing to: awardsearch@nsf.gov.