Raw image format

A file format that contains minimally processed data from the image sensor.

A camera raw image file contains minimally processed data from the image sensor of either a digital camera, a motion picture film scanner, or other image scanners. Raw files are named so because they are not yet processed and therefore are not ready to be printed or edited with a bitmap graphics editor. Normally, the image is processed by a raw converter in a wide-gamut internal color space where precise adjustments can be made before conversion to a "positive" file format such as TIFF or JPEG for storage, printing, or further manipulation. There are dozens of raw formats in use by different manufacturers of digital image capture equipment.

Raw image files are sometimes incorrectly described as "digital negatives", but neither are they negatives nor do the unprocessed files constitute visible images. Rather, the Raw datasets are more like exposed but an undeveloped film that can be converted (electronically developed) in a non-destructive manner multiple times in observable, reversible steps to reach a visually desired image. (With exposed film, development is a single event that physically transforms the unexposed film irreversibly.)

Like the undeveloped photographic film, a raw digital image may have a wider dynamic range or color gamut than the developed film or print. Unlike physical film after development, the Raw file preserves the information captured at the time of exposure. The purpose of raw image formats is to save, with minimum loss of information, data obtained from the sensor.

Raw image formats are intended to capture the radiometric characteristics of the scene, that is, physical information about the light intensity and color of the scene, at the best of the camera sensor's performance. Most raw image file formats store information sensed according to the geometry of the sensor's individual photo-receptive elements (sometimes called pixels) rather than points in the expected final image: sensors with hexagonal element displacement, for example, record information for each of their hexagonally-displaced cells, which a decoding software will eventually transform into the rectangular geometry during "digital developing".

File contents

Raw files contain the information required to produce a viewable image from the camera's sensor data. The structure of raw files often follows a common pattern:

  • A short file header which typically contains an indicator of the byte-ordering of the file, a file identifier, and an offset into the main file data
  • Camera sensor metadata which is required to interpret the sensor image data, including the size of the sensor, the attributes of the CFA, and its color profile
  • Image metadata which can be useful for inclusion in any CMS environment or database. These include the exposure settings, camera/scanner/lens model, date (and, optionally, place) of shoot/scan, authoring information, and others. Some raw files contain a standardized metadata section with data in Exif format.
  • An image thumbnail
  • Most raw files contain a full-size JPEG conversion of the image, which is used to preview the file on the camera's LCD panel.
  • In the case of motion picture film scans, either the timecode, keycode or frame number in the file sequence which represents the frame sequence in a scanned reel. This item allows the file to be ordered in a frame sequence (without relying on its filename).

The sensor image data

Many raw file formats, including IIQ (Phase One), 3FR (Hasselblad), DCR, K25, KDC (Kodak), CRW CR2 CR3 (Canon), ERF (Epson), MEF (Mamiya), MOS (Leaf), NEF NRW (Nikon), ORF (Olympus), PEF (Pentax), RW2 (Panasonic) and ARW, SRF, SR2 (Sony), are based on the TIFF file format. These files may deviate from the TIFF standard in a number of ways, including the use of a non-standard file header, the inclusion of additional image tags, and the encryption of some of the tagged data.

Panasonic's raw converter corrects geometric distortion and chromatic aberration on such cameras as the LX3, with necessary correction information presumably included in the raw. Phase One's raw converter Capture One also offers corrections for geometrical distortion, chromatic aberration, purple fringing, and keystone correction emulating the shift capability of tilt-shift in software and specially designed hardware, on most raw files from over 100 different cameras. The same holds for Canon's DPP application, at least for all more expensive cameras like all EOS DSLRs and the G series of compact cameras.

DNG, the Adobe digital negative format, is an extension of the TIFF 6.0 format and is compatible with TIFF/EP, and uses various open formats and/or standards, including Exif metadata, XMP metadata, IPTC metadata, CIE XYZ coordinates, ICC profiles, and JPEG.

Sensor image data

In digital photography, the raw file plays the role that photographic film plays in film photography. Raw files thus contain the full resolution (typically 12- or 14-bit) data as read out from each of the camera's image sensor pixels.

The camera's sensor is almost invariably overlaid with a color filter array (CFA), usually, a Bayer filter, consisting of a mosaic of a 2x2 matrix of red, green, blue, and (second) green filters.

One variation on the Bayer filter is the RGBE filter of the Sony Cyber-shot DSC-F828, which exchanged the green in the RG rows with "emerald" (a blue-green or cyan color). Other sensors, such as the Foveon X3 sensor, capture information directly in RGB form (using three-pixel sensors in each location). This RGB raw data still needs to be processed to make an image file, because the raw RGB values correspond to the responses of the sensors, not to a standard color space like sRGB. As there is no Bayer or other mosaic, there is no need for demosaicing.

Flatbed and film scanner sensors are typically straight narrow RGB or RGBI (where "I" stands for the additional infrared channel for automatic dust removal) strips that are swept across an image. The HDRi raw data format is able to store the infrared raw data, which can be used for infrared cleaning, as an additional 16-bit channel. The remainder of the discussion about raw files applies to them as well. Some scanners do not allow the host system access to the raw data at all, as a speed compromise. The raw data are processed very rapidly inside the scanner to select out the best part of the available dynamic range so only the result is passed to the computer for permanent storage, reducing the amount of data transferred and therefore the bandwidth requirement for any given speed of image throughput.

To obtain an image from a raw file, this mosaic of data must be converted into standard RGB form. This is often referred to as "raw development".

When converting from the four-sensor 2x2 Bayer-matrix raw form into RGB pixels, the green pair is used to control the luminance detail of the processed output pixel, while the red and blue, which each have half as many samples, are used mostly for the more slowly-varying chroma component of the image.

If raw format data is available, it can be used in high-dynamic-range imaging conversion, as a simpler alternative to the multi-exposure HDI approach of capturing three separate images, one underexposed, one correct, and one overexposed, and "overlaying" one on top of the other.

Standardization

Providing a detailed and concise description of the content of raw files is highly problematic. There is no single raw format; formats can be similar or radically different. Different manufacturers use their own proprietary and typically undocumented formats, which are collectively known as raw format. Often they also change the format from one camera model to the next. Several major camera manufacturers, including Nikon, Canon, and Sony, encrypt portions of the file in an attempt to prevent third-party tools from accessing them.

This industry-wide situation of inconsistent formatting has concerned many photographers who worry that their valuable raw photos may someday become inaccessible, as computer operating systems and software programs become obsolete and abandoned raw formats are dropped from new software. The availability of high-quality open-source software which decodes raw image formats, particularly dcraw, has helped to alleviate these concerns. An essay by Michael Reichmann and Juergen Specht stated "here are two solutions – the adoption by the camera industry of A: Public documentation of RAW formats; past, present, and future, or, more likely B: Adoption of a universal RAW format". "Planning for Library of Congress Collections" identifies raw-file formats as "less desirable file formats", and identifies DNG as a suggested alternative.

DNG is the only raw image format for which industry-wide buy-in is being sought. It is based upon, and compatible with, the ISO standard raw image format ISO 12234-2, TIFF/EP, and is being used by ISO in their revision of that standard.

The ISO standard raw image format is ISO 12234-2, better known as TIFF/EP. (TIFF/EP also supports "non-raw", or "processed", images). TIFF/EP provided a basis for the raw image formats of a number of cameras. For example, Nikon's NEF raw files are based on TIFF/EP, and include a tag that identifies the version of TIFF/EP they are based on. Adobe's DNG raw file format was based on TIFF/EP, and the DNG specification states "DNG ... is compatible with the TIFF-EP standard". Several cameras use DNG as their raw image format, so in that limited sense, they use TIFF/EP too.

Adobe Systems launched this DNG raw image format in September 2004. By September 2006, several camera manufacturers had started to announce support for DNG in newer camera models, including Leica, Samsung, Ricoh, Pentax, Hasselblad (native camera support); and, Better Light (export). The Leica Digital-Modul-R (DMR) was the first to use DNG as its native format. In September 2009 Adobe stated that there were no known intellectual property encumbrances or license requirements for DNG. There is a "Digital Negative (DNG) Specification Patent License", but it does not actually state that there are any patents held on DNG, and the September 2009 statement was made at least 4 years after this license was published.

TIFF/EP began its 5-year revision cycle in 2006. Adobe offered the DNG specification to ISO to be part of ISO's revised TIFF/EP standard. A progress report in October 2008 from ISO about the revision of TIFF/EP stated that the revision "... currently includes two "interoperability-profiles," "IP 1" for processed image data, using ".TIF" extension, and "IP 2" for "raw" image data, ".DNG" extension". It is "IP 2" that is relevant here. A progress report in September 2009 states that "This format will be similar to DNG 1.3, which serves as the starting point for development."

DNG has been used by open-source developers. Use by camera makers varies: the largest companies such as Canon, Nikon, Sony, and some others, do not use DNG. Smaller companies and makers of "niche" cameras who might otherwise have difficulty getting support from software companies frequently use DNG as their native raw image format. Pentax uses DNG as an optional alternative to their own raw image format. There are 15 or more such companies, even including a few that specialize in movie cameras. In addition, most Canon point & shoot cameras can support DNG by using CHDK.

Canon Raw v2, CR2, is mostly based on TIFF and lossless Jpeg ITU-T81

Canon Raw v3, CR3 is based on ISO Base Media File Format (ISO/IEC 14496-12), with custom tags, and an unknown "crx" codec.

acronymn
  • RAW
resources
source
Adapted from content published on wikipedia.org
Last modified on June 13, 2021, 8:31 pm
Videocide.com is a service provided by Codecide, a company located in Chicago, IL USA.
magnifier