Guidelines: Embedding Metadata in DPX Files
Audio-Visual Working Group

Embedded metadata can travel with a digital object during its life cycle and often exists in synergy with metadata in an organization's databases or other information technology systems. Embedded metadata enables people in and outside an organization to work more efficiently, provides valuable data to the systems that preserve digital content, and can assist in disaster recovery. In addition to this project to develop guidelines for DPX files, FADGI offers embedding guidelines for WAVE audio files and two embedding guidelines for still images: for minimal descriptive metadata and for the TIFF header. In addition, the Working Group has developed the BWF MetaEdit tool to support users of the WAVE format.

Overview of DPX Format

Digital Picture Exchange (DPX) is a pixel-based (raster) file format intended for very high quality moving image content with attributes defined in a binary file header. There are two versions of the DPX format, version 1 defined by SMPTE ST 268M-1994 and version 2 defined by SMPTE ST 268M-2003 and Amd. 1:2012. DPX images are produced by scanning motion picture film or by using a camera that produces a DPX output.

Each DPX file represents a single image or frame in a sequence of a motion picture or video data stream with a single component, e.g., luma, or multiple components, e.g., red, green, blue; or Cb, Y, Cr (chroma-luma data). Many variations in multiple component data are supported. As a structured raster image format, DPX is intended to carry only picture or imagery data with corresponding sound carried in a separate format, typically WAVE files. In practice, this means that a single digitized motion picture film will consists of a sequence of tens of thousands of individual DPX files, each file corresponding to a frame of scanned film with sequentially numbered file names as well as a separate audio file for sound data.

FADGI Guidelines for Embedded Metadata in DPX Files

In spring 2016, FADGI Audio-Visual Working Group initiated a project to review the state of embedded metadata in DPX headers from a wide variety of film scanners in use at federal agencies and beyond. The results of the analysis demonstrate that metadata implementation is inconsistent, even in SMPTE core fields.

FADGI has authored guidelines for embedding metadata in the DPX header. The guidelines outline FADGI implementations of the SMPTE Core fields as well as other elements Strongly Recommended, Recommended or Optional for FADGI use. The non-Core fields take advantage of existing header structures as well as define new metadata elements for the User Defined fields to document, among other things, digitization process history.

This document is limited in scope to embedded metadata guidelines and does not look to define other technical characteristics of what a DPX file might carry such as image tonal settings, aspect ratios, bit depths, color models and resolution.  Recommended capture settings are defined for a variety of source material in the companion FADGI document, Digitizing Motion Picture Film: Exploration of the Issues and Sample SOW.

This project is led by the FADGI Film Scanning subgroup with active participation from the Smithsonian's National Museum of African American History and Culture (NMAAHC), the National Archives and Records Administration (NARA), and the Library of Congress including Digital Collections and Management Services, the Packard Campus for Audio-Visual Conservation and the American Folklife Center. Many other agencies also participate, including the National Air and Space Administration (NASA).

Guidelines

Current Version

• Guidelines for Embedded Metadata within DPX File Headers for Digitized Motion Picture Film (PDF, 421 KB). April 23, 2019. CC0 1.0 Universal License
  Changed "Digitization Process History" to "FADGI Process History" to fit in allotted bytes.


• Anonymized analysis of embedded metadata and data structure in sample DPX files. (XLSX File)

Earlier Versions

• Guidelines for Embedded Metadata within DPX File Headers for Digitized Motion Picture Film (PDF, 451 KB). May 7, 2018. CC0 1.0 Universal License
  Corrected language in Source image date/time (field 37) to clarify FADGI Use intends this field to capture the creation date/time of the source material.
• Guidelines for Embedded Metadata within DPX File Headers for Digitized Motion Picture Film (PDF, 413 KB) Approved by Working Group - August 14, 2017. CC0 1.0 Universal License
  The approved version includes significant revisions from the 12/16/16 draft version including a justification about the rationale for embedded metadata, explanations with date/time formatting issues and data overruns as well as other minor adjustments. FADGI will work with SMPTE to resolve some of the inconsistencies in the ST 268M specification uncovered during this project.
• Embedding Metadata in Scanned Motion Picture Film Files: Guideline for Federal Agency Use of DPX Files (PDF, December 16, 2016). Draft for Public Comment.

embARC Open Source Software Beta Release

To faciliate the implementation of the DPX Embedded Metadata Guidelines, FADGI has released the beta version of the embARC open source software. embARC, for Metadata Embedded for Archival Content, is a new open source tool for format validation and batch embedding and correcting metadata within file headers. For more information including downloads, please see embARC.

Presentations

Kate Murray introduced the DPX Embedded Metadata Guidelines at the 2017 IASA Annual Meeting in Berlin, Germany.

A poster and handout (PDF) about the project was presented at the 2016 Association of Moving Image Archivists conference in Pittsburgh, PA.

Working Groups

Still Image Working Group
This group is involved in a cooperative effort to develop common digitization guidelines for still image materials.

Audio-Visual Working Group
This group works collaboratively on common and sustainable technical guidelines, methods, and practices for digitized and born digital sound recordings and moving images.