Federal Agencies
Digital Guidelines Initiative

Audio System Performance | Content Categories & Digitization Objectives | Digital Imaging Framework | Digitizing Motion Picture Film | Digitization Activities - Project Planning | Embedded Metadata in Broadcast WAVE Files | Embedded Metadata in Digital Still Images | Embedded Metadata in DPX Files | Embedded Metadata in TIFF Images |  File Format ComparisonsMXF Application Specification | Technical Guidelines for Digitizing Cultural Heritage Materials

Home > Guidelines > Audio System Performance

Guidelines: Audio Digitization System Performance
Audio-Visual Working Group

Performance of audio digitization systems: overview. The Working Group has been exploring the issues pertaining to the performance testing of audio digitization systems. This activity responds to recommendation 2.4 of the National Recording Preservation Plan and is also intended to parallel efforts by the Still Image Working Group to develop metrics and methods for the performance testing of still image scanners and cameras. For audio digitization, two aspects have been under study: (a) the performance of analog-to-digital converters (ADCs) and (b) the problem of interstitial errors, i.e., accidental loss or transformations of audio samples within the digitizing system before the data stream is written to file. This activity is supported by the Working Group's expert consultant Chris Lacinak of Audiovisual Preservation Solutions.

Performance Guideline for ADCs: Approved and Draft Versions

These guideline pertain to the metrics and measurement methods for ADCs. Its starting point for metrics is Guidelines on the Production and Preservation of Digital Audio Objects (TC-04, Second Edition, 2009; International Association of Sound and Audiovisual Archives). For methods, its starting point is AES standard method for digital audio engineering — Measurement of digital audio equipment (AES17-1998, r2009; Audio Engineering Society), and other standards cited in the guidelines.

FADGI has two levels of ADC performance testing guidelines. Each guideline consists of a test method - instructions on how to perform the tests - and an associated set of performance metrics - a set of pass-fail specifications for the results produced when utilizing the test method. The focus of this endeavor is to evaluate the performance of the ADC device, but it is important to also recognize that the results of the testing are inextricably bound to the quality of the test and measurement system. In other words, the quality of the system used to do the testing will have an impact on the results. This reality has led us to think carefully about the naming and framing of the two guidelines produced to date.

The first guideline addresses the highest level of performance testing for ADCs. To perform the test method and confidently analyze the specified performance metrics requires a sophisticated, high precision test system. Such a test system is able to test ADCs of all performance levels, up to the highest levels of performance, using the guideline. Thus, the “high level performance” designation speaks to both the test system and the performance specifications set for the ADCs under test. In this guideline, performance metrics are primary and the test method and system are crafted around them.

The second guideline addresses the need that many organizations have for a low-cost alternative test system. The performance metrics within the low-cost guideline represent the minimal acceptable results using the low-cost test system. Unlike the high level performance guideline, the low-cost test guideline is not intended or able to test ADC’s of all performance levels. The primary purpose of this guideline is to provide users with the ability to perform a routine protocol that establishes a norm for a given ADC, so that failures within that system are able to be identified.

Therefore these two guidelines should not be thought of as nested documents, where the low-cost guideline is simply a subset of tests and lesser performance metrics compared to the high level performance guideline. Despite the fact that they come out of the same body of work, they are purposefully different in intent, design, and application.

High Level Performance

The pass-fail specifications recommended in this guideline are for the highest level of ADC performance which require relatively expensive equipment and advanced engineering skills. The rationale and implementation details for testing against this standard are presented in two related documents: Audio Analog-to-Digital Converter Performance Specification and Test Method: Introduction, August 20, 2012 and ADC Performance Testing: Report on Project Development During 2015.

Audio Analog-to-Digital Converter Performance Specification and Testing: High Level Performance, v. 1.1 (PDF, 550 KB)
Approved by Working Group, July 2016

Audio Analog-to-Digital Converter Performance Specification and Testing: High Level Performance, v. 1.0 (PDF, 240 KB)
Approved by Working Group, August 20, 2012.

Audio Analog-to-Digital Converter Performance Specification and Testing (PDF, 217 KB)
Earlier version. Drafted in February 2012 and reformatted June 19, 2012.

Low Cost Performance

The pass-fail specifications recommended in this guideline are for modified performance testing which takes into account minimum levels of performance, with lower equipment costs and requiring more modest engineering skills.

Audio Analog-to-Digital Converter Performance Testing Specification: Low Cost (PDF, 154 KB)
Draft for Working Group Review, August 11, 2017

Audio Analog-to-Digital Converter Performance Testing: Low Cost Test Setup User Guide (PDF, 99 KB)
Submitted in September 2016 and draft released to Working Group for review in August 2017

Audio Analog-to-Digital Converter Performance Testing: Template for Recording Results of Low Cost Performance Testing (XLXS, 75 KB)
Submitted in September 2016 and draft released to Working Group for review in August 2017

Audio Analog-to-Digital Converter Performance Testing: Low Cost Performance Test Setup and Test Method (PDF, 194 KB)
Submitted in September 2016 and draft released to Working Group for review in August 2017.
This report was submitted in September 2016 before the software development phase of the project began. As research and development progressed, it became clear that some of the concepts and ideas expressed within could not be executed or supported in the proposed manner and adjustments have been made along the way. Although some aspects have changed on the path from planning to execution of the final low cost testing system, FADGI sees value in making this report available as background material for the overall ADC Performance Testing project which continues to evolve.

Performance Guideline for ADCs: Related Documents

ADC Performance Testing: Report on Project Development During 2015 (PDF 4.6 MB). Detailed technical report that includes comparisons of the 2016 and 2012 guidelines, a discussion (and proposed guideline) for lower-cost testing of performance at lower levels of quality, and a sketch of actions planned for the next round of work. Appendix to the report (ZIP file, 1 MB) includes instructions and device scripts used in 2015 testing for high level performance.

Audio Analog-to-Digital Converter Performance Specification and Test Method: Introduction, August 20, 2012. (PDF, 621 KB) Archived for reference: The earlier version drafted in February 2012 and reformatted June 19, 2012. (PDF, 506 KB)

Summaries of comments received on the guideline and introduction as of June 21, 2012 (PDF, 84 KB) and July 12, 2012 (PDF, 119 KB). These comments were considered when the August 20, 2012, version of the guideline was drafted.

Analog-to-Digital Converter Performance Specification and Testing, the original version of the combined explanatory report and recommended guideline dated February 24, 2012. (PDF, 499 KB).

Performance Impairments Caused by Interstitial Errors: Field Test Report

Interstitial errors consist of lost or altered samples within the recorded file, resulting in the loss of content and integrity. These errors--often very momentary--result from a failure in the chain of digital data, i.e., in the handoff from the ADC to the digital audio workstation (DAW), and in the DAW's writing of the file to a storage medium. The first item linked below, titled Study Report, describes a 2012 field test carried out in three federal audio facilities, designed to identify occurrences of interstitial errors in the course of actual work. The second item, titled Appendixes and completed in 2012, provides background information as well as re-presenting sections from the consultant's initial 2011 report.

Interstitial Error Study, Volume I. The Study Report. Presented to the Working Group April 8, 2013. (PDF, 3.6 MB).

Interstitial Error Study, Volume II. Appendixes. Presented to the Working Group September 11, 2012. (PDF, 1.9 MB).


Working Groups

Still Image Working Group
This group is involved in a cooperative effort to develop common digitization guidelines for still image materials.

Audio-Visual Working Group
This group works collaboratively on common and sustainable technical guidelines, methods, and practices for digitized and born digital sound recordings and moving images.

Last Updated: 08/24/2017