Technical Guidelines Third Edition Revision Comments

"I was impressed with "DRAFT Technical Guidelines for Digitizing Cultural Heritage Materials - 3rd Edition", especially the care and thought that went into the metadata section. The emphasis on evaluation and sampling was also impressive. My only minor critique is on this sentence in section 6.7: "Optical Character Recognition (OCR) is the process of converting a raster image of text into searchable American Standard Code for Information Exchange (ASCII) data" I realize most docs the federal gov't deals with are in English or other ascii'able language, but it might be better to use "searchable electronic text" or something that doesn't narrow the encoding to something as limited as ASCII. Hopefully, the output of OCR would be in UTF-8 or UTF-16, for example."

Received June 6, 2022

"Is there a version or edit history that documents the differences between the 2nd and 3rd editions of the FADGI Guidelines? Your posting on The Signal gives an overview of the additions and changes. A detailed edit history would be useful since it would allow a reviewer to focus on the sections that have changed and ignore those that are the same as the 2nd edition, which presumably are no longer up for comment. Or are comments being invited on the entire document and just not the amendments to the 2nd edition?"

Received June 13, 2022

Response from Working Group: In section 1.2 of the document (beginning on page 9) there is a more complete list of what was update between the 2nd and 3rd editions, however, it is not a granular list. Comments are invited on the entire document, please feel free to share thoughts or recommendations on any area.

"In Section 2.4.3, I was delighted to see the acknowledgement that gray target patches are not—and are not guaranteed to be—truly colorless. However, because of this, when setting white balance in software, not all gray patches will return the same white balance values. Perhaps it's worth mentioning that the a* and b* values printed on a DICE target can be used as a guide to identify the most spectrally neutral gray patch for the user to define the imaging system's white balance during calibration. Then, once optimized, white balance settings can (should?) be saved for future use. Accurate and precise white balance over time also relies on the cleanliness and/or age of the target patch.

In Section 3.6, it would be interesting to include a rule of thumb that the Library of Congress follows to define an "oversize" item's dimensions. Larger than what will fit in a traditional manuscript box? Larger than 30 x 40 inches? This might help give some direction to institutions balancing digitizing historically significant posters that warrant 4-star performance and are technically prints, but may be impractical to digitize at the [Prints and Photographs] 4-star sampling frequency (i.e. 600ppi) due to maximum imaging system resolution, limited field of view, stitching methodologies/capabilities, etc.

It is great to see FADGI shift toward star-level tolerances that use colorimetric values rather than RGB count values in the measurement parameters for each material type."

Received July 5, 2022

"I noticed today that the Dynamic Range metric is grayed out in the new version and also noticed that Dynamic Range is also not defined. I'm assuming it its an optical density value based on the numbers given but I think it would be helpful to define it. Most of the other metrics are defined earlier in the document and perhaps a brief description could be included there.

I'm also a bit confused by the Highlight/Shadow description: "2.4.14 Highlight/Shadow, Tolerance These values remain under review. References to legacy digital counts are being converted to L* values."

Is this to mean in the final document that the given RGB values in the draft are going to be converted to L* values? (to clarify, the metric in transmissive materials is given in RGB)"

Received July 7, 2022

"FADGI is going in the right direction. Changes are positive and switching to Lab units is good. Here are a question and some comments. Question: I don’t really understand what the tolerances written in light gray are: (9, 6 and 3 values for example for [Tone Response (OECF)] “Bound Volumes: Rare and Special Materials”).

2022:

It seems to be the [same] ones used in the 2010 version:

In the 2016 version, I understood that the light gray tolerances were those from 2010: 9, 6 and 3. 2016 tolerances are different (8, 5 and 2 count levels for example for [Tone Response (OECF) (Luminance)]"Bound Volumes: Rare and Special Materials").

2016:

Comments: Why not use delta E 2000 formula with SL=1 like in ISO19264-1? See Roy Berns for more details about SL=1.

Tone scale tolerances (from "Bound Volumes: Rare and Special Materials"): +/- 2L* for 4 star is OK (same as ISO19264-1 level A) but +/- 5L* for 3 star looks wide (L* go from 0 to 100 and RGB from 0 to 255 so +/-5L* means +/- 12 RGB levels as it was +/- 4 in 2016 version). And 2 star gives +/-8L* which is more than 20 RGB levels! Is there a mistake in units? (RGB instead of L*?). Looks like unit changed from RGB to L* but values stayed the same.

2022:

2016:

1% lightness uniformity is difficult to achieve and maintain, especially on large formats like A0. Did you consider enlarging tolerances for large formats?"

Received July 15, 2022

Comments submitted by Digital Transitions, Inc.:

"2.4.4 Lightness Uniformity. We suggest this section be extended to discuss the inherent complexity objects with a glossy or semi-gloss surface and for material that has dimensionality. Targets tend to have glossy or semi-gloss patches which will provide a measurement of illumination uniformity that is not relevant to a matte-paper subject in the same lighting. This can lead to situations where the target indicates unacceptable illumination uniformity while real-world subjects produce excellent results. Even worse, it can lead to situations where the target indicates acceptable illumination uniformity while real-world subjects do not. A robust way to check for genuine illumination uniformity is to capture the subject at 0º and 90º rotations and then digitally rotate them to match; it's the same subject in each case so all parts of the subject should be the same brightness in both captures.

2.4.5 Color Accuracy. "targets with a small number of color and density patches can provide a good analysis of system consistency, but cannot provide information suitable for creating ICC color profiles. Targets with 100 or more measurement patches will provide a better input to ICC profile creation software" This is clearly labeled as a "general rule" but I think it's worth fleshing out some of what the "general rule" will miss. We would suggest...

'However, quantity of patches is not the only driver of how well suited a given physical target is serve as an input for calibration. Other factors include how well matched the patches are to the range of material in the collection being imaged, the flatness/parallelism of the patches (some targets tend to bow over time). It's also important to keep targets clean over time, and not all targets can be cleaned. The Library of Congress has sponsored considerable research in this area, including the creation of a Next Generation Target for color profile creation and validation.'(*)

Also I think there is a minor typo in this section. "routing" should be "rotating".

*A reference to "Next Generation Target (NGT) Evaluation Task A: Evaluate the effectiveness of the three color targets" can be included in the references.

2.4.6 Color Channel Mis-Registration. "Poor registration of the three color channels is a consequence of poor lens design or aperture choice." We suggest this continue... or can indicate damage to the physical lens. In systems that do not capture R,G, and B in a single sensor capture color channel misregistration can also indicate issues of mechanical calibration such as a failing stepper motor on a scanner.

2.4.7 SFR* (Spatial Frequency Response) "It is related to the sharpness, focus, and ability to resolve fine details." - we suggest vibration and alignment to the system attributes that SFR elucidates. A perfectly sharp imaging system, exposed to problematic vibration, produces poor SFR results.

We also suggest this section be extended to include a statement such as: 'As with all metrics in this guide, the specific points of the SFR curve used for pass/fail metrics in FADGI are a surrogate rather than the ends to themselves. The underlying goal should be a system which is evenly and correctly rendering detail and in which the naturalistic appearance of detail is not compromised by post processing such as sharpening in such a way that the nature of the material is misrepresented. Especially at high resolutions, and in non-continuos subjects such as photographic film, the calculation of SFR from a target leads to a "noisy" curve which may undulate; it is the overall shape of the curve that is of underlying interest as an evaluative tool rather than the point measurement at exactly 50% and 10% of the modulation which forms the metric. See figure 3 for an example of the ideal overall curve that should be the goal of an imaging system.'

2.6.1 Room. "The working environment should be painted/decorated a neutral, matte gray with a 60% reflectance or less to minimize flare and perceptual biases." We suggest also indicating a desirable minimum gray reflectance. Digitization is done by people, and people do poorly (both psychologically and in regards to the utility of their vision systems) in extremely dark environments. Institutions that read "60% or less" may assume that "darker is better" and that gray in this context only means "neutral" not "midtone". A medium or dark gray is preferable to a black.

2.6.2 Monitor, Light Boxes, and Viewing Booths AND 2.6.6 Lighting. Technology has improved since the previous iteration of this document. We would suggest the minimum CRI for viewing booths and light boxes be elevated to 94, and we would suggest CQS or TM30-15 be provided as a secondary metric as many lighting manufacturers "teach the test" of CRI in their manufacturing decisions, achieving CRI scores that aren't warranted by a holistic analysis of their spectral output. We would also suggest that "A CRI above (X) is generally accepted as appropriate for most cultural heritage imaging" be continued with "; however, when available and within budget a higher CRI/CQS is more desirable.

"Single exposure total area capture scanning systems are considered the most appropriate technologies when imaging special collections materials, including documents." The use of the word "scanning" here is misleading. The point of this statement, as we understand, is that instant capture single-shot camera systems are preferable to planetary scanners or other devices that use motion during capture or between multiple captures. Suggest changing "scanning" to "camera" or "instant capture".

"Given the lack of calibration targets available for negative films, and appropriate software with which to create calibrations, it is recommended to manually establish scan settings based on highlight, shadow, and midtone measurements of the image being scanned. These settings may need to be changed with every scan, based on the original."

We do not agree. Modifying exposure for each image is both impractical, subjective, and removes the ability for the viewer to understand the original nature of the material. We suggest this be expanded to allow for a workflow that uses absolute reference rather than subjective and subject-based decisions. For example "targets for film scanning are less common and less broadly available than for reflective material scanning. In the case that a suitable target is not available at an institution it is minimally acceptable to (insert the workflow you currently suggest here). However, as with reflective materials it is preferable to digitize transmissive materials using targets and profiles to create absolute reference points of tone and color, at least for the master file.

Also, we would advocate here for the inclusion of a call for object level targets for transmissive materials so that a target can be included in frame as is the case for reflective material today. This document needn't be only retrospective on current options available but can serve as a call to arms for the community to create new tools where a gap (as noted in this paragraph) exists.

"Imaging with narrow band blue light has been shown to increase the resolution and reduce the effects of Newton’s rings when film is imaged between glass."

We suggest clarifying that this is only applicable for monochrome cameras. A narrow band blue light harms the resolution for a standard Bayer sensor camera. This is clarified in the earlier paragraph but not in this bullet point.

"X-Ray Film (Radiographs)". This section recommends against the use of "color digital cameras" and for "monochrome cameras with HDR capabilities" – the use of the term "HDR" here might confuse or mislead readers. The core need here is for accurate rendering of tone across a wide range of subject density. Some cameras may use a "HDR" mode which combines multiple raw files while others might use a different sensor mode during the creation of a single raw files.

We suggest changing this to "specialized cameras with high dynamic range".

4.1 Camera. "In essence, the true resolution is far less than the stated resolution, and the color is interpolated from the data from four pixels." – as technology in demosaicing continues to improve since the previous version of this document we would advise to remove the word "far" from this sentence. Interpolation always causes loss, but the amount of loss in high-end raw processors such as Capture One CH is now quite small.

4.2 Scanner. "To this day, drum scanners provide the highest image quality of all imaging devices" As technology continues to improve we would advise to caveat this statement. Something like "drum scanners outperform most modern imaging devices". For example, the dynamic range and noise of a Phase One back using the dual-exposure or frame-averaging mode is several times better than the noise floor produced by a PMT.

5.1.3 Insource vs. Outsource. We suggest a closing sentence be included reading "The recommendations for image quality and the tools and methods to monitor them remain the same whether an image is produced in-house or via outsourcing, and these guidelines and a FADGI star rating are an excellent tool to include in the covenants of any outsourcing contract as they are objective, independent and testable.

Received July 15, 2022

"The "Technical Guidelines for Digitizing Cultural Heritage Materials - 3rd Edition” (draft 3.5) is largely a step in the right direction. Two items of note are moving to L* colorimetric certifications and adding the Modern Textual Records section. The L* colorimetric certifications are a significant step towards achieving test criterion that is consistent across encoding spaces. The addition of the Modern Textual Records adds a set of test criterion that matches the scanning needs of commonly printed documents.

However, the sharpness metrics are largely unchanged. Section 2.4.7 renames MTF to SFR and includes a slight narrowing in the allowable range for the SFR50 metric (formerly “MTF50”) – increasing the lower limit from 0.35 to 0.40 for FADGI 3-star. The number of SFR metrics are excessive, especially in comparison to the other FADGI evaluation metrics. SFR10 measurements are noisy, easy to undermine with post-filtering and not highly correlated with visible image quality. Interpreting SFR10 metrics is problematic and we would recommend eliminating them from FADGI 2022.

Section 3.4 Documents (Unbound): General Collections includes a sub-section listing “Not Recommended Technologies”. This list includes “Pass through manual or automatically fed document scanners” and states that “This class of equipment often introduces streak artifacts in the imaging process, which are not FADGI compliant”. This unfairly limits the utility of ADF scanning. When used as directed, these platforms can deliver streak-free imagery and at speeds that far exceed alternate methods. ADF scanners have been used for archival document scanning for decades already. Service Providers have experience in utilizing ADFs to provide streak-free scans. The definition of materials in this category includes “new, clean and easy to handle materials”. There is no reason that these materials could not be scanned, FADGI compliant, using an ADF scanner.

In Section 3.5, Documents (Unbound): Modern Textual Records, the commentary suggests the L* change only impacts OECF Tonescale & White Balance metrics, but it also impacts the Noise metric test. Noise level is no longer tested against separate R, G, B channels – it is to be tested against a single channel noise calculated in L*. This is described in section 2.4.10 and again in the table in all the sections. However, the comment “This will impact two metrics, OECF and White Balance when using digital image conformance evaluation test patterns.” is listed below the table in Section 3.5. It should be changed to read “This will impact three metrics, OECF, White Balance, and Noise, when using digital image conformance evaluation test patterns.”

Received August 4, 2022

"A few questions / clarifications requested re: the 2022 Revised Guidelines: Regarding 4" x 5" and larger transmissive film formats: as of August 2022, the largest commercially available imaging system appropriate for what would be considered mass digitization has a native pixel size of 14,204 x 10,652. This means that digitization personnel are unable to capture 4" x 5" films in a single shot at the 3,000 / 2,940 ppi minimum resolution required for FADGI 3-Star. Similarly, an 8" x 10" film or glass plate cannot achieve the proposed resolution specs for 3- or 4-Star in a single shot using this imaging system. Can FADGI please clarify the recommended approach to achieve 3-Star digitization of 4x5 and larger formats for mass digitization? In practice, stitching multiple panels is usually cost- and time-prohibitive on larger projects, and stitching continuous tone photos without introducing undesirable artifacts may be impossible.

Regarding 3.15 Printer Matter, Manuscripts, and Other Documents on Microfilm: FADGI defines resolution as pixels per inch relative to the microfilm itself and acknowledges that the reduction ratio originally used to create the microfilm may be unknown or lost. By itself this is fine, but in the context of actual projects where source documents are required to be scaled back up from microfilm upon output, digitization personnel are often tasked with delivering images with a specific resolution relative to the source documents instead of the microfilm (e.g., 300-400ppi for NDNP newspapers conversions from microfilm). As well, many OCR engines expect input files to be in the 200-400ppi range and may behave differently than expected if they attempt to process an input image showing resolution in the thousands of pixels per inch. The resolution measurement itself may confuse many end users if it is not clear if it relates to the microfilm or the source documents. Can FADGI clarify its recommendations on converting between resolution-on-film versus resolution-of-source-documents on FADGI-compliant microfilm projects, especially when reduction ratio may be unknown but when a specific resolution of the scaled-up source documents is required?"

Received August 5, 2022

"PFU/Fujitsu’s area of expertise is in capture and digitization of modern paper-based information. Our request for consideration is for change within the 3.5 Documents (Unbound): Modern Textural Records specifications.

3.5 Documents (Unbound): Modern Textural Records – Considerations

• These documents are categorized as newer, text-based documents with a high contrast between text and background on modern office paper. These documents were created primarily for the information contained in the text, and therefore capture of the text is important.
• While not requiring a specific star rating, 3.5 Documents (Unbound): Modern Textural Records have the similar specification requirements as the 3.4 Documents (Unbound): General Collections 3 Star rating.
• The Recommended Technologies for 3.5 Documents (Unbound): Modern Textural Records include ADF or Sheetfed scanners. This technology is the predominantly used scanning technology for these types of documents today. All but the most expensive ADF or Sheetfed scanners struggle to maintain the 3.4 Documents (Unbound): General Collections 3 Star rating, or equivalent specifications. Requiring these similar specifications will increase the cost of scanning 3.5 Documents (Unbound): Modern Textural Records exponentially, with little to no improvement in the digitization and capture of the text information. This, in turn, may potentially impose an undue financial burden on end users that need to meet such standards.

PFU/Fujitsu Recommendation: Based on our understanding on the information above, we would like to propose changes for consideration, to the 3.5 Documents (Unbound): Modern Textural Records requirements. Our proposed changes are shown in the column heading “PFU/Fujitsu Recommendation” in the table below. This proposal reduces the specification requirements in areas that would be less important for capturing text-based information in the Modern Textural Records category."

Received August 5, 2022