Risk Properties Definition and Extraction (CPP-023)

CPP-IdentifierCPP-023
CPP-LabelRisk Properties Definition and Extraction
AuthorBertrand Caron
ContributorsKris Dekeyser
EvaluatorsFranziska Schwab, Maria Benauer, Felix Burger
Change historyComments
Version 1.0.0 - 2025-08-29Milestone version
Version 2.0.0 - 2025-04-21Substantial revision of the description section and the process description to align interdependencies with CPP-012: Short definition and label modified - Revision of the global outputs and the structure of the Definition and scope section - Separation of the step-by-step description into Assessment and Extraction - Migration to XML

1. Description of the CPP

The TDA monitors preservation risks and defines detection methods for determining whether the deposited digital objects are subject to these risks.

Inputs and outputs

Input(s)
Alerts
Technology alerts
Community alerts
Output(s)
Documentation/guidance
Risk property
Risk property detection method
Updated risk inventory

Definition and scope

Risk Properties Definition and Extraction is a multi-tiered process that a) identifies risks caused by the specific properties and content of a file, b) defines methods to detect these risks based on file analysis (i.e. format identification, metadata extraction and format validation), and c) holds responsibility for performing technology watch. Together with CPP-012 Risk Mitigation, which defines which actions can be undertaken to reduce the likelihood or impact of the risk, CPP-023 Risk Properties Definition and Extraction is an essential process in the scope of the broader, on-going “risk management” activity.

Risk Properties Definition and Extraction is also the process responsible for technology watch. It receives updates from Community Watch (CPP-018), such as conference papers, specialised literature and journals, vendor announcements, etc. Based on that, it evaluates the impact of the risk with regard to the skills, objectives and means of the Designated Community. In particular, the two following activities are in scope of this CPP:

  • Risk Identification involves cataloging potential threats to digital materials, including technological obsolescence (e.g. hardware or software becoming unavailable), media degradation (e.g. physical deterioration of storage devices), format obsolescence ( e.g. file formats being no longer supported), organisational risks (e.g. loss of institutional knowledge or funding), and environmental hazards (e.g. human failure and disasters such as power failures or security breaches).
  • Risk Assessment evaluates the likelihood and potential impact of identified risks. This typically involves analysing object- and organisation-related factors like the criticality of the digital materials; the object’s primary and secondary value; the stability of their formats; and the resources available for preservation activities.

Problems in the rendering or reuse of Files by the TDA’s end users may stem from several reasons. For example, one common reason is the use of certain file format features, that rendering tools may misinterpret. To mitigate such misinterpretation, preservation actions (e.g. CPP-014 File Migration, CPP-027 File Repair, CPP-015 Emulation and Rendering Tools, etc.) should go beyond file format or validity status and consider risk properties. Risk properties are File or Representation properties that help identify whether a File is subject to a specific risk. Some examples of risk properties are:

  • Discontinued support of file format;
  • Erroneous file format structures;
  • Encryption;
  • External dependencies (e.g. non-embedded fonts);
  • Embedded Files;
  • Advanced multimedia features;
  • Specific creating application, when said application is known to have created faulty Files;
  • etc.

It is important to note that risk properties, in some cases, may be also considered significant properties, however, this is not necessarily the case For example, a valid PDF may be encrypted with an empty password. This is a risk property (it may endanger the following preservation processes) but it may not be considered a significant property (the TDA may decide to decrypt the PDF and keep it decrypted). Risk properties are typically defined by digital preservation analysts, while significant properties should be defined by the TDA management and stakeholders.

The step-by-step description below focuses more specifically on the risk extraction part of the process. Like Risk Mitigation, Risk Properties Definition and Extraction focuses on a specific subset of risks - in this case, risks related to inherent properties of Files. Although of major impact, the definition and detection of risks caused by organisational, financial or security issues are not described.

Process description

Trigger event(s)

Trigger EventCPP-identifier
Community alertsCPP-018 (Community Watch)
Technology alerts

Step-by-step description

NoSupplierInputStepsOutputCustomer
 sequence
1CPP-018 (Community Watch)Information sources for digital preservationReceive community and technology alertsRisk
CPP-010 (File Format Validation)
CPP-027 (File Repair)
New findings gained through technical analysis
CPP-023 (Risk Properties Definition and Extraction)Risk properties policy and detection method
2RiskEvaluate the risk likelihood and impact.Assessment of risk likelihood and impact
3RiskGather a test set of Files (compiled of both files that are subject to this risk and a control group).Test set
4RiskSelect a candidate tool for identifying the riskCandidate extractor tool
5Test setRun the candidate extractor tool on the test setCandidate extractor tool output
Candidate extractor tool
6Candidate extractor tool outputDetermine which property in the candidate extractor tool output helps distinguish those Files that are affected by the risk and those that are not.Successful identification of a risk property detection method (extractor tool) (step 7)
No method identified: resume steps 3-5
7Chosen extractor tool outputDetermine the interpretation of the extractor tool outputRisk property detection method: path / expression to extract the risk property from the tool’s output.(This interpretation should resolve to a true/false statement about the file being subject to the risk or not. It can be simple (an XPATH to the textual content of an element in an XML output) or more complex. See, about this, the use case about multipage TIFFs below.)
8RiskUpdate the risk inventory with the risk, risk property and risk property detection method.Updated risk inventoryCPP-009 (Metadata Extraction)
Risk property
Risk property detection method: path / expression to extract the risk property from the tool’s output

Rationale(s) and worst case(s)

RationaleImpact of inaction or failure of the process
Risk assessment is critical for digital preservation activities because identifying risks related to the inherent characteristics of Files is required for a semantic / logical preservation approach.The impact of the absence of risk assessment on Files’ characteristics may lead to different problems:
  • Broken, undetected dependencies;
  • Unusual features badly supported by rendering tools;
  • etc.

2. Dependencies and relationships with other CPPs

Dependencies

CPP-IDCPP-TitleRelationship description
CPP-008File Format IdentificationRisks can be related to file format generic properties and concern all instances of it. In such a case, File Format Identification is sufficient to determine whether the file is affected by the risk. For example, Microsoft announced the discontinuation of its software product Publisher. As a consequence, all .pub files are now affected by the absence of a suitable software environment.
CPP-009Metadata ExtractionRisks can be related to properties common to one or several file formats. In such a case, Metadata Extraction is required to determine whether the file is affected by the risk. For example, running a metadata extractor like Apache Tika is required to check whether a PDF is encrypted or not.
CPP-018Community WatchRisk is measured against the skills and tools available in the Designated Community.

Other relations

RelationCPP-IDCPP-TitleRelationship description
Affinity withCPP-019Data Quality AssessmentRisk Properties Definition and Extraction and CPP-019 both define properties that the TDA should consider and assess against the result of CPP-009 (Metadata extraction).
Required byCPP-009Metadata ExtractionRisk Properties Definition and Extraction provides the requirements for the selection of an appropriate extractor tool.
Required byCPP-012Risk MitigationRisk Properties Properties Definition and Extraction maintains a risk inventory that guides the Risk mitigation process.
Required byCPP-014File MigrationRisk Properites Definition and Extraction identifies risks related to file format that would trigger File Migration and provides the method for their detection.

4. Reference implementations

Use cases

Identification of PDF preservation risks

Institutional background
InstitutionKoninklijke Bibliotheek, NL
Hyperlink https://bitsgalore.org/2023/05/25/identification-of-pdf-preservation-risks-with-verapdf-and-jhove.html
Description
Trigger eventIn this blog post, Johan van der Knijff lists a number of PDF features that represent a risk for the rendering or reuse of the content of Files. Van der Knijff also demonstrates how to detect them, using JHOVE and veraPDF (i.e. two of the most widely used PDF metadata extractors among the digital preservation community).
Problem statementEncryption or password-protection, multimedia content or external dependencies can prevent TDAs from making the content of Files accessible to its end users. File Format Identification and File Format Validation are not sufficient to identify these risks, because these features are used by only some instances of the file format and because encryption and multimedia objects are perfectly valid and standard PDF features.
Proposed solutionVan der Knijff provides the reader with a method to determine whether a File is using such features, and therefore has to be flagged at risk.

Multi-page TIFFs

Institutional background
InstitutionRosetta Users Group, N/A
Hyperlink https://docs.google.com/spreadsheets/d/1tSpS_1rCeVgOI0dEkmRu8cR9VjlLlYj_CnehswD1aM0/edit?gid=0#gid=0
Description
Trigger eventAlthough TIFF files may contain multiple images, this is not a common feature. As a result, some rendering tools cannot handle images beyond the first one properly, as they only take into account the “baseline” features described in the TIFF specification but ignore its extensions.
Problem statementThe preservation software Rosetta has a functionality for providing risk reports that help users to monitor risks and identify Files or Representations that are at risk in order to take batch action on all previously ingested Objects. Triggers for such risk reports include: Certificate, Encryption, End of life, Non-Standard, Obsolete, Outdated, Propriety.This spreadsheet was elaborated by the Rosetta Users Group as a risk inventory that provides a list of risks and methods for their detection The detection can be based on the file format identifier (PUID, in this case) or on a file property.
Proposed solutionThe line titled “TIFF with multiple pages” is a risk property, identified based on technology and community alerts. Multiple tools can detect this risk but JHOVE was preferred as most of the Rosetta implementations use it for both file format validation and metadata extraction of TIFFs. The interpretation of JHOVE’s output being complex, the extraction of the risk property relies on the analysis of several elements in JHOVE’s output. Those are described in column D as a set of conditions resolving to TRUE or FALSE.

4-channel JPEGs

Institutional background
InstitutionKU Leuven, BE
Description
Trigger eventMultiple end users reported that some ingested JPEGs did not display correctly, being rendered as a black rectangle. A number of reports had entered, but without a reference to the faulty JPEGs. At last a report was filed with a PID that contained 4 JPEGs, of which 1 did not render.
Problem statementBy comparing the Technical metadata of the correct and misrendered JPEG, it was noticed that the misrendered JPEG had four samples per pixel instead of three, as it happens with most RGB images. Since color space information was missing, it was first assumed that there was an alpha channel added for transparency.However, further investigation into the specification and internet articles revealed that an alpha channel was highly unlikely. In a next step, the TDA therefore downloaded the files and ran more tools against the JPEGs. Exiftool listed a Color Transform value of YCCK, which was confirmed by checking the file’s APP14 block data.Closer visual inspection of the web viewer revealed indeed small variations of different shadings of black in places where there was a lot of contrast in the original image. This pattern also seemed consistent with a YCbCr interpretation of the YCCK data. An investigation of all the JPEG files with 4-channel images and all of them turned out to be indeed in the YCCK colorspace.
Proposed solutionThe TDA at KU Leuven decided to convert all JPEG files to YCbCr. However, all employed tools still seem to result in a file with colors slightly different from the originals, leading to on-going discussions with the producer on the conversion process.

Publicly available documentation

InstitutionOrganisation typeLanguageHyperlink
TIB – Leibniz Information Centre for Science and Technology and University Library, DENational library
Non-commercial digital preservation service
Research infrastructure
Research performing organisation
English https://wiki.tib.eu/confluence/spaces/lza/pages/93608641/Preservation+Management#PreservationManagement-Riskmanagement
CSC – IT Center for Science Ltd., FINon-commercial digital preservation serviceEnglishhttps://urn.fi/urn:nbn:fi-fe2023062157386
(section 2.1)
Archivematica, CADigital preservation systemEnglish https://www.archivematica.org/en/docs/archivematica-1.17/user-manual/transfer/transfer/#transfer-tab-microservices
(Archivematica can provide some of the information needed to support risk assessment as part of its Transfer process.)