Risk Properties Definition and Extraction (CPP-023)
| CPP-Identifier | CPP-023 |
| CPP-Label | Risk Properties Definition and Extraction |
| Author | Bertrand Caron |
| Contributors | Kris Dekeyser |
| Evaluators | Franziska Schwab, Maria Benauer, Felix Burger |
| Change history | Comments |
|---|---|
| Version 1.0.0 - 2025-08-29 | Milestone version |
| Version 2.0.0 - 2025-04-21 | Substantial revision of the description section and the process description to align interdependencies with CPP-012: Short definition and label modified - Revision of the global outputs and the structure of the Definition and scope section - Separation of the step-by-step description into Assessment and Extraction - Migration to XML |
1. Description of the CPP
The TDA monitors preservation risks and defines detection methods for determining whether the deposited digital objects are subject to these risks.
Inputs and outputs
| Input(s) | |||||||
|---|---|---|---|---|---|---|---|
| Alerts |
| ||||||
| Output(s) | |||||||
| Documentation/guidance |
| ||||||
Definition and scope
Risk Properties Definition and Extraction is a multi-tiered process that a) identifies risks caused by the specific properties and content of a file, b) defines methods to detect these risks based on file analysis (i.e. format identification, metadata extraction and format validation), and c) holds responsibility for performing technology watch. Together with CPP-012 Risk Mitigation, which defines which actions can be undertaken to reduce the likelihood or impact of the risk, CPP-023 Risk Properties Definition and Extraction is an essential process in the scope of the broader, on-going “risk management” activity.
Risk Properties Definition and Extraction is also the process responsible for technology watch. It receives updates from Community Watch (CPP-018), such as conference papers, specialised literature and journals, vendor announcements, etc. Based on that, it evaluates the impact of the risk with regard to the skills, objectives and means of the Designated Community. In particular, the two following activities are in scope of this CPP:
- Risk Identification involves cataloging potential threats to digital materials, including technological obsolescence (e.g. hardware or software becoming unavailable), media degradation (e.g. physical deterioration of storage devices), format obsolescence ( e.g. file formats being no longer supported), organisational risks (e.g. loss of institutional knowledge or funding), and environmental hazards (e.g. human failure and disasters such as power failures or security breaches).
- Risk Assessment evaluates the likelihood and potential impact of identified risks. This typically involves analysing object- and organisation-related factors like the criticality of the digital materials; the object’s primary and secondary value; the stability of their formats; and the resources available for preservation activities.
Problems in the rendering or reuse of Files by the TDA’s end users may stem from several reasons. For example, one common reason is the use of certain file format features, that rendering tools may misinterpret. To mitigate such misinterpretation, preservation actions (e.g. CPP-014 File Migration, CPP-027 File Repair, CPP-015 Emulation and Rendering Tools, etc.) should go beyond file format or validity status and consider risk properties. Risk properties are File or Representation properties that help identify whether a File is subject to a specific risk. Some examples of risk properties are:
- Discontinued support of file format;
- Erroneous file format structures;
- Encryption;
- External dependencies (e.g. non-embedded fonts);
- Embedded Files;
- Advanced multimedia features;
- Specific creating application, when said application is known to have created faulty Files;
- etc.
It is important to note that risk properties, in some cases, may be also considered significant properties, however, this is not necessarily the case For example, a valid PDF may be encrypted with an empty password. This is a risk property (it may endanger the following preservation processes) but it may not be considered a significant property (the TDA may decide to decrypt the PDF and keep it decrypted). Risk properties are typically defined by digital preservation analysts, while significant properties should be defined by the TDA management and stakeholders.
The step-by-step description below focuses more specifically on the risk extraction part of the process. Like Risk Mitigation, Risk Properties Definition and Extraction focuses on a specific subset of risks - in this case, risks related to inherent properties of Files. Although of major impact, the definition and detection of risks caused by organisational, financial or security issues are not described.
Process description
Trigger event(s)
| Trigger Event | CPP-identifier |
|---|---|
| Community alerts | CPP-018 (Community Watch) |
| Technology alerts |
Step-by-step description
| No | Supplier | Input | Steps | Output | Customer |
|---|---|---|---|---|---|
| sequence | |||||
| 1 | CPP-018 (Community Watch) | Information sources for digital preservation | Receive community and technology alerts | Risk | |
| CPP-010 (File Format Validation) CPP-027 (File Repair) | New findings gained through technical analysis | ||||
| CPP-023 (Risk Properties Definition and Extraction) | Risk properties policy and detection method | ||||
| 2 | Risk | Evaluate the risk likelihood and impact. | Assessment of risk likelihood and impact | ||
| 3 | Risk | Gather a test set of Files (compiled of both files that are subject to this risk and a control group). | Test set | ||
| 4 | Risk | Select a candidate tool for identifying the risk | Candidate extractor tool | ||
| 5 | Test set | Run the candidate extractor tool on the test set | Candidate extractor tool output | ||
| Candidate extractor tool | |||||
| 6 | Candidate extractor tool output | Determine which property in the candidate extractor tool output helps distinguish those Files that are affected by the risk and those that are not. | Successful identification of a risk property detection method (extractor tool) (step 7) | ||
| No method identified: resume steps 3-5 | |||||
| 7 | Chosen extractor tool output | Determine the interpretation of the extractor tool output | Risk property detection method: path / expression to extract the risk property from the tool’s output.(This interpretation should resolve to a true/false statement about the file being subject to the risk or not. It can be simple (an XPATH to the textual content of an element in an XML output) or more complex. See, about this, the use case about multipage TIFFs below.) | ||
| 8 | Risk | Update the risk inventory with the risk, risk property and risk property detection method. | Updated risk inventory | CPP-009 (Metadata Extraction) | |
| Risk property | |||||
| Risk property detection method: path / expression to extract the risk property from the tool’s output | |||||
Rationale(s) and worst case(s)
| Rationale | Impact of inaction or failure of the process |
|---|---|
| Risk assessment is critical for digital preservation activities because identifying risks related to the inherent characteristics of Files is required for a semantic / logical preservation approach. | The impact of the absence of risk assessment on Files’
characteristics may lead to different problems:
|
2. Dependencies and relationships with other CPPs
Dependencies
| CPP-ID | CPP-Title | Relationship description |
|---|---|---|
| CPP-008 | File Format Identification | Risks can be related to file format generic properties and concern all instances of it. In such a case, File Format Identification is sufficient to determine whether the file is affected by the risk. For example, Microsoft announced the discontinuation of its software product Publisher. As a consequence, all .pub files are now affected by the absence of a suitable software environment. |
| CPP-009 | Metadata Extraction | Risks can be related to properties common to one or several file formats. In such a case, Metadata Extraction is required to determine whether the file is affected by the risk. For example, running a metadata extractor like Apache Tika is required to check whether a PDF is encrypted or not. |
| CPP-018 | Community Watch | Risk is measured against the skills and tools available in the Designated Community. |
Other relations
| Relation | CPP-ID | CPP-Title | Relationship description |
|---|---|---|---|
| Affinity with | CPP-019 | Data Quality Assessment | Risk Properties Definition and Extraction and CPP-019 both define properties that the TDA should consider and assess against the result of CPP-009 (Metadata extraction). |
| Required by | CPP-009 | Metadata Extraction | Risk Properties Definition and Extraction provides the requirements for the selection of an appropriate extractor tool. |
| Required by | CPP-012 | Risk Mitigation | Risk Properties Properties Definition and Extraction maintains a risk inventory that guides the Risk mitigation process. |
| Required by | CPP-014 | File Migration | Risk Properites Definition and Extraction identifies risks related to file format that would trigger File Migration and provides the method for their detection. |
3. Links to frameworks
Certification
| Certification framework | Term used in framework to refer to the CPP | Section |
|---|---|---|
| CTS Link | CTS does not explicitly mention Risk Properties Definition and Extraction, although it is within the scope of R09 (Preservation planning). | |
| Nestor Seal Link | Nestor Seal does not explicitly mention Risk Properties Definition and Extraction, although C11 Preservation measures are based on a risk properties definition process as described in this CPP. | |
| ISO 16363 Link | “identifying each preservation risk” | ISO 16363 mentions Risk Properties Definition and Extraction in section 4.3.1: “The repository shall have documented preservation strategies relevant to its holdings.” |
4. Reference implementations
Use cases
Identification of PDF preservation risks
| Institutional background | |
|---|---|
| Institution | Koninklijke Bibliotheek, NL |
| Hyperlink | https://bitsgalore.org/2023/05/25/identification-of-pdf-preservation-risks-with-verapdf-and-jhove.html |
| Description | |
| Trigger event | In this blog post, Johan van der Knijff lists a number of PDF features that represent a risk for the rendering or reuse of the content of Files. Van der Knijff also demonstrates how to detect them, using JHOVE and veraPDF (i.e. two of the most widely used PDF metadata extractors among the digital preservation community). |
| Problem statement | Encryption or password-protection, multimedia content or external dependencies can prevent TDAs from making the content of Files accessible to its end users. File Format Identification and File Format Validation are not sufficient to identify these risks, because these features are used by only some instances of the file format and because encryption and multimedia objects are perfectly valid and standard PDF features. |
| Proposed solution | Van der Knijff provides the reader with a method to determine whether a File is using such features, and therefore has to be flagged at risk. |
Multi-page TIFFs
| Institutional background | |
|---|---|
| Institution | Rosetta Users Group, N/A |
| Hyperlink | https://docs.google.com/spreadsheets/d/1tSpS_1rCeVgOI0dEkmRu8cR9VjlLlYj_CnehswD1aM0/edit?gid=0#gid=0 |
| Description | |
| Trigger event | Although TIFF files may contain multiple images, this is not a common feature. As a result, some rendering tools cannot handle images beyond the first one properly, as they only take into account the “baseline” features described in the TIFF specification but ignore its extensions. |
| Problem statement | The preservation software Rosetta has a functionality for providing risk reports that help users to monitor risks and identify Files or Representations that are at risk in order to take batch action on all previously ingested Objects. Triggers for such risk reports include: Certificate, Encryption, End of life, Non-Standard, Obsolete, Outdated, Propriety.This spreadsheet was elaborated by the Rosetta Users Group as a risk inventory that provides a list of risks and methods for their detection The detection can be based on the file format identifier (PUID, in this case) or on a file property. |
| Proposed solution | The line titled “TIFF with multiple pages” is a risk property, identified based on technology and community alerts. Multiple tools can detect this risk but JHOVE was preferred as most of the Rosetta implementations use it for both file format validation and metadata extraction of TIFFs. The interpretation of JHOVE’s output being complex, the extraction of the risk property relies on the analysis of several elements in JHOVE’s output. Those are described in column D as a set of conditions resolving to TRUE or FALSE. |
4-channel JPEGs
| Institutional background | |
|---|---|
| Institution | KU Leuven, BE |
| Description | |
| Trigger event | Multiple end users reported that some ingested JPEGs did not display correctly, being rendered as a black rectangle. A number of reports had entered, but without a reference to the faulty JPEGs. At last a report was filed with a PID that contained 4 JPEGs, of which 1 did not render. |
| Problem statement | By comparing the Technical metadata of the correct and misrendered JPEG, it was noticed that the misrendered JPEG had four samples per pixel instead of three, as it happens with most RGB images. Since color space information was missing, it was first assumed that there was an alpha channel added for transparency.However, further investigation into the specification and internet articles revealed that an alpha channel was highly unlikely. In a next step, the TDA therefore downloaded the files and ran more tools against the JPEGs. Exiftool listed a Color Transform value of YCCK, which was confirmed by checking the file’s APP14 block data.Closer visual inspection of the web viewer revealed indeed small variations of different shadings of black in places where there was a lot of contrast in the original image. This pattern also seemed consistent with a YCbCr interpretation of the YCCK data. An investigation of all the JPEG files with 4-channel images and all of them turned out to be indeed in the YCCK colorspace. |
| Proposed solution | The TDA at KU Leuven decided to convert all JPEG files to YCbCr. However, all employed tools still seem to result in a file with colors slightly different from the originals, leading to on-going discussions with the producer on the conversion process. |
Publicly available documentation
| Institution | Organisation type | Language | Hyperlink |
|---|---|---|---|
| TIB – Leibniz Information Centre for Science and Technology and University Library, DE | National library Non-commercial digital preservation service Research infrastructure Research performing organisation | English | https://wiki.tib.eu/confluence/spaces/lza/pages/93608641/Preservation+Management#PreservationManagement-Riskmanagement |
| CSC – IT Center for Science Ltd., FI | Non-commercial digital preservation service | English | https://urn.fi/urn:nbn:fi-fe2023062157386 (section 2.1) |
| Archivematica, CA | Digital preservation system | English |
https://www.archivematica.org/en/docs/archivematica-1.17/user-manual/transfer/transfer/#transfer-tab-microservices (Archivematica can provide some of the information needed to support risk assessment as part of its Transfer process.) |