Community Watch (CPP-018)

CPP-IdentifierCPP-018
CPP-LabelCommunity Watch
AuthorMicky Lindlar
ContributorsMattias Levlin
EvaluatorsFelix Burger, Maria Benauer
Change historyComments
Version 1.0.0 - 2025-08-29Milestone version
Version 1.1.0 - 2026-03-25Migration to XML
Version 1.1.1 - 2026-07-24Replaced br by span elements.

1. Description of the CPP

The TDA monitors its Designated Community in order to identify its evolving needs and knowledge.

Inputs and outputs

Input(s)
Documentation/guidance
Preservation Policy
Mission Statement
File Format Policy - Preferred formats
Emulation and rendering policy
List of emulation and rendering tools
File Format Policy - Identification
File Format Policy - Validation
List of identification and validation tools
Access Policy
List of tools used for derivatives generation or normalisation
Metadata recording policy
List of metadata schema or serialisation used
List of tools used for (automated) Metadata generation and handling
Designated Community definition
Description of Designated Community’s Knowledge Base
Preservation Objective
Output(s)
Documentation/guidance
Report on changed Designated Community structure
Report on updated file formats or data types used by Designated Community
Report on changed or new significant properties
Report on updated access, use and re-use requirements
Report on updated Metadata requirements
Report on updated legal or policy impact
Report on updated methods, processes or concepts used by Designated Community
Report on updated requirements for Preservation objective
Report on risk
Report on software changes
Report on updates to file format identification and validation
Report on changes to migration tools
Report on changes to Metadata generation and handling tools

Definition and scope

The TDA must monitor its Designated Community in order to identify its evolving needs and knowledge.

Information packages within a TDA are stored for a group of current and future users (i.e. the Designated Community). A Designated Community is always specific to a TDA and it may change over time (e.g. as data from a different sub-discipline are being deposited or the method of intended access or interaction to data changes over time). In addition to the Designated Community itself, the TDA should define what can be assumed as knowledge within the Designated Community (e.g. what language they speak, what scientific methods they understand and what file formats or metadata standards they can interact with). This is referred to as the Designated Community’s Knowledge Base. Lastly, the TDA must plan for the Preservation Objective, meaning how the Designated Community can use the data. A Preservation Objective should be specific, actionable and measurable. While the Designated Community is the “who” and “with what knowledge or skills”, the Preservation Objective is the “how”.

For example, see these two fictive TDAs:

  • TDA1 preserves civil engineering data for research and practice within Germany. The data preserved is currently 3D point-cloud scans in file formats LAS/LAZ and e57. Based on the content they currently preserve, they define their Designated Community as researchers from the discipline of civil engineering including all subdisciplines as well as specialists working in these fields (e.g. as urban engineers within Germany). The Designated Community’s knowledge base is built on the German language; includes principles such as mechanics and materials science; methods such as simulation and surveying as well as tools such as Computer-Aided Design; and 3D point-cloud scanning. One of the Preservation Objectives is for the Designated Community to be able to render separate point-cloud scans of the same location on their own systems and to perform tasks including deformation monitoring on the data.
  • TDA2 preserves oral history data in a university located in Spain. The data preserved is from a long-term collaborative research project between the institutes for political science and for recent Spanish history and contains interviews with contemporary witnesses on the Spanish Transformation to Democracy (La Transición Española). The data can currently only be accessed by staff from the two affiliated institutes. In 2040, access to the data will be opened to the wider research community. The TDA defines its Designated Community as the researchers from the two affiliated institutes as well as researchers for those disciplines throughout Spain. The Designated Community ’s knowledge base is built on the Spanish, Catalan, Galician and Basque languages; includes the historical context of the data; political science concepts; and oral history and testimonial methods. One of the Preservation Objectives is for the Designated Community to be able to load the original data into their qualitative data software for coding and thematic analysis.

The CPPs assume that the TDA’s Designated Community including its knowledge base and the Preservation Objective have been defined and are available in guiding documentation such as a mission statement, a preservation policy or preservation action plan(s). Such a Designated Community definition typically includes current depositors or data producers and current consumers of the data, but also needs to take into account potential future user groups.

Often, the Designated Communites themselves look to the TDA for advice on data handling practices, especially connected to file formats, data packaging or Metadata. To adequately support current day and future users, it is therefore crucial to also monitor usage trends, good practice and risks associated with file formats, data types and Metadata which the TDA already supports or intends to support in the future. This information is typically found outside of the research communities themselves. Communities to turn to for this include digital preservation, digital curation and research data management communities as well as existing communities around file formats, Metadata and software.

OAIS has the functions “Monitor Designated Community” and “Monitor Technology” feed into the “Preservation Watch” function. All three are part of the preservation planning functional entity. In addition, OAIS mentions the environment itself (e.g. financial, political, environmental) as a potential source to monitor and feed into Preservation Watch. In practice, the monitoring functions are often described as “Technology Watch”, “Organisation Watch”, “Community Watch” and “Preservation Watch”.

Within the scope of the CPP, Community Watch covers monitoring of:

  • The Designated Community, its Knowledge Base and Usage Scenarios;
  • Digital Preservation, Digital Curation, Research Data Management communities;
  • File Format Communities and (emerging) good practice, risks and developments;
  • Metadata Communities and (emerging) good practice, risks and developments;
  • Software Communities and (emerging) good practice risks and developments.

The CPP Community Watch therefore covers more than monitoring the Designated Community. It integrates information sources closer to the Technology Watch or Monitor Technology (OAIS) function. Since it is important to stress the relevance of the unique Designated Communites whose Preservation Objective needs to be supported by matching tools and processes within the TDA, the name Community Watch was chosen here.

Process description

Trigger event(s)

Trigger EventCPP-identifier
Continuously running activitiy

Step-by-step description

NoSupplierInputStepsOutputCustomer
 sequence
1 sequence - Designated Community and Knowledge Base Description
1.1Mission StatementChoose, design and implement methods to monitor changes in the Designated Community (e.g. user survey, feedback form, web/usage analytics, support channel, scientific advisory boards etc.)
Preservation Policy
1.2 parallel - Evaluate feedback from evaluation method
1.2.IOptional: If an indicator is given for a changed Designated Community (i.e. new discipline uses data or previously defined discipline no longer uses data):review of guidance documents (designated community definition) by an administrative entity Report on changed Designated Community structure
Update to Retention PolicyCPP-017 (Disposal)
1.2.IIOptional: If an indicator is given for a changed Knowledge Base impacting file formats or data types usedReport on updated file formats or data types used by Designated Community
Update to File format policyCPP-008 (File Format Identification)
Update to Emulation and rendering policyCPP-015 (Emulation and Rendering Tools)
Update to Significant Properties PolicyCPP-022 (Significant Properties Definition)
Update to preferred file formats listCPP-026 (File Normalisation)
Update File format policy - Derivatives: formatsCPP-028 (Creation of Derivatives)
1.2.IIIOptional: If an indicator is given for a changed Knowledge Base impacting significant properties used or extracted from File formats or data Objects Report on changed or new significant properties
Update to Significant Properties PolicyCPP-022 (Significant Properties Definition)
Update to queries leveraging changed or new significant propertiesCPP-024 (Enabling Discovery)
1.2.IVOptional: If an indicator is given for a changed Knowledge Base impacting software used in accessing or interacting with data or re-use scenariosReport on updated access, use and re-use requirements
Update to Emulation and rendering PolicyCPP-015 (Emulation and Rendering Tools)
Update to queriesCPP-024 (Enabling Discovery)
1.2.VOptional: If indicator is given for changed Metadata width, depth, syntax or serialisationReport on updated Metadata requirements
Update to metadata policyCPP-016 (Metadata Ingest and Management)
Update to queriesCPP-024 (Enabling Discovery)
1.2.VIOptional: If indicator is given for legal or policy impacting restricted or extended access to Objects Report on updated legal or policy impact
Update on Rights StatementsCPP-020 (Rights Management)
Update on Access PolicyCPP-025 (Enabling Access)
1.2.VIIOptional: If indicator is given for changed methods, processes or concepts:review of guidance documents (Knowledge Base description) by an administrative entityReport on updated methods, processes or concepts by Designated Community
1.3Mission StatementPreservation PolicyCheck all steps above for impact on Preservation Objective. If an indicator is given for a changed Preservation Objective:review of guidance documents (Preservation objective, Designated Community Knowledge Base) by an administrative entity Report on updated requirements for Preservation objective
Update on Quality propertiesCPP-019 (Data Quality Assessment)
2 sequence - Supported file formats and data types description
2.1File format policy - preferred file formatsChoose, design and implement methods to monitor changes in file formats and data types, including networks around supporting software (e.g. observing file format networks, observing digital preservation and digital curation networks, conference proceedings, journal articles, querying knowledge bases etc.)
2.2 parallel - Evaluate information
2.2.IOptional: If risk in file format or data type is raisedReport on risk
Update on riskCPP-023 (Risk Properties Definition and Extraction)
2.2.IIOptional: If file format is no longer supported or rationale behind initial choice for format is no longer true (e.g. no longer widely used):review of guidance documents (Preservation Intent definition) by an administrative entityReport on risk
Update on riskCPP-023 (Risk Properties Definition and Extraction)
2.2.IIIEmulation and rendering PolicyOptional: If tool to support file formats rendering or emulation is no longer supported or available or a new version has become availableReport on software changes
List of emulation and rendering toolsUpdate on riskCPP-023 (Risk Properties Definition and Extraction)
Update to Emulation and rendering PolicyCPP-015 (Emulation and Rendering Tools)
2.2.IVFile format Identification and Validation PoliciesOptional: If new tool or method to support file format identification or validation becomes availableORIf file format identification or validation tools used is no longer supported or availableORIf new versions of the tools have become availableReport on updates for file format identification
List of identification and validation toolsUpdate to File Format PolicyCPP-008 (File Format Identification)
Updates to File Format Validation PolicyCPP-010 (File Format Validation)
2.2.VAccess PolicyOptional: If there is a migration path defined in the Access policy (e.g. to create derivatives or to normalise Files in pre-ingest) and the tool used for this is no longer supported or available, or a new version has become availableReport on changes to migration tools
File Format PolicyUpdate to support for DIP creationCPP-025 (Enabling Access)
List of tools used to derivatives generation or normalisationUpdate to tools used for normalisationCPP-026 (File Normalisation)
Update to File format policy - Derivatives: toolsCPP-028 (Creation of Derivatives)
3 sequence - Supported Metadata
3.1Metadata policyChoose, design and implement methods to monitor changes in syntax, schema and serialisation used for Metadata in the TDA
List of used metadata schema / serialisation in the TDA
3.2 parallel - Evaluate information
3.2.IOptional: If risk in Metadata syntax, schema or serialisation is raised:review of guidance documents (Preservation Intent definition) by an administrative entityReport on risk
Update on riskCPP-023 (Risk Properties Definition and Extraction)
Update to queriesCPP-024 (Enabling Discovery)
3.2.IIOptional: If Metadata syntax, schema or serialisation is no longer supported or rationale behind initial choice for syntax, schema or serialisation is no longer true (e.g. no longer widely used):review of guidance documents (Preservation Intent definition) by an administrative entityReport on risk
Update on riskCPP-023 (Risk Properties Definition and Extraction)
Update to queriesCPP-024 (Enabling Discovery)
3.2.IIIList of tools used within TDA for (automated) Metadata generation and handling (e.g. validation, conversion)Optional: If new method to support Metadata generation and handling becomes availableORIf tools used are no longer supported/availableORIf new versions of the tools have become availableReport on changes to Metadata generation and handling tools tools
Update to methods or tools for metadata extractionCPP-009 (Metadata Extraction)
Update to methods or tools for metadata ingest and managementCPP-016 (Metadata Ingest and Management)
Update to queriesCPP-024 (Enabling Discovery)

Rationale(s) and worst case(s)

RationaleImpact of inaction or failure of the process
The TDA takes responsibility for the long-term stewardship of data for the Designated Community. TRUST : R-Reponsibility The TDA must monitor and identify expectations and requirements of its Designated Community and act to meet these changing requirements. TRUST : U-User Focus The TDA must use adequate tools to support the needs of the Designated Community and update these accordingly. TRUST : T-Technology Data will not meet the needs of the Designated Community.
The TDA should conduct its own watch of the digital preservation, digital curation and research data management communities TRUST : R - Responsibility The TDA should conduct its own watch of those file format, software and Metadata communities relevant to the TDA’s holdings TRUST : T-Technology Processes adopted might not meet the needs of the Designated Community specific to the TDA.

2. Dependencies and relationships with other CPPs

Dependencies

CPP-IDCPP-TitleRelationship description
///

Other relations

RelationCPP-IDCPP-TitleRelationship description
Affected byCPP-009Metadata ExtractionEither due to changing significant properties or due to updated tools, metadata extraction requirements can be affected.
Required byCPP-012Risk MitigationChanging community needs affect the risks and the mitigation of those.
Required byCPP-019Data Quality AssessmentThe signals from the community may affect the Data Quality Assessment. The Data Quality Assessment performed during Ingest may for example result in extraction of quality properties that are required by the Designated Community.
Required byCPP-022Significant Properties DefinitionSignificant Properties Definition is fundamental to ensure that information and features supported by the Objects are those that the TDA’s Designated Community is expecting.
Required byCPP-023Risk Properties Definition and ExtractionRisk is measured against the skills and tools available in the Designated Community.
Required byCPP-024Enabling DiscoveryThe TDA must have identified the needs of its Designated Community in order to enable queries that support the community's defined query scenarios.
May be required byCPP-017DisposalCommunity watch treats Disposal as a customer in cases where, for instance, a collection previously archived within the TDA has been also added to a different TDA and will be preserved there. Required actions downstream must be taken into account within the disposal process (e.g. notifying stakeholders about the changed preservation location).

4. Reference implementations

Publicly available documentation

InstitutionOrganisation typeLanguageHyperlink
TIB – Leibniz Information Centre for Science and Technology and University Library, DENational library
Non-commercial digital preservation service
Research infrastructure
Research performing organisation
Englishhttps://wiki.tib.eu/confluence/spaces/lza/pages/93608286/Digital+preservation+at+TIB#DigitalpreservationatTIB-Cooperationandnetworks
CSC – IT Center for Science Ltd., FINon-commercial digital preservation serviceEnglishhttps://urn.fi/urn:nbn:fi-fe2023062157386
(Sections TBA)
Englishhttps://urn.fi/urn:nbn:fi-fe2025040925236
(Sections V10, V14, V69, V80, 6.2.2, 7.1)
Archivematica, CADigital preservation systemEnglishhttps://www.archivematica.org/en/community/resources/