Building a rights and provenance register for dormant archives.

A register is the control layer for any dormant archive.

When a product line closes or an acquisition integrates, the data left behind rarely arrives with a clean pedigree. Repositories change hands. Schema documentation rots. The people who negotiated the original agreements move on. A rights and provenance register captures what is known, flags what is not, and creates a stable reference for every downstream decision.

The register is not a catalog. A catalog describes content. A register describes authority. It answers: where did this come from, who created it, under what terms, what has been done to it, and who holds it now. Each row represents a distinct data source. Each column captures a dimension of custody and permission.

This discipline sits at the center of our /collections/rights-and-provenance/. It feeds the inventory, informs the PII mapping, and governs the sample manifests that buyers and compliance teams review.

Define the grain before you open the spreadsheet.

A register fails when its grain is ambiguous. Decide up front what constitutes a row. Common grains include:

• A logical dataset (e.g., “customer transaction history 2018-2023”)
• A physical store (e.g., “PostgreSQL cluster prod-db-07”)
• A delivered artifact (e.g., “monthly Parquet export to S3 bucket X”)

Choose one grain and apply it consistently. If a single logical dataset spans three physical stores with different custody histories, it needs three rows. If three logical datasets share identical provenance, they may share a row with a scope note.

Document the grain decision in a preamble sheet. Future custodians will thank you.

Columns: the minimum viable register.

The following columns form the baseline. Add domain-specific columns as needed, but do not remove these.

Column	Purpose	Source of truth
source_id	Unique, stable identifier for the row. Use a UUID or a prefixed code (e.g., DS-2024-0042).	Assigned at registration.
source_name	Human-readable label. Include version or date range if relevant.	Documentation, schema, or custodian interview.
grain_note	One-sentence description of what this row covers.	Author judgment.
origin_system	The system of record where the data was first created.	Architecture diagrams, repo README, interview.
creator_team	Organizational unit or external party that produced the data.	HR records, org charts, vendor contracts.
creation_period	Start and end dates of data generation.	System logs, retention schedules.
governing_agreement	Contract, license, or policy that governs use and transfer.	Legal repository, contract management system.
agreement_clause	Specific section numbers that grant or restrict rights.	Legal review.
data_controller	Legal entity that determines purposes and means.	Privacy notices, corporate records.
data_processor	Entities that process on behalf of the controller.	DPAs, vendor lists.
pii_classification	None, low, medium, high - or a reference to a detailed PII map.	PII mapping worksheet.
retention_duty	Statutory or contractual retention requirement.	Legal register, regulatory guidance.
deletion_trigger	Event or date that initiates deletion.	Policy, contract, law.
transformation_log	Summary of every ETL, anonymization, aggregation, or schema change.	Pipeline code, runbooks, commit history.
current_custodian	Team or individual with operational control today.	Interview, access control lists.
custody_transfer_date	Date the current custodian assumed responsibility.	Ticket history, email thread, handover doc.
access_controls	Summary of who can read, write, delete.	IAM policies, ticket requests.
known_unknowns	Explicit list of gaps: missing contract, unknown creator, undocumented transform.	Author judgment.
research_owner	Person tasked with resolving each known unknown.	Assignment at register creation.
research_deadline	Target date for resolution.	Project plan.
register_version	Semantic version of the register row (e.g., 1.0, 1.1).	Incremented on each edit.
last_reviewed	Date of last custodian sign-off.	Review calendar.
reviewer	Name and role of signatory.	Review record.

Populate from three evidence streams.

No single source holds the full picture. Triangulate across contracts, repositories, and people.

Contracts and legal artifacts.

Start with the governing agreement column. Locate the master services agreement, data processing addendum, source code license, or internal policy. Extract the clause numbers that address ownership, permitted use, transfer rights, and deletion obligations. Paste the clause reference into the register. Attach the PDF or link to the contract management system in a supporting folder.

If the agreement is missing, mark the cell “MISSING” and add a row to the known_unknowns column. Assign a research owner from legal.

Repositories and technical artifacts.

Clone the repository. Read the README, the license file, the CONTRIBUTING guide, and the CI configuration. Check the git log for the earliest commits - they often reveal the originating team. Examine pipeline definitions (Airflow DAGs, dbt models, Spark jobs) to reconstruct the transformation log. Note every anonymization step, every column drop, every aggregation.

If the repository is archived or inaccessible, flag it in known_unknowns. Request restoration from infrastructure.

Interviews with custodians.

Schedule 30-minute sessions with the current custodian and, if possible, the original creator. Use a structured interview template:

1. Walk me through the data flow from creation to current store.
2. What agreements governed the initial collection?
3. What transformations have been applied since?
4. Who else has held custody?
5. What deletion or retention duties apply?
6. What do you not know?

Record the session (with consent). Transcribe key passages into the register. Capture the custodian’s name in the reviewer column and the interview date in last_reviewed.

Flag unknowns explicitly.

A blank cell is ambiguous. It could mean “not applicable,” “not yet researched,” or “researched and truly unknown.” Use a controlled vocabulary for gaps:

• MISSING - The artifact should exist but cannot be located.
• UNCLEAR - The artifact exists but does not answer the question.
• PENDING - Research assigned, deadline set.
• NOT_APPLICABLE - Confirmed irrelevant for this source.

Every MISSING or UNCLEAR entry must have a corresponding line in known_unknowns with a research_owner and research_deadline. This turns gaps into work items.

Version the register like code.

The register changes. A contract is found. A transformation is documented. A custodian transfers. Treat each row as a versioned record.

• Increment register_version on every edit.
• Keep a changelog sheet: date, source_id, column_changed, old_value, new_value, author, reason.
• Require a reviewer sign-off for any change to governing_agreement, data_controller, pii_classification, or retention_duty.
• Store the register in a version-controlled repository (Git) with branch protection on main. Approve changes via pull request.

This discipline mirrors the rigor applied to the /collections/data-inventory/ and /preparation/dataset-preparation/ workstreams. The register becomes the single source of truth for every downstream package.

Checklist: register readiness review.

Before you hand the register to licensing or compliance, confirm each item.

• Grain defined and documented in preamble.
• Every row has a stable source_id.
• All baseline columns populated or flagged with controlled vocabulary.
• Governing agreement clause references attached or linked.
• Transformation log traces every material change from origin to current state.
• PII classification cross-referenced to detailed PII map.
• Retention duty and deletion trigger verified against current law and contracts.
• Known unknowns list has owner and deadline for each entry.
• Changelog captures every edit since register creation.
• Current custodian has signed last_reviewed within the past 90 days.
• Register stored in version-controlled repository with protected main branch.
• Access limited to authorized reviewers and research owners.

Keep the register adjacent to the inventory.

The inventory lists what exists. The register explains what may be done with it. Link them by source_id. When the inventory adds a new source, the register gets a new row. When the register resolves a gap, the inventory confidence score improves.

This coupling is operationalized in our /preparation/process/ process. The readiness review at /preparation/recovery-planner/ validates that both artifacts are present, aligned, and current.

A sample register excerpt.

source_id	source_name	grain_note	origin_system	creator_team	governing_agreement	agreement_clause	pii_classification	transformation_log	current_custodian	known_unknowns	register_version
DS-2024-0042	Customer transaction history 2018-2023	Logical dataset covering all completed orders	payments-service v3	Payments Engineering	MSA-2017-ACME §4.2, DPA-2018-ACME §7	MSA §4.2 (ownership), DPA §7 (transfer)	High - contains name, email, IP	2020-03: PII hash applied to email; 2021-06: amount rounded to cents; 2022-11: partitioned by quarter	Data Platform Team	Missing DPA amendment 2021; research_owner: J. Rivera; deadline: 2026-11-15	1.3
DS-2024-0057	Product telemetry raw events	Physical store: Kafka topic telemetry.raw retained 13 months	Telemetry Pipeline	Platform Infrastructure	Internal Policy POL-2019-DATA §3	POL §3.1 (retention), §3.4 (access)	Low - pseudonymous IDs only	2019-01: schema v1; 2020-08: added session_id; 2023-02: migrated to Avro	Platform Infrastructure	Creator team disbanded; research_owner: K. Chen; deadline: 2026-11-30	1.1

The register is a living instrument.

A rights and provenance register is not a project deliverable. It is an operational control. It survives the acquisition, the shutdown, the migration. It travels with the data. When a buyer asks “can we use this for model training?” the register answers. When a regulator asks “what is the lawful basis?” the register answers. When a new custodian asks “who do I call?” the register answers.

Build it once. Maintain it forever. Version it rigorously.

---

Ready to assess your dormant archives? Request a readiness review at /accession/.

Where to go next.

The register builds on a data inventory. If you have not started one, use the inventory template, then run the readiness check. For help, request a readiness review.