The single bit that answers nothing

Every data catalog and warehouse toolchain lets you tag a column as PII. One flag: pii: true or pii: false. This feels like governance, but it discards information.

When our DPO looks at a data product, the question is never "does it contain PII, yes or no". The questions are:

• Does this column identify a person on its own, or only in combination with other columns?

• Is the data raw, pseudonymised, or anonymised? And is the pseudonym mapping still around?

• Did a human assert this classification, or did a tool infer it?

A boolean collapses all of these into one bit and turns the rest of the audit trail into guesswork. GDPR itself distinguishes categories of personal data and processing states, and a German university's Verarbeitungsverzeichnis (records of processing activities) needs those distinctions, not a bit.

So we classify on two orthogonal axes, plus a provenance tag.

Axis one: sensitivity

The first axis describes what kind of data the column contains:

• A direct identifier uniquely identifies one person on its own: email address, student number, user ID, full name. A pseudonym which maps 1:1 to a person via a lookup table is also a direct identifier.

• An indirect identifier identifies a person in combination with other fields, also called a quasi-identifier: zip code, birthdate, gender, a small-cohort course ID. This category is where the boolean flag fails most badly, because each field looks harmless on its own. Latanya Sweeney showed that zip code, birthdate, and gender together uniquely identify roughly 87% of US residents [1] .

• For sensitive content the structure of the column is not identifying, but its content might be: forum post text, file uploads, free-form survey answers. You cannot classify these by schema, they need content inspection.

• None marks data which is safely not personal: course titles, semester codes, aggregated counts.

Axis two: state

The second axis describes what processing has been applied:

• Raw data is stored as collected from the source system.

• Pseudonymised data has a reversible mapping which is held separately (GDPR Art. 4(5)). Legally this is still personal data. The separated mapping table is what keeps it pseudonymous instead of identifying.

• Anonymised data has no reversible mapping any more (GDPR Recital 26). This requires genuinely irreversible destruction of identifiers, or aggregation below a re-identification threshold.

• Dropped marks a column which was removed at this stage. An explicit marker that an upstream PII column is intentionally absent downstream is much more useful than a silent absence.

The axes combine: An email address is {direct_identifier, raw}, a pseudonym ID is {direct_identifier, pseudonymised}, and a completion rate aggregated over fifty students is {none, anonymised}. None of these pairs survives the round trip through a boolean.

Provenance: who says so?

The third piece is not about the data at all, but about the classification itself. Every computed classification carries a provenance tag:

• Declared means a human explicitly authored the classification on this column.

• Inherited means the classification was computed by walking the column lineage and applying most-sensitive-input-wins.

• Unknown means there is no declaration and no resolvable inputs.

I have not seen this part in off-the-shelf tooling, and it is the part the audit actually hinges on. A reviewer looking at a data product needs to distinguish asserted facts from inferred ones. We weight a contract's risk summary by provenance: If all fields are declared, the contract counts as audited. If any field is inherited, it is auto-classified and a review is recommended. If any field is unknown, the contract cannot ship. And any raw direct identifier exposed at a contract surface is flagged regardless of provenance, because no amount of paperwork makes that one fine.

Propagation through lineage

Classifying every column by hand does not scale and would rot immediately. Classifications therefore propagate through column-level lineage, which we extract from our dbt models. One design rule keeps the propagation trustworthy:

Inheritance is conservative and always produces an upper bound on risk. Declarations exist only to relax that bound.

• The most sensitive input wins. A column derived from one PII input and one harmless input inherits the PII classification.

• When the SQL parser cannot resolve column lineage, which happens on any real pipeline, we fall back to the maximum sensitivity across all columns of all upstream tables. This over-classifies, but it over-classifies safely.

• An explicit human declaration trumps inheritance. This is how a pseudonymisation or anonymisation step which the lineage tooling cannot see gets authored: The human who knows the mapping was destroyed says so, on the record.

• If nothing works, the column is unknown, and an unknown column blocks its data product from being released.

This asymmetry carries the whole design: Automation may only ever make the assessment stricter. Only an accountable human may relax it, and the relaxation is recorded as declared provenance. A failure of the lineage parser produces over-classification and a review burden, never silent under-classification.

Two pragmatic escape hatches keep the authoring overhead sane. Source columns must always be classified explicitly, and a missing classification is a validation error, because an absent flag is ambiguous between "safe" and "forgot". Downstream, a handful of well-known column names produced by our consent subsystem (the pseudonym and anonymous IDs) carry conventional classifications, so they do not need re-declaring in every model they flow through. A column using one of those names without coming from the consent subsystem is a bug to be renamed, not a classification problem.

One enforcement detail sounds petty, but the whole design depends on it: SELECT * is banned in transformation models. Column-level lineage is only trustworthy if every column reference is explicit. The privacy architecture rests, in part, on a lint rule.

Summary

Coming from commerce architecture, I did not know what to expect from privacy-first engineering. The actual work turned out to be metadata engineering: making classifications first-class data with types, lineage, provenance, and CI gates, with the same discipline we apply to the data itself. The trust of our DPO in the platform does not come from a policy document. It comes from being able to trace, for any column in any data product, what it is, what was done to it, and who said so. A boolean flag answers none of those questions; two axes and a provenance tag answer all three.

I'll write more about how this classification plays together with consent withdrawal in one of the next posts.

References

1

Sweeney, L. (2000). "Simple Demographics Often Identify People Uniquely." Carnegie Mellon University, Data Privacy Working Paper 3. The basis of the widely cited result that {ZIP, birthdate, gender} uniquely identifies ~87% of the US population.