Testing with production-like data without breaking GDPR

Almost every team that tests a data-heavy product arrives at the same shortcut: restore last night’s production backup into staging. It is the fastest route to realistic data, and it is the point at which a testing problem quietly becomes a data protection one.

This is the engineering side of that problem — what changes, what the alternatives cost in practice, and what to agree with a supplier before anyone touches a database.

This is how we approach it as engineers. It is not legal advice, and it does not replace your own counsel or DPO.

The copy itself is processing

Under the GDPR, processing covers storage, adaptation, retrieval and use — not just the original collection. Copying a table of customers into a second environment is therefore an act of processing in its own right, and it inherits every obligation the original collection had.

Three of those obligations do most of the work here:

• Purpose limitation. The data was collected to deliver a service to those people. Software testing is a different purpose, and it needs to be either compatible with the original one or independently justified. "We already have the data" is not the same thing as "we may use it for this".
• Data minimisation. If you need ten thousand rows to reproduce a pagination defect, you do not have a basis for restoring four million. Volume that exists only because it was easier to copy everything is the hardest thing to defend.
• Security of processing. The obligation to protect the data follows the copy. This is where test environments do badly: broader access than production, weaker authentication, snapshots nobody tracks, and a habit of handing credentials to whoever needs to look at something.

That last point is the practical risk rather than the theoretical one. A breach in a staging environment is still a breach, with the same notification clock attached, and staging is usually where the controls are thinnest.

The production database is guarded because everyone agrees it matters. The copy of it in staging is guarded by whoever set up staging, on the afternoon they set it up.

Anonymised, pseudonymised, masked: three different words

These get used interchangeably in engineering conversations, and the distinction is the single most useful thing to get right.

• Anonymised data is data where the individual is no longer identifiable by anyone, by any means reasonably likely to be used. Genuinely anonymised data falls outside the GDPR altogether. That is a high bar, not a synonym for "we removed the name column".
• Pseudonymised data replaces identifiers with tokens while a key or a route back still exists somewhere. It is a recognised and encouraged safeguard, and it is still personal data with the full set of obligations attached.
• Masked data is an engineering technique, not a legal category. Whether masking achieves anonymisation or only pseudonymisation depends entirely on what it leaves behind.

The reason the bar for anonymisation is high is that identity leaks through combinations rather than through single fields. Three tests are worth applying to any dataset you intend to call anonymous: can a single individual still be picked out of it, can a record be linked to a record in another dataset, and can an attribute be inferred about someone with reasonable confidence. A postcode, a date of birth and a diagnosis will often single out one person with no name in sight.

What usable test data actually needs

The reason teams reach for a production restore is not laziness. Real data has properties that are genuinely hard to fabricate:

• Referential integrity. Orders point at customers, invoices point at orders. Scrambling one table independently produces a database that no longer holds together, and defects that exist only in your test environment.
• Distribution. Real data is lumpy. A handful of accounts have thousands of records and most have three, and performance problems live entirely in that tail.
• Awkwardness. Apostrophes in surnames, addresses with no postcode, a customer who changed their name twice, historic rows written by a schema you no longer run. These are where the defects are, and nobody invents them by hand.

Any approach that throws all three away in the name of safety will be abandoned by the first engineer who needs to reproduce something real, which is why the honest options preserve shape while removing identity.

Four approaches that work

1. 01Subset, then transform. Take a coherent slice rather than the whole database — a few thousand accounts with all their related rows — and transform the identifying fields within it. Smaller is both safer and faster to restore, and it is the option that most often survives contact with a data protection review.
2. 02Deterministic substitution. Replace each identifier with a stable pseudonym derived consistently, so the same input maps to the same output everywhere it appears. Joins keep working, the data stays coherent, and no real name survives. Note that a substitution you can reverse, or a mapping table you keep, leaves you with pseudonymised data rather than anonymised data.
3. 03Synthetic generation against real constraints. Generate data from the schema plus a description of the real distributions, rather than from the real rows. Slow to set up, excellent afterwards, and it is the only approach that is unambiguously outside the scope of the regulation.
4. 04Reproduce from the defect, not the database. For a specific bug, a curated fixture that recreates the failing state is usually more useful than a million rows, and it can be committed to the repository and reused as a regression test.

In practice most teams end up with the first two for exploratory and manual test passes, and the fourth for anything that becomes part of an automated suite — because a fixture that lives in version control is repeatable, and a database restore never is.

If an external team does the testing

Bringing in a supplier does not change the obligations, it distributes them. You remain the controller, deciding why and how the data is processed; the supplier is a processor, acting on your documented instructions. Four things are worth settling in writing before access is granted:

• A data processing agreement covering the categories of data, the purpose, the retention period and what happens at the end of the engagement.
• Whether personal data is in scope at all. The cleanest answer is that the supplier only ever sees transformed data, and that this is stated rather than assumed.
• Sub-processors. Cloud test-runners, device farms and screenshot services are sub-processors, and a screenshot of a defect can contain more personal data than the row that caused it.
• Where the data physically sits. A supplier inside the EU keeps this a domestic question; outside it, transfer mechanisms and their ongoing validity become part of the engagement rather than a footnote.

This tends to matter most in sectors where the data is the sensitive part — healthcare, finance, HR platforms — which are also the sectors where realistic test data is least optional.

What to write down

A test plan that touches personal data should be able to answer four questions on a single page: what data the test environment holds, how it got there and what was transformed on the way, who can reach it, and when it is destroyed. If those answers exist, most of the compliance argument is already made.

It is also worth checking who can actually reach the environment, rather than who is supposed to. Access control on non-production systems is consistently the weakest link, and it is one of the first things an OWASP-based assessment looks at — a staging box with production data and a shared password is a finding, not a workflow.