Identity field consistency is the property that makes a set of generated values usable at all. Every field in a record can be individually plausible while the record as a whole describes something impossible — a street in one country, a postal code from another, a telephone number belonging to a third — and that is precisely the state in which testing stops telling the truth.
This article explains how inconsistent records arise, why they let tests pass when they should fail, and what it takes to keep a test dataset coherent as it grows.
Why do individually correct fields make a wrong record?
Randomness is the usual culprit, and it is counter-intuitive how quickly it produces nonsense. Draw a country, then draw a city independently, and the pair will disagree with probability near one. The values are each drawn from the right pool; nothing is malformed; the combination simply never occurs.
Hand-built records fail differently but no less often. Whoever writes them typically works field by field, checking each value against their own knowledge, and has no reason to hold the country, the division, the postal code and the telephone prefix in mind at the same time. The result is a record that looks careful and is internally contradictory.
There is a third source, quieter than either: records assembled from several sources. A staging row may take its name from one seed, its address from a fixture, and its contact details from whatever the test author had open. Each part was fine where it came from.
Where inconsistency turns into a false pass
Consider a rule that says a customer must be at least a certain age to buy a product, and a test that checks the rule works. With a birth date in one century and a stored age from another, the test may exercise the accept branch while the record it is testing should have been rejected — and nothing reports a problem, because no check failed.
| Inconsistent pair | What it hides |
|---|---|
| Country and postal code shape | All postal-code validation, because the code is still well formed |
| Country and telephone calling prefix | Locale and routing logic, which silently falls back to a default |
| Birth date and age threshold | Age gates, so a passing test proves only that the accept path runs |
| Address and identifier format | Identifier validation, which is skipped when the country is ambiguous |
| Division and postcode range | Address normalisation, because no rule can be applied to a contradiction |
The pattern in every row is the same: the inconsistency removes the input that would have triggered the failure. A test suite full of such records is not weak because it lacks assertions; it is weak because its inputs never reach the branches where the assertions live.
Should inconsistent data be rejected or repaired?
It depends who is holding it, and getting this wrong causes its own damage. Incoming data from a person should be repaired where the intent is unambiguous and queried where it is not, because a human being is waiting. Generated and fixture data should be rejected outright, because there is no intent to recover and a repaired record hides the defect in whatever produced it.
The rule of thumb for a generator is that inconsistency is a bug in the generator rather than a property to be tolerated downstream. Anything else trains the team to write tolerant code that will later meet real input and swallow it.
For validation in the product, the useful question is what a failed consistency check should do to the record. A missing postal code is a data-entry gap, and the user can fix it. A postal code that exists but belongs to a different region is a contradiction, and no amount of retrying by the user will resolve it without the region changing too. Those two deserve different treatment.
How strong should a consistency rule be?
Make it as weak as it can be while still catching the failures that matter, and be explicit about which it is. Rules come in three strengths, and mixing them without labels is how a codebase ends up with validation nobody can explain.
- A warning records the disagreement and lets the record through, which is right for inferred rather than stated relationships.
- A rejection blocks the record, which is right when the disagreement is impossible rather than merely unusual.
- A repair rewrites one value to agree with another, which is the most dangerous of the three and should be used only where the precedence is documented.
Most of the confusion in this area comes from treating a statistical correlation as a hard rule. A telephone prefix usually implies a country, but not always; a postal code usually implies a region, but border areas and exceptions exist. Encoding a tendency as a prohibition rejects real people, and it does so most often for the users who least resemble the assumptions in the data.
Do generated records need to be consistent across the whole batch?
Within a record, yes. Across a batch, no — and confusing the two wastes effort in one direction and creates defects in the other. A batch of a thousand records should contain variety: different regions, different name shapes, different age brackets, some records with fields left absent. It should not contain a thousand records that each claim to be the same person.
There is a practical reason to keep batches internally diverse. A test that runs against a hundred near-identical records exercises one code path a hundred times and reports success. The same test against a hundred varied records exercises the branches that matter, which is what a load or migration rehearsal is actually trying to establish.
Where a batch does need homogeneity is reproducibility: the same input should produce the same batch, so that a failure found once can be found again. Generated records from the identity and test data generator keep that property by deriving everything from an identity key, which is what lets a batch be varied and repeatable at the same time. The values remain synthetic records for software testing, not real people’s details.
Which pairings break most often in practice?
Four pairings account for the majority of defects. The country with the address block is the first, because address shapes are the most obviously national part of a record. The country with the telephone prefix is the second, because calling codes are short and easy to leave unlinked. The birth date with the age gate is the third, because the gate is usually expressed as a number rather than as a comparison. The country with the identification number is the fourth, because a number that is merely digit-shaped passes a weak check anywhere.
Each of these has a cheap test: generate a record, alter one member of the pair by hand, and confirm the system notices. If it does not, the pairing was never actually validated, and the record that should have failed was passing all along.
For developers: where to enforce the rules
Generate in dependency order and validate in the same order. Country first, then the division that belongs to it, then everything derived from those two — address, postal code, telephone prefix, identifier format, currency and locale. A generator that draws fields in the order the form displays them will produce contradictions, because the form’s order is about presentation and the data’s order is about causation.
Put cross-field validation in one place rather than scattering it across the form, the API and the database. Duplicated rules drift, and the version that drifts is always the one nobody is reading. Model each pairing explicitly so that a reviewer can see which value constrains which, and give every rule a strength label so that a later reader knows whether it blocks or merely warns.
For fixtures, include a deliberately inconsistent record alongside the valid ones — the same person with exactly one pairing broken — and assert that the system rejects it. That single case is the most valuable fixture in the set, because it is the only one that proves the consistency rules are still switched on. Again, all of this is synthetic data for testing: it is not a description of any real person, and it must not be used to stand in for one.
Next steps
Write down the five field pairings your product relies on and check each one in the code; if a pairing is enforced nowhere, it is being assumed rather than verified. Then pull a batch of generated records from the identity generator and run them through a migration or import path, watching for rows that succeed for the wrong reason. The test fixtures article explains how to keep one broken record permanently in the suite so this property never regresses.