A generated address that fails at checkout is one of the most irritating classes of test failure, because every individual field looks fine. The city is real. The postal code has the right number of characters. The phone number has a plausible area code. Nothing is wrong except that the three belong to different places.
The error shows up late and far from the fixture. A shipping service rejects the destination, an address validator says the town and the code do not belong together, a tax calculation quietly falls back to a default rate, or a support agent reads back an address that sounds right and is wrong.
How a mismatch is created in the first place
Naive generators build an address one field at a time. The country is drawn from a list. The city is drawn from a global pool of city names, sometimes filtered by country and sometimes not. The postal code is produced from a per-country pattern rule. The phone number is assembled from a dialling code plus random digits.
Each step is individually defensible; the product is not. Postal codes are not random strings with a shape, they are allocated geographically. The leading characters of a code identify where it belongs, which means a code can be structurally valid and still impossible for the city printed beside it. Treating the format as the only constraint throws away exactly the information that would have made the tuple check out.
Phone numbers fail the same way. National numbering plans allocate number blocks by region, and most countries expect the national trunk prefix to be dropped before a country code is added. A form that concatenates a dialling code with a number that still carries its leading zero produces something syntactically plausible and never dialable.
How do you spot a mismatch in data you already have?
Tuple invariants are testable, and they catch this cheaply.
- Group the rows by postal code and count the distinct cities. In real data a code covers one place or a small delivery area; in naive generated data it is a random string that collisions scatter across many towns.
- Check for impossible blanks. Some countries have no postal code system in everyday use, so a table that holds a code for every country on earth contains fabrications.
- Read twenty rows by hand. A person spots that a southern city paired with a north-eastern code is nonsense in a way that a length check never will.
- Look at the phone column. If every stored value has the same length regardless of country, the generator is filling digits rather than following a numbering plan.
Not every mismatch is geographic, though. Some are formatting disagreements that only look like data problems, and the shapes are catalogued in postal code formats by country. One side stores a Canadian code without its internal space while the other expects the spaced form; one side uppercases everything and the other preserves case. The tuple is correct and the comparison still fails. Normalise both sides before concluding that two values disagree.
The mismatch is a family, not a single bug
Disagreement between a city and a code is not one failure but a family, and each member needs its own expectation written down.
| Fixture | Expected outcome |
|---|---|
| Code from another region | Rejected by the relationship rule |
| Code that does not exist | Rejected at lookup against reference data |
| City name carrying diacritics | Accepted after Unicode normalisation |
| Country with no postal code system | Accepted with the field left empty |
| Foreign code pasted into the wrong country | A policy decision that must not crash |
| One code covering several towns | Accepted, because a real code can do that |
That last row matters more than it looks. A postal code does not map to exactly one city everywhere; some cover a group of delivery areas across several municipalities. The invariant is not one code to one city, it is one code to a small and stable set of cities, so a test asserting strict uniqueness will fail against real reference data and send someone hunting for a bug that is not there.
Name the expected outcome in every fixture and keep the negative members in the same suite as the positive ones. A suite made only of valid tuples hides the interesting case.
Which form defects does a fake address expose fastest?
Generated records are not only for checking that a form accepts them. Because a generator produces values at the edges of a country’s format, it finds these defects quickly.
- Truncation at a fixed length. A column or input sized for the shortest realistic value silently cuts a longer one, and a truncated postal code is stored without complaint. A long division name in a two-line address does the same thing.
- Requiredness that does not follow the country. A city, region or postal code marked required everywhere contradicts the countries that have no such field. The form is not wrong for the country it was written in; it is wrong for the second country it was extended to.
- Silent normalisation that changes the stored value. Uppercasing a case-sensitive code, trimming internal whitespace, or routing the input through a numeric type rewrites what the user typed. The user sees a green form and the record holds something else.
Why do the front end and the back end disagree?
Address validation usually exists twice: a pattern in the browser for fast feedback, and a rule on the server because the browser cannot be trusted. The two drift, and the drift produces the worst error state of all, a form that says everything is fine and a request that comes back rejected.
Three shapes of drift are common. The same value is validated at different stages, so the browser checks the string as typed while the server checks a value consumer code has already normalised. The two rule sets differ, with the browser knowing a pattern per country and the server also applying a length and a relationship rule. And the two sides disagree about types, with an empty string crossing the wire where the server expects a null.
The way to keep them aligned is a shared corpus and one source of truth. Take the generated records you already produce, feed the same list to the browser validator in a unit test and to the server validator through its endpoint, and assert that both return the same verdict for every record. Anything that disagrees is drift you found before a user did. Address data in test fixtures covers how to keep that corpus maintainable.
Generating the tuple instead of the fields
The unit of generation should be an address record that already contains a consistent country, top-level division, city and postal code. When the division, city and code come from one data record, they cannot disagree with each other: the generator is choosing a place, not three strings that happen to sit next to one another.
Run the validator you ship in production over the generated rows as a batch, which is what the validation tool is for. If the validator is what rejects a row, you have found the mismatch before a test did, and you have exercised the validator against a large and varied corpus, which is hard to do with hand-written samples.
Treat the phone number as part of the address too. Store international format, display national format, and derive the national format from the same country record that supplied the rest of the address. A number and a country chosen independently will eventually contradict each other.
Keep negative cases deliberately. A suite needs rows that should fail: a code attached to a city in the wrong country, a phone number with its trunk prefix left in, a country without postal codes carrying one anyway. Generate a clean set for the happy path and hand-write a small dirty set for the failure path.
The distribution check that samples miss
Sample-based checks find gross errors; a distribution check finds subtle ones. In real address data the number of distinct postal codes per country follows that country’s own allocation, and a generated set with a few hundred codes spread evenly across many countries is filling formats rather than geography.
The same holds for cities. Real address data concentrates: a handful of cities account for most rows. Uniform selection across every city in a country produces data that looks wrong in every aggregate chart and, worse, never exercises the code paths that assume any city can appear at any time.
Writing a bug report someone else can reproduce
A defect report about address data is only useful if the next person can rebuild the exact record: the input, the country, the environment and the expected result. A screenshot rarely carries the input, because a form shows the value after the browser has formatted it.
Use a pinned sample rather than a real one. When the generator returns the same record for the same key, the report can carry the key instead of a copy of the record, and anyone can paste that key into the address generator to get the identical person, street, city and postal code. Include the stored value as well as the entered one, because the difference between them is usually the defect and it is invisible in a screenshot of the form.
Never attach a real person’s address, phone number or identity number to make a report look realistic. The pinned key gives you a realistic record that anyone can regenerate, and the country directory shows what each country’s records are built from. Every sample discussed on this page is synthetic test material and stands in for nobody.