Farpost · Data Ingestion
$ Two Weak Signals Beat One Strong-Looking One
Matching a government address record against Farpost's own building records sounds like a string-matching problem — normalize both addresses and compare. Canada's civic-address data is fragmented enough that string matching alone isn't reliable, so this ingestion path never trusts it alone.
A string match is corroborated by an independent signal: is there a building within 50 meters of the government record's actual coordinates? If both agree, it's a confident match. If the string match says one building but the coordinates say another one two kilometers away, that's not resolved by trusting either signal — it's logged as ambiguous for a human to look at.
async def ingest_nar_record(record, building_repo, ambiguity_repo, write_buffer=None) -> str:
"""Returns "matched", "created", or "ambiguous" for caller-side stats."""
loc_guid = record["loc_guid"]
location = record.get("location")
candidates = await building_repo.find_by_normalized_address(
record["postal_code"], record["civic_no"], record["street_name"],
)
if len(candidates) > 1:
await ambiguity_repo.record(
loc_guid, "multiple_string_matches", [c.slug for c in candidates], record,
)
return "ambiguous"
if len(candidates) == 1:
candidate = candidates[0]
if location is None or candidate.location is None:
# No location data on one side to corroborate with — still unambiguous.
await _backfill_match(candidate, loc_guid, location, write_buffer)
return "matched"
proximate = await building_repo.find_by_proximity(location, PROXIMITY_RADIUS_METERS)
if any(str(c.id) == str(candidate.id) for c in proximate):
await _backfill_match(candidate, loc_guid, location, write_buffer)
return "matched"
# String match says one thing, geography says another — don't guess.
await ambiguity_repo.record(
loc_guid, "string_match_contradicted_by_proximity", [candidate.slug], record,
)
return "ambiguous"
await _create_building(record, write_buffer)
return "created"The fix
A single string-match candidate isn't auto-accepted until an independent 50-metre geospatial proximity check corroborates it. No location data on either side falls back to accepting the string match, since there's nothing to contradict it with; a contradiction between the two signals — matched by address, but not by geography — is recorded to an ambiguity repository rather than guessed at, under a different reason code than a genuine multiple-candidate string match.
Why this matters
It's more work to build two independent signals and reconcile disagreements than to trust one, but it's the more defensible choice against genuinely messy, fragmented real-world data. Requiring corroboration instead of trusting either signal alone is a concrete instance of defensive design — the system is built to recognize when it doesn't actually know something, rather than confidently guessing.