Turning Repeat Passes Into a Change-Detection Layer

Every drive-by pass logs a GNSS trajectory: latitude, longitude, timestamp, usually at 1 to 10 Hz depending on the receiver. The naive assumption, the one that shows up in a lot of pitch decks, is that two passes down the same street give you two versions of the same line. They don't. Consumer-grade GNSS, even with WAAS or EGNOS correction, drifts by several meters depending on satellite geometry, multipath bouncing off buildings, and whatever the atmosphere is doing that day. A truck driving the same lane at 9 a.m. Tuesday and 2 p.m. Thursday can log centerlines five or six meters apart, and that's plenty to put a pothole on the wrong side of a two-lane road in your database. Raw coordinates therefore can't be the join key across passes. You need map matching: snap each trajectory to a known road network, whether that's OpenStreetMap, a state DOT centerline file, or something proprietary, and then describe position as distance along the segment plus offset from centerline, rather than a lat/lon pair floating in space. Hidden Markov Model map matchers, the approach Newson and Krumm laid out at Microsoft back in 2009, handle this well because they weigh both the raw GNSS reading and whether the route makes sense given how roads actually connect. Once you've got that linear reference instead of a raw coordinate, comparing pass 14 to pass 87 turns into matching along one shared, stable axis instead of trying to line up two noisy point clouds floating in open space. Skip this step and every later stage inherits the error. A crack flagged at meter 340 on one pass and meter 344 on another is either the same crack seen twice or two separate cracks four meters apart, and without map matching there's no way to tell which. That one choice decides whether your change-detection layer holds up or just adds noise. ## Deduplication is where the real engineering lives Say alignment works and every detection now sits at a stable linear reference. The next headache: the same pothole gets flagged over and over, by different trucks, at different times of day, under different light and weather, with a slightly different bounding box each time. One pothole can throw off thirty or forty raw detections over a month of drive-bys. Treat each one as its own event and you'll flood the maintenance queue with duplicates, and any trend line you try to draw becomes meaningless. Deduplication here plays out a lot like entity resolution on a customer list, except the entities are potholes and cracks, and the messy identifiers are location, size estimate, and visual signature instead of name and mailing address. A workable approach clusters detections by spatial proximity along the linear reference, anything within roughly 1.5 meters counts as a candidate match, combined with a similarity score off the defect's visual embedding, assuming the detection pipeline runs a CNN or vision transformer that spits out feature vectors and not just boxes. DBSCAN, or something in that family, works well here because it doesn't ask you to guess the number of clusters up front, and nobody knows in advance how many defects sit on a given street segment. Time matters just as much, and this is what actually separates a change-detection layer from a static map of flagged spots. A cluster that starts showing up in week 3 and keeps showing up through week 9 tells you something very different from a detection that fires once and vanishes, which is far more likely a shadow, glare off a manhole cover, or road debris a street sweeper cleared the next morning. Persistence across passes from different vehicles on different days is the strongest evidence you'll get that something real sits on the ground. One detection, no matter how confident the model claims to be, is just a claim. ## Setting the threshold before the work order fires Here's where engineering judgment turns into an operational, and frankly political, call. Every vision model spits out a confidence score, and somewhere a threshold gets drawn: above it, a detection counts as real; below it, it gets tossed. Set that number too low and crews get sent out to fix tar patches and oil stains. Set it too high and real deterioration sits untouched until it's a pavement failure costing ten times as much to repair. This is a precision-recall tradeoff, plain and simple, and there's no universally correct point on that curve. It comes down to which kind of mistake the agency can stomach. A false positive costs a truck roll, maybe twenty minutes of a crew's time confirming nothing's wrong. A false negative costs whatever it costs to let a crack turn into a pothole and a pothole turn into a liability claim over a cracked axle or a cyclist going down. Pavement management researchers have tracked this deterioration curve for decades: a crack that's cheap to seal early becomes exponentially more expensive once water gets under the base course. That asymmetry argues for tolerating more false positives than most engineers feel comfortable with, if they're judging purely on model accuracy. So in practice, one confidence threshold isn't enough. A two-stage design works better: a low bar that puts a detection into a "watch" state, needing confirmation across two or three independent passes before it turns into a work order, plus a separate, higher bar that fires an immediate alert for defects severe enough to damage a vehicle the first time a sensor sees them. Fraud detection systems use the same logic, tiered thresholds instead of one binary cutoff, for the same reason: the cost of a false positive and a false negative are almost never symmetric. ## What the confidence score actually needs to represent Model confidence out of a vision pipeline isn't ground-truth probability, and treating it as such is a mistake I've seen more than once. A softmax output of 0.87 just reflects the model's own internal calibration at that number, and that calibration can be way off, especially for defect types underrepresented in training data, or for road surfaces the model saw less of during training (chip seal shows up far less often than plain asphalt in most datasets). Platt scaling or isotonic regression, run against a held-out set of confirmed field inspections, turns a raw score into something closer to an actual probability. Skip that step and you end up with a threshold that looked great in a lab evaluation and then misfires constantly once it hits streets the model was never tuned on. The multi-pass persistence check from earlier is, in a sense, a cheap stand-in for proper calibration. If you can't fully trust a single-pass confidence number, requiring the same defect to show up across independent passes gives you an empirical confidence measure grounded in repeated real-world confirmation instead of one model's internal certainty. This approach is slower. It also tends to be more honest. ## Why this only works at fleet scale None of this, not the map matching, not the clustering, not the tiered thresholds, is worth building for one vehicle making a single pass down a street once a month. The whole thing depends on enough independent passes to make persistence mean something, which means either a large municipal fleet (garbage trucks, buses, street sweepers already covering every road on a fixed schedule) or a deal with delivery or rideshare fleets that happen to cover the same ground repeatedly just by doing their normal job. Coverage isn't even across a road network, though. Arterial roads on bus routes get seen daily, sometimes hourly. A residential side street might get one qualifying pass a week, from a mail truck if it's lucky. Any agency running a system like this needs to say plainly that time-to-detection varies wildly across the network, and set maintenance response expectations to match. Selling this as uniform, real-time coverage, when the fleet doing the sensing only physically drives some streets once a week, is the kind of overpromise that blows up the first time a resident reports a pothole the "system" was supposed to have caught already. The upside, when the pipeline actually works, is a maintenance program that finds problems before residents call them in, riding on trips the fleet was already making. That's a genuinely different cost structure than a dedicated inspection vehicle or a crowd-sourced reporting app, and it's why DOTs and municipal fleet operators have started putting real engineering hours into the alignment, the deduplication, and the threshold logic, instead of treating all of it as an afterthought bolted onto a computer vision demo.