Five federal feeds publish product recalls, and no two of them describe a recall the same way. A service checks all five on a timer, rewrites every record into one common format, and puts them all on a single four-level severity scale. That half runs today; the app a person would open is not built yet.
A car seat, a bag of spinach, and a blood pressure pill can all be recalled, and the three notices arrive in three different shapes. Five federal feeds publish them: the Consumer Product Safety Commission through SaferProducts, the FDA through its public data service openFDA on separate food and drug feeds, NHTSA for vehicles, and the Agriculture Department's Food Safety and Inspection Service, FSIS, for meat and poultry.
The five disagree about danger in kind, not merely in wording. The FDA stamps a recall with a numbered class, Class I through Class III, where Class I is the tier that can kill you. NHTSA classifies nothing and instead tells owners to park the car, or park it outside. CPSC and FSIS publish no severity at all, so two sources give a code, one gives a phrase, two give nothing, and nothing converts one agency's word into another's. Nobody is going to read five feeds to find out which one owns the model number in their hand.
The feeds are polled rather than pushed: the system asks each agency for recent recalls on a timer instead of being told when something appears, so every poll returns records already in the database. Being handed the same recall repeatedly is ordinary, not an error.
Severity cannot be copied across either, because for two of the five there is nothing to copy, and the classifier still has to answer from the recall's own text.
The ingestion engine is the back end that collects records, cleans them up, and stores them with no screen attached. It polls CPSC/SaferProducts, openFDA Food, openFDA Drug, NHTSA, and USDA FSIS on cron schedules staggered so no two fire at the same moment. Each record is normalized, meaning rewritten into one shared set of fields called the schema, so a spinach recall and a truck recall become the same kind of row, then given one of four severity levels and written behind a uniqueness rule described below. One REST API, a single web address returning rows as JSON, serves the result.
All 728 recalls in the database carry a severity: 273 Critical, 339 High, 42 Moderate, 74 Low.
The part that has to make a judgement is worth reading slowly. classifyDanger is handed the recall's text, the FDA class if one arrived, and the NHTSA advisory if one arrived, then checks a fixed list of conditions from the top down and stops at the first match, which is why the list is a ladder and its entries are rungs.
The first rung is a keyword rung, above every agency field, and that is deliberate: explicit death language in the recall's own text outranks the agency's classification, so a recall whose description says someone died comes out Critical even when the agency filed it as Class II. Read the agency field first instead and the same recall shows up as High.
Those keyword rungs are the only reason two of the five sources can be classified at all: CPSC and FSIS publish no severity field, so their records either land on rungs 1, 5, and 7 or fall to the bottom. Critical keywords include death, suffocation, and stop using immediately; High includes choking, listeria, and electrocution; Moderate includes undeclared allergen and missing label. Each tier is longer than these examples.
Low is the fall-through, not a judgement: no severity field arrived and no keyword matched. 74 of the 728 records sit there.
Five cron expressions, no two identical: every two hours at :15, every three hours on the hour, every four hours at :10, every six hours at :30, and every six hours at :45. Two share a cadence, but the minute offsets all differ, so no two polls start in the same minute. They write into one SQLite file in WAL mode, where a write does not block the API's reads.
Re-polling is made safe by the database, not by timing. A recall is identified by the pair of source and source id rather than the id alone, as two agencies can issue the same number, and a uniqueness constraint on that pair means an already-stored record cannot be stored twice, which is what deduplication means here and why the cron can run as often as it likes.
The API could have handed each source out in its native shape with a tag naming the agency, which is less code at ingestion. But every consumer would then have to learn five field layouts and keep up as the agencies change them. Normalizing on the way in pays that cost once.
Passing their language through untranslated, Class II on a drug and park outside on a truck, is truer to the source and produces a screen nobody can use: the two cannot be ranked against each other, neither means much to a reader who does not know the scheme, and for CPSC and FSIS there is no phrase to pass through.
The alternative is an application-level check: look for the record, write it if it is absent. With five schedules writing into one file, that is two operations with a gap between them, and the gap is where a duplicate gets in. The constraint is one operation, and it holds however careless the code around it is.
The ingestion engine is live and the consumer app is not. The React front end is 122 lines. What works is the half underneath, built in this order: polling, normalization, the classifier, the deduplication constraint, and an API that returns rows.
All five feeds are wired up, and one of them, the USDA FSIS feed, has never returned a record. It does not crash, and a crash would have announced itself on the first run. It runs clean, raises nothing, and yields nothing, so the only place the failure shows is a table with no meat or poultry recalls in it.
The system has no users, because the interface they would use does not exist. It is meant to be one place to ask whether something in the house has been recalled, without having to know that the crib belongs to CPSC, the formula to the FDA, and the car seat to NHTSA. On one ladder the record whose text says someone died sorts above the record whose label was printed wrong.
That answer exists today, and reaching it means writing an HTTP request by hand. Closing the gap is front-end work against an API that already returns rows.
← All projects