What parsing a legislature's PDFs taught me about data I could not get
Four lessons from turning a legislature's floor minutes into a public scoreboard, and why the hardest part was deciding what not to show
Costa Rica's Legislative Assembly publishes the minutes of every floor session. They are public, official, and delivered as PDFs. I wanted a public scoreboard for the 57 legislators of the 2026-2030 term, the kind of standings table you would see for a football league, so I had to turn those documents into data.
I expected the hard part to be the PDF. It was not. Converting a PDF to text is a solved problem. What the text looks like afterwards is where the work starts.
The names were harder than the structure
Finding the roll call inside a session record is easy. The section has a recognizable heading and a recognizable end. The trouble is what sits between them: every legislator present, printed as one unpunctuated run of words. No commas, no line breaks, no numbering.
Spanish naming makes this worse than it sounds. Everyone has two surnames, but the number of given names varies. Some legislators have one, some have two. So you cannot split on a fixed word count, and there is no delimiter to split on at all.
What worked was turning the problem around. I already had the official roster from the electoral authority, so I knew who could appear. The two surnames of each person are stable and, as a pair, unique within the chamber. The parser anchors on each surname pair it finds in the run, and treats whatever sits between one anchor and the next as the given name of the following person. A closed, known list beat clever tokenizing.
Public does not mean usable
Some of the numbers I wanted most are technically public and practically out of reach. Motions are the clearest case. The Assembly publishes them, but as scanned images inside a PDF, one file per bill, with no index by legislator. To count motions per person you would need to OCR every bill file in the term and then attribute each motion by hand.
Other data does not exist in any public form. Legislative advisors require a formal records request. Visits received by each office have no public register at all. I stopped at those walls instead of estimating. The scoreboard has a page that lists what it does not know and why, and I think that page does more for trust than any chart on the site.
A rule with no real example is not a metric
I wrote a rule to count interruptions on the floor, based on how the minutes transcribe them. It looked reasonable. Then I went looking for a session record that actually contains an interruption, so I could check the rule against a real one. I never found one in the material I had.
A rule that has never met a true positive is a guess with a regular expression attached. So that column ships as pending. The same logic applies across the whole table: every value carries its status and its source, and pending is never rendered as zero. Zero means it was counted and there were none. Pending means nobody has counted yet. On a page that ranks real people by name, mixing those two up is not a rounding error.