StateBizMap
A network map of who's really behind a state's businesses.
A geospatial network application over public business registries — 339,000 Iowa entities plus 136,000 on the Gulf Coast — that renders every business as a point and draws the hidden connections between them, linking companies that share a registered agent of record.
The network
Rendered live in your browser from real registry records — the same data and the same shared-agent logic the production app runs on. Open the full map for all 339,000 points, satellite basemaps, the agent explorer, and the collaboration layer.
Real records, not mock data: 30,222 sampled from 292,203 in-state entities (a bounding-box clip drops rows the source mis-geocoded). The point cloud is the only geography here — no basemap is drawn; the state takes shape purely from where businesses are. Each colour is one registered agent of record; hover a node for the business, click an agent to isolate its network. Commercial filing mills are filtered out, so what's left are genuine clusters — law firms and individual attorneys fronting dozens of entities across dozens of counties.
The problem
Every state publishes its business registry, and almost none of it is usable. It arrives as a 95MB flat file or a 1.74GB fixed-width archive: one row per company, no relationships, no map, no way to see that forty separate LLCs are all fronted by the same attorney two counties away.
The interesting information isn't in any single row — it's in what the rows have in common. StateBizMap turns those files into a map you can fly around and a network you can trace, so patterns that were structurally invisible in a spreadsheet become the first thing you see.
What I built
- Renders 339,000 Iowa businesses at full density — no sampling. An early version drew a random 12k subset, so zooming a city showed 3% of it; the fix ships all points as a browser-cached static payload the map fetches once instead of re-serializing 81MB on every interaction.
- Builds the connection graph from shared registered agents, then filters out commercial filing mills so the remaining clusters are genuine — a single law firm acting as agent for dozens of property LLCs, visible as one bright constellation across the state.
- Multi-source ingestion, all free: Iowa's bulk CSV (which ships coordinates), Florida's Sunbiz bulk archive over public SFTP (fixed-width 1,440-character records, 6 officers each), and Overture Maps places pulled from S3 parquet with DuckDB and clipped to a county polygon.
- Addresses without coordinates are resolved through the free Census batch geocoder at ~9,000 per request with a bounding-box sanity clip that drops fuzzy mismatches.
- A 32× memory reduction, driven by measurement rather than guesswork: the service idled at 1.5GB and peaked at 2.5GB on a box with 420MB free. Profiling found unbounded caches and string-typed parsing, not data volume — pre-baked Parquet with categorical dtypes, bounded cache entries, and a systemd memory ceiling brought idle RSS to 46MB.
- Multi-user collaboration on top of the map: accounts, saved companies, shared notes, a multiplayer pin board, and an agent explorer that ranks agents by reach and flies the camera to a cluster.
- Region-modular by design — every state-specific fact (source URL, column mapping, coordinate pairs, filing-mill list) lives in one config registry, so adding a region is a config entry rather than a code change.
Architecture
- 1Bulk registry sources (Iowa CSV · Florida Sunbiz SFTP · Overture/DuckDB) → parsers
- 2Normalize + geocode (free Census batch) → canonical schema
- 3Offline pre-bake → Parquet with categorical dtypes (95MB CSV → 23MB, 0.46s load)
- 4Connection builder → shared-agent edges, filing mills excluded
- 5Streamlit + PyDeck app → full-density static point payload + arc layer
- 6Self-hosted on a VPS behind a private tunnel, memory-capped by systemd
What it demonstrates
Data engineering against sources that don't want to cooperate: a 1.74GB archive compressed with Deflate64, which every standard Python library fails to open, streamed through an alternate decompressor rather than expanded to 18GB on disk; fixed-width records parsed by byte offset; names normalized; hundreds of thousands of addresses geocoded for free.
It's also the clearest example I have of performance work done properly. The instinct was that the problem was data size. Measuring showed it was unbounded caching and dtype choices — a per-filter-combination cache that never evicted, and 24 string columns where five categoricals would do. The fix was four reversible levers and a hard systemd ceiling so the failure mode is bounded even if I'm wrong again.
And it's a system-design problem in the way that matters commercially: the binding constraint on adding a new state isn't code, it's whether the data is free and whether the box can hold it. The architecture is arranged so both questions have cheap answers.
A note on scope
Two states are live; a third was surveyed and deliberately dropped when the data turned out to cost $1,000 up front, and a 3.3M-row candidate is held until the pre-baking work makes it affordable to serve. Knowing which regions not to build is part of the design.
Stack
More work