4 posts tagged with "r-runtime"

Every platform has a weak link. For Parthenon, it was the R container.

PHP handled 200 concurrent API requests without breaking a sweat. Python served AI inference with async workers. PostgreSQL managed million-row queries across six schemas. Redis cached sessions at sub-millisecond latency. And then there was R — single-threaded, fragile, running bare Rscript as PID 1 with no supervision, no timeouts, and a health check that lied.

This is the story of how we tore it down and built Darkstar — a production-grade R analytics engine that runs OHDSI HADES analyses concurrently, recovers from crashes automatically, and executes 35% faster than the container it replaced.

The R runtime was the single most fragile component in the entire Parthenon stack. Every other service — PHP, Python AI, Solr, Redis, PostgreSQL — could handle concurrent requests gracefully. The R container could not. A single CohortMethod estimation on 1 million patients takes 5-30 minutes. During that time, the entire R process was locked — health checks timed out, status queries hung, and any other analysis request queued behind it with no feedback. This devlog covers the six-phase hardening effort that replaced the entire R runtime infrastructure in a single day.

We ran every analysis type in Parthenon — estimation, prediction, SCCS, evidence synthesis, pathways, characterization, and incidence rates — against our full Acumenus CDM with 1 million synthetic patients. Thirteen seeded analyses. Seven different OHDSI methodologies. One session.

This post covers what happened when we moved from the 2,694-patient Eunomia demo dataset to production scale, the bugs that only surface at a million patients, and the hard-won lessons about propensity score modeling on synthetic data.

The OHDSI HADES ecosystem is remarkable. CohortMethod, PatientLevelPrediction, SelfControlledCaseSeries — these R packages encode decades of pharmacoepidemiology methodology into reusable software. In theory, you point them at an OMOP CDM database, call a few functions, and get publication-ready causal inference results.

In practice, getting these packages to run correctly in a modern production environment required solving problems that no documentation warned us about.

This is the story of what we encountered building Parthenon's R runtime — a Plumber API sidecar that executes HADES analyses against a 1-million-patient CDM — and the specific, reproducible bugs we had to fix before a single analysis could complete.