37 posts tagged with "ohdsi"

One of OHDSI's greatest strengths is database agnosticism. The HADES ecosystem — via SqlRender and DatabaseConnector — lets researchers write analyses once and run them against SQL Server, PostgreSQL, Oracle, Snowflake, BigQuery, and seven other platforms without modification. Today, Parthenon achieved full parity with that capability: all 12 HADES-supported database dialects are now covered across both the PHP SQL translator and the R runtime.

Every platform has a weak link. For Parthenon, it was the R container.

PHP handled 200 concurrent API requests without breaking a sweat. Python served AI inference with async workers. PostgreSQL managed million-row queries across six schemas. Redis cached sessions at sub-millisecond latency. And then there was R — single-threaded, fragile, running bare Rscript as PID 1 with no supervision, no timeouts, and a health check that lied.

This is the story of how we tore it down and built Darkstar — a production-grade R analytics engine that runs OHDSI HADES analyses concurrently, recovers from crashes automatically, and executes 35% faster than the container it replaced.

A big architectural day on Parthenon: we shipped the new Workbench launcher experience and drafted the formal plan to collapse our multi-database mess into a single, schema-isolated parthenon database. A noticeably cleaner codebase on the other side.

A massive 116-commit push today centered almost entirely on maturing the Evidence Investigation workbench — from retiring the old FinnGen UI to hardening the investigation experience with proper navigation, KPI metrics, URL-synced state, and ARIA accessibility. We also landed multi-dataset support in Morpheus and set the stage for one of the most requested features on the roadmap: volcano plots powered by the newly-renamed Darkstar R runtime.

A big day on the quality and resilience front: 34 commits landed in Parthenon focused on a comprehensive codebase health audit, a major expansion of our Playwright E2E test suite, and a fork of the StudyAgent submodule. No flashy new features today — instead, we did the unglamorous but essential work of making sure what we've already built actually works, is safe to change, and won't silently break in production.

The R runtime was the single most fragile component in the entire Parthenon stack. Every other service — PHP, Python AI, Solr, Redis, PostgreSQL — could handle concurrent requests gracefully. The R container could not. A single CohortMethod estimation on 1 million patients takes 5-30 minutes. During that time, the entire R process was locked — health checks timed out, status queries hung, and any other analysis request queued behind it with no feedback. This devlog covers the six-phase hardening effort that replaced the entire R runtime infrastructure in a single day.

In a single development session, we shipped three phases of a cognitive architecture that transforms Abby from a stateless RAG chatbot into a persistent, intelligent, context-aware research assistant. She now remembers who you are, routes complex questions to a more powerful brain, traverses clinical concept hierarchies, and warns you when your data has gaps. This post tells the complete story — the problems we solved, the architecture we built, and the engineering decisions behind 188 passing tests across 60+ new files.

What started as "Abby's responses are slow" turned into an 18-hour deep dive that touched every layer of the AI stack — from GPU driver backends to embedding model race conditions to the fundamental question of why a 4-billion-parameter medical LLM was confidently inventing researcher names. By the end, Abby went from 15-25 second hallucinated responses to 2-5 second grounded answers backed by 167,000 vectors of medical knowledge — and we found that an 8-megabyte systemd memory lock was silently killing 25% of all GPU inference requests.

Abby can now answer "What concept sets do we have for diabetes?" and "How many patients are in our CDM?" with real data — queried live from the Parthenon PostgreSQL database at response time. Eight contextual tools give her awareness of concept sets, cohort definitions, vocabulary concepts, Achilles characterization stats, data quality results, cohort generation counts, CDM summaries, and analysis executions.

A productive Sunday on the Parthenon platform — 70 commits landed today covering four distinct themes: hardening the real-time Commons presence system, fixing a persistent CSRF authentication bug, overhauling the Grafana observability stack, and significantly expanding the medical knowledge base powering Abby, our AI research assistant.