Overview
Executive Summary
RWE Evidence Studio is an analytical product designed to guide researchers from a structured clinical question through cohort definition, SQL generation, cohort diagnostics, and evidence interpretation in a reproducible, auditable workflow.
The product uses the PICOT framework as the structured input layer and translates the resulting cohort logic into OMOP CDM-compliant SQL, making evidence generation from observational health data more transparent and reproducible.
The design prioritizes analytical transparency: each step from clinical question to final output is documented, linked to upstream design decisions, and configurable without requiring the researcher to work directly in code.
Central Question
Research Question
“How can the process of translating a clinical question into a real-world evidence analysis be made more structured, transparent, and reproducible without sacrificing analytical rigor or researcher control?”
Background
Project Context
Motivation
Generating real-world evidence from observational health data requires a sequence of decisions that are often made implicitly and documented inconsistently: how a clinical question is operationalized, how cohort membership is defined, how confounding is handled, and how results are interpreted. These decisions have large downstream effects on study validity and are not reliably reproducible across analysts or institutions. RWE Evidence Studio is designed to make this decision chain explicit, auditable, and reusable.
Background
Real-world evidence studies using claims data, electronic health records, or other observational sources require careful cohort design to avoid biases including prevalent user bias, immortal time bias, and indication confounding. The OMOP Common Data Model provides a standardized representation of patient-level health data that, when paired with reproducible cohort logic, enables cross-institutional evidence generation. Cohort diagnostic tools characterize cohort composition before analysis, providing a critical safeguard against specification errors reaching the results stage.
Contributions
My Role
End-to-end responsibility for product concept design, analytical workflow architecture, and implementation.
Designed the PICOT-to-cohort translation framework mapping clinical question components to OMOP CDM concept domains and tables
Defined the cohort logic schema: index event, inclusion and exclusion criteria, baseline window, follow-up window, and washout period specification
Architected the SQL generation layer targeting OMOP CDM observation periods, condition occurrence, drug exposure, measurement, and visit occurrence tables
Specified the cohort diagnostics workflow including attrition tracking, covariate distribution tables, and standardized mean difference checks
Designed the results interpretation layer with structured framing for effect estimates, uncertainty, and design-linked limitations
Developed the analytical safeguards layer ensuring that all design decisions are captured and versioned before analysis proceeds
Inputs
Data Sources & Scale
Workflow Stages
7
Target Data Model
OMOP CDM
Project Status
Completed
Primary Output
Reproducible SQL
OMOP Common Data Model
Period
Target data standard for all generated SQL
Scale
Standardized schema across participating institutions
Tables used
Target data standard for all generated cohort SQL. Workflow examples are illustrative and do not contain identifiable patient data.
PICOT Clinical Question Framework
Period
Applied at study design stage
Scale
Five-component structured clinical question input
Input framework for decomposing clinical questions into operationalizable cohort components: Population, Intervention, Comparator, Outcome, and Time horizon.
Pipeline
Data Engineering Workflow
Target data standard for all generated cohort SQL. Workflow examples are illustrative and do not contain identifiable patient data.
The researcher enters a structured clinical question. The product guides decomposition using the PICOT framework: Population, Intervention, Comparator, Outcome, and Time horizon.
Population criteria, intervention and comparator exposures, outcome events, and time windows are mapped to OMOP CDM concepts. Inclusion and exclusion criteria are specified in structured form with concept set references.
The index date anchor is defined: the event that marks cohort entry, such as a first drug dispensing, diagnosis date, or procedure date. Baseline and follow-up windows are set relative to the index date.
The cohort logic specification is translated into OMOP CDM-compliant SQL. Queries target the appropriate observation tables and apply the specified criteria, time windows, and concept sets in a reproducible, versioned form.
The generated cohort is characterized before analysis: attrition tracking documents criterion application, covariate distributions are computed for each exposure group, and standardized mean differences flag residual imbalance.
With a diagnostics-reviewed cohort, the appropriate statistical analysis is specified and executed. Propensity score methods are available for confounding control in comparative effectiveness applications.
Evidence outputs are structured with explicit framing of limitations, generalizability constraints, and design assumptions. Each output is linked to the upstream design decisions that produced it, creating an auditable evidence record.
Methods
Analytical Methodology
Expand each method for description and purpose.
Analysis Outputs
Selected Output Previews
All analytical outputs shown below are illustrative. Data is entirely synthetic and does not represent real patients, real clinical records, or a completed study. These visuals demonstrate the type of evidence the product is designed to generate.
This view shows how the study population narrows step by step from all available patients in the OMOP CDM source to the final analysis cohort, documenting the number removed at each eligibility and design criterion. Each step is traceable back to the upstream cohort specification.
This panel shows how the dashboard translates a structured cohort definition into OMOP CDM-compliant SQL, including the new-user design logic, concept set references, and observation period constraints. The generated query is reproducible and versioned — the same cohort specification produces the same SQL on any compatible data source.
This output checks whether the two study groups are comparable at baseline after propensity score matching. Each covariate is plotted twice: open circles show the imbalance before matching and filled circles show the residual difference after. Points to the left of the threshold line indicate acceptable balance.
This summary condenses the full analysis into a short, decision-ready brief that highlights the study question, population, result, confidence interval, and key interpretation notes. It is designed for audiences who need the main finding and its context without reviewing the full analytic output.
Rigor
Data Quality & Methodological Safeguards
All cohort design decisions are captured in a structured specification before SQL generation begins, preventing post-hoc design changes that cannot be traced back to the original study intent
Attrition tracking documents the number of records excluded at each inclusion and exclusion criterion step, providing a complete audit trail for cohort assembly
Cohort diagnostics are conducted and reviewed before the primary analysis as a required precondition, not an optional post-hoc check
Covariate balance assessment confirms that comparison groups are sufficiently similar after confounding adjustment before effect estimates are interpreted
The new-user, active-comparator design is the default for comparative effectiveness analyses, reducing prevalent user bias and confounding by indication at the design stage
All workflow runs are versioned: the same PICOT specification and cohort logic produce the same generated SQL on the same data, supporting reproducibility across analysts and institutions
Responsible Interpretation
Limitations
Understanding these limitations is essential for correctly interpreting results.
Illustrative analytical outputs
All workflow examples and output visualizations use synthetic data and illustrative cohort counts. The product design has not been validated against a live OMOP CDM implementation or a completed external observational study.
All outputs are clearly labeled as illustrative and synthetic. Product capabilities reflect design intent and implemented workflow architecture, not production performance on a deployed data environment.
OMOP CDM implementation variation
OMOP CDM implementations vary across institutions in vocabulary mapping, ETL decisions, and concept coding practices. Generated SQL may require institution-specific customization to run correctly on a given database.
The product generates standard OMOP-compliant SQL as a starting point. Analyst review and customization before execution are expected and documented in the workflow.
Unmeasured confounding
Observational studies cannot control for confounders not captured in the available data. Even well-designed OMOP cohort studies with propensity score adjustment remain subject to unmeasured and residual confounding.
The product includes structured prompts for identifying potential unmeasured confounders at the design stage and provides sensitivity analysis options for assessing robustness to uncontrolled confounding assumptions.
Index event definition requires clinical judgment
The choice of index event has large effects on study validity that an automated product cannot fully determine. The clinical judgment required to define a valid index event exceeds what a workflow tool can enforce.
The product guides index event specification with structured prompts and documented decision rules but does not automate the clinical judgment required. Analyst review of index event logic is required before SQL generation.
Data source generalizability
RWE findings are specific to the patient population, care setting, and time period captured in the underlying data source. OMOP-standardized data from different institutions represent different populations and may not be combined without harmonization.
The results interpretation layer includes structured framing for generalizability constraints. Cohort diagnostics characterize the study population before results are interpreted.
Stack
Technology Stack
Data standard
Query language
Data pipeline
Statistical analysis
Reflection
Lessons Learned
Translating a clinical question into a valid cohort design requires more structured guidance than most RWE tools provide. The gap between a researcher's intent and a correctly specified OMOP cohort is a common failure point in observational research.
Cohort diagnostics should be a precondition for interpretation, not an optional step. Building diagnostics into the required workflow before analysis results are accessible fundamentally changes how errors are caught.
SQL generation from a structured specification surfaces ambiguities in the cohort logic that are not apparent in natural language descriptions. The translation step itself functions as a form of specification review.
The new-user, active-comparator design resolves several common confounding problems and should be the workflow default, not an advanced option requiring deep methodological knowledge to select.
Forward
Next Steps
Validate the PICOT-to-cohort translation framework against published observational studies with known cohort specifications
Connect the SQL generation layer to a test OMOP CDM instance to validate query correctness on real database structure
Implement the cohort diagnostics module using OHDSI CohortDiagnostics outputs as a reference standard for characterization tables
Design the results interpretation template with structured framing for effect estimate limitations, generalizability, and design assumptions
Evaluate integration with OHDSI Atlas as a front-end option for cohort specification and vocabulary browsing