Fully Open Meditron:
An Auditable Pipeline for Clinical LLMs

Clinical decision support systems (CDSS) require scrutable, auditable pipelines that enable rigorous, reproducible validation. Yet current LLM-based CDSS (LLM-CDSS) remain largely opaque. Most "open" models are open-weight only, releasing parameters while withholding the data provenance, curation procedures, and generation pipelines that determine model behavior. Fully Open (FO) models, which expose the complete training stack end-to-end, do not currently exist in medicine. We introduce Fully Open Meditron, the first fully open pipeline for building LLM-CDSS, comprising a clinician-audited training corpus, a reproducible data construction and training framework, and a use-aligned evaluation protocol. The corpus unifies eight public medical QA datasets into a normalized conversational format and expands coverage with three clinician-vetted synthetic extensions: exam-style QA, guideline-grounded QA derived from 46,469 clinical practice guidelines, and clinical vignettes. We apply the recipe to five FO base models. Apertus-70B-MeditronFO improves +6.6 points over its base (47.2% to 53.8%) on aggregate medical benchmarks, establishing a new FO SoTA for LLM-CDSS, and Gemma-3-27B-MeditronFO surpasses MedGemma on HealthBench (58% vs 55.9%). These results show that fully open pipelines can achieve state-of-the-art domain-specific performance without sacrificing auditability or reproducibility.

Our release includes:

• A reproducible end-to-end framework spanning corpus construction, synthetic data generation, decontamination, training, and evaluation.
• A structured, clinician-audited knowledge corpus normalizing eight public medical QA datasets and expanding coverage via clinician-vetted synthetic generation.
• Auto-MOOVE, an automated, open-ended clinical evaluation protocol validated against 204 human raters.
• A family of fully open medical specialists across three model families (Apertus, OLMo-2, EuroLLM).

We argue that clinically competitive medical specialization can be achieved using reproducible, auditable pipelines rather than opaque adaptation procedures, providing a reusable foundation for training and evaluating future fully open medical models.

Medical benchmark accuracy (%). Best within partition in bold; best fully open underlined. HealthBench uses the full benchmark with Qwen3-235B-A22B as judge.

Auto-MOOVE pairwise preference results: every MeditronFO model is preferred over its corresponding base (left), and Gemma-3-27B-MeditronFO is preferred over MedGemma (right).

Per-criterion Likert profiles show improvements across all nine clinical evaluation dimensions.

• Fully open state of the art. Fine-tuning on the Fully Open Meditron corpus consistently improves base models. Apertus-70B-MeditronFO reaches a 53.8% average on aggregate medical benchmarks (+6.6 over its base), establishing a new fully-open state of the art. Smaller bases benefit most, with Apertus-8B improving +12.8 points.
  
  
• Surpassing closed-data specialists. On the open-ended Auto-MOOVE protocol, validated against 204 human raters, every MeditronFO variant is preferred over its base, with adjusted win rates from 67.2% to 92%. Gemma-3-27B-MeditronFO surpasses MedGemma, a closed-data specialist, on both HealthBench (58.0 vs 55.9) and Auto-MOOVE (preferred in 58.6% of comparisons), despite being built from a fully open, auditable pipeline.

Model	Base	Base model openness			Medical adaptation openness				License
		Data	Code	Weights	Data	Synth	Code	Weights
Closed
Med-Gemini	Gemini								R
Open Weights
BioMistral	Mistral								O
Meditron-70B	Llama 2								IC
Meditron-3	Llama 3.1								IC
Aloe Beta	Llama 3.1								R
MedGemma-27B	Gemma-3-27B-it								C
Partially Open
Gemma-3-27B-MeditronFO	Gemma-3-27B-it								IC
Fully Open
Apertus-70B-MeditronFO	Apertus-70B								O
OLMo-2-32B-MeditronFO	OLMo-2-32B								O
EuroLLM-22B-MeditronFO	EuroLLM-22B								O

Openness across medical LLMs. MeditronFO is the first family to satisfy every dimension end-to-end. License: O = permissive open, C = community/commercial with restrictions, IC = inherited base-model license (adaptation permissively released), R = restrictive/research-only/proprietary.