CLINICAL BENCHMARK // 301 NEJM CPC EVALUATED CASES
The hardest cases in medicine. Ranked. Published.
On the publicly available NEJM Clinicopathological Conference cases, the diagnoses that stump specialists, DeepSensi does not just score well. It refuses to miss the ones that matter. In clinical systems, state is not declared by code. State is a function of proven evidence.
Diagnostic accuracy on NEJM Clinicopathological Conference cases
Test conditions. NEJM Clinicopathological Conference (CPC) cases, text only (no vision), in a double-blinded test. DeepSensi was not trained on these cases: it reaches each diagnosis by medical deduction. The published results below it come from another protocol and are shown with their source.
DeepSensi test: double-blinded, text only (no vision). Comparison models run zero-shot on the same 100 cases.
DeepSensi Cognitive Engine FAST Challenger N=301
Gemini 3.8 Flash (high reasoning) Zero-shot N=100
Claude Opus 4.8 Zero-shot N=100
Gemini 2.5 Pro Zero-shot N=100
Claude Fable 5 Zero-shot N=100
Published results on NEJM CPC cases: Microsoft SDBench, sequential diagnosis protocol.
Microsoft MAI-DxO (orchestrating o3) Sequential N=304
Missed criticalUnavailable OpenAI o3 Sequential N=304
Missed criticalUnavailable Gemini 2.5 Pro Sequential N=304
Missed criticalUnavailable Claude 4 Opus Sequential N=304
Missed criticalUnavailable OpenAI GPT-4o Sequential N=304
Missed criticalUnavailable Practising physicians, unaided Sequential N=56
Missed criticalUnavailable Missed critical: the correct diagnosis is not among the top three. Sources, per model: DeepSensi Cognitive Engine: DeepSensi development benchmark, double-blinded, 301 NEJM CPC cases (2014 to 2023), text only, DeepSensi “FAST Challenger” configuration. Gemini 3.8 Flash (high reasoning): DeepSensi zero-shot comparison study, double-blinded, 100 NEJM CPC cases, single model, model gemini-3.8-flash, high reasoning, 18 September 2026. Claude Opus 4.8: DeepSensi zero-shot comparison study, double-blinded, 100 NEJM CPC cases, single model, model claude-opus-4-8, September 2026. Gemini 2.5 Pro: DeepSensi zero-shot comparison study, double-blinded, 100 NEJM CPC cases, single model, model gemini-2.5-pro, September 2026. Claude Fable 5: DeepSensi zero-shot comparison study, double-blinded, 100 NEJM CPC cases, single model, model claude-fable-5, September 2026. MAI-DxO, o3, Gemini 2.5 Pro, Claude 4 Opus, GPT-4o and physicians: Nori et al. 2025, “Sequential Diagnosis with Language Models”, Microsoft AI, arXiv:2506.22405: 304 NEJM CPC cases (2017 to 2025), sequential protocol in which the model asks questions and orders tests before giving one final diagnosis; MAI-DxO in its best configuration; Gemini 2.5 Pro and Claude 4 Opus as off-the-shelf baselines (Figure 8); physicians on the 56-case test set.
DeepSensi, zero-shot models and published results on NEJM CPC cases| System | Protocol | Cases | Top-1 | Top-3 | Missed critical | Source |
|---|
| DeepSensi Cognitive Engine | FAST Challenger | 301 | 93.3-96.7% | 100% | Zero | DeepSensi development benchmark, double-blinded |
| Gemini 3.8 Flash (high reasoning) | Zero-shot | 100 | 82.0% | 91.0% | 9.0% | DeepSensi zero-shot comparison study, double-blinded |
| Claude Opus 4.8 | Zero-shot | 100 | 70.0% | 83.0% | 17.0% | DeepSensi zero-shot comparison study, double-blinded |
| Gemini 2.5 Pro | Zero-shot | 100 | 58.0% | 82.0% | 18.0% | DeepSensi zero-shot comparison study, double-blinded |
| Claude Fable 5 | Zero-shot | 100 | 39.0% | 55.0% | 45.0% | DeepSensi zero-shot comparison study, double-blinded |
| Microsoft MAI-DxO (orchestrating o3) | Sequential | 304 | 85.5% | Not measured | Unavailable | Nori et al. 2025, arXiv:2506.22405 (SDBench) |
| OpenAI o3 | Sequential | 304 | 78.6% | Not measured | Unavailable | Nori et al. 2025, arXiv:2506.22405 (SDBench) |
| Gemini 2.5 Pro | Sequential | 304 | 68.8% | Not measured | Unavailable | Nori et al. 2025, arXiv:2506.22405 (SDBench) |
| Claude 4 Opus | Sequential | 304 | 68.4% | Not measured | Unavailable | Nori et al. 2025, arXiv:2506.22405 (SDBench) |
| OpenAI GPT-4o | Sequential | 304 | 49.3% | Not measured | Unavailable | Nori et al. 2025, arXiv:2506.22405 (SDBench) |
| Practising physicians, unaided | Sequential | 56 | 19.9% | Not measured | Unavailable | Nori et al. 2025, arXiv:2506.22405 (SDBench) |
Performance results. The figures in this section were measured on the diagnostic device configuration, DeepSensi Cognitive Medical OS, in its development phase: an internal measurement that is not independent of the Company, on 301 NEJM Clinicopathological Conference cases (2014 to 2023), with single frontier models run zero-shot as comparators (cases per row as shown); published results of other systems are cited with their source. Regulatory status: deepsensi.com/regulatory