Vertical AI vs. General AI: AppliedXL's Floor Beats Claude, ChatGPT, and Perplexity's Ceiling on Biopharma Agentic Research

April 2026 benchmark — Will Katzka, AI Analyst, AppliedXL

Five top-performing AI systems answered ten biopharma strategy queries, one shot, no follow-ups. Responses were scored on depth, domain specificity, completeness, signal quality, and actionability. AXL (Vulcan) and AXL (Minerva) took the top two spots on the leaderboard. Full methodology is disclosed at the end for replicability.

Performance overview

Scores shown here use the four-rater non-self panel described below.

Key findings

Capability spread

Average score per criterion across the ten queries, out of 10.

Per-query totals

Out of 50 points per prompt. AXL (Vulcan) ranges from 46.5 to 48.5 across the ten queries; AXL (Minerva) from 45.0 to 47.0. The next-highest system, Claude (Opus), ranges from 33.0 to 35.0. These totals use the four-rater non-self panel, with self-rater inflation removed for Claude (Opus), ChatGPT (5.4), and Perplexity.

Ten scored queries

Conclusion

Methodology

Blinded multi-rater scoring with self-preference correction

Design. Four systems (AXL, Claude (Opus), ChatGPT (5.4), Perplexity Pro) received identical prompts across ten clinical intelligence queries. Each system produced one response per query in a single shot, with no follow-ups or reprompting.

Scoring. Responses were scored by four independent LLM raters: Gemini, Claude, ChatGPT (5.4), and Perplexity. Each rater scored the full set three times independently, yielding 36 scoring passes per response and 1,800 scored data points across the benchmark. Scoring used an anchored 1–10 rubric across five criteria: Depth, Domain Specificity, Completeness, Signal Quality, and Actionability.

Blinding. System identities were replaced with randomized A–E labels that were re-randomized for each query. System headers and identifying formatting were stripped from all responses before scoring. The decode key was held separately and not joined to the scores until all rating passes were complete.

Inter-rater agreement. Spearman rank correlations across raters: Claude–ChatGPT (5.4) ρ = 0.938 (strong), Perplexity vs. others ρ = 0.64–0.67 (moderate-to-strong), Gemini vs. others ρ = 0.53–0.58 (moderate). No single rater's calibration dominates the panel average.

Self-preference correction. LLM raters are documented to inflate scores for responses generated by their own model family (Panickssery et al., 2024). To correct for this, a non-self panel was constructed for each scored system: Claude (Opus)'s scores exclude the Claude rater (measured self-inflation +9.6%), ChatGPT (5.4)'s scores exclude the ChatGPT (5.4) rater (+2.6%), and Perplexity's scores exclude the Perplexity rater (+3.6%). AXL has no model-family overlap with any rater and uses the full four-rater average.

Parameter definition & scoring

Depth — how substantive and detailed is the response?

• 1–3: Surface-level overview; could be a Wikipedia summary or press release digest.
• 4–5: Solid narrative with some specifics (asset names, general mechanisms).
• 6–7: Detailed analysis with multiple data points, named trials or programs.
• 8–9: Granular, multi-layered analysis with quantified metrics and cross-referenced data.
• 10: Exhaustive — every claim grounded in specific, verifiable data with contextual interpretation.

Domain Specificity — does it reflect biopharma and regulatory expertise?

• 1–3: Could apply to any industry; no regulatory, clinical, or competitive framing.
• 4–5: Correct terminology, basic awareness of drug development stages.
• 6–7: Reflects working knowledge of trial design, regulatory pathways, competitive dynamics.
• 8–9: Uses domain-native frameworks (termination rates, enrollment behavioral signals, landscape scoring).
• 10: Indistinguishable from output produced by a senior biopharma analyst or portfolio manager.

Completeness — does it fully address the question?

• 1–3: Addresses one dimension of the question, misses major angles.
• 4–5: Covers the main theme but omits adjacent considerations.
• 6–7: Addresses most dimensions with reasonable breadth.
• 8–9: Comprehensive coverage across scientific, operational, competitive, and strategic dimensions.
• 10: No material angle left unaddressed.

Signal Quality — does it surface non-obvious or early-stage insights?

• 1–3: Consensus knowledge only — available in any review article or press coverage.
• 4–5: One non-obvious observation, but ungrounded or speculative.
• 6–7: Multiple non-obvious insights with partial data support.
• 8–9: Novel inferences from primary data; falsifiable claims; counter-signals identified.
• 10: Surfaces signals invisible without proprietary or behavioral data analysis.

Actionability — can a decision-maker act on this response?

• 1–3: Reader understands the topic better but cannot make a specific decision.
• 4–5: Directional guidance without specifics (e.g., "this space has risk").
• 6–7: Named entities and general recommendations, but no prioritization framework.
• 8–9: Specific trials, assets, or opportunities identified with risk/reward framing.
• 10: A decision-maker could act on the output directly — invest, avoid, monitor, partner.