BatchHaiku

Home · Claude Haiku guides

Claude Haiku 5.5 benchmarks

Public scores for planning bulk pipelines — not a substitute for your own golden CSV. We restate published numbers; we do not run these harnesses ourselves.

Headline scores

MetricHaiku 5.5Note
Intelligence Index (max)43Artificial Analysis
Terminal-Bench 4.0 (AA)33%Vendor card may cite 39.2%
AutomationBench-AA~35%Luna often higher here
Context / max out1M / 128KProvider docs

Charts at a glance

Every figure below is traced to a public source (see caption under each chart). We do not run these harnesses ourselves — charts only restate published numbers for comparison.

Intelligence Index (max / high effort)

Independent leaderboard — small/flash peers vs Haiku & Sonnet

  • Sonnet 5.5 (max)56
  • Kimi K344
  • Haiku 5.5 (max)43
  • GLM-5.3 Flash42
  • Gemini 3.8 Flash41
  • GPT-6 Luna (max)38
  • Haiku 5.5 (high)38
  • Prior Haiku gen (≈)≈17

Source: Artificial Analysis article, 7 Oct 2026 (max 43; high 38; peers as cited). Prior Haiku ≈17 from AA’s “+26 vs last Haiku” (43−26).

SWE-bench Pro (max effort)

Vendor-reported — Anthropic system card §8.2 (Pro version not specified)

  • Sonnet 5.581.3% %
  • Haiku 5.564.8% %

Source: Anthropic Claude Haiku 5.5 System Card, Table / §8.2 — Haiku 64.8%, Sonnet 81.3% (five-trial averages).

Terminal-Bench 4.0

Independent AA measurement (Anthropic’s own card reports 39.2% — different protocol)

  • Haiku 5.5 (AA)33% %
  • Haiku 4.5 (AA)0% %

Source: Artificial Analysis article, 7 Oct 2026 — Haiku 5.5 33%, Haiku 4.5 0%.

Effort vs tokens (Intelligence Index)

AA: max buys Index points but multiplies output tokens — costly for one-label batch rows.

Indextokens / task →High: 38~55k tokensMax: 43~162k tokens

Source: Artificial Analysis (7 Oct 2026) — high ~55k / max ~162k output tokens per Index task

Haiku 5.5 score suite

Mix of AA independent + Anthropic vendor SWE — different harnesses; compare within source

  • SWE Multilingual (vendor)83.7%
  • SWE-bench Pro (vendor)64.8%
  • AA-Omniscience (AA)36%
  • AutomationBench-AA~35%
  • Terminal-Bench (AA)33%
  • SWE Multimodal (vendor)30.7%

Source: AA article: Omniscience 36%, AutomationBench-AA ~35% (noted understated), Terminal 33%. System card §8.2: SWE Multilingual 83.7%, Pro 64.8%, Multimodal 30.7%.

List price by prompt size (Anthropic)

$ per 1M tokens — 5× step-up above 100k prompt tokens

  • Input ≤100k$0.10
  • Output ≤100k$0.50
  • Input >100k$0.50
  • Output >100k$2.50

Source: Anthropic Claude Haiku 5.5 overview / AA article — $0.10/$0.50 ≤100k; $0.50/$2.50 >100k.

Tokenizer inflation

Anthropic docs: same text ≈ 30% more tokens on Haiku 5.5 vs 4.5 (illustrative 100 → 130).

Haiku 4.5 (same string)100 tokens*
Haiku 5.5 (same string)130 tokens*

≈ +30% tokens → higher bill

Source: Anthropic Claude Haiku 5.5 overview (tokenizer note). *Bars are proportional illustration of the stated ≈30% increase, not a measured prompt sample.

Tokens per Index task (order of magnitude)

Why max effort is expensive for bulk labeling

  • Haiku 5.5 high~55k
  • Haiku 5.5 max~162k
  • GPT-6 Luna max~50k

Source: Artificial Analysis article — Haiku high ~55k; Haiku max ~162k; GPT-6 Luna max ~50k output tokens per Index task.

Independent = Artificial Analysis. Vendor = Anthropic system card (self-reported; SWE-Bench Pro version unspecified on the card). Scores move when harnesses, effort, or safety filters change — check live AA / Anthropic cards before procurement.

What this means for batch labeling

Leaderboard wins do not guarantee your review/ticket taxonomy accuracy. Use Haiku 5.5 when throughput and $/1k rows matter; spot-check against Sonnet on a gold set. Compare Luna on Haiku 5.5 vs GPT-6 Luna or open the homepage comparison tool.

BatchHaiku is an independent third-party tool, NOT affiliated with or endorsed by Anthropic. Live inference runs Claude Haiku 5.5 via Cloudflare AI under your account usage terms.

Test labels on your CSV

Related guides