claude-haiku-4-5

New Claude Haiku 4.5 and OpenAI GPT-5 Pro models have been released. I wanted to test them for code analysis for my csfa.sh nftables wrapper script and GitHub workflow action test against other LLM models I use. This is for code analysis and not code generation. Code analysis would be useful for understanding code bases, writing documentation, troubleshooting code and planning.

CSFA (v1.3.1) is a CSF-like wrapper for nftables that provides familiar ConfigServer Security & Firewall commands mapped to modern nftables equivalents. The project uses a single Bash script (csfa.sh) that manages firewall rules through a dedicated inet table called "csfa".

I have paid subscriptions and accounts with:

• OpenAI ChatGPT Plus
• Claude AI Max $100
• Gemini AI Pro
• T3 Chat
• OpenRouter AI
• KiloCode

I tested 27 AI LLM models for code analysis and summaries and then used Claude Code Sonnet 4.5 to evaluate and rank all 27 AI LLM model responses.

The 27 AI LLM models evaluated are (including costs for usage):

• Claude Code Sonnet 4.5 included in subscription cost
• OpenAI Codex GPT-5 Medium Thinking included in subscription cost
• OpenAI ChatGPT GPT-5 Thinking included in subscription cost
• Claude Code Opus 4.1 included in subscription cost
• Claude AI Web Opus 4.1 Thinking included in subscription cost
• KiloCode Claude Sonnet 4 $0.240 | inputs tokens: 80,596 | output tokens: 4,822 | cache hits: 38,818
• Google Gemini 2.5 Pro Web included in subscription cost
• KiloCode Sonoma Dusk Alpha $0.000 | inputs tokens: 66,302 | output tokens: 3,049 | cache hits: 32,168
• KiloCode Sonoma Sky Alpha $0.000 | inputs tokens: 31,761 | output tokens: 2,684 | cache hits: 397
• KiloCode MoonshotAI Kimi K2 0905 $0.020 | inputs tokens: 30,763 | output tokens: 1,484 | cache hits: 0
• KiloCode xAI Grok Code Fast 1 $0.000 | inputs tokens: 30,649 | output tokens: 1,025 | cache hits: 576
• KiloCode Qwen3 Coder $0.010 | inputs tokens: 34,309 | output tokens: 1,422 | cache hits: 0