Some announcements sound too good to be true. Subquadratic — a Miami-based AI startup — might be one of them. They emerged from stealth in May with $29 million in seed funding, claiming they’d cracked one of the deepest bottlenecks in modern AI. One month later, the story just got more interesting.
What Subquadratic claims
The problem is real: every transformer model computes attention quadratically. Double the text, quadruple the compute. That’s why long context windows are so expensive — and why every major lab is working on alternatives.
Subquadratic says they’ve solved it. Their model SubQ uses sparse attention instead of the standard dense attention. The numbers they’re putting out are staggering: 56x faster than FlashAttention, a context window of 12 million tokens, and a RULER 128 benchmark run costing $8 instead of $2,600 for Anthropic Opus. On LiveCodeBench — a coding benchmark — they hit 89.7%. Needle-in-a-haystack: 98% at both 6 million and 12 million tokens.
The MIT Technology Review report
Today, MIT Technology Review reports that an independent evaluation by Appen backs up many of those claims. That’s meaningful — Appen is an established name in AI evaluation. But it’s not the full picture.
There are legitimate concerns. Subquadratic reused weights from Qwen, a Chinese open-source model, and hasn’t made SubQ widely available. You can’t just download it and test it yourself. And that’s where things get uncomfortable.
The Theranos question
Investor Dan McAteer put it bluntly: SubQ is either the biggest breakthrough since the Transformer itself — or it’s ‘AI Theranos.’ Will Depue, formerly at OpenAI, was more measured: the public evidence doesn’t yet justify the stronger claim that they’ve solved the quadratic attention bottleneck.
CEO Justin Dangel, meanwhile, isn’t hedging: he says nobody will be building on transformers in a few years.
My take
This story is fascinating precisely because it’s so polarizing. The numbers are impressive, and independent validation is a good sign. But as long as SubQ isn’t broadly accessible, there’s a big question mark hanging over everything. Real breakthroughs don’t prove themselves through press releases — they prove themselves when others can reproduce them.
Here’s what I keep thinking: if it’s true, this is genuinely a paradigm shift. And if it’s not, it’ll become a case study in why independent reproducibility in AI research isn’t negotiable. Either way, this is one to watch closely.
Sources: MIT Technology Review, VentureBeat