What Speedometer 3.1 actually measured
We published a browser benchmark chart. The developer of Helium said we faked it. Rather than argue, we re-ran everything under controlled conditions, recorded it, and published the whole dataset, including the runs where we lose.
Summary
- Our original chart was run on a real daily-driver profile with extensions loaded. Speedometer's instructions ask for a clean profile. We didn't disclose ours. That is a legitimate methodological error and it's ours.
- We re-ran it clean, twice, on the same hardware, screen-recorded start to finish. Aside came out ahead in both rounds: 51.09 and 50.10, versus Chrome 49.7 / 48.9 and Helium 49.1 / 48.6.
- We are not going to claim that generalizes, because our own data says it doesn't. On a second Mac in the same office, Helium wins and Aside finishes last. On wukko's M5 Max, Chrome wins and Aside finishes last.
- Across three machines, all three browsers finish first exactly once. Changing the hardware moved the same browser by 24.3 points. Changing the browser moved the same hardware by 1.5.
- Scores move around a lot depending on the machine they're run on. So take any Speedometer number you see, very much including ours, as a rough indicator rather than a ranking. It's a fun number, not a verdict.
1. What we published
@AsideAI · August 13, 2026 · 117K views
Aside is now faster than Chrome, Helium, and Arc. Aside scored 49.9 on Speedometer 3.1 benchmark, outperforming previous version (25.6), Arc (33.2), Helium (44.4), and Chrome (46.7) on the same machine. Available today.
The measurement we cared about was internal and repeatable: our own engine moved from 25.6 to 49.9 on one machine across one release. The cross-browser comparison was context we added around it, and that is the part that did not hold up.
2. The accusation
wukko (@uwukko), developer of Helium · August 13, 2026 · 71K views
the results seemed off to me, so i ran the same benchmark across major browsers in a proper environment, as per speedometer guidelines i was unable to get the same results, so i believe aside faked their benchmark results or used "dirty" environments for other browsers
The environmental criticism is correct, and it is the part that matters. The Chrome and Arc figures in our chart came from real profiles with extensions loaded. Speedometer's instructions say to use a clean profile. We should have either run it clean or labeled the chart. We did neither.
The reasoning at the time was that users don't browse in a clean state, so a loaded profile is closer to lived performance. That's a defensible thing to measure. It is not a defensible thing to publish unlabeled, next to browsers we didn't configure ourselves.
3. The conditions the benchmark asks for
Speedometer 3 general instructions, verbatim
- Ensure you are running the latest stable browser version.
- Prefer using a separate clean browser profile — extensions and non-default browser settings can have a large impact on the score.
- Don't run any programs in the background.
- Restart the browser and close all other tabs and windows.
- Keep the Speedometer page focused and do not interact with the device while running the benchmark.
- Keep your battery-powered device plugged into a charger.
- Some devices might need time to cool down between consecutive runs.
Seven conditions. Here is our compliance, honestly marked, for the original chart and for the re-run.
| Condition | Original chart | Re-run |
|---|---|---|
| Latest stable browser version | Yes | Yes |
| Separate clean profile | No | Yes |
| No background programs | No | Yes except the screen recorder |
| Browser restarted, other tabs closed | Yes | Yes |
| Page focused, no interaction | Yes | Yes |
| Plugged into charger | Yes | Yes |
| Cooldown between consecutive runs | Not controlled | Not controlled |
Note the last three rows. Power state, background load, and thermal state are never visible in a screenshot of a score, whoever posts it. That's not a dig at anyone, it's just a limitation of sharing benchmarks as screenshots, and it's why we filmed ours this time.
4. The controlled re-run
Clean installs, default settings, everything quit, recorded end to end, two rounds
5. What the numbers support
Three findings, in descending order of how much they should change your mind
Finding 1 — The reported error bars overlap the entire gap
Speedometer publishes a confidence interval with every score. Our margin over Chrome and Helium is smaller than that interval. Below, both rounds on identical hardware, plotted with the intervals the benchmark itself reported.
In round 1, Aside's interval clears Chrome's by 0.31 points. In round 2, on the same machine ten minutes later, Chrome's interval and Helium's interval each fully contain Aside's. The ordering did not change. The evidence for the ordering evaporated.
A single Speedometer run quoted as one clean number, without its interval, is being read with more precision than it really has. Our original chart included.
Finding 2 — Hardware moves the score roughly 16x more than the browser does
Hold the browser constant and change the machine, and the number swings enormously. Hold the machine constant and change the browser, and it barely moves.
The variable doing the work in a Speedometer comparison is the silicon, the thermal headroom, and the OS build. Browser choice is a rounding error next to it. Comparing an absolute score from one person's laptop to an absolute score from another person's laptop is not a comparison of browsers.
Finding 3 — The ranking is not stable across machines
If a benchmark measured a durable property of a browser engine, the ordering should survive a change of hardware. It doesn't.
| Environment | 1st | 2nd | 3rd |
|---|---|---|---|
| Machine A · M4 Max · rounds 1 & 2 | Aside | Chrome | Helium |
| Machine B · M4 Pro | Helium | Chrome | Aside (Canary) |
| wukko · M5 Max | Chrome | Helium | Aside |
Three environments, three different winners, and every browser takes first place exactly once. Which is a pretty good sign that what's mostly being measured here is the environment.
The honest reading of this table is symmetric, and we'll state the half that costs us: it means our re-run does not prove Aside is faster. It shows Aside was faster on that machine, on that day, twice, in front of a camera. That's a much smaller claim than the one we originally made, and it is the largest claim this benchmark can carry.
6. The raw recordings
Machine A, round 2. Unedited, launch to score.
The screen recorder was running during these three runs and costs a little performance, which is part of why every round 2 score sits about a point below round 1. We considered that a fair trade for being able to show the runs rather than assert them.
Round 1 results
The original uncontrolled runs
These are the runs behind the chart we posted: real profile, extensions loaded, the machine in the state it is actually used in.
7. Full dataset
| Run | Environment | Browser | Score |
|---|---|---|---|
| Original post Machine A | Real profile, extensions loaded | Aside | 49.9 |
| Chrome | 46.7 | ||
| Helium | 44.4 | ||
| Arc | 33.2 | ||
| Aside (previous build) | 25.6 | ||
| Re-run round 1 Machine A | Clean profile | Aside | 51.09 ±0.44 |
| Chrome | 49.70 ±0.64 | ||
| Helium | 49.10 ±1.10 | ||
| Re-run round 2 Machine A | Clean profile, recorded | Aside | 50.10 ±0.43 |
| Chrome | 48.90 ±1.80 | ||
| Helium | 48.60 ±2.70 | ||
| Second machine Machine B | Clean profile | Helium | 41.60 |
| Chrome | 41.40 | ||
| Dia | 38.70 | ||
| Aside Canary | 36.60 | ||
| wukko's run M5 Max, 36GB | Clean, per instructions | Chrome | 65.73 |
| Helium (protections off) | 65.71 | ||
| Brave | 62.76 | ||
| Dia | 60.25 | ||
| Aside | 53.36 |
8. What would actually settle it
If anyone does want a cross-browser number worth arguing over, the recipe isn't complicated:
- One machine, one operator, one session. Cross-machine score comparison is not meaningful and should be dropped entirely.
- Interleave, don't block. Run A, B, C, A, B, C rather than five of A then five of B, so thermal drift is distributed across browsers instead of penalising whichever went last.
- n ≥ 5 per browser, with a fixed cooldown between runs.
- Report median and spread, never a single number without its interval.
- Pin and publish versions of every browser and the OS build.
- Publish the recording and the raw per-workload output, not a screenshot of the final score.
We'll run exactly that protocol and publish the result whatever it says, including if it says we're slower. If wukko wants to set the conditions, or run it himself on our hardware, that offer is genuinely open. We'd rather end up with one measurement everyone trusts than two charts nobody does.
9. Where we stand
What we got wrong. We published a cross-browser chart from an uncontrolled environment without labeling it, with bars that weren't to scale. If you put four logos on a chart, you owe people the conditions. We didn't provide them. That was careless and we're sorry for it.
What we stand behind. Aside's engine went from 25.6 to 49.9 on the same machine across one release. That is a same-machine, same-conditions comparison, which is the only kind this benchmark supports, and it holds on every machine we've tested.
What we now think about the benchmark. Speedometer 3.1 is a good instrument for measuring one browser against itself on one machine over time. It's a lot shakier for ranking Chromium-derived browsers against each other, since they land within a couple of points of one another, well inside the noise floor, and the ordering reshuffles depending on the hardware. If the winner changes with the laptop, the leaderboard is mostly entertainment.
So run it yourself, on your own machine, and trust that number over ours or anyone else's. That is the only version of this claim that survives contact with the data.