For as long as we’ve had benchmarks, we’ve had companies willing to cheat on those benchmarks to make themselves look good. It’s a never-ending cycle to identify what companies do and how they do it, and we don’t often get a clear window into the process.
XDA-Developers managed to get their hands on a OnePlus 5, however, and the results are damning. This is a phone that’s designed to deliberately obfuscate its own performance in the name of looking better than the competition. Here’s how they describe the problem:
The OnePlus 5, on the other hand, is an entirely different beast — it resorts to the kind of obvious, calculated cheating mechanisms we saw in flagships in the early days of Android, an approach that is clearly intended to maximize scores in the most misleading fashion. While there are no governor switches when a user enters a benchmark (at least, we can’t seem to see that’s the case), the minimum frequency of the little cluster jumps to the maximum frequency as seen under performance governors. All little cores are affected and kept at 1.9GHz, and it is through this cheat that OnePlus achieves some of the highest GeekBench 4 scores of a Snapdragon 835 to date – and likely the highest attainable given its no-compromise configuration with its specific configuration.
The slide below shows how the OnePlus 5 performs if you obfuscate the benchmark it’s running, and compares it with how the phone behaves in normal circumstances:
That’s the little, non-cheating cluster on top, and the big, cheating cluster below it. According to XDA, the cheating boosts performance by about 5 percent compared with other solutions equipped with a Snapdragon 835.
There are several problems with what the company is doing here. First and foremost, they’re driving the phone’s SoCs in a way Qualcomm didn’t necessarily design for, and that the chip may not be able to handle in the long term. Second, there’s no indication these cheating optimizations are applied in any non-benchmark tests. One of the differences between the Turbo Mode frequencies that Intel and AMD offer and this kind of skullduggery is that Intel and AMD are both up front about the base and max frequency of their processors. Digging around in those settings may require some basic understanding of how UEFI works (or how to use third party tools to accomplish the same thing), but they do work and fall under user control. There’s no such option here; users can’t turn this feature off in the Settings menu or an equivalent.
And finally, this company is advertising its own capabilities as being ahead of other Snapdragon 835 devices when it’s cheating to get there. Again, we’ve seen similar patterns from motherboard manufacturers in the past, where companies would goose their clock or FSB speeds to run a chip just a tiny bit faster than it otherwise would. But these optimizations don’t always work — I’ve tested high-end chips in motherboards before and had them destabilize for no apparent reason until I realized that the UEFI had automatically applied an overclock I wasn’t even aware of. Keeping benchmark workloads consistent is critical for testing mobile devices, and frankly, most companies don’t do a very good job of it.
Given the heat and limited dissipation of a smartphone, I’ve got to agree with XDA on this one. Overclocking a smartphone (which is what’s going on here, more or less) is not a smart move. And the company’s decision to try and pull this trick to look better in benchmarks isn’t going to win it any points from us. We didn’t like it when Samsung pulled these tricks, and we certainly don’t like it now.
The worst thing about this kind of cheating is that it’s done in the name of making a product look faster at the expense of other factors. Higher clock speeds mean hotter products and less battery life. This doesn’t matter much when you work with a desktop, but in mobile devices weight and battery life count for far more. It’s the kind of issue developers should have long since adjusted for. Five percent extra performance in a few synthetic tests of already marginal value just isn’t worth the hassle.