OnePlus & Meizu get caught cheating on benchmark tests

Was there ever a test that could not be cheated? Turns our benchmark cheating is back, but hopefully not here to stay

  1. Techook
  2. News
  3. OnePlus & Meizu get caught cheating on benchmark tests

Highlights

    • OnePlus and Meizu articiaally boosting benchmark scores
    • Thermal limits far more relaxed for benchmarks
    • Benchmark cheating was originally exposed as an industry wide practice in 2013

When we’re looking to understand just how poorly or well a phone performs, we turn to benchmark apps. These apps are designed to objectively measure the throughput of a phone’s CPU-RAM-GPU  combination across varying phones, resulting in a number that can be translated across all price and specification points. Suffice to say, Benchmark results have become a testament of a phone’s performance, so when its found that some manufacturers are tweaking their phone’s OS to tamper with benchmark results, one can’t help but not be surprised.

According to a report by XDA, it turns out that OnePlus 3 and Meizu have been caught cheating on these benchmark tests. In collaboration with Geekbench, XDA found that the operating system on the OnePlus 3T and the Meizu Pro 6 were desgiend to identify when a benchmark was launched and accordingly increase the resting CPU clock-speed, along with relaxing the thermal limitations on the chip. The company ran a version of Geekbench that was written from the ground up to be recognized as a “Mini Golf game” to see if there would be a difference in performance reported. The benchmark remains the same, only the name changed at every level so that the OS wouldn’t recognize it as a benchmark. While the OnePlus 3T showed a very slight difference in CPU performance, it should show the phone being lenient with thermal temperatures up to 3 degrees. At peak load on Geekbench (actual), the temperature registered was 48 degrees celcius while that on the Mini Golf version of the benchmark, it was 44 degrees. The Benchmark scores saw a variance of anywhere between 50-200 points, which isn’t as big of a deal.

While the OnePlus 3T showed only very minor performance jumps, the Meizu Pro 6, on the other hand, was a whole different story. XDA found that performance scores varied by anywhere between 1000-1400 points between the regular benchmark and the mini-golf game. The phone offers a performance mode, which allegedly only offers actual performance boost when benchmarks are run and drops the CPU clocks back to those of Balance Mode otherwise.

Cheating on benchmark tests is nothing new, given the huge expose of 2013 where many companies including Samsung, Sony, HTC etc were found to be boosting performance for benchmarks. The practices were stopped after an industry wide shaming, however, it is shocking to see similar actions be repeated years later. While OnePlus 3T does boost its scores artificially for benchmark apps, we found that in our review, the phone’s real life performance to be top-of-the-line. There wasn’t a task (or several) that the phone couldn’t handle, so it’s a little surprising to see the company resort to such methods when they already have such an able performer in their stable. The Meizu 6 Pro, on the other hand, skews the results far more.

This case has shown the importance of relying on real world usage to corroborate the benchmark scores even more. For this very reason, we make it a point to use the phone over the course of a number of days to see whether the performance reported by the benchmarks matched the phone’s performance in real life. At the end of the day, benchmark scores only matter if the phone is able to reflect that performance in a person’s usage pattern otherwise, they’re just worthless number.