Primate Labs, the developer behind Geekbench, will now display warnings on benchmark results for Intel’s new Arrow Lake Plus and Panther Lake processors when Intel’s Binary Optimization Tool (BOT) is active. The BOT tool has the potential to boost overall Geekbench 6 scores by up to 8%, and even individual workloads by as much as 40%. Primate Labs asserts that these altered results are not comparable to standard benchmark runs, as BOT modifies the benchmark in a manner that remains undisclosed.
Given that Geekbench cannot detect whether BOT was utilized, all benchmark results for Arrow Lake Plus chips will feature a disclaimer: “This benchmark result may be invalid due to binary modification tools that can run on this system.” Primate Labs emphasized the critical importance of this action for upholding the integrity of scores reported on the Geekbench Browser, committing to update these warnings as Intel’s list of supported CPUs evolves.
The justification for this decision is subject to debate. Some critics contend that Geekbench results inherently vary across different platforms, and that BOT’s function is merely to optimize instruction execution, akin to the internal processes of various CPU architectures. BOT essentially reorders instructions to maximize pipeline utilization without skipping any actual computations. However, a significant concern is that BOT is not universally compatible with all software; it necessitates application-specific profiles. Consequently, results enhanced by BOT might not accurately represent the CPU’s native performance without the tool.
Intel has a history marked by contentious benchmark optimizations, including SPEC invalidating Intel benchmark results in 2024 due to “unfair” compiler optimizations, and its ICC compiler deliberately degrading AMD CPU performance in 2009 by removing optimizations for rival architectures. The apprehension is that BOT could initiate an “optimization arms race,” further obscuring genuine CPU performance metrics. Considering these precedents, marking BOT-modified results appears to be a sensible approach. Intel likens BOT’s operation to arranging instructions optimally, much like a game of Tetris.