Accelerating Decision Tree Ensemble with Guided Branch Approximation

Processing lightweight machine learning (ML) algorithms, such as decision tree ensemble (DTE), on low-power edge devices is beneficial; however, these devices usually have limited resources, and domain-specific accelerators are not readily available. Therefore, energy- and resource-efficient acceleration mechanisms for ML workloads on lightweight embedded microcontrollers without additional hardware accelerators are desired. However, the penalties associated with branch mispredictions can be performance bottlenecks when executing DTE on conventional in-order pipelined processors. This study proposes the Guided Branch Approximation (GBA), an approximate computing approach to improve the performance of DTE on lightweight general-purpose processors by selectively ignoring the correctness of branch instructions. GBA enhances the performance by speculatively executing selected branch instructions without any rollback on branch mispredictions. GBA allows programmers and high-level ML frameworks to annotate approximal branch instructions and to ensure target applications’ quality of service (QoS). GBA comprises the following: 1) the approximate branch instruction format, a new type of branch instruction that ignores the wrong prediction of branch predictors, and 2) a hardware-based QoS mechanism that dynamically manages the execution of approximable branch instructions to prevent undesirable QoS degradation. We evaluate the proposed idea on an in-order pipeline processor using a software simulator. Experiments show that GBA can reduce the total execution time by more than 15 % while preserving the QoS of the DTE algorithm in the best-case scenario with a slight modification to the hardware.