Computer Science > Hardware Architecture
[Submitted on 29 Jun 2024]
Title:A Quality-Aware Voltage Overscaling Framework to Improve the Energy Efficiency and Lifetime of TPUs based on Statistical Error Modeling
View PDFAbstract:Deep neural networks (DNNs) are a type of artificial intelligence models that are inspired by the structure and function of the human brain, designed to process and learn from large amounts of data, making them particularly well-suited for tasks such as image and speech recognition. However, applications of DNNs are experiencing emerging growth due to the deployment of specialized accelerators such as the Google Tensor Processing Units (TPUs). In large-scale deployments, the energy efficiency of such accelerators may become a critical concern. In the voltage overscaling (VOS) technique, the operating voltage of the system is scaled down beyond the nominal operating voltage, which increases the energy efficiency and lifetime of digital circuits. The VOS technique is usually performed without changing the frequency resulting in timing errors. However, some applications such as multimedia processing, including DNNs, have intrinsic resilience against errors and noise. In this paper, we exploit the inherent resilience of DNNs to propose a quality-aware voltage overscaling framework for TPUs, named X-TPU, which offers higher energy efficiency and lifetime compared to conventional TPUs. The X-TPU framework is composed of two main parts, a modified TPU architecture that supports a runtime voltage overscaling, and a statistical error modeling-based algorithm to determine the voltage of neurons such that the output quality is retained above a given user-defined quality threshold. We synthesized a single-neuron architecture using a 15-nm FinFET technology under various operating voltage levels. Then, we extracted different statistical error models for a neuron corresponding to those voltage levels. Using these models and the proposed algorithm, we determined the appropriate voltage of each neuron. Results show that running a DNN on X-TPU can achieve 32% energy saving for only 0.6% accuracy loss.
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.