research-article

Open access

ISA mapper: a compute and hardware agnostic deep learning compiler

Authors:

Matthew Sotoudeh,

Michael Anderson,

Evangelos Georganas,

Alexander Heinecke,

Jason KnightAuthors Info & Claims

CF '19: Proceedings of the 16th ACM International Conference on Computing Frontiers

Pages 164 - 173

https://doi.org/10.1145/3310273.3321559

Published: 30 April 2019 Publication History

Abstract

Domain specific accelerators present new challenges for code generation onto novel instruction sets, communication fabrics, and memory architectures. We introduce a shared intermediate representation to describe both deep learning programs and hardware capabilities, then formulate and apply instruction mapping to determine how a computation can be performed on a hardware system. Our scheduler chooses a specific mapping and determines data movement and computation order.

With this system, we demonstrate automated extraction of matrix multiplication kernels from recent deep learning operations. We demonstrate 2--5X better performance on GEMM and GRU execution versus state-of-the-art on new hardware and up to 85% of state-of-the-art performance on existing hardware.

References

[1]

Alfred V Aho, Mahadevan Ganapathi, and Steven WK Tjiang. 1989. Code generation using tree matching and dynamic programming. ACM Transactions on Programming Languages and Systems (TOPLAS) 11, 4 (1989), 491--516.

Digital Library

[2]

Baidu. 2017. DeepBench: Benchmarking Deep Learning operations on different hardware. https://github.com/baidu-research/DeepBench

[3]

Tianqi Chen, Thierry Moreau, Ziheng Jiang, Haichen Shen, Eddie Yan, Leyuan Wang, Yuwei Hu, Luis Ceze, Carlos Guestrin, and Arvind Krishnamurthy. 2018. TVM: End-to-End Optimization Stack for Deep Learning. arXiv preprint arXiv:1802.04799 (2018).

[4]

Tianqi Chen, Lianmin Zheng, Eddie Yan, Ziheng Jiang, Thierry Moreau, Luis Ceze, Carlos Guestrin, and Arvind Krishnamurthy. 2018. Learning to Optimize Tensor Programs. arXiv preprint arXiv:1805.08166 (2018).

Digital Library

[5]

Kyunghyun Cho et al. 2014. On the Properties of Neural Machine Translation: EncoderâĂŞDecoder Approaches. Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation (2014).

[6]

Luigi P. Cordella, Pasquale Foggia, Carlo Sansone, and Mario Vento. 2004. A (sub)graph isomorphism algorithm for matching large graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence 26, 10 (2004), 1367--1372.

Digital Library

[7]

Scott Cyphers, Arjun K Bansal, Anahita Bhiwandiwalla, Jayaram Bobba, Matthew Brookhart, Avijit Chakraborty, Will Constable, Christian Convey, Leona Cook, Omar Kanawi, et al. 2018. Intel nGraph: An Intermediate Representation, Compiler, and Executor for Deep Learning. arXiv preprint arXiv:1801.08058 (2018).

[8]

James Demmel and Grace Dinh. 2018. Communication-Optimal Convolutional NeuralNets. CoRR abs/1802.06905 (2018). arXiv:1802.06905 http://arxiv.org/abs/1802.06905

[9]

E. Georganas, S. Avancha, K. Banerjee, D. Kalamkar, G. Henry, H. Pabst, and A. Heinecke. 2018. Anatomy Of High-Performance Deep Learning Convolutions On SIMD Architectures. ArXiv e-prints (Aug. 2018). arXiv:cs.DC/1808.05567

Digital Library

[10]

K. Keutzer. 1987. DAGON: Technology Binding and Local Optimization by DAG Matching. In Proceedings of the 24th ACM/IEEE Design Automation Conference (DAC '87). ACM, New York, NY, USA, 341--347.

Digital Library

[11]

Chris Lattner and Vikram Sadanand Adve. 2004. LLVM: a compilation framework for lifelong program analysis & transformation. In Proceedings of the international symposium on Code generation and optimization. 75--86.

Digital Library

[12]

Chris Leary and Todd Wang. 2017. XLA: TensorFlow, compiled! TensorFlow Dev Summit.

[13]

Jaroslav Nešetril and P Ossona de Mendez. 2012. Sparsity: Graphs, Structures, and Algorithms, volume 28 of Algorithms and Combinatorics.

[14]

Jonathan Ragan-Kelley, Connelly Barnes, Andrew Adams, Sylvain Paris, Frédo Durand, and Saman Amarasinghe. 2013. Halide: a language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines. ACM SIGPLAN Notices 48, 6 (2013), 519--530.

Digital Library

[15]

Laurent Sifre and Stéphane Mallat. 2014. Rigid-motion scattering for image classification. Ph.D. Dissertation. Citeseer.

[16]

Nicolas Vasilache, Oleksandr Zinenko, Theodoros Theodoridis, Priya Goyal, Zachary DeVito, William S. Moses, Sven Verdoolaege, Andrew Adams, and Albert Cohen. {n. d.}. Tensor Comprehensions: Framework-Agnostic High-Performance Machine Learning Abstractions. arXiv preprint arXiv:1802.04730 ({n. d.}).

[17]

Vertex.ai. 2017. Announcing PlaidML: Open Source Deep Learning for Every Platform. http://vertex.ai/blog/announcing-plaidml

Cited By

Xie AHu YWang YLi ZGao YCheng Z(2025)GTA: Generating high-performance tensorized program with dual-task schedulingJournal of Systems Architecture10.1016/j.sysarc.2025.103359160(103359)Online publication date: Mar-2025
https://doi.org/10.1016/j.sysarc.2025.103359
Zheng SChen RWei AJin YHan QLu LWu BLi XYan SLiang YSalapura VZahran MChong FTang L(2022)AMOSProceedings of the 49th Annual International Symposium on Computer Architecture10.1145/3470496.3527440(874-887)Online publication date: 18-Jun-2022
https://dl.acm.org/doi/10.1145/3470496.3527440
Man XZhu JSong GYin SWei SLiu LSalapura VZahran MChong FTang L(2022)CaSMapProceedings of the 49th Annual International Symposium on Computer Architecture10.1145/3470496.3527426(259-273)Online publication date: 18-Jun-2022
https://dl.acm.org/doi/10.1145/3470496.3527426
Show More Cited By

Recommendations

Multithreading extension for Thumb ISA and decoder support
EHAC'06: Proceedings of the 5th WSEAS International Conference on Electronics, Hardware, Wireless and Optical Communications

Dual width instruction set embedded processors such as ARM provide 16-bit instruction set in addition to the 32-bit instructions set for lower energy and memory cost. The combination of hardware multithreading technique with the 16-bit code design can ...
ISA Wars: Understanding the Relevance of ISA being RISC or CISC to Performance, Power, and Energy on Modern Architectures

RISC versus CISC wars raged in the 1980s when chip area and processor design complexity were the primary constraints and desktops and servers exclusively dominated the computing landscape. Today, energy and power are the primary design constraints and ...
A cycle-approximate, mixed-ISA simulator for the KAHRISMA architecture
DATE '12: Proceedings of the Conference on Design, Automation and Test in Europe

Processor architectures that are capable to reconfigure their instruction set and instruction format dynamically at run time offer a new flexibility exploiting instruction level parallelism vs. thread level parallelism. Based on the characteristics of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CF '19: Proceedings of the 16th ACM International Conference on Computing Frontiers

April 2019

414 pages

ISBN:9781450366854

DOI:10.1145/3310273

General Chairs:
Francesca Palumbo
Universit`a degli Studi di Sassari, IT
,
Michela Becchi
North Carolina State University
,
Program Chairs:
Martin Schulz
Technical University of Munich, DE
,
Kento Sato
RIKEN R-CCS, JP

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMICRO: ACM Special Interest Group on Microarchitectural Research and Processing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 April 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Conference

CF '19

Sponsor:

SIGMICRO

CF '19: Computing Frontiers Conference

April 30 - May 2, 2019

Alghero, Italy

Acceptance Rates

Overall Acceptance Rate 273 of 785 submissions, 35%

Upcoming Conference

CF '25

Sponsor:
sigmicro

22nd ACM International Conference on Computing Frontiers

May 28 - 30, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
589
Total Downloads

Downloads (Last 12 months)108
Downloads (Last 6 weeks)13

Reflects downloads up to 17 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Xie AHu YWang YLi ZGao YCheng Z(2025)GTA: Generating high-performance tensorized program with dual-task schedulingJournal of Systems Architecture10.1016/j.sysarc.2025.103359160(103359)Online publication date: Mar-2025
https://doi.org/10.1016/j.sysarc.2025.103359
Zheng SChen RWei AJin YHan QLu LWu BLi XYan SLiang YSalapura VZahran MChong FTang L(2022)AMOSProceedings of the 49th Annual International Symposium on Computer Architecture10.1145/3470496.3527440(874-887)Online publication date: 18-Jun-2022
https://dl.acm.org/doi/10.1145/3470496.3527440
Man XZhu JSong GYin SWei SLiu LSalapura VZahran MChong FTang L(2022)CaSMapProceedings of the 49th Annual International Symposium on Computer Architecture10.1145/3470496.3527426(259-273)Online publication date: 18-Jun-2022
https://dl.acm.org/doi/10.1145/3470496.3527426
Rieber DAcosta AFröning H(2021)Joint Program and Layout Transformations to Enable Convolutional Operators on Specialized Hardware Based on Constraint ProgrammingACM Transactions on Architecture and Code Optimization10.1145/348792219:1(1-26)Online publication date: 6-Dec-2021
https://dl.acm.org/doi/10.1145/3487922
Rieber DFröning H(2021)Search Space Complexity of Iteration Domain Based Instruction Embedding for Deep Learning AcceleratorsIoT Streams for Data-Driven Predictive Maintenance and IoT, Edge, and Mobile for Embedded Machine Learning10.1007/978-3-030-66770-2_16(213-228)Online publication date: 10-Jan-2021
https://doi.org/10.1007/978-3-030-66770-2_16
Sotoudeh MThakur AKell SVerna D(2020)Analogy-making as a Core primitive in the software engineering toolboxProceedings of the 2020 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software10.1145/3426428.3426918(101-121)Online publication date: 18-Nov-2020
https://dl.acm.org/doi/10.1145/3426428.3426918
Kim RKim GKim HYoon GYoo H(2020)A Method for Optimizing Deep Learning Object Detection in Edge Computing2020 International Conference on Information and Communication Technology Convergence (ICTC)10.1109/ICTC49870.2020.9289529(1164-1167)Online publication date: 21-Oct-2020
https://doi.org/10.1109/ICTC49870.2020.9289529

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten