research-article

The Image Calculator: 10x Faster Image-AI Inference by Replacing JPEG with Self-designing Storage Format

Authors:

Stratos IdreosAuthors Info & Claims

Proceedings of the ACM on Management of Data, Volume 2, Issue 1

Article No.: 52, Pages 1 - 31

https://doi.org/10.1145/3639307

Published: 26 March 2024 Publication History

Abstract

Numerous applications today rely on artificial intelligence over images. Image AI is, however, extremely expensive. In particular, the inference cost of image AI dominates the end-to-end cost. We observe that the image storage format lies at the root of the problem. Images today are predominantly stored in JPEG format. JPEG is a storage format designed for the human eye; it maximally compresses images without distorting the components of an image that are visible to the human eye. However, our observation is that during image AI, images are "seen'' by algorithms, not humans. In addition, every AI application is different regarding which data components of the images are the most relevant.

We present the Image Calculator, a self-designing image storage format that adapts to the given AI task, i.e., the specific neural network, the dataset, and the applications' specific accuracy, inference time, and storage requirements. Contrary to the state-of-the-art, the Image Calculator does not use a fixed storage format like JPEG. Instead, it designs and constructs a new storage format tailored to the context. It does so by constructing a massive design space of candidate storage formats from first principles, within which it searches efficiently using composite performance models (inference time, accuracy, storage). This way, it leverages the given AI task's unique characteristics to compress the data maximally. We evaluate the Image Calculator across a diverse set of data, image analysis tasks, AI models, and hardware. We show that the Image Calculator can generate image storage formats that reduce inference time by up to 14.2x and storage by up to 8.2x with a minimal loss in accuracy or gain, compared to JPEG and its state-of-the-art variants.

References

[1]

Muhammad Adnan, Yassaman Ebrahimzadeh Maboud, Divya Mahajan, and Prashant J. Nair. 2021. Accelerating Recommendation System Training by Leveraging Popular Choices. Proc. VLDB Endow., Vol. 15, 1 (2021), 127--140.

Digital Library

[2]

Gbeminiyi Ajayi. 2018. Multi-class Weather Dataset for Image Classification. Mendeley Data, Vol. 1 (2018). https://doi.org/10.17632/4drtyfjtfy.1

[3]

Baba Fakruddin Ali B H and Prakash Ramachandran. 2022. Compressive Domain Deep CNN for Image Classification and Performance Improvement Using Genetic Algorithm-Based Sensing Mask Learning. Applied Sciences, Vol. 12, 14 (2022).

[4]

AWS. 2019. Amazon EC2 Update -- Inf1 Instances with AWS Inferentia Chips for High Performance Cost-Effective Inferencing. https://aws.amazon.com/blogs/aws/amazon-ec2-update-inf1-instances-with-aws-inferentia-chips-for-high-performance-cost-effective-inferencing/ Accessed on May 16, 2023.

[5]

Yuanchao Bai, Xu Yang, Xianming Liu, Junjun Jiang, Yaowei Wang, Xiangyang Ji, and Wen Gao. 2022. Towards End-to-End Image Compression and Analysis with Transformers. In AAAI. 104--112.

[6]

Favyen Bastani, Songtao He, Arjun Balasingam, Karthik Gopalakrishnan, Mohammad Alizadeh, Hari Balakrishnan, Michael Cafarella, Tim Kraska, and Sam Madden. 2020a. MIRIS: Fast Object Track Queries in Video. In SIGMOD. 1907--1921.

[7]

Favyen Bastani, Songtao He, Arjun Balasingam, Karthik Gopalakrishnan, Mohammad Alizadeh, Hari Balakrishnan, Michael Cafarella, Tim Kraska, and Sam Madden. 2020b. MIRIS: Fast Object Track Queries in Video. In SIGMOD. 1907--1921.

[8]

Vasudev Bhaskaran and Konstantinos Konstantinides. 1995. Image and Video Compression Standards. Springer.

[9]

Imene Bouderbal, Abdenour Amamra, M. El-Arbi Djebbar, and Mohamed Akrem Benatia. 2022. Towards SSD Accelerating for Embedded Environments: A Compressive Sensing Based Approach. Journal of Real-Time Image Processing, Vol. 19, 6 (2022), 1199--1210.

Digital Library

[10]

Matthew Butrovich, Wan Shen Lim, Lin Ma, John Rollinson, William Zhang, Yu Xia, and Andrew Pavlo. 2022. Tastes Great! Less Filling! High Performance and Accurate Training Data Collection for Self-Driving Database Management Systems. In SIGMOD. 617--630.

[11]

Han Cai, Chuang Gan, Tianzhe Wang, Zhekai Zhang, and Song Han. 2020. Once-for-All: Train One Network and Specialize it for Efficient Deployment. In ICLR.

[12]

Han Cai, Ji Lin, Yujun Lin, Zhijian Liu, Haotian Tang, Hanrui Wang, Ligeng Zhu, and Song Han. 2022. Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications. ACM Transactions Design Automation of Electronic Systems, Vol. 27, 3 (2022).

Digital Library

[13]

Han Cai, Ligeng Zhu, and Song Han. 2019. ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware. In ICLR.

[14]

Jiashen Cao, Karan Sarkar, Ramyad Hadidi, Joy Arulraj, and Hyesoon Kim. 2022. FiGO: Fine-Grained Query Optimization in Video Analytics. In SIGMOD. 559--572.

[15]

Lahiru D. Chamain, Siyu Qi, and Zhi Ding. 2021a. An End-to-End Learning Architecture for Efficient Image Encoding and Deep Learning. In 29th European Signal Processing Conference (EUSIPCO). 691--695.

[16]

Lahiru D. Chamain, Siyu Qi, and Zhi Ding. 2022. End-to-End Image Classification and Compression With Variational Autoencoders. IEEE Internet of Things Journal, Vol. 9, 21 (2022), 21916--21931.

[17]

Lahiru D. Chamain, Fabien Racapé, Jean Bégaint, Akshay Pushparaja, and Simon Feltman. 2021b. End-to-End Optimized Image Compression for Machines, A Study. In 31st Data Compression Conference (DCC). 163--172.

[18]

Subarna Chatterjee, Meena Jagadeesan, Wilson Qin, and Stratos Idreos. 2022. Cosine: A Cloud-Cost Optimized Self-Designing Key-Value Storage Engine. Proc. VLDB Endow., Vol. 15, 1 (2022), 112--126.

Digital Library

[19]

Subarna Chatterjee, Mark Pekala, Lev Kruglyak, and Stratos Idreos. 2024. Limousine: Blending Learned and Classical Indexes to Self-Design Larger-than-Memory Cloud Storage Engines. Proc. ACM Manag. Data (2024).

Digital Library

[20]

Sien Chen, Jian Jin, Lili Meng, Weisi Lin, Zhuo Chen, Tsui-Shan Chang, Zhengguang Li, and Huaxiang Zhang. 2021. A New Image Codec Paradigm for Human and Machine Uses. CoRR, Vol. abs/2112.10071 (2021).

[21]

Hyomin Choi and Ivan V. Bajic. 2018. Deep Feature Compression for Collaborative Object Detection. In 25th IEEE International Conference on Image Processing (ICIP). 3743--3747.

[22]

Hyomin Choi and Ivan V. Bajic. 2021. Latent-Space Scalability for Multi-Task Collaborative Intelligence. In 28th IEEE International Conference on Image Processing (ICIP). 3562--3566.

[23]

Hyomin Choi and Ivan V. Bajic. 2022. Scalable Image Coding for Humans and Machines. IEEE Transactions on Image Processing, Vol. 31, 1 (2022), 2739--2754.

Digital Library

[24]

Hyomin Choi, Robert A. Cohen, and Ivan V. Bajic. 2020. Back-And-Forth Prediction for Deep Tensor Compression. In 45th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 4467--4471.

[25]

Jungwook Choi, Zhuo Wang, Swagath Venkataramani, Pierce I-Jen Chuang, Vijayalakshmi Srinivasan, and Kailash Gopalakrishnan. 2018. PACT: Parameterized Clipping Activation for Quantized Neural Networks. CoRR, Vol. abs/1805.06085 (2018).

[26]

Pramod Chunduri, Jaeho Bang, Yao Lu, and Joy Arulraj. 2022. Zeus: Efficiently Localizing Actions in Videos Using Reinforcement Learning. In SIGMOD. 545--558.

[27]

Felipe Codevilla, Jean Gabriel Simard, Ross Goroshin, and Chris Pal. 2021. Learned Image Compression for Machine Perception. CoRR, Vol. abs/2111.02249 (2021).

[28]

Nilaksh Das, Sanya Chaba, Renzhi Wu, Sakshi Gandhi, Duen Horng Chau, and Xu Chu. 2020. GOGGLES: Automatic Image Labeling with Affinity Coding. In SIGMOD. 1717--1732.

[29]

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. ImageNet: A Large-scale Hierarchical Image Database. In CVPR. 248--255.

[30]

Yingpeng Deng and Lina J. Karam. 2021. Learning-based Compression for Material and Texture Recognition. CoRR, Vol. abs/2104.10065 (2021).

[31]

Radosvet Desislavov, Fernando Martínez-Plumed, and José Hernández-Orallo. 2023. Trends in AI Inference Energy Consumption: Beyond the Performance-vs-Parameter Laws of Deep Learning. Sustainable Computing: Informatics and Systems, Vol. 38 (2023), 100--857.

[32]

Jialin Ding, Vikram Nathan, Mohammad Alizadeh, and Tim Kraska. 2020. Tsunami: A Learned Multi-Dimensional Index for Correlated Data and Skewed Workloads. Proc. VLDB Endow., Vol. 14, 2 (2020), 74--86.

Digital Library

[33]

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In ICLR.

[34]

Iddo Drori and Joaquin Vanschoren. 2021. AAAI 2021 Meta Learning Tutorial. https://sites.google.com/mit.edu/aaai2021metalearningtutorial Accessed on May 16, 2023.

[35]

Lingyu Du and Guohao Lan. 2022. FreeGaze: Resource-efficient Gaze Estimation via Frequency Domain Contrastive Learning. CoRR, Vol. abs/2209.06692 (2022).

[36]

Max Ehrlich. 2020. TorchJPEG. https://torchjpeg.readthedocs.io/en/latest/ Accessed on May 16, 2023.

[37]

Max Ehrlich, Larry Davis, Ser-Nam Lim, and Abhinav Shrivastava. 2020. Quantization Guided JPEG Artifact Correction. ECCV (2020).

[38]

Jingzhi Fang, Yanyan Shen, Yue Wang, and Lei Chen. 2021. ETO: Accelerating Optimization of DNN Operators by High-Performance Tensor Program Reuse. Proc. VLDB Endow., Vol. 15, 2 (2021), 183--195.

Digital Library

[39]

Forbes. 2019. Google Cloud Doubles Down On NVIDIA GPUs For Inference. https://www.forbes.com/sites/moorinsights/2019/05/09/google-cloud-doubles-down-on-nvidia-gpus-for-inference/ Accessed on May 16, 2023.

[40]

Dan Fu and Gabriel Guimaraes. 2016. Using Compression to Speed Up Image Classification in Artificial Neural Networks. (2016). https://www.danfu.org/files/CompressionImageClassification.pdf

[41]

Fangeheng Fu, Jiawei Jiang, Yingxia Shao, and Bin Cui. 2019. An Experimental Evaluation of Large Scale GBDT Systems. Proc. VLDB Endow., Vol. 12, 11 (2019), 1357--1370.

Digital Library

[42]

Fangcheng Fu, Xupeng Miao, Jiawei Jiang, Huanran Xue, and Bin Cui. 2022. Towards Communication-Efficient Vertical Federated Learning Training via Cache-Enabled Local Updates. Proc. VLDB Endow., Vol. 15, 10 (2022), 2111--2120.

Digital Library

[43]

Shay Gershtein, Tova Milo, Slava Novgorodov, and Kathy Razmadze. 2022. Classifier Construction Under Budget Constraints. In SIGMOD. 1160--1174.

[44]

Stefan Grafberger, Paul Groth, and Sebastian Schelter. 2023. Automating and Optimizing Data-Centric What-If Analyses on Native Machine Learning Pipelines. Proc. ACM Manag. Data, Vol. 1, 2 (2023).

Digital Library

[45]

Lionel Gueguen, Alex Sergeev, Ben Kadlec, Rosanne Liu, and Jason Yosinski. 2018. Faster Neural Networks Straight from JPEG. In NeurIPS. 3937--3948.

[46]

Isabelle Guyon, Jan N. van Rijn, Sé bastien Treguer, and Joaquin Vanschoren (Eds.). 2021. AAAI Workshop on Meta-Learning and MetaDL Challenge, MetaDL@AAAI 2021, virtual, February 9, 2021. Proceedings of Machine Learning Research, Vol. 140. PMLR. https://proceedings.mlr.press/v140/

[47]

Song Han, Huizi Mao, and William J. Dally. 2016. Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding. In ICLR.

[48]

Song Han, Jeff Pool, John Tran, and William J. Dally. 2015. Learning both Weights and Connections for Efficient Neural Network. In NuerIPS. 1135--1143.

[49]

Brandon Haynes, Maureen Daum, Dong He, Amrita Mazumdar, Magdalena Balazinska, Alvin Cheung, and Luis Ceze. 2021. VSS: A Storage System for Video Analytics. In SIGMOD. 685--696.

[50]

Kaiming He, Georgia Gkioxari, Piotr Dollá r, and Ross B. Girshick. 2017a. Mask R-CNN. In ICCV. 2980--2988.

[51]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In CVPR. 770--778.

[52]

Wenjia He, Michael R. Anderson, Maxwell Strome, and Michael Cafarella. 2020. A Method for Optimizing Opaque Filter Queries. In SIGMOD. 1257--1272.

[53]

Wenjia He and Michael Cafarella. 2022. Controlled Intentional Degradation in Analytical Video Systems. In SIGMOD. 2105--2119.

[54]

Yihui He, Ji Lin, Zhijian Liu, Hanrui Wang, Li-Jia Li, and Song Han. 2018. AMC: AutoML for Model Compression and Acceleration on Mobile Devices. In ECCV, Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss (Eds.). 815--832.

[55]

Yihui He, Xiangyu Zhang, and Jian Sun. 2017b. Channel Pruning for Accelerating Very Deep Neural Networks. In ICCV. 1398--1406.

[56]

Andrew Howard, Ruoming Pang, Hartwig Adam, Quoc V. Le, Mark Sandler, Bo Chen, Weijun Wang, Liang-Chieh Chen, Mingxing Tan, Grace Chu, Vijay Vasudevan, and Yukun Zhu. 2019. Searching for MobileNetV3. In ICCV. 1314--1324.

[57]

HPCwire. 2019. AWS to Offer Nvidia's T4 GPUs for AI Inferencing. https://www.hpcwire.com/2019/03/19/aws-upgrades-its-gpu-backed-ai-inference-platform/ Accessed on May 16, 2023.

[58]

Bo Hu, Peizhen Guo, and Wenjun Hu. 2022. Video-Zilla: An Indexing Layer for Large-Scale Video Analytics. In SIGMOD. 1905--1919.

[59]

Yueyu Hu, Shuai Yang, Wenhan Yang, Ling-Yu Duan, and Jiaying Liu. 2020. Towards Coding For Human And Machine Vision: A Scalable Image Coding Approach. In 21st IEEE International Conference on Multimedia and Expo (ICME). 1--6.

[60]

Yueyu Hu, Wenhan Yang, Haofeng Huang, and Jiaying Liu. 2021. Revisit Visual Representation in Analytics Taxonomy: A Compression Perspective. CoRR, Vol. abs/2106.08512 (2021).

[61]

Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. 2016. Binarized Neural Networks. In NeurIPS.

[62]

Stratos Idreos, Niv Dayan, Wilson Qin, Mali Akmanalp, Sophie Hilgard, Andrew Ross, James Lennon, Varun Jain, Harshita Gupta, David Li, and Zichen Zhu. 2019a. Design Continuums and the Path Toward Self-Designing Key-Value Stores that Know and Learn. In CIDR.

[63]

Stratos Idreos and Tim Kraska. 2019. From Auto-Tuning One Size Fits All to Self-Designed and Learned Data-Intensive Systems. In SIGMOD. 2054--2059.

[64]

Stratos Idreos, Kostas Zoumpatianos, Manos Athanassoulis, Niv Dayan, Brian Hentschel, Michael S. Kester, Demi Guo, Lukas M. Maas, Wilson Qin, Abdul Wasay, and Yiyou Sun. 2018a. The Periodic Table of Data Structures. IEEE Data Eng. Bull., Vol. 41, 3 (2018), 64--75.

[65]

Stratos Idreos, Kostas Zoumpatianos, Subarna Chatterjee, Wilson Qin, Abdul Wasay, Brian Hentschel, Mike S. Kester, Niv Dayan, Demi Guo, Minseo Kang, and Yiyou Sun. 2019b. Learning Data Structure Alchemy. IEEE Data Eng. Bull., Vol. 42, 2 (2019), 47--58.

[66]

Stratos Idreos, Konstantinos Zoumpatianos, Brian Hentschel, Michael S. Kester, and Demi Guo. 2018b. The Data Calculator: Data Structure Design and Cost Synthesis From First Principles, and Learned Cost Models. In SIGMOD. 535--550.

[67]

iMerit. 2021. Top 13 Machine Learning Image Classification Datasets. https://imerit.net/blog/top-13-machine-learning-image-classification-datasets-all-pbm/ Accessed on Jan. 02, 2023.

[68]

Alexander Isenko, Ruben Mayer, Jeffrey Jedele, and Hans-Arno Jacobsen. 2022. Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines. In SIGMOD. 1825--1839.

[69]

Jiawei Jiang, Shaoduo Gan, Yue Liu, Fanlin Wang, Gustavo Alonso, Ana Klimovic, Ankit Singla, Wentao Wu, and Ce Zhang. 2021. Towards Demystifying Serverless Machine Learning Training. In SIGMOD. 857--871.

[70]

Wenbin Jiao, Xuemin Cheng, Yao Hu, Qun Hao, and Hongsheng Bi. 2022. Image Recognition Based on Compressive Imaging and Optimal Feature Selection. IEEE Photonics Journal, Vol. 14, 2 (2022), 1--12.

[71]

Sian Jin, Chengming Zhang, Xintong Jiang, Yunhe Feng, Hui Guan, Guanpeng Li, Shuaiwen Leon Song, and Dingwen Tao. 2021. COMET: A Novel Memory-Efficient Deep Learning Training Framework by Using Error-Bounded Lossy Compression. Proc. VLDB Endow., Vol. 15, 4 (2021), 886--899.

Digital Library

[72]

Kaggle. 2023 a. Cigarette Filter Detection Task. https://www.kaggle.com/datasets/estebanpacanchique/cigarette-butt Accessed on Sept. 20, 2023.

[73]

Kaggle. 2023 b. Micro-controller Detection and Segmentation Task. https://www.kaggle.com/datasets/tannergi/microcontroller-segmentation Accessed on Sept. 20, 2023.

[74]

Daniel Kang, Peter Bailis, and Matei Zaharia. 2019. BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video Analytics. Proc. VLDB Endow., Vol. 13, 4 (2019), 533--546.

Digital Library

[75]

Daniel Kang, Ankit Mathur, Teja Veeramacheneni, Peter Bailis, and Matei Zaharia. 2020. Jointly Optimizing Preprocessing and Inference for DNN-Based Visual Analytics. Proc. VLDB Endow., Vol. 14, 2 (2020), 87--100.

Digital Library

[76]

Barrie Kersbergen, Olivier Sprangers, and Sebastian Schelter. 2022. Serenade - Low-Latency Session-Based Recommendation in e-Commerce at Scale. In SIGMOD. 150--159.

[77]

Michael S. Kester, Manos Athanassoulis, and Stratos Idreos. 2017. Access Path Selection in Main-Memory Optimized Data Systems: Should I Scan or Should I Probe?. In SIGMOD. 715--730.

[78]

Alexandros Koliousis, Pijika Watcharapichat, Matthias Weidlich, Luo Mai, Paolo Costa, and Peter Pietzuch. 2019. Crossbow: Scaling Deep Learning with Small Batch Sizes on Multi-GPU Servers. Proc. VLDB Endow., Vol. 12, 11 (2019), 1399--1412.

Digital Library

[79]

Tim Kraska. 2021. Towards Instance-Optimized Data Systems. Proc. VLDB Endow., Vol. 14, 12 (2021), 3222--3232.

Digital Library

[80]

Tim Kraska, Mohammad Alizadeh, Alex Beutel, Ed H. Chi, Ani Kristo, Guillaume Leclerc, Samuel Madden, Hongzi Mao, and Vikram Nathan. 2019. SageDB: A Learned Database System. In CIDR.

[81]

Tim Kraska, Alex Beutel, Ed H. Chi, Jeffrey Dean, and Neoklis Polyzotis. 2018. The Case for Learned Index Structures. In SIGMOD. 489--504.

Digital Library

[82]

Ani Kristo, Kapil Vaidya, Ugur cCetintemel, Sanchit Misra, and Tim Kraska. 2020. The Case for a Learned Sorting Algorithm. In SIGMOD. 1001--1016.

[83]

Nikolina Kubiak and Simon Hadfield. 2021. TACTIC: Joint Rate-Distortion-Accuracy Optimisation for Low Bitrate Compression. CoRR, Vol. abs/2109.10658 (2021).

[84]

Andreas Kunft, Asterios Katsifodimos, Sebastian Schelter, Sebastian Breß, Tilmann Rabl, and Volker Markl. 2019. An Intermediate Representation for Optimizing Machine Learning Pipelines. Proc. VLDB Endow., Vol. 12, 11 (2019), 1553--1567.

Digital Library

[85]

Andrew Lavin and Scott Gray. 2016. Fast Algorithms for Convolutional Neural Networks. In CVPR.

[86]

Nam Le, Honglei Zhang, Francesco Cricri, Ramin Ghaznavi-Youvalari, and Esa Rahtu. 2021. Image Coding For Machines: An End-To-End Learned Approach. In 46th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 1590--1594.

[87]

Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. 2017. Pruning Filters for Efficient ConvNets. In ICLR.

[88]

Pengfei Li, Hua Lu, Qian Zheng, Long Yang, and Gang Pan. 2020. LISA: A Learned Index Structure for Spatial Data. In SIGMOD. 2119--2133.

[89]

Yang Li, Yu Shen, Huaijun Jiang, Wentao Zhang, Jixiang Li, Ji Liu, Ce Zhang, and Bin Cui. 2022. Hyper-Tune: Towards Efficient Hyper-Parameter Tuning at Scale. Proc. VLDB Endow., Vol. 15, 6 (2022), 1256--1265.

Digital Library

[90]

Yang Li, Yu Shen, Wentao Zhang, Jiawei Jiang, Bolin Ding, Yaliang Li, Jingren Zhou, Zhi Yang, Wentao Wu, Ce Zhang, and Bin Cui. 2021. VolcanoML: Speeding up End-to-End AutoML via Scalable Search Space Decomposition. Proc. VLDB Endow., Vol. 14, 11 (2021), 2167--2176.

Digital Library

[91]

Ji Lin, Wei-Ming Chen, Han Cai, Chuang Gan, and Song Han. 2021. Memory-efficient Patch-based Inference for Tiny Deep Learning. In NeurIPS. 2346--2358.

[92]

Ji Lin, Wei-Ming Chen, Yujun Lin, John Cohn, Chuang Gan, and Song Han. 2020. MCUNet: Tiny Deep Learning on IoT Devices. In NeurIPS.

[93]

Ji Lin, Chuang Gan, and Song Han. 2019. Defensive Quantization: When Efficiency Meets Robustness. In ICLR.

[94]

Ji Lin, Yongming Rao, Jiwen Lu, and Jie Zhou. 2017. Runtime Neural Pruning. In NeurIPS.

[95]

Jinming Liu, Heming Sun, and Jiro Katto. 2021b. Learning in Compressed Domain for Faster Machine Vision Tasks. In 36th International Conference on Visual Communications and Image Processing (VCIP). 1--5.

[96]

Jinming Liu, Heming Sun, and Jiro Katto. 2022b. Improving Multiple Machine Vision Tasks in the Compressed Domain. In 26th International Conference on Pattern Recognition (ICPR). 331--337.

[97]

Jinming Liu, Heming Sun, and Jiro Katto. 2022c. Semantic Segmentation in Learned Compressed Domain. CoRR, Vol. abs/2209.01355 (2022).

[98]

Linfeng Liu, Tong Chen, Haojie Liu, Shiliang Pu, Li Wang, and Qiu Shen. 2022a. 2C?Net: Integrate Image Compression and Classification via Deep Neural Network. Multimedia Systems (2022).

[99]

Xingyu Liu, Jeff Pool, Song Han, and William J. Dally. 2018. Efficient Sparse-Winograd Convolutional Neural Networks. In ICLR.

[100]

Zhuang Liu, Jianguo Li, Zhiqiang Shen, Gao Huang, Shoumeng Yan, and Changshui Zhang. 2017. Learning Efficient Convolutional Networks through Network Slimming. In ICCV. 2755--2763.

[101]

Zihao Liu, Sicheng Li, Yen-kuang Chen, Tao Liu, Qi Liu, Xiaowei Xu, Yiyu Shi, and Wujie Wen. 2020. Orchestrating Medical Image Compression and Remote Segmentation Networks. In 23rd International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI). 406--416.

[102]

Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021a. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. In ICCV. IEEE, 9992--10002.

[103]

Zechun Liu, Haoyuan Mu, Xiangyu Zhang, Zichao Guo, Xin Yang, Kwang-Ting Cheng, and Jian Sun. 2019. MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning. In ICCV. 3295--3304.

[104]

Shao-Yuan Lo and Hsueh-Ming Hang. 2019. Exploring Semantic Segmentation on the DCT Representation. In 1st ACM International Conference on Multimedia in Asia (MMASIA). 1--6.

[105]

Guo Lu, Xingtong Ge, Tianxiong Zhong, Jing Geng, and Qiang Hu. 2022. Preprocessing Enhanced Image Compression for Machine Vision. CoRR, Vol. abs/2206.05650 (2022).

[106]

Shangyu Luo, Dimitrije Jankov, Binhang Yuan, and Chris Jermaine. 2021. Automatic Optimization of Matrix Implementations for Distributed Machine Learning and Linear Algebra. In SIGMOD. 1222--1234.

[107]

Kaihao Ma, Xiao Yan, Zhenkun Cai, Yuzhen Huang, Yidi Wu, and James Cheng. 2023. FEC: Efficient Deep Recommendation Model Training with Flexible Embedding Communication. Proc. ACM Manag. Data, Vol. 1, 2 (2023), 21 pages.

Digital Library

[108]

Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun. 2018. ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design. In ECCV. 122--138.

[109]

Samuel Madden, Jialin Ding, Tim Kraska, Sivaprasad Sudhir, David Cohen, Timothy G. Mattson, and Nesime Tatbul. 2022. Self-Organizing Data Containers. In CIDR.

[110]

Ryan Marcus, Parimarjan Negi, Hongzi Mao, Nesime Tatbul, Mohammad Alizadeh, and Tim Kraska. 2021. Bao: Making Learned Query Optimization Practical. In SIGMOD. 1275--1288.

[111]

Ryan Marcus, Parimarjan Negi, Hongzi Mao, Chi Zhang, Mohammad Alizadeh, Tim Kraska, Olga Papaemmanouil, and Nesime Tatbul. 2019. Neo: A Learned Query Optimizer. Proc. VLDB Endow., Vol. 12, 11 (2019), 1705--1718.

Digital Library

[112]

Yixin Mei, Fan Li, Li Li, and Zhu Li. 2021. Learn A Compression for Objection Detection - VAE With A Bridge. In 36th International Conference on Visual Communications and Image Processing (VCIP). 1--5.

[113]

Xupeng Miao, Xiaonan Nie, Yingxia Shao, Zhi Yang, Jiawei Jiang, Lingxiao Ma, and Bin Cui. 2021. Heterogeneity-Aware Distributed Machine Learning Training via Partial Reduce. In SIGMOD. 2262--2270.

[114]

Oscar Moll, Favyen Bastani, Sam Madden, Mike Stonebraker, Vijay Gadepally, and Tim Kraska. 2022a. ExSample: Efficient Searches on Video Repositories through Adaptive Sampling. In ICDE.

[115]

Oscar Moll, Manuel Favela, Samuel Madden, and Vijay Gadepally. 2022b. SeeSaw: Interactive Ad-hoc Search Over Image Databases. CoRR, Vol. abs/2208.06497 (2022).

[116]

Paul Mooney. 2018. Blood Cell Images. https://www.kaggle.com/datasets/paultimothymooney/blood-cells Accessed on May 16, 2023.

[117]

Supun Nakandala and Arun Kumar. 2022. Nautilus: An Optimized System for Deep Transfer Learning over Evolving Training Datasets. In SIGMOD. 506--520.

[118]

Supun Nakandala, Yuhao Zhang, and Arun Kumar. 2020. Cerebro: A Data System for Optimized Deep Learning Model Selection. Proc. VLDB Endow., Vol. 13, 12 (2020), 2159--2173.

Digital Library

[119]

Vikram Nathan, Jialin Ding, Mohammad Alizadeh, and Tim Kraska. 2020. Learning Multi-Dimensional Indexes. In SIGMOD. 985--1000.

[120]

Xiaonan Nie, Xupeng Miao, Zilong Wang, Zichao Yang, Jilong Xue, Lingxiao Ma, Gang Cao, and Bin Cui. 2023. FlexMoE: Scaling Large-Scale Sparse Pre-Trained Model Training via Dynamic Device Placement. Proc. ACM Manag. Data, Vol. 1, 1 (2023), 19 pages.

Digital Library

[121]

Kwanghyun Park, Karla Saur, Dalitso Banda, Rathijit Sen, Matteo Interlandi, and Konstantinos Karanasos. 2022. End-to-End Optimization of Machine Learning Prediction Queries. In SIGMOD. 587--601.

[122]

Neel Patwa, Nilesh Ahuja, Srinivasa Somayazulu, Omesh Tickoo, Srenivas Varadarajan, and Shashidhar Koolagudi. 2020. Semantic-Preserving Image Compression. In 27th IEEE International Conference on Image Processing (ICIP). 1281--1285.

[123]

Jingshu Peng, Zhao Chen, Yingxia Shao, Yanyan Shen, Lei Chen, and Jiannong Cao. 2022. Sancus: Staleness-Aware Communication-Avoiding Full-Graph Decentralized Training in Large-Scale Graph Neural Networks. Proc. VLDB Endow., Vol. 15, 9 (2022), 1937--1950.

Digital Library

[124]

Hieu Pham, Melody Guan, Barret Zoph, Quoc Le, and Jeff Dean. 2018. Efficient Neural Architecture Search via Parameters Sharing. In ICML. 4095--4104.

[125]

Arnab Phani, Lukas Erlbacher, and Matthias Boehm. 2022. UPLIFT: Parallelization Strategies for Feature Transformations in Machine Learning Workloads. Proc. VLDB Endow., Vol. 15, 11 (2022), 2929--2938.

Digital Library

[126]

Arnab Phani, Benjamin Rath, and Matthias Boehm. 2021. LIMA: Fine-Grained Lineage Tracing and Reuse in Machine Learning Systems. In SIGMOD. 1426--1439.

[127]

Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, and Ali Farhadi. 2016. XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks. In ECCV. 525--542.

[128]

Alexander Ratner, Stephen H. Bach, Henry Ehrenberg, Jason Fries, Sen Wu, and Christopher Ré. 2017. Snorkel: Rapid Training Data Creation with Weak Supervision. Proc. VLDB Endow., Vol. 11, 3 (2017), 269--282.

Digital Library

[129]

Luis Remis and Chaunté W. Lacewell. 2021. Using VDMS to Index and Search 100M Images. Proceedings of the VLDB Endowment, Vol. 14, 12 (2021), 3240--3252.

Digital Library

[130]

Shaoqing Ren, Kaiming He, Ross B. Girshick, and Jian Sun. 2015. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In NIPS. 91--99.

[131]

Alexander Renz-Wieland, Rainer Gemulla, Zoi Kaoudi, and Volker Markl. 2022. NuPS: A Parameter Server for Machine Learning with Non-Uniform Parameter Access. In SIGMOD. 481--495.

[132]

Alexander Renz-Wieland, Rainer Gemulla, Steffen Zeuch, and Volker Markl. 2020. Dynamic Parameter Allocation in Parameter Servers. Proc. VLDB Endow., Vol. 13, 12 (2020), 1877--1890.

Digital Library

[133]

Ties Robroek, Aaron Duane, Ehsan Yousefzadeh-Asl-Miandoab, and Pinar Tozun. 2023. Data Management and Visualization for Benchmarking Deep Learning Training Systems. In DEEM Workshop.

[134]

Samuel Felipe dos Santos, Nicu Sebe, and Jurandy Almeida. 2020. The Good, The Bad, and The Ugly: Neural Networks Straight From JPEG. In 27th IEEE International Conference on Image Processing (ICIP). 1896--1900.

[135]

Sebastian Schelter, Stefan Grafberger, and Ted Dunning. 2021. HedgeCut: Maintaining Randomised Trees for Low-Latency Machine Unlearning. In SIGMOD. 1545--1557.

[136]

Sebastian Schelter, Tammo Rukat, and Felix Biessmann. 2020. Learning to Validate the Predictions of Black Box Classifiers on Unseen Data. In SIGMOD. 1289--1299.

[137]

Saurabh Singh, Sami Abu-El-Haija, Nick Johnston, Johannes Ballé, Abhinav Shrivastava, and George Toderici. 2020. End-to-End Learning of Compressible Features. In 27th IEEE International Conference on Image Processing (ICIP). 3349--3353.

[138]

Utku Sirin and Stratos Idreos. 2024. The Image Calculator Technical Report. (2024).

[139]

Samuel L. Smith, Andrew Brock, Leonard Berrada, and Soham De. 2023. ConvNets Match Vision Transformers at Scale. CoRR, Vol. abs/2310.16764 (2023).

[140]

Simeng Sun, Tianyu He, and Zhibo Chen. 2021. Semantic Structured Image Coding Framework for Multiple Intelligent Applications. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 31, 9 (2021), 3631--3642.

[141]

Peter Symes. 1998. Video Compression: Fundamental Compression Techniques and an Overview of the JPEG and MPEG Compression Systems. McGraw-Hill.

[142]

Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, and Quoc V. Le. 2019. MnasNet: Platform-Aware Neural Architecture Search for Mobile. In CVPR. 2820--2828.

[143]

Mingxing Tan and Quoc V. Le. 2019. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. In ICML. 6105--6114.

[144]

Róbert Torfason, Fabian Mentzer, Eiríkur Ágústsson, Michael Tschannen, Radu Timofte, and Luc Van Gool. 2018. Towards Image Understanding from Deep Compression Without Decoding. In ICLR. 1--17.

[145]

Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan C. Bovik, and Yinxiao Li. 2022. MaxViT: Multi-axis Vision Transformer. In ECCV. 459--479.

[146]

Paroma Varma and Christopher Ré. 2018. Snuba: Automating Weak Supervision to Label Training Data. Proc. VLDB Endow., Vol. 12, 3 (2018), 223--236.

Digital Library

[147]

Xinchen Wan, Kaiqiang Xu, Xudong Liao, Yilun Jin, Kai Chen, and Xin Jin. 2023. Scalable and Efficient Full-Graph GNN Training for Large Graphs. Proc. ACM Manag. Data, Vol. 1, 2 (2023), 23 pages.

Digital Library

[148]

Kuan Wang, Zhijian Liu, Yujun Lin, Ji Lin, and Song Han. 2019b. HAQ: Hardware-Aware Automated Quantization With Mixed Precision. In CVPR. 8612--8620.

[149]

Shurun Wang, Zhao Wang, Shiqi Wang, and Yan Ye. 2021. End-to-End Compression Towards Machine Vision: Network Architecture Design and Optimization. IEEE Open Journal of Circuits and Systems, Vol. 2, 1 (2021), 675--685.

[150]

Tianzhe Wang, Kuan Wang, Han Cai, Ji Lin, Zhijian Liu, Hanrui Wang, Yujun Lin, and Song Han. 2020. APQ: Joint Search for Network Architecture, Pruning and Quantization Policy. In CVPR. 2075--2084.

[151]

Zeke Wang, Kaan Kara, Hantian Zhang, Gustavo Alonso, Onur Mutlu, and Ce Zhang. 2019a. Accelerating Generalized Linear Models with MLWeaving: A One-Size-Fits-All System for Any-Precision Learning. Proc. VLDB Endow., Vol. 12, 7 (2019), 807--821.

Digital Library

[152]

Zixi Wang, Fan Li, Jing Xu, and Pamela C. Cosman. 2022a. Human--Machine Interaction-Oriented Image Coding for Resource-Constrained Visual Monitoring in IoT. IEEE Internet of Things Journal, Vol. 9, 17 (2022), 16181--16195.

[153]

Zhenzhen Wang, Minghai Qin, and Yen-Kuang Chen. 2022b. Learning From the CNN-based Compressed Domain. In 22nd IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). 4000--4008.

[154]

Maurice Weber, Cedric Renggli, Helmut Grabner, and Ce Zhang. 2020. Observer Dependent Lossy Image Compression. In 8th DAGM German Conference on Pattern Recognition (GCPR). 130--144.

[155]

Nathaniel Weir, Prasetya Utama, Alex Galakatos, Andrew Crotty, Amir Ilkhechi, Shekar Ramaswamy, Rohin Bhushan, Nadja Geisler, Benjamin H"attasch, Steffen Eger, Ugur Cetintemel, and Carsten Binnig. 2020. DBPal: A Fully Pluggable NL2SQL Training Pipeline. In SIGMOD. 2347--2361.

[156]

Bichen Wu, Xiaoliang Dai, Peizhao Zhang, Yanghan Wang, Fei Sun, Yiming Wu, Yuandong Tian, Peter Vajda, Yangqing Jia, and Kurt Keutzer. 2019. FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search. In CVPR. 10734--10742.

[157]

Yuncheng Wu, Tien Tuan Anh Dinh, Guoyu Hu, Meihui Zhang, Yeow Meng Chee, and Beng Chin Ooi. 2022. Serverless Data Science - Are We There Yet? A Case Study of Model Serving. In SIGMOD. 1866--1875.

[158]

Jiuhong Xiao, Lavisha Aggarwal, Prithviraj Banerjee, Manoj Aggarwal, and Gerard Medioni. 2022. Identity Preserving Loss for Learned Image Compression. In CVPR Workshop on New Trends in Image Restoration and Enhancement and Challenges. 517--526.

[159]

Doris Xin, Stephen Macke, Litian Ma, Jialin Liu, Shuchen Song, and Aditya Parameswaran. 2018. HELIX: Holistic Optimization for Accelerating Iterative Machine Learning. Proc. VLDB Endow., Vol. 12, 4 (2018), 446--460.

Digital Library

[160]

Kai Xu. 2021. Learning in Compressed Domains. Ph.,D. Dissertation. Arizona State University.

[161]

Kai Xu, Minghai Qin, Fei Sun, Yuhao Wang, Yen-Kuang Chen, and Fengbo Ren. 2020. Learning in the Frequency Domain. In CVPR. 1740--1749.

[162]

Zhiqiang Xu, Dong Li, Weijie Zhao, Xing Shen, Tianbo Huang, Xiaoyun Li, and Ping Li. 2021. Agile and Accurate CTR Prediction Model Training for Massive-Scale Online Advertising Systems. In SIGMOD. 2404--2409.

[163]

Shuai Yang, Yueyu Hu, Wenhan Yang, Ling-Yu Duan, and Jiaying Liu. 2021. Towards Coding for Human and Machine Vision: Scalable Face Image Coding. IEEE Transactions on Multimedia, Vol. 23, 1 (2021), 2957--2971.

[164]

Gyeong-In Yu, Saeed Amizadeh, Sehoon Kim, Artidoro Pagnoni, Ce Zhang, Byung-Gon Chun, Markus Weimer, and Matteo Interlandi. 2021. WindTunnel: Towards Differentiable ML Pipelines beyond a Single Model. Proc. VLDB Endow., Vol. 15, 1 (2021), 11--20.

Digital Library

[165]

Binhang Yuan, Cameron R. Wolfe, Chen Dun, Yuxin Tang, Anastasios Kyrillidis, and Chris Jermaine. 2022. Distributed Learning of Fully Connected Neural Networks Using Independent Subnet Training. Proc. VLDB Endow., Vol. 15, 8 (2022), 1581--1590.

Digital Library

[166]

Xin Zhang, Yanyan Shen, Yingxia Shao, and Lei Chen. 2023. DUCATI: A Dual-Cache Training System for Graph Neural Networks on Giant Graphs with the GPU. Proc. ACM Manag. Data, Vol. 1, 2 (2023).

Digital Library

[167]

Yuhao Zhang and Arun Kumar. 2019. Panorama: A Data System for Unbounded Vocabulary Querying over Video. Proc. VLDB Endow., Vol. 13, 4 (2019), 477--491.

Digital Library

[168]

Yuhao Zhang, Frank McQuillan, Nandish Jayaram, Nikhil Kak, Ekta Khanna, Orhan Kislal, Domino Valdano, and Arun Kumar. 2021. Distributed Deep Learning on Data Systems: A Comparative Analysis of Approaches. Proc. VLDB Endow., Vol. 14, 10 (2021), 1769--1782.

Digital Library

[169]

Chenguang Zheng, Hongzhi Chen, Yuxuan Cheng, Zhezheng Song, Yifan Wu, Changji Li, James Cheng, Hao Yang, and Shuai Zhang. 2022. ByteGNN: Efficient Graph Neural Network Training at Large Scale. Proc. VLDB Endow., Vol. 15, 6 (2022), 1228--1242.

Digital Library

[170]

Hongkuan Zhou, Da Zheng, Israt Nisa, Vasileios Ioannidis, Xiang Song, and George Karypis. 2022. TGL: A General Framework for Temporal GNN Training on Billion-Scale Graphs. Proc. VLDB Endow., Vol. 15, 8 (2022), 1572--1580.

Digital Library

[171]

Shuchang Zhou, Zekun Ni, Xinyu Zhou, He Wen, Yuxin Wu, and Yuheng Zou. 2016. DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients. CoRR, Vol. abs/1606.06160 (2016).

[172]

Chenzhuo Zhu, Song Han, Huizi Mao, and William J. Dally. 2017. Trained Ternary Quantization. In ICLR.

Cited By

Chatterjee SPekala MKruglyak LIdreos S(2024)Limousine: Blending Learned and Classical Indexes to Self-Design Larger-than-Memory Cloud Storage EnginesProceedings of the ACM on Management of Data10.1145/36393022:1(1-28)Online publication date: 26-Mar-2024
https://dl.acm.org/doi/10.1145/3639302

Index Terms

The Image Calculator: 10x Faster Image-AI Inference by Replacing JPEG with Self-designing Storage Format
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision representations
        Image representations
2. Information systems
  1. Information storage systems
    1. Record storage systems
      1. Record storage alternatives

Recommendations

Mixed Image Format Environments: Comparisons, Tradeoffs, and Issues
IMAC '95: Proceedings of the 4th International Conference on Image Management and Communication (IMAC '95)

Abstract: As the extent of the networked image environments grows, it becomes increasing likely that multiple image formats will be present within a network. This paper addresses the general evolution of this environment through a specific examination ...
JPEG Image Steganalysis Improvement Via Image-to-Image Variation Minimization
ICACTE '08: Proceedings of the 2008 International Conference on Advanced Computer Theory and Engineering

In this research, we introduce an approach to enhance the discriminative capability of features by employing image-to-image variation minimization. In order to minimize image-to-image variation, we will estimate the cover image from the stego image by ...
Multi-dimensional color image storage and retrieval for a normal arbitrary quantum superposition state

Multi-dimensional color image processing has two difficulties: One is that a large number of bits are needed to store multi-dimensional color images, such as, a three-dimensional color image of $$1024 \times 1024 \times 1024$$1024 1024 1024 needs $$1024 ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Management of Data

Proceedings of the ACM on Management of Data Volume 2, Issue 1

SIGMOD

February 2024

1874 pages

EISSN:2836-6573

DOI:10.1145/3654807

Editor:
Divyakant Agrawal
UC Santa Barbara, United States

Issue’s Table of Contents

Copyright © 2024 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 March 2024

Published in PACMMOD Volume 2, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Author Tags

Qualifiers

Research-article

Funding Sources

Swiss National Science Foundation
USA Department of Energy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
254
Total Downloads

Downloads (Last 12 months)254
Downloads (Last 6 weeks)49

Reflects downloads up to 14 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chatterjee SPekala MKruglyak LIdreos S(2024)Limousine: Blending Learned and Classical Indexes to Self-Design Larger-than-Memory Cloud Storage EnginesProceedings of the ACM on Management of Data10.1145/36393022:1(1-28)Online publication date: 26-Mar-2024
https://dl.acm.org/doi/10.1145/3639302

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents