research-article

Batch-file Operations to Optimize Massive Files Accessing: Analysis, Design, and Application

Authors:

Li YangAuthors Info & Claims

ACM Transactions on Storage (TOS), Volume 16, Issue 3

Article No.: 19, Pages 1 - 25

https://doi.org/10.1145/3394286

Published: 16 July 2020 Publication History

Abstract

Existing local file systems, designed to support a typical single-file access mode only, can lead to poor performance when accessing a batch of files, especially small files. This single-file mode essentially serializes accesses to batched files one by one, resulting in a large number of non-sequential, random, and often dependent I/Os between file data and metadata at the storage ends. Such access mode can further worsen the efficiency and performance of applications accessing massive files, such as data migration. We first experimentally analyze the root cause of such inefficiency in batch-file accesses. Then, we propose a novel batch-file access approach, referred to as BFO for its set of optimized Batch-File Operations, by developing novel BFOr and BFOw operations for fundamental read and write processes, respectively, using a two-phase access for metadata and data jointly. The BFO offers dedicated interfaces for batch-file accesses and additional processes integrated into existing file systems without modifying their structures and procedures. In addition, based on BFOr and BFOw, we also propose the novel batch-file migration BFOm to accelerate the data migration for massive small files. We implement a BFO prototype on ext4, one of the most popular file systems. Our evaluation results show that the batch-file read and write performances of BFO are consistently higher than those of the traditional approaches regardless of access patterns, data layouts, and storage media, under synthetic and real-world file sets. BFO improves the read performance by up to 22.4× and 1.8× with HDD and SSD, respectively, and it boosts the write performance by up to 111.4× and 2.9× with HDD and SSD, respectively. BFO also demonstrates consistent performance advantages for data migration in both local and remote situations.

References

[1]

Vasily Tarasov and George Amvrosiadis. 2018. Filebench. Retrieved from http://sourceforge.net/projects/filebench/.

[2]

Skyvia.com. 2018. Skyvia. Retrieved from https://skyvia.com/data-integration/synchronization.

[3]

Alibaba. 2018. TFS Project. Retrieved from http://code.taobao.org/p/tfs/src/.

[4]

William E. Allcock, John Bresnahan, Rajkumar Kettimuthu, and Michael Link. 2005. The globus striped GridFTP framework and server. In Proceedings of the ACM/IEEE Supercomputing Conference (SC’05). 54.

Digital Library

[5]

Michael P. Andersen and David E. Culler. 2016. BTrDB: Optimizing storage system design for timeseries processing. In Proceedings of the USENIX Conference on File and Storage Technologies. 39--52.

[6]

Apache. 2018. Hadoop. Retrieved from http://hadoop.apache.org/.

[7]

Jens Axboe. 2018. Blktrace. Retrieved from https://git.kernel.org/pub/scm/linux/kernel/git/axboe.

[8]

Doug Beaver, Sanjeev Kumar, Harry C. Li, Jason Sobel, and Peter Vajgel. 2010. Finding a needle in haystack: Facebook’s photo storage. In Proceedings of the 9th USENIX Symposium on Operating Systems Design and Implementation (OSDI’10). 47--60.

Digital Library

[9]

Binfer. 2018. High-speed File Transfer Software. Retrieved from https://www.binfer.com/high-speed-file-transfer-software/.

[10]

Vijay Chidambaram, Tushar Sharma, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. 2012. Consistency without ordering. In Proceedings of the USENIX Conference on File and Storage Technologies. 9.

[11]

Xiaoning Ding, Song Jiang, Feng Chen, Kei Davis, and Xiaodong Zhang. 2007. DiskSeen: Exploiting disk layout and access history to enhance I/O prefetch. In Proceedings of theUSENIX Annual Technical Conference. USENIX, 261--274.

[12]

John Esmet, Michael A. Bender, Martin Farach-Colton, and Bradley C. Kuszmaul. 2012. The TokuFS streaming file system. In Proceedings of the 4th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage’12).

[13]

Songling Fu, Ligang He, Chenlin Huang, Xiangke Liao, and Kenli Li. 2015. Performance optimization for managing massive numbers of small files in distributed file systems. IEEE Trans. Parallel Distrib. Syst. 26, 12 (2015), 3433--3448.

Digital Library

[14]

GNU. 2018. Linux scp. Retrieved from http://www.gnu.org/software/coreutils/coreutils.html.

[15]

GNU. 2018. Linux tar. Retrieved from http://www.gnu.org/software/coreutils/coreutils.html.

[16]

Yunhong Gu and Robert L. Grossman. 2007. UDT: UDP-based data transfer for high-speed wide area networks. Comput. Netw. 51, 7 (2007), 1777--1799.

Digital Library

[17]

Sangtae Ha, Injong Rhee, and Lisong Xu. 2008. CUBIC: A new TCP-friendly high-speed TCP variant. Operat. Syst. Rev. 42, 5 (2008), 64--74.

Digital Library

[18]

Andrew Hanushevsky. 2018. BBCP. Retrieved from http://www.slac.stanford.edu/ abh/bbcp/.

[19]

William Jannen, Jun Yuan, Yang Zhan, Amogh Akshintala, John Esmet, Yizheng Jiao, Ankur Mittal, Prashant Pandey, Phaneendra Reddy, Leif Walsh, Michael A. Bender, Martin Farach-Colton, Rob Johnson, Bradley C. Kuszmaul, and Donald E. Porter. 2015. BetrFS: A right-optimized write-optimized file system. In Proceedings of the USENIX Conference on File and Storage Technologies. 301--315.

[20]

Yongsoo Joo, Sangsoo Park, and Hyokyung Bahn. 2017. Exploiting I/O reordering and I/O interleaving to improve application launch performance. Trans. Stor. 13, 1 (2017), 8:1–8:17.

[21]

Tom Kelly. 2003. Scalable TCP: Improving performance in highspeed wide area networks. Comput. Commun. Rev. 33, 2 (2003), 83--91.

Digital Library

[22]

Youngjae Kim, Scott Atchley, Geoffroy Vallée, and Galen M. Shipman. 2015. LADS: Optimizing data transfers using layout-aware data scheduling. In Proceedings of the USENIX Conference on File and Storage Technologies. 67--80.

[23]

Changman Lee, Dongho Sim, Joo Young Hwang, and Sangyeun Cho. 2015. F2FS: A new file system for flash storage. In Proceedings of the USENIX Conference on File and Storage Technologies. 273--286.

[24]

Tan Li, Yufei Ren, Dantong Yu, and Shudong Jin. 2017. RAMSYS: Resource-aware asynchronous data transfer with multicore SYStems. IEEE Trans. Parallel Distrib. Syst. 28, 5 (2017), 1430--1444.

Digital Library

[25]

Jie Liang, Yinlong Xu, Yongkun Li, and Yubiao Pan. 2017. ISM- An intra-stripe data migration approach for RAID-5 scaling. In Proceedings of the International Conference on Networking, Architecture, and Storage (NAS’17). 1--10.

[26]

LinuxKernel. 2018. CFQ. Retrieved from https://www.kernel.org/doc/Documentation/block/cfq-iosched.txt.

[27]

LinuxKernel. 2018. Deadline. Retrieved from https://www.kernel.org/doc/Documentation/block/deadline-iosched.txt.

[28]

Yue Liu, Songlin Hu, Tilmann Rabl, Wantao Liu, Hans-Arno Jacobsen, Kaifeng Wu, Jian Chen, and Jintao Li. 2014. DGFIndex for smart grid: Enhancing hive with a cost-effective multidimensional range index. Proc. VLDB Endow. 7, 13 (2014), 1496--1507.

Digital Library

[29]

Lanyue Lu, Thanumalayan Sankaranarayana Pillai, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. 2016. WiscKey: Separating keys from values in SSD-conscious storage. In Proceedings of the 14th USENIX Conference on File and Storage Technologies (FAST’16). 133--148.

Digital Library

[30]

Marshall Kirk Mckusick and T. J. Kowalski. 2007. Fsck—The UNIX file system check program. Retrieved from https://dl.acm.org/doi/10.5555/107172.107210.

[31]

Netapp. 2018. Cloud Sync. Retrieved from https://cloud.netapp.com/cloud-sync.

[32]

Nexor. 2018. Secure and Efficient Manual Release of Files Across Networks. Retrieved from https://www.nexor.com/case-studies/files-transfer-secure-networks/.

[33]

Thanumalayan Sankaranarayana Pillai, Ramnatthan Alagappan, Lanyue Lu, Vijay Chidambaram, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. 2017. Application crash consistency and performance with CCFS. In Proceedings of the 15th USENIX Conference on File and Storage Technologies (FAST’17). 181--196.

Digital Library

[34]

Kai Ren and Garth A. Gibson. 2013. TABLEFS: Enhancing metadata efficiency in the local file system. In Proceedings of the USENIX Annual Technical Conference. 145--156.

Digital Library

[35]

Ohad Rodeh, Josef Bacik, and Chris Mason. 2013. BTRFS: The Linux B-tree filesystem. ACM Trans. Storage 9, 3 (2013), 1--32.

Digital Library

[36]

Bradley W. Settlemyer, Jonathan D. Dobson, Stephen W. Hodson, Jeffery A. Kuehn, Stephen W. Poole, and Thomas Ruwart. 2011. A technique for moving large data sets over high-performance long distance networks. In Proceedings of the IEEE Conference on Mass Storage Systems and Technologies (MSST’11). 1--6.

Digital Library

[37]

Philip Shilane, Mark Huang, Grant Wallace, and Windsor Hsu. 2012. WAN optimized replication of backup datasets using stream-informed delta compression. In Proceedings of the USENIX Conference on File and Storage Technologies. 5.

Digital Library

[38]

Adam Sweeney, Doug Doucette, Wei Hu, Curtis Anderson, Mike Nishimoto, and Geoff Peck. 1996. Scalability in the XFS file system. In Proceedings of the USENIX Annual Technical Conference. 1--14.

[39]

Textfiles.com. 2018. TextFiles. Retrieved from http://www.textfiles.com/bbs/.

[40]

Andrew Tridgell. 2018. Rsync. Retrieved from https://rsync.samba.org/.

[41]

Stephen C. Tweedie. 2000. EXT3, journaling filesystem. In Proceedings of the Ottowa Linux Symposium.

[42]

Wenrui Yan, Jie Yao, Qiang Cao, Changsheng Xie, and Hong Jiang. 2017. ROS: A rack-based optical storage system with inline accessibility for long-term data preservation. In Proceedings of the 12th European Conference on Computer Systems (EuroSys’17). 161--174.

Digital Library

[43]

Wangdong Yang, Kenli Li, and Keqin Li. 2019. A pipeline computing method of SpTV for three-order tensors on CPU and GPU. Trans. Knowl. Discov. Data 13, 6 (2019), 63:1–63:27.

[44]

Weikuan Yu, Jeffrey S. Vetter, Shane Canon, and Song Jiang. 2007. Exploiting lustre file joining for effective collective IO. In Proceedings of the 7th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID’07). 267--274.

Digital Library

[45]

Jun Yuan, Yang Zhan, William Jannen, Prashant Pandey, Amogh Akshintala, Kanchan Chandnani, Pooja Deo, Zardosht Kasheff, Leif Walsh, Michael A. Bender, Martin Farach-Colton, Rob Johnson, Bradley C. Kuszmaul, and Donald E. Porter. 2016. Optimizing every operation in a write-optimized file system. In Proceedings of the USENIX Conference on File and Storage Technologies. 1--14.

[46]

Haoyu Zhang, Brian Cho, Ergin Seyfe, Avery Ching, and Michael J. Freedman. 2018. Riffle: Optimized shuffle service for large-scale data analytics. In Proceedings of the 13th EuroSys Conference (EuroSys’18). 43:1–43:15.

[47]

Shuanglong Zhang, Helen Catanese, and An-I Andy Wang. 2016. The composite-file file system: Decoupling the one-to-one mapping of files and metadata for better performance. In Proceedings of the USENIX Conference on File and Storage Technologies. 15--22.

Cited By

Ueno RHaneda HHomma NInoue AMinematsu KLuo BLiao XXu JKirda ELie D(2024)Crystalor: Recoverable Memory Encryption Mechanism with Optimized Metadata StructureProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security10.1145/3658644.3670273(228-242)Online publication date: 2-Dec-2024
https://dl.acm.org/doi/10.1145/3658644.3670273
Huang QLi PHuang YShuang FCai Y(2024)Region-Focused Network for Dense CaptioningACM Transactions on Multimedia Computing, Communications, and Applications10.1145/364837020:6(1-20)Online publication date: 26-Mar-2024
https://dl.acm.org/doi/10.1145/3648370
Yang ZYang ZGuo ZLin ZZhu HLi QLiu W(2023)Towards Temporal Event Detection: A Dataset, Benchmarks and ChallengesIEEE Transactions on Multimedia10.1109/TMM.2023.327652326(1102-1113)Online publication date: 16-May-2023
https://dl.acm.org/doi/10.1109/TMM.2023.3276523
Show More Cited By

Index Terms

Batch-file Operations to Optimize Massive Files Accessing: Analysis, Design, and Application
1. Information systems
  1. Information storage systems
    1. Information storage technologies
2. Software and its engineering
  1. Software organization and properties
    1. Contextual software domains
      1. Operating systems
        File systems management

Recommendations

A multiple-file write scheme for improving write performance of small files in Fast File System

Fast File System (FFS) stores files to disk in separate disk writes, each of which incurs a disk positioning (seek + rotation) limiting the write performance for small files. We propose a new scheme called co-writing to accelerate small file writes in ...
Research of Massive Small Files Reading Optimization Based on Parallel Network File System
HPCC-CSS-ICESS '15: Proceedings of the 2015 IEEE 17th International Conference on High Performance Computing and Communications, 2015 IEEE 7th International Symposium on Cyberspace Safety and Security, and 2015 IEEE 12th International Conf on Embedded Software and Systems

With the rapid development of cloud computing and big data, there are more and more small files. How to manage those massive small files efficiently and provide low-latency service is becoming a hot topic in Parallel Network File System (pNFS). When ...
Accessing Files on Unmounted Filesystems
LISA '01: Proceedings of the 15th USENIX conference on System administration

This paper describes a utility named <tt>ruf</tt> that reads files from an unmounted file system. The files are accessed by reading disk structures directly so the program is peculiar to the specific file system employed. The current implementation ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Storage

ACM Transactions on Storage Volume 16, Issue 3

August 2020

150 pages

ISSN:1553-3077

EISSN:1553-3093

DOI:10.1145/3410885

Editor:
Sam H. Noh
Ulsan National Institute of Science and Technology, Ulsan, Republic of Korea

Issue’s Table of Contents

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 July 2020

Online AM: 07 May 2020

Accepted: 01 April 2020

Revised: 01 March 2020

Received: 01 September 2019

Published in TOS Volume 16, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Creative Research Group Project of NSFC
the US NSF
National key research and development program of China
NSFC

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
236
Total Downloads

Downloads (Last 12 months)26
Downloads (Last 6 weeks)2

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ueno RHaneda HHomma NInoue AMinematsu KLuo BLiao XXu JKirda ELie D(2024)Crystalor: Recoverable Memory Encryption Mechanism with Optimized Metadata StructureProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security10.1145/3658644.3670273(228-242)Online publication date: 2-Dec-2024
https://dl.acm.org/doi/10.1145/3658644.3670273
Huang QLi PHuang YShuang FCai Y(2024)Region-Focused Network for Dense CaptioningACM Transactions on Multimedia Computing, Communications, and Applications10.1145/364837020:6(1-20)Online publication date: 26-Mar-2024
https://dl.acm.org/doi/10.1145/3648370
Yang ZYang ZGuo ZLin ZZhu HLi QLiu W(2023)Towards Temporal Event Detection: A Dataset, Benchmarks and ChallengesIEEE Transactions on Multimedia10.1109/TMM.2023.327652326(1102-1113)Online publication date: 16-May-2023
https://dl.acm.org/doi/10.1109/TMM.2023.3276523
Yang ZXiang JYou JLi QLiu W(2023)Event-Oriented Visual Question Answering: The E-VQA Dataset and BenchmarkIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.326703635:10(10210-10223)Online publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1109/TKDE.2023.3267036
Zubair KAbdullah RMohaisen DAwad A(2023)RC-NVM: Recovery-Aware Reliability-Security Co-Design for Non-Volatile MemoriesIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2023.327903121:4(1817-1830)Online publication date: 3-Jul-2023
https://dl.acm.org/doi/10.1109/TDSC.2023.3279031
Zhang GDeng YZhou YPang SYue JZhu Y(2023)Cocktail: Mixing Data With Different Characteristics to Reduce Read Reclaims for nand Flash MemoryIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2022.321467942:7(2336-2349)Online publication date: 1-Jul-2023
https://dl.acm.org/doi/10.1109/TCAD.2022.3214679
Kim JLee WHong JKim S(2022)Efficient Integrity-Tree Structure for Convolutional Neural Networks through Frequent Counter Overflow Prevention in Secure MemoriesSensors10.3390/s2222876222:22(8762)Online publication date: 13-Nov-2022
https://doi.org/10.3390/s22228762
Lin HLi MXia QFei YYin BYang X(2022)6-DoF Pose Relocalization for Event Cameras With Entropy Frame and Attention NetworksProceedings of the 18th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry10.1145/3574131.3574457(1-8)Online publication date: 27-Dec-2022
https://dl.acm.org/doi/10.1145/3574131.3574457
Lei MWang FFeng DShuai XCao Y(2022)A Dynamic and Recoverable BMT Scheme for Secure Non-Volatile MemoryProceedings of the 51st International Conference on Parallel Processing10.1145/3545008.3545061(1-11)Online publication date: 29-Aug-2022
https://dl.acm.org/doi/10.1145/3545008.3545061
Zou YZubair KAlwadi MShadab RGandham SAwad ALin M(2022)ARES: Persistently Secure Non-Volatile Memory with Processor-transparent and Hardware-friendly Integrity Verification and Metadata RecoveryACM Transactions on Embedded Computing Systems10.1145/349273521:1(1-32)Online publication date: 10-Feb-2022
https://dl.acm.org/doi/10.1145/3492735
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Issue’s Table of Contents