Synchronous metadata management of large storage systems
Authors: 
Günter Hackl, Wolfgang Pausch, Sebastian Schönherr, Günther Specht, Gunther ThielAuthors Info & Claims
Günter Hackl, Wolfgang Pausch, Sebastian Schönherr, Günther Specht, Gunther ThielAuthors Info & ClaimsAbstract
Today filesystems of big companies are both huge and distributed amongst the world. They contain huge sets of metadata, but are not optimized to analyze them. In contrast, if metadata is stored in a database system and updated synchronously, it could be analyzed and processed in a much easier and straightforward way. Then even adding new attributes, not natively supported by the underlying filesystem, is easily possible. Thus, synchronous metadata storage in a database system can help managing and administrating huge filesystems efficiently but must not slow down the filesystem significantly. The aim of this paper is to describe possible solutions for synchronous metadata storage, inspect how such an integration of filesystem and database system might look like and evaluate the performance.
References
[1]
}}SQLAPI++: A C++ library for accessing multiple SQL databases. last visited on January 2010. {Online}. http://www.sqlapi.com.
[2]
}}Andrew Tridgell. DBench. last visited on January 2010. {Online}. http://samba.org/ftp/tridge/dbench.
[3]
}}J. Celko. Joe Celko's Trees and Hierarchies in SQL for Smarties. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, May 2004.
[4]
}}Lustre file system: High-performance storage architecture and scalable cluster file system. Lustre white paper, October 2008. last visited on January 2010. {Online}. http://tinyurl.com/y9z6de9.
[5]
}}H. Garcia-Molina and K. Salem. Main memory database systems: An overview. IEEE Trans. on Knowl. and Data Eng., 4(6):509--516, 1992.
[6]
}}M. Jarke, M. Lenzerini, Y. Vassiliou, and P. Vassiliadis. Fundamentals of Data Warehouses. Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2001.
[7]
}}MemcacheDB. A distributed key-value storage system designed for persistent. last visited on January 2010. {Online}. http://memcachedb.org/.
[8]
}}Mikio Hirabayashi. Tokyo Cabinet: a modern implementation of DBM. last visited on January 2010. {Online}. http://1978th.net/tokyocabinet.
[9]
}}D. Nagle, D. Serenyi, and A. Matthews. The panasas activescale storage cluster: Delivering scalable high bandwidth storage. In Proc. of the 2004 ACM/IEEE Conf. on Supercomputing (SC '04), pages 53--63, Washington, DC, USA, 2004. IEEE Computer Society.
[10]
}}Oracle Corporation. TimesTen. last visited on January 2010. {Online}. http://www.oracle.com/timesten/index.html.
[11]
}}Redis. A persistent key-value database with built-in net interface written in ansi-c for posix systems. last visited on January 2010. {Online}. http://code.google.com/p/redis/.
[12]
}}M. Stonebraker, S. Madden, D. J. Abadi, S. Harizopoulos, N. Hachem, and P. Helland. The end of an architectural era (it's time for a complete rewrite). In Proc. of the 33rd International Conf. on Very Large Data Bases (VLDB), pages 1150--1160, Vienna, Austria, 2007. ACM Press.
[13]
}}Tropashko. Nested intervals tree encoding in sql. SIGMOD Rec., 34(2):47--52, 2005.
[14]
}}V. Tropashko. Nested intervals tree encoding with continued fractions. Computing Research Repository (CoRR), cs.DB/0402051, 2004.
[15]
}}S. Weil, S. A. Brandt, E. L. Miller, D. D. E. Long, and C. Maltzahn. Ceph: A scalable, high-performance distributed file system. In Proc. of the 7th Conf. on Operating Systems Design and Implementation (OSDI '06), pages 307--320, Berkeley, CA, USA, Nov 2006. USENIX Association.
[16]
}}M. Xiong, H. Jin, and S. Wu. Fdsss: An efficient metadata management scheme in large scale data environment. In Proc. of the Fifth International Conf. on Grid and Cooperative Computing Workshops (GCCW '06), pages 71--77, Washington, DC, USA, 2006. IEEE Computer Society.
Recommendations
Metadata management and relational databases
ACMSE '05 vol 1: Proceedings of the 43rd annual ACM Southeast Conference - Volume 1
A simple extension to the relational model that permits meta-data (more generally metaknowledge) to be stored and manipulated as first class data is proposed. The accompanying query language is SQL extended with reification and reflection capabilities. ...
Comments
Information & Contributors
Information
Published In
August 2010
282 pages
Copyright © 2010 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
- ACM: Association for Computing Machinery
- BytePress
- Concordia University: Concordia University
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 16 August 2010
Check for updates
Qualifiers
- Research-article
Conference
IDEAS '10
Sponsor:
- ACM
- Concordia University
IDEAS '10: Fourteenth International Database Engineering & Applications
August 16 - 18, 2010
Quebec, Montreal, Canada
Acceptance Rates
Overall Acceptance Rate 74 of 210 submissions, 35%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 225Total Downloads
- Downloads (Last 12 months)2
- Downloads (Last 6 weeks)0
Reflects downloads up to 25 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in