article

Free access

Database compression

Authors:

Scott J. Van HornAuthors Info & Claims

ACM SIGMOD Record, Volume 22, Issue 3

Pages 31 - 39

https://doi.org/10.1145/163090.163096

Published: 01 September 1993 Publication History

Abstract

Despite the fact that computer memory costs have decreased dramatically over the past few years, data storage still remains, and will probably always remain, an important cost factor for many large scale database applications. Compressing data in a database system is attractive for two reasons: data storage reduction and performance improvement. Storage reduction is a direct and obvious benefit, while performance improves because smaller amounts of physical data need to be moved for any particular operation on the database.

We address several aspects of reversible data compression and compression techniques:

general concepts of data compression;

a number of compression techniques;

a comparison of the effects of compression on common data types;

advantages and disadvantages of compressing data; and

future research needs.

References

[1]

1. Anderson, Robert, et al. "A Very High Speed Noiseless Data Compression Chip for Space Imaging Applications." In Storer and Reif¹², 462.

[2]

2. Bassiouni, M. A. "Data Compression in Scientific and Statistical Databases," IEEE Transactions on Software Engineering, SE-11(10): 1047- 1058 (October 1985).

Digital Library

[3]

3. Fang, Wai-Chi, et al. "A Neural Network Based VLSI Vector Quantizer for Real-Time Image Compression." In Storer and Reif¹², 342-351.

[4]

4. Huffman, David A. "A Method for the Construction of Minimum-Redundancy Codes," Proceedings of the IRE, 40: 1098-1101 (September 1952).

[5]

5. Jones, Douglas W. "Application of Splay Trees to Data Compression," Communications of the ACM, 31(8): 996-1007 (August 1988).

Digital Library

[6]

6. Martin, James. Computer Database Organization . New Jersey: Prentice-Hall, Inc., 1977.

Digital Library

[7]

7. Mukherjee, Amar, et al. "Multibit Decoding/Encoding of Binary Codes Using Memory Based Architectures." In Storer and Reif¹², 352- 361.

[8]

8. Perl, Yehoshua, et al. "The Cascading of the LZW Compression Algorithm with Arithmetic Coding." In Storer and Reif¹², 277-286.

[9]

9. Reghbati, H.K. "An Overview of Data Compression Techniques," Computer, 71-75 (April 1981).

[10]

10. Ruth, Stephen S. and Paul J. Kreutzer. "Data Compression for Large Business Files," Datamation , 62-66 (September 1972).

[11]

11. Severance, Dennis G. "A Practitioner's Guide to Database Compression--A Tutorial," Information Systems, 8(1): 51-62 (January 1983).

[12]

12. Storer, James A. and John H. Reif, editors. DCC '91 Data Compression Conference, IEEE Computer Society Press, 1991.

Digital Library

[13]

13. Teorey, Toby J. and James P. Fry. Design of Database Structures. New Jersey: Prentice-Hall, Inc., 1982.

Digital Library

[14]

14. Venbrux, Jack and Norley Liu. "A Very High Speed Lossless Compression/Decompression Chip Set." In Storer and Reif¹², 461.

[15]

15. Welch, Terry A. "A Technique for High Performance Data Compression," Computer, 8-19 (June 1984).

[16]

16. Wiederhold, Gio. Database Design. New York: McGraw-Hill Book Co, 1983.

Digital Library

[17]

17. Witten, Ian H., et al. "Arithmetic Coding for Data Compression," Communications of the ACM, 30(6): 520-540 (June 1987).

Digital Library

Cited By

Chen XTian JBeaver IFreeman CYan YWang JTao D(2024)FCBench: Cross-Domain Benchmarking of Lossless Compression for Floating-Point DataProceedings of the VLDB Endowment10.14778/3648160.364818017:6(1418-1431)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.14778/3648160.3648180
Zeng XZhang S(2024)CStream: Parallel Data Stream Compression on Multicore Edge DevicesIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.338686236:11(5889-5904)Online publication date: Nov-2024
https://doi.org/10.1109/TKDE.2024.3386862
Zhang YZhang FLi HZhang SGuo XChen YPan ADu X(2024)Data-Aware Adaptive Compression for Stream ProcessingIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.337771036:9(4531-4549)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1109/TKDE.2024.3377710
Show More Cited By

Index Terms

Database compression
1. Information systems
  1. Data management systems
    1. Data structures
      1. Data layout
        Data compression
  2. Information storage systems
    1. Storage management
2. Software and its engineering
  1. Software organization and properties
    1. Contextual software domains
      1. Operating systems
        File systems management
        Memory management

Recommendations

Database compression with data mining methods
Information organization and databases

Despite the drop in prices, storage cost is still a major cost factor in large scale database applications, such as data warehouses. Data compression is needed to reduce the cost. Many data compression techniques have been proposed and the issue of ...
Relational Database Compression Using Augmented Vector Quantization
ICDE '95: Proceedings of the Eleventh International Conference on Data Engineering

Data compression is one way to alleviate the I/O bottleneck problem faced by I/O-intensive applications such as databases. However, this approach is not widely used because of the lack of suitable database compression techniques. In this paper, we ...
Lossless Data Compression: Data compression, Algorithm, Lossy compression, Bit rate, ZIP (file format), Unix, Gzip, Portable Network Graphics, Graphics Interchange Format, Tagged Image File Format

Comments

Information & Contributors

Information

Published In

cover image ACM SIGMOD Record

ACM SIGMOD Record Volume 22, Issue 3

Sept. 1993

98 pages

ISSN:0163-5808

DOI:10.1145/163090

Editor:
Arie Segev
Univ. of California, Berkeley

Issue’s Table of Contents

Copyright © 1993 Authors.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 September 1993

Published in SIGMOD Volume 22, Issue 3

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

112
Total Citations
View Citations
2,178
Total Downloads

Downloads (Last 12 months)271
Downloads (Last 6 weeks)33

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chen XTian JBeaver IFreeman CYan YWang JTao D(2024)FCBench: Cross-Domain Benchmarking of Lossless Compression for Floating-Point DataProceedings of the VLDB Endowment10.14778/3648160.364818017:6(1418-1431)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.14778/3648160.3648180
Zeng XZhang S(2024)CStream: Parallel Data Stream Compression on Multicore Edge DevicesIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.338686236:11(5889-5904)Online publication date: Nov-2024
https://doi.org/10.1109/TKDE.2024.3386862
Zhang YZhang FLi HZhang SGuo XChen YPan ADu X(2024)Data-Aware Adaptive Compression for Stream ProcessingIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.337771036:9(4531-4549)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1109/TKDE.2024.3377710
Chen HLiu LMeng JLu W(2024)AFC: An adaptive lossless floating-point compression algorithm in time series databaseInformation Sciences10.1016/j.ins.2023.119847654(119847)Online publication date: Jan-2024
https://doi.org/10.1016/j.ins.2023.119847
Liu CPavlenko AInterlandi MHaynes B(2023)A Deep Dive into Common Open Formats for Analytical DBMSsProceedings of the VLDB Endowment10.14778/3611479.361150716:11(3044-3056)Online publication date: 24-Aug-2023
https://dl.acm.org/doi/10.14778/3611479.3611507
Afroozeh AKuffo LBoncz P(2023)ALP: Adaptive Lossless floating-Point CompressionProceedings of the ACM on Management of Data10.1145/36267171:4(1-26)Online publication date: 12-Dec-2023
https://dl.acm.org/doi/10.1145/3626717
Wan WZhang FZhang CZhang MZhai JChai YZhang HLu WChen YLi HPan ADu X(2023)Compressed Data Direct Computing for DatabasesIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.331627436:5(1902-1918)Online publication date: 18-Sep-2023
https://dl.acm.org/doi/10.1109/TKDE.2023.3316274
Huang YWen DLai LQian ZQin LZhang Y(2023)Efficient and Effective Path Compression in Large Graphs2023 IEEE 39th International Conference on Data Engineering (ICDE)10.1109/ICDE55515.2023.00237(3093-3105)Online publication date: Apr-2023
https://doi.org/10.1109/ICDE55515.2023.00237
Zhang YZhang FLi HZhang SDu X(2023)CompressStreamDB: Fine-Grained Adaptive Stream Processing without Decompression2023 IEEE 39th International Conference on Data Engineering (ICDE)10.1109/ICDE55515.2023.00038(408-422)Online publication date: Apr-2023
https://doi.org/10.1109/ICDE55515.2023.00038
Hildebrandt JHabich DLehner W(2023)BOUNCE: memory-efficient SIMD approach for lightweight integer compressionDistributed and Parallel Databases10.1007/s10619-023-07426-041:3(439-466)Online publication date: 10-May-2023
https://dl.acm.org/doi/10.1007/s10619-023-07426-0
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents