Parallel Maintenance of Materialized Views in Large-Scale Analytic Platforms
Pages 1 - 19
Abstract
Speeding up queries processing is an important issue in database management. Materialized views are largely used to address this issue. They have been proven successful for query performance optimization. However, updating data sources of the corresponding view requires maintaining the related views. Therefore, a view maintenance strategy is required. This paper presents a novel approach for materialized view maintenance that overcomes the limitations of prior approaches using a parallel “Divide and Conquer” strategy. We modeled the view maintenance problem using a new concept called “Multiple-Views-Matrix” as a matrix that brings up all affected views and their corresponding base relations. Moreover, we introduce a new algorithm for performing maintenance of multiple views, the proposed algorithm is able to use multiple parallelism and recursivity; this allows it to maintain multiple views and to process several updates at the same time. We show that our method provides a significant improvement in terms of maintenance process cost.
References
[1]
Abderrazak, S. (2022). View Materialization for Query Processing in IoT Systems. International Journal of Technology Diffusion, 13(1).
[2]
Agrawal, D., El Abbadi, A., Singh, A., & Yurek, T. (1997). Efficient view maintenance at data warehouses. SIGMOD Record, 26(2), 417–427.
[3]
Babu, S., & Herodotou, H. (2013). Massively Parallel Databases and MapReduce Systems. Foundations and Trends in Databases, 5(1), 1–104.
[4]
Bailey, J., Dong, G., Mohania, M., & Wang, X. S. (1998). Incremental view maintenance by base relation tagging in distributed databases. Distributed and Parallel Databases, 6(3), 287–309.
[5]
Bamha, M., Bentayeb, F., & Hains, G. (1999). An efficient scalable parallel view maintenance algorithm for shared nothing multi-processor machines. In Database and Expert Systems Applications. (pp. 616-625).
[6]
Chen, J., & Rundensteiner, E. A. (2000). Txnwrap: A transactional approach to data warehouse maintenance. Technical Report WPI-CS-TR-00-26, Worcester Polytechnic InstituteKrueger, J., Kim, C., Grund, M., Satish, N., Schwalb, D., Chhugani, J., Plattner, H., Dubey, P., & Zeier, A. (2011). Fast Updates on Read-Optimized Databases Using Multi-Core CPUs . Proceedings of the VLDB Endowment International Conference on Very Large Data Bases, 5(1), 61–72.
[7]
Chen, S. (2010). Cheetah: A high performance, custom data warehouse on top of MapReduce. Proceedings of the VLDB Endowment International Conference on Very Large Data Bases, 3(1-2), 1459–1468.
[8]
Gupta, A., Jagadish, H., & Mumick, I. (1994). Data integration using self-maintainable views. Technical Memorandum 113880-941101-32, AT&T Bell Laboratories.
[9]
Gupta, A., Mumick, I. S., & Subrahmanian, V. S. (1993). Maintaining Views Incrementally. SIGMOD Record, 22(2), 157–166.
[10]
Gupta, H., & Mumick, I. S. (2006). Incremental maintenance of aggregate and outer-join expressions. Information Systems, 31(6), 435–464.
[11]
Héman, S., Zukowski, M., Nes, N. J., Sidirourgos, L., & Boncz, P. A. (2010). Positional update handling in column stores. In SIGMOD, pp 543–554.
[12]
Hu, Y., & Dessloch, S. (2014). Extracting deltas from column oriented NoSQL databases for different incremental applications and diverse data targets. Data & Knowledge Engineering, 93, 42–59.
[13]
HuynN. (1996). Efficient View Self-Maintenance. In Proceedings of the Workshop on Materialized Views: Techniques and Applications, pp. 17–25.
[14]
Huyn, N. (1997). Multiple-View Self-Maintenance. In Data Warehousing Environments, the VLDB Conference, pp 25-29.
[15]
DingL.ZhangX.RundensteinerE. A. (1999). The MRE wrapper approach: enabling incremental view maintenance of data warehouses defined on multi-relation information sources. In Proceedings of the 2nd ACM international workshop on Data warehousing and OLAP, ACM. pp. 30-35. 10.1145/319757.319784
[16]
Ise, Y., Yamamoto, S., Matsumoto, S., Saiki, S., & Nakamura, M. (2013, July). Implementing Materialized View of Large-Scale Power Consumption Log Using MapReduce. In Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, 14th ACIS International Conference, (pp. 523-528). IEEE. 10.1109/SNPD.2013.60
[17]
Kim, J., Lee, B. S., Moon, Y. S., Ok, S. H., & Lee, W. (2005). Parallel consistency maintenance of materialized views using referential integrity constraints in data warehouses. Data Warehousing and Knowledge Discovery. Springer Berlin Heidelberg.
[18]
Kim, N., & Moon, S. (2007). Concurrent View Maintenance Scheme for Soft Real-Time Data Warehouse Systems. Journal of Information Science and Engineering, 23, 723–739.
[19]
Labio, W. J., Yerneni, R., & Garcia-Molina, H. (1999). Shrinking the Warehouse Update Window. In ACM SIGMOD international conference on Management of data, pp 383-394.
[20]
Laurent, D., Lenchtenboerger, J., Spyratos, N., & Vossen, G. (2001). Monotonic Complements for Independent Data Warehouses. The VLDB Journal, 10(4), 295–315.
[21]
LeeK. Y.KimM. H. (2005). Optimizing the Incremental Maintenance of Multiple Join Views. In ACM International Workshop on Data Warehousing and OLAP, pp 107-113. 10.1145/1097002.1097021
[22]
LingT. W.SzeE. K. (1999). Materialized view maintenance using version numbers. In proceedings of 6th International Conference on Database Systems for Advanced Applications. IEEE, pp. 263-270. 10.1109/DASFAA.1999.765760
[23]
Liu, B., Chen, S., & Rundensteiner, E. A. (2002). A transactional approach to parallel data warehouse maintenance. In Data Warehousing and Knowledge Discovery, pp. 307-316.
[24]
LiuB.RundensteinerE. A. (2005). Cost-Driven General Join View Maintenance over Distributed Data Sources. In International Conference on Data Engineering, pp 578-579.
[25]
LuoG.NaughtonJ. F.EllmannC. J.WatzkeM. W. (2003, March). A comparison of three methods for join view maintenance in parallel RDBMS. In Proceedings 19th International Conference on Data Engineering (Cat. No. 03CH37405) (pp. 177-177). IEEE Computer Society. 10.1109/ICDE.2003.1260791
[26]
Mohania, M. K., Krishna, P. R., Kumar, K. P., Karlapalem, K., & Vincent, M. W. (2005). Functional Dependency Driven Auxiliary Relation Selection for Materialized Views Maintenance. In COMAD, pp 37-45.
[27]
Mûller, S., Butzmann, L., Howelmeyer, K., Klauck, S., & Plattner, H. (2013). Efficient View Maintenance for Enterprise Applications in Columnar In-Memory Databases. EDOC.
[28]
Mumick, I., Quass, D., & Mumick, B. (1997). Maintenance of Data Cubes and Summary Tables in a Warehouse. In ACM SIGMOD international conference on Management of data, pp 100-111.
[29]
NguyenT. V. A.BimonteS.d’OrazioL.DarmontJ. (2012, March). Cost models for view materialization in the cloud. In Proceedings of the 2012 Joint EDBT/ICDT Workshops, (pp. 47-54). ACM. 10.1145/2320765.2320788
[30]
O’Gorman, K., Agrawal, D., & El Abbadi, A. (1999). Posse: A Framework for Optimizing Incremental View Maintenance at Data Warehouses. In DaWaK, pp 106-115.
[31]
Onizuka, M., Kato, H., Hidaka, S., Nakano, K., & Hu, Z. (2013). Optimization for iterative queries on MapReduce. Proceedings of the VLDB Endowment International Conference on Very Large Data Bases, 7(4), 241–252.
[32]
Qu, W., & Dessloch, S. (2014). A Real-time Materialized View Approach for Analytic Flows in Hybrid Cloud Environments. Datenbank-Spektrum: Zeitschrift fur Datenbanktechnologie: Organ der Fachgruppe Datenbanken der Gesellschaft fur Informatik e.V, 14(2), 97–106.
[33]
Quass, D., Gupta, A., Mumick, I., & Widom, J. (1996). Making Views Self-Maintainable for Data Warehousing. In ICPADS, pp 158-169.
[34]
Samtani, S., Kumar, V., & Mohaina, M. (1999). Self-Maintenance of multiple Views in Data Warehousing. In International conference on Information and Knowledge Management, (pp 292-299).
[35]
Sebaa, A., Nouicer, A., & Tari, A. (2019). Impact of technology evolution on the materialized views: Current issues and future trends. International Journal of Business Information Systems, 30(4), 427–462.
[36]
Sebaa, A., & Tari, A. (2019). Materialized view maintenance: Issues, classification, and open challenges. International Journal of Cooperative Information Systems, 28(01), 1930001.
[37]
ShuZ.ZuoY.TangY. (2008). A collaborative framework for parallel view maintenance”. In 12th International Conference on. Computer Supported Cooperative Work in Design. pp. 81-86. IEEE.
[38]
Varde, A. S., & Rundensteiner, E. A. (2002). MedWrap: consistent view maintenance over distributed multi-relation sources. In DEXA, pp 341–350.
[39]
XuC.ZhouM.QianW. (2010, October). Materialized view maintenance in columnar storage for massive data analysis. In Universal Communication Symposium (IUCS), 2010 4th International, pp. 69-76. IEEE. 10.1109/IUCS.2010.5666768
[40]
ZhangX.DingL.RundensteinerE. A. (2001). PVM: Parallel view maintenance under concurrent data updates of distributed sources, in Proceedings of the 2001 International Conference on Data Warehousing and Knowledge Discovery, pp. 230–239. Munich, Germany. 10.1007/3-540-44801-2_23
[41]
Zhang, X., Ding, L., & Rundensteiner, E. A. (2004). Parallel multisource view maintenance. In The VLDB Journal, 13(1), pp. 22-48.
[42]
Zhang, X., & Rundensteiner, E. A. (2002). Integrating the maintenance and synchronization of data warehouses using a cooperative framework . Information Systems, 27(4), 219–243.
[43]
Zhang, X., Yang, L., & Wang, D. (2010). Incremental view maintenance based on data source compensation in data warehouses. In ICCASM, pp 287-291.
[44]
Zhou, J., Larson, P., & Elmongui, H. G. (2007). Lazy Maintenance of Materialized Views. In VLDB conference, pp. 231-242.
Index Terms
- Parallel Maintenance of Materialized Views in Large-Scale Analytic Platforms
Index terms have been assigned to the content through auto-classification.
Recommendations
Incremental maintenance of materialized views with outerjoins
Queries using outerjoins appear very frequently in traditional applications such as data warehousing. Lately, they have been widely used in newly emerged systems such as Object-Relational Mapping (ORM) tools, schema integration and information exchange ...
Efficient processing of monotonic linear progressive queries via dynamic materialized views
CASCON '10: Proceedings of the 2010 Conference of the Center for Advanced Studies on Collaborative ResearchThere is an increasing demand to process emerging types of queries, such as progressive queries (PQs), from numerous contemporary database applications including telematics, ecommerce, business intelligence, and decision support. Unlike a conventional ...
Processing Aggregate Queries with Materialized Views in Data Warehouse Environment
Materialized views, which are derived from base relations and stored in the database, offer opportunities for significant performance gain in query evaluation by providing quick access to the pre-computed data. A materialized view can be utilized in ...
Comments
Information & Contributors
Information
Published In
Copyright © 2022.
Publisher
IGI Global
United States
Publication History
Published: 21 July 2022
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 30 Aug 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in