For most enterprises these days, hyperscale data workloads and AI have become the norm and are driving data strategies. However, while conventional wisdom dictates that this means greater reliance on cloud services and cutting-edge approaches to manage data pipelines, there’s a place for more well-tread approaches as well. Many data managers intend to beef up their data warehouse capabilities and even return to on-prem systems in the near future.
These are the findings of a recent survey of 500 data and IT managers who handle data workloads of 150 terabytes or more. Enterprises are ramping up their investments in data analytics and warehousing to meet the challenges of increasing data volumes and business requirements, according to the survey, released by Ocient, a data analytics platform. While cloud has been the dominant technology model for a number of years, the survey detects a return to on-prem capabilities for some data analytics operations.
More than one-third of respondents, 34%, say their data analytics road map for the next 12–18 months will include a partial return to an on-prem mode. In addition, 27% say they intend to prioritize an on-prem deployment model during the next 1–3 years.
In addition, data warehousing will also be playing a prominent role in the intelligent data enterprises that are evolving. At least 44% of data and IT leaders are prioritizing bringing AI and machine learning capabilities into their data warehouse deployments, which will help “streamline operational environments and accelerate data science,” the Ocient survey authors stated.
The need for greater data analytics capabilities is undisputed. About all respondents, 99%, agree that it’s “somewhat” or “very important” to increase the amount of data their organizations analyze in the next 1–3 years. “Although the growth of data continues unhindered, 2023 has brought about some interesting changes,” the study’s authors pointed out. “Perhaps no story has captured our attention more than generative artificial intelligence and machine learning. People everywhere, from virtually every industry, are looking at large language models with new eyes, imagining the myriad ways in which this notso- new technology can and will change our lives.”
AI readiness is top of mind for many data managers, the survey confirms. However, AI efforts are often tempered by security, accuracy, and trust concerns. Close to two-thirds, 63%, report being “very concerned” about the security and compliance of their large datasets. Another 49% are looking for cost-effective ways to scale data management and analysis.
Data quality is also vital, according to the survey. Close to half of the managers surveyed, 47%, cite data quality as a top data analytics priority for this year. In addition, while the top three issues remained the same as the previous year’s survey, an increasing number—from 14% in 2022 to 25% in 2023—were also concerned about the reliability of their data.
Simultaneously, the time, talent, and tools mismatch is hindering data-driven innovation, the survey shows. More than 70% of survey respondents believe that a comprehensive data strategy should include the development of new products, services, and revenue streams. However, 55% say data is growing too fast for them to harness its full capacity. Meanwhile, 40% say the lack of talent available to analyze their data is an issue holding back progress.
“Companies need sufficient time to collaborate and iterate, tools that support their goals effectively, and talent pools deep enough to drive innovation,” the survey’s authors said. “But for many of our survey respondents, supply and demand for these three critical components of innovation are mismatched. … Advanced analytics and data science, as well as AI and ML, require specialized talent and powerful, leading-edge tools that can deliver results quickly—often, in near real time.”
The survey “makes it clear that the world is continuing its steady march past big data toward a hyperscale future,” the analysts pointed out. “As leaders prepare for this new reality, their survey responses reveal that it is more critical than ever for organizations to develop a solid foundation for data integration, transformation, processing, and analysis.”