Dr. Ben E. Kuzey

Today, I had the opportunity to deliver the opening keynote at the Industry Summit on Data Product Oriented Architectures (jads.nl). The topic was centered around unraveling #dataproducts. What were the main conclusions? - There is no universal definition for data products, as there lacks a taxonomy, metadata standard, and interoperability standard. Therefore, to implement this concept within your organization, you need to create your own organizational standard with specific guidelines. - Deliver a minimum viable data product that enables all teams to utilize it. (Avoid creating a data product solely for specialized delivery!) - Refine your guidelines based on the lessons learned. - Ensure that all teams reach a consensus on cataloging, monitoring KPIs, etc. - Implement a data product dispute board that allows data product managers to reach agreements. - Invest in training and community building. - Measure and communicate the success of data products to ensure continued support. See the attached presentation for more information.

346

21 commentaren

🚀 DE Interview 📢 Cloud Optimization ⌛ 1-Min-Read Optimizing Cloud Storage Using Spark With a Housekeeping Script 🧔🏽‍♂️Interviewer: Can you explain how you optimized cloud storage using Spark with a housekeeping script? 👨‍💻Candidate: 1️⃣ In one of our projects, we were facing challenges with rapidly increasing storage costs due to accumulating data in AWS S3. To address this, I implemented an optimization strategy using Spark along with a housekeeping script. 🧔🏽‍♂️Interviewer: That sounds interesting. Could you walk me through the steps you took? 👨‍💻Candidate: 1️⃣ First, I analyzed the storage patterns and identified that a significant portion of the data was either stale or less frequently accessed. 2️⃣ I then developed a Spark job to classify this data based on its usage and relevance, segregating it into active and inactive datasets. 3️⃣ The inactive data was compressed using efficient algorithms, and a housekeeping script was created to automate the compression process and manage the data lifecycle. 4️⃣ The script would regularly run to compress, archive, and move infrequently accessed data to a low-cost storage tier while deleting unnecessary files. Versioning in S3 was enabled to safeguard against accidental deletion. 🧔🏽‍♂️Interviewer: How did this impact your overall cloud storage management? 👨‍💻Candidate: 1️⃣ The implementation of the Spark-based solution and housekeeping script led to significant cost savings by reducing the storage footprint. 2️⃣ It ensured that only relevant data occupied primary storage layers, improving data retrieval speeds and system efficiency. 3️⃣ Additionally, the automated process minimized manual intervention, reducing errors and operational overhead. Summary: 1️⃣ Problem: Escalating cloud storage costs and inefficient data management. 2️⃣ Solution: Implemented Spark-based optimization and automated housekeeping script for data lifecycle management. 3️⃣ Impact: Reduced storage costs, enhanced data management efficiency, and streamlined cloud operations. #CloudOptimization #Spark #AWS #DataManagement #CostSavings #Automation #TechInnovation #CloudComputing #S3 #DataLifecycle

14

Language Proficiency and AI. Peter Hinssen already wrote in ‘The End of Horrible’ that computer-corrected language is so dull and mechanical that it quickly irritates. Flawlessly written blocks of text without a message. Without passionate content. Without any humanity. Great for manuals, training, software, and other business but emotionless texts. Not inviting to read. Quickly so irritating that you put it aside and prefer a well, excitingly written article or book. AI is a tool, but it never delivers the human creativity and passion that makes normal communication so lively and exciting. So never let ChatGPT write your job applications, personal letters, and especially not your love letters… for the experienced reader and writer, you’ll be caught out immediately.

1

Summer day with lots of sun, beach, refreshing drinks and a book which was already long on my reading list. Finally was able to read “Data management at scale” from Piethein Strengholt. Create read if you are interested in data management, data platforms, data governance, data products, etc. Or if your organisation is looking on becoming data driven. Doesn’t matter if you are a small company or a large enterprise the book covers all aspects of setting op a data platform. Got some nice insights like: - start from your organisation capabilities - how to create context boundaries and why they are important - architecture choices to go for centralised or decentralised - importance of master data management - data products vs domain data stores - governance models - etc. But most important, it’s a journey. You will never get their without some trail and error. Let’s put these new insights into practice for some customers that are also making the shift of becoming data driven. #data #datamanagement #datagovernance #dataplatform #azure #azurefabric

71

4 commentaren

Day 2 of our course “Requirements gathering for Data Professionals” that I am conducting! Super exited to work with this amazing group of people again! Interested? Drop me a message academy@b21s.nl #DataLiteracy #DataAndAILiteracy #Requirementsgathering #Education #Lovetoteach #BDC Business Data Challengers Business Data Challengers Academy Wortell - Smart

25