In today’s data-driven world, a robust data strategy is the linchpin of success for businesses across industries. As organizations grapple with massive data volumes and increasing complexity, and with the landscape of the AI and data science industry continuously changing ever so quickly, data strategies need to stay relevant in order to be effective. Here are 6 important trends to consider that can either ‘make or break’ your data strategy.

1. The Rise of Citizen Data Scientists

A citizen data scientist (a term coined by Gartner), also often referred to as a citizen data analyst, is defined as “a person who creates or generates models that leverage predictive or prescriptive analytics, but whose primary job function is outside of the field of statistics and analytics.” They typically come from the business side and have a good understanding of the data and business processes.

Citizen data scientists (CDSs) epitomize the democratization of data, as they can bridge the gap between self-service data discovery by business users and the more advanced analytics done by expert data scientists. Cultivating CDSs in your business units (or a combined CDS team for the organization) can accelerate and scale up the utilization of your data. To help CDSs thrive, your data strategy would need to be adjusted to provide the right environment for them. This includes aspects such as:

• Data proficiency programs — find the right candidates and the appropriate level of training, diversify the types of expertise if appropriate
• Data and analytics democratization — allow access to a wider team while maintaining good data stewardship and analytics best practices
• Self-service data marketplace — depending on what architecture is already in place, explore internal solutions or potential external partners, invest as needed
• Low-code/no-code development tools — evaluate platforms based on your use cases and needs, remember to consider scalability and integration potentials

Nevertheless, it is crucial to balance your strategy and investment between CDSs and expert data scientists. Remember that even though CDSs would ideally have a quantitative background, they are not as deeply trained in advanced statistics and analytics as the expert data scientists, and hence are meant to work together with rather than replacing the latter. Models should be evaluated by the experts before getting deployed. The overall environment should also be complemented by other important roles in the data lifecycle such as data engineers, ML engineers, etc. Hence, creating a Data Science Center of Excellence is vital in your data strategy.

2. Real-time Data

The era of real-time data analytics has transformed the way we extract value from information. Unlike traditional post-processing data analysis, real-time analytics provides immediate insights, allowing organizations to respond quickly to changing conditions, make informed decisions on the fly, and enhance customer experiences. In contrast, post-processing data analysis looks at historical data to identify patterns, which remains important for long-term strategies and trend analysis.

Enabling real-time data analytics often requires a mature data strategy that is aligned with your specific business strategies. This may include adjustments such as:

• Real-time access to operational systems — e.g. live sensors, beacons, web visitor activity tracking
• Streaming data — strategically prioritize the data to be made accessible on live streaming based on your use cases, keep in mind data regulations and governance best practices
• Contextual enrichment of data streams — enhancing the collected data with other (often external) data sources to make them more useable or insightful (e.g. weather/traffic data to augment live route optimization); carefully evaluate for the right method/tool/platform
• Real-time analytics — final and often the most difficult step; to enable this, assess your current step, identify blockers and gaps (e.g. data silos), and map out the data strategy to achieve your desired data state

Remember that storing unnecessary old data can get expensive and the value of data often decays with time. Striking a balance between real-time insights and historical analysis is key in today’s data-driven landscape, where agility and foresight are paramount for success.

3. Data & AI Regulations

The state of data and AI regulations is in a constant state of evolution. Governments and regulatory bodies worldwide are grappling with the rapid advancements in the field, seeking to strike a balance between fostering innovation and safeguarding privacy, ethics, and security. Recent years have seen the introduction of landmark data privacy regulations such as GDPR in Europe, state regulations in the US (e.g. CCPA), PIPEDA and Quebec’s Law 25 in Canada, etc. There are many more draft AI regulations soon to be finalized, such as the EU AI Act, AIDA and CPPA in Canada, and the newly signed executive order on Safe, Secure and Trustworthy AI in the US. For an organization utilizing AI, it is vital to stay informed about these regulations and guidance, as it may significantly impact the direction of your data product development and consequently your data strategy.

To help keep up with the evolving regulations, there are many frameworks of ‘Responsible AI’ that can be adopted. One example is the 6 principles of responsible AI outlined by Microsoft: accountability, inclusiveness, reliability and safety, privacy and security, fairness, and transparency. Another similar example is the 10 guiding principles laid out by McKinsey. Promoting the adoption of responsible AI principles within your organization’s data strategy is crucial to not only mitigate legal and ethical risks but also to enhance decision-making, improve customer satisfaction, and establish a reputation for responsible innovation in the competitive landscape of today’s data-driven world.

4. FinOps

Cloud computing, with its pay-as-you-go model and flexible scalability, has indeed saved organizations significant upfront capital expenditures and operational costs compared to setting up and maintaining on-premises infrastructure. It has become the go-to solution of the industry, with 56% of SMBs spending at least $600K per year and 78% of larger enterprises spending at least $1.2M (29% of which spends >$12M) per year on public cloud services (source: Flexera State of the Cloud Survey 2023). However, the issue of avoidable cloud spend or ‘cloud waste’ has escalated with the years. Across all cloud-maturity levels, 94% of organizations reported that they had cloud waste, mainly due to overprovisioning, idle or underused resources (source: HashiCorp State of Cloud Strategy Survey 2023).

The need for FinOps (Financial Operations) best practices is imperative in your data strategy. This practice involves continuous collaboration between technical and financial teams, utilizing tools and processes to monitor, analyze, and optimize cloud expenses. Depending on the size of your organization, the complexity of your cloud infrastructure, and your commitment to optimizing cloud spending, you may benefit from forming a dedicated FinOps team.

Another strategy that can be incorporated into your FinOps practice is collaboration with your cloud provider’s discount programs which could identify cost-reduction opportunities. Plenty of programs are available, such as AWS EDP (Enterprise Discount Program), Azure EA (Enterprise Agreement), Reserved Instances, Savings Plans, and more.

5. Logical Data Architecture and Data Fabric

Data architecture is crucial in data strategy as it provides the structured data management framework for organizing and managing data, ensuring efficiency, integrity, and accessibility, which are essential for making informed decisions, fostering innovation, and achieving strategic business objectives. It is important to keep up with the challenges and evolution of data architecture solutions.

The current accepted best practice of data architecture is Logical Data Warehouse (LDW), which has provided a unified and abstracted view of an organization’s data assets, allowing users to access and analyze data from various sources as if it were a single, logically organized repository, even though they physically are not. However, several requirements persist that are not satisfied by LDW, such as the need to access data from different types of repositories with one standard method/API, integrating data across the repositories, using a different best tool for each individual data processing task, and applying real-time processing to enable augmented analysis.

Data fabric, with its underlying logical data architecture, is a system that can implement these requirements. It democratizes access to data across an enterprise at scale by focusing on metadata management, orchestration, and contextual data enrichment by making use of the semantic layer. With these, it can provide solutions such as breaking data silos and enabling quicker insights or even real-time analytics. Instead of trying to build a single central repository for all your business units’ data and analytics, encourage your organization’s data strategy to strive towards logical data architecture/data fabric and getting real-time data by connecting to your systems.

6. GenAI & LLMs

Generative AI (GenAI) and LLMs are not — and probably will never be — perfect. There are many widely known issues surrounding them, such as hallucination, sensitive information handling, bias and fairness, and many others. Understandably, opinions on their readiness for real-world applications and deployments are widely varied.

This section is where my opinion respectfully differed from Paul’s. Even with their current limitations, GenAIs and LLMs have proven useful in being counterparts to humans in their daily tasks and boosting efficiency. Rather than the much-dreaded use of replacing humans, they have provided great functions in serving as complementary support or brainstorming partners to humans, from suggesting Python codes, initial screening with chatbots, to producing creative media. One needs to be careful of GenAI and LLMs’ limitations and find ways to make use of them incrementally and strategically.

Organizations that strategically incorporate GenAIs and LLMs into their data strategies stand to gain a competitive edge by unlocking new opportunities for innovation, customer engagement, and efficiency in handling large volumes of textual data. However, it is essential to address ethical considerations, and data privacy concerns, and continuously monitor and fine-tune these models for optimal performance within the broader data strategy framework.

In conclusion, the evolving landscape of data strategy reflects a dynamic interplay between technological advancements and the increasing demands of a data-driven world. As organizations navigate the complex terrain of data, these trends illuminate the path forward. Embracing them with a strategic mindset empowers organizations to not only unlock the full potential of their data but also to navigate challenges effectively. The future of data strategy lies in its adaptability, with organizations continuously refining their approaches to harness the transformative power of data and stay at the forefront of innovation in an ever-changing digital era.

Note : This Post is written and published by Priscilla Marianni to summarize the presentation “6 Trends That Can ‘Make or Break’ Your Data Strategy” by Paul Moxon, the SVP Data Architecture and Chief Evangelist of Denodo.

Reference :
https://medium.com/@priscilla.mariani1/6-trends-that-can-make-or-break-your-data-strategy-a7415da77f69