Data Mesh
LOREM IPSUM
LOREM IPSUMLOREM IPSUMLOREM IPSUMLOREM IPSUMLOREM IPSUMLOREM IPSUMLOREM IPSUMLOREM IPSUMLOREM IPSUMLOREM IPSUM
What is a Data Mesh
A Data Mesh is a decentralised approach to your enterprise data architecture. The concept provides end-users with the ability to find and access cross-domain data easily. The approach is designed and operates based on a level of data ownership and responsibility within different domains, diverging from the legacy centralised monolithic data architectures of old.
The approach was originally theorised by Zhamak Degani, building the concept on Four Essential Pillars; Domain Ownership, Data as a Product, Self-serve data platform, and Federated Computational Governance. Learn about this state-of-the-art enterprise data approach and explore how Data Watchdog can be utilised in constructing your own Data Mesh Architecture.
Explore WCKD RZR Solutions
Find out how WCKD RZR can help you implement Data Mesh architecture.
Why a Data Mesh?
Existing Data Estates are commonly monolithic and centralised around complex and convoluted pipelines and processes. These processes are overseen by dedicated teams; however, as organisations are becoming more data-centric these archaic processes are becoming increasingly overwhelmed by the ever-growing demands for their data from across the enterprise.
- Different data use cases require different types of transformation, increasing the strain on monolithic processes.
- Constantly growing volumes of enterprise data usage will overwhelm a central data pipeline.
- Increasingly complex data queries slow the response time and agility of teams and their processes.
- Centralised Data Estates require costly and time-consuming transporting and importing of data assets to a central data lake.
A Data Mesh approach to enterprise data architecture will enable your organisation to handle and grow with the increasing importance and volume of data requests.
The Four Pillars of Data Mesh
Explore the four pillars of Data Mesh and how Data Watchdog can help you implement this state-of-the-art enterprise data approach.
Benefits of Data Mesh
In moving away from the centralised processes and pipelines of the monolithic data estate, a modern Data Mesh moves the responsibility and control of data access and processing to those stakeholders with the contextual knowledge of their own information. This contextual knowledge of their own means you avoid the back-and-forth communications, contextual questions and requests for approvals of existing monolithic processes.
- A Data Mesh approach creates an agile data enterprise architecture, capable of handling the growing and changing influence of data within modern organisations.
- This agility in the construction of the Data Mesh will allow the organisation to adjust and evolve in line with everchanging regulations, growing data estates and sources, and increasingly important analytical demands.
- Through a centralised access infrastructure, Domain users are able to access the critical data they require directly, quickly and without intervention.
Ultimately, the Data Mesh approach is designed to put the stakeholders with the relevant knowledge in a position to carry out their own data access processes without time-consuming interactions with external teams and convoluted processes.
Domain Ownership
In existing monolithic data processes, external data teams hold the responsibility for the handling of data assets between stakeholders and data sources, without the context of the assets intended deployment. Without a link between data providers and consumers, direct feedback and context is lost leading to the end-to-end processes of the data pipeline becoming inefficient.
- A Data Mesh operates through Domain Ownership of enterprise data processes and responsibilities.
- Domains become responsible for the data they produce and utilise for ingestion, transformation, and serving that data to end users.
- In the shifting responsibility and handling of data assets to those invested in its application, adequate context and feedback ensure an accurate and direct data pipeline.
In a Data Mesh, Domain owners are able to leverage data assets directly in line with their operational needs eliminating time-consuming and convoluted back-and-forth processes associated with existing data access pipelines.
Back to the Pillars
Data as a Product
Data products or 'Data Assets' are data sets served by a domain and transformed into business value. The Data product is key to the operation of a Data Mesh, existing attitudes treat the Data Product as a by-product of business processes, collected and dumped in central stores.
- A Data Mesh prioritises the quality in the collection and processing of data assets as a product in themselves.
- When handled correctly, with relevant context, quality data assets can be combined and analysed to generate further high-quality data assets.
- Domain Ownership ensures that collected data products are handled and transformed by stakeholders with the most subject matter expertise.
When treating Data as a Product, a Data Mesh approach ensures high quality data is captured and delivered efficiently to the relevant domains and stakeholders.
Back to the Pillars
Self-Serve Data Platform
The end-to-end process for accessing a relevant data resource within a centralised enterprise data pipeline involves a number of external processes and stakeholders, outside of the business users' domain. These convoluted processes require a chain of approvals, requests, and delays in the simple task of providing business users access to valuable data resources. The Data Mesh approach deploys the Domain Ownership in designing a Self-service infrastructure for all data processes.
- This platform allows business users to design, specify, and adjust their data accesses in line with the context of their domain and task.
- A self-service platform will be managed by central IT services; handling design, management, and design.
- A centralised self-service platform will also provide domain data engineers the ability to focus on building business driven domain-specific value, free from the strain of centralised IT requirements and design.
The Self-serve Data Platform within a Data Mesh will allow domain users and engineers the ability to carry out key tasks on a centrally managed and operated tool; however, without the constraints of centralised requirements, communication, and requests. This streamlines processes within domains and enables the true Domain Ownership of case-specific data at a reduced cost.
Back to the Pillars
Domain Driven Data Governance
The Data Mesh approach proposes decentralisation of data requests and processes, with Domains taking ownership of their pipeline; however, with the stakeholders and risks involved in corporate governance, a Data Mesh approach requires a shared responsibility between domains and centralised processes for data governance.
- Shared responsibility for Data Governance allows Domains a clear framework and responsibilities to adhere to and enforce in their processes.
- Domain-driven enforcement utilises domain-specific context to ensure data products and processes are handled in line with the correct standards.
- A single data enforcement capability with the ability for domain driven policy definition ensures the universal compliance of all domains to all required Governance policies on a firm-wide level.
A Domain Driven Governance function utilises domain ownership of data processes in ensuring all actions taken within the Data Mesh are fully compliant with relevant requirements, whilst also being held in compliance with universal requirements and policies through centralised enforcement infrastructure.
Back to the Pillars
