A framework introducing Capability, Capacity, Competency, and Compliance to guide data strategy and highlight improvement areas within enterprises.
Data Governance in a Nutshell
Data governance, as defined by DAMA, is “the exercise of authority, control, and shared decision-making (planning, monitoring, and enforcement) over the management of data assets” [1]. operates within businesses through three core actions: alignment, definition, and control.
In each of these 3 actions, the data governance function relies on its ability to establish metrics, extract measurements, and take action.
The Essence of Data Governance Measurement: The 4 Cs
At the heart of data governance decision-making lie four essential Cs: Capability, Capacity, Competency, and Compliance. These distinct dimensions not only steer the data strategy of the enterprise, but also pinpoint specific areas deserving of attention, investment, and enhancement.
The subsequent sections of this article will accomplish the following:
- Highlight the origin of the 4 Cs
- Provide a clear definition of each C metric in the 4 Cs
- Explore the interactions and implication of the 4 Cs
- Examine a real-world scenario illustrating the adoption of the 4 Cs
Measurement Categories
When evaluating a business function, a strategic program, or the overall business, there are three key measurement categories to consider:
- Maturity: This measures the highest level of capability achieved for a specific function or its subset. It typically covers the entire enterprise and determines the advancement or maturity level of the function. While this could be termed “Efficacy” in different contexts, in the realm of Data Management, it’s referred to as Maturity. This maturity is often scored against industry averages and placed on a defined scale.
- Efficiency: This measures how effectively resources are utilized to achieve desired results. It assesses resource productivity and aims to minimize waste. Metrics in this category could include total cost of ownership, cost reduction, or cost avoidance..
- Effectiveness: This measures how well a solution performs and adds value. It’s a measure of output quality and success in achieving objectives. Metrics in this category could include net revenue generated, cost of risk avoided, or captured market share.
Collectively, these three categories provide a comprehensive assessment of: (1) team potential, (2) task execution quality, and (3) impact of the solution.
The Two Sides of Every Team
Efficiency and Effectiveness measurements highlight a crucial duality: each team has two sides that contribute to their success:
- The Process to the Result: How the Work is Accomplished
- The Final Outcome: The Actual Work Achieved
Both aspects hold equal significance for teams and organizations. Often, the “How We Work” facet is treated as a product in itself and is entrusted to a specialized Center of Excellence (CoE) that focuses on delivering excellence. This might involve collaboration with a Scrum Master in an Agile project management context or partnering with a Quality Expert for software product quality assurance. In either scenario, the team seeks expertise to execute their roles as closely as possible to the theoretically optimal state of the business.
The diagram below provides a summarized comparison of the “How” and the “What,” representing the two sides of a team’s functionality.
Metrics Overview
The three foundational measurement categories can be further deconstructed into specific metrics. Maturity is quantified through a metric that measures the level of capability attained. Efficiency is measured by metrics that assess the capacity within a given capability, the competence of resources for execution, and the cost incurred for task completion. Effectiveness is measured by how well completed work aligns with compliance, mitigates enterprise risk, or influences other crucial business metrics.
Moreover, these metrics target distinct subsets of subjects. Capability metrics are most effective when applied to enterprise-level measurements. Capacity metrics yield optimal insights when focused on individual business units within the enterprise. Competency, Cost, Compliance, Risk, and Impact deliver the most value when they drill down into finer units of the enterprise. These subjects span across people, processes, data, and technology.
Lastly, there are shared mechanisms or tools employed to facilitate measurement and drive metric enhancement. These intricacies, in addition to subject and metric specifics, are encapsulated within the forthcoming overview diagram. Further elaboration on each metric is provided in the subsequent section.
1. Capability
Capability measures the ability (potential) to perform the work.
It represents a high level of adeptness achievable under the right circ*mstances. It encompasses the articulation of necessary expertise, resources, and materials an organization requires to carry out its core functions. However, defining capability doesn’t guarantee its deployment; the organization must possess the capacity to implement it.
Generally, periodic capability assessments are conducted at the enterprise level. A defined framework aids in measuring capability and comparing it to an industry average.
A measurement of capability, in conjunction with a overarching data strategy, should drive the data management roadmap. Although a capability metric can measure roadmap progress, its scope is often too broad to accurately measuring project success.
2. Capacity
Capacity measures the extent to which work can be executed .
Capacity encompasses the organization’s ability to deliver, and is closely linked with the potential coverage of enterprise scope. Capacity isn’t solely tied to headcount; it primarily focuses on measuring resources with allocated time and assigned priority to accomplish capability related tasks.
Measurement of capacity occurs either at the business unit level or enterprise-wide. It can be quantified by hours or the percentage of time allocated for the work.
Digital transformation, change management, and continuous improvement can enhance capacity. As resources become more proficient or adapt to a data-centric culture, business units recognize dedicated capacity for data management tasks.
3. Competency
Competency measures the acquired skills, knowledge, and expertise required for the work.
It signifies the functional adequacy achieved through possessing ample strengths, skills, and understanding to deliver what’s needed. Competence embodies the ‘know-how’ or ‘skill’ of an individual or business entity.
Competency evaluation often targets specific individuals, processes, data sources, or technological solutions. These individual assessments can be aggregated to describe a larger company group.
Similar to capacity, competency can also be improved through digital transformation, change management, or continuous improvement.
Similar to capacity, competency is amenable to enhancement through digital transformation, change management, or continuous improvement. Capacity and competency are intertwined; resources must initially possess capacity before activating, enabling, or expanding their competency. Elevated competency typically leads to increased capacity. If unused capacity isn’t redirected elsewhere, it can be reinvested in data management knowledge, fostering an efficiency-enhancing cycle.
4. Compliance
Compliance measures the adherence to stipulated required.
Compliance signifies an organization’s alignment with regulations, standards, policies, or laws. The objective is for organizations to be conscious of and undertake measures to comply with prescribed regulations or requirements.
Compliance assessment often addresses controls outlined in standards, which can subsequently be aggregated to depict adherence to specific data governance standards or overarching policies. Due to the detailed nature of compliance data collection, it can also be sliced to focus on specific teams, business units, or processes.
While specific measurement approaches may vary for each control, aggregated compliance typically materializes as a score representing the degree of compliance achieved. At its core, this score usually calculates the ratio of successful audits to total audits, accompanied by a remediation strategy to bridge gaps in the targeted compliance level.
Enhancing compliance involves establishing goals, devising alignment strategies, and overseeing progress. Special projects led by a project management office can influence improvements in compliance. General oversight by data governance ensures that compliance remains a central focus. In collaboration with the agile framework, specific data management requirements can be integrated into the Definition of Done (DoD) or Definition of Ready (DoR).
Through the integration of the 4 Cs, businesses can achieve a comprehensive perspective on the transformation of data culture fueled by investments in data management capabilities.
Each metric within the diagram interacts either in additive or multiplicative ways. As capabilities are measured and prioritized, they serves as a guide to build improved data management capabilities. As capabilities mature, teams might need additional capacity to harness the newfound capability effectively. Further resource enablement is almost always required to activate teams and adopt new processes or tools. Organization wide change management, coupled with process integration, aligns the competency linked to the expanded enterprise-level capability. The likelihood of compliance and adherence to governance standards experiences a substantial boost through investments in competency activation and ensuring sufficient capacity.
Ultimately, an enterprise will codify required data management practices in the organization standards and leverage the compliance metric to measure the desired results. The 4Cs framework empowers organizations to effectively diagnose issues and pinpoint specific areas for driving change. The outlined framework also underscores three critical misconceptions that hinder compliance with data management standards:
The Fallacy of Maturity
An mature data management enterprise might not necessarily excel in executing its own data management. Solely concentrating on enhancing capabilities without activating employees, implementing change management, and nurturing competency in data management won’t yield the sought-after outcomes. Maturity of capability doesn’t automatically translate to compliance.
The Fallacy of Competency
Conversely, a competent organization might not be inherently mature as per the outlined data management standards. While an organization might possess in-depth knowledge of data management best practices, compliance success remains elusive until these practices are aligned within a more mature enterprise capability framework.
Competency is further complicated by capacity. A few highly competent resources, though measured, won’t efficiently drive enterprise-level data management excellence. A certain threshold of capacity is essential for mature organizations, and a broader employee activation is likely required to accurately measure the organization’s competency.
The Fallacy of Capacity
Merely having a sufficient number of personnel for the task doesn’t equate to having the right individuals and appropriate tools for the job. Blindly adding more personnel in endless quantities is neither effective nor efficient.
A mature data management organization demands enhanced capabilities, superior tools, and more robust business processes implemented successfully through a robust change management program. Success also hinges on a more educated, data-literate general workforce, complemented by data professionals equipped with subject matter expertise.
As previously mentioned, capacity and competency are closely interlinked. To incorporate the right capacity, it’s essential to comprehend the skills possessed by the additional resources. An evaluation of skills allows for a better grasp of what capacity should be allocated versus what capacity needs to be cultivated through further activation, empowerment, and training.
Meet NimbusDataTech, a cloud software company embarking on its data management journey. The company has a long road before it becomes a leader in data management. But its leadership knows that with the right focus, it can become an example for all of its customers and get deeper value from its data assets.
Company Overview
NimbusDataTech is a prominent player in cloud software, specializing in innovative and trusted solutions. Their business model entails selling software through a subscription framework, requiring annual contract renewals. Throughout the customer lifecycle, they gather customer information and leverage embedded telemetry in their cloud products for insights into adoption and usage patterns. The company aspires to reduce customer churn by utilizing advanced analytics and the wealth of data they’ve amassed.
Data Landscape
At the core of the data landscape are data analytics teams, comprised of data scientists, data engineers, and data visualization specialists. These teams are the driving force behind advanced analytics, devoting substantial time to sourcing, comprehending, and cleansing data for analytical purposes. Across the enterprise, data teams function within a federated model, resulting in fragmentation. They often find themselves recruiting highly seasoned experts to fulfill project demands.
Data Strategy
The appointment of a Chief Data Officer marked a pivotal juncture, with a mandate to establish a robust data strategy and lead a transformative data management initiative. This revamped data strategy zeroes in on enhancing enterprise-wide data management maturity. Success is measured annually through maturity assessments, measuring year-over-year improvements. Yearly growth targets are selected based historical performance.
Current State
The Data Management Maturity (DMM) program’s inception dates back to 2020, when the team selected the Data Management Maturity (DMM) Model created by the CMMI Institute [3]. The baseline DMM score was assessed at 0.50 on a 5 level scoring scale established by the framework. A final DMM score target of 4.00 was selected in 2020 when the program outlined their 5years long range plan.
The Data Management Maturity (DMM) program is the main driver of improving data management practices. It has a singular focus on increasing the DMM score year-over-year. Over the past two years, the score has shown incremental growth, rising from 1.50 in 2021 to 2.25 in 2022. A new target of 2.75 for 2023 was selected, aiming to surpass the industry average. However, after two years of focused efforts to elevate the maturity score, the team is now encountering resistance and grappling with several issues.
The Problem
While the team has consistently met their Data Management Maturity (DMM) score targets and achieved planned program objectives, they are currently facing challenges in garnering interest from the broader data community and securing additional funding for further advancements. A survey conducted through interviews with their community members revealed three problems serving as the root causes of resistance:
Success Metric Misalignment: While the DMM score, a primary success metric, indicates progress and success, qualitative program outcomes remain lacking. Interview findings reveal that the teams engaged in data management work perceive minimal impact and little change in their day-to-day responsibilities. While some capabilities might be showing improvement in the best-case scenario, the median capability employed by most teams — often utilizing homegrown solutions or subsets of enterprise-level capabilities — has demonstrated minimal change over the past two years.
Business Value Realization: Despite increasing DMM scores, business unit leaders did not experience improved capability translating into business value. Across different business units, different key performance indicators (KPIs), are used to measure success. The KPIs range from increasing revenue in sales to reducing risk and cost in operations. These leaders find it challenging to discern how a higher DMM score directly affects their primary KPIs or influences their teams. The enterprise data team has struggled to demonstrate concrete business value and establish a “what’s in it for me” narrative tailored to each business unit’s leaders.
Internal Customer Satisfaction: The narrative of enterprise-level success doesn’t align with the actual experiences of federated data teams. This disconnect in perception has frustrated many within the data community responsible for daily data management tasks. Enterprise solutions often lack accompanying manuals, training, or processes to bridge understanding gaps. The enterprise data management solution offerings left the data community perplexed. As the enterprise team introduced governance controls and auditing measures, the broader data community’s response was dismissive and frustrated. Many perceived data governance as punitive due to the lack of clear expectations or guidance.
The Solution
The identified problems underscore a clear gap in the approach. While the enterprise team made commendable strides in advancing Data Management Capabilities (the first “C”) to enhance enterprise maturity, they lacked critical components essential for driving internal customer success. Moreover, their goals and strategy lacked essential metrics.
To address these issues, the team decided to adopt the 4 Cs of Data Governance Measurement. This framework mandated integrating metrics for Competency, Capacity, and Compliance alongside the existing Capability metric. They also repositioned their focus, shifting from Data Management Maturity to Data Management Measurement ( a new DMM).
With this broader measurement perspective, the enterprise team identified gaps in their process of implementing and overseeing enterprise data management capabilities. They initiated a comprehensive retooling effort.
Their revised plan retained the initial Capability-focused steps, involving data management maturity assessments and delivering a product roadmap to develop data management capabilities. However, three new steps were introduced:
- Team Measurement: The team shifted from an assessment of business units capabilities to measuring data management capacity and the corresponding competency within that capacity. A comprehensive picture of team competency was constructed, followed by a plan to address gaps in data literacy, data fluency, and process knowledge.
- Change Management: The change in measurement necessitated a larger investment in a dedicated change management function. This function encompassed business architecture definition, role delineation, process development, self-service resource creation, and enablement delivery. The objective was to ensure the enterprise recruited requisite data skills and fostered those skills internally. By taking ownership of the competency metric, they aimed to measure how well the data community understood their assigned tasks.
- Control and Audit: While this function had been emerging, it was previously separated from maturity considerations. The team recognized the significance of compliance as a critical success metric. Rather than focusing solely on achieving a specific level of maturity, they shifted their mindset to whether tasks were actually performed at the specified level of maturity. This change in perspective directed their attention to task outcomes and ensured that initiatives initiated at the enterprise level effectively translated into the tasks executed by individual contributors engaged in data management.
Conclusion
“If you can’t measure it, you can’t manage it.” — Peter Drucker [4]
Implementing the 4 Cs of Data Governance Measurement broadened the enterprise team’s scope of responsibility significantly. In many ways, they assumed ownership of the entire data management product, bearing the onus of driving adoption and success within the wider data management community.
Enhanced ownership of data management prompted a shift away from breadth towards depth. The team prioritized a small subset of specific data management capabilities and ensuring their functionality and adoption before moving on to the next batch. This concentrated approach led to greater realization of business value, with iterative releases delivering incremental value on regular cadence.
With a broader mandate in data management measurement, the enterprise governance team continued to explore additional metrics, aiding their efforts to align the business, set priorities, and exercise data-driven oversight.
If there is one takeaway, it is this: start measuring data management within your business, crafting an approach that builds a 360 view of the end-to-end process. A great starting point is implementing The 4 Cs of Data Governance Measurement.
- Capability (Maturity): Increase the potential of the enterprise.
- Capacity (Efficiency): Identify the individuals executing the work.
- Competency (Efficiency): Evaluate their skill in executing the work.
- Compliance (Effectiveness): Appraise the tangible accomplishments.
For practical recommendations on where to start, consider focusing on these three projects:
Measure Data Management Maturity:
- Select a DMM Framework
- Conduct a Comprehensive Assessment
- Establish an Initial Baseline Score
- Build a Data Management Roadmap
- Allocate Resources and Fund the “How” Projects
- Refine and Iterate on a Regular Cadence
Develop Scalable Team Capability Assessments:
- Create a User-Friendly Self-Service Tool
- Measure Specific Capabilities
- Develop Enablement to Foster Skill Development
- Mentor on Deliverable Execution
- Augment and Grow Capacity
- Refine and Iterate on a Regular Cadence
Cultivate a Data Culture:
- Measure Data Literacy Levels
- Measure Data Fluency Levels
- Develop Accelerators for Capabilities
- Activate Resources through Process & Education
- Build a Collaborative Community
- Facilitate Collaboration and Growth
In essence, these steps illuminate the path forward, promoting the systematic transformations to intensify measurement practices and ensure that capability translates into competency across the entire enterprise. As you embark on this journey, remember that success stems from a continuous cycle of improvement. So, start somewhere, iterate, and best of luck!
[1] DAMA International, et al. The DAMA Guide to the Data Management Body of Knowledge (DAMA-DMBOK Guide), Second Edition. Technics Publications, 2017.
[2] “Capability Maturity Model Integration.” Wikipedia, Wikimedia Foundation, 17 July 2023, https://en.wikipedia.org/w/index.php?title=Statistical_hypothesis_testing&oldid=1090223185.
[3] “Data Management Maturity (DMM) Model: At-A-Glance.” CMMI Institute, An ISACA Enterprise, 2019, https://stage.cmmiinstitute.com/getattachment/cb35800b-720f-4afe-93bf-86ccefb1fb17/attachment.aspx.
[4] “Measurement Myopia.” Drucker Institute, 04 July 2013, https://drucker.institute/thedx/measurement-myopia/#:~:text=%E2%80%9CIf%20you%20can't%20measure,quotations%20attributed%20to%20Peter%20Drucker.
