More business leaders are becoming aware of the tremendous impact big data has on the trajectory of the enterprise organization as it relates to:
- Predicting customer expectations;
- Assisting with effective product management;
- Being available on-demand to influence top-down decision making;
- Tailoring customer service innovations by investigating shopping habits of customers; and
- Providing organizations with competitor information
However, there’s one big caveat — if your data isn’t accurate, complete and consistent it can lead to major missteps when making business decisions. In fact, Gartner estimates the average financial impact of poor data quality on businesses at $15 million per year1, which means you can’t afford to not make data quality management a priority starting right now.
What is Data Quality Management?
Data quality management (DQM) refers to a business principle that requires a combination of the right people, processes and technologies all with the common goal of improving the measures of data quality that matter most to an enterprise organization. That last part is important: the ultimate purpose of DQM is not just to improve data quality for the sake of having high-quality data but rather to achieve the business outcomes that depend upon high-quality data. The big one is customer relationship management or CRM. As often cited, “CRM systems are only as good as the information they contain”.
A Foundation for High-Quality Data
Effective data quality management requires a structural core that can support data operations. Here are five foundational principles to implement high-quality big data within your data infrastructure:
#1 Organizational Structure
IT leadership should consider the following roles when implementing DQM practices across the enterprise:
DQM Program Manager: This role sets the tone with regard to data quality and helps to establish data quality requirements. He or she is also responsible for keeping a handle on day-to-day data quality management tasks, ensuring the team is on schedule, within budget and meeting predetermined data quality standards.
Organization Change Manager: This person is instrumental in the change management shift that occurs when data is used effectively, they make decisions about data infrastructure and processes.
Data Analyst I or Business Analyst: This individual interprets and reports on data.
Data steward: The data steward is charged with managing data as a corporate asset.
#2 Data Quality Definition
Very simply, if you don’t have a defined standard for quality data, how can you know if you are meeting or exceeding it? While data quality definitions as to what data quality means varies from organization to organization.
The most critical points of defining data quality may vary across industries and from organization to organization. But defining these rules is essential to the successful use of business intelligence software.
Your organization may wish to consider the following characteristics of high-quality data in creating your data quality definitions:
- Integrity: how does the data stack up against pre-established data quality standards?
- Completeness: how much of the data has been acquired?
- Validity: does the data conform to the values of a given data set?
- Uniqueness: how often does a piece of data appear in a set?
- Accuracy: how accurate is the data?
- Consistency: in different data sets does the same data hold the same value?
In addition, to ensure these characteristics are satisfied each time, experts in data protection recommend the following guiding governance principles when implementing your DQM strategy:
- Accountability: who’s responsible for ensuring DQM?
- Transparency: how is DQM documented and where are these documents available?
- Protection: what measures are taken to protect data?
- Compliance: what compliance agencies ensure governance principles are being met?
#3 Data Profiling Audits
Data profiling is an audit process that ensures data quality. During this process auditors look for validation of data against meta-data and existing measures. Then they report on the quality of data. Conducting data profiling activities routinely is a sure way to ensure your data is the quality needed to keep your organization ahead of the competition.
#4 Data Reporting and Monitoring
For most organizations, this refers to the process of monitoring, reporting and recording exceptions. These exceptions can be captured by business intelligence (BI) software for automated solutions that capture bad data before it becomes usable.
#5 Correcting Errors
Once potentially bad or incomplete data has been sorted out by BI systems, it’s time to make appropriate data corrections such as completing the data, removing duplicates or addressing some other data issue.
