Mastek Blog

Metadata Management - Bring Method in the Madness

19-Mar-2015 07:25:00 / by Nilesh Pandey

Nilesh Pandey

“I'm not dumb. I just have a command of thoroughly useless information,”

Calvin from the comic strip Calvin and Hobbes once famously said. This is a situation that most IT users will identify with as they try to make sense out of the reams of data they have to sift through on a daily basis whilst adding updates to the system.

What is needed is a proper handling of this information using metadata management. Forrester defines metadata management as information that describes or provides context for data, content, business processes, services, business rules and policies that support an organisation’s information systems. Metadata management ensures visibility and delivers the control that is required to manage change in a complex, enterprise data integration environment.

Why don’t you see what I see?

It is a given that most enterprise data warehouse (EDW) projects typically extend from 18 to 36 months. EDW projects are complex and supported by numerous, meticulously prepared documents, such as the traceability matrix, ETL specs and design documents. These documents are crucial and require regular updating at every phase of the project. Quite often, users must collaborate across different time zones, on different areas of the project.

Potential Problems

What most IT project managers fail to consider, are the imminent changes that can occur over the duration of the project. This includes alterations in project requirements, modifications to the data architecture during the design phase, to replacement of resources such as business analysts moving over to different projects. In such a situation, IT users tend to get a distributed view of information, which conveys a lopsided picture.

Even worse, if these changes are mismanaged, it could give rise to several issues including:

  • Rework or the project going haywire
  • Multiple versions of the truth
  • Redundant data across numerous systems
  • Source systems out of sync
  • Difficulties with delivering analytics

No single user has end-to-end visibility and ownership into the data flow of the project. Therefore, it is essential to maintain a consistent understanding within the team, as well as with the wider company stakeholders such as management and analysts. Tracking and evaluating changes and understanding their impact during the various phases is critical to the delivery of the project.

Impact on Business and IT

From an IT and business perspective, there is a strong case for building a metadata management system.

Business Drivers

  • Data/ Regulatory governance
  • Support mergers and acquisitions
  • Improve risk management
  • Process optimisation
  • Address rapid changing business needs
  • Ensure information quality

Technical Drivers

  • Lower development and testing costs
  • Reduce risk of changes and updates
  • Reduce support costs for business users

Gaining a 360 Degree View

Mastek’s metadata management solution offers business and technical users the ability to quickly find, understand and manage information that supports their daily business operations. It provides answers to the difficult questions that users want to resolve, as detailed in Figure 1.

Figure 1: What Users Really Want to Know

Users benefit from a single view of the data.

Full Audit Trail

Tracking data creators, including edits, approvals or deletions to the information, at any point in time

Data Lineage

Origination of the dataset, date and schedule, and period covered, contents, change impact, etc.

Wiki-style Functionality

Easy visualisation of data flow and management

Search and Storage

  • Ability to carry out fast and flexible advanced searches
  • Store each version of a data element using relevant timestamps

Data Enrichment and Connectivity

  • Ability by authorised users to enrich and expose data to EDW
  • Link to technical metadata
  • Management of reference data
  • Common vocabulary of business terms

Usage History

Identifies teams that have used the dataset, the purpose and the level of success achieved

Depending upon the unique IT and business requirements, customers can decide to opt for an off-the-shelf or bespoke, metadata management solution.

Topics: TechNovation

Nilesh Pandey

Written by Nilesh Pandey

Principal Consultant

Subscribe to Email Updates

Lists by Topic

see all

Posts by Topic

see all

Recent Posts