The increasingly high volume and lack of insight, into quality/content/context and understanding of data pose challenges for decision making and inhibit gaining more business value out of corporate information. Additionally, the lack of consensus on the relevance of information among stakeholders affects the usefulness of delivered information.
A metadata repository is the key to gather, retain, and disseminate Data, Information and Knowledge for the greater Understanding. Metadata is used to describe, find, use, and repurpose the data; publishing it facilitates data sharing. Hence Metadata is essential for maintaining an organization's investment in its data.
Metadata includes data sources/aliases/access rights/usage, business rules, data transformation rules, summarization levels, technical configurations, and much more. The critical part of its implementation is integrating components of metadata from its various sources, to be utilized by stakeholders of multiple roles.
Our consultants can help your organisation, and take responsibility of your metadata strategy and its implementation using standards, practices and tools. Our approach will also include how metadata is stored in a way people can interact with it.
An abstract representation of data is data model and the process of constructing them is data modeling. There are three different views of data - Business, Logical and Physical. For building a new database start at conceptual level as business requirements, at logical level use the design to meet those requirements and finally at physical level specification of database that has to be implemented.In other type of project where we integrate, modify and migrate existing data type we start with implemented data, extract the structure of that data to create a logical view and then derive the business context. There are different types of data models itself - Hierarchical, Network, Columnar etc. We have strong expertise on Dimensional, Relational data models, and recommend the use one or the other based upon their applicability.
We can help you build Database schemas based upon your requirements, implement and maintain their instances. We can help implement specific application program to access / modify the stored data in an easy and user friendly approach. We can also perform the Database administrative task to provide access control to the data and restrict unauthorized usage.
Metadata is "information about data", it is the background information which describes the content, quality, condition, and other appropriate characteristics of the data. Metadata defines the elements of any Information Management system, and how they work together. From Information Management (IM) perspective, Ralph Kimball describes metadata as the DNA of the Data Warehouse (DWH), and classifies it into following types, based on the source of the metadata.
- Technical metadata defines the data model of the objects and processes in a DW/BI system, hence is primarily structural.
- Business metadata is the content from DWH described in more user friendly terms. The business metadata tells you what data you have, where it comes from, what it means and what its relationship is to other data in the DWH, and enables you to generate Impact reports and Lineage reports. It also serves as documentation for the DWH/BI system. Users who browse the DWH are primarily viewing the business metadata.
- Process metadata is used to describe the results of various operations in the DWH. Within the ETL process all key data from tasks are logged on execution. This includes start time, end time, CPU seconds used, disk reads, disk writes and rows processed. When troubleshooting the ETL or query process, this sort of data becomes valuable. Business metadata and process metadata are primarily descriptive.
The critical part of the implementation of Metadata Management is integrating components of metadata from its various sources (as shown in the above diagram), to be utilized by stakeholders of multiple roles.
Our consultants can help your organisation, and take responsibility of your metadata strategy and its implementation using standards, practices and tools. Our approach will also include how metadata is stored in a way people can interact with it.
Our consultants have implemented a Common Frame Work, driven by metadata to manage pre and post DWH tasks such as file auditing/staging/archiving/transfer & notification, Job Auditing, common Error Management, Data Error/Reject Logging and job restart in DWH environment at Fortune 50 Insurance Business and at a Regional Bank.