Monday, February 6, 2012

Why multi domain MDM has become the talk of town?

[Multi - Domain] Master Data management

My blog started with the concept of MDM explained in
Master Data Management (CDI/IR/PIM) published in 2008.

I will try to redefine some of the terms from the previous MDM blog.

Master Data: It is the information that may include data about customers, products, employees, materials, suppliers, etc. often it is non-transactional in nature. In the context of MDM, master data can support transactional processes and operations also. Business-wise the analytics/reporting of an enterprise greatly depends on its master data. Master data is often used by several functional groups and stored in different data systems across an enterprise.
Digitizing all the information has led us to a huge amount of data. The same is true for Master Data. Worldwide organizations are now aware of how easy it is to manage their customers or products if they have that information in digital form. Several data entry solutions are available and also several enterprise-level managing solutions like ERP solutions, CRM solutions, etc. It often so happens that in the same organizations we have different functional group storing and dealing with the same or similar data. It is but obvious that this leads to multiple copies of the master record in an organization.

Master Data Management: In computing, master data management (MDM) comprises a set of processes and tools that consistently defines and manages the master data of an organization (which may include reference data). MDM has the objective of providing processes for collecting, aggregating, matching, consolidating, quality-assuring, persisting, and distributing such data throughout an organization to ensure consistency and control in the ongoing maintenance and application use of this information. Thereby producing a master file which is believed to be the best version of truth or the “Golden Data”
Managing Master Data is of utmost importance because it is used by multiple applications, an error in master data can cause errors in all the applications that use it. E.g. an incorrect address in the customer master might mean orders, bills, and marketing literature is all sent to the wrong address. Similarly, an incorrect price on an item master can be a marketing disaster, and an incorrect account number in an Account Master can lead to huge fines.

DW & MDM: In the IT world we have three main categories of enterprise one is the product company (people who develop and produce the MDM tools), the Service Implementers, and the Client (The organization where MDM will be implemented). Educating the third organization about the importance of MDM was the prior task of the first two. There were a different thought process and lots of challenging questions. Clients always had this question for the MDM vendors, why do we need an MDM solution at all when we already have a Data warehouse. At that point it was very important that all should understand where Data warehouse ends, MDM begins.
In many ways, the processes are similar to those used in populating a data warehouse or datamart, particularly with the consolidation style of MDM. However, despite the close relationship between MDM and data warehousing, a glance at the recent literature reveals that these two important areas tend to be treated as separate:
Volume and Type of Data: MDM systems involve only the master data, not terabytes of transaction data (such as telecommunication companies' call detail records), which may need to be stored at the detail level, in addition to various levels of aggregation for analysis purposes.
Database Design: The master data is typically held in a relatively normalized form, whereas many analytical systems depend on specialized designs, such as star schemas to improve analytical performance.
Use of the Data: MDM reconciles semantic differences to give a single view of master data, usually for operational purposes, which is independent of any single application system. Data warehousing supports BI and analytics but does not usually support transactional activities. MDM systems and data warehouses are complementary, and the introduction of MDM systems into the operational area should reduce the work required to integrate and consolidate master data as it's prepared for loading into the data warehouse.
Two approaches to master data management are widely taken in practice: MDM for operational purposes and MDM for analytical purposes. Operational MDM focuses on ensuring that data in multiple operational systems that should be the same actually is the same. Analytic MDM is usually associated with data warehousing and seeks to leave the operational world alone, instead of focusing on compiling a view of data that can be used for analytic BI and management information purposes. Operational MDM is generally being adopted by organizations with a need to focus on ensuring consistency of master data definitions across the operational systems landscape (e.g., enterprise resource planning, customer relationship management, etc.). Analytic MDM has become established as a style of implementation adopted by businesses needing to effect a significant improvement in the speed and quality of their business reporting, often centered on one or more national, regional, or enterprise data warehouses.

About Multi-Domain MDM: The term multi-domain is very confusing at times. 
For a Client, it looks like different business domains whereas for the vendors it is actually the same business but different nature of data stored or the different types of entities, which the client organization is managing. So do you just manage the customer entity, just the product entity, or manage multiple types of entities. Sometimes this views suppliers and customers as being indifferent “domains” as they might have a different database structure. 
Multi-Domain MDM is actually a technical view of a domain-driven from a product perspective. It should have been Multi-Entity MDM. Frankly speaking, that will make life much easier for the sales reps who want to push a multi-domain MDM implementation. Otherwise from a business perspective, often the clients get confused at the proposal itself and start wondering why we need connectivity between Customer-Centric, Enterprise Centric, and Supply Centric domains.

The MDM product companies are now offering that with a single instance of MDM product one can:
• Manage all domains of Master Data as well as Reference Data.
• Implement both Operational MDM and Analytical MDM solutions.
• Support multiple MDM architecture styles: Transactional, Consolidated, Hybrid.
• Meet both B2B and B2C requirements.
• Manage and harmonize metadata.
• Manage unstructured data (such as documents and images) related to Master Data entities.

Kinshuk Dutta 

Scala & Spark for Managing & Analyzing Big Data (Using Machine Learning)

Managing & Analyzing Big Data using Apache Scala & Apache Spark In this blog we will see how to use Scala and Spark to analyze Big D...