An article to understand data warehouse: metadata classification, metadata management

Catalog

​ ​1、 Business Metadata ​​

​ ​2、 Technical Metadata ​​

​ ​3、 Management metadata ​​

​ ​4、 Xiaobian has something to say ​​

---

1、 Business Metadata

1. describe ” data ” The business implications behind it
2. Topic definition ： Each paragraph ETL、 The business subject behind the table .
3. Business description ： The specific business logic implemented by each piece of code .
4. Standard indicators ： Be similar to BI Semantic layer in 、 Consistent facts in the data warehouse ; Standardize the indicators in the analysis .
5. Standard dimensions ： Same as Standard indicators , Standardize the definition of each dimension of analysis 、 Standardization .
6. Maintain continuously and communicate with the business party for confirmation .

2、 Technical Metadata

• Data source metadata

• for example ： Data sources IP、 port 、 Database type ; The way data is obtained ; The structure of data storage ; Definition and of each column of the original data key Refers to the corresponding value .

• ETL Metadata

• according to ETL Different purposes , It can be divided into two categories ： Data cleaning metadata ; Data processing metadata .
• Data cleaning , The main purpose is to eliminate dirty data and standardize data format ; Therefore, the metadata here is mainly ： Of each table and column " correct " Data rules ; Default data type " correct " The rules .
• Data processing , For example, common table input and table output ; Unstructured data ; Splitting of special fields, etc . Source data to data warehouse 、 Various rules of data mart layer . Like content 、 clear 、 Data refresh rules .

• Data Warehouse Metadata

• Description of data warehouse structure , Including warehouse mode 、 View 、 dimension 、 Hierarchy and the location and content of the data mart ; Business system 、 The architecture and pattern of data warehouse and data mart .

• BI Metadata

• The algorithm used to aggregate 、 Including various metrics and dimension definition algorithms . Data granularity 、 Subject areas 、 Gather 、 Summary 、 Predefined queries and reports .

3、 Management metadata

Management related , Including management process 、 Personnel organization 、 Roles, responsibilities, etc .

 

4、 Xiaobian has something to say

In daily work , Metadata management is mainly reflected in metadata collection 、 Storage 、 Inquire about 、 Several aspects of application . In principle, we should Normalization , To scripting , To tools Construction in the direction of .

• collection ： Metadata collection should be as detailed as possible , real , It can be generated by tools or checked , Avoid problems such as nonstandard manual entry
• Storage ： Store metadata without distortion , Synchronize metadata changes in time
• Inquire about ： Through web pages or library tables , See metadata easily and quickly , Assist in development
• application ： Data consanguinity 、 Optimal scheduling dependency 、 Data governance, etc