Data center concept

Data center

The overall architecture of the supporting technology in the data center

What problems will be solved by China and Taiwan

1. The indicators are inconsistent . One of the two data products contains tax , One does not include tax , Their same indicator name is sales , The result is different . When the operation faces these indicators , I don't know the business caliber of the indicator , It's hard to use these data .

2. Data duplication , Long demand response time . As demand grows , Operations and analysts continue to complain about the longer delivery time of requirements , In the face of fast changing business , The demand response time can no longer meet the business requirements for agile data development

3. The efficiency of data retrieval is low . Facing hundreds of thousands of watches , Our operations and analysts look for data 、 It's very difficult to understand the data accurately , Want to find the data you want , Make sure the data matches your needs , They often take more than three days , For new people , It's going to take longer .

4. Poor data quality . Data is often because BUG The calculation result is wrong , Eventually lead to wrong business decisions .

5. Data costs grow linearly . Data costs increase linearly with demand

The data center is the standard built by enterprises 、 Safe 、 A unified 、 Shared data organization , Support front-end data applications through data service .

How does the data center realize that all data is processed only once ？

Simply speaking , For data warehouse , We require that measures or indicators of the same granularity be processed only once , Build a globally consistent public dimension table .

To achieve the above , Two tool products are needed ：

One is shucang Design Center , In the model design stage , Force models with the same aggregation granularity , Measurement cannot be repeated .

The other is data map , Convenient data development can quickly understand the exact meaning of a table .

Several positions ：

The theme

A topic domain is a high-level abstraction of a business process , Like a commodity 、 transaction 、 user 、 Traffic can be used as a subject field , You can think of it as a directory of the data warehouse . The data in the data warehouse is generally stored by time , Usually keep 5 In the above , The data in each time partition is written by appending , A record is not updatable . Warehouse modeling

Warehouse modeling

Enmen modeling ： The top-down （ The top here refers to the source of the data , In a traditional data warehouse , Business databases ）, Based on the entities in the business and the relationships between entities , Building a data warehouse

Kimball modeling ： Contrary to enmen , It is a bottom-up model design method , Starting from the needs of data analysis , Split dimensions and facts

Because the current business changes faster , So I recommend Kimball's modeling design method .