Case sharing ｜ integrated construction of data operation and maintenance in the financial industry

author ： Cloud wisdom

 

pre-sale

 

Architects  Jaden Li（ Li Junjie ）

One 、 Case background

The financial industry is entering a critical period of rapid digital oriented development , Compared with other industries , The operation, maintenance and construction of the financial industry has  IT  The scale of the system is huge 、 The particularity of cross generational and complex business systems , Moreover, the financial regulatory agencies are concerned about the stability of the financial industry 、 Security has strict policy requirements , However, many financial institutions have problems due to nonstandard top-level data indicators and imperfect index system construction , Lead to  IT  The data quality of technical indicators and business operation indicators is poor 、 Weak correlation 、 Problems such as scattered scene coverage .

When formulating the phased objectives of the project, this case fully takes into account the matching degree between the real needs of each construction stage of the project and the product capacity , And the depth of enabling business scenario construction , Assist customers in sustainability with phased ideas 、 Systematically build an integrated operation and maintenance platform . Next, we will introduce the positioning of cloud smart product system in target planning at different stages of the project .

• Stage 1 ： Ability
   
  platform
   
  ,
    When formulating the objectives of this stage, we should pay more attention to how to break up the product capabilities and carry out functional reorganization , So as to create platform functions and system capabilities that meet the comprehensive needs of the operation and maintenance of the financial industry . At the same time, in the construction process, comprehensively sort out the enterprise operation and maintenance scenario data model , All aspects of the financial industry attributes are carried and precipitated through this platform , Including data governance methodology system .

• Stage two ： Platform scenario ,
    In phase I, the business scenario capability based on platform capability construction has been preliminarily reusable , The goal of this stage will focus on the construction of core business scenarios , And continue to explore how to improve the reuse efficiency of business scenario capabilities , Lay a good foundation for the subsequent continuous deepening of scene construction .

• Stage three ： Deepen scene construction ,
    This stage mainly includes the upgrading of multi center deployment architecture in different places and the improvement of comprehensive operation capability , Ensure the operation and maintenance results that have been completed in the construction of each core business scenario .

Two 、 Business scenario

（ One ）  Customer service scenario

• Business pain points
  ： When the customer service department solves the complaint work order for all business lines , You need to log in six or seven systems to complete the result query , When abnormal marketing activities are involved, the colleagues in the computer room should use the command line terminal to analyze the business log ...

• Technical difficulties
  ： There is a large amount of marketing business system logs （40+ Billion / Every day ）、 The log link specification governance is complex 、 The final consistency requirements of business sensitive data are high 、 Business scenario construction planning cycle is short （ Within a week 、 Bi weekly online ）.

• Solution
  ： In response to enterprise customer complaints, business data queries and other related scenarios , Connect all data sources through the products of the cloud intelligence unified acquisition and control center , Use cloud smart operation and maintenance data platform products to provide streaming data computing and data warehouse capabilities to complete data governance , Use the cloud smart form low code product to design the business query interface in a visual drag and drop way , Finally, it has been launched in the production environment, including but not limited to remote transfer 、 Living expenses, etc  50+  Customer service query scenario , It meets the one-stop analysis of users' whole scene behavior by the customer service department , Conveniently query the business data involved in the work order , Solve and close the work order as quickly as possible , Reduce work order circulation time , Reduce the workload of the customer service department while improving consumer satisfaction .

（ Two ）  Business monitoring scenarios

• Business pain points
  ： Because the full dimensional insight of marketing activity data is limited by the inefficient traditional big data division mode , The results of all major marketing activities have been impacted by the behavior of the wool party , Less means in advance 、 The reaction is slow 、 Afterwards, I sighed again .

• Technical difficulties
  ： Streaming data has multiple streams and complicated logic （ Pre decryption is used to associate spoons 、 Post desensitization is used for compliance storage ）、 The whole process of streaming data processing requires high timeliness （ Near real time  <= 4min）.

• Solution
  ： With the help of the streaming computing power of cloud smart operation and maintenance data platform products , The abnormal behavior process of users in the analysis of marketing activities by the operation Department is precipitated into a set of task arrangement of normalization of business abnormality monitoring , For example, the path monitoring task of ticket verification in different places 、 Wechat single free arbitrage trend analysis task .

（ 3、 ... and ）&nbsp; Operational analysis scenario

• Business pain points
  ： Customers face dozens of hundreds of business trading systems , It is difficult to judge whether the operation indicators and technical indicators of each core business scenario are normal based on the monitoring strategy configured by human experience , Furthermore, it lacks the means of similar business monitoring templates to uniformly manage similar business scenarios 、 Operation monitoring analysis , For example, business transaction quality monitoring 、 Business transaction trend analysis and other scenarios .

• Technical difficulties
  ： Intelligent algorithms are difficult to be flexible 、 Stable 、 Accurately learn according to the data characteristics of each core business scenario and automatically generate monitoring thresholds （ Dynamic baseline 、 Frequency domain analysis 、 Automatic threshold, etc ）、 The granularity of log pattern recognition varies dynamically .

• Solution
  ： Design business theme monitoring Kanban by dragging and dropping visual controls freely through cloud intelligent monitoring center products 、 Use the algorithm Laboratory of the cloud intelligent algorithm center product to train algorithm generics 、 Automatically mark noise , Train algorithm generics that adapt to the characteristics of customer business data , At this stage, it has been launched  1000+  Data analysis model support  110+  Business theme operation analysis scenarios .

3、 ... and 、 Functional framework

The overall functional framework of this case can be divided into data platform layer from bottom to top 、 Data governance layer 、 Functional services can 、 There are five levels in the business scenario layer , The functions of each level are as follows ：

• Data platform layer
  ： It has the ability of remote Multi Center disaster recovery ; It implies the data access layer that has been built in phase I and phase II of the project , Support real-time synchronization of production environment business database sensitive data 、API/RPC  Mode interface type data 、 similar  Kafka  Message queue data, etc ,  Manage the state of data flow in the process of data access 、 Traffic monitoring 、 Comprehensive data quality standardization inspection such as data quality monitoring .

• Data governance layer
  ： Data cleaning of real-time access business data flow 、 Data blending 、 Data compression and other data quality standardization processing , Ensure that the business data written to the platform meets the data quality specification requirements of the data warehouse . The real-time data warehouse system is layered into the data paste source layer （ODS）、 Data warehouse layer （DW）、 Data application layer （ADS）, The data paste source layer （ODS） It is the surface layer of mirrored data without any data processing after the data source is connected to the platform , Data application layer （ADS） It is the finished product data that can be consumed directly for business scenarios ;

• Functional service layer
  ： This layer includes an index system based on data governance 、 Business engine based on business link monitoring Kanban management 、 Integrated  7  Class algorithm operation and maintenance scenario  24  Intelligent engine of intelligent algorithm type （ Unified algorithm Center ）, And alarm engine .

• Business scenario layer
  ： The operation side includes customer service query 、 Transaction quality analysis 、 Product map management 、 Business engine and other scenario modules , The operation and maintenance side includes business indicator monitoring 、 Log pattern recognition 、 Log anomaly detection and algorithm automatic recommendation 、 Expert annotation and other ability modules .

Four 、 Case summary

Cloud intelligence, as a leading service provider of full stack intelligent business operation and maintenance solutions in China , We are committed to exploring the intelligent upgrading of business operation and maintenance , Take the full scenario intelligent business operation and maintenance solution as the starting point , Ensure customer business continuity 、 Promote customer service innovation .

The platform in this case goes through three stages of continuous iterative upgrading , Successfully broke the barrier between global data , It makes up for the weakness of traditional big data platforms in real-time data analysis , It has been widely used in many business departments and received good word-of-mouth feedback , The success of this case represents, in a sense, a major breakthrough in the construction of the integration of operation and maintenance in the financial industry . meanwhile , The scenario capacity-building of the platform also provides more imagination for the future operation and maintenance work , Through multi-dimensional analysis and mining of global data, it is more possible to activate the innovation of the integrated operation and maintenance mode , Help customers continue to maintain the competitive advantage of the industry leader .

Open source benefits

Cloud intelligence has become an open source data visualization platform  FlyFish . By configuring the data model, it provides users with hundreds of visual graphics components , Zero coding can achieve a cool visual large screen that meets your business needs .  meanwhile , Flying fish also provides flexible expansion ability , Support component development 、 Customize the configuration of functions and global events ,  Facing complex demand scenarios can ensure efficient development and delivery .

Click the address link below , Welcome to

&nbsp;

FlyFish

&nbsp;

Like to send  Star. Participate in component development , There are ten thousand yuan in cash waiting for you to get .

GitHub  Address ：&nbsp;

https://github.com/CloudWise-...

Gitee  Address ：

https://gitee.com/CloudWise/f...

Super experience officer activities ：

&nbsp;

http://bbs.aiops.cloudwise.co...

Ten thousand yuan cash activities ：

&nbsp;

http://bbs.aiops.cloudwise.co...

Wechat scanning identifies the QR code below , remarks 【 Flying fish 】 Join in AIOps Community flying fish developer exchange group , And  FlyFish  project  PMC  Face to face communication ～