How to hand over complex legacy systems?

This article is written by huangyaming , Zhengjianing , Wangweibo , Wang Yue , Gu Hao , Gao Shuo , Sunmengyang jointly completed .

More than half of new projects , All begin with handover . The handover period varies , There are many forms of handover . No matter what , From the customer relationship 、 Team work style and other aspects , The handover period has laid the keynote for the project to enter the stable delivery or maintenance period .

2020 year 10 month ,Thoughtworks Of C The team has handed over a customer team that has nearly 20 Payment gateway system with a history of years . This payment gateway mainly provides credit card payment to enterprises in English speaking regions , Payment related functions such as savings card payment , The monthly trading volume is more than 100 million .

2021 year 1 Month begins ,C The team formally took over the daily operation and maintenance of the project . It is not only necessary to ensure the stable operation of the system , Provide 7×24 Hours On Call Support , Also respond to the needs of daily business , At the same time, ensure that the entire payment gateway conforms to the data security standards of the payment card industry （Payment Card Industry Data Security Standard, Abbreviation for PCI-DSS）.

In the process of handover , The team faces many challenges , Tried a lot of ways , At the same time, some experience has been accumulated . We will share our experience and practice through this article , Hope to help more people .

Challenge

As a long history “ Old age ” payment gateway , During the handover process, we encountered a series of challenges , It can be roughly divided into the following two categories ：

1. High business complexity

Business , This payment network only supports card payment 8 Technology , There are also credit card related security functions , Countless reports and various value-added services .

technical , All in all 100 Multiple services and 300 Multiple code bases , Deployed over 200 individual EC2 On ; The coupling between services is serious ; Many services do not deploy pipelines 、 No test environment or even source code ; It is often necessary to manually operate the production environment database to solve problems ; The operating system and software package versions are very old .

Project management , No complete and clear business and technical documents have been summarized and precipitated .

2. There are many handover contents 、 Time is short 、 The scope is not clear

Before handover , The only information the team received was 100 Names of multiple services , The content is very limited ; The handover time cycle is quite tense （ The preliminary plan is only 30 A working day ）, Not enough time to understand all the functions of the system .

practice

1. Set goals in stages 、 Establish focus

How do we generally measure the maintenance quality of a legacy project ？

• short-term ： At least be the same as the previous team . in other words , When account team members leave , The team has enough knowledge and skills to deal with online accidents and daily business work .
• long-term ： reflect Thoughtworks Something different . Business for the project 、 Know enough about technology and development history , Enough to give an improvement plan , Land in the future for a long time 、 Bring greater value to customers .

Given the complexity of the project , It is basically impossible to achieve this goal within the limited handover period . But if you stretch the timeline , Implement in stages , It is easier to make a practical plan ; meanwhile , It also maximizes the value of the handover period , Let the team work in one direction from day one .

Based on this , The team starts from the actual situation , Divide the project into three phases ：

Through the consistent understanding of the objectives of different stages of the project , It reduces the anxiety and panic of some teams during the handover period , To come up with more creative ideas , And try bravely 、 feedback 、 iteration , Achieve the objectives of each stage .

2. utilize C4 The model combs the system architecture

Usually, the problems are business problems , It is meaningless if you can not understand the services in the business process . therefore , After we hand over an independent service or several related services , Will try to use C4 Model Draw their C1(System Context Diagram) and C2(Container Diagram) Two high-level diagrams , Display the system input in a visual way 、 The dependency between the output and each service .

Practice has proved , The process of drawing pictures can help us better absorb fragmented knowledge , It is helpful for the whole team to summarize and precipitate knowledge . meanwhile , Compared with repeated explanations , Graph is a more effective language .

Some independent modules are relatively easy to draw , But different versions are involved API Payment process , We need to constantly obtain more information to improve , Repeatedly confirm with the customer . Even after the handover, some links are still not connected or there is no time to sort them out , Only after handover , As the goal of the in-depth understanding period, continue to improve .

Payment system C1 Simplify the picture （ Simplified edition ）

3. Share context within the team by pairing

During the first phase of handover , We and the customer team are “1+1” The mode of knowledge transfer , Business knowledge is scattered among members like an island . in addition , Our team has different time and skill background for each person to join the project , For some background information 、 Business context 、 There are some gaps in the mastery of technology realization .

therefore , Start at the second stage of project handover , For most of the work , We all do it in pairs . According to different business and priorities , We have divided several important topics , such as ： Tasks related to daily needs ,PCI Relevant tasks and production environment changes, etc . We will pair up through specialization and familiarity with services , Let these two people become experts in the corresponding fields in the team .

The main benefits are ： Ensure that the corresponding knowledge can be spread among the team , Eliminate knowledge silos ; Avoid that a member cannot carry out important tasks due to leave ; Important online operations can be checked by one more person .

I'm arranging Primary On Call and Secondary On Call When , take “Dev + DevOps” The combination of , Make sure you have enough skills to deal with online accidents . When an online accident occurs , Two people work together in pairs .

Although pairing can affect efficiency in the early stage , But it can ensure that at least two people in the team are familiar with the specific business , Finally, the whole team can have the ability to respond to accidents . From the present results , It is this form of pairing , It ensures the “ High availability ”.

4. Improve the team through online accident drill On Call The confidence of the

7 × 24 Hours On Call For the team , It will undoubtedly be a very big challenge . Before officially taking over the system , The team feels a lot of pressure . On the one hand, these pressures are due to the lack of On Call Practical experience of , On the other hand, because in the first stage of handover , We lack in-depth understanding of business implementation details and systems .

On Call Engineers should not only refer to the standard treatment process , It is also necessary to assess the impact of online problems in a short time and solve them accurately , So it is the best way for us to learn during the in-depth understanding period to drill with the accidents that have occurred before .

In the formal undertaking On Call Before , Each iteration will have an activity simulating online accident handling , The main process is ：

• The organizer will choose a representative accident from the past online failures to simulate , For example, it is an accident of integrating services with other gateways ;
• Team agreement 2 Hours to simulate online accidents , The organizer restored the scene , Other members conduct appropriate questioning according to their own understanding without knowing ;
• Split into two groups , Position the problem according to the existing situation , And give the solution ;
• The organizer makes a replay , Sort out relevant knowledge points .

Through the above methods , We were able to adapt quickly On Call The rhythm of . Up to now , Every member of our team has something to do Primary On Call I've learned a lot from you .

Conclusion

In the three months of handover , We continue to improve the handover methods , Finally, the project was successfully taken over from the customer team . Whether delivery Supervisor , Or the customer teams we worked with all praised our work . In the process of exploring the handover , We tried different ways to make our handover smooth , And share the practices that are helpful for handover , I hope that's helpful .

The copyright of this article belongs to Thoughtworks Owned by the company , If you need to reprint, please leave a message in the background .