Neglected problem: test environment configuration management

Last night, a classmate of the test exchange group asked a question , The problem is about this ：

Various configurations of the company are chaotic , Online always makes mistakes , such as API Of key, The production environment uses the configuration of the test environment . As QA, In addition to checking these related items before going online , Have you used any good tools to manage these ？

This problem has exposed many bad phenomena and problems in her team , such as ：

• Online problems occur frequently ;

• Configuration management is not standardized ;

• The production and test environments are not isolated ;

Of course , The above problems are only superficial , There are more hidden problems and bad factors reflected , such as ：

• The quality of online delivery is not high ;

• The process specification is not perfect , Insufficient landing effect ;

• QA Lack of good tools and means to carry out quality assurance ;

On the topic of quality assurance , There is too much to talk about . This article , I want to talk about the above problems , How to manage the testing environment to solve some thoughts and methods that affect the quality of online delivery .

How important is the test environment

Talk about the test environment before configuration management , Let's first look at the normal software development and delivery process , What environment should we go through ：

The local environment （host）

： Develop the environment for requirement development on your own computer or local server , As long as the part in charge can realize the established function ;

development environment （dev）

： Develop an environment in which students uniformly publish their locally implemented code for function joint debugging , General unit tests are also conducted in this environment ;

Test environment （sit）

： The environment in which most testing activities are carried out , Interface in this environment / function / performance / Automation and a series of testing activities , Implement code functions / exception handling / Build success rate / Test and verify whether there are performance bottlenecks ;

Acceptance environment （uat）

： Release the tested code , Let the product participate in the acceptance , Confirm whether it meets its expected environment ;

Pretest environment （ppe/ade

）： It is also called gray environment , Simply put, it is to release the code that has passed the test acceptance to the environment , Run for a period of time and make continuous follow-up observation , Are there any other problems or missing items （ There are also technical means to route the requests of a small number of users to the environment , To verify ）;

Production environment （prd/prod）

： The so-called online environment , That is, the environment used by users ;

Be careful , All the above environments , Basically, the following specifications should be followed ！

• Network isolation ： That is, requests cannot be accessed across environments , Especially in non production and production environments ;

• Data isolation ： That is, different environments should have their own independent data sources , In principle, the same data source cannot be shared ;

• Circulation card points ： Publish the code to the next environment node , In principle, it should meet the state or standard of circulation , You can't publish at will ;

• Other matters ： If there are multiple sets in a certain environment , Suggest sharing data sources , In this way, the cost of maintenance and change is low ;

Finish the interpretation of facing the environment , I believe some students have found the importance of testing environment .

Most testing activities are in the test environment （ contain sit/aut/perf） Develop , If the test environment is unstable , The test results cannot guarantee the quality above the level .

How to manage the test environment

Above, I talked about the impact and importance of a stable testing environment on delivery quality , In this part, let's talk about some common test environment management methods .

Change management

How to understand change ？ Change is not a simple code release , It also includes configuration changes / Server changes /DDL Change even the switch of business activities , In theory, all changes will affect the result of code running . According to statistics , Most online problems or online failures , It's all caused by changes , Therefore, change management is very important .

For changes , The commonly used means are changed review/ change check/ Change approval . On the one hand, they participate through different roles , From different angles review, Leak filling ; On the other hand, by check And the way of examination and approval , In this way, all team members can improve their awareness of change risk and quality assurance .

It's changing review and double check When , The applicant needs to explain the content of the change / scope / Solutions to abnormal situations , In this way, even if the change goes wrong , It can also be repaired in time , Control the risk within a reasonable range .

Of course , Changed review And approval process , It needs to be implemented to varying degrees depending on the environment .

The following is a paragraph of my previous work and DBA Dialogue with the person in charge ：

I ：XX bosses , I want to do testing environmental governance , Want to reduce random table structure changes , Keep the underlying data consistent , I need your help ;

DBA person in charge ： That's great , I've always wanted to do this , But when I went to promote business, I was ignored , There's a lot of resistance ...

I ： What do you think of our cooperation , I'll coordinate with business development and testing , You get the underlying technical specifications done ？

DBA person in charge ： That's all right. , Authority closing , The change of table structure will follow the work order , Reduce the frequency of changes , Standardize the database and SQL standard , I'll take care of it ;

I ： That line , You should first prepare relevant technical solutions and specifications , Let's first choose an environmental pilot , I will communicate with the business side , OK, let's start ;

DBA person in charge ： Sure , If you need any help, just say , It's good for us to do this DBA It's also a good thing ...

Share the above conversation process , In fact, what we want to express is ： Many times, we face pain points in our work , It is also a problem that others want to solve , Seek a community of interests , Work together to achieve one thing , It's easier than fighting alone , The effect will be better .

Rights management

A few days ago, it was revealed that the data of a state-owned enterprise was leaked , It is said that the reason is that some R & D will accesskey Uploaded to csdn Lead to .

This event is not evaluated here , Instead, I hope to use this event to illustrate the importance of permission management .

From my personal work experience , The best authority of the test environment is to let the test students maintain and manage themselves , Especially the release permission .

In principle, , Start with the test environment , The release authority of each subsequent circulation environment , It's best for the test students to be responsible , This convenience can avoid unauthorized publishing , On the other hand, the test students know and evaluate the release and change in time , It also helps to control risks and timely know and remedy problems . Here are a few cases ：

• There are no restrictions on service publishing
  ： Set the service publishing time window through the publishing platform , Negotiate, communicate and confirm with the R & D and test teams （ Reduce service unavailability caused by arbitrary publishing ）;

• Anyone can publish it at any time
  ： Each application or business domain application set specifies Services owner, Service publishing needs to go through owner The publishing process can only be executed after the second confirmation ;

• Information out of sync & High communication cost
  ： Establish a special service release information synchronization group , An application needs to be released , Automatic Aite corresponding owner Make notification confirmation （ It can be set to be undisturbed , But the impact needs owner Be responsible for ）;

Data management

The importance of test data is self-evident , Every test activity needs the support of data , This is also the reason why I mentioned above about data source isolation and sharing data sources in the same type of environment , It seems contradictory , In fact, it's a summary of experience after stepping on many pits .

Students with similar experience believe that they have the following work experience ：

• Table structure changes in multiple test environments , It needs to be mentioned many times DDL The repair order ,DBA Students have a large amount of tasks ;

• Suppose the test environment switches during the test , The change will be repeated , It is easy to omit or change incorrectly ;

• Even if there is a special test data embedding tool , However, multiple environments and data sources will lead to more time-consuming data preparation , Increase complexity ;

• Different environments and different data sources , When performing automated regression , test case And data may need to be modified and adapted , Time consuming and laborious ;

• Even if multiple projects are carried out at the same time , But the final release online is only a set of environment and data sources , This will lead to the online risk probability caused by frequent changes ;

In response to these phenomena , How did I do it ？

First , Select an environment as the base environment , be-all DDL Change to the benchmark environment first , Then automatically change to the data source of other environments ;

secondly ,stable Environment landing , The phenomenon of environmental divergence will gradually converge , After the cost of construction, maintenance and change is reduced , Instead, I have time and resources to do special optimization work ;

then , Change permissions and entry convergence , Through a unified work order system , Reduce the overall change chaos , All changes are traceable and recorded ;

Tools and means

The importance of testing environment and some common environmental management methods are discussed above , In fact, to sum up , The core idea is as follows ：

• All changes require check, Major changes require approval ;

• With tools and platforms , Close the change permission , Use a unified process to standardize the operation process ;

• Reduce the frequency of changes , Clarify the access and exit criteria for change circulation , Test students to do a good job in quality control ;

Other management methods

In addition to the above cases and technical solutions , There are also the following means , such as ：

• Code branch naming conventions ;

• Service unavailable subscription notification ;

• The service release notification function is online ;

• Training on environmental unavailability problems and solutions ;

• Set up a virtual group , Each domain is designated to be responsible for the testing of that domain owner;

• Integrate automation frameworks and methods of different business lines , Provide a unified technical solution ;

• Test environment access log monitoring alarm , From the service layer -DB layer , Monitor and alarm automatically and transparently ;

• Environment and service unavailability are included in the fault SLA Measurement dimension , Regularly repeat and analyze , And constantly improve ;