Boast about Devops

One 、 The limitations of agile drive devops Be born

Limitations of agile ： Agile only focuses on agility in the development phase , It does not involve other links of the whole product life cycle process, resulting in no obvious effect after adopting the agile development process .

devops Become the booster of enterprise digital transformation , Play the role of infrastructure , By accelerating business innovation , Rapid deployment , Finally, ensure the stable operation of the business system .

Domestic companies have begun to use self-developed or open source tools devops, But the current devops Still more basic , Start with automation , Will be compiled manually , pack , Deployment and other links have been transformed into an automated mode . Most of them only stay at the primary stage

Why do most companies devops All stay in the primary stage ？
　　1、 There is a shortage of talents
　　2、devops Your skills need to be improved , Most professional engineers are transferred halfway , Such as operation and maintenance , test , Only familiar fields , But I don't have a deep understanding of the whole process （ In my current company , It is done by some testers devops　　　　 Platform development ）
　　3、 Lack of learning materials from the overall perspective

Actually devops An idea and thought of , Apply modern science and technology to the end-to-end value chain , Through to the enterprise Culture , technological process , and The system The transformation of has achieved greater success

End to end yes devops The main line
devops The main idea is continuous improvement

 

Software development process in the new era --devops

Then how can we do well devops, There are three major principles
　　1、 Take a holistic view
　　2、 Start with the beginning
　　3、 Keep improving

The current market competition is fierce , Enterprises expect to put their products into the market faster , therefore devops It is a topic that every enterprise can't get around

 

Two 、devops The history of development

Lean software development and agile software development are devops The foundation of development ,devops Not subordinate to agile , It is a development method improved on the basis of agile

Lean and agile
　　　　 Lean thinking comes from 20 Toyota in the 21st century , A system that absolutely eliminates waste

　　　　 Jingti is now pursuing the quality of products , Keep improving
　　　　 The benefits are reflected in the low cost , Less consumption , Enterprises can gain profits

 

Waste things

　　　　 stock --- Represent in the software development process ： Unfinished work
　　　　 Over handling --- Represent in the software development process ： Unnecessary process , Documents that no one reads
　　　　 Overproduction --- Represent in the software development process ： Additional features
　　　　 transport ------- Represent in the software development process ： Knowledge loss of information transmission
　　　　 wait for ------- Represent in the software development process ： Delay , Waiting for decision , Wait for resource allocation , Release
　　　　 Move ------- Represent in the software development process ： Task switching
　　　　 defects ------- Represent in the software development process ： defects ,bug

Through the value flow diagram, we can realize whether the carding process in the development process is valuable , valuable , Worthless , Worthless but must

 

agile 2001 Snowbird Conference , Announce agile , Establish Agile Alliance

Key points of agile development
　　 Developers and business people work together , Be happy to face the changes in demand , Even in the late stage of development , Value for customers . Respond to change through agility

　　 Reduce the batch size of each iteration delivery , Short lead time delivery , Often deliver working software , commonly 2 To 4 Release a version this week , Customers can use the product earlier , Developers can also respond to customer needs faster

　　 Agile only involves the software development stage , Other processes do not involve , So the feeling of enterprises is unclear

devops Through the end-to-end software development process , Integrate development and operation and maintenance , Ultimately efficient , High quality delivery of user value , Win the market for enterprises

 

3、 ... and 、 Impact map ： A sharp tool for solving product planning and requirements

End to end ： The whole process from requirement proposal to requirement release to production environment delivery to users

Best practices ： It refers to , Proven effective ways and means

Use impact maps to ensure ： Product planning , The general direction of milestone planning and user analysis is correct

Why product planning
　　 In the process of software development , How many functions do users really need
　　 In the process of software development , Are there any functions and products that are cancelled halfway

 

The software industry is an industry with a low success rate
1994 year , The success rate of the software project is 16%（ Statistics 8000 A project ）
2004, The success rate of the software project is 29%（ Adopt agile and lean agile ideas , Statistics 40000 A project ）

 

Why is the success rate of software projects so low ？
　　1、 Do not understand the real needs of users

　　2、 Pay more attention to the software function itself , Ignoring the business objectives that need to be delivered ,

　　3、 Lack of overall knowledge and understanding of products

 

Therefore, it is very important to tap the real needs of customers , So we need to find a tool to tap the real needs of customers

 

Impact map ： It is a technology of strategic planning , Influence the map by answering why who how what To output thoughts leading to

　　why Why do you do this , Why develop this feature . Answer certain business goals , Don't be unrealistic , accord with smart principle

　　who Achieving this goal has something to do with those people or roles , Including positive people and negative people or roles , The main character minor character Off field role

　　how Consider how these roles affect the achievement of the ultimate goal , Consider how roles help or hinder the achievement of goals

　　what Discuss what can be done to achieve this impact , Select valuable departments , Finish first

Influence the characteristics of the map
　　 structured ： The first level is the goal , The second level is the role , The third layer is the influence , The fourth level is the deliverables

　　 entirety ： From the target , To the role Tree like attempts to deliver

　　 Collaboration ： A bridge for all roles to communicate and assist

　　 dynamic ： It needs constant iteration and updating , Constantly improving . Impact maps are value based

 

Four 、 A user story ： A powerful tool for reaching agreement on user needs

For a user's needs , product Development Testing has a completely different understanding of requirements , The final product delivered is also , Not what customers need

 

A user story , A tool for visualizing user needs

 

Software development is a team project , It consists of one function after another , Throughout the development process , The function is like basketball in basketball match , Pass from the hands of the players , Finally, the shooting is the most accurate

People input , The common goal of every team member is to throw the basketball into the basket

 

In the software development process , The product manager analyzes the functional requirements , Developers implement the function code ,

The tester tests the function , Finally, the operation and maintenance personnel deploy the functions to the production environment , Deliver to customer

A common goal ： Deliver functionality to customers

Team assistance it is not easy to achieve efficient collaboration

 

 

The way to reach consensus ： Let every role in the software development process understand what users' needs are , What's the job

that , Can writing documents solve the problem ？
　　 Documents cannot reach a consensus

　　 People habitually think within the scope of documents according to their own experience , It is inevitable that the content is not well understood , Cause a misunderstanding

　　 Looking at documents is like looking at photos of tourism , It has something to do with your mood at that time , Different people see different final results

　　 The only way to reach a consensus is for all relevant parties to sit together , Discuss in an effective way , Finally, seek a consensus understanding .
　　 The effective way is to tell user stories

A user story From the perspective of users Talk about how users use , Let everyone understand the same , In order to make software more in line with user needs

How to tell user stories well ？
Storytelling is the core point

　　1、 Focus on the panorama ： Split a large demand reasonably , The split requirements are called user stories , need
　　 From the perspective of the whole , Understand the product being developed with the team , Discuss the positioning of each story in the whole product
　　, priority , Weigh together


　　2、3C principle ： Proposed in extreme programming
　　 card , Write down the desired software features , It is the carrier of user stories , Record the functional features of the product that each user wants

　　 conversation , Get together and have an in-depth discussion on the software to be developed , Ask relevant personnel to discuss the user story in the card
　　 Each participant needs to ask questions , Others answered . Let participants reach an agreement on the understanding of the user story

　　 confirm , Confirm the completed characteristics （ Acceptance conditions ）, Consider how to judge that the user story has been completed , Clear acceptance
　　 Conditions

　　3、 Use story templates ： Unify the language of team communication , It can improve the effect of discussion

 

The following template can be used to explain user stories

As xxx role . I need it xxx features , To facilitate the resolution of xxxx problem
Discuss the user roles , Different roles have different demands for software
Discuss the functions to be done , Not just about the function of the page , Also discuss the logic behind , Implied demand
Discuss the value of functionality
Discuss anomalies , And the solutions after the emergence
Discuss the development cycle

Software development is a team project , Communication and coordination in the team is very important

 

5、 ... and 、 Kanban method ： Expose problems and methods for continuous improvement

The purpose of Kanban method is to make ： Workflow Visualization

Limit the number of work items in progress by limiting the number of cards in each cycle , As long as the team has the processing ability
To pull in new task items

 

Kanban also comes from Toyota in Japan

The number of cards codes the maximum number of tasks the team can handle

In order to solve the problem of team overload , You need to visualize the work of team members

Kanban method includes visual workflow , Define process operation rules , Pull system to limit the number of WIP ,
It is a management method to achieve on-time delivery on the premise that team members are not overloaded

Proceed from the current situation , Progressive type , Incremental feature improvement methods

How to use Kanban method
1、 Value stream mapping , A series of related actions that need to be implemented from responding to customer requests to create value to completing the request and delivering user value . If you want to deliver user value faster , You need to know the operation of the whole process , Value stream mapping
It can help understand the whole process .
There will be three data in each link

　　LT： Lead time or delivery time , It refers to the time consumed by the work project from the beginning to the end , Including waiting time
　　PT： Processing time refers to the time for processing the work item
　　%C/A： Percentage of completeness and accuracy , This value represents the rate of rework , Generally, it is estimated value


Using Kanban method, we can find bottlenecks that affect value delivery

2、 Define the type of work item
Demand centric work items , Including demand , Features and user stories

Development centric work items , Including refactoring , defects , Optimization, etc.

Work items centered on operation and maintenance , Including expansion , Backup , fault

Other transactional work items . Including training , meeting


3、 Visual design of Kanban

Abstract the steps into analysis , Design , Development 、 test 、 Release key links , You can design the work items that need to flow through
Kanban columns


Each column is divided into ongoing , complete


Visualization of team success work items , Visualization of the end-to-end delivery process , Visualization of system bottlenecks and problems


Is the most basic practice , Expose the problem


4、 Realistically define rules

Define the rules , For example, what conditions need to be met for the work item to enter the next stage

5、 Limit the quantity of reprocessed products . It is necessary to specify the quantity of WIP in each link

WIP is the number of task items in progress or waiting , When the quantity is less than the limit, new task items can be pulled


6、 Track work items

In order to smoothly deliver high-quality user value

There is usually a standing meeting . Check the completion of the work against the Kanban , Look at Kanban . Track work items , Promote value flow

Focus on Backlog of demand The need for interruption Hindered demand Requirements that are about to expire or have expired have not been completed

The need to stop for a long time

7、 Continuous improvement

Small batch Short period The team should collect users' feedback on the software as soon as possible , Continuous improvement

Deming's can be used PDCA

plan perform check / Inspection summary Handle , Summarize and analyze the output of the previous step

 

6、 ... and 、 Non functional requirements ： Key factors affecting user satisfaction

How to effectively focus on non functional requirements

according to FURPS+ Requirements classification model , Requirements are divided into the following categories
　　 Functional requirements
　　 Ease of use needs
　　 Reliability requirements
　　 Performance requirements
　　 Supportability requirements

 

 

Nonfunctional requirements ： Standards for system delivery , Difficult to define , Measure , test , And tracking , It is generally described as a system
How to provide functions （how）

Functional requirements are generally described as （what）

Functional requirements are only visible icebergs at sea level , But below sea level , There are many non functional implicit requirements

 

The person in charge of the product should pay attention to the needs of the whole product , Pay attention to the development of non functional requirements from the source , Teams can really deliver working software

In the process of software development , The implementation of clearly defined non functional requirements is the key to evaluate the success of software

Nonfunctional requirements Yes System architecture , System delivery time The total cost Testing strategies have an impact

Non functional requirements should be included in the unified process management in the requirement management stage
Specific tasks need to be created to manage these non functional requirements

 

The realization of non functional requirements is costly , There may be exclusion between them , In the actual development process , Will be based on
The difficulty of realizing non functional requirements , Make Tradeoffs

How to analyze non functional requirements （ Four level non functional requirements analysis ）

Any function can be described as the following sentence
Every 《 System 》 Provide 《 function 》 to 《 party 》
Every 《 function 》 Must satisfy 《 Conditions 》 To satisfy 《 party 》 The needs of

Functional requirements indicate what the system should do
Non functional requirements indicate what the system should do

Analysis steps
　　1、 Identify stakeholders in the system
　　2、 Based on the knowledge and experience of developers, set functional goals from the perspective of stakeholders
　　3、 Disassembly function is sub function
　　4、 Identify non functional requirements in each subfunction

 

Nonfunctional metrics （ Be measurable ）
　　 throughput （qps,tps）
　　 reliability
　　 Extensibility

Design principles of non functional requirements

　　SOLID Principles and IDEALS principle

Testing of non functional requirements
　　 Defining benchmarks
　　 Defining a scene
　　 Define success and failure

　　 Monitoring of non functional requirements （ Monitor the time consumption of the complete call link for each request ）

 

7、 ... and 、 Code pre check ： How to provide code warehousing quality

Check the code before it is submitted to the code warehouse , Including static inspection ,Code Review, test , compile
And so on , The purpose is to ensure the quality of the code submitted to the code base

Local inspection Generally, it is checked locally by developers

Local submission Submit the code to the local warehouse for inspection

Remote submitted Check the code when it is submitted to the remote warehouse

Branch merging Check when the code merges from one branch to another

Why do you want to do code pre check ： Software is the core of business development , High quality software can bring better income , Win the market

The start-up function can appropriately reduce the quality of code

 

Readability ： Code is not just for machines , It's for people to see
Maintainability
Reducing technology debt

Practice of code pre checking
Quality is not a behavior , It's a habit

Continuously improve in a circular way

1、 Practice of local inspection
　　 advantage
　　　　 The lowest cost of finding and fixing problems
　　　　 The most efficient

　　 shortcoming
　　　　 Developers need to have a strong sense of consciousness

　　 Tools
　　　　SonarLint FindBugs CheckStyle,PMD, Alibaba standard plug-ins
　　　　 Local building maven compile ,gradle compile
　　　　 Local testing unit testing , Check the logic of the code

2、 Local submission check （ perform git commit）
　　 advantage
　　　　 The timing is right
　　　　 More efficient
　　　　 be based on git Of hook Mechanism , It can automatically identify simple problems

　　 shortcoming
　　　　 Only static checks can be performed
　　　　 Similarly, the consciousness of developers


　　 Tools
　　　　pre-commit


3、 Remote commit check （git push'）
　　 advantage
　　　　 Deep examination can be carried out
　　　　 Mandatory inspection can be achieved
　　　　 You can join the manual review process
　　　　 You can control the code quality of each submission
　　 shortcoming
　　　　 The timing is later , Long feedback period
　　　　 You need to build a code server
　　　　 Maintenance costs are high
　　　　 Check every submission , Just slow down the pace of development


　　 Tools （ Consider doing incremental checks ）
　　　　SonarQube
　　　　 Manual review
　　　　 automated testing （jacoco）

 

8、 ... and 、 Technical debt ： Borrow and return frequently , It's not difficult to borrow again

How to effectively manage technical debt

What technical debt
Due to the pressure of delivery , There are the following problems
　　1、 The document does not write , Or it does not match the current version
　　2、 The architecture design only meets the needs at that time
　　3、 There are many code fragments left in the code to be optimized
　　4、 Legacy code is missing documentation , Missing unit test , No one can change , No one dares to change

The concept of technical debt
It refers to those who don't do it now , But if you don't do it all the time , Things that will affect the healthy development of software , Such as outdated architectural design , For example, the code that needs to be refactored , But unrealized demand is not technical debt



How does technology debt arise
　　 Production stage
　　 Consciousness stage
　　 critical point ： Has hindered the normal development of software
　　 Remedy ： Pay off the debt , The software has entered normal development

Cause of occurrence
　　1、 It's about business （ Pressure of time and cost , Business objectives do not match , When the requirements are not clear, the main non functional requirements ）
　　2、 Context changes （ Changes in business context , Technology changes, software and hardware middleware are constantly changing , Natural evolution ）
　　3、 Development process （ Invalid documents, key documents are still necessary , Insufficient Test Automation , Process mismatch ）
　　4、 The team （ Lack of experience , Distributed teams ： Communication problems and coordination lead to inconsistent understanding , Non exclusive team ）

Principles and practices for managing technical debt
Managing technical debt is not a one-off activity . He is part of the software development process
　　1、 Avoid technical debt
　　　　 People with rich development experience and ability participate in the project
　　　　 People with rich experience and ability control the progress and quality of the project
　　　　 Establish systematic business knowledge and skill training , Improve the overall strength of the team

　　2、 Discover technical debt in advance
　　　　 Unit test framework , Discover through unit tests
　　　　 Tools for static code analysis
　　　　 Continuous integration

　　3、 Repay the technical debt as soon as possible , According to the size of technical debt , It's easy to be in a hurry
　　　　 Immediate repayment of technical debt
　　　　 Mark technical debt

 

Nine 、 Configuration Management ： A sure way to achieve one package in the end

Configuration Management ： Use the same deployment package in each environment , Manage the differences of each environment through configuration items

What is configuration management

Originally referred to as version control , Now refers to a process , Through this process, all products related to the project and the relationship between them are uniquely defined , modify , Storage and Retrieval

The application is made up of software code , The run data , And configuration information

How configuration information is described and stored
Store in the form of key value pairs
Store through configuration file , At present yaml Is the most used form
Can be stored in a database , Version control library , File directory , environment variable

 

Ten 、 Environmental management

How to quickly deliver a set of available environments ---- Environmental management

 

Environment refers to all the resources and configuration information required by the application to run
　　1、 Server hardware information
　　2、 Operating system information
　　3、 The information of middleware needed by the application to run
　　4、 The version of the application itself , Data and configuration information

Environmental management is the process of preparing to deploy the environment and the control of the environment after deployment , It can ensure the preparation of the environment
Fast and consistent , It also enables the deployed environment to be effectively utilized

Why should we do environmental management
　　1、 Manually deploying the environment can cause many problems
　　　　 It's easy to make mistakes
　　　　 Very inefficient
　　　　 It is not convenient to do test verification

In general , Physical server of the environment , The operating system will not change often , What changes is only the upper level
Applications and dependent middleware services . At this time, you can completely take git Configuration management to quickly create
Environmental Science

Design idea of deployment platform
　　git Warehouse （ Deployment scripts ）----》CMDB（ Configuration management database ）----》 Deployment platform ----》 Environmental Science

Git Warehouse
Infrastructure is code , It is the method of infrastructure automation based on software development practice , By checking the code
Make changes , Achieve consistency and repeatability in building and changing infrastructure . And through automated testing
Practice verifies the availability of infrastructure

advantage
　　1、 Reusability
　　2、 Uniformity
　　3、 transparency , Everyone can look at the code

Configuration management database （CMDB） Used to store information related to hardware and software
How to model to include the elements involved in the environment , And the relationship between elements
How to enter data ？ Can we automate , Can I customize the query interface
How to integrate with deployment scripts , Get what data

CMDB Data preparation , A lot of work
　　1、 Automatic scanning
　　2、 Manually enter

Deployment platform
　　1、 Clone the script of the deployment code
　　2、 call CMDB Interface
　　3、 Execute deployment script

 

 

11、 ... and 、 Continuous integration

Through frequent submissions , Integrate the work content of team members many times every day . Through the automated test platform
Verify whether the integrated system is available , Problems found this morning , Fix the problem , Keep the software under development
Working state

 

What is continuous integration
1、 Every time someone submits code, they integrate it


Why continuous integration
Software has a feature . Before the development is completed , Applications cannot be run for a long time , It's usually
Early development of their respective functional modules , Finally, integrate the functions together in the integration stage , To test
　　1、 Due to long-term development on their respective branches , This leads to a large number of conflicts when merging branches in the integration phase , Can't merge
　　2、 Due to no previous integration, the integration phase takes too long or cannot be integrated at all
　　3、 Since no tests have been carried out before , After the system integration, it is found that it does not meet the requirements


The solution is continuous integration ： There are two main types of continuous integration
　　1、 Instant integration ： Team members test after each submission , And perform the compilation , Build automated tests and other tasks to check whether the code submitted by individuals is available

　　2、 Timing integration ： Similar to daily build , Automatically execute an integration process at a specified time

Current continuous integration mainly refers to instant integration

Workflow
　　1、 Developers submit code to the code repository in the local workspace
　　2、 The version control system passes webhook And other mechanisms to notify the continuous integration server in real time
　　3、 Continue to integrate the server to clone the latest code to the local server , Or dedicated server
　　4、 Execute the build script on the continuous integration server or dedicated server , Check the latest code , Compiling and constructing 、 Code static scan 、 Code dynamic scanning 、 Unit test, etc
　　5、 Run end automatic production report
　　6、 The implementation report is notified to the stakeholders by email

Version control system is used to store all content related to software . Such as application source code , Database script , Build script , And deployment scripts , Is based on git Distributed version control system

For application continuous integration practice , Version control system provides basic version control capability

To improve the efficiency of continuous integration , When the code is submitted , Real time notification mechanism will be adopted , At present, the version control system provides a version based on webhook Or message based notifications i Mechanism , When different types of events occur , The registrant will be notified

 

Continuous integration tools , at present jenkins More commonly used
How to trigger a build
1、 Time to build
2、 When submitting code to gitlab Trigger a build
3、 Polling method SCM


automated testing
　　 The purpose of continuous integration is to find problems in the code in advance , Make sure the software works

The following three tests need to be added to the continuous integration species
　　 unit testing
　　 Integration testing
　　 The acceptance test


For continuous integration , Requirements for developers
　　1、 Commit code frequently , At least once a day , You also need to merge the trunk branches
　　2、 Each time you need to submit a complete task , Don't submit half of a function
　　3、 Repair immediately after the build fails

 

Twelve 、API management ： Application programming interface

It's about putting api Life cycle of ,api Registration of , Use , Version management , Provide unified

api Control platform

At present commonly used api The architectural style of is REST API ,REST Depends on statelessness , It can cache the communication protocol with client and server , such as http

REST Principles
　　1、 Unified interface description
　　2、 Both client-side and server-side should follow a unified interface
　　3、 No state
　　4、 Cacheability
　　5、 Layered system

structure RESTful api Best practices
　　1、 Use RESTful url Design API
　　2、 Use HTTP The verb to execute a resource CURD operation
　　3、 At present HTTP Verbs cannot be mapped to operation time , Use URL The operation
　　4、API Version of （ stay url The path includes version information ）

API Life cycle management of
　　api establish
　　　　1、API The creation of , be based on api The generator generates api, This method is mainly used in the initial stage of the project
　　　　2、 Generated from code scanning api, This method is mainly used in the middle and later stages of the project

　　API Test of
　　API The interface logic of
　　API Documents
　　API Performance of
　　API The safety of the



openapiif You can compare each submission api The difference of

API Release
　　1、api The version should not destroy any existing clients
　　2、 Change as little frequency as possible api Major version number of
　　3、 Backward compatibility , Avoid generating new api Version of
　　4、api The version should not be linked to the software version

management api Version of （ stay url Specify , stay http Of header It is specified in , For example X-API-Version Medium header identification ）

API It is a very critical part of the current era of micro Services , Connect the various components of the software , Form an overall external service , Yes api The effective management of can also speed up the development of software and improve the quality of software

 

13、 ... and 、 automated testing ： The only choice to improve the efficiency of testing , Only as comprehensive as possible test coverage , To reduce the probability of software errors

Automated testing is nothing ？
It doesn't mean automatically generating test code , Instead, it automates the execution of test code written by developers or testers

Never do anything manually that can be automated

What is not suitable for automated testing ？
　　1、 User experience testing
　　2、 Exploratory testing

Why automated testing
　　1、 Speed up the test , Shorten the test time
　　2、 Enough time , There are more functions
　　3、 Save time and reduce execution costs
　　 In the whole life cycle of software development , Testing is a frequent and repetitive activity , Every time you submit code, you need to test it to ensure that the changes of new code will not be affected , Before each software release , System regression testing is also required , Once automated testing 　　　　 The construction is complete , You can run unattended , Even on multiple machines
Parallel execution


　　4、 Reduce the probability of error , Improve accuracy
　　 Automated tests perform the same steps every time they are executed , And a detailed test report will be generated every time ,
　　 The test report is not affected by human factors , Manual testing is easily influenced by personal experience and emotions , Error prone ,
　　 The mobility of personnel makes the test knowledge unable to precipitate

　　5、 Improve test coverage
　　 Automated testing can increase the depth and scope of testing , So as to improve the quality of software


　　6、 It can speed up the efficiency of feedback
　　 Automated testing occurs after each code submission , Automatically trigger and notify the developers in the team of the test results , Greatly shorten the time for developers to get feedback

　　7、 It can simulate the test scenario that cannot be completed manually
　　 For example, the scenario of concurrent access

　　 therefore , Automated testing through rapid batch execution of test cases , Reduce test time , Accelerate the feedback loop , Improve software quality

How to achieve automated testing
Implementing automated testing also requires cost

　　1、 Define the scope of automated testing ： Distinguish which are suitable for automated testing , Which are suitable for manual testing
　　　　 Business oriented , Evaluate product testing -- Manual testing
　　　　 Business oriented , Support team testing -- automation & manual
　　　　 Technology oriented , Support team testing ---- automation
　　　　 Technology oriented , Evaluate product testing ---- automation & manual

　　2、 Define the level of automated testing
　　　　UI test （10% The energy of ）《 Service Testing （20% The energy of ）《 unit testing （70% The energy of ）《 Basic settings automated testing （ Efficiency of preparation of test data and test environment ）

　　　　 Infrastructure layer ： Prepare data and environment for automated testing , You can use automation or container based
　　　　 The way to build , Commonly used tools ansible,chef,puppt,jenkins

　　　　 unit testing ： Compete for code methods , Classes and packages , Generally, it belongs to code level testing and is related to
　　　　 Continuous integration within the enterprise, pipeline integration , Commonly used tools xunit System

　　　　 Service test layer ： Test the interface between services , Interaction testing between general service interfaces , Commonly used tools postman,soapUi etc.

　　　　UI test ： Test the function of the interface , Generally, end-to-end process testing is carried out in one or more applications , Focus on key functions , Common tools selenium,appium etc.

　　3、 Integrate with continuous integration pipeline
　　　　 Build automatic test platform and continuous integration pipeline integration
　　　　　　a、 Developers submit code to git Branch and merge the warehouse
　　　　　　b、 After the continuous integration server receives the merge event , Trigger compile build unit test and other checks , take
　　　　　　 The result is notified to the developer
　　4、 Deploy to the test environment , This environment is an integrated environment , Deploy other components that the service depends on
　　5、 Run automated test cases

 

Testing is a very important stage in the software life cycle , It is also a very time-consuming process

fourteen 、 Deployment pipeline ： Build a key platform for one-stop deployment

Build a pipeline from development to testing to operation and maintenance , It can realize one click software deployment to the generation environment

The stage of deploying the pipeline

　　 Submission phase

　　 Automated testing phase

　　 Manual test stage
　　 Release stage

Deploy pipeline related practices

　　1、 One bag to the end
　　　　 Compile the software from the source code to build a deployment package , It is used uniformly in subsequent processes
　　　　 This is a deployment package
　　　　 Reduced compilation time
　　　　 Ensure the consistency of the deployment package

　　2、 The same deployment method
　　　　 Use the relevant pipeline , Deploy any set of environment in the same way , Including the generation environment
　　　　 Separate deployment scripts from configuration information
　　　　 Adopting relevant deployment methods is one of the ways to reduce the risk of software release

　　3、 Smoke test for deployment
　　　　 Call the interface to check whether it can return normally


　　4、 Automated deployment platform
　　　　 Deploy important components of the pipeline , By encapsulating a unified deployment process , Provide an easy-to-use user interface , Provide unified software deployment capability

 

15、 ... and 、 Chaos Engineering ： Improve the reliability of the system through problem injection

The robustness of the software is continuously enhanced by destroying the system in the generation environment

Chaos engineering is a special method of deliberately destroying things , By making trouble in the production environment , To discover possible Hide the problem

Chaos engineering is not just about sabotage , Chaos engineering is a supplement to traditional testing , After traditional testing, the system is stable , Can be arbitrarily destroyed in the production environment , To further enhance the stability of the system

The core idea of chaos engineering is to actively inject faults in a controllable way , To verify whether the behavior of the system meets our expectations , And repair it under abnormal conditions , So as to improve the stability of the system

Why should we implement chaos Engineering
　　 Creating reliable software is the current enterprise to obtain users , The basis for winning market competition
　　 Traditional testing can only guarantee the quality of software application layer , There is no guarantee that the application and various services or the whole system can be used normally under any circumstances

Chaos engineering can actively test the behavior under various pressures in the production environment , By comparing hypothetical behavior with actual behavior . You can find and fix problems before the system breaks down

Software and infrastructure can be tested and verified more widely than traditional forms

Find problems that cannot be found by traditional tests

Help the team understand the behavior of the system in the real environment , How and what services are interrupted bug

Enhanced system stability and reliability , Provide the user experience


How to implement chaos Engineering
　　1、 First, collect a set of baseline indicator data
　　　　 Infrastructure monitoring indicators （ Server's cpu、 Memory 、io、 Disk usage , Network delay , Packet loss ）
　　　　 Warning indicators （ You can count the number of alarms per week by service , The time of processing alarms and the most frequent alarm types of each service every week ）

　　　　 Severity indicators （ The number of events with different severity levels per week can be counted according to service, and the number of events with each severity level can be counted according to service MTTD,MTTR,MTBF）

　　　　 Application indicators （ Observability metrics for applications , Number of events , Requested response event , Number of database connections ,QPS,TPS）

　　2、 Simulate real events
　　
　　　　 attack ： Input the fault into the system , Such as consumption of computing resources , Shutdown system , Methods such as discarding network packets , An attack is the injection of a single fault


　　　　 scene ： Save a set of tools as a collection , The attacks of the scenario can be executed in sequence , But better
　　　　 Control the attack mode and simulate the scenario under complex fault preservation , It can be executed repeatedly and can observe the behavior change of the system over time


　　3、 Analyze the observations
　　　　 Whether the system behavior is in line with expectations

　　　　 If the system has a monitoring alarm system , Whether it can run as expected

　　　　 What new problems have been found in this experiment

　　　　 How long does the alarm system detect problems , And give notice , Is this time acceptable

　　　　 After the experiment , Whether the system automatically recovers to the normal state , Still need human intervention

　　4、 Repeat the experiment
　　　　 After fixing the problem , Repeat the test

　　　　 Modify the content of the attack


　　5、 Convert manual tests to automated tests


Chaos engineering cannot be achieved without tools and platforms chaoblade

 

sixteen 、 Metrics ： Look for really useful indicators

In fact, many teams and organizations are implementing devops They all focus on technology and ignore measurement and culture

Measurement is implementation devops The key elements of , If you put devops Compared to a car , Before that, building tools and platforms is
body , Measurement is the dashboard of the car ,devops The measurement of also needs some indicators to guide devops Continuous improvement of

What kind of indicators are good indicators , How can we find good indicators

Why measure indicators
　　1、 The premise of measurement is to have a set of end-to-end devops platform , Otherwise, the measurement of excellence is just
　　 Local metrics
　　2、 The input-output ratio of the measurement itself is not like CICD Obvious effects , Many measurements are just for the above , Instead of really trying to solve the problem





The measure is devops You need to think about it from the beginning
The core idea of lean thinking is continuous improvement , Clear and clear metrics as a guide , To achieve the goal of continuous improvement , On the road of continuous improvement , No end point , Always on the road

Metrics can provide information to help us know where we are now , How far away from the target , Are we moving forward or backward along the goal , To what extent

Metrics are needed from devops Platform , What metrics should be considered at the beginning , How to get , Yes devops The design of the platform is instructive


Metrics are not an end （ It is easy to give people the illusion of reaching the point ）, It's the means ( Identify potential problems ),
It's not control （ It is easy to give people a psychological hint of static goals ） It's improvement （ Direct to the hearts of the people with dynamic goals ）

What kind of indicators are good indicators
　　 About finding good indicators , Some enterprises have a misunderstanding , Is to measure all content

　　 More indicators need to invest more resources to pay attention to all aspects of software research and development , Eventually, the effect of each indicator is not good

　　 With kpi Complete the index in the form of , The final completion is quantity, not quality

The standard of good indicators
　　1、 Measurable The indicator must be measurable, a quantitative indicator, not very good , Very fast qualitative indicators

　　2、 The associated
　　 Indicators must be able to measure factors that have a significant impact on the business

　　3、 Unalterable
　　 Team members should not affect the results of the metrics

　　4、 Implementable
　　 The index can be obtained by technical means and the value is true and reliable

　　5、 Traceable
　　 The index must be able to directly reflect the problems existing in the software development process


Before finding the indicators that need to be tracked , We need to identify the challenges facing the organization and the problems to be solved （ The standard of measurement is to solve or improve existing problems ）
　　1、 Code quality
　　2、 Team members
　　3、 Release efficiency


Good indicators are used to solve practical business problems
　　1、 Shorten time to market ： Lead time
　　　　 Measure the time from user requirements to final delivery to users

　　2、 Improve the efficiency of software development （ Flow efficiency ）
　　　　 Use flow efficiency , Check the bottleneck , And focus on how to improve the flow bottleneck , Wait for time to decrease , Software development efficiency is high

　　3、 Resolve conflicts between what the team is dealing with and unplanned matters
　　　　 Quantity of WIP used , To expose teams or team members who are overloaded with work , Make the work of each team member more balanced

　　4、 Solve the problem that the unfinished important work will not be forgotten
　　　　 Use residence time , Or expiration time , To measure how long the unfinished work stays in the system , If the set threshold is exceeded, an early warning will be given to expose the risk

　　5、 Reduce the number of problems found by users in the build environment
　　　　 Use defect escape rate . bring bug Try to find

Avoid non conformance devops Indicators of the times
　　1、 Traditional engineering indicators
　　 Such as MTBF（ Mean time between failures ） stay devops The significance of the times is not great , The long-term stability of the system is not the primary goal ,devops The era is to ensure the stability of the system through rapid deployment

　　2、 Competition based indicators
　　 Never build metrics based on team members or competition between teams , The purpose of metrics is to solve business problems , It's not a means to show the technical level of team members

　　3、 Vanity indicators
　　 For example, weekly code line statistics , The workload of developers should not be judged by the meaningless number of lines of code , The timeliness and quality of the final delivered functions are the most important

　　4、 Don't choose indicators based on the difficulty of obtaining indicators

 

How to use these indicators
Feedback loop is the basis of effective improvement , Feedback through metrics helps to adjust the team's actions more accurately , Improve communication throughout the organization


　　step1： collecting data （ Collect data about the software development process , As raw materials for subsequent analysis ）
　　 The platform itself has the ability to collect data , The design platform should be designed for measurement indicators , For example, each task must have a start time and an end time , Every event should happen 、 Handle 、 Time record of solution , The relationship between affairs 　　　　 United

　　 Provide unified reports

　　 The human side , Effective participation of team members can give full play to the ability of the platform ,devops Although the platform will automate the R & D process as much as possible , Some contents still need manual cooperation , Submit the code according to the specification , demand id And defects id Operations such as Association



　　step2： Analyze the data


　　step3： According to the analysis data , Adjust the process

　　step4： repeat

 

seventeen 、 Team capability ： Team capability == Ability to deliver

When people are too professional , Silos will be formed , When conducting software R & D activities , It is necessary to carry out multiple communication and handover between different departments , When there is no free time, you still need to queue , This leads to a delay in delivery

How to solve this problem
Team T Type talent view , To solve such a problem

I type （ Experts ）
　　 Be proficient in a certain field
　　 Few skills or experience in other fields
　　 Soon encountered a bottleneck
　　 Insensitive to downstream waste and impact
　　 Resist flexible or variable plans

T type （ Generalist , Full stack engineer ）
　　 Be proficient in a certain field
　　 Have skills in many fields
　　 Can break through the bottleneck
　　 Sensitive to downstream waste and impact
　　 Help develop flexible and variable plans

E type
　　 Proficient in several fields
　　 Have practical experience in many fields , Strong executive ability , Can continue to innovate
　　 Unlimited potential

Cross training can be used in enterprises , Provide learning opportunities for engineers , And regularly let them rotate between different positions , To improve relevant skills

T Type team needs to master the skill proficiency of team members , And carry out targeted training for personnel in weak links

Make skill map , You can understand the strengths and weaknesses of team members , And it needs to be updated regularly , Every three months

　　 Skill stack Team members A Team members B

　　java
　　go
　　python


How to measure the delivery ability of the team
　　 If there is an urgent project , Which team should be handed over to ： The ability of the team needs to be measured by specific indicators

　　 The visualization of R & D process is poor ： It is difficult to prepare to define how long it takes for a requirement to be developed to be completed , And the specific output of each iteration of each team

　　 The randomness of work segmentation ： There is no unified standard for the work of software development , Demand can be large or small , There is no fixed term

 

Misunderstanding of measuring team effectiveness
Focus on output rather than results
Focus on individual or local measurement rather than team or overall measurement

For example, the statistics of the number of lines of code , Write as much code as possible when completing code logic , Cause the software to become bloated , Make code maintenance cost higher

For example, the statistics of story points , Break requirements into stories , Use user story points to estimate the size of each story , To express the expected workload to complete these stories

A good indicator of team effectiveness
Focus on the overall results ： Reward developers for high throughput , And the system stability of the operation and maintenance team

Focus on results rather than outputs ： It is meaningless not to create output for enterprises

Indicators of effectiveness measurement
　　1、 Lead time for software development （ The time from the request to the deployment to the customer environment ）
　　2、 Deployment frequency （ The sooner the better ）
　　3、 The change of software deployment failed （ It refers to the probability of rollback after the customer environment of version deployment fails ）
　　4、 recovery time （ Time from service failure to service recovery ）
　　5、 Availability of service operation and maintenance （ The ability of the team to ensure the availability of software and services ）

 

eighteen 、 Response speed ; Martial arts in the world , Fast break not only

In today's digital enterprise era , Software release speed , The speed of problem repair is enterprise
The magic weapon to win

Yes 2 Enterprises have found opportunities in the market , At this time, who can turn the opportunity into a product and release it to the market , Who can win the first chance

What is response speed
　　1、 The speed of software release , From software project approval to release to production environment , Time delivered to the user , It's the time of end-to-end release , It represents the response speed of the team to market opportunities

　　2、 Release of features , From the completion of feature review to integration test acceptance test , At the time of release to the production environment , It represents the response speed of the team to user needs

　　3、 Defect release , Only this defect was found , Time to fault location and repair , It represents the response speed of the team to testing and operation and maintenance

The fundamental to improve response speed is to shorten the software delivery cycle

What factors will affect the response speed
　　 Software development is a complex , Work to deliver final value
　　 Resource utilization ： It refers to the proportion of relevant resources that affect software delivery in software delivery activities , For example, team members , Server devices , Time and so on ; But software development is a sequential job , Each stage depends on the previous stage 　　　　　　        paragraph , This means that no matter how much resources are invested immediately, the delivery cycle cannot be shortened indefinitely ; In addition, the higher the utilization rate of resources , The longer the work task waits in the queue , The lead time will also increase , If one person is dealing with multiple at the same time         Mission , Switching tasks will also take extra time , Key resources , Key people are overstocked in long task queues , It will gradually become the bottleneck of the whole delivery


　　 The size of the task , It is also a factor that affects software delivery , When the size of the task is inconsistent , It will lead to inconsistent processing time , There will be waiting , Waiting is a waste , It will prolong the software delivery cycle , therefore , When splitting requirements , Try to split the tasks in a balanced way , Reduce waiting time

　　 Quantity of work in process , Visualize the work in progress through Kanban method , So as to control the quantity of WIP , The more work in progress , Team members' continuous task switching leads to waiting for other tasks and high utilization of team members, resulting in bottlenecks . You can limit the number of WIP

　　 Degree of Automation , Reduce manual operation Automated compilation , Scanning scanning 、 Deploy 、 test


How to measure the response speed

　　 Efficiency of value delivery , Such as how long the user needs to be released , That is, the lead time ; Cumulative flow graph is an important measure in Kanban system , It is used to show the number of WIP in each stage of the development process




Efficiency of the project , finger devops The ability of the platform to record key information such as time , Compilation efficiency , Test efficiency 、 Deployment efficiency
　　 Compilation efficiency , Code compilation requires cpu resources , The tasks that each compiler can handle at the same time are limited , When multiple compilation tasks are executed , There will be tasks waiting , In order to maximize the efficiency of the compiler , Can be built with containers 　　　　 The way , When there is a compilation task , You can start one docker Containers , Destroy after compilation ; The size of the code base will also affect the compilation efficiency ; Settings of compilation tools , The compilation tool uses the latest version

　　 Efficiency of testing , regression testing 、 A functional test 、 Non functional testing , Automate as much as possible , Number of cases , Test success rate



　　 Deployment efficiency , The time from the beginning of deployment to the end of deployment , Deployment frequency is a global indicator

The improvement of efficiency will not be achieved overnight , But through continuous optimization and continuous improvement , This is also the purpose of guiding improvement with metrics

 

nineteen 、 Software quality ： The key to the success of the system

High quality software , Efficient enterprises go faster and farther
Poor software quality , Efficient enterprises will die faster

 

What is software quality
Leaders need to know what the current software quality is , What are the problems , What is the development trend of software quality , These are the measures of software quality

Software quality is divided into internal quality and external quality

　　 Internal quality refers to the quality perceived by developers , In the development process , Find out as soon as possible , Repair as soon as possible , Internal quality problems , It can improve the external quality of products released into the production environment

　　 External quality refers to the quality that can be perceived by users , External quality is the key to the success of a product , Improving external quality is the ultimate goal of team members

　　 Test coverage is a measure of code quality , Code coverage in automated testing , Include unit tests , Integration testing , Test coverage of regression test . The industry generally believes that the test coverage reaches 80% That's all right. , Test one 　　　　 It must be an effective test , Invalid tests even 100% Coverage doesn't make any sense

　　 Number of test defects the number of defects found during the test phase , Testing phase , The purpose is to find problems , Don't be afraid to find problems , Testers should distinguish which are bug, What is the need for improvement

　　 The inspection of test quality cannot be used as a basis to measure the ability of team members , Nor as the standard of performance appraisal , As a result , The overall indicator is the final indicator

External quality
　　 The internal quality has been done very well , Then the external quality must be very good ？ This is not necessarily

　　 From the perspective of user satisfaction , Non functional products to assess external quality
　　 User satisfaction , Judging products from the perspective of end users , Collect users' evaluation information about products or services . Net recommended value （ recommend - Not recommended ）, It is the golden standard to measure user satisfaction

　　 Non functionality of the product , reliability （ The probability of service unavailability ）、 performance （ Fluency of using the product 、 Carton 、 Delay ） wait



Software quality , This has determined the key factors of product success and failure , It is divided into internal quality and external quality , They complement each other , Interact with each other , Internal quality is the source , External quality is the result

 

twenty 、 Business value ： The ultimate goal of software release

In the Agile Manifesto , The first principle stipulates ： Our most important goal is to deliver valuable software as early as possible through continuous efforts , Make users satisfied

By incorporating valuable software , Satisfied users are linked to the ultimate business goals of the enterprise , Can realize the value of enterprise business , Business objectives , For example, the growth of users , An increase in income , Cost reduction

 

The implementation of devops What is the purpose ？
devops The goal of （ Increase deployment frequency , Reduce recovery time , Improve service availability ）, Ultimately achieve user satisfaction , Improve user satisfaction , Achieving business goals （ Market share growth , User growth , User usage market growth ）, Finally achieve the enterprise goal （ Revenue growth ）

You can see devops The goal is not to implement devops Ultimate goal , Business goals and enterprise goals are the implementation devops Ultimate goal

How to measure the value of the business
devops The measurement of is not only to measure the frequency of release , Number of code defects , Demand and other R & D indicators , Also measure users 、 Market share 、 Changes in data such as net recommendations

Software delivers user requirements through every release , Only after the software is released , All the abilities you have done before can be valued

How to measure the incremental value of software release ？
For example, will it increase income , Will it reduce costs , Will it attract more users , Will existing users be prevented from leaving

In the release phase, it is necessary to determine the functional scope of the release , Also measure the value after release , In order to determine whether the project income matches the actual income , Whether it really realizes the value of the business

It can be measured by the following indicators
　　1、 User visits ,PV
　　2、 Number of unique visitors ,UV
　　3、 Number of new users
　　4、 User loyalty , User stickiness （ Number of repeat purchases 、 The length of decision-making time , Attitude towards competitors , Attitude towards product problems ）
　　5、 User retention
　　6、 Market share , Determines the profit of the enterprise in probability
　　7、 Net recommended value eNPS