Don't ask me how to do UI automation test again

Key words of this article ： Mobile UI Automation thinking

Hello everyone , I'm little brother . Before the test, some students in the communication group asked “ Are there any cases of automated testing in the work that can be shared “, Yes, yes .

Today I will describe in detail a 【UI The application of automation in practical work 】, This is a real one that our team did before UI Automation case , because APP Version update iterations are too many , The code is also missing , So some details can not be fully described .

But I will put the idea and process of this project , Share in detail , I hope that helps .

This article dry goods warning , The full text 5300 word , Estimated reading time 14 minute .

Catalog

One 、 The main categories of automated testing

Two 、 Fit to do UI Automated scenarios

• 1）【 Regression test 】

• 2）【 On-Site Inspection 】

• 3）【 Embedded point test at mobile end 】

3、 ... and 、UI Automation case

• 1） Project background

• 2） Project selection

• 3） Start installing deployment

• 4） Execute the test plan

Four 、 Yes UI Automated thinking

5、 ... and 、 Automated test learning resource sharing

One 、 The main categories of automated testing

The forms of automation can be divided into interface automation and UI automation .

According to the test pyramid theory , More and more to the top ,ROI（ Input output ratio ） The higher .

Generally, everyone will increase efforts to invest in the construction of interface automation , and UI Automation because the input-output ratio is too low , So basically right UI There will be less investment in the construction of automation .

But that doesn't mean UI Automation is completely unnecessary , In the next section, I will list 3 Kind of 【 Fit to do UI Automated scenarios 】.

Two 、 Fit to do UI Automated scenarios

Generally speaking, there is 3 This scenario is more suitable for UI Automated ： Regression test 、 On-Site Inspection 、 Mobile end embedded point test .

1） Regression test

The project goes online or APP release , Traditional manual testing is time-consuming . On the first release date , Especially for the distribution with large changes , Colleagues use manual case walk through to verify regression , The pressure is very high ！

At that time, it was generally verified by online regression , Basically, it costs 30 minute -2 Hours , For testers and developers , It's all a kind of suffering .

Sometimes there are not enough people , We also need products 、 The operation team assists in regression verification , Very inefficient .

This is the time , It can be used UI Automation to cover some functions with few changes .

2） On-Site Inspection

The online environment is complex , It is prone to unexpected problems , The pressure of manual inspection and test is great .

such as ：

• Operation configuration error ;

• Some unconventional operations of users ;

• Compatibility of different mobile terminals .

These problems are exposed to users , To feedback to development , And then to the development and repair , It may have been a long time .

In the past , Colleagues usually conduct regular manual online inspection to find some online problems .

It does have an effect , But the problem is ：

Manual inspection will also cost a lot of testing time , Sometimes busy , It's hard to stick to it .

This is the time , It can be used UI Automation to cover some main processes , Make sure there are no problems with the main process .

3） Embedded point test at mobile end

If you keep doing simple and repeatable tests , Not only is it inefficient , It also severely dampens the enthusiasm of testers .

The most typical is the embedded point test of the mobile terminal , Traditional test methods , Is to operate on the mobile phone , Trigger buried point report , Then grab the bag through the mobile phone , Obtain buried point data , And then according to the buried point document , To manually verify each field one by one .

This operation is very mindless , But it actually takes quite a lot of time , There is a need for more buried sites , It may not be finished in one day , It also wastes the energy of testers ： After each test, the buried point , I feel that my eyesight has decreased a lot （ Blind to data ）.

This is the time , You can try UI Automatically trigger the burying point , And through some verification logic , Automatic calibration of embedded points .

4） besides ,UI Automation can also solve ：

• APP Compatibility test

• APP Comparative test of competing products

• APP Data capture

• ...

If you have more UI Automation practice cases , Welcome to comment area .

3、 ... and 、UI Automation case

Directory as follows ：

1） Project background

2） Project selection

3） Start installing deployment

4） Execute the test plan

1） Project background

When I was looking for a room in a shell , As the company's business continues to grow ,APP More and more functions , every time APP Big edition , Testers need to spend more energy on online regression verification than ever before .

According to statistics , At present, the number of manual use cases for online regression verification in our department is about 1000 Bar or so , If all use cases require regression testing , It will cost at least 2 Hours to complete , This is undoubtedly APP The test brings great challenges to students .

If use case regression is not sufficient , hold Bug Left in the production environment , It will bring bad experience to users .

Traditional manual testing , It has been difficult to meet the needs of daily testing , We found that , Many use cases can be realized in the form of automation , This will greatly alleviate APP Test your classmates' stress .

Now let's show you some “ Manual use cases ” convert to “ automation “ Table screenshot of , Although the method is stupid , But that's how we counted the coverage of automated scenes .

Now that the use cases have been sorted out , We began to prepare to develop UI Automated frameworks and scripts , It is definitely impossible to develop from scratch , There is not so much time to build the wheel repeatedly in the actual work , So the first thing we have to do , Is to choose a good wheel , And then do development on this basis .

2） Project selection

stay APP Of UI Automation , In recent years, many excellent  UI Automated testing framework （ Tools ）.

such as ：Appium、Robotium、Instrumentation、UIAutomator、ATX wait .

Because the small partners in the team , It's basically Pythoner, So when we test the framework selection , It is preferable to consider Python A well supported framework .

After a round of comparison , Find out Appium and ATX Yes Python The support is better , And both have Android and iOS Mature solutions .

So it was decided later that Appium and ATX Choose between .

Here are Appium and ATX Some of the contrast ：

At the beginning of the project , We also built Appium and ATX Operating environment .

Later, because Appium Installation and deployment are relatively complex 、 Project startup and operation ratio ATX A little slower （ The frame is heavy ）、Appium Configuration is tedious .

So we gradually gave up Appium, In the end ATX Conduct UI Construction of automation engineering .

Here we can briefly introduce ATX The ecosystem ：

You can see it ATX It's still very powerful , The project is also being maintained and updated .

however , Light has a bottom layer UI The automation driven framework is not good yet , You also need a professional testing framework to manage your automated testing projects （ Use cases ）.

pytest and unittest Is the most popular solution .

We are glad to find that , stay GitHub Someone has already done the scaffolding work ：ATX-Test.

ATX-Test Is based on ATX-Server Of UI Automated testing framework , It can realize the parallel test of multiple devices , And generate a unified test report .

The technology stack used includes ：

Now that there are more useful projects , We can directly carry out secondary development under this framework , Avoid making wheels repeatedly .

ATX-Test Currently not supported iOS, During the second transformation , You can put the facebook-wda The driver is integrated .

3） Steps to install the deployment

Use ATX The preconditions are ：

Python Running environment （Python 3.6+ ( Community feedback 3.8.0 I won't support it , however 3.8.2 Support ）

Android edition 4.4+（ Need to install uiautomator2）

iOS（ Need to install facebook-wda）

Installation and deployment steps

First step , install uiautomator2 

（ fit Android）

pip install --upgrade --pre uiautomator2

Yes Android Initialize the mobile phone

# init  All devices connected to the computer 
python -m uiautomator2 init

#  High-order usage 
# init and set atx-agent listen in all address
python -m uiautomator2 init --addr :7912

About u2 Detailed use documents of , You can refer to ：

https://github.com/openatx/uiautomator2

No more details here .

The second step , install weditor

pip install -U weditor

start-up weditor

python -m weditor

weditor You can grab Android control elements 、 Automatically generate script code and Debug.

The third step , install facebook-wda（ fit iOS）

need MacOS Running environment , And you need to install Xcode.

pack WebDriverAgentRunner

xcodebuild -project WebDriverAgent.xcodeproj -scheme WebDriverAgentRunner -destination 'platform=iOS Simulator,name=iPhone 6' test

WDA Running on a real machine requires some configuration , You can refer to this article ：

ATX file - iOS How to install the real machine  

WebDriverAgent：

https://testerhome.com/topics/7220

After configuration, run the following command （ Need to use Mac Password , And equipment UDID）

#  Unlock keychain, So that the normal signature can be applied 
security unlock-keychain -p $your-mac-password-here ~/Library/Keychains/login.keychain

#  Acquiring equipment UDID
UDID=$(idevice_id -l | head -n1)

#  Run the test 
xcodebuild -project WebDriverAgent.xcodeproj -scheme WebDriverAgentRunner -destination "id=$UDID" test

install facebook-wda

pip3 install -U facebook-wda

About facebook-wda The use of documents , You can refer to ：

https://github.com/openatx/facebook-wda

No more details here .

Step four ,iOS The method to grab the control , Generally, there are two common ways ：

Appium Provided by the team appium-ios-inspector Project address ：

https://github.com/mykola-mokhnach/Appium-iOS-Inspector

Macaca Of app-inspector Tools Project address ：

https://macacajs.github.io/app-inspector/cn/

Step five , About cloud real machine cluster deployment

What is cloud real machine cluster , To put it bluntly, it means that the mobile terminal real machine can be remotely scheduled on the cloud platform （ Mobile management platform ）.

Existing cloud real machine clusters , In general, it can be divided into ： Public cloud real machine cluster and private cloud real machine cluster .

Public cloud cluster , It is the cloud real machine service provided by some companies in the testing industry .

Well known are Testin、WeTest etc. .

• advantage ： There's no need to build your own , And there are a large number of mobile devices to choose from .

• shortcoming ： Asking for money , your .

Private cloud real machine cluster , You need to purchase real equipment and special cabinets by yourself .

We finally decided to build a private cloud real machine environment , To perform real machine dispatching and management .

The real machine adopts the machine eliminated from the test , Sometimes I will buy some popular models for UI automated testing .

Are there any advantages and disadvantages of private cloud real machine ？

advantage ： The machine is its own , The platform is its own , Play as you like .

shortcoming ：

• The cost of real machines is also very high .

• If it's a wired connection , Charge the battery for a long time , There is great loss , The battery will bulge , If the holiday is unattended , In the worst case, the cabinet will explode .

• If it is a wireless connection , Battery monitoring is another big problem , Frequent manual care is required .

Now commonly used private cloud real machine management platform , May adopt ：

STF：https://github.com/openstf/stf

perhaps ,

atxserver2：https://github.com/openatx/atxserver2

atxserver2 Seamless docking ATX, So what we're using is atxserver2 To manage our cloud machines .

4） Execute the test plan

Let's take a look at our entire test architecture ：

Generate test reports ：

If you prefer pytest Words , You can put uiautomator2 + htmltestrunner Switch to pytest + allure It's delicious, too .

Because it involves some company secrets , There is no specific demonstration in this part .

If you are interested, you can build a set by yourself UI Automated test run environment .

Four 、 Yes UI Automated thinking

If you really touch UI In terms of Automation , You'll find that UI Automatic playability is also very strong , And for novice testing , from UI Getting started with automation to learn automated testing , It is a very good route .

When you see your phone operating according to your preset script APP, There will be an inexplicable sense of achievement , And it will encourage you to continue to study deeply .

I see a lot ”XXX From entry to abandonment “ The case of , Most of them are because the learning path is too steep , It leads to hesitation in acquiring knowledge , Slowly gave up .

however  UI automation It is not difficult to practice , It may be a little troublesome to set up the environment , in general ,UI automation For beginners , It's still friendly .

Of course UI Automation does not solve all problems , such as ：

• APP Version iteration is too fast ,UI Frequent changes .

• Test scenarios with complex operation processes . such as , There are often various pop ups （ Pinduoduo makes you cut ）, The pass rate of use cases will be very low .

• Base codes don't make sense , Test scenario design is better than implementation . such as , In order to pursue the ultimate coverage of automated testing , May do automated testing for some very marginal functions , These functions are rarely even triggered by users , Do this part of function automation , It's obviously meaningless .

• Platform is the trend , Some teams abstract the writing of automated scripts onto the platform ” Dot ” Operating the , It seems more convenient , But in fact, the efficiency of writing is not as good as Coding, So we should weigh the pros and cons , Choose an automation solution that suits your team .

Last , Whether it's interface automation or UI automation , Have their own limitations and scope of application , The two should be complementary .

Never put UI automation Work that you're not good at , Throw it to UI automation To do .

UI automation There are more ways to play , Interested students can try according to the business of their own company ：

• utilize AI technology , Predict positioning elements , Improve the efficiency of scripting .

• UI Automation can make use of image recognition for comparative testing .

• combination Maxim、Monkey perhaps Traversal test Tools , Conduct stability precision test .

• combination Perfdog And other performance testing tools , Conduct mobile terminal performance test .

If there's something wrong with it , Please give me more advice . If there are other ideas , You are also welcome to communicate with me in the comment area .

Learning resource sharing

Finally, thank everyone who reads my article carefully , Watching the rise and attention of fans all the way , Reciprocity is always necessary , Although it's not very valuable , If you can use it, you can take it

These materials , For those who want to advance 【 automated testing 】 For our friends, it should be the most comprehensive and complete war preparation warehouse , This warehouse also accompanied me through the most difficult journey , I hope it can help you ！ Everything should be done as soon as possible , Especially in the technology industry , We must improve our technical skills . I hope that's helpful …….