Test cases should never be used casually, recording the thinking caused by the exception of a test case

High quality resource sharing

Learning route guidance （ Click unlock ）	Knowledge orientation	Crowd positioning
🧡 Python Actual wechat ordering applet 🧡	Progressive class	This course is python flask+ Perfect combination of wechat applet , From the deployment of Tencent to the launch of the project , Create a full stack ordering system .
Python Quantitative trading practice	beginner	Take you hand in hand to create an easy to expand 、 More secure 、 More efficient quantitative trading system

One

Do you usually write test cases ？

I used to write test cases only for business interfaces , Write one for each interface , data case It's just a test . It's ok if you can run through . Different scenes case, Then change the data . Run again . Simple and easy .

But since I started to maintain open source in my spare time , Began to deepen the understanding of test cases . Even now I have promoted the status of test cases to the same important status as the core code , I once joked that those who only write core code but not test case code are hooligans .

Open source projects face everyone , Everyone and every company have different environments , The project structure is also different ,jdk,spring The version of the system , Third party dependency packages are different . So the open source framework must work well in all scenarios . So many function points , So many scenes , Even if I am the author , It's impossible to remember so many details just by familiarity , At this time, test cases are very important , It is the most critical quality assurance of the whole project . A lot of times , I rely on test cases to find some small edges bug Of . At present, my open source project has 870 Test cases , It covers about 90% The above scenario .

This article discusses a problem of test case operation mechanism caused by test cases .

Two

The cause of the matter is that a small partner in the group finds that a unit test case has a wrong configuration item ,spring The context actually executes 2 Time , And with the right configuration , It is normal to start only once . This puzzled him , Thought there was something wrong with the framework .

The reason why he felt spring Launched the 2 Time , Is to see in the log 2 Time springboot Of logo Print ,2 The same error report again ：

  .   \_\_\_\_          \_ \_\_ \_ \_
 /\\ / \_\_\_'\_ \_\_ \_ \_(\_)\_ \_\_ \_\_ \_ \ \ \ \
( ( )\\_\_\_ | '\_ | '\_| | '\_ \/ \_` | \ \ \ \
 \\/ \_\_\_)| |\_)| | | | | || (\_| | ) ) ) )
 ' |\_\_\_\_| .\_\_|\_| |\_|\_| |\_\\_\_, | / / / /
 =========|\_|==============|\_\_\_/=/\_/\_/\_/
 :: Spring Boot :: (v2.0.5.RELEASE)
 
com.yomahub.liteflow.exception.ELParseException:  Program error , Does not meet the grammatical specification , There is no matching grammar , Maximum match to [0:7]
 at com.yomahub.liteflow.builder.el.LiteFlowChainELBuilder.setEL(LiteFlowChainELBuilder.java:124) ~[liteflow-core-2.8.2.jar:na]
 at com.yomahub.liteflow.parser.helper.ParserHelper.parseOneChainEl(ParserHelper.java:391) ~[liteflow-core-2.8.2.jar:na]
 at com.yomahub.liteflow.parser.el.XmlFlowELParser.parseOneChain(XmlFlowELParser.java:20) ~[liteflow-core-2.8.2.jar:na]
 at java.util.ArrayList.forEach(ArrayList.java:1259) ~[na:1.8.0\_292]
 at com.yomahub.liteflow.parser.helper.ParserHelper.parseDocument(ParserHelper.java:217) ~[liteflow-core-2.8.2.jar:na]
 at com.yomahub.liteflow.parser.base.BaseXmlFlowParser.parse(BaseXmlFlowParser.java:40) ~[liteflow-core-2.8.2.jar:na]
 . \_\_\_\_ \_ \_\_ \_ \_
 /\\ / \_\_\_'\_ \_\_ \_ \_(\_)\_ \_\_ \_\_ \_ \ \ \ \
( ( )\\_\_\_ | '\_ | '\_| | '\_ \/ \_` | \ \ \ \
 \\/ \_\_\_)| |\_)| | | | | || (\_| | ) ) ) )
 ' |\_\_\_\_| .\_\_|\_| |\_|\_| |\_\\_\_, | / / / /
 =========|\_|==============|\_\_\_/=/\_/\_/\_/
 :: Spring Boot :: (v2.0.5.RELEASE)
 
com.yomahub.liteflow.exception.ELParseException:  Program error , Does not meet the grammatical specification , There is no matching grammar , Maximum match to [0:7]
 at com.yomahub.liteflow.builder.el.LiteFlowChainELBuilder.setEL(LiteFlowChainELBuilder.java:124) ~[liteflow-core-2.8.2.jar:na]
 at com.yomahub.liteflow.parser.helper.ParserHelper.parseOneChainEl(ParserHelper.java:391) ~[liteflow-core-2.8.2.jar:na]
 at com.yomahub.liteflow.parser.el.XmlFlowELParser.parseOneChain(XmlFlowELParser.java:20) ~[liteflow-core-2.8.2.jar:na]
 at java.util.ArrayList.forEach(ArrayList.java:1259) ~[na:1.8.0\_292]
 at com.yomahub.liteflow.parser.helper.ParserHelper.parseDocument(ParserHelper.java:217) ~[liteflow-core-2.8.2.jar:na]
 at com.yomahub.liteflow.parser.base.BaseXmlFlowParser.parse(BaseXmlFlowParser.java:40) ~[liteflow-core-2.8.2.jar:na]

The test case code is ：

@RunWith(SpringRunner.class)
@TestPropertySource(value = "classpath:/whenTimeOut/application1.properties")
@SpringBootTest(classes = WhenTimeOutELSpringbootTestCase.class)
@EnableAutoConfiguration
@ComponentScan({"com.yomahub.liteflow.test.whenTimeOut.cmp"})
public class WhenTimeOutELSpringbootTestCase {

    @Resource
    private FlowExecutor flowExecutor;

    // among b and c stay when Timeout in case , So it threw out WhenTimeoutException This is a mistake 
    @Test
    public void testWhenTimeOut1() throws Exception{
        LiteflowResponse response = flowExecutor.execute2Resp("chain1", "arg");
        Assert.assertFalse(response.isSuccess());
        Assert.assertEquals(WhenTimeoutException.class, response.getCause().getClass());
    }
}

Open source framework at the source level , It is impossible to take the initiative to start again spring Context ( In fact, I don't know how to do it if I want to ). And with the correct configuration , It's normal . and spring Of @Configuration Also started 2 Time , From the thread stack , Also by Junit This triggers ：

It is worth mentioning that , The error reported is springboot Start the process . So I didn't enter at all @Test Decorated test case code . So it has nothing to do with what the code is written . I tested it , If an exception is thrown in the test code ,spring The context is started only once .

So this problem may end here , Because it's not the framework itself ,Junit Itself is starting spring Failure triggers 2 Secondary initialization spring The action of , It could be a kind of Junit The mechanism of retry . This is beyond my control , Anyway, there's really something wrong , It will also be thrown out , Also need not care Initialize several times , It doesn't affect the overall effect of my test cases , Just correct the specific test cases .

3、 ... and

But when I was dealing with a test case later, I suddenly thought about test cases Spring Loading mechanism , Thus associating with the previous problem . A sudden Epiphany .

The structure of our use cases is generally , A test case represents a large scenario , Each method in it represents a specific case. hypothesis 1 Class with 10 individual test Specific use cases , So when you click on the class Run Test When ,spring How many times will it be initialized .

The answer is 1 Time ,springboot test To speed up the process of running test cases , It is impossible to initialize every method spring Of . In this class spring The context of will be cached , this 10 All methods will share the same spring Context .

The specific operating mechanism is ： Click on the class Run Test When , Will go first initialization spring, Then start running test methods , When the test method runs , If found not initialized spring, It will be initialized again spring. This explains , When we run the method alone run test When , It will also be initialized spring.

Now we can explain the above problem , Because initialization failed , When running the method, it is found that it has not been initialized , So we initialize again .

But for different Test Class words , It will be initialized many times . in other words , Each class will be initialized spring. This should be found when you run multiple test cases .

Four

One more question ： Has anyone ever encountered that when running all test cases, there will always be several errors , But the single operation is completely normal ？

If you've ever met one , That must be ignoring the following points ：

If you choose to run all test cases , Although each test case class is initialized spring, however JVM From beginning to end, it only started once . And those of you defined in the class static The variable of , Not as spring Start and change . When you are all running , Some of the test cases you may make mistakes refer to static Variables are also data left over from the last test case . So you may report an error . And in a single run , There is no such phenomenon .

If you encounter this situation , You have to use in test cases @AfterClass This annotation , In the method of annotation declaration, put the static Variables are cleared . So we can run together . For example, each of my test cases inherits one BaseTest Method , Write this method inside for emptying static The cache of ：

public class BaseTest {
    @AfterClass
    public static void cleanScanCache(){
        ComponentScanner.cleanCache();
        FlowBus.cleanCache();
        ExecutorHelper.loadInstance().clearExecutorServiceMap();
        SpiFactoryCleaner.clean();
        LiteflowConfigGetter.clean();
    }
}

5、 ... and

How to write test cases , What is the common way of writing . I don't want to explain it too much here , Baidu for a while , You should find a lot of tutorials , Or interested , You can also read my open source project LiteFlow Test cases in .

Test cases can not only ensure the quality of your project , You can also clearly see how many lines of code your entire test case covers . The test cases here are written by a separate project . To distinguish the core engineering package .

And then in IDEA Go inside and configure it separately testcase The task of ：

Then go to some run xxx with coverage Button to run the test case ：

Between multiple test projects , Run a dialog box that will pop up and ask if you want to add the results of this time to the total results , Direct point add That's all right. ：

All your test case projects are running well , The following report page will be displayed on the right ：

At the top, you can see that the number of lines covered by my entire test case is 79%. However, this does not mean that the project covers only scenarios 79%. Row coverage and function scenario coverage are 2 A concept , It just means that all test cases have been run , Ran the proportion of all code lines .

Finally, I hope you can't ignore the test cases , Although sometimes I feel sick when I write , But in the end you will realize its sweetness .