聊聊Go代码覆盖率技术与最佳实践

{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"\"聊点干货\""}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"覆盖率技术基础"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"截止到Go1.15.2以前，关于覆盖率技术底层实现，以下知识点您应该知道："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"go语言采用的是插桩源码的形式，而不是待二进制执行时再去设置breakpoints。这就导致了当前go的测试覆盖率收集技术，一定是侵入式的，会修改目标程序源码。曾经有同学会问，插过桩的二进制能不能放到线上，所以建议最好不要。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"到底什么是\"插桩\"？这个问题很关键。大家可以任意找一个go文件，试试命令"},{"type":"codeinline","content":[{"type":"text","text":"go tool cover -mode=count -var=CoverageVariableName xxxx.go"}]},{"type":"text","text":"，看看输出的文件是什么？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"笔者以这个文件为例"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"https://github.com/qiniu/goc/blob/master/goc.go"}]},{"type":"text","text":", 得到以下结果:"}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/e9/e9db85d547914df99c6d84bcac7bbf29.png","alt":null,"title":"","style":[{"key":"width","value":"100%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"可以看到，执行完之后，源码里多了个"},{"type":"codeinline","content":[{"type":"text","text":"CoverageVariableName"}]},{"type":"text","text":"变量，其有三个比较关键的属性:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"Count"}]},{"type":"text","text":" uint32数组，数组中每个元素代表相应基本块(basic block)被执行到的次数"},{"type":"codeinline","content":[{"type":"text","text":"CoverageVariableName"}]},{"type":"text","text":"变量会在每个执行逻辑单元设置个计数器，比如"},{"type":"codeinline","content":[{"type":"text","text":"CoverageVariableName.Count[0]++"}]},{"type":"text","text":", 而这就是所谓插桩了。通过这个计数器能很方便的计算出这块代码是否被执行到，以及执行了多少次。相信大家一定见过表示go覆盖率结果的coverprofile数据，类似下面: "},{"type":"codeinline","content":[{"type":"text","text":"github.com/qiniu/goc/goc.go:21.13,23.2 1 1"}]}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这里的内容就是通过类似上面的变量"},{"type":"codeinline","content":[{"type":"text","text":"CoverageVariableName"}]},{"type":"text","text":"得到。其基本语义为 \""},{"type":"text","marks":[{"type":"strong"}],"text":"文件:起始行.起始列,结束行.结束列 该基本块中的语句数量 该基本块被执行到的次数"},{"type":"text","text":"\""}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"Pos"}]},{"type":"text","text":" 代表的各个基本块在源码文件中的位置，三个为一组。比如这里的"},{"type":"codeinline","content":[{"type":"text","text":"21"}]},{"type":"text","text":"代表该基本块的起始行数，"},{"type":"codeinline","content":[{"type":"text","text":"23"}]},{"type":"text","text":"代表结束行数,"},{"type":"codeinline","content":[{"type":"text","text":"0x2000d"}]},{"type":"text","text":"比较有趣，其前16位代表结束列数，后16位代表起始列数。通过行和列能唯一确定一个点，而通过起始点和结束点，就能精确表达某基本块在源码文件中的物理范围"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"NumStmt"}]},{"type":"text","text":" 代表相应基本块范围内有多少语句(statement)"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"依托于go语言官方强大的工具链，大家可以非常方便的做单测覆盖率收集与统计。但是集测/E2E就不是那么方便了。不过好在我们现在有了 https://github.com/qiniu/goc。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"集测覆盖率收集利器 - Goc原理"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"关于单测这块，深入go源码，我们会发现"},{"type":"codeinline","content":[{"type":"text","text":"go test -cover"}]},{"type":"text","text":"命令会自动生成一个"},{"type":"codeinline","content":[{"type":"text","text":"_testmain.go"}]},{"type":"text","text":" 文件。这个文件会Import各个插过桩的包，这样就可以直接读取插桩变量，从而计算测试覆盖率。实际上"},{"type":"codeinline","content":[{"type":"text","text":"goc"}]},{"type":"text","text":"也是类似的原理(PS: 关于为何不直接用"},{"type":"codeinline","content":[{"type":"text","text":"go test -c -cover"}]},{"type":"text","text":" 方案，可以参考这里: "},{"type":"link","attrs":{"href":"http://mp.weixin.qq.com/s?__biz=Mzg5NDEzNzk3OQ==&mid=2247483688&idx=1&sn=ddb905febdd9adfe5769b5b82d70a2e0&chksm=c025623cf752eb2abf7d2378ccd0054758c505cae7799bcd518dd149888ccec0479b34d1e784&scene=21#wechat_redirect","title":null},"content":[{"type":"text","text":"开源啦 | go语言系统测试覆盖率收集利器goc"}]},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不过集测时，被测对象通常是完整产品，涉及到多个long running的后端服务。所以goc在设计上会自动化会给每个服务注入HTTP API，同时通过服务注册中心"},{"type":"codeinline","content":[{"type":"text","text":"goc server"}]},{"type":"text","text":"来管理所有被测服务。如此的话，就可以在运行时，通过命令"},{"type":"codeinline","content":[{"type":"text","text":"goc profile"}]},{"type":"text","text":"实时获取整个集群的覆盖率结果，当真非常方便。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"整体架构参见:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/7a/7ac8b0998f6e9f35ea7c28a923472014.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"代码覆盖率的最佳实践"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"技术需要为企业价值服务，不然就是在耍流氓。可以看到，目前玩覆盖率的，主要有以下几个方向:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"度量 - 深度度量，各种包，文件，方法度量，都属于该体系。其背后的价值在于反馈与发现。反馈测试水平如何，发现不足或风险并予以提高。比如常见的作为流水线准入标准，发布门禁等等。度量是基础，但不能止步于数据。覆盖率的终极目标，是提高测试覆盖率，尤其是自动化场景的覆盖率，并一以贯之。所以基于此，业界我们看到，做的比较有价值的落地形态是增量覆盖率的度量。goc diff 结合Prow平台也落地了类似的能力，如果您内部也使用Kubernetes，不妨尝试一下。当然同类型的比较知名的商业化服务，也有CodeCov/Coveralls等，不过目前她们多数是局限在单测领域。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"精准测试方向 - 这是个很大的方向，其背后的价值逻辑比较清晰，就是建立业务到代码的双向反馈，用于提升测试行为的精准高效。但这里其实含有悖论，懂代码的同学，大概率不需要无脑反馈；不能深入到代码的同学，你给代码级别的反馈，也效果不大。所以这里落地姿势很重要。目前业界没还看到有比较好的实践例子，大部分都是解决特定场景下的问题，有一定的局限。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"而相较于落地方向，作为广大研发同学，下面这些最佳实践可能对您更有价值："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"高代码覆盖率并不能保证高产品质量，但低代码覆盖率一定说明大部分逻辑没有被自动化测到。后者通常会增加问题遗留到线上的风险，当引起注意。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"没有普适的针对所有产品的严格覆盖率标准。实际上这更应该是业务或技术负责人基于自己的领域知识，代码模块的重要程度，修改频率等等因素，自行在团队中确定标准，并推动成为团队共识。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"低代码覆盖率并不可怕，能够主动去分析未被覆盖到的部分，并评估风险是否可接受，会更加有意义。实际上笔者认为，只要这一次的提交比上一次要好，都是值得鼓励的。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"谷歌有篇博客(见参考资料)提到，其经验表明，重视代码覆盖率的团队通常会更加容易培养卓越工程师文化，因为这些团队在设计产品之初就会考虑可测性问题，以便能更轻松的实现测试目标。而这些措施反过来会促使工程师编写更高质量的代码，更注重模块化。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最后，欢迎加入七牛云Goc交流群，我们一起聊聊goc，聊聊研发效能那些事。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"参考资料:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"https://testing.googleblog.com/2020/08/code-coverage-best-practices.html"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"系列文章"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://xie.infoq.cn/article/0d1b06597b8345708606fc073","title":""},"content":[{"type":"text","text":"https://xie.infoq.cn/article/0d1b06597b8345708606fc073"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"觉得不错，欢迎关注公众号: 大卡尔"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"                                     点个赞、在看和转发是最大的支持"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}