ADA logics:cri-o overall safety audit project

author ：David Korczynski + Adam Korczynski

Before the article by Ada Logics Safety research and safety engineering David Korczynski And safety engineering and safety automation Adam Korczynski stay Ada Logics Blog [1] Published on

Ada Logics Ltd. Recently on CRI-O[2] Conducted a comprehensive safety audit .CRI-O yes Kubernetes Container runtime interface [3] An implementation of , stay Kubernetes Used as the core component in the cluster , Orchestrate containers on each node . This makes it an important software package , because CRI-O The problems in will have a far-reaching impact , Because it's a lot of Kubernetes A key part of the cluster .

The main security finding of the audit project was a serious denial of service problem . For anyone who has the ability to Kubernetes Create on Cluster pod For people who , This is a node level denial of service attack . Essentially , If one can in a given Kubernetes Create a node in the cluster pod, Then it can cause a denial of service attack by running out of memory . This is for dependencies CRI-O Runtime nodes , Interestingly , The vulnerability also exists in another popular container runtime Containerd[4] in .

We want to thank CRI-O The cooperation of defenders . This job has got CNCF[5] The support of , And got OSTIF[6] With the help of . Besides our work ,Chainguard[7] Also participated in software security supply chain audit . We would like to thank all our collaborators .

In this blog post , We will give an overview of the project , Complete details can be found in CRI-O Found in the warehouse report .

Summary of audit and results

The objective of the audit project is to CRI-O Extensive analysis of the security situation , It is divided into the following tasks ：

1. CRI-O The threat model is formalized
2. Security audit of code
3. CRI-O Fuzzy test integration , Including passage OSS-Fuzz[8] Continuous fuzzy test integration
4. Review documents and tests

The main safety finding of this work is a single high severity problem . Also found some small problems , however , From the perspective of completing this audit ,CRI-O Is a well written project , High level of safety assurance .

Cluster denial of service through deployment

The most serious discovery is a denial of service attack on a given cluster by exhausting node resources . The attack is by creating pod In the way of , This means that anyone can create pod Of users can be used to create pod Cause a denial of service on the given node of . Of this vulnerability CVE yes CVE-2022-1708,Github Safety bulletins can be found here ：https://github.com/cri-o/cri-o/security/advisories/GHSA-fcm2-6c3h-pg6j

Interestingly , Denial of service attacks also occur in other container runtime interface implementations , Most notably Containerd. say concretely , Run out of CRI-O Memory attacks in can be used to exhaust Containerd Of memory .containerd The problem of CVE yes CVE-2022-31030, About containerd Of Github Safety bulletins can be found here ：https://github.com/containerd/containerd/security/advisories/GHSA-5ffw-gxpp-mxpf

stay Kubernetes In the world of , about CRI-O and Containerd Come on , This problem can lead to denial of service of nodes through deployment without any user interaction , If the cluster uses any of these container runtime interface implementations , The cluster will be enabled effectively DOS. Patching is highly recommended .

CRI-O Continuous fuzzy test integration

We aim at CRI-O The infrastructure integrates an extensive fuzzy test suite . The main challenge in this regard is to build infrastructure , send CRI-O The fuzzy test can work , It's very difficult , because CRI-O Is an interconnected system , Depends on many components , For example, binary files on the system , Mainly through Kubelet communicate ,kubelet Restrict sending to CRI-O The data of , It also uses a fairly complex testing framework , Many, many simulations are involved .

in general , We aim at CRI-O Code 、containers/image[9] and containers/storage[10] The warehouse implements 14 A blur Tester , And integrate the project into OSS-Fuzz. The blur tester can be accessed from https://github.com/cncf/cncf-fuzzing/tree/main/projects/cri-o get ,OSS-Fuzz The integration of is in https://github.com/google/oss-fuzz/tree/master/projects/cri-o.

The main focus of fuzzy testing is on gRPC The handler . This is mainly due to fuzz_server[11] complete , It is a fairly large blur Tester , contain 900 Line code . This blur tester starts a gRPC The server , And send a random message sequence to the server . In this way , The blur tester is CRI-O Has a significant impact on the entire code .

Reference resources

• complete PDF The report ：https://github.com/cri-o/cri-o/blob/main/security/2022_security_audit_adalogics.pdf
• CRI-O Safety notice ：https://github.com/cri-o/cri-o/security/advisories/GHSA-fcm2-6c3h-pg6j
• Containerd Safety notice ：https://github.com/containerd/containerd/security/advisories/GHSA-5ffw-gxpp-mxpf
• OSTIF Notice ：https://ostif.org/our-audit-of-cri-o-is-complete-high-severity-issues-found-and-fixed/

Reference material

[1]

Ada Logics Blog : https://adalogics.com/blog/cri-o-security-engagement

[2]

CRI-O: https://github.com/cri-o/cri-o

[3]

Kubernetes Container runtime interface : https://github.com/kubernetes/cri-api

[4]

Containerd: https://containerd.io/

[5]

CNCF: https://www.cncf.io/

[6]

OSTIF: https://ostif.org/

[7]

Chainguard: https://www.chainguard.dev/

[8]

OSS-Fuzz: https://google.github.io/oss-fuzz/

[9]

containers/image: https://github.com/containers/image

[10]

containers/storage: https://github.com/containers/storage

[11]

fuzz_server: https://github.com/cncf/cncf-fuzzing/blob/main/projects/cri-o/fuzz_server.go

---

CNCF (Cloud Native Computing Foundation) Founded on 2015 year 12 month , Affiliated to the Linux Foundation, It's a non-profit organization .

CNCF（ Cloud native Computing Foundation ） Committed to fostering and maintaining a vendor neutral open source ecosystem , To promote cloud native technology . By democratizing the most cutting-edge model , Let these innovations be used by the public .