当前位置：网站首页>Design and implementation of kubelet garbage collection mechanism to protect nodes from being preempted by containers image GC high threshold

Design and implementation of kubelet garbage collection mechanism to protect nodes from being preempted by containers image GC high threshold

2022-07-03 04:02:00 【Zhang quandan, Foxconn quality inspector】

kubelet Protect nodes from being preempted by containers ：
  • If Image takes up disk space The proportion exceeds the high water level （ Default The value is 90%, You can use the parameter ImageGCHighThresholdPercent To configure ）,kubelet Will clean up no Image used .
  • When node CPU、Memory Or the disk is less than a certain value or proportion （ By the parameter EvictionHard To configure ） when ,kubelet Will expel low priority Pod（ for example BestEffort Of Pod）. Through these operations , guarantee Existing on node Pod Can be guaranteed in QoS（Quality of Service） Continue normal operation under .

kubernetes The garbage collection mechanism in is mainly composed of two parts ：

  • One is by kube-controller-manager Medium gc controller Automatic recovery kubernetes Objects deleted in and their dependent objects ;
  • The second is to recycle the exited containers on each node and when node Recycle container images that are no longer used when the disk resources on the are insufficient ;

The main analysis of this paper kubelet Garbage collection mechanism in , The main purpose of garbage collection is to save resources on the host ,gc controller The recycling mechanism of can refer to previous articles  garbage collector controller Source code analysis .

kubelet There are three main parameters related to container garbage collection in :

  • --maximum-dead-containers-per-container: It means a pod The maximum number of containers that can be saved that have been stopped , The default is 1;（maxPerPodContainerCount）
  • --maximum-dead-containers： One node How many stopped containers can be reserved on the , The default is -1, There is no limit ;
  • --minimum-container-ttl-duration： The minimum time that an exited container can survive , The default is 0s;

There are three main parameters related to image recycling ：

  • --image-gc-high-threshold： When kubelet When the disk reaches ,kubelet Start reclaiming the image , The default is 85% Start recycling , Root directory and data disk ;
  • --image-gc-low-threshold： When recycling an image, stop recycling when the disk utilization is reduced to , The default is 80%;
  • --minimum-image-ttl-duration： The minimum lifetime of an unused image before it is recycled , The default is 2m0s;

 kubelet The recycling process of the medium container is as follows :
pod The exit time of the container in exceeds --minimum-container-ttl-duration Will be marked as recyclable , One pod At most --maximum-dead-containers-per-container Containers that have stopped , One node At most --maximum-dead-containers Containers stopped . When recycling containers ,kubelet It will be sorted according to the exit time of the container , Recycle the container with the longest exit time first . It should be noted that ,kubelet When recycling, it will pod Medium container And sandboxes Recycle separately , After recycling the container, the corresponding log dir Also recycle ;

kubelet The image recycling process in is as follows :
When the disk utilization of the container image mount point file system is greater than --image-gc-high-threshold when （containerRuntime by docker when , The image storage directory defaults to  /var/lib/docker）,kubelet Start deleting unused container images in the node , Until disk usage drops to --image-gc-low-threshold  Stop the garbage collection of the image .

 

 

 

kubelet GarbageCollect Source code analysis

kubernetes edition ：v1.16

GarbageCollect Is in kubelet Started after object initialization , stay  createAndInitKubelet  Method first calls  kubelet.NewMainKubelet  Initialize the kubelet object , Then call  k.StartGarbageCollection  Launched the GarbageCollect.

k8s.io/kubernetes/cmd/kubelet/app/server.go:1089

1
2
3
4
5
6
7
8
9
10
11
12
13
14

func createAndInitKubelet(......) {
    k, err = kubelet.NewMainKubelet(
        ......
    )
    if err != nil {
        return nil, err
    }

    k.BirthCry()

    k.StartGarbageCollection()

    return k, nil
}

k.StartGarbageCollection

stay kubelet The life cycle of the image in and the life cycle of the container are through imageManager and containerGC Managed . stay  StartGarbageCollection  The container and image garbage collection tasks will be started in the method , Its main logic is ：

  • 1、 start-up containerGC goroutine,ContainerGC The interval time defaults to 1 minute ;
  • 2、 Check  --image-gc-high-threshold  The value of the parameter , if 100 Then disable imageGC;
  • 3、 start-up imageGC goroutine,imageGC The interval time defaults to 5 minute ;

k8s.io/kubernetes/pkg/kubelet/kubelet.go:1270

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

func (kl *Kubelet) StartGarbageCollection() {
    loggedContainerGCFailure := false
    
    // 1、 Start the container garbage collection service 
    go wait.Until(func() {
        if err := kl.containerGC.GarbageCollect(); err != nil {
            loggedContainerGCFailure = true
        } else {
            var vLevel klog.Level = 4
            if loggedContainerGCFailure {
                vLevel = 1
                loggedContainerGCFailure = false
            }

            klog.V(vLevel).Infof("Container garbage collection succeeded")
        }
    }, ContainerGCPeriod, wait.NeverStop)
    
    // 2、 Check  ImageGCHighThresholdPercent  The value of the parameter 
    if kl.kubeletConfiguration.ImageGCHighThresholdPercent == 100 {
        return
    }

    // 3、 Start the image garbage collection service 
    prevImageGCFailed := false
    go wait.Until(func() {
        if err := kl.imageManager.GarbageCollect(); err != nil {
            ......
            prevImageGCFailed = true
        } else {
            var vLevel klog.Level = 4
            if prevImageGCFailed {
                vLevel = 1
                prevImageGCFailed = false
            }
        }
    }, ImageGCPeriod, wait.NeverStop)
}

kl.containerGC.GarbageCollect

kl.containerGC.GarbageCollect  It's called ContainerGC manager The method in ,ContainerGC Is in  NewMainKubelet  Initialized in ,ContainerGC You need to specify a runtime, The runtime namely ContainerRuntime, stay kubelet Middle is kubeGenericRuntimeManager, Also in  NewMainKubelet  Initialized in .

k8s.io/kubernetes/pkg/kubelet/kubelet.go

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

func NewMainKubelet(){
    ......
    // MinAge、MaxPerPodContainer、MaxContainers  They are related to container garbage collection mentioned at the beginning of the above article 
    //  Three parameters 
    containerGCPolicy := kubecontainer.ContainerGCPolicy{
        MinAge:             minimumGCAge.Duration,
        MaxPerPodContainer: int(maxPerPodContainerCount),
        MaxContainers:      int(maxContainerCount),
    }
    
    //  initialization  containerGC  modular 
    containerGC, err := kubecontainer.NewContainerGC(klet.containerRuntime, containerGCPolicy, klet.sourcesReady)
    if err != nil {
        return nil, err
    }
    ......
}

Here are ContainerGC Initialization and GarbageCollect Start of ：

k8s.io/kubernetes/pkg/kubelet/container/container_gc.go:68

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

func NewContainerGC(runtime Runtime, policy ContainerGCPolicy, sourcesReadyProvider SourcesReadyProvider) (ContainerGC, error) {
    if policy.MinAge < 0 {
        return nil, fmt.Errorf("invalid minimum garbage collection age: %v", policy.MinAge)
    }

    return &realContainerGC{
        runtime:              runtime,
        policy:               policy,
        sourcesReadyProvider: sourcesReadyProvider,
    }, nil
}

func (cgc *realContainerGC) GarbageCollect() error {
    return cgc.runtime.GarbageCollect(cgc.policy, cgc.sourcesReadyProvider.AllReady(), false)
}

You can see ,ContainerGC Medium GarbageCollect Finally, call runtime Medium GarbageCollect Method ,runtime namely kubeGenericRuntimeManager.

cgc.runtime.GarbageCollect

cgc.runtime.GarbageCollect The implementation of kubeGenericRuntimeManager in , Its main logic is ：

  • 1、 Recycling pod Medium container;
  • 2、 Recycling pod Medium sandboxes;
  • 3、 Recycling pod as well as container Of log dir;

k8s.io/kubernetes/pkg/kubelet/kuberuntime/kuberuntime_gc.go:378

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

func (cgc *containerGC) GarbageCollect(gcPolicy kubecontainer.ContainerGCPolicy, allSourcesReady bool, evictTerminatedPods bool) error {
    errors := []error{}
    // 1、 Recycling  pod  Medium  container
    if err := cgc.evictContainers(gcPolicy, allSourcesReady, evictTerminatedPods); err != nil {
        errors = append(errors, err)
    }

    // 2、 Recycling  pod  Medium  sandboxes
    if err := cgc.evictSandboxes(evictTerminatedPods); err != nil {
        errors = append(errors, err)
    }

    // 3、 Recycling  pod  as well as  container  Of  log dir
    if err := cgc.evictPodLogsDirectories(allSourcesReady); err != nil {
        errors = append(errors, err)
    }
    return utilerrors.NewAggregate(errors)
}

cgc.evictContainers

stay  cgc.evictContainers  All recyclable containers will be recycled in the method , Its main logic is ：

  • 1、 First call  cgc.evictableContainers  Get recyclable containers as evictUnits, Recyclable containers refer to non running Status and creation time exceeds MinAge,evictUnits The array contains pod And container Correspondence of ;
  • 2、 Recycling deleted State and terminated State of pod, Traverse evictUnits, if pod Is it in deleted perhaps terminated state , Call  cgc.removeOldestN  Recycling pod All containers in .deleted State means pod Has been deleted or its  status.phase  by failed And its  status.reason  by evicted perhaps pod.deletionTimestamp != nil And pod Of all containers in status by terminated perhaps waiting state ,terminated State means pod be in Failed perhaps succeeded state ;
  • 3、 For non deleted perhaps terminated State of pod, call  cgc.enforceMaxContainersPerEvictUnit  Keep it  MaxPerPodContainer  Containers that have exited , Sort according to the exit time of the container, and delete the one with the longest exit time first ,MaxPerPodContainer  As mentioned above , It means a pod The maximum number of containers that can be saved that have been stopped , The default is 1, have access to  --maximum-dead-containers-per-container  Specified at startup ;
  • 4、 if kubelet Specified at startup --maximum-dead-containers（ The default is -1 There is no limit to ）, That is to say, it needs to be node Keep the number of containers exited , if node The number of containers that have been stopped on the reservation exceeds  --maximum-dead-containers, First calculate the need for each pod How many exited containers should be reserved to ensure that the total number does not exceed  --maximum-dead-containers  Value , If the calculation result is less than 1 Then take 1, That is, keep at least one , Then delete each pod Containers that don't need to be kept in , If node The number of containers that have been stopped on the reservation still exceeds the maximum value that needs to be reserved , Will evictUnits The containers in are sorted according to the exit time. Delete the container with the longest exit time , send node The number of containers that have been stopped on the reservation meets  --maximum-dead-containers  value ;

k8s.io/kubernetes/pkg/kubelet/kuberuntime/kuberuntime_gc.go:222

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

func (cgc *containerGC) evictContainers(gcPolicy kubecontainer.ContainerGCPolicy, allSourcesReady bool, evictTerminatedPods bool) error {
    // 1、 Get the list of containers that can be recycled 
    evictUnits, err := cgc.evictableContainers(gcPolicy.MinAge)
    if err != nil {
        return err
    }

    // 2、 Recycling  Deleted  State and  Terminated  State of  pod, here  allSourcesReady  finger  kubelet 
    //     Three kinds of support  podSource  Are they all available 
    if allSourcesReady {
        for key, unit := range evictUnits {
            if cgc.podStateProvider.IsPodDeleted(key.uid) || (cgc.podStateProvider.IsPodTerminated(key.uid) && evictTerminatedPods) {
                cgc.removeOldestN(unit, len(unit)) 
                delete(evictUnits, key)
            }
        }
    }
    
    // 3、 For the wrong  Deleted  State and  Terminated  State of  pod  Retain  MaxPerPodContainer  Containers that have exited 
    if gcPolicy.MaxPerPodContainer >= 0 {
        cgc.enforceMaxContainersPerEvictUnit(evictUnits, gcPolicy.MaxPerPodContainer)
    }

    // 4、 if  kubelet  Specified at startup  --maximum-dead-containers（ The default is  -1  There is no limit to ） Parameters ,
    //    At this point, you need to be  node  The number of containers reserved for exit cannot exceed  --maximum-dead-containers  individual 
    if gcPolicy.MaxContainers >= 0 && evictUnits.NumContainers() > gcPolicy.MaxContainers {
        numContainersPerEvictUnit := gcPolicy.MaxContainers / evictUnits.NumEvictUnits()
        if numContainersPerEvictUnit < 1 {
            numContainersPerEvictUnit = 1
        }
        cgc.enforceMaxContainersPerEvictUnit(evictUnits, numContainersPerEvictUnit)

        numContainers := evictUnits.NumContainers()
        if numContainers > gcPolicy.MaxContainers {
            flattened := make([]containerGCInfo, 0, numContainers)
            for key := range evictUnits {
                flattened = append(flattened, evictUnits[key]...)
            }
            sort.Sort(byCreated(flattened))

            cgc.removeOldestN(flattened, numContainers-gcPolicy.MaxContainers)
        }
    }
    return nil
}

cgc.evictSandboxes

cgc.evictSandboxes  Method will recycle all recyclable sandboxes, Its main logic is ：

  • 1、 First of all get node On all the containers and sandboxes;
  • 2、 structure sandboxes And pod And save it in sandboxesByPodUID in ;
  • 3、 Yes sandboxesByPodUID The list is sorted by creation time ;
  • 4、 if sandboxes Where pod be in deleted state , Delete the pod All of the sandboxes Otherwise, only the one with the shortest exit time is reserved sandboxes,deleted The status is above  cgc.evictContainers  The method has been explained ;

k8s.io/kubernetes/pkg/kubelet/kuberuntime/kuberuntime_gc.go:274

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54

func (cgc *containerGC) evictSandboxes(evictTerminatedPods bool) error {
    // 1、 obtain  node  On all the  container
    containers, err := cgc.manager.getKubeletContainers(true)
    if err != nil {
        return err
    }
    // 2、 obtain  node  On all the  sandboxes
    sandboxes, err := cgc.manager.getKubeletSandboxes(true)
    if err != nil {
        return err
    }

    // 3、 Collect all  container  Of  PodSandboxId 
    sandboxIDs := sets.NewString()
    for _, container := range containers {
        sandboxIDs.Insert(container.PodSandboxId)
    }

    // 4、 structure  sandboxes  And  pod  And save it in  sandboxesByPodUID  in 
    sandboxesByPod := make(sandboxesByPodUID)
    for _, sandbox := range sandboxes {
        podUID := types.UID(sandbox.Metadata.Uid)
        sandboxInfo := sandboxGCInfo{
            id:         sandbox.Id,
            createTime: time.Unix(0, sandbox.CreatedAt),
        }
        
        if sandbox.State == runtimeapi.PodSandboxState_SANDBOX_READY {
            sandboxInfo.active = true
        }

        if sandboxIDs.Has(sandbox.Id) {
            sandboxInfo.active = true
        }

        sandboxesByPod[podUID] = append(sandboxesByPod[podUID], sandboxInfo)
    }

    // 5、 Yes  sandboxesByPod  Sort 
    for uid := range sandboxesByPod {
        sort.Sort(sandboxByCreated(sandboxesByPod[uid]))
    }

    // 6、 Traverse  sandboxesByPod, if  sandboxes  Where  pod  be in  deleted  state ,
    //  Delete the  pod  All of the  sandboxes  Otherwise, only the one with the shortest exit time is reserved  sandboxes
    for podUID, sandboxes := range sandboxesByPod {
        if cgc.podStateProvider.IsPodDeleted(podUID) || (cgc.podStateProvider.IsPodTerminated(podUID) && evictTerminatedPods) {
            cgc.removeOldestNSandboxes(sandboxes, len(sandboxes))
        } else {
            cgc.removeOldestNSandboxes(sandboxes, len(sandboxes)-1)
        }
    }
    return nil
}

cgc.evictPodLogsDirectories

cgc.evictPodLogsDirectories  Method will recycle all recyclable pod as well as container Of log dir, Its main logic is ：

  • 1、 Recycle first deleted state pod logs dir, Traverse pod logs dir /var/log/pods,/var/log/pods  by pod logs Default directory for ,pod logs dir The format is  /var/log/pods/NAMESPACE_NAME_UID, analysis pod logs dir obtain pod uid, Judge pod Is it in deleted state , If in deleted The status is deleted logs dir;

  • 2、 Recycling deleted state container logs Linked directory ,/var/log/containers  by container log Default directory for , It will soft link to pod Of log dir Next , for example ：

    1

    /var/log/containers/storage-provisioner_kube-system_storage-provisioner-acc8386e409dfb3cc01618cbd14c373d8ac6d7f0aaad9ced018746f31d0081e2.log -> /var/log/pods/kube-system_storage-provisioner_b448e496-eb5d-4d71-b93f-ff7ff77d2348/storage-provisioner/0.log

k8s.io/kubernetes/pkg/kubelet/kuberuntime/kuberuntime_gc.go:333

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

func (cgc *containerGC) evictPodLogsDirectories(allSourcesReady bool) error {
    osInterface := cgc.manager.osInterface
    // 1、 Recycling  deleted  state  pod logs dir
    if allSourcesReady {
        dirs, err := osInterface.ReadDir(podLogsRootDirectory)
        if err != nil {
            return fmt.Errorf("failed to read podLogsRootDirectory %q: %v", podLogsRootDirectory, err)
        }
        for _, dir := range dirs {
            name := dir.Name()
            podUID := parsePodUIDFromLogsDirectory(name)
            if !cgc.podStateProvider.IsPodDeleted(podUID) {
                continue
            }
            err := osInterface.RemoveAll(filepath.Join(podLogsRootDirectory, name))
            if err != nil {
                klog.Errorf("Failed to remove pod logs directory %q: %v", name, err)
            }
        }
    }
    // 2、 Recycling  deleted  state  container logs  Linked directory 
    logSymlinks, _ := osInterface.Glob(filepath.Join(legacyContainerLogsDir, fmt.Sprintf("*.%s", legacyLogSuffix)))
    for _, logSymlink := range logSymlinks {
        if _, err := osInterface.Stat(logSymlink); os.IsNotExist(err) {
            err := osInterface.Remove(logSymlink)
            if err != nil {
                klog.Errorf("Failed to remove container log dead symlink %q: %v", logSymlink, err)
            }
        }
    }
    return nil
}

kl.imageManager.GarbageCollect

The main process of container recycling has been analyzed above , Next, we will continue to analyze the process of image recycling ,kl.imageManager.GarbageCollect  It is the method to start the image recycling task , The image recycling process is in imageManager In the , First of all, understand imageManager The initialization ,imageManager Also in  NewMainKubelet  Method .

k8s.io/kubernetes/pkg/kubelet/kubelet.go

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

func NewMainKubelet(){
    ......    
    //  Three parameters need to be specified during initialization , Three parameters have been mentioned above 
    imageGCPolicy := images.ImageGCPolicy{
        MinAge:               kubeCfg.ImageMinimumGCAge.Duration,
        HighThresholdPercent: int(kubeCfg.ImageGCHighThresholdPercent),
        LowThresholdPercent:  int(kubeCfg.ImageGCLowThresholdPercent),
    }
    ......
    imageManager, err := images.NewImageGCManager(klet.containerRuntime, klet.StatsProvider, kubeDeps.Recorder, nodeRef, imageGCPolicy, crOptions.PodSandboxImage)
    if err != nil {
        return nil, fmt.Errorf("failed to initialize image manager: %v", err)
    }
    klet.imageManager = imageManager
    ......
}

kl.imageManager.GarbageCollect  The main logic of the method is ：

  • 1、 First call  im.statsProvider.ImageFsStats  Get the disk information of the container image storage directory mount point file system ;
  • 2、 Get the of the mount point available and capacity Information and calculate its utilization ;
  • 3、 If the utilization rate is greater than  HighThresholdPercent, First of all, according to the  LowThresholdPercent  Value to calculate the amount of disk that needs to be released , And then call  im.freeSpace  Release unused image Until the disk idle rate is met ;

k8s.io/kubernetes/pkg/kubelet/images/image_gc_manager.go:269

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

func (im *realImageGCManager) GarbageCollect() error {
    // 1、 Get the disk information of the container image storage directory mount point file system 
    fsStats, err := im.statsProvider.ImageFsStats()
    if err != nil {
        return err
    }

    var capacity, available int64
    if fsStats.CapacityBytes != nil {
        capacity = int64(*fsStats.CapacityBytes)
    }
    if fsStats.AvailableBytes != nil {
        available = int64(*fsStats.AvailableBytes)
    }

    if available > capacity {
        available = capacity
    }

    if capacity == 0 {
        err := goerrors.New("invalid capacity 0 on image filesystem")
        im.recorder.Eventf(im.nodeRef, v1.EventTypeWarning, events.InvalidDiskCapacity, err.Error())
        return err
    }
    // 2、 If the utilization rate is greater than  HighThresholdPercent, At this point, you need to recycle the image 
    usagePercent := 100 - int(available*100/capacity)
    if usagePercent >= im.policy.HighThresholdPercent {
        // 3、 Calculate the amount of disk that needs to be freed 
        amountToFree := capacity*int64(100-im.policy.LowThresholdPercent)/100 - available
        
        // 4、 call  im.freeSpace  Recycle unused image information 
        freed, err := im.freeSpace(amountToFree, time.Now())
        if err != nil {
            return err
        }

        if freed < amountToFree {
            err := fmt.Errorf("failed to garbage collect required amount of images. Wanted to free %d bytes, but freed %d bytes", amountToFree, freed)
            im.recorder.Eventf(im.nodeRef, v1.EventTypeWarning, events.FreeDiskSpaceFailed, err.Error())
            return err
        }
    }

    return nil
}

im.freeSpace

im.freeSpace  It is a method of recycling unused images , Its main logic is ：

  • 1、 First call  im.detectImages  Get used images List as imagesInUse;
  • 2、 Traverse  im.imageRecords  according to imagesInUse Get all unused images Information ,im.imageRecords  Record node On all the images Information about ;
  • 3、 According to the use time, the unused images Sort the list ;
  • 4、 Traverse unused images List and then call  im.runtime.RemoveImage  delete mirror , Until all unused are recycled images Or meet the idle rate ;

k8s.io/kubernetes/pkg/kubelet/images/image_gc_manager.go:328

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54

func (im *realImageGCManager) freeSpace(bytesToFree int64, freeTime time.Time) (int64, error) {
    // 1、 Get used  images  list 
    imagesInUse, err := im.detectImages(freeTime)
    if err != nil {
        return 0, err
    }

    im.imageRecordsLock.Lock()
    defer im.imageRecordsLock.Unlock()

    // 2、 Get all unused  images  Information 
    images := make([]evictionInfo, 0, len(im.imageRecords))
    for image, record := range im.imageRecords {
        if isImageUsed(image, imagesInUse) {
            klog.V(5).Infof("Image ID %s is being used", image)
            continue
        }
        images = append(images, evictionInfo{
            id:          image,
            imageRecord: *record,
        })
    }
    // 3、 Sort by image usage time 
    sort.Sort(byLastUsedAndDetected(images))
    // 4、 Recycle unused images 
    var deletionErrors []error
    spaceFreed := int64(0)
    for _, image := range images {
        if image.lastUsed.Equal(freeTime) || image.lastUsed.After(freeTime) {
            continue
        }

        if freeTime.Sub(image.firstDetected) < im.policy.MinAge {
            continue
        }

        // 5、 call  im.runtime.RemoveImage  delete mirror 
        err := im.runtime.RemoveImage(container.ImageSpec{Image: image.id})
        if err != nil {
            deletionErrors = append(deletionErrors, err)
            continue
        }
        delete(im.imageRecords, image.id)
        spaceFreed += image.size
        if spaceFreed >= bytesToFree {
            break
        }
    }

    if len(deletionErrors) > 0 {
        return spaceFreed, fmt.Errorf("wanted to free %d bytes, but freed %d bytes space with errors in image deletion: %v", bytesToFree, spaceFreed,   errors.NewAggregate(deletionErrors))
    }
    return spaceFreed, nil
}

summary

This paper mainly analyzes kubelet Implementation of garbage collection mechanism in ,kubelet It will be recycled regularly node The container that has exited on has been pawned node When disk resources are insufficient, recycle images that are no longer used to free disk resources , Container and image recycling strategies are mainly through kubelet The threshold value of several parameters in .

 

原网站

版权声明
本文为[Zhang quandan, Foxconn quality inspector]所创，转载请带上原文链接，感谢
https://yzsam.com/2022/02/202202150718296229.html

边栏推荐

猜你喜欢

随机推荐