垃圾回收
Garbage collection is a collective term for the various mechanisms Kubernetes uses to clean up cluster resources. This allows the clean up of resources like the following
- Terminated pods
- Completed Jobs
- Objects without owner references
- Unused containers and container images
- Dynamically provisioned PersistentVolumes with a StorageClass reclaim policy of Delete
- Stale or expired CertificateSigningRequests (CSRs)
- Nodes deleted in the following scenarios
- On a cloud when the cluster uses a cloud controller manager
- On-premises when the cluster uses an addon similar to a cloud controller manager
- Node Lease objects
Owners and dependents
Many objects in Kubernetes link to each other through owner references. Owner references tell the control plane which objects are dependent on others. Kubernetes uses owner references to give the control plane, and other API clients, the opportunity to clean up related resources before deleting an object. In most cases, Kubernetes manages owner references automatically.
Ownership is different from the labels and selectors mechanism that some resources also use. For example, consider a Service that creates EndpointSlice
objects. The Service uses labels to allow the control plane to determine which EndpointSlice
objects are used for that Service. In addition to the labels, each EndpointSlice
that is managed on behalf of a Service has an owner reference. Owner references help different parts of Kubernetes avoid interfering with objects they don’t control.
Note
Cross-namespace owner references are disallowed by design. Namespaced dependents can specify cluster-scoped or namespaced owners. A namespaced owner must exist in the same namespace as the dependent. If it does not, the owner reference is treated as absent, and the dependent is subject to deletion once all owners are verified absent.
Cluster-scoped dependents can only specify cluster-scoped owners. In v1.20+, if a cluster-scoped dependent specifies a namespaced kind as an owner, it is treated as having an unresolvable owner reference, and is not able to be garbage collected.
In v1.20+, if the garbage collector detects an invalid cross-namespace ownerReference
, or a cluster-scoped dependent with an ownerReference
referencing a namespaced kind, a warning Event with a reason of OwnerRefInvalidNamespace
and an involvedObject
of the invalid dependent is reported. You can check for that kind of Event by running kubectl get events -A --field-selector=reason=OwnerRefInvalidNamespace
.
Cascading deletion
Kubernetes checks for and deletes objects that no longer have owner references, like the pods left behind when you delete a ReplicaSet. When you delete an object, you can control whether Kubernetes deletes the object's dependents automatically, in a process called cascading deletion. There are two types of cascading deletion, as follows
- Foreground cascading deletion
- Background cascading deletion
You can also control how and when garbage collection deletes resources that have owner references using Kubernetes finalizers.
Foreground cascading deletion
In foreground cascading deletion, the owner object you're deleting first enters a deletion in progress state. In this state, the following happens to the owner object
- The Kubernetes API server sets the object's
metadata.deletionTimestamp
field to the time the object was marked for deletion. - The Kubernetes API server also sets the
metadata.finalizers
field toforegroundDeletion
. - The object remains visible through the Kubernetes API until the deletion process is complete.
After the owner object enters the deletion in progress state, the controller deletes dependents it knows about. After deleting all the dependent objects it knows about, the controller deletes the owner object. At this point, the object is no longer visible in the Kubernetes API.
During foreground cascading deletion, the only dependents that block owner deletion are those that have the ownerReference.blockOwnerDeletion=true
field and are in the garbage collection controller cache. The garbage collection controller cache may not contain objects whose resource type cannot be listed / watched successfully, or objects that are created concurrent with deletion of an owner object. See Use foreground cascading deletion to learn more.
Background cascading deletion
In background cascading deletion, the Kubernetes API server deletes the owner object immediately and the garbage collector controller (custom or default) cleans up the dependent objects in the background. If a finalizer exists, it ensures that objects are not deleted until all necessary clean-up tasks are completed. By default, Kubernetes uses background cascading deletion unless you manually use foreground deletion or choose to orphan the dependent objects.
See Use background cascading deletion to learn more.
Orphaned dependents
When Kubernetes deletes an owner object, the dependents left behind are called orphan objects. By default, Kubernetes deletes dependent objects. To learn how to override this behaviour, see Delete owner objects and orphan dependents.
Garbage collection of unused containers and images
The kubelet performs garbage collection on unused images every five minutes and on unused containers every minute. You should avoid using external garbage collection tools, as these can break the kubelet behavior and remove containers that should exist.
To configure options for unused container and image garbage collection, tune the kubelet using a configuration file and change the parameters related to garbage collection using the KubeletConfiguration
resource type.
Container image lifecycle
Kubernetes manages the lifecycle of all images through its image manager, which is part of the kubelet, with the cooperation of cadvisor. The kubelet considers the following disk usage limits when making garbage collection decisions
HighThresholdPercent
LowThresholdPercent
Disk usage above the configured HighThresholdPercent
value triggers garbage collection, which deletes images in order based on the last time they were used, starting with the oldest first. The kubelet deletes images until disk usage reaches the LowThresholdPercent
value.
Garbage collection for unused container images
Kubernetes v1.30 [beta]
(enabled by default: true)As a beta feature, you can specify the maximum time a local image can be unused for, regardless of disk usage. This is a kubelet setting that you configure for each node.
To configure the setting, you need to set a value for the imageMaximumGCAge
field in the kubelet configuration file.
The value is specified as a Kubernetes duration. See duration in the glossary for more details.
For example, you can set the configuration field to 12h45m
, which means 12 hours and 45 minutes.
Note
This feature does not track image usage across kubelet restarts. If the kubelet is restarted, the tracked image age is reset, causing the kubelet to wait the fullimageMaximumGCAge
duration before qualifying images for garbage collection based on image age.Container garbage collection
The kubelet garbage collects unused containers based on the following variables, which you can define
MinAge
: kubelet 可以回收容器的最小年龄。设置为0
可禁用此功能。MaxPerPodContainer
:每个 Pod 可以拥有的已终止容器的最大数量。设置为小于0
的值可禁用此功能。MaxContainers
:集群可以拥有的已终止容器的最大数量。设置为小于0
的值可禁用此功能。
除了这些变量之外,kubelet 还会回收未识别和已删除的容器,通常从最旧的容器开始。
在某些情况下,保留每个 Pod 的最大容器数 (MaxPerPodContainer
) 可能会超出全局已终止容器的总允许数量 (MaxContainers
),MaxPerPodContainer
和 MaxContainers
可能会相互冲突。在这种情况下,kubelet 会调整 MaxPerPodContainer
以解决冲突。最坏的情况是将 MaxPerPodContainer
降级为 1
并驱逐最旧的容器。此外,一旦由已删除的 Pod 拥有的容器早于 MinAge
,它们也会被移除。
Note
kubelet 只回收它管理的容器。配置垃圾回收
你可以通过配置特定于管理这些资源的控制器的选项来调整资源的垃圾回收。以下页面将向你展示如何配置垃圾回收
下一步
- 了解更多关于 Kubernetes 对象所有权的信息。
- 了解更多关于 Kubernetes 终结器的信息。
- 了解清理已完成 Job 的 TTL 控制器。