配置 kubelet 镜像凭证提供程序
Kubernetes v1.26 [stable]
从 Kubernetes v1.20 开始,kubelet 可以使用执行插件 (exec plugins) 动态检索容器镜像仓库的凭据。kubelet 和执行插件通过标准 I/O (stdin, stdout 和 stderr) 使用 Kubernetes 版本化 API 进行通信。这些插件允许 kubelet 动态请求容器镜像仓库的凭据,而不是将静态凭据存储在磁盘上。例如,插件可以与本地元数据服务器通信,以检索 kubelet 正在拉取镜像所需的短期凭据。
如果满足以下任一条件,你可能会对使用此功能感兴趣
- 需要调用云服务提供商的 API 来检索仓库的认证信息。
- 凭据过期时间较短,需要频繁请求新凭据。
- 将仓库凭据存储在磁盘或 imagePullSecrets 中是不可接受的。
本指南演示如何配置 kubelet 的镜像凭据提供者插件机制。
用于镜像拉取的 ServiceAccount 令牌
Kubernetes v1.33 [alpha]
(默认未启用)从 Kubernetes v1.33 开始,kubelet 可以配置为将绑定到正在执行镜像拉取的 Pod 的 ServiceAccount 令牌发送给凭据提供者插件。
这使得插件可以交换该令牌,获取访问镜像仓库所需的凭据。
要启用此功能,必须在 kubelet 上启用 KubeletServiceAccountTokenForCredentialProviders
特性门控,并且必须在插件的 CredentialProviderConfig
文件中设置 tokenAttributes
字段。
tokenAttributes
字段包含将传递给插件的 ServiceAccount 令牌的相关信息,包括令牌的预期受众以及插件是否要求 Pod 具有 ServiceAccount。
使用 ServiceAccount 令牌凭据可以实现以下用例
- 避免需要基于 kubelet/节点的身份来从仓库拉取镜像。
- 允许工作负载基于其自身的运行时身份拉取镜像,而无需长期存在/持久化的密钥。
开始之前
- 你需要一个带有支持 kubelet 凭据提供者插件的节点的 Kubernetes 集群。此支持在 Kubernetes 1.33 中可用;Kubernetes v1.24 和 v1.25 将其作为 Beta 功能包含在内,并默认启用。
- 如果你正在配置一个需要 ServiceAccount 令牌的凭据提供者插件,你需要一个运行 Kubernetes v1.33 或更高版本的节点的 Kubernetes 集群,并且在 kubelet 上启用了
KubeletServiceAccountTokenForCredentialProviders
特性门控。 - 一个可用的凭据提供者执行插件实现。你可以构建自己的插件或使用云提供商提供的插件。
要检查版本,输入 kubectl version
。
在节点上安装插件
凭据提供者插件是一个可由 kubelet 运行的可执行二进制文件。确保插件二进制文件存在于集群中的每个节点上,并存储在一个已知目录中。该目录将在稍后配置 kubelet 标志时需要。
配置 Kubelet
为了使用此功能,kubelet 需要设置两个标志
--image-credential-provider-config
- 凭据提供者插件配置文件的路径。--image-credential-provider-bin-dir
- 凭据提供者插件二进制文件所在的目录路径。
配置 kubelet 凭据提供者
kubelet 读取传递给 --image-credential-provider-config
的配置文件,以确定应对哪些容器镜像调用哪些执行插件。如果你正在使用 基于 ECR 的插件,以下是一个你可能会使用的示例配置文件
apiVersion: kubelet.config.k8s.io/v1
kind: CredentialProviderConfig
# providers is a list of credential provider helper plugins that will be enabled by the kubelet.
# Multiple providers may match against a single image, in which case credentials
# from all providers will be returned to the kubelet. If multiple providers are called
# for a single image, the results are combined. If providers return overlapping
# auth keys, the value from the provider earlier in this list is used.
providers:
# name is the required name of the credential provider. It must match the name of the
# provider executable as seen by the kubelet. The executable must be in the kubelet's
# bin directory (set by the --image-credential-provider-bin-dir flag).
- name: ecr-credential-provider
# matchImages is a required list of strings used to match against images in order to
# determine if this provider should be invoked. If one of the strings matches the
# requested image from the kubelet, the plugin will be invoked and given a chance
# to provide credentials. Images are expected to contain the registry domain
# and URL path.
#
# Each entry in matchImages is a pattern which can optionally contain a port and a path.
# Globs can be used in the domain, but not in the port or the path. Globs are supported
# as subdomains like '*.k8s.io' or 'k8s.*.io', and top-level-domains such as 'k8s.*'.
# Matching partial subdomains like 'app*.k8s.io' is also supported. Each glob can only match
# a single subdomain segment, so `*.io` does **not** match `*.k8s.io`.
#
# A match exists between an image and a matchImage when all of the below are true:
# - Both contain the same number of domain parts and each part matches.
# - The URL path of an matchImages must be a prefix of the target image URL path.
# - If the matchImages contains a port, then the port must match in the image as well.
#
# Example values of matchImages:
# - 123456789.dkr.ecr.us-east-1.amazonaws.com
# - *.azurecr.io
# - gcr.io
# - *.*.registry.io
# - registry.io:8080/path
matchImages:
- "*.dkr.ecr.*.amazonaws.com"
- "*.dkr.ecr.*.amazonaws.com.cn"
- "*.dkr.ecr-fips.*.amazonaws.com"
- "*.dkr.ecr.us-iso-east-1.c2s.ic.gov"
- "*.dkr.ecr.us-isob-east-1.sc2s.sgov.gov"
# defaultCacheDuration is the default duration the plugin will cache credentials in-memory
# if a cache duration is not provided in the plugin response. This field is required.
defaultCacheDuration: "12h"
# Required input version of the exec CredentialProviderRequest. The returned CredentialProviderResponse
# MUST use the same encoding version as the input. Current supported values are:
# - credentialprovider.kubelet.k8s.io/v1
apiVersion: credentialprovider.kubelet.k8s.io/v1
# Arguments to pass to the command when executing it.
# +optional
# args:
# - --example-argument
# Env defines additional environment variables to expose to the process. These
# are unioned with the host's environment, as well as variables client-go uses
# to pass argument to the plugin.
# +optional
env:
- name: AWS_PROFILE
value: example_profile
# tokenAttributes is the configuration for the service account token that will be passed to the plugin.
# The credential provider opts in to using service account tokens for image pull by setting this field.
# if this field is set without the `KubeletServiceAccountTokenForCredentialProviders` feature gate enabled,
# kubelet will fail to start with invalid configuration error.
# +optional
tokenAttributes:
# serviceAccountTokenAudience is the intended audience for the projected service account token.
# +required
serviceAccountTokenAudience: "<audience for the token>"
# requireServiceAccount indicates whether the plugin requires the pod to have a service account.
# If set to true, kubelet will only invoke the plugin if the pod has a service account.
# If set to false, kubelet will invoke the plugin even if the pod does not have a service account
# and will not include a token in the CredentialProviderRequest. This is useful for plugins
# that are used to pull images for pods without service accounts (e.g., static pods).
# +required
requireServiceAccount: true
# requiredServiceAccountAnnotationKeys is the list of annotation keys that the plugin is interested in
# and that are required to be present in the service account.
# The keys defined in this list will be extracted from the corresponding service account and passed
# to the plugin as part of the CredentialProviderRequest. If any of the keys defined in this list
# are not present in the service account, kubelet will not invoke the plugin and will return an error.
# This field is optional and may be empty. Plugins may use this field to extract additional information
# required to fetch credentials or allow workloads to opt in to using service account tokens for image pull.
# If non-empty, requireServiceAccount must be set to true.
# The keys defined in this list must be unique and not overlap with the keys defined in the
# optionalServiceAccountAnnotationKeys list.
# +optional
requiredServiceAccountAnnotationKeys:
- "example.com/required-annotation-key-1"
- "example.com/required-annotation-key-2"
# optionalServiceAccountAnnotationKeys is the list of annotation keys that the plugin is interested in
# and that are optional to be present in the service account.
# The keys defined in this list will be extracted from the corresponding service account and passed
# to the plugin as part of the CredentialProviderRequest. The plugin is responsible for validating the
# existence of annotations and their values. This field is optional and may be empty.
# Plugins may use this field to extract additional information required to fetch credentials.
# The keys defined in this list must be unique and not overlap with the keys defined in the
# requiredServiceAccountAnnotationKeys list.
# +optional
optionalServiceAccountAnnotationKeys:
- "example.com/optional-annotation-key-1"
- "example.com/optional-annotation-key-2"
providers
字段是 kubelet 使用的已启用插件列表。每个条目都有几个必需字段
name
:插件的名称,必须与--image-credential-provider-bin-dir
指定目录中存在的可执行二进制文件的名称匹配。matchImages
:用于匹配镜像以确定是否应调用此提供者的一系列字符串。详情见下文。defaultCacheDuration
:如果插件未指定缓存持续时间,kubelet 在内存中缓存凭据的默认持续时间。apiVersion
:kubelet 和执行插件通信时将使用的 API 版本。
每个凭据提供者也可以提供可选的参数和环境变量。请咨询插件实现者,以确定给定插件所需的参数和环境变量集合。
如果你正在使用 KubeletServiceAccountTokenForCredentialProviders 特性门控,并通过设置 tokenAttributes 字段将插件配置为使用 ServiceAccount 令牌,则需要以下字段
serviceAccountTokenAudience
:投射的 ServiceAccount 令牌的预期受众。此字段不能为空字符串。requireServiceAccount
:插件是否要求 Pod 具有 ServiceAccount。- 如果设置为
true
,kubelet 将仅在 Pod 具有 ServiceAccount 时调用插件。 - 如果设置为
false
,即使 Pod 没有 ServiceAccount,kubelet 也会调用插件,并且不会在CredentialProviderRequest
中包含令牌。
- 如果设置为
这对于用于拉取没有 ServiceAccount 的 Pod(例如静态 Pod)的镜像的插件非常有用。
配置镜像匹配
kubelet 使用每个凭据提供者的 matchImages
字段来确定是否应针对 Pod 正在使用的给定镜像调用插件。matchImages
中的每个条目都是一个镜像模式,可以可选地包含端口和路径。可以在域中使用通配符(Glob),但不能在端口或路径中使用。Glob 支持子域,例如 *.k8s.io
或 k8s.*.io
,以及顶级域,例如 k8s.*
。也支持匹配部分子域,例如 app*.k8s.io
。每个通配符只能匹配一个子域段,所以 *.io
不会匹配 *.k8s.io
。
当以下所有条件都为真时,镜像名称与 matchImage
条目匹配
- 两者包含相同数量的域部分,并且每个部分都匹配。
- 匹配镜像的 URL 路径必须是目标镜像 URL 路径的前缀。
- 如果 matchImages 包含端口,则镜像中的端口也必须匹配。
matchImages
模式的一些示例值
123456789.dkr.ecr.us-east-1.amazonaws.com
*.azurecr.io
gcr.io
*.*.registry.io
foo.registry.io:8080/path
下一步
- 阅读 kubelet 配置 API (v1) 参考 中关于
CredentialProviderConfig
的详细信息。 - 阅读 kubelet 凭据提供者 API 参考 (v1)。