配置 kubelet 镜像凭证提供程序

功能状态: Kubernetes v1.26 [stable]

从 Kubernetes v1.20 开始,kubelet 可以使用 exec 插件动态地检索容器镜像仓库的凭证。kubelet 和 exec 插件通过 stdio(stdin、stdout 和 stderr)使用 Kubernetes 版本化 API 进行通信。这些插件允许 kubelet 动态请求容器仓库的凭证,而不是将静态凭证存储在磁盘上。例如,插件可以与本地元数据服务器通信,以检索由 kubelet 拉取的镜像的短期凭证。

如果以下任何条件为真,你可能对使用此功能感兴趣:

  • 需要调用云提供商服务 API 以检索注册表的身份验证信息。
  • 凭证有效期短,需要频繁请求新凭证。
  • 将注册表凭证存储在磁盘上或 imagePullSecrets 中是不可接受的。

本指南演示了如何配置 kubelet 的镜像凭证提供程序插件机制。

用于拉取镜像的服务账号令牌

特性状态: Kubernetes v1.34 [beta] (默认启用:true)

从 Kubernetes v1.33 开始,kubelet 可以配置为将绑定到要拉取镜像的 Pod 的服务账号令牌发送到凭证提供程序插件。

这允许插件将令牌换取访问镜像仓库的凭证。

要启用此功能,必须在 kubelet 上启用 `KubeletServiceAccountTokenForCredentialProviders` 特性门控,并且必须在插件的 `CredentialProviderConfig` 文件中设置 `tokenAttributes` 字段。

`tokenAttributes` 字段包含将传递给插件的服务账号令牌的信息,包括令牌的预期受众以及插件是否要求 Pod 具有服务账号。

使用服务账号令牌凭证可以实现以下用例:

  • 避免需要基于 kubelet/节点的身份才能从注册表拉取镜像。
  • 允许工作负载根据其自身的运行时身份拉取镜像,而无需长期/持久的 Secret。

准备工作

  • 你需要一个具有支持 kubelet 凭证提供程序插件的节点的 Kubernetes 集群。此支持在 Kubernetes 1.34 中可用;Kubernetes v1.24 和 v1.25 将此作为 Beta 功能包含在内,默认启用。
  • 如果你正在配置一个需要服务账号令牌的凭证提供程序插件,则需要一个具有运行 Kubernetes v1.33 或更高版本的节点的 Kubernetes 集群,并且在 kubelet 上启用了 `KubeletServiceAccountTokenForCredentialProviders` 特性门控。
  • 一个可工作的凭证提供程序 exec 插件实现。你可以构建自己的插件或使用云提供商提供的插件。
你的 Kubernetes 服务器版本必须等于或晚于 v1.26。

要检查版本,请输入 kubectl version

在节点上安装插件

凭证提供程序插件是 kubelet 将运行的可执行二进制文件。确保插件二进制文件存在于集群中的每个节点上,并存储在已知目录中。稍后在配置 kubelet 标志时将需要该目录。

配置 kubelet

为了使用此功能,kubelet 需要设置两个标志:

  • `--image-credential-provider-config` - 凭证提供程序插件配置文件的路径。
  • `--image-credential-provider-bin-dir` - 凭证提供程序插件二进制文件所在的目录的路径。

配置 kubelet 凭证提供程序

传递给 `--image-credential-provider-config` 的配置文件由 kubelet 读取,以确定哪些容器镜像应该调用哪些 exec 插件。如果你正在使用 基于 ECR 的插件,这有一个你可能最终会使用的配置文件示例

apiVersion: kubelet.config.k8s.io/v1
kind: CredentialProviderConfig
# providers is a list of credential provider helper plugins that will be enabled by the kubelet.
# Multiple providers may match against a single image, in which case credentials
# from all providers will be returned to the kubelet. If multiple providers are called
# for a single image, the results are combined. If providers return overlapping
# auth keys, the value from the provider earlier in this list is used.
providers:
  # name is the required name of the credential provider. It must match the name of the
  # provider executable as seen by the kubelet. The executable must be in the kubelet's
  # bin directory (set by the --image-credential-provider-bin-dir flag).
  - name: ecr-credential-provider
    # matchImages is a required list of strings used to match against images in order to
    # determine if this provider should be invoked. If one of the strings matches the
    # requested image from the kubelet, the plugin will be invoked and given a chance
    # to provide credentials. Images are expected to contain the registry domain
    # and URL path.
    #
    # Each entry in matchImages is a pattern which can optionally contain a port and a path.
    # Globs can be used in the domain, but not in the port or the path. Globs are supported
    # as subdomains like '*.k8s.io' or 'k8s.*.io', and top-level-domains such as 'k8s.*'.
    # Matching partial subdomains like 'app*.k8s.io' is also supported. Each glob can only match
    # a single subdomain segment, so `*.io` does **not** match `*.k8s.io`.
    #
    # A match exists between an image and a matchImage when all of the below are true:
    # - Both contain the same number of domain parts and each part matches.
    # - The URL path of an matchImages must be a prefix of the target image URL path.
    # - If the matchImages contains a port, then the port must match in the image as well.
    #
    # Example values of matchImages:
    # - 123456789.dkr.ecr.us-east-1.amazonaws.com
    # - *.azurecr.io
    # - gcr.io
    # - *.*.registry.io
    # - registry.io:8080/path
    matchImages:
      - "*.dkr.ecr.*.amazonaws.com"
      - "*.dkr.ecr.*.amazonaws.com.cn"
      - "*.dkr.ecr-fips.*.amazonaws.com"
      - "*.dkr.ecr.us-iso-east-1.c2s.ic.gov"
      - "*.dkr.ecr.us-isob-east-1.sc2s.sgov.gov"
    # defaultCacheDuration is the default duration the plugin will cache credentials in-memory
    # if a cache duration is not provided in the plugin response. This field is required.
    defaultCacheDuration: "12h"
    # Required input version of the exec CredentialProviderRequest. The returned CredentialProviderResponse
    # MUST use the same encoding version as the input. Current supported values are:
    # - credentialprovider.kubelet.k8s.io/v1
    apiVersion: credentialprovider.kubelet.k8s.io/v1
    # Arguments to pass to the command when executing it.
    # +optional
    # args:
    #   - --example-argument
    # Env defines additional environment variables to expose to the process. These
    # are unioned with the host's environment, as well as variables client-go uses
    # to pass argument to the plugin.
    # +optional
    env:
      - name: AWS_PROFILE
        value: example_profile

    # tokenAttributes is the configuration for the service account token that will be passed to the plugin.
    # The credential provider opts in to using service account tokens for image pull by setting this field.
    # if this field is set without the `KubeletServiceAccountTokenForCredentialProviders` feature gate enabled, 
    # kubelet will fail to start with invalid configuration error.
    # +optional
    tokenAttributes:
      # serviceAccountTokenAudience is the intended audience for the projected service account token.
      # +required
      serviceAccountTokenAudience: "<audience for the token>"
      # cacheType indicates the type of cache key use for caching the credentials returned by the plugin
      # when the service account token is used.
      # The most conservative option is to set this to "Token", which means the kubelet will cache
      # returned credentials on a per-token basis. This should be set if the returned credential's
      # lifetime is limited to the service account token's lifetime.
      # If the plugin's credential retrieval logic depends only on the service account and not on
      # pod-specific claims, then the plugin can set this to "ServiceAccount". In this case, the
      # kubelet will cache returned credentials on a per-serviceaccount basis. Use this when the
      # returned credential is valid for all pods using the same service account.
      # +required
      cacheType: "<Token or ServiceAccount>"
      # requireServiceAccount indicates whether the plugin requires the pod to have a service account.
      # If set to true, kubelet will only invoke the plugin if the pod has a service account.
      # If set to false, kubelet will invoke the plugin even if the pod does not have a service account
      # and will not include a token in the CredentialProviderRequest. This is useful for plugins
      # that are used to pull images for pods without service accounts (e.g., static pods).
      # +required
      requireServiceAccount: true
      # requiredServiceAccountAnnotationKeys is the list of annotation keys that the plugin is interested in
      # and that are required to be present in the service account.
      # The keys defined in this list will be extracted from the corresponding service account and passed
      # to the plugin as part of the CredentialProviderRequest. If any of the keys defined in this list
      # are not present in the service account, kubelet will not invoke the plugin and will return an error.
      # This field is optional and may be empty. Plugins may use this field to extract additional information
      # required to fetch credentials or allow workloads to opt in to using service account tokens for image pull.
      # If non-empty, requireServiceAccount must be set to true.
      # The keys defined in this list must be unique and not overlap with the keys defined in the
      # optionalServiceAccountAnnotationKeys list.
      # +optional
      requiredServiceAccountAnnotationKeys:
      - "example.com/required-annotation-key-1"
      - "example.com/required-annotation-key-2"
      # optionalServiceAccountAnnotationKeys is the list of annotation keys that the plugin is interested in
      # and that are optional to be present in the service account.
      # The keys defined in this list will be extracted from the corresponding service account and passed
      # to the plugin as part of the CredentialProviderRequest. The plugin is responsible for validating the
      # existence of annotations and their values. This field is optional and may be empty.
      # Plugins may use this field to extract additional information required to fetch credentials.
      # The keys defined in this list must be unique and not overlap with the keys defined in the
      # requiredServiceAccountAnnotationKeys list.
      # +optional
      optionalServiceAccountAnnotationKeys:
      - "example.com/optional-annotation-key-1"
      - "example.com/optional-annotation-key-2"

`providers` 字段是 kubelet 使用的已启用插件列表。每个条目都有几个必需字段:

  • `name`: 插件的名称,必须与传递给 `--image-credential-provider-bin-dir` 的目录中存在的二进制可执行文件的名称匹配。
  • `matchImages`: 用于匹配镜像以确定是否应调用此提供程序的字符串列表。详见下文。
  • `defaultCacheDuration`: 如果插件未指定缓存持续时间,kubelet 将在内存中缓存凭证的默认持续时间。
  • `apiVersion`: kubelet 和 exec 插件在通信时将使用的 API 版本。

每个凭证提供程序还可以获得可选的参数和环境变量。请咨询插件实现者以确定给定插件所需的参数和环境变量集。

如果你正在使用 `KubeletServiceAccountTokenForCredentialProviders` 特性门控,并通过设置 `tokenAttributes` 字段将插件配置为使用服务账号令牌,则需要以下字段:

  • `serviceAccountTokenAudience`: 预期投射服务账号令牌的受众。这不能是空字符串。
  • `cacheType`: 当使用服务账号令牌时,用于缓存插件返回的凭证的缓存键类型。最保守的选项是将其设置为 `Token`,这意味着 kubelet 将按令牌缓存返回的凭证。如果返回的凭证的生命周期限制为服务账号令牌的生命周期,则应设置此项。如果插件的凭证检索逻辑仅依赖于服务账号而不依赖于 Pod 特定的声明,则插件可以将其设置为 `ServiceAccount`。在这种情况下,kubelet 将按服务账号缓存返回的凭证。当返回的凭证对所有使用相同服务账号的 Pod 都有效时,请使用此项。
  • `requireServiceAccount`: 插件是否要求 Pod 具有服务账号。
    • 如果设置为 `true`,则 kubelet 仅在 Pod 具有服务账号时才调用插件。
    • 如果设置为 `false`,即使 Pod 没有服务账号,kubelet 也会调用插件,并且不会在 `CredentialProviderRequest` 中包含令牌。

这对于用于为没有服务账号的 Pod(例如静态 Pod)拉取镜像的插件很有用。

配置镜像匹配

kubelet 使用每个凭证提供程序的 `matchImages` 字段来确定是否应为 Pod 使用的给定镜像调用插件。`matchImages` 中的每个条目都是一个镜像模式,可以选择包含端口和路径。域中可以使用 glob 模式,但不能在端口或路径中使用。glob 模式支持子域,如 `*.k8s.io` 或 `k8s.*.io`,以及顶级域,如 `k8s.*`。还支持匹配部分子域,如 `app*.k8s.io`。每个 glob 模式只能匹配一个子域段,因此 `*.io` 不匹配 `*.k8s.io`。

当以下所有条件都为真时,镜像名称和 `matchImage` 条目之间存在匹配:

  • 两者包含相同数量的域部分,并且每个部分都匹配。
  • 匹配镜像的 URL 路径必须是目标镜像 URL 路径的前缀。
  • 如果 `matchImages` 包含端口,则镜像中的端口也必须匹配。

`matchImages` 模式的一些示例值是:

  • 123456789.dkr.ecr.us-east-1.amazonaws.com
  • *.azurecr.io
  • gcr.io
  • *.*.registry.io
  • foo.registry.io:8080/path

下一步

上次修改时间:2025 年 6 月 25 日太平洋标准时间上午 9:19:添加 Kubelet 镜像凭证提供程序测试版的 PSAT 文档 (a618b01c1a)