Authentication practices with Tekton

The upstream official documentation around authentication is available here. It mainly focus on authenticating through annotated secrets and service accounts. This document will try to explore the different way to use "secrets" for authentication in your Pipeline and Task.

There is at least three ways to use secrets to authenticate to Github or Docker or.

  • Using secrets and workspaces highly recommended

  • Using a ServiceAccount recommended

  • Using VolumeMount not recommended

Using Secret(s) and Workspace(s)

This is the most flexible approach, especially if you use the optional workspace feature. The general idea behind this approach is: - We use a workspace to bind a secret - In the Task definition, we tell the command line (through a flag or an environment variable) where to find the secret content (aka the authentication content).

We’ll take the example with a very simple git-clone Task, as well as a very simple docker build, but it can apply to anything else.

A git clone example

For this example, we assume we are using git with ssh to authenticate. Later in the document, we will show different approaches.

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: git-clone
spec:
  workspaces:
    - name: output
      description: The git repo will be cloned onto the volume backing this Workspace.
    - name: ssh-directory # (1)
      description: |
        A .ssh directory with private key, known_hosts, config, etc. Copied to
        the user's home before git commands are executed. Used to authenticate
        with the git remote when performing the clone. Binding a Secret to this
        Workspace is strongly recommended over other volume types
  params:
    - name: url
      description: Repository URL to clone from.
      type: string
    - name: revision
      description: Revision to checkout. (branch, tag, sha, ref, etc...)
      type: string
      default: ""
    - name: gitInitImage
      description: The image providing the git-init binary that this Task runs.
      type: string
      default: "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init:v0.37.0"
  results:
    - name: commit
      description: The precise commit SHA that was fetched by this Task.
    - name: url
      description: The precise URL that was fetched by this Task.
  steps:
    - name: clone
      image: "$(params.gitInitImage)"
      script: |
        #!/usr/bin/env sh
        set -eu
        # This is necessary for recent version of git
        git config --global --add safe.directory '*'
        # (2)
        cp -R "$(workspaces.ssh-directory.path)" "${HOME}"/.ssh
        chmod 700 "${HOME}"/.ssh
        chmod -R 400 "${HOME}"/.ssh/*
        CHECKOUT_DIR="$(workspaces.output.path)/"
        /ko-app/git-init \
          -url="$(params.url)" \
          -revision="$(params.revision)" \
          -path="${CHECKOUT_DIR}"
        cd "${CHECKOUT_DIR}"
        RESULT_SHA="$(git rev-parse HEAD)"
        EXIT_CODE="$?"
        if [ "${EXIT_CODE}" != 0 ] ; then
          exit "${EXIT_CODE}"
        fi
        printf "%s" "${RESULT_SHA}" > "$(results.commit.path)"
        printf "%s" "$(params.url)" > "$(results.url.path)"

  1. Defines the workspace ; here the name is ssh-directory, and the description explain how should the secret be created. An example of an ssh secret with a ed25519 private key.

    apiVersion: v1
kind: Secret
metadata:
  name: my-github-ssh-credentials
type: Opaque
data:
  id_ed25519: # […]
  known_hosts: # […]
  # config: # […] # optional

    This can be created with the following command-line:

    $ kubectl create secret generic my-github-ssh-credentials \
  --from-file=id_ed25519=/path/to/.ssh/id_ed25519 \
  --from-file=known_hosts=/path/to/.ssh/known_hosts
secret/my-github-ssh-credentials created

  2. In the script, we are copying the content of the secret (as a folder) to ${HOME}/.ssh which is the standard folder where ssh (and thus git) will look for credentials.

$ tkn task start git-clone \
      --param url=git@github.com:vdemeester/buildkit-tekton \
      --workspace name=output,emptyDir="" \
      --workspace name=ssh-directory,secret=my-github-ssh-credentials \
      --use-param-defaults --showlog
TaskRun started: git-clone-run-kt7fv
Waiting for logs to be available...
[clone] {"level":"warn","ts":1657102809.7319033,"caller":"git/git.go:273","msg":"URL(\"git@github.com:vdemeester/buildkit-tekton\") appears to need SSH authentication but no SSH credentials have been provided"}
[clone] {"level":"info","ts":1657102813.1506214,"caller":"git/git.go:178","msg":"Successfully cloned git@github.com:vdemeester/buildkit-tekton @ e6afd054a907ee447a040c6e95f23fabe038ce6d (grafted, HEAD) in path /workspace/output/"}
[clone] {"level":"info","ts":1657102813.1600826,"caller":"git/git.go:217","msg":"Successfully initialized and updated submodules in path /workspace/output/"}

Note that the warning URL(\"git@github.com:vdemeester/buildkit-tekton\") appears to need SSH authentication but no SSH credentials have been provided appears because the entrypoint didn’t see any credentials coming from the annotated secrets attached to a ServiceAccount (see Using ServiceAccount(s)). It can safely be ignored in that case.

A docker configuration

For this example, we will be using an existing docker configuration file to be used inside a Task, similar to the A git clone example example. The Task definition is slightly similar to the A git clone example one. Here we will use skopeo to copy an image to our own repository — it could be applied to other tools (podman, buildah, docker, …), basically any tool that is capable of reading a docker client configuration file.

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: skopeo-copy
spec:
  workspaces:
    - name: dockerconfig # (1)
      description: Includes a docker `config.json`
  steps:
    - name: clone
      image: quay.io/skopeo/stable:v1.8.0
      env:
      - name: DOCKER_CONFIG
        value: $(workspaces.dockerconfig.path) # (2)
      script: |
        #!/usr/bin/env sh
        set -eu
        skopeo copy docker://docker.io/library/ubuntu:latest docker://docker.io/vdemeester/ubuntu-copy:latest

  1. Similar to A git clone example, we define a workspace that should contain a config.json file. For a secret, this means a key named config.json. An example of an ssh secret with a ed25519 private key.

    apiVersion: v1
kind: Secret
metadata:
  name: regcred
type: Opaque
data:
  config.json: # […]

    This can be created with the following command-line:

    $ kubectl create secret generic regcred \
  --from-file=config.json=/path/to/.docker/config.json
secret/regcred created

  2. Here we are just setting the DOCKER_CONFIG environment variable to point to the dockerconfig workspace path. skopeo (as a lot of docker-ish client) do read this environment variable to get the docker client configuration information, and in our case, the authentication informations.

$ tkn task start skopeo-copy --workspace name=dockerconfig,secret=regcred --showlog
TaskRun started: skopeo-copy-run-cfg7l
Waiting for logs to be available...
[clone] DOCKER_CONFIG=/workspace/dockerconfig
Getting image source signatures
[clone] Copying blob sha256:405f018f9d1d0f351c196b841a7c7f226fb8ea448acd6339a9ed8741600275a2
[clone] Copying config sha256:27941809078cc9b2802deb2b0bb6feed6c236cde01e487f200e24653533701ee
[clone] Writing manifest to image destination
[clone] Storing signatures

With optional workspaces

This approach is extremely similar to the one above but using the optional feature of workspaces. In a gist, this means a Workspace might be bound to it (a.k.a. mounted as a volume in the Pod/Container) ; but it could also not be present.

The only difference here will be, to make sure our Task takes this optional property into account. The git-clone upstream Task shows an example of that. Let’s adapt the A git clone example example with optional Workspace support.

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: git-clone
spec:
  workspaces:
    - name: output
      description: The git repo will be cloned onto the volume backing this Workspace.
    - name: ssh-directory # (1)
      description: |
        A .ssh directory with private key, known_hosts, config, etc. Copied to
        the user's home before git commands are executed. Used to authenticate
        with the git remote when performing the clone. Binding a Secret to this
        Workspace is strongly recommended over other volume types
  params:
    - name: url
      description: Repository URL to clone from.
      type: string
    - name: revision
      description: Revision to checkout. (branch, tag, sha, ref, etc...)
      type: string
      default: ""
    - name: gitInitImage
      description: The image providing the git-init binary that this Task runs.
      type: string
      default: "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init:v0.37.0"
  results:
    - name: commit
      description: The precise commit SHA that was fetched by this Task.
    - name: url
      description: The precise URL that was fetched by this Task.
  steps:
    - name: clone
      image: "$(params.gitInitImage)"
      script: |
        #!/usr/bin/env sh
        set -eu
        # This is necessary for recent version of git
        git config --global --add safe.directory '*'
        # (2)
        if [ "$(workspaces.ssh-directory.bound)" = "true" ] ; then
          cp -R "$(workspaces.ssh-directory.path)" "${HOME}"/.ssh
          chmod 700 "${HOME}"/.ssh
          chmod -R 400 "${HOME}"/.ssh/*
        fi
        CHECKOUT_DIR="$(workspaces.output.path)/"
        /ko-app/git-init \
          -url="$(params.url)" \
          -revision="$(params.revision)" \
          -path="${CHECKOUT_DIR}"
        cd "${CHECKOUT_DIR}"
        RESULT_SHA="$(git rev-parse HEAD)"
        EXIT_CODE="$?"
        if [ "${EXIT_CODE}" != 0 ] ; then
          exit "${EXIT_CODE}"
        fi
        printf "%s" "${RESULT_SHA}" > "$(results.commit.path)"
        printf "%s" "$(params.url)" > "$(results.url.path)"

  1. Defines the workspace ; here the name is ssh-directory, and the description explain how should the secret be created. The example in A git clone example apply here as well.

  2. In the script, we are conditionally copying the content of the secret (as a folder) to ${HOME}/.ssh which is the standard folder where ssh (and thus git) will look for credentials. If the Workspace is bound we copy, if it’s not bound, we don’t do anything.

The main advantage of using optionnal Workspace is that it makes your Task a bit more flexible. For example, with git-clone, if you are going to clone a publicly available git repository, you won’t bind a Secret to the Workspace.

With git-credentials

We can apply both the approach with and without the optional Workspace with other means of authenticating than ssh keys for git-clone, for example. Git has a notion of git-credential that could be used here. Using optional workspace as above, it could look like the following.

        if [ "$(workspaces.basic_auth.bound)" = "true" ] ; then
          cp "$(workspaces.basic_auth.path)/.git-credentials" "${HOME}/.git-credentials"
          cp "$(workspaces.basic_auth.path)/.gitconfig" "${HOME}/.gitconfig"
          chmod 400 "${HOME}/.git-credentials"
          chmod 400 "${HOME}/.gitconfig"
        fi

Using ServiceAccount(s)

The ServiceAccount approach is the most widely documentated, especially upstream. The goal here is not to paraphrase the upstream documentation, so we’ll just describe what the flow is.

This methods constits, in a gist, of the following: - Secret(s) annotated specifcally, see Annotations - ServiceAccount(s) that are linked to those annotated Secret(s), see Link Secret(s) to ServiceAccount(s) - PipelineRun(s) or TaskRun(s) using this/those ServiceAccount(s), see Specify SerivceAccount(s) on PipelineRun/TaskRun(s)

Annotations

Tetkon support two different type of authentication for secrets : git and docker. The annotation "reflect" this :

  • tekton.dev/git-{} mark the annotated secret as a git secret.

  • tekton.dev/docker-{} mark the annotated secret as a docker secret.

In addition, Tekton supports different types of secret per type of authentication. See the Types of Secrets kubernetes documentation for more details on those secrets.

  • For git secrets:

  • kubernetes.io/basic-auth : basic authentication

  • kubernetes.io/ssh-auth : ssh based authentication (keys, …)

  • For docker secrets:

  • kubernetes.io/basic-auth : basic authentications

  • kubernetes.io/dockercfg : serialized ~/.dockercfg file

  • kubernetes.io/dockerconfigjson : serialized ~/.docker/config.json file

We’ll just show a set of example of annotated secrets with a quick description of them.

Almost all the example below are using stringData, which are direct content of the secret (visible for everybody that can get Secret). They can also all work with data, which is the base64 encoded version of the data, that gives a little bit more obfuscation to the Secrets.

  • basic authentication for git : for gitlab.com as well as github.com. It seems not very secure to use the same user/password for authenticating different git provider, but that’s just to show that a Secret can be used to target multiple host.

    apiVersion: v1
kind: Secret
metadata:
  annotations:
    tekton.dev/git-0: https://github.com
    tekton.dev/git-1: https://gitlab.com
type: kubernetes.io/basic-auth
stringData:
  username: <cleartext username>
  password: <cleartext password>

  • ssh authentication for git:

    apiVersion: v1
kind: Secret
metadata:
  annotations:
    tekton.dev/git-0: github.com
type: kubernetes.io/ssh-auth
stringData:
  ssh-privatekey: <private-key>
  # This is non-standard, but its use is encouraged to make this more secure.
  # Omitting this results in the server's public key being blindly accepted.

  • basic authentication for docker

    apiVersion: v1
kind: Secret
metadata:
  annotations:
    tekton.dev/docker-0: https://gcr.io
type: kubernetes.io/basic-auth
stringData:
  username: <cleartext username>
  password: <cleartext password>

  • kubernetes.io/dockerconfigjson authentication for docker. This type of secret (with kubernetes.io/dockercfg) do not need to be annotated as it can contain authentication for multiple hosts.

    apiVersion: v1
kind: Secret
metadata:
  name: regcred
type: kubernetes.io/dockerconfigjson
data:
  .dockerconfigjson: # base64 encoded content of a docker `config.json` file

    It can be created easily with the following command-line

    $ kubectl create secret generic regcred \
          --from-file=.dockerconfigjson=<path/to/.docker/config.json> \
          --type=kubernetes.io/dockerconfigjson`.

To link a Secret to a ServiceAccount, you need to reference the Secret, by name, in the secrets field of the ServiceAccount.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: build-bot
secrets:
  - name: regcred
  - name: a-git-auth-secret

Specify SerivceAccount(s) on PipelineRun/TaskRun(s)

To add a ServiceAccount to a PipelineRun or a TaskRun you can use the serviceAccountName field.

# For a TaskRun
apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
  name: build-with-basic-auth
spec:
  serviceAccountName: build-bot
  taskRef:
    name: demo-task
  # ...
---
# For a PipelineRun
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  name: demo-pipeline
  namespace: default
spec:
  serviceAccountName: build-bot
  pipelineRef:
    name: demo-pipeline
  # […]

It is also possible to use tkn to achieve the same.

---
# For a TaskRun
$ tkn task start demo-task --serviceaccount build-bot
# For a PipelineRun
$ tkn pipeline start demo-pipeline --serviceaccount build-bot --param # […]
---

Using VolumeMount(s)

Because this solution is highly discouraged, mainly due to the fact that Volume and VolumeMount might go away in the API, see Remove "volumes" from Task before V1, we’ll only scratch the surface on how it would work.

The idea of this solution is very similar to Using Secret(s) and Workspace(s) but using Volume and VolumeMount instead. You wount a Secret in a Volume using VolumeMount in the Task, and you act based on the path it is mounted on to copy the secrets to the right place.