In this blog post, we will look at Skaffold & how it can be used to improve the developer workflow for Kubernetes applications.
Skaffold is a Google container tool, which handles the workflow for building, pushing and deploying the Kubernetes applications.
Skaffold comes as a client-side only single binary cli tool. Its
installation is as simple as downloading the binary & copying it to
/usr/local/bin
. Detailed Skaffold Installation steps.
We will look at how we can use Skaffold to build & deploy Botkube.
We can generate simple skaffold.yaml
configuration file with skaffold
init
command in the project’s root directory.
skaffold init
command will take us through a wizard to generate a
simple skaffold.yaml
file, as below :
$ skaffold init
apiVersion: skaffold/v2alpha3
kind: Config
metadata:
name: botkube
build:
artifacts:
- image: infracloudio/botkube
context: build
deploy:
kubectl:
manifests:
- deploy-all-in-one-tls.yaml
- deploy-all-in-one.yaml
Do you want to write this configuration to skaffold.yaml? [y/n]: y
Configuration skaffold.yaml was written
You can now run [skaffold build] to build the artifacts
or [skaffold run] to build and deploy
or [skaffold dev] to enter development mode, with auto-redeploy
But, we need to make some more changes to the generated skaffold.yaml
file to make the build & deploy work with Botkube’s file structure. The
changes we need to make for build the step work are:
>>>
artifacts:
- image: infracloudio/botkube
context: build
>>>
<<<
artifacts:
- image: infracloudio/botkube
context: .
docker:
dockerfile: build/Dockerfile
<<<
We can set a Docker image tagging policy with below configuration in build section:
<<<
build:
local:
push: false # Set true to a push built images to remote Docker repository
tagPolicy:
gitCommit: {}
<<<
Skaffold supports below tag policies:
git commit
current date & time
environment variables based template
digest of the Docker image
The updated skaffold.yaml
file will look like this:
apiVersion: skaffold/v2alpha3
kind: Config
metadata:
name: botkube
build:
local:
push: false # Setting false will not push the image to remote repository
tagPolicy:
gitCommit: {}
artifacts:
- image: botkubetest
context: .
docker:
dockerfile: build/Dockerfile
deploy:
kubectl:
manifests:
- deploy-all-in-one.yaml
Skaffold workflow pipeline can be run in below modes:
skaffold run
– to build & deploy onceskaffold dev
– to trigger the watch loop build & deploy
workflow with cleanup on exitskaffold debug
– to run a pipeline in debug modeNow, we can run skaffold dev
command to build the Docker image &
deploy to the Kubernetes cluster (uses current Kubernetes context). We
can also add kubeContext
parameter in deploy section of
skaffold.yaml
to explicitly specify Kubernetes context.
NOTE : For Minikube, we need to run eval $(minikube docker-env)
&
set imagePullPolicy
to Never
, so that Minikube can get access to the
locally built Docker images, else we will get ImagePullBackOff
error.
The skaffold dev
output looks like below & Botkube will be deployed to
Minikube:
$ skaffold dev
Listing files to watch...
- botkubetest
Generating tags...
- botkubetest -> botkubetest:v0.10.0-6-gf90361d-dirty
Checking cache...
- botkubetest: Not found. Building
Found [minikube] context, using local docker daemon.
Building [botkubetest]...
Sending build context to Docker daemon 9.087MB
Step 1/17 : FROM golang:1.12-alpine3.10 AS BUILD-ENV
---> 6d6865cf688e
...
...
...
...
Step 17/17 : ENTRYPOINT /go/bin/botkube
---> Using cache
---> a610c9924625
Successfully built a610c9924625
Successfully tagged botkubetest:v0.10.0-6-gf90361d-dirty
Tags used in deployment:
- botkubetest -> botkubetest:a610c9924625c825bc7775ab959921adcf075f83c64474ef1899174d884821e2
local images can't be referenced by digest. They are tagged and referenced by a unique ID instead
Starting deploy...
- namespace/botkube created
- configmap/botkube-configmap created
- secret/botkube-communication-secret created
- serviceaccount/botkube-sa created
- clusterrole.rbac.authorization.k8s.io/botkube-clusterrole created
- clusterrolebinding.rbac.authorization.k8s.io/botkube-clusterrolebinding created
- deployment.apps/botkube created
Watching for changes...
For deploying the applications to a remote Kubernetes cluster, we need to either push locally built images to remote Docker repository & pull image in remote cluster or use in-cluster builders like kaniko.
We can to set push: true
in skaffold.yaml
file in order to push the
built image to remote Docker repository. Skaffold uses local Docker
login credentials to push to the remote Docker repository. Also, we need
to create a custom image pull secret in the cluster & provide it in the
Kubenetes deployment specification to be able to pull the images in the
cluster.
Skaffold currently supports following Builders:
- Dockerfile
- locally with Docker
- in-cluster with Kaniko
- on cloud with Google Cloud Build
- Jib Maven and Gradle
- locally
- on cloud with Google Cloud Build
- Bazel locally
- Cloud Native Buildpacks
- locally with Docker
- on cloud with Google Cloud Build
- Custom script
- locally
- in-cluster
source https://skaffold.dev/docs/
Skaffold currently supports following Deployers:
- Kubernetes Command-Line Interface (kubectl)
- Helm
- Kustomize
In order to use Helm as Deployer, we need to update the skaffold.yaml
file as below:
apiVersion: skaffold/v2alpha3
kind: Config
metadata:
name: botkube
build:
local:
push: false # Setting false will not push the image to remote repository
tagPolicy:
gitCommit: {}
artifacts:
- image: botkubetest
context: .
docker:
dockerfile: build/Dockerfile
deploy:
statusCheckDeadlineSeconds: 600 # Period Skaffold waits for deployment to stabilise
kubeContext: minikube # Context to deploy the application to
helm:
flags:
upgrade:
[--timeout=600] # Helm upgrade timeout
install:
[--timeout=600] # Helm install timeout
releases:
- name: botkube
chartPath: helm/botkube/
skipBuildDependencies: true # Skip Helm dependencies build
namespace: botkube
version: v0.10.0
setValues:
communications.webhook.enabled: true
communications.webhook.url: <WEBHOOK_URL>
config.settings.clustername: <CLUSTER_NAME>
#image.repository: infracloudio/botkube
#image.tag: v0.10.0
imageStrategy:
helm: {}
As per Skaffold docs, when we run skaffold dev
, following steps are
carried out:
1) Collects and watches your source code for changes
2) Syncs files directly to pods if user marks them as syncable
3) Builds artifacts from the source code
4) Tests the built artifacts using container-structure-tests
5) Tags the artifacts
6) Pushes the artifacts
7) Deploys the artifacts
8) Monitors the deployed artifacts
9) Cleans up deployed artifacts on exit (Ctrl+C)
source: https://skaffold.dev/docs/
Skaffold supports multiple profiles to allow deployment to different
environment/clusters. Profiles can be specified in skaffold.yaml
file
as follows:
apiVersion: skaffold/v2alpha3
kind: Config
profiles:
- name: minikube
activation:
- kubeContext: minikube
- env: ENV=local_dev
build:
...
# Build configuration for minikube profile
...
deploy:
...
# Deploy configuration for minikube profile
...
- name: eks-cluster
activation:
- kubeContext: <EKS_CLUSTER_CONTEXT_NAME>
- env: ENV=integration
build:
...
# Build configuration for eks-cluster profile
...
deploy:
...
# Deploy configuration for eks-cluster profile
...
Below command can be used to run Skaffold with different profiles:
skaffold [run|dev] -p <profile_name> -f <skaffold_config_yaml_file>
We can use Skaffold in CI/CD pipeline for building, deploying separately or both as below:
We can just build the project with below command.
skaffold build [-p <profile>] > build_result.json
build_result.json
file mainly contains image tags that are generated,
among other things.
This is useful for deploying pre-built images with Skaffold.
skaffold deploy [-p <profile>] --build-artifacts=build_result.json
build_result.json
can also consist of just the image tags like below.
{
"builds": [
{
"imageName": "REPO_NAME/IMAGE_NAME",
"tag": "REPO_NAME/IMAGE_NAME:IMAGE_TAG"
},
{
"imageName": "REPO_NAME/IMAGE_NAME",
"tag": "REPO_NAME/IMAGE_NAME:IMAGE_TAG"
}
]
}
This will delete all the cluster resources created by Skaffold.
skaffold delete [options]
Log Tailing is enabled by default for dev
and debug
mode & disabled
by default for run
mode. It can be enabled with the --tail
flag.
skaffold run --tail
Port forwarding is disabled by default, can be enabled by using
--port-forward
. Port forwarding is supported only in dev
& debug
modes.
There are two types of port forwarding:
Details about port forwarding.
Skaffold comes as a client-side only single binary cli tool. As there is no on-cluster component, hence no overhead or maintenance work on cluster.
Skaffold has a pluggable architecture, so we have a choice of using suitable tools at each stage in pipeline.
Source: https://skaffold.dev/docs/design/
Skaffold provides built-in support for the following tools, at each stage in pipeline.
Source: https://skaffold.dev/docs/design/
It is very easy to share projects (code & configuration) with just two
steps git clone
and skaffold run
.
Skaffold automatically manages logging and port-forwarding for the
deployments in dev
& debug
mode.
Single skaffold.yaml
file can be used to deploy to multiple
environments using below mechanisms:
Skaffold is very easy to use tool to automate the end-to-end developer workflow for Kubernetes applications. The pipelines created with skaffold are fast & efficient. It has potential to be used as the CI/CD tool as well. It supports multiple tools at each stage of the workflow, It has a lot of features compared to other such tools, making it even better choice.
If you would like to hear more about Improving Kubernetes Developer Workflow with Kind & Skaffold, take a look at this talk from one of my InfraCloud colleagues, Vishal Biyani, from PDC Tech Carnival 2020.
Looking for help with building your DevOps strategy or want to outsource DevOps to the experts? learn why so many startups & enterprises consider us as one of the best DevOps consulting & services companies.
skaffold.yaml
.