Let’s say you are using GitOps to deploy your applications on Kubernetes. Let’s say you want to be sure you wrote your manifests right. Then kubeconform
and kube-score
are the tools you are looking for. And you can of course check it on every push or pull/merge request.
kubeconform
By default, kubeconform
will use vanilla Kubernetes schemas but if you use Custom Resource Definitions, kubeconform
can support them. Use the -schema-location
to indicate where to find the JSON schemas. It accepts templated URLs and it’s handy because there’s a GitHub repository with the most common CRD’s out there: https://github.com/datreeio/CRDs-catalog.
Use openapi2jsonschema.py
to generate JSON schemas from CRD yaml definitions.
Use crd-extractor.sh to generate JSON schemas for all the CRD’s in your cluster. It will save the schemas into ~/.datree/crdSchemas/
.
I suggest you pass the -verbose
, -summary
, -ignore-missing-schemas
and -strict
options to kubeconform
. The first three are self-explanatory. The last one disallows additional properties not in schema or duplicated keys, which helps identifying typos for example:
✖ /data/apps/test-app1/test-app1.yaml: Deployment test-app1 is invalid: problem validating schema. Check JSON formatting: jsonschema: '/spec/template/spec/volumes/0' does not validate with https://raw.githubusercontent.com/yannh/kubernetes-json-schema/master/master-standalone-strict/deployment-apps-v1.json#/properties/spec/properties/template/properties/spec/properties/volumes/items/additionalProperties: additionalProperties 'PersistentVolumeClaim' not allowed
You can validate your entire deployment by piping the output of kustomize build
to kubeconform
, however you will not know which files failed. This is most useful when you have a valid deployment and are doing small incremental changes, for example in CI/CD pipeline or on merge/pull requests.
kustomize build clusters/mycluster/flux-system/ | docker run --rm -i ghcr.io/yannh/kubeconform:v0.6.7-alpine -summary -verbose -strict -ignore-missing-schemas
If you want to know which files are problematic, then point kubeconform
to a directory. However, do note that it will analyze all the files in the directory, including the ones that would not be picked by kustomize
.
docker run --rm -i -v .:/data:ro ghcr.io/yannh/kubeconform:v0.6.7-alpine -summary -verbose -strict -ignore-missing-schemas -output pretty /data
kube-score
Use kube-score
to check your manifests for best practices, such as specifying limits or Image Pull Policy. Make sure to mention your Kubernetes version as the default is v1.18.
Use it the same way as kubeconform:
kustomize build clusters/mycluster/flux-system/ | docker run --rm -i zegl/kube-score:v1.19.0 score --kubernetes-version 1.32 --color always -
GitHub/Gitea actions
To validate your manifests using GitHub/Gitea actions, use the following yaml.
name: Validate Kubernetes Manifests
on:
push:
pull_request:
branches: [master]
jobs:
kubeconform:
runs-on: linux_amd64
steps:
- uses: actions/checkout@v4.2.2
- uses: yokawasa/action-setup-kube-tools@v0.11.2
with:
setup-tools: |
kustomize
kubeconform
kube-score
kustomize: '5.6.0'
kubeconform: '0.6.7'
kube-score: '1.19.0'
- continue-on-error: true
run: |
kustomize build clusters/mycluster/flux-system/ | docker run --rm -i ghcr.io/yannh/kubeconform:v0.6.7-alpine -verbose -strict -summary -schema-location default -schema-location 'https://raw.githubusercontent.com/datree/CRDs-catalog/refs/heads/main/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json'
- continue-on-error: true
run: |
kustomize build clusters/mycluster/flux-system/ | docker run --rm -i zegl/kube-score:v1.19.0 score --kubernetes-version 1.32 -
The output of kube-score typically looks like this at first:
apps/v1/Deployment adguard-home in adguard-home 💥
path=STDIN
[CRITICAL] Container Security Context ReadOnlyRootFilesystem
· adguardhome -> Container has no configured security context
Set securityContext to run the container in a more secure context.
[CRITICAL] Container Resources
· adguardhome -> CPU limit is not set
Resource limits are recommended to avoid resource DDOS. Set
resources.limits.cpu
· adguardhome -> Memory limit is not set
Resource limits are recommended to avoid resource DDOS. Set
resources.limits.memory
· adguardhome -> CPU request is not set
Resource requests are recommended to make sure that the application
can start and run without crashing. Set resources.requests.cpu
· adguardhome -> Memory request is not set
Resource requests are recommended to make sure that the application
can start and run without crashing. Set resources.requests.memory
[CRITICAL] Container Image Pull Policy
· adguardhome -> ImagePullPolicy is not set to Always
It's recommended to always set the ImagePullPolicy to Always, to
make sure that the imagePullSecrets are always correct, and to
always get the image you want.
[CRITICAL] Container Ephemeral Storage Request and Limit
· adguardhome -> Ephemeral Storage limit is not set
Resource limits are recommended to avoid resource DDOS. Set
resources.limits.ephemeral-storage
· adguardhome -> Ephemeral Storage request is not set
Resource requests are recommended to make sure the application can
start and run without crashing. Set
resource.requests.ephemeral-storage
[CRITICAL] Pod Probes
· Container is missing a readinessProbe
A readinessProbe should be used to indicate when the service is
ready to receive traffic. Without it, the Pod is risking to receive
traffic before it has booted. It's also used during rollouts, and
can prevent downtime if a new version of the application is failing.
More information: https://github.com/zegl/kube-score/blob/master/README_PROBES.md
[CRITICAL] Pod NetworkPolicy
· The pod does not have a matching NetworkPolicy
Create a NetworkPolicy that targets this pod to control who/what
can communicate with this pod. Note, this feature needs to be
supported by the CNI implementation used in the Kubernetes cluster
to have an effect.
[CRITICAL] Container Security Context User Group ID
· adguardhome -> Container has no configured security context
Set securityContext to run the container in a more secure context.
[WARNING] Deployment Replicas
· Deployment few replicas
Deployments targeted by Services are recommended to have at least 2
replicas to prevent unwanted downtime.