Using CEL Expressions in Kyverno Policies
Kyverno, in simple terms, is a policy engine for Kubernetes that can be used to describe policies and validate resource requests against those policies. It allows us to create policies for our Kubernetes cluster on different levels. It enables us to validate, change, and create resources based on our defined policies.
A Kyverno policy is a collection of rules. Whenever we receive an API request to our Kubernetes cluster, we validate it with a set of rules.
A policy consists of different clauses, such as:
- Match: It selects the resources to be included in a rule.
- Exclude: It selects a subset of the resources from the match block which should be excluded from a rule.
Match and Exclude are used to select resources, users, user groups, service accounts, namespaced roles, and cluster-wide roles.
- Validate: It validates the properties of the new resource, and it is created if it matches what is declared in the rule.
- Mutate: It modifies matching resources.
- Generate: It creates additional resources.
- Verify Images: It verifies container image signatures using Cosign and Notary.
Refer to Selecting Resources for more information.
Each rule can contain only a single validate, mutate, generate, or verifyImages child declaration.
In this post, I will show you how to write CEL expressions in Kyverno policies for resource validation. Common Expression Language (CEL) was first introduced to Kubernetes for the validation rules for CustomResourceDefinitions, and then it was used by Kubernetes ValidatingAdmissionPolicies in 1.26.
CEL Expressions in validate rules
Creating a policy to disallow host paths for Deployments
The below policy ensures no hostPath volumes are in use for Deployments.
1kubectl apply -f - <<EOF
2apiVersion: kyverno.io/v1
3kind: ClusterPolicy
4metadata:
5 name: disallow-host-path
6spec:
7 validationFailureAction: Enforce
8 background: false
9 rules:
10 - name: host-path
11 match:
12 any:
13 - resources:
14 kinds:
15 - Deployment
16 validate:
17 cel:
18 expressions:
19 - expression: "!has(object.spec.template.spec.volumes) || object.spec.template.spec.volumes.all(volume, !has(volume.hostPath))"
20 message: "HostPath volumes are forbidden. The field spec.template.spec.volumes[*].hostPath must be unset."
21EOF
spec.rules.validate.cel contains CEL expressions that use the Common Expression Language (CEL) to validate the request. If an expression evaluates to false, the validation check is enforced according to the spec.validationFailureAction field.
Now, let’s try deploying an app that uses a hostPath:
1kubectl apply -f - <<EOF
2apiVersion: apps/v1
3kind: Deployment
4metadata:
5 name: nginx
6spec:
7 replicas: 2
8 selector:
9 matchLabels:
10 app: nginx
11 template:
12 metadata:
13 labels:
14 app: nginx
15 spec:
16 containers:
17 - name: nginx-server
18 image: nginx
19 volumeMounts:
20 - name: udev
21 mountPath: /data
22 volumes:
23 - name: udev
24 hostPath:
25 path: /etc/udev
26EOF
We can see that our policy is enforced. Great!
Error from server: error when creating "STDIN": admission webhook "validate.kyverno.svc-fail" denied the request:
resource Deployment/default/nginx was blocked due to the following policies
disallow-host-path:
host-path: HostPath volumes are forbidden. The field spec.template.spec.volumes[*].hostPath
must be unset.
Creating a policy to check StatefulSet Namespaces
The below policy ensures that any StatefulSet is created in the production Namespace
1kubectl apply -f - <<EOF
2apiVersion: kyverno.io/v1
3kind: ClusterPolicy
4metadata:
5 name: check-statefulset-namespace
6spec:
7 validationFailureAction: Enforce
8 background: false
9 rules:
10 - name: statefulset-namespace
11 match:
12 any:
13 - resources:
14 kinds:
15 - StatefulSet
16 validate:
17 cel:
18 expressions:
19 - expression: "namespaceObject.metadata.name == 'production'"
20 message: "The StatefulSet must be created in the 'production' namespace."
21EOF
Let’s try creating a StatefulSet in the default Namespace.
1kubectl apply -f - <<EOF
2apiVersion: apps/v1
3kind: StatefulSet
4metadata:
5 name: bad-statefulset
6spec:
7 replicas: 1
8 selector:
9 matchLabels:
10 app: app
11 template:
12 metadata:
13 labels:
14 app: app
15 spec:
16 containers:
17 - name: container2
18 image: nginx
19EOF
As expected, the Statefulset creation is blocked because it violates the rule
Error from server: error when creating "STDIN": admission webhook "validate.kyverno.svc-fail" denied the request:
resource StatefulSet/default/bad-statefulset was blocked due to the following policies
check-statefulset-namespace:
statefulset-namespace: The StatefulSet must be created in the 'production' namespace.
Let’s create a Statefulset in the production Namespace.
1kubectl apply -f - << EOF
2apiVersion: apps/v1
3kind: StatefulSet
4metadata:
5 name: good-statefulset
6 namespace: production
7spec:
8 replicas: 1
9 selector:
10 matchLabels:
11 app: app
12 template:
13 metadata:
14 labels:
15 app: app
16 spec:
17 containers:
18 - name: container2
19 image: nginx
20EOF
The StatefulSet is successfully created. Great!
statefulset.apps/good-statefulset created
In the previous two examples, we have used object in CEL expressions which refers to the incoming object and namespaceObject which refers to the Namespace that the incoming object belongs to.
Some other useful variables that we can use in CEL expressions are
- oldObject: The existing object. The value is null for CREATE requests.
- authorizer: It can be used to perform authorization checks.
- authorizer.requestResource: A shortcut for an authorization check configured with the request resource (group, resource, (subresource), namespace, name).
CEL Preconditions in Kyverno Policies
The below policy ensures the hostPort field is set to a value between 5000 and 6000 for pods whose metadata.name set to nginx
1kubectl apply -f - <<EOF
2apiVersion: kyverno.io/v1
3kind: ClusterPolicy
4metadata:
5 name: disallow-host-port-range
6spec:
7 validationFailureAction: Enforce
8 background: false
9 rules:
10 - name: host-port-range
11 match:
12 any:
13 - resources:
14 kinds:
15 - Pod
16 celPreconditions:
17 - name: "first match condition in CEL"
18 expression: "object.metadata.name.matches('nginx')"
19 validate:
20 cel:
21 expressions:
22 - expression: "object.spec.containers.all(container, !has(container.ports) || container.ports.all(port, !has(port.hostPort) || (port.hostPort >= 5000 && port.hostPort <= 6000)))"
23 message: "The only permitted hostPorts are in the range 5000-6000."
24EOF
spec.rules.celPreconditions are CEL expressions. All celPreconditions must be evaluated to true for the resource to be evaluated. Therefore, any Pod with nginx in its metadata.name will be evaluated.
Let’s try deploying an Apache server with hostPort set to 80.
1kubectl apply -f - <<EOF
2apiVersion: v1
3kind: Pod
4metadata:
5 name: apache
6spec:
7 containers:
8 - name: apache-server
9 image: httpd
10 ports:
11 - containerPort: 8080
12 hostPort: 80
13EOF
You’ll see that it’s successfully created because the validation rule wasn’t applied on the new Pod as it doesn’t satisfy the celPreconditions. That’s exactly what we need.
Pod/apache created
Let’s try deploying an Nginx server with hostPort set to 80.
1kubectl apply -f - <<EOF
2apiVersion: v1
3kind: Pod
4metadata:
5 name: nginx
6spec:
7 containers:
8 - name: nginx-server
9 image: nginx
10 ports:
11 - containerPort: 8080
12 hostPort: 80
13EOF
Since the new Pod satisfies the celPreconditions, the validation rule will be applied. As a result, the creation of the Pod will be blocked as it violates the rule.
Error from server: error when creating "STDIN": admission webhook "validate.kyverno.svc-fail" denied the request:
resource Pod/default/nginx was blocked due to the following policies
disallow-host-port-range:
host-port-range: The only permitted hostPorts are in the range 5000-6000.
Parameter Resources in Kyverno Policies
The below policy ensures the deployment replicas are less than a specific value. This value is defined in a parameter resource.
1kubectl apply -f - <<EOF
2apiVersion: kyverno.io/v1
3kind: ClusterPolicy
4metadata:
5 name: check-deployment-replicas
6spec:
7 validationFailureAction: Enforce
8 background: false
9 rules:
10 - name: deployment-replicas
11 match:
12 any:
13 - resources:
14 kinds:
15 - Deployment
16 validate:
17 cel:
18 paramKind:
19 apiVersion: rules.example.com/v1
20 kind: ReplicaLimit
21 paramRef:
22 name: "replica-limit-test.example.com"
23 parameterNotFoundAction: "Deny"
24 expressions:
25 - expression: "object.spec.replicas <= params.maxReplicas"
26 messageExpression: "'Deployment spec.replicas must be less than ' + string(params.maxReplicas)"
27EOF
The cel.paramKind and cel.paramRef specify the resource used to parameterize this policy. For this example, it is configured by ReplicaLimit custom resources.
The ReplicaLimit could be as follows:
1kubectl apply -f - <<EOF
2apiVersion: rules.example.com/v1
3kind: ReplicaLimit
4metadata:
5 name: "replica-limit-test.example.com"
6maxReplicas: 3
7EOF
Here’s the corresponding custom resource definition:
1kubectl apply -f - <<EOF
2apiVersion: apiextensions.k8s.io/v1
3kind: CustomResourceDefinition
4metadata:
5 name: replicalimits.rules.example.com
6spec:
7 group: rules.example.com
8 names:
9 kind: ReplicaLimit
10 plural: replicalimits
11 scope: Namespaced
12 versions:
13 - name: v1
14 served: true
15 storage: true
16 schema:
17 openAPIV3Schema:
18 type: object
19 properties:
20 apiVersion:
21 type: string
22 kind:
23 type: string
24 metadata:
25 type: object
26 maxReplicas:
27 type: integer
28EOF
Now, let’s try deploying an app with five replicas.
1kubectl apply -f - <<EOF
2apiVersion: apps/v1
3kind: Deployment
4metadata:
5 name: nginx
6spec:
7 replicas: 5
8 selector:
9 matchLabels:
10 app: nginx
11 template:
12 metadata:
13 labels:
14 app: nginx
15 spec:
16 containers:
17 - name: nginx-server
18 image: nginx
19EOF
As expected, the deployment creation will be blocked because it violates the rule.
Error from server: error when creating "STDIN": admission webhook "validate.kyverno.svc-fail" denied the request:
resource Deployment/default/nginx was blocked due to the following policies
check-deployment-replicas:
deployment-replicas: Deployment spec.replicas must be less than 3
Let’s try deploying an app with two replicas.
1kubectl apply -f - <<EOF
2apiVersion: apps/v1
3kind: Deployment
4metadata:
5 name: nginx
6spec:
7 replicas: 2
8 selector:
9 matchLabels:
10 app: nginx
11 template:
12 metadata:
13 labels:
14 app: nginx
15 spec:
16 containers:
17 - name: nginx-server
18 image: nginx
19EOF
The deployment is created successfully. Great!
deployment.apps/nginx created
CEL Variables in Kyverno Policies
If an expression grows too complicated, or part of the expression is reusable and computationally expensive to evaluate, We can extract some parts of the expressions into variables. A variable is a named expression that can be referred later as variables in other expressions.
The order of variables is important because a variable can refer to other variables defined before it. This ordering prevents circular references.
The below policy enforces that image repo names match the environment defined in its Namespace. It enforces that all containers of deployment have the image repo match the environment label of its Namespace except for “exempt” deployments or any containers that do not belong to the “example.com” organization (e.g., common sidecars). For example, if the Namespace has a label of {“environment”: “staging”}, all container images must be either staging.example.com/* or do not contain “example.com” at all, unless the deployment has {“exempt”: “true”} label.
1kubectl apply -f - <<EOF
2apiVersion: kyverno.io/v1
3kind: ClusterPolicy
4metadata:
5 name: image-matches-namespace-environment.policy.example.com
6spec:
7 validationFailureAction: Enforce
8 background: false
9 rules:
10 - name: image-matches-namespace-environment
11 match:
12 any:
13 - resources:
14 kinds:
15 - Deployment
16 validate:
17 cel:
18 variables:
19 - name: environment
20 expression: "'environment' in namespaceObject.metadata.labels ? namespaceObject.metadata.labels['environment'] : 'prod'"
21 - name: exempt
22 expression: "has(object.metadata.labels) && 'exempt' in object.metadata.labels && object.metadata.labels['exempt'] == 'true'"
23 - name: containers
24 expression: "object.spec.template.spec.containers"
25 - name: containersToCheck
26 expression: "variables.containers.filter(c, c.image.contains('example.com/'))"
27 expressions:
28 - expression: "variables.exempt || variables.containersToCheck.all(c, c.image.startsWith(variables.environment + '.'))"
29 messageExpression: "'only ' + variables.environment + ' images are allowed in namespace ' + namespaceObject.metadata.name"
30EOF
Let’s start with creating a Namespace that has a label of environment: staging
1kubectl apply -f - <<EOF
2apiVersion: v1
3kind: Namespace
4metadata:
5 name: staging-ns
6 labels:
7 environment: staging
8EOF
And then create a deployment whose image is example.com/nginx in the staging-ns Namespace.
1kubectl apply -f - <<EOF
2apiVersion: apps/v1
3kind: Deployment
4metadata:
5 name: deployment-fail
6 namespace: staging-ns
7spec:
8 replicas: 1
9 selector:
10 matchLabels:
11 app: app
12 template:
13 metadata:
14 labels:
15 app: app
16 spec:
17 containers:
18 - name: container2
19 image: example.com/nginx
20EOF
As expected, the deployment creation will be blocked since its image must be staging.example.com/nginx
Let’s try setting the deployment image to staging.example.com/nginx instead
1kubectl apply -f - <<EOF
2apiVersion: apps/v1
3kind: Deployment
4metadata:
5 name: deployment-pass
6 namespace: staging-ns
7spec:
8 replicas: 1
9 selector:
10 matchLabels:
11 app: app
12 template:
13 metadata:
14 labels:
15 app: app
16 spec:
17 containers:
18 - name: container2
19 image: staging.example.com/nginx
20EOF
The deployment is created successfully. Great!
deployment.apps/deployment-pass created
Auto-Gen Rules for CEL Expressions
Since Kubernetes has many higher-level controllers that directly or indirectly manage Pods: Deployment, DaemonSet, StatefulSet, Job, and CronJob resources, it’d be inefficient to write a policy that targets Pods and every higher-level controller. Kyverno solves this issue by supporting the automatic generation of policy rules for higher-level controllers from a rule written exclusively for a Pod.
Check the autogen rules for more information.
For example, when creating a validation policy like below, which disallows latest image tags, the policy applies to all resources capable of generating Pods.
1kubectl apply -f - <<EOF
2apiVersion: kyverno.io/v1
3kind: ClusterPolicy
4metadata:
5 name: disallow-latest-tag
6spec:
7 validationFailureAction: Enforce
8 rules:
9 - name: disallow-latest-tag
10 match:
11 any:
12 - resources:
13 kinds:
14 - Pod
15 validate:
16 cel:
17 expressions:
18 - expression: "object.spec.containers.all(container, !container.image.contains('latest'))"
19 message: "Using a mutable image tag e.g. 'latest' is not allowed."
20EOF
Once the policy is created, these other resources can be shown in auto-generated rules which Kyverno adds to the policy under the status object.
1status:
2 autogen:
3 rules:
4 - exclude:
5 resources: {}
6 generate:
7 clone: {}
8 cloneList: {}
9 match:
10 any:
11 - resources:
12 kinds:
13 - DaemonSet
14 - Deployment
15 - Job
16 - StatefulSet
17 - ReplicaSet
18 - ReplicationController
19 resources: {}
20 mutate: {}
21 name: autogen-disallow-latest-tag
22 validate:
23 cel:
24 expressions:
25 - expression: object.spec.template.spec.containers.all(container, !container.image.contains('latest'))
26 message: Using a mutable image tag e.g. 'latest' is not allowed.
27 - exclude:
28 resources: {}
29 generate:
30 clone: {}
31 cloneList: {}
32 match:
33 any:
34 - resources:
35 kinds:
36 - CronJob
37 resources: {}
38 mutate: {}
39 name: autogen-cronjob-disallow-latest-tag
40 validate:
41 cel:
42 expressions:
43 - expression: object.spec.jobTemplate.spec.template.spec.containers.all(container,
44 !container.image.contains('latest'))
45 message: Using a mutable image tag e.g. 'latest' is not allowed.
Let’s try creating an nginx deployment with the latest tag.
1kubectl apply -f - << EOF
2apiVersion: apps/v1
3kind: Deployment
4metadata:
5 name: nginx-deployment
6 labels:
7 app: nginx
8spec:
9 replicas: 3
10 selector:
11 matchLabels:
12 app: nginx
13 template:
14 metadata:
15 labels:
16 app: nginx
17 spec:
18 containers:
19 - name: nginx
20 image: nginx:latest
21EOF
As expected the deployment creation is blocked.
Error from server: error when creating "STDIN": admission webhook "validate.kyverno.svc-fail" denied the request:
resource Deployment/default/nginx-deployment was blocked due to the following policies
disallow-latest-tag:
autogen-disallow-latest-tag: Using a mutable image tag e.g. 'latest' is not allowed.
Conclusion
This blog post explains how to use CEL expressions in Kyverno policies to validate resources covering all the features introduced in Kubernetes ValidatingAdmissionPolicies. Stay tuned for our next post, where we’ll show you how to generate Kubernetes ValidatingAdmissionPolicies from Kyverno policies.