8000 [Bug] Admission controller returns "disallowed operation" for resources explicitly allowed through additional rbac role · Issue #11667 · kyverno/kyverno · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

[Bug] Admission controller returns "disallowed operation" for resources explicitly allowed through additional rbac role #11667

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
achaikaJH opened this issue Nov 25, 2024 · 3 comments
Open

[Bug] Admission controller returns "disallowed operation" for resources explicitly allowed through additional rbac role #11667

achaikaJH opened this issue Nov 25, 2024 · 3 comments
Labels
bug Something isn't working triage Default label assigned to all new issues indicating label curation is needed to fully organize. type:controller controller related issue validation Issues pertaining to the validate ability.

Comments

@achaikaJH
Copy link

Kyverno Version

1.13.0

Kubernetes Version

1.30.x

Kubernetes Platform

AKS

Kyverno Rule Type

Validate

Description

I'm running kyverno 1.13.1 on AKS cluster. I have set of policies which validate istio service mesh virtualservices, serviceentries as well as mutate some scale resources.
All these policies are working fine with kyverno 1.12.4 version.
With kyverno 1.13.1 I get following messages in the admission controller log (provided in kyverno logs section)
I added following permissions to the helm chart values

admissionController:
      featuresOverride: {}
      rbac:
        create: true
        clusterRole:
          # -- Extra resource permissions to add in the cluster role
          extraResources:
            - apiGroups:
                - "*"
              resources:
                - "*"
              verbs:
                - get
                - list
                - watch
            - apiGroups:
                - networking.k8s.io
              resources:
                - ingresses
                - ingressclasses
                - networkpolicies
              verbs:
                - create
                - update
                - patch
                - delete
            - apiGroups:
                - networking.istio.io
              resources:
                - serviceentries
                - virtualservices
              verbs:
                - get
                - list
                - watch
                - create
                - update
                - patch
            - apiGroups:
                - rbac.authorization.k8s.io
              resources:
                - rolebindings
                - roles
              verbs:
                - create
                - update
                - patch
                - delete
            - apiGroups:
                - ""
              resources:
                - configmaps
                - secrets
                - resourcequotas
                - limitranges
              verbs:
                - create
                - update
                - patch
                - delete
            - apiGroups:
                - ""
              resources:
                - pods
                - namespaces
              verbs:
                - update
                - patch
            - apiGroups:
                - apps
              resources:
                - deployments
                - replicasets
                - daemonsets
                - statefulsets
              verbs:
                - update
                - patch
            - apiGroups:
                - policy
              resources:
                - poddisruptionbudgets
              verbs:
                - update
                - patch
            - apiGroups:
                - batch
              resources:
                - cronjobs
                - jobs
              verbs:
                - update
                - patch
            - apiGroups:
                - autoscaling
              resources:
                - horizontalpodautoscalers
                - deployments/scale
              verbs:
                - get
                - list
                - watch
                - update
                - patch

and I see them reflected in the kyverno:admission-controller:additional clusterole:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  annotations:
    meta.helm.sh/release-name: kyverno
    meta.helm.sh/release-namespace: kyverno
  creationTimestamp: "2024-11-22T19:04:57Z"
  labels:
    app.kubernetes.io/component: admission-controller
    app.kubernetes.io/instance: kyverno
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/part-of: kyverno
    app.kubernetes.io/version: 3.3.3
    helm.sh/chart: kyverno-3.3.3
    helm.toolkit.fluxcd.io/name: kyverno
    helm.toolkit.fluxcd.io/namespace: kyverno
  name: kyverno:admission-controller:additional
  resourceVersion: "99991100"
  uid: 99909885-1f1d-4b88-9e63-a2b37415275a
rules:
- apiGroups:
  - '*'
  resources:
  - '*'
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses
  - ingressclasses
  - networkpolicies
  verbs:
  - create
  - update
  - patch
  - delete
- apiGroups:
  - networking.istio.io
  resources:
  - serviceentries
  - virtualservices
  verbs:
  - get
  - list
  - watch
  - create
  - update
  - patch
- apiGroups:
  - rbac.authorization.k8s.io
  resources:
  - rolebindings
  - roles
  verbs:
  - create
  - update
  - patch
  - delete
- apiGroups:
  - ""
  resources:
  - configmaps
  - secrets
  - resourcequotas
  - limitranges
  verbs:
  - create
  - update
  - patch
  - delete
- apiGroups:
  - ""
  resources:
  - pods
  - namespaces
  verbs:
  - update
  - patch
- apiGroups:
  - apps
  resources:
  - deployments
  - replicasets
  - daemonsets
  - statefulsets
  verbs:
  - update
  - patch
- apiGroups:
  - policy
  resources:
  - poddisruptionbudgets
  verbs:
  - update
  - patch
- apiGroups:
  - batch
  resources:
  - cronjobs
  - jobs
  verbs:
  - update
  - patch
- apiGroups:
  - autoscaling
  resources:
  - horizontalpodautoscalers
  - deployments/scale
  verbs:
  - get
  - list
  - watch
  - update
  - patch

Steps to reproduce

  1. Create policies to validate istio service mesh resources and scale as following
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: restrict-service-entry-antipattern-host
  annotations:
    policies.kyverno.io/title: Restrict ServiceEntry Host with anti-patterns
    policies.kyverno.io/category: Istio
    policies.kyverno.io/severity: medium
    kyverno.io/kyverno-version: 1.8.0
    policies.kyverno.io/minversion: 1.6.0
    kyverno.io/kubernetes-version: "1.24"
    policies.kyverno.io/subject: ServiceEntry
    policies.kyverno.io/description: >-
      ServiceEntries in some cases may be too permissive as it
      would direct unintended traffic to the given resource. This
      policy enforces that any ServiceEntry host does not contain
      antipattern subdomains/domains.
spec:
  validationFailureAction: Audit
  background: true
  rules:
  - name: block-service-entry-antipattern-host
    context:
      - name: namespacefilters
        configMap:
          name: namespace-filters
          namespace: kyverno
    match:
      any:
      - resources:
          kinds:
          - ServiceEntry
    preconditions:
      all:
      - key: "{{ request.namespace || '' }}"
        operator: AnyNotIn
        value: "{{ \"namespacefilters\".data.\"infralist\" | parse_json(@) }}"
    validate:
      message: >
        ServiceEntry cannot contain antipatterns like * and _. 
      foreach:
      - list: "request.object.spec.hosts"
        deny:
          conditions:
            any:
            - key: "{{ contains(element, '*.') }}"
              operator: Equals
              value: true
            - key: "{{ contains(element, '_') }}"
              operator: Equals
              value: true
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: mutate-force-costscaledown-deploy-hpa-pdb
  annotations:
    policies.kyverno.io/title: MUTATE - Force Cost Scaledown Deploy/Hpa/Pdb
    policies.kyverno.io/category: Cost Optimzation
    policies.kyverno.io/severity: medium
    policies.kyverno.io/subject: Deployment, HorizontalPodAutoscaler, PodDisruptionBudget
    policies.kyverno.io/description: >-
      This forces deployment replicas to 1, hpa minReplicas to 1 and disables pdb to allow disruptions.      
spec:
  mutateExistingOnPolicyUpdate: true
  background: true
  rules:
  - name: mutate-deployment-replicas
    context:
      - name: namespacefilters
        configMap:
          name: namespace-filters
          namespace: kyverno
    match:
      any:
      - resources:
          kinds:
          - Deployment
          - Deployment/scale
    exclude:
      any:
      - resources:
          kinds:
            - Deployment
            - Deployment/scale
          namespaces:
          - "*-uat"
          - "*-uat-ext"
          - "*-prod"
          - "*-prod-ext"
          selector:
            matchExpressions:
              - key: "app.kubernetes.io/managed-by"
                operator: In
                values:
                  - "Helm"
    preconditions:
      all:
      - key: "{{ request.namespace || '' }}"
        operator: AnyNotIn
        value: "{{ \"namespacefilters\".data.\"infralist\" | parse_json(@) }}"
      - key: "{{request.operation || 'BACKGROUND'}}"
        operator: NotEquals
        value: DELETE
    mutate:
      targets:
      - apiVersion: apps/v1
        kind: Deployment
        name: "{{ request.object.metadata.name }}"
        namespace: "{{ request.object.metadata.namespace }}"
      patchStrategicMerge:
        metadata:
          labels:
            random_trigger: "forcemutate-{{ random('[0-9a-z]{8}') }}"
        spec:
          replicas: 1

Expected behavior

kyverno should be able to process and validate resources without errors.

Screenshots

No response

Kyverno logs

kyverno-admission-controller-7d5448df6d-7klv5 kyverno 2024-11-25T19:21:43Z INF github.com/kyverno/kyverno/pkg/auth/auth.go:83 > disallowed operation evaluationError= gvr="autoscaling/v1, Resource=deployments/scale" kind=Deployment/scale logger=auth namespace= reason="Azure does not have opinion for this user." v=0 verb=list
kyverno-admission-controller-7d5448df6d-7klv5 kyverno 2024-11-25T19:21:43Z INF github.com/kyverno/kyverno/pkg/auth/auth.go:83 > disallowed operation evaluationError= gvr="autoscaling/v1, Resource=deployments/scale" kind=Deployment/scale logger=auth namespace= reason="Azure does not have opinion for this user." v=0 verb=watch
kyverno-admission-controller-7d5448df6d-7klv5 kyverno 2024-11-25T19:21:47Z INF github.com/kyverno/kyverno/pkg/auth/auth.go:83 > disallowed operation evaluationError= gvr="networking.istio.io/v1beta1, Resource=serviceentries" kind=ServiceEntry logger=auth namespace= reason="Azure does not have opinion for this user." v=0 verb=get
kyverno-admission-controller-7d5448df6d-7klv5 kyverno 2024-11-25T19:21:47Z INF github.com/kyverno/kyverno/pkg/auth/auth.go:83 > disallowed operation evaluationError= gvr="networking.istio.io/v1beta1, Resource=serviceentries" kind=ServiceEntry logger=auth namespace= reason="Azure does not have opinion for this user." v=0 verb=list
kyverno-admission-controller-7d5448df6d-7klv5 kyverno 2024-11-25T19:21:47Z INF github.com/kyverno/kyverno/pkg/auth/auth.go:83 > disallowed operation evaluationError= gvr="networking.istio.io/v1beta1, Resource=serviceentries" kind=ServiceEntry logger=auth namespace= reason="Azure does not have opinion for this user." v=0 verb=watch
kyverno-admission-controller-7d5448df6d-7klv5 kyverno 2024-11-25T19:21:47Z INF github.com/kyverno/kyverno/pkg/auth/auth.go:83 > disallowed operation evaluationError= gvr="networking.istio.io/v1beta1, Resource=serviceentries" kind=ServiceEntry logger=auth namespace= reason="Azure does not have opinion for this user." v=0 verb=get
kyverno-admission-controller-7d5448df6d-7klv5 kyverno 2024-11-25T19:21:47Z INF github.com/kyverno/kyverno/pkg/auth/auth.go:83 > disallowed operation evaluationError= gvr="networking.istio.io/v1beta1, Resource=serviceentries" kind=ServiceEntry logger=auth namespace= reason="Azure does not have opinion for this user." v=0 verb=list
kyverno-admission-controller-7d5448df6d-7klv5 kyverno 2024-11-25T19:21:47Z INF github.com/kyverno/kyverno/pkg/auth/auth.go:83 > disallowed operation evaluationError= gvr="networking.istio.io/v1beta1, Resource=serviceentries" kind=ServiceEntry logger=auth namespace= reason="Azure does not have opinion for this user." v=0 verb=watch
kyverno-admission-controller-7d5448df6d-7klv5 kyverno 2024-11-25T19:21:47Z INF github.com/kyverno/kyverno/pkg/auth/auth.go:83 > disallowed operation evaluationError= gvr="networking.istio.io/v1beta1, Resource=serviceentries" kind=ServiceEntry logger=auth namespace= reason="Azure does not have opinion for this user." v=0 verb=get
kyverno-admission-controller-7d5448df6d-7klv5 kyverno 2024-11-25T19:21:48Z INF github.com/kyverno/kyverno/pkg/auth/auth.go:83 > disallowed operation evaluationError= gvr="networking.istio.io/v1beta1, Resource=serviceentries" kind=ServiceEntry logger=auth namespace= reason="Azure does not have opinion for this user." v=0 verb=list
kyverno-admission-controller-7d5448df6d-7klv5 kyverno 2024-11-25T19:21:48Z INF github.com/kyverno/kyverno/pkg/auth/auth.go:83 > disallowed operation evaluationError= gvr="networking.istio.io/v1beta1, Resource=serviceentries" kind=ServiceEntry logger=auth namespace= reason="Azure does not have opinion for this user." v=0 verb=watch
kyverno-admission-controller-7d5448df6d-7klv5 kyverno 2024-11-25T19:21:48Z INF github.com/kyverno/kyverno/pkg/auth/auth.go:83 > disallowed operation evaluationError= gvr="networking.istio.io/v1beta1, Resource=virtualservices" kind=VirtualService logger=auth namespace= reason="Azure does not have opinion for this user." v=0 verb=get
kyverno-admission-controller-7d5448df6d-7klv5 kyverno 2024-11-25T19:21:48Z INF github.com/kyverno/kyverno/pkg/auth/auth.go:83 > disallowed operation evaluationError= gvr="networking.istio.io/v1beta1, Resource=virtualservices" kind=VirtualService logger=auth namespace= reason="Azure does not have opinion for this user." v=0 verb=list
kyverno-admission-controller-7d5448df6d-7klv5 kyverno 2024-11-25T19:21:48Z INF github.com/kyverno/kyverno/pkg/auth/auth.go:83 > disallowed operation evaluationError= gvr="networking.istio.io/v1beta1, Resource=virtualservices" kind=VirtualService logger=auth namespace= reason="Azure does not have opinion for this user." v=0 verb=watch


### Slack discussion

_No response_

### Troubleshooting

- [X] I have read and followed the documentation AND the [troubleshooting guide](https://kyverno.io/docs/troubleshooting/).
- [X] I have searched other issues in this repository and mine is not recorded.
@achaikaJH achaikaJH added bug Something isn't working triage Default label assigned to all new issues indicating label curation is needed to fully organize. labels Nov 25, 2024
@welcome Welcome
Copy link
welcome bot commented Nov 25, 2024

Thanks for opening your first issue here! Be sure to follow the issue template!

@dosubot dosubot bot added type:controller controller related issue validation Issues pertaining to the validate ability. labels Nov 25, 2024
@slawekww
Copy link

The same issue experienced.

2025-03-13T09:05:55Z INF github.com/kyverno/kyverno/pkg/auth/auth.go:83 > disallowed operation evaluationError= gvr="networking.istio.io/v1, Resource=gateways" kind=Gateway logger=auth namespace= reason= v=0 verb=get
2025-03-13T09:05:55Z INF github.com/kyverno/kyverno/pkg/auth/auth.go:83 > disallowed operation evaluationError= gvr="networking.istio.io/v1, Resource=gateways" kind=Gateway logger=auth namespace= reason= v=0 verb=list
2025-03-13T09:05:55Z INF github.com/kyverno/kyverno/pkg/auth/auth.go:83 > disallowed operation evaluationError= gvr="networking.istio.io/v1, Resource=virtualservices" kind=VirtualService logger=auth namespace= reason= v=0 verb=get
2025-03-13T09:05:55Z INF github.com/kyverno/kyverno/pkg/auth/auth.go:83 > disallowed operation evaluationError= gvr="networking.istio.io/v1, Resource=gateways" kind=Gateway logger=auth namespace= reason= v=0 verb=watch
2025-03-13T09:05:55Z INF github.com/kyverno/kyverno/pkg/auth/auth.go:83 > disallowed operation evaluationError= gvr="networking.istio.io/v1, Resource=virtualservices" kind=VirtualService logger=auth namespace= reason= v=0 verb=list

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working triage Default label assigned to all new issues indicating label curation is needed to fully organize. type:controller controller related issue validation Issues pertaining to the validate ability.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@slawekww @achaikaJH
0