What exactly is RawDeployment?

[iciclespider]

I have looked high and low in the documentation and the source code, but I have been unable to find a nice explanation of what Raw Deployment does, and does not, do. I did a quick test and it appears it does not do storage initialization, is that correct?

[titmuthuraj]

Kserve supports two types of deployment modes. one is Serverless mode and another one is RawDeployment mode.

1. We can use the Serverless mode to enable autoscaling based on request volume and supports scale down to and from zero using Knative.

2. We can use the RawDeployment mode to enable InferenceService deployment with Kubernetes resources Deployment, Service, Ingress and Horizontal Pod Autoscaler. Comparing to serverless deployment it unlocks Knative limitations such as mounting multiple volumes. But "Scale down to and from Zero" is not supported in RawDeployment mode.

Note: Both the Serverless and RawDeployment mode does the storage initialization.

[israel-hdez]

Would you confirm if you are using the same 0.11.0 version as in #3499?

[iciclespider]

I am using v0.12.0, from a kubectl describe pod of the kserve-controller:

    Image:         kserve/kserve-controller:v0.12.0
    Image ID:      docker.io/kserve/kserve-controller@sha256:59d7d878a430722a7c7ad9a95bef082fb3ddc419506bdacf3f8a9ceeb2d5975d

[iciclespider]

And this is the kustomization.yaml configuration I am using to deploy KServe:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: kserve

resources:
- https://github.com/kserve/kserve/releases/download/v0.12.0/kserve.yaml

patches:
- target:
    kind: ConfigMap
    name: inferenceservice-config
  patch: |
    - path: /data/deploy
      op: replace
      value: |-
        {
            "defaultDeploymentMode": "RawDeployment"
        }
    - path: /data/ingress
      op: replace
      value: |-
        {
            "disableIstioVirtualHost": true,
            "disableIngressCreation": true,
            "ingressDomain": "svc.cluster.local",
            "ingressGateway" : "disabled",
            "ingressService" : "disabled"
        }

[iciclespider]

I just reread the comments in this issue #3499 and in my case I was also deploying the InferenceService to the kserve namespace. When I deployed it to a different namespace, storage was properly initialized. Mystery solved I believe. However, some documentation is needed clarifying this. Actually, IMO, I do not see a reason for not supporting InferenceServices in the kserve namespace.

[iciclespider]

On the other, other hand...

When deployed to the "kserve" namespace in RawDeployment mode, the kserve operator did create the Deployment and the Pod did get created. It was then the kserve mutating webhook that did not process the needed storage initialization. If deploying to the kserve namespace is not supported, the kserve operator should not be creating the Deployment and other resources. If it is supported, storage initialization is broken.