Python SDK for KFServing Server and Client.
KFServing Python SDK can be installed by pip or Setuptools.
pip install kfservingInstall via Setuptools.
python setup.py install --user(or sudo python setup.py install to install the package for all users)
KFServing's python server libraries implement a standardized KFServing library that is extended by model serving frameworks such as Scikit Learn, XGBoost and PyTorch. It encapsulates data plane API definitions and storage retrieval for models.
KFServing provides many functionalities, including among others:
- Registering a model and starting the server
- Prediction Handler
- Liveness Handler
- Readiness Handlers
KFServing supports the following storage providers:
- Google Cloud Storage with a prefix: "gs://"
- By default, it uses
GOOGLE_APPLICATION_CREDENTIALSenvironment variable for user authentication. - If
GOOGLE_APPLICATION_CREDENTIALSis not provided, anonymous client will be used to download the artifacts.
- By default, it uses
- S3 Compatible Object Storage with a prefix "s3://"
- By default, it uses
S3_ENDPOINT,AWS_ACCESS_KEY_ID, andAWS_SECRET_ACCESS_KEYenvironment variables for user authentication.
- By default, it uses
- Azure Blob Storage with the format: "https://{$STORAGE_ACCOUNT_NAME}.blob.core.windows.net/{$CONTAINER}/{$PATH}"
- By default, it uses anonymous client to download the artifacts.
- For e.g. https://kfserving.blob.core.windows.net/triton/simple_string/
- Local filesystem either without any prefix or with a prefix "file://". For example:
- Absolute path:
/absolute/pathorfile:///absolute/path - Relative path:
relative/pathorfile://relative/path - For local filesystem, we recommended to use relative path without any prefix.
- Absolute path:
- Persistent Volume Claim (PVC) with the format "pvc://{$pvcname}/[path]".
- The
pvcnameis the name of the PVC that contains the model. - The
[path]is the relative path to the model on the PVC. - For e.g.
pvc://mypvcname/model/path/on/pvc
- The
KFServing's python client interacts with KFServing APIs for executing operations on a remote KFServing cluster, such as creating, patching and deleting of a InferenceService instance. See the Sample for KFServing Python SDK Client to get started.
| Class | Method | Description |
|---|---|---|
| KFServingClient | set_credentials | Set Credentials |
| KFServingClient | create | Create InferenceService |
| KFServingClient | get | Get or watch the specified InferenceService or all InferenceServices in the namespace |
| KFServingClient | patch | Patch the specified InferenceService |
| KFServingClient | replace | Replace the specified InferenceService |
| KFServingClient | rollout_canary | Rollout the traffic on canary version for specified InferenceService |
| KFServingClient | promote | Promote the canary version of the InferenceService to default |
| KFServingClient | delete | Delete the specified InferenceService |
| KFServingClient | wait_isvc_ready | Wait for the InferenceService to be ready |
| KFServingClient | is_isvc_ready | Check if the InferenceService is ready |
- KnativeAddressable
- KnativeCondition
- KnativeURL
- KnativeVolatileTime
- NetUrlUserinfo
- V1alpha2AlibiExplainerSpec
- V1alpha2Batcher
- V1alpha2CustomSpec
- V1alpha2DeploymentSpec
- V1alpha2EndpointSpec
- V1alpha2ExplainerSpec
- V1alpha2InferenceService
- V1alpha2InferenceServiceList
- V1alpha2InferenceServiceSpec
- V1alpha2InferenceServiceStatus
- V1alpha2Logger
- V1alpha2ONNXSpec
- V1alpha2PredictorSpec
- V1alpha2PyTorchSpec
- V1alpha2SKLearnSpec
- V1alpha2StatusConfigurationSpec
- V1alpha2TritonSpec
- V1alpha2TensorflowSpec
- V1alpha2TransformerSpec
- V1alpha2XGBoostSpec