Kubernetes

Running LLMs on Kubernetes means packaging model-serving workloads and using Kubernetes APIs and extensions to place, scale, secure, observe, and update them on GPU nodes. For how to run LLMs on Kuber