Things built to be run

A self-hosted AI stack for an on-prem Kubernetes cluster, in one chart: a local LLM engine, a chat frontend, and an image-generation workflow runner. Each piece is its own subchart and can be switched off alone, and a shared library chart keeps naming, labels, scheduling, persistence and GPU handling under one set of rules instead of three.

Ollama

LLM inference engine, holds the model weightscluster-internal

Open WebUI

chat frontendIngress or HTTPRoute

ComfyUI

image generation workflowsport-forward

# install from the published repository
helm repo add klahio https://charts.klah.io
helm repo update
helm install ai klahio/ai-stack-lite -n ai-stack --create-namespace

Profiles ship for the two common shapes: values-single-gpu.yaml for one consumer GPU shared by Ollama and ComfyUI, and values-cpu.yaml for a cluster with no GPU at all.

The chart does not install GPU support, on purpose. A device plugin or the GPU Operator is node-level infrastructure with its own lifecycle, and bundling it would mean a helm uninstall of this stack takes the cluster's GPU support with it. The chart checks the prerequisites at install time and fails with instructions instead.

A kubectl plugin for the day-to-day operation of a single KServe InferenceService or LLMInferenceService. One of them turns into a serving runtime, Deployments, pods with a model-download init container, Services, an Ingress or HTTPRoutes and an autoscaler — so when a model is not serving, the reason is usually several objects away, and KServe's own status reports the symptom rather than the cause. Three commands answer three questions.

status

Is my model serving, and if not, why?follows InferenceService → Deployment → pod → container and names the cause

test

Does it actually answer a request?finds the route and the protocol itself; --prompt for an LLM

logs

What did its containers print?the right containers, without pod names

# install with krew, or go install
kubectl krew install inferon
kubectl inferon status sklearn-iris -n kserve-test

Failed model downloads, images that cannot be pulled, crashing servers, OOM kills, pods that cannot be scheduled, a model format with no serving runtime, an external address that does not answer even while KServe reports IngressReady=True — each is reported as a diagnosis with the pod it came from, the relevant log lines, and a hint at the field to fix.

The plugin never creates, changes or deletes a cluster object; get, list and a port-forward for test are all it asks for. It reads KServe objects as unstructured data and checks for fields rather than version numbers, so it does not depend on KServe's Go module. Standard deployment mode only — Knative serverless is not supported yet.