What happened
The New Stack published a piece arguing that running GPUs on Kubernetes is “bring-your-own-everything”, and that this starts with the bill. The article opens with a familiar picture: most Kubernetes platforms already have a Grafana dashboard showing resource use for workloads, but that habit does not automatically carry over to GPUs. We only had the opening of the piece to work from, so treat what follows as our reading of the theme, not a summary of every detail.
Why it matters
For CPU and memory, most teams get something useful with little effort: requests, limits and usage show up in the usual dashboards. GPUs tend to be different. They are expensive, they are often requested as whole units, and the tooling you rely on for everything else may tell you little about who is using them or how busy they really are.
If you have no platform team, this is where surprises happen. A developer asks for a GPU for an experiment, the pod finishes, and the node stays up. Or a team reserves a card and uses a fraction of it. Without a cost and utilisation view that you have set up yourself, the first clear signal is the cloud invoice.
The point of the headline is that nobody ships this view for you. Scheduling, monitoring, cost attribution and cleanup are all choices you make and then maintain.
What to do this week
- List every workload that requests a GPU, and who owns it. If you cannot name an owner, that is your first finding.
- Check whether your current dashboards show GPU utilisation at all, not just that a GPU is allocated. Allocated and busy are different things.
- Look at how GPU nodes scale down. Confirm that idle nodes actually go away, and how long that takes.
- Add labels or namespaces so GPU spend can be tied to a team or project, even roughly.
- Set a budget alert on GPU instance spend in your cloud account, so a surprise reaches you in days rather than at month end.
None of this needs new platforms. It needs someone to decide what “good” looks like and write it down.
Need a hand?
If you are starting to run GPU workloads on Kubernetes and want the cost and utilisation view in place before the bill arrives, our Kubernetes consulting is built for that. We can review your cluster, show you where GPU spend is hiding and suggest the smallest set of changes that helps. The first call is free and it is with an engineer, not a sales team.
Source: GPUs on Kubernetes are bring-your-own-everything, starting with the bill
Related service: Kubernetes Consulting