On a lot of our first calls, someone pulls up a GPU dashboard to show us how busy their cluster is. The graph looks healthy. Then we ask what those GPUs were doing at 2am on a Saturday, and the call gets a little quieter.
Nothing is broken. The dashboard measures something real. It shows how hard a GPU works while a job is running, and that is a different question from how many hours a day the GPU is doing anything at all. Your budget cares about the second question.
Full transparency, we built Orion to share GPUs better, so we have skin in this game. Take that into account. I am also not going to pretend this is a deep dive into what happens inside the chip. Plenty of people go much further into GPU profiling than I will here. This post stays at the level where IT teams and studio leads actually make buying decisions.
Two numbers with the same name
When someone tells you their GPU utilization is 80%, they usually mean one of two things.
The first is how busy a GPU is while a job is on it. Most dashboards show this one. A render or a training run kicks off, the graph climbs, and it looks great. It is also the number vendors like to quote, because it is the flattering one.
The second is fleet utilization. Take every GPU you own and every hour of the month, and ask what share of those hours had real work on them. That number tells you whether the hardware is paying for itself. It is also the hardest one to find in most monitoring setups.
Both numbers are real. They answer different questions, and mixing them up is how a team sitting on a lot of idle hardware ends up asking for more.
Why the dashboard looks fine
Averages hide most of it. A GPU that ran flat out for an hour and sat idle for three reports 25% over that four-hour block. The math is right. But roll that into a daily average for a team that works business hours, and the number you look at each morning tells you very little about how much capacity you really use.
Allocated is not the same as used, either. A workload gets a GPU, then spends twenty minutes waiting on a file copy, or on a person who went to lunch. The GPU belongs to that workload the whole time, nothing useful runs on it, and most monitoring still counts it as in use.
And most people look one machine at a time. Each machine seems to have a good reason to exist. Very few teams look at the whole fleet across a whole month.
What the real number usually looks like
While jobs are running, the numbers tend to look reasonable, and 60-80% is common in what we have seen. The problem is how many hours a day anything is running.
Batch jobs run in windows. A researcher submits a job, goes to a meeting, comes back, and waits on results. Development and testing come in bursts. Interactive work follows people, so it stops whenever the person does.
Fold all of that idle time into the fleet number and it drops fast. Goldman Sachs and Cast AI have both put average enterprise on-premises GPU utilization at 10-15% (sources on our methodology page). What we see on first calls with new teams lands in the same range.
When people push back with higher figures, they are usually quoting the busy-while-running number. The 10-15% is fleet-wide and averaged over time. Both are real, and they mean different things.
The 10-15% figure is the uncomfortable one. It means you are paying for a cluster that does useful work for roughly one hour out of every eight. For the other seven, those GPUs are waiting.
The workstation problem
Interactive GPU workstations are where this gets really bad, because they behave nothing like a render farm.
A farm can be tuned. You queue jobs, schedule around demand, and keep the machines fed. The GPU is either working or waiting for the next job, and you control how long it waits.
A workstation has a person in front of it, so usage follows the person. It spikes during a heavy preview, drops to nothing while they read notes or take a call, and spikes again when they change something and preview again. The GPU is sized for the artist’s busiest moment and runs at the artist’s average. Those two are nowhere near each other.
Giving every artist their own GPU means paying for that busiest moment around the clock. If an artist works eight hours and the GPU is busy for three of them, you are at about 37% on a good day. Nights, weekends, and vacations cost the same and produce nothing.
Sharing is the fix we built Orion around. Orion runs workstations as containers on a shared pool, so several of them can sit on one GPU, and a workstation that nobody is using can shut itself down and hand its share back. We do not slice the GPUs ourselves, NVIDIA does that. We decide what runs where. In 2025, R3D Studios moved from one GPU per artist to shared scheduling and reported 2:1 GPU density, with 10 artists across 5 GPU instances, and up to ~40% compute cost reduction. The artists did not notice the change. The capacity had been in the hardware the whole time. Nobody was using it.
How to measure it properly
Start with the fleet number over 30 days. Add up every GPU-hour you had available across the cluster. Then count the hours where a GPU was doing meaningful work, say above 10% busy. Divide the second by the first. That tells you whether your hardware spend is working.
Then check how often GPUs are handed to a workload at all, separately from how often they are busy. A GPU can belong to a workload and still sit completely idle, and you want to see both.
Split it by workload type. Training, inference, workstations, and rendering each behave differently and each has its own fix. Average them together and you will miss what is going wrong.
Look at idle time by hour of day as well. Most clusters go quiet overnight and on weekends. If your business runs 9-5 in one timezone, your GPUs are probably idle for more than half the hours you are paying for.
One day’s snapshot will not tell you much, because GPU work comes in bursts and depends on the project. Thirty days with a daily breakdown will.
What good utilization looks like
There is no universal target. It depends on what you run.
For batch training where throughput is the goal, 70-80% across scheduled windows that cover most of the day is very good. It takes real scheduling discipline to get there and stay there.
For interactive workstations, two or three people per GPU with good scheduling is a realistic step up from one each. Past that, people start to feel the sharing, and most creative teams will not accept that.
For inference serving, utilization is almost the wrong thing to watch. What matters is whether answers come back fast enough and what each one costs. A quiet inference cluster is fine if it keeps up.
The number to compare against is your own, measured properly, next to what better scheduling would give you. An industry-wide “good” line will not tell you much.
Before you buy more
GPU lead times are long right now, and cloud GPUs get scarce when demand peaks. If you are planning AI work that needs more GPU capacity, the first question is whether you are using what you already have.
A cluster at 12% fleet utilization has a lot of room before it needs new hardware. With better scheduling, the same cluster might run twice the work at the same cost. So this is about what your team can take on as much as what it costs. The work people turned down because they thought they were out of capacity is the hardest thing to put a number on, and it is usually the biggest.
Our number, for what it is worth
The concrete result we can point to is the 2025 one above, 2:1 density for one team of artists. That is one workload type at one scale. Your result depends on where you start, what you run, and what capacity means for your team, and other workloads will land somewhere else.
If you want to know what yours might look like, start by measuring your fleet utilization. Use the fleet number, not the dashboard one.
See how Orion handles your workload mix. Book a demo
WRITTEN BY
Alex Hatfield
Alex co-founded Juno to fix how enterprise compute gets done. He leads product vision and customer strategy, and spends most of his time working directly with infrastructure and research teams pushing the limits of what their hardware can do.