Autonomous GPU performance engineer

Run every model at redline

run 07 · llama-3.1-70bH100 SXM 80 GBTensorRT-LLMExample
  1. 00:00turbokern run llama-3.1-70b · target p95 < 45 ms
  2. 00:02baseline p95 88.4 ms · 1,020 tok/s · 2× H100 needed
  3. 00:09profyle attention is memory-bound · 41% of step timemap
  4. 00:31simyx ranked 212 candidates → benchmark top 18prune
  5. 01:12kernl fused attention, 128×64 tiles −14.2 mskept
  6. 01:40kernl split-K GEMM on o_proj +2.1 msrejected
  7. 02:26quantra FP8 on 76 of 80 layers · acc −0.3% −19.6 mskept
  8. 02:51quantra INT4 on all layers · acc −2.8% over budgetrejected
  9. 03:34servex in-flight batch 64, CUDA Graphs −12.8 mskept
  10. 03:58result p95 41.8 ms · 1,912 tok/s · 1× H100met
p95 latency per step, mstarget 45 ms
baselinestep 7
Example run. Values are illustrative, not a measured result.

Point Turbokern at a model and a target: a lower cost per token, a latency SLO or a GPU budget. It profiles the workload on your NVIDIA GPU, rewrites kernels, quantizes, tunes serving and benchmarks every change. It keeps only what it measured faster and still correct.

Target2×throughput per GPU on a typical LLM endpoint
Target<1%accuracy change allowed by default
TargetDaysfrom first profile to deployable engine

Targets are design goals for the pilot, not measured results.

What it changes

The work of a performance team, run as a loop

Most models run well below what the GPU can do. Closing that gap takes rare engineers and has to be redone for every model and every GPU generation. Turbokern does that work continuously.

Target−50%

Inference cost per million tokens on the same hardware.

Target1 GPU

Serve a 70B model inside its p95 target on one H100 instead of two.

Design rule100%

Of kept changes pass a correctness or accuracy check first.

Target0 rewrites

Hand-written kernels you must maintain when Blackwell arrives.

Targets and design rules for the platform. Turbokern is pre-launch and has no customer results yet.

The problem

Inference is the bill that never stops

Inference is the recurring cost of running AI, and for AI-native companies it is often the largest line on the cloud bill. Most models still run far below the hardware's potential.

Closing that gap means profiling with Nsight, writing CUDA kernels, choosing quantization schemes and tuning TensorRT-LLM configs. That is slow, per-model, per-GPU work for engineers few teams can hire, and every new GPU generation resets it.

  • Teams over-provision GPUs to hit latency targets.
  • Quantization is skipped because a quality regression is riskier than the bill.
  • Hand-tuned kernels go stale when Hopper gives way to Blackwell.
One LLM endpoint, before and afterExample
BaselineAfter TurbokernExample endpoint
p95 latency
88.4 ms
41.8 ms
Throughput
1,020 tok/s
1,912 tok/s
GPUs to serve the SLO
2× H100
1× H100
Example. Values are illustrative, not measured results.

How it works

From a model and a target to a deployable engine

  1. 01Hour 1

    Ingest and baseline

    Connect a model, a target GPU and an objective. Turbokern measures the starting point you will be compared against.

  2. 02Hours 1-4

    Profile

    Profyle captures Nsight and CUPTI traces and ranks bottlenecks by their cost to latency and throughput.

  3. 03Days 1-3

    Optimize and measure

    Candidates from Kernl, Quantra and Servex run on the real GPU. Each one is kept only if it is faster and passes its check.

  4. 04Ongoing

    Ship and re-run

    You get a TensorRT-LLM, Triton or NIM package, a recipe and a report. New models and GPUs trigger a new run.

Timings are planning estimates for a single LLM endpoint.

Who it is for

Built for the people who own the GPU bill

AI-native companies

Cut cost per token

LLM, vision and speech products where inference is the largest cost and latency is the product.

Inference providers

More requests per GPU

Clouds, MSPs and API providers whose margin is served throughput per GPU-hour.

Robotics and edge

Fit the power envelope

Teams deploying perception on Jetson, IGX and DRIVE under latency, power and thermal limits.

Platform teams

One optimization layer

ML infrastructure leads who want every model optimized the same way before it ships.

Built on the NVIDIA stack

We tune the tools you already deploy

Turbokern does not replace your serving stack. It produces better engines, kernels and configs for it.

ProfileNsight Systems and ComputeTimeline and kernel traces
ProfileCUPTIProgrammatic telemetry
KernelsCUDA and CUTLASSGenerated and tuned kernels
KernelscuDNNLibrary baselines
ServeTensorRT-LLMOptimized LLM engines
ServeTriton and NIMPackaged serving
PrecisionFP8 and INT4Hopper and Blackwell tensor cores
EdgeJetson, IGX, DRIVEEmbedded targets

For developers

One command, one config file

Turbokern runs from the CLI or the API. Describe the model, the target hardware and what good looks like. The run report lists every candidate, what was measured and why it was kept or rejected.

Runs happen on your GPUs or ours. Outputs are plain artifacts you can diff, review and roll back.

  • Targets: p50/p95/p99 latency, throughput, cost per token, GPU count.
  • Guards: accuracy budget, numerical tolerance, memory ceiling.
  • Outputs: engine, recipe, report, all versioned.
turbokern.yaml
model: meta-llama/Llama-3.1-70B-Instruct
hardware: h100-sxm-80gb
objective:
  latency_p95_ms: 45
  minimize: gpus
guards:
  max_accuracy_drop: 0.01   # 1% on your eval set
  eval_set: s3://evals/chat-regression-v4
traffic: replay://prod-sample-2026-10
output: [tensorrt-llm, triton, report]

$ turbokern run -f turbokern.yaml

FAQ

Questions engineers ask

Is Turbokern available today?

No. Turbokern is pre-launch. We are recruiting design partners for a pilot that starts in 2027. Everything on this site that looks like a result is an example or a target, and is labelled that way.

What do I need to provide?

A model you can run, access to the GPU type you deploy on (yours or ours), an evaluation set that defines acceptable quality, and a target. Traffic samples help Servex but are optional.

Does Turbokern change model quality?

Only within the budget you set. Quantization and kernel changes are measured against your evaluation set and numerical tolerances. Anything outside the budget is rejected and listed in the report.

Which GPUs are supported?

The pilot targets H100 and H200, with Blackwell and Jetson Orin next. Turbokern is built for NVIDIA GPUs only, because it optimizes directly against CUDA, TensorRT and Nsight.

Do our weights leave our environment?

Not if you choose the on-prem or air-gapped deployment. Runs, artifacts and traces stay on your hardware. See the security page for detail.

Next step

Bring one expensive workload

Request a benchmark on one model and one target. We plan the baseline run with you, and you keep the engine, the recipe and the report whatever happens.