Rebuild the comparison
Bind the workload, correctness rules, and comparable alternatives.
Product vision
The workload performance platform for AI infrastructure
Find the better configuration.Understand the gain.
Turn AI performance hypotheses into correct, reproducible improvements.
For applied researchers, performance engineers, and teams that co-design AI hardware and software.
Make the method reusable
A stack change can mean rebuilding the study.
Reconstruct what ran
Keep settings, versions, every attempt, and known gaps together.
Hand off the method
Show what the evidence supports and how another researcher can rerun it.
Make this decision and keep the method for the next one.
One continuous record
Keep the workload tied to the whole observed stack.
Keep the workload and correctness rule tied to every attempt, its observed stack, versions, gaps, comparison, and rerun recipe.
- Inference engine
- Engine and launch settings
- Compute
- GPU, driver, and firmware
- Fabric
- Topology and links
- Storage
- Data and artifact paths
- Version rail
- Configuration and library versions such as CUDA, Torch, and NCCL
From hypothesis to the next experiment
Change one variable. See what the evidence allows.
- Hypothesis
Name the expected effect and the mechanism that might cause it.
- Workload
Bind the exact workload with the correctness and performance measures that matter, its resource ceiling and stop conditions.
- Controlled variables
Declare what changes and what stays fixed. Keep requested settings apart from observed ones.
- Run and observe
Capture every attempt with its actual configuration, measurements and failures.
- Compare correctness and performance
Check correctness and hard limits first, then compare performance among eligible results.
- Recipe and next experiment
Keep the retained runtime recipe (requested runtime and bundle digest) with its uncertainty and limits, then choose the smallest change that can alter the conclusion.
Output: the Performance Report, one exact record of the plan, evaluation, outcome, and runtime recipe.
Example research questionExplore one controlled comparison
Start with a bounded question. Run only when the controls and evidence sources are available.
Could changing vLLM sequence capacity improve throughput within the same quality and latency limits?
Reference
Record the reference max_num_seqs setting.
Candidate
Change only max_num_seqs.
- Hold fixed
- Model revision, workload profile, prompts, precision, sampling, engine image, GPU placement, request load, and comparison requirements.
- Preview the plan
Confirm the control is supported. Bound requests, duration, and resource use before launch.
- Run and observe
Capture the actual configurations and measurements for every attempt, including failures.
- Inspect the evidence
Read charts beside the configuration difference, correctness checks, latency, throughput, and retained stack diagnostics.
- Report or HOLD
Issue a Performance Report only after required checks pass. Otherwise retain HOLD and the next collection step.
- Rerun and extend
Rerun the retained runtime recipe, then choose the next change from what the evidence explains.
A method that compounds
Grow a private experiment history for each customer across clusters and hardware.
Start with one workload and one cluster. Extend across inference providers, GPU clouds, hardware builders, and research groups as each target earns fresh evidence.
- Decide
Resolve one workload question with a comparison your team can inspect.
- Repeat
Rerun the retained runtime recipe and keep every attempt and gap.
- Choose across targets
Compare fresh, eligible evidence for each cluster and hardware target.
- Guide co-design
Use the observed tradeoffs to choose the next experiment with infrastructure partners.
Fits the workflow you already have
Keep your tools. Keep the research history.
Performance IQ connects the question, correctness rules, observed stack and versions, every attempt and gap, comparison limits, runtime recipe, and next experiment.
- AIPerf and the NVIDIA Dynamo Profiler
- Benchmark and search candidate configurations for the workload question.
- MLflow and Weights & Biases
- Track models and runs while Performance IQ keeps the infrastructure decision in context.
- Your serving platform
- Keep Prometheus, Grafana, NetBox, Kubernetes, Slurm, and Kueue in their existing roles.
- Scripts and manual reconciliation
- Keep local glue where it helps. Performance IQ binds the context and method across the handoff.