Observation only · 2026-10-31T16:47:59.282Z
Performance Report
Recorded performance, request populations, MFU, recipes, and full-stack telemetry for 3 retained runs. This report does not select a winner or establish scientific qualification.
Compare retained runs
3 retained runs collected across 2 independently retained source programs. 2 report-local source epochs are represented; the source epochs differ across runs. Exact private source identities are withheld; this is not one matched program and no winner is claimed.
Observation only. These values do not select a winner or establish scientific qualification.
Viewer-selected reference. Results stay unchanged.
Output throughput
measured-window · tokens/sSeparate retained runs · measured-window · tokens/s. Metric samples and request populations are retained separately.
Model Qwen/Qwen3-8B · Scenario chat · Precision bfloat16 · Engine custom · Hardware NVIDIA B200 · source program 1 · source epoch 1 · Observation completed 2026-10-31T09:33:52.779058Z
Model Qwen/Qwen3-8B · Scenario chat · Precision bfloat16 · Engine vllm · Hardware NVIDIA B200 · source program 2 · source epoch 2 · Observation completed 2026-10-31T02:51:49.686775Z
Model Qwen/Qwen3-8B · Scenario chat · Precision bfloat16 · Engine sglang · Hardware NVIDIA B200 · source program 2 · source epoch 2 · Observation completed 2026-10-31T02:56:09.373007Z
Open circles: display reference. Filled circles: each alternative.
Latency and throughput in each retained run
Each point pairs two recorded measurements from the same alternative.
- 1. Modular MAX 26.7.0 · Run 1 · source program 1 · source epoch 1: End-to-end latency 23436.336 ms; Output throughput 260.87 tokens/s
- 2. vLLM 0.26.0 · Run 2 · source program 2 · source epoch 2: End-to-end latency 27352.546 ms; Output throughput 233.376 tokens/s
- 3. SGLang 0.5.16 · Run 3 · source program 2 · source epoch 2: End-to-end latency 21734.789 ms; Output throughput 252.273 tokens/s
Evidence and coverage
- Engine & requests
- Retained findingsModular MAX 26.7.0 · Run 1: request-timing (partial); Example recorded measurement: engine-queue-time Unavailable · p50 · 0 samples · native-timing-unavailable; engine-cache (unavailable); Example recorded measurement: kv-cache-occupancy Unavailable · p50 · 0 samples · not-recorded; workload-outcomes (unavailable); Example recorded measurement: attempted-requests Unavailable · maximum · 0 samples · details-withheld · Window 2026-10-31T09:32:55.030402Z to 2026-10-31T09:33:42.050859Z. vLLM 0.26.0 · Run 2: request-timing (partial); Example recorded measurement: engine-queue-time Unavailable · p50 · 0 samples · native-timing-unavailable; engine-cache (observed); Example recorded measurement: kv-cache-occupancy 0.37739789373191224 percent · p50 · 56 samples · 1 metric details withheld; workload-outcomes (observed); Example recorded measurement: attempted-requests 2 count · maximum · 1 samples · Window 2026-10-31T02:50:41.288277Z to 2026-10-31T02:51:36.145911Z. SGLang 0.5.16 · Run 3: request-timing (partial); Example recorded measurement: engine-queue-time Unavailable · p50 · 0 samples · native-timing-unavailable; engine-cache (partial); Example recorded measurement: kv-cache-occupancy 0.32890940565493365 percent · p50 · 45 samples; workload-outcomes (observed); Example recorded measurement: attempted-requests 2 count · maximum · 1 samples · Window 2026-10-31T02:55:17.055483Z to 2026-10-31T02:56:00.670065Z Inspect evidence
- Compute & memory
- Retained findingsModular MAX 26.7.0 · Run 1: gpu-activity (observed); Example recorded measurement: gpu-sm-active-ratio 0.6909596306275767 ratio · p50 · 48 samples; host-pressure (observed); Example recorded measurement: cpu-throttled-time 0 microseconds · sum · 47 samples · Window 2026-10-31T09:32:55.030402Z to 2026-10-31T09:33:42.050859Z. vLLM 0.26.0 · Run 2: gpu-activity (observed); Example recorded measurement: gpu-sm-active-ratio 0.6227619740928886 ratio · p50 · 56 samples; host-pressure (observed); Example recorded measurement: cpu-throttled-time 0 microseconds · sum · 55 samples · Window 2026-10-31T02:50:41.288277Z to 2026-10-31T02:51:36.145911Z. SGLang 0.5.16 · Run 3: gpu-activity (observed); Example recorded measurement: gpu-sm-active-ratio 0.6695563471607625 ratio · p50 · 45 samples; host-pressure (observed); Example recorded measurement: cpu-throttled-time 0 microseconds · sum · 44 samples · Window 2026-10-31T02:55:17.055483Z to 2026-10-31T02:56:00.670065Z Inspect evidence
- Fabric & network
- Retained findingsModular MAX 26.7.0 · Run 1: network-tcp (observed); Example recorded measurement: network-tcp-rtt 22 microseconds · p50 · 47 samples; topology-placement (observed); Example recorded measurement: placement-rank 0 count · maximum · 1 samples; collective-progress (unavailable); Example recorded measurement: collective-arrival-spread Unavailable · maximum · 0 samples · retained-capture-unavailable; network-fabric (inapplicable); Example recorded measurement: network-fabric-receive-errors Unavailable · maximum · 0 samples · not-applicable-engine-http-loopback · Window 2026-10-31T09:32:55.030402Z to 2026-10-31T09:33:42.050859Z. vLLM 0.26.0 · Run 2: network-tcp (observed); Example recorded measurement: network-tcp-rtt 56 microseconds · p50 · 55 samples; topology-placement (observed); Example recorded measurement: placement-rank 0 count · maximum · 1 samples; collective-progress (inapplicable); Example recorded measurement: collective-arrival-spread Unavailable · maximum · 0 samples · not-applicable-single-rank; network-fabric (inapplicable); Example recorded measurement: network-fabric-receive-errors Unavailable · maximum · 0 samples · not-applicable-engine-http-loopback · Window 2026-10-31T02:50:41.288277Z to 2026-10-31T02:51:36.145911Z. SGLang 0.5.16 · Run 3: network-tcp (observed); Example recorded measurement: network-tcp-rtt 21 microseconds · p50 · 43 samples; topology-placement (observed); Example recorded measurement: placement-rank 0 count · maximum · 1 samples; collective-progress (inapplicable); Example recorded measurement: collective-arrival-spread Unavailable · maximum · 0 samples · not-applicable-single-rank; network-fabric (inapplicable); Example recorded measurement: network-fabric-receive-errors Unavailable · maximum · 0 samples · not-applicable-engine-http-loopback · Window 2026-10-31T02:55:17.055483Z to 2026-10-31T02:56:00.670065Z Inspect evidence
- Storage
- Retained findingsModular MAX 26.7.0 · Run 1: storage-wait (observed); Example recorded measurement: storage-io-pressure 499826 microseconds · sum · 47 samples · Window 2026-10-31T09:32:55.030402Z to 2026-10-31T09:33:42.050859Z. vLLM 0.26.0 · Run 2: storage-wait (observed); Example recorded measurement: storage-io-pressure 590519 microseconds · sum · 55 samples · Window 2026-10-31T02:50:41.288277Z to 2026-10-31T02:51:36.145911Z. SGLang 0.5.16 · Run 3: storage-wait (observed); Example recorded measurement: storage-io-pressure 185916 microseconds · sum · 44 samples · Window 2026-10-31T02:55:17.055483Z to 2026-10-31T02:56:00.670065Z Inspect evidence
Observed collection scope
3 retained runs collected across 2 independently retained source programs.
2 report-local source epochs are represented; the source epochs differ across runs. Exact private source identities are withheld; this is not one matched program and no winner is claimed.
All measurements, populations and PNG export
Every retained metric, request population and MFU basis remains available here. The PNG contains the original measured charts.
Output throughput
Measured window · tokens/s
Modular MAX 26.7.0 · source program 1 · source epoch 1260.87 tokens/s
2 samples
vLLM 0.26.0 · source program 2 · source epoch 2233.38 tokens/s
2 samples
SGLang 0.5.16 · source program 2 · source epoch 2252.27 tokens/s
2 samples
0260.87 tokens/s
End-to-end latency
p50 · ms
Modular MAX 26.7.0 · source program 1 · source epoch 123,436.34 ms
2 samples
vLLM 0.26.0 · source program 2 · source epoch 227,352.55 ms
2 samples
SGLang 0.5.16 · source program 2 · source epoch 221,734.79 ms
2 samples
027,352.55 ms
Time to first token
p50 · ms
Modular MAX 26.7.0 · source program 1 · source epoch 129.94 ms
2 samples
vLLM 0.26.0 · source program 2 · source epoch 231.25 ms
2 samples
SGLang 0.5.16 · source program 2 · source epoch 226.51 ms
2 samples
031.25 ms
Time per output token
p50 · ms
Modular MAX 26.7.0 · source program 1 · source epoch 13.81 ms
2 samples
vLLM 0.26.0 · source program 2 · source epoch 24.26 ms
2 samples
SGLang 0.5.16 · source program 2 · source epoch 23.94 ms
2 samples
04.26 ms
Correctness
Observed rate · Percent
Modular MAX 26.7.0 · source program 1 · source epoch 1100% (2/2)
2 evaluated requests
vLLM 0.26.0 · source program 2 · source epoch 2100% (2/2)
2 evaluated requests
SGLang 0.5.16 · source program 2 · source epoch 2100% (2/2)
2 evaluated requests
0%100%
Inference MFU (2N estimate)
Measured window · Percent
Modular MAX 26.7.0 · source program 1 · source epoch 10.219%
2 measured requests
MFU reason: None recorded
Reference basis: sha256:6fcc9b7038473d25687399fd78a317bee826ea9423d7cf6c851116bc42b459f0
Operand modelParameterCount: 8190735360
Operand successfulMeasuredInputTokens: 1882
Operand measuredOutputTokens: 12255
Operand measuredWindowMs: 46977.38
Operand totalTokensPerSecond: 300.9320656026368435191575179
Operand acceleratorCount: 1
Operand denseBfloat16FlopsPerAccelerator: 2250000000000000
vLLM 0.26.0 · source program 2 · source epoch 20.195%
2 measured requests
MFU reason: None recorded
Reference basis: sha256:6fcc9b7038473d25687399fd78a317bee826ea9423d7cf6c851116bc42b459f0
Operand modelParameterCount: 8190735360
Operand successfulMeasuredInputTokens: 1882
Operand measuredOutputTokens: 12790
Operand measuredWindowMs: 54804.211
Operand totalTokensPerSecond: 267.7166541089333445563152072
Operand acceleratorCount: 1
Operand denseBfloat16FlopsPerAccelerator: 2250000000000000
SGLang 0.5.16 · source program 2 · source epoch 20.215%
2 measured requests
MFU reason: None recorded
Reference basis: sha256:6fcc9b7038473d25687399fd78a317bee826ea9423d7cf6c851116bc42b459f0
Operand modelParameterCount: 8190735360
Operand successfulMeasuredInputTokens: 1882
Operand measuredOutputTokens: 10991
Operand measuredWindowMs: 43567.806
Operand totalTokensPerSecond: 295.4704673446259836908014142
Operand acceleratorCount: 1
Operand denseBfloat16FlopsPerAccelerator: 2250000000000000
0%100%
Correctness and samples
Modular MAX 26.7.0 · source program 1 · source epoch 1
2 correct / 2 evaluated
1 warmup · 2 measured · 2 successful · 0 failed
Observation completed 2026-10-31T09:33:52.779Z · 0 measured requests without a correctness result
Count coverage reason: Not recorded
Measurement basis and coverage
Count coverage reason: Not recorded
MFU reason: None recorded
Reference basis: sha256:6fcc9b7038473d25687399fd78a317bee826ea9423d7cf6c851116bc42b459f0
Estimated inference MFU uses the 2N total measured token proxy and theoretical dense BF16 peak.
- modelParameterCount
- 8190735360
- successfulMeasuredInputTokens
- 1882
- measuredOutputTokens
- 12255
- measuredWindowMs
- 46977.38
- totalTokensPerSecond
- 300.9320656026368435191575179
- acceleratorCount
- 1
- denseBfloat16FlopsPerAccelerator
- 2250000000000000
vLLM 0.26.0 · source program 2 · source epoch 2
2 correct / 2 evaluated
1 warmup · 2 measured · 2 successful · 0 failed
Observation completed 2026-10-31T02:51:49.686Z · 0 measured requests without a correctness result
Count coverage reason: Not recorded
Measurement basis and coverage
Count coverage reason: Not recorded
MFU reason: None recorded
Reference basis: sha256:6fcc9b7038473d25687399fd78a317bee826ea9423d7cf6c851116bc42b459f0
Estimated inference MFU uses the 2N total measured token proxy and theoretical dense BF16 peak.
- modelParameterCount
- 8190735360
- successfulMeasuredInputTokens
- 1882
- measuredOutputTokens
- 12790
- measuredWindowMs
- 54804.211
- totalTokensPerSecond
- 267.7166541089333445563152072
- acceleratorCount
- 1
- denseBfloat16FlopsPerAccelerator
- 2250000000000000
SGLang 0.5.16 · source program 2 · source epoch 2
2 correct / 2 evaluated
1 warmup · 2 measured · 2 successful · 0 failed
Observation completed 2026-10-31T02:56:09.373Z · 0 measured requests without a correctness result
Count coverage reason: Not recorded
Measurement basis and coverage
Count coverage reason: Not recorded
MFU reason: None recorded
Reference basis: sha256:6fcc9b7038473d25687399fd78a317bee826ea9423d7cf6c851116bc42b459f0
Estimated inference MFU uses the 2N total measured token proxy and theoretical dense BF16 peak.
- modelParameterCount
- 8190735360
- successfulMeasuredInputTokens
- 1882
- measuredOutputTokens
- 10991
- measuredWindowMs
- 43567.806
- totalTokensPerSecond
- 295.4704673446259836908014142
- acceleratorCount
- 1
- denseBfloat16FlopsPerAccelerator
- 2250000000000000
Model: Qwen/Qwen3-8B · Precision: bfloat16 · Concurrency: 1 · Hardware: 1 × NVIDIA B200
Full telemetry, recipes and coverage
- Model
- Qwen/Qwen3-8B
- Precision
- bfloat16
- Concurrency
- 1
- Hardware
- 1 × NVIDIA B200
Recorded recipe
- Engine
- custom
- Engine version
- 26.7.0
- Adapter version
- 0.20.0-dev
- Model
- Qwen/Qwen3-8B
- Scenario
- chat
- Framework
- Modular MAX
- Precision
- bfloat16
- concurrency
- 1
- max Tokens
- 39731
- stream
- True
- temperature
- 0.6
- warmup Requests
- 1
- Model
- Qwen/Qwen3-8B
- Precision
- bfloat16
- Concurrency
- 1
- Hardware
- 1 × NVIDIA B200
Recorded recipe
- Engine
- vllm
- Engine version
- 0.26.0
- Adapter version
- 0.20.0-dev
- Model
- Qwen/Qwen3-8B
- Scenario
- chat
- Framework
- vllm
- Precision
- bfloat16
- concurrency
- 1
- max Tokens
- 39731
- stream
- True
- temperature
- 0.6
- warmup Requests
- 1
- Model
- Qwen/Qwen3-8B
- Precision
- bfloat16
- Concurrency
- 1
- Hardware
- 1 × NVIDIA B200
Recorded recipe
- Engine
- sglang
- Engine version
- 0.5.16
- Adapter version
- 0.20.0-dev
- Model
- Qwen/Qwen3-8B
- Scenario
- chat
- Framework
- sglang
- Precision
- bfloat16
- concurrency
- 1
- max Tokens
- 39731
- stream
- True
- temperature
- 0.6
- warmup Requests
- 1
Observed recipes and system diagnostics
Measurements stay attached to their recorded alternative and run. Diagnostic windows overlap the run in time; they do not establish cause or change the recorded result.
Request & engine
Serving latency, throughput, request outcomes and cache behavior.
Engine queue time
Per-run p50 · ms
Modular MAX 26.7.0 · Run 1Unavailable
vLLM 0.26.0 · Run 1Unavailable
SGLang 0.5.16 · Run 1Unavailable
0 of 3 retained values plotted. Missing and not-applicable values retain their recorded reasons below.
Exact values, samples and coverage
| Alternative / run | Exact value | Population and status |
|---|
Engine prefill time
Per-run p50 · ms
Modular MAX 26.7.0 · Run 1Unavailable
vLLM 0.26.0 · Run 1Unavailable
SGLang 0.5.16 · Run 1Unavailable
0 of 3 retained values plotted. Missing and not-applicable values retain their recorded reasons below.
Exact values, samples and coverage
| Alternative / run | Exact value | Population and status |
|---|
Engine generation time
Per-run p50 · ms
Modular MAX 26.7.0 · Run 1Unavailable
vLLM 0.26.0 · Run 1Unavailable
SGLang 0.5.16 · Run 1Unavailable
0 of 3 retained values plotted. Missing and not-applicable values retain their recorded reasons below.
Exact values, samples and coverage
| Alternative / run | Exact value | Population and status |
|---|
Time to first token
Per-run p50 · ms
KV cache occupancy
Per-run p50 · percent
Attempted requests
Per-run maximum · count
Successful requests
Per-run maximum · count
Failed requests
Per-run maximum · count
Cancelled requests
Per-run maximum · count
Request success rate
Per-run maximum · ratio
Achieved request rate
Per-run maximum · request/s
Output token throughput
Per-run maximum · token/s
Measured window duration
Per-run maximum · ms
GPU
Compute activity, GPU memory, clocks, power and diagnostic codes.
GPU SM activity
Per-run p50 · ratio
GPU tensor pipe activity
Per-run p50 · ratio
GPU DRAM activity
Per-run p50 · ratio
GPU SM clock
Per-run p50 · megahertz
GPU memory clock
Per-run p50 · megahertz
GPU power
Per-run p50 · watt
GPU energy
Per-run sum · joule
GPU temperature
Per-run p50 · celsius
Clock throttle mask
Per-run maximum · bitmask
GPU framebuffer free bytes
Per-run p50 · byte
GPU framebuffer used bytes
Per-run p50 · byte
GPU ECC SBE total
Per-run p50 · count
GPU ECC DBE total
Per-run p50 · count
Observed XID code
Per-run maximum · count
CPU & memory
Host CPU pressure and memory observations.
Request retractions
Per-run p50 · count
Modular MAX 26.7.0 · Run 1Unavailable
0 of 1 retained values plotted. Missing and not-applicable values retain their recorded reasons below.
CPU throttled time
Per-run sum · microseconds
Memory major page faults
Per-run sum · count
Request retractions
Per-run sum · count
SGLang 0.5.16 · Run 1Unavailable
0 of 1 retained values plotted. Missing and not-applicable values retain their recorded reasons below.
Storage
Recorded storage pressure and wait.
| Alternative | Run | Completed | Recorded recipe |
|---|---|---|---|
| Modular MAX 26.7.0 | Run 1 | Oct 31, 2026, 9:33 AM UTC |
|
| vLLM 0.26.0 | Run 1 | Oct 31, 2026, 2:51 AM UTC |
| Alternative | Run | Measurement window | Clock basis | Rows | Coverage |
|---|---|---|---|---|---|
| Modular MAX 26.7.0 | Run 1 | Oct 31, 2026, 9:32 AM UTC – Oct 31, 2026, 9:33 AM UTC | Observer read return | 13 returned / 13 selected / 13 total | Diagnostic metrics: 30 of 50 recorded. Proven not-applicable metrics: 4. Diagnostic collections: 1 N/A · 4 incomplete.
Coverage checks are derived from retained catalogs. Metric and collection counts describe separate populations. |
| vLLM 0.26.0 | Run 1 | Oct 31, 2026, 2:50 AM UTC – Oct 31, 2026, 2:51 AM UTC | Observer read return | 13 returned / 13 selected / 13 total | Diagnostic metrics: 40 of 51 recorded. Proven not-applicable metrics: 8. Diagnostic collections: 2 N/A · 1 incomplete.
Coverage checks are derived from retained catalogs. Metric and collection counts describe separate populations. |
| SGLang 0.5.16 | Run 1 | Oct 31, 2026, 2:55 AM UTC – Oct 31, 2026, 2:56 AM UTC | Observer read return | 13 returned / 13 selected / 13 total | Diagnostic metrics: 39 of 51 recorded. Proven not-applicable metrics: 8. |
Association: temporal overlap only, not causality. Unavailable measurements are not shown as zero.
Report provenance and limits
Report identity: sha256:4e1b24c0f3416f4cc2ebabd45ffe16f3df4e389a152a82599218f937f33d560b
Projection digest: 9af3a860188614b590e7aad086925faf12be508ebfb8a896d1de34743e08e116
Modular MAX 26.7.0: Native timing selector not recorded
vLLM 0.26.0: Native timing selector not recorded
SGLang 0.5.16: Native timing selector not recorded
This retained-run collection spans independently retained source programs and source epochs; it is not one matched program.
Source program and source epoch ordinals are report-local labels; private source identities are withheld.
Observation only. The values describe the recorded requests and times.
Estimated inference MFU uses a 2N token proxy and theoretical dense BF16 peak.
Diagnostic association is temporal overlap, not evidence of causality.