Status
To be run. Supplier contracts are not final, so there are no plans to test yet. When a plan is contracted, we run this method on it and publish the results here, with the date, the raw output and the exact server specification.
Principles
- Same method on every plan; any change to the method is versioned and dated.
- At least three runs at different times of day; we report the median and the range, never only the best run.
- Fresh servers from the normal provisioning path, not hand-picked hardware.
- Raw output published with each result.
- Results are labelled with the plan, region, supplier and date they apply to.
- No comparisons with competitors unless we ran the same method on their service ourselves and say when.
The tests
| Area | Tool and settings | What we report | Status |
|---|---|---|---|
| CPU, single and multi-thread | sysbench cpu and 7-Zip benchmark (7z b), plus the OpenSSL tests above | Score per thread and for all threads; spread across runs; CPU steal time during the test | To be run |
| Memory bandwidth | STREAM (triad), all threads and one thread | GB/s, best of 10 iterations, median of 3 runs | To be run |
| Disk: random 4 KiB | fio, randread and randwrite, iodepth 1 and 32, direct=1, 60 s, 10 GiB file | IOPS and p99 latency | To be run |
| Disk: sequential 1 MiB | fio, read and write, iodepth 8, direct=1 | MB/s | To be run |
| Disk: sync commit latency | fio, 4 KiB writes with fsync after each | p50 and p99 latency | To be run |
| Network throughput | iperf3 to two public endpoints and between two servers in the same region, 30 s, 4 streams | Mbit/s each direction | To be run |
| Network latency | mtr to reference endpoints, 100 cycles | Median and p95 RTT; loss | To be run |
| Web stack | k6 against a reference WordPress and a reference API container | Requests per second at p95 latency under 200 ms | To be run |
| GPU inference | vLLM benchmark_serving with Llama 3.1 8B, fixed prompt and output lengths, concurrency 1 to 64 | Output tokens per second and time to first token | To be run |
Example: the fio random-read job
[global]
ioengine=libaio
direct=1
time_based=1
runtime=60
size=10G
filename=/root/fio.test
[randread-qd1]
rw=randread
bs=4k
iodepth=1
[randread-qd32]
stonewall
rw=randread
bs=4k
iodepth=32Ask for a test on your configuration
If you are choosing between configurations, ask in your quote request for one of these tests on the exact configuration. Where the supplier allows a trial server, we will run it and send you the raw output.
Sources
- fio documentation, fio project
- STREAM benchmark, University of Virginia
- iperf3, ESnet
- vLLM benchmarks, vLLM project