An independent benchmarking study published on arXiv (arXiv:2608.05202) provides a protocol-level evaluation of commercial quantum error suppression and error mitigation software stacks. Conducted by researchers from The Catholic University of America, University of Deusto, and Universidad de los Andes, the paper compares native IBM Qiskit Runtime primitives against third-party managed pipelines in the Qiskit Functions Catalog—specifically Q-CTRL Performance Management and Qedma QESEM. All experiments were executed on ibm_pittsburgh, a 156-qubit IBM Quantum Heron r3 superconducting processor.
The benchmark evaluates two execution models: sampling tasks (bitstring counts) via Sampler interfaces, and observable expectation values via Estimator interfaces. To ensure like-for-like comparisons, IBM and Q-CTRL configurations operated under matched shot budgets (215 = 32,768 shots per instance), while Qedma QESEM was configured through its native precision-and-time-budgeted interface (0.1 precision target, 600-second QPU time cap). Abstract workloads were submitted across all providers within identical hardware calibration windows.
In the Sampler Track, tested across Bernstein–Vazirani (25–75 qubits), Quantum Phase Estimation (10–30 counting qubits), GHZ-state preparation (25–50 qubits), and Randomized Mirror Circuits (25–100 qubits), Q-CTRL’s autonomous suppression pipeline delivered the highest exact-output success probabilities across all 12 test cases. On 30-qubit Quantum Phase Estimation (QPE), where raw and measurement-twirled IBM runs yielded zero exact-match shots out of 32,768 (0.00% success), Q-CTRL maintained a 12.69% exact success probability. On 100-qubit Randomized Mirror Circuits, Q-CTRL retained a 76.45% exact 100-bit target string success rate, compared to 9.14% on IBM raw execution.
| [ Estimator Track Accuracy & QPU Overhead Benchmarks (TFIM 25–75 Qubits) ] | ||
|---|---|---|
| Software Stack Configuration | Error Metrics vs. MPS Exact Reference | Error Reduction & QPU Runtime |
| IBM Raw Execution (SamplerV2 / EstimatorV2 Level 0) | • Magnetization MAE (mX): 0.1484 • Correlator MAE (cZZ): 0.0282 • Overall MAE: 0.0883 | • 1.00× (Baseline) • Reported QPU Time: ~17.8 seconds per job |
| IBM TREX + Twirling (Resilience Level 1 + Gate Twirling) | • Magnetization MAE (mX): 0.1026 • Correlator MAE (cZZ): 0.0587 • Overall MAE: 0.0807 | • 1.09× Error Reduction • Reported QPU Time: ~28.0–37.8 seconds per job |
| Q-CTRL Performance Management (Autonomous Suppression Pipeline) | • Magnetization MAE (mX): 0.0376 • Correlator MAE (cZZ): 0.0194 • Overall MAE: 0.0285 | • 3.10× Error Reduction • Reported QPU Time: ~28.0 seconds per job |
| Qedma QESEM (Characterization & Unbiased Mitigation) | • Magnetization MAE (mX): 0.0278 • Correlator MAE (cZZ): 0.0097 • Overall MAE: 0.0188 | • 4.70× Error Reduction • Reported QPU Time: ~211–311 seconds per job |
In the Estimator Track, evaluated on an eight-layer Transverse-Field Ising Model (TFIM) circuit at 25, 50, and 75 qubits against an exact Matrix Product State (MPS) reference, Q-CTRL and Qedma QESEM reduced aggregate mean absolute error (MAE) by factors of 3.10× and 4.70× over raw execution, respectively. However, the two third-party stacks demonstrated distinct resource-accuracy profiles: Q-CTRL achieved its error reduction using 28 seconds of QPU time per job (matching IBM baseline runtime), whereas Qedma QESEM achieved the highest precision by consuming 211 to 311 seconds of QPU time per job (7.5× to 11.1× the runtime of Q-CTRL).
Review the research preprint on arXiv (Quantum Error Management Benchmark) here and explore cloud primitive specifications at IBM Qiskit Runtime here and here.
In GQI Portal
The players behind the news
The team that writes QCR tracks every company, deal and technology in the GQI Factory, GQI's verified database of the quantum industry. Next up: every story linked to its players, coming to QCR's paid plans.
- Players Companies and institutions across the quantum industry, by segment.
- Scorecards How the players compare on hardware, software, funding and more.
Newsletter
QCR Alerts in your inbox
The latest reporting and analysis from Quantum Computing Report. Free, unsubscribe any time.