Many of the new announcements regarding advancements in quantum computing technology revolve around new milestones about the qubits or the discovery of new algorithms. However, another important area that is also continuing to improve is what GQI calls the MidStack. As can be seen in the picture above, this encompasses portions in the middle of the quantum stack that includes the Framework, Architecture, and Control Logic. (You can download several of the reports from GQI dealing with the MidStack from the GQI Reports page.)
IBM and its partners have just made several announcements that will improve the user experience with Qiskit by providing improved performance, more efficiency, and easier use through additional functions and tools. The first advance that IBM has announced is increased performance with the Qiskit transpiler. It turns out that transpiling an user input quantum program into the most efficient implementation of gates on a quantum processor is a complex optimization problem. Not only do you want the transpiler to create an implementation with the fewest number of levels and the fewest number of gates, but you also want the transpiler to do it in the shortest amount of time. This is all classical processing, but in some cases a transpiler can take hours or even days to complete the job if you want to have the highest level of optimization. As we have previously reported in our article about Qiskit 1.2, IBM has been working to improve the performance of Qiskit by converting internal sections of the Qiskit code from Python to the higher performance and more efficient Rust programming language.
It helps to have a way of measuring progress in this effort, so IBM has also created an open source benchmarking suite called Benchpress that can test and evaluate the performance of different software development kits. This benchmark currently contains 1066 test cases derived from third party libraries that include test cases that use up to 930 qubits and up to roughly a million two-qubit gates. The Benchpress software will test and report on the number of tests that have passed, skipped, or failed to complete the compilations. It will also report on the runtime, two-qubit gate count, and the two-qubit gate depth. IBM has run the Benchpress tests to compare the Qiskit 1.2.0 results versus BQSKit 1.1.2, Braket 1.86.1 , Cirq 1.4.1, Staq 3.5, and TKET 1.31.0.
As shown in the graphs below, Qiskit was able to successfully compile more test cases and do it in a much smaller amount of time that the other SDKs.
With regards to the level of optimization that each SDK’s achieved, TKET came the closest to the performance of Qiskit, although the results did depend upon the topology of the target processor (All-to-All, Square, Heavy-Hex, or Linear) and the individual test case. On average, Qiskit produced 24% fewer two-qubit gates than TKET with the compiled code exhibiting roughly similar gate depths. (See Figure 3 and Table II in the arXiv paper referenced below for the details.)
We will caution our readers that these results represent a snapshot in time. As future versions of these SDKs are released the performance of each SDK will likely improve and the relative results may change. In addition, the results do vary depending upon the particular test case that is run and the specific target machine architecture. But it is helpful to have a new tool in which to benchmark the SDK performance so the performance of new releases can be compared.
Additional information about IBM’s SDK benchmarking exercise and the Benchpress tool can be found in a blog posted on the IBM website here, a technical paper posted on arXiv here, and the Benchpress open source code repository on GitHub here.
September 16, 2024
