The metric Quantum Volume (QV) has been in the news lately and it deserves a few more words of explanation. This metric was first created by IBM in 2017 and modified in 2018 as a metric that would allow comparison of different quantum computers. It has many valuable characteristics. It is a well-documented, straightforward way of comparing different quantum computers that can be run on any gate level quantum computer, not just ones that use a superconducting technology. Most important, it considers qubit count, qubit quality and other factors so it does not emphasize the number of qubits alone. It is a good tool for use by quantum hardware engineers to measure their progress in development. If they are able to come up with a new generation of hardware that increases this metric, they are going in the right direction. However, it is a poor tool for end users to use if they want to measure the goodness of a quantum computer for solving their computation problems.

We will explain why Quantum Volume is not really appropriate
for end users in this brief. But first,
we will provide a simplified explanation of how this measure is derived. The quantum volume measurement involves
testing a series of circuits with a square configuration. By that, we mean that the number of qubits
equals the gate depth of the circuit.
For example, a test may begin by trying out a 2×2 configuration. This would mean a 2 qubit circuit with each
gate going through 2 gate levels. In
these tests, the gate sequences are alternating sequences of single qubit gates
followed by two qubit gates. The circuit
is run multiple times with the results measured and the results are analyzed
with a certain statistical test to see how accurate the answer is to what would
be the theoretical result. If the test
passes a certain criteria, then the test is repeated for a 3×3 configuration, a
4×4 configuration, etc. Because qubits
are imperfect, each increased level gets harder and harder because the gate
errors start stacking up and the test no longer passes the criteria. Once the
largest square array is found that will pass the test, the Quantum Volume is
calculated as 2^{n} where n is the number of qubits and gate depth. So a 4×4 circuit would have a QV of 16, a 5×5
circuit would have a QV of 32, a 6×6 circuit would have a QV of 64, etc. For
complete details on how this test is conducted, we refer you to the paper that IBM
posted on arXiv.

Here are our concerns about Quantum Volume:

- The test is all based upon a square circuit configuration, but very few quantum programs really have a square configuration. Some of the algorithms being developed for NISQ computers such as VQE and QAOA, are wide and shallow. This means that use a larger number of qubits but only a few levels of gate depth. Others, may have a much larger number of gate operations versus the numbers of qubits. For example, Shor’s algorithm can theoretically factor a 2048 bit number using about 4100 logical qubits, but it requires about 8.6×10
^{9}gate operations. (Note that is based upon logical or error-free qubits and not physical qubits). - We do not agree with calculating the quantum volume by using a formula of 2
^{n}. We think this gives a distorted view of how fast the quantum computer performance is increasing. Let us explain this by an analogy. Let’s suppose we were looking for office space and the landlord shows us two offices spaces with dimensions 5×5 or 6×6. (You can use either feet or meters as the dimension depending upon what country you are in.) Would you expect to pay twice as much for the 6×6 office as the 5×5 office? No! You could calculate the worth by looking at the square area and determine that the 6×6 is about 44% more valuable (36/25) than the 5×5, not 100%. In a quantum computing algorithm, we do not think an end user would be able to increase their problem size by 100% if they were provided a new quantum computer that had just one more qubit and one more gate level. - The focus for anyone developing a quantum computer should be how to make it achieve quantum advantage and solve problems better than a classical computer. Since classical computers are error free, the equivalent quantum volume for a quantum program running on a quantum simulator on a classical computer can be very high. For example, in 2019, Google ran a quantum benchmark on the Summit supercomputer at the Oak Ridge National Lab that successfully calculated the results of a 49×40 circuit. So the equivalent QV for Summit would be 2
^{40}or about 1.1 x 10^{12}.

So for anyone claiming to have the world’s highest performance quantum computer, a high Quantum Volume figure is helpful, but we would not regard it as definitive proof. Perhaps a more appropriate challenge would be to replicate or beat Google’s Quantum Supremacy experiment. At this point in time, anyone who can achieve that would indeed have something noteworthy.

March 5, 2020

Paul NationApril 4, 2020 at 4:16 am1) Quantum Volume (QV) is designed to quantify performance for an “average” quantum circuit. This is why it is square and random. If you look at extreme limits either way, they do not capture the progress you want to track in a universal quantum computer. The example of wide but shallow circuits is not ideal because it does not adequately track qubit fidelity improvements. Circuits that are deep, but contain few qubits, can be classically simulated. QV takes the middle road in demanding that both the search space and time allowed to search it (determined by inverse error rates) are both increasing at the same time.

2) QV measures the largest quantum computational space a device can explore by a number of layers of random two-qubit unitaries that is equal to the number of qubits used in the circuit. Since adding another qubit does indeed double the quantum computational (Hilbert) space, this scaling is sound.

3) This is of course one of many goals, but it does not track performance improvements. Following this logic, we cannot benchmark quantum computers until they have beaten a classical computer at some task. Additionally, doing well at one specific task need not equate to doing well at another. Highlighted here is an experiment with no practical purpose, and one from which the authors make no claim as to the usefulness of their device in practical computations of interest. The claim that running a non-QV circuit on a classical machine defines the QV of that devices also makes little sense. Given that Google claims to have outperformed Summit, this statement is claiming that Google’s Sycamore device has a QV of at least 2^40. I don’t think the community is holding its breath waiting for the paper claiming this to be true.

Doug FinkeApril 6, 2020 at 10:59 amThank you for your comments Paul. As we said at the beginning of the article, Quantum Volume is a useful metric. However, describing the performance of a quantum computer can get quite complicated. So we don’t want to leave the impression that this is the only performance metric that needs to be used. It should be supplemented by others that may provide some additional insights that would be helpful in other situations.

Doug Finke

Managing Editor