Yuval Boger interviews Professor Eduardo Miranda and Brian Siegelwax, authors of “Teaching Qubits to Sing: Mission Impossible?”, a fascinating work about creating music with quantum computers. Eduardo, Brian and Yuval talk about how the program works, their plans for futue work, finishing Schubert’s Unfinished Symphony and much more.

Transcript

Yuval Boger: Hello, professor Miranda. Hello, Brian. Thanks for joining me today.

Brian Siegelwax: Thank you.

Eduardo Miranda:Hello.

Yuval: So Eduardo, who are you and what do you do?

Eduardo: I am Eduardo Miranda. I am a professor of computer music at the University of Plymouth. And currently I am developing research into using quantum computers to create music.

Yuval: And Brian, who are you? And what do you do?

Brian: I am Brian Siegelwax. I write code, I write about code, and I also recruit people who write code.

Yuval: Fantastic. And we are here to talk about singing qubit, right? The qubit choir.

Eduardo: Okay.

Yuval: Eduardo, you’ve been on the Qubit Guy’s podcast before, and I know you are using quantum computers to compose music. Could one of you or both of you tell me how this singing qubit project came about?

Eduardo: Well, I think Brian and I have been communicating on social media, exchanging some ideas, and so on. I think both of us wanted to develop something together. And then I think the opportunity to develop a generative music system came about. We exchanged some ideas about how we could implement, how we could find a simple way to program a quantum computer to generate sequences of things. Brian can probably tell you a little bit more about how the technical idea came up, but the inspiration I had was, well, if you can generate sequences of musical notes, what would be an interesting instrument to play the sequences? And at the time, coincidentally, I was working with a vocal synthesis, then I said, “Okay, so let’s hook this quantum system to a vocal synthesizer and let’s make the qubit sing.

Yuval: So Brian, if I understand correctly, there’s a vocal synthesizer, which means that I could put in any file or collection of notes, and it would generate the singing sound. And the quantum part is the part that composes the sequence of the notes. Is that correct?

Brian: Correct. So we started with a sample file, and we used that to generate the rules and then the rules get encoded into the amplitudes of the qubits. And then the qubits decide what the sequence will be.

Yuval: So let me get this straight. So if I have do, re, mi, fa, so, la, si. Usually, the next note would be a do, right? Or in English A, B, C, D, E, F, G and then A. So that sounds like a probabilistic state machine, right?

Eduardo: Correct.

Yuval: You can say, well, if this is the sequence, then next note with high probability or with some probability is this and that and so on.

Eduardo: That’s it.

Yuval: How is that different than the quantum generation?

Eduardo: Okay. So the system has two phases, what we call a learning phase and a generative phase. The learning phase is pretty classical, so to speak. What we do, we input music into the system. Then we statistically measure how many times not X follows, not Y, or how many times three notes follow the other four notes, things like that. So we build a set of rules. Then we put these rules into what would be the equivalent of transition metrics. Okay. So that is the learning phase. So we can input any musical sequences to the system. The system will work out what these rules are.

Then the classical way of doing this would be to program the machine to go through this transition table and pick notes that would follow certain probabilistic Gaussian or whatever tendencies. But we found an easier way to do that, which is use the quantum machine to build quantum states with the amplitudes reflecting those probabilities. So it is a much more, in my view, elegant way. And I think we can scale this up in the future much easier than if we were going to continue doing this classically.

Yuval: So Brian, maybe you could talk to me a little bit about what’s going on inside. How does the quantum circuit look? How do you prepare the state? What are you measuring on the output?

Brian: Yeah. The interesting thing is there’s no one circuit to show. They’re all generated on the fly. So let’s say we want to use three notes to generate the next note. And let’s say we have ABC, and we go through this original tune, and we’re counting how many times ABC appears. So let’s say, for argument’s sake, ABC appears 10 times and seven times the next note in the sequence is D, and three times the next note in the sequence is E, and they’re all binary coded. So let’s say 000 is A, 001 is B and so forth. So what we have is, and from your standpoint, Yuval, it’s a state preparation function. So we want 0% probability of 000 (A), zero for B, zero for C, then 70% for D, 30% for E, zero for the others.

So you have eight probabilities, except that we used Qiskit’s initialize() function, which requires us to convert that into amplitude, so we have to take the square root of the probabilities. We have to take one extra step, and then the initialized function generates the circuit on the fly. So it’ll generate a circuit in this case that when we measure it, we’ll have a 70% probability of measuring 0110. I’m ahead of myself. What’s D? 10, 11, whatever it is. But 011 I think, which is D and then 100, or whatever E is. And then let’s say we measure D. I should be able to do this in my head faster, but anyway. So then we have B, C, D as the sequence.

Then we look back in the transition matrix, and we have another round of amplitudes, and the initialized function generates a completely different circuit that, when measured, produces the output and the probabilities that not that we want, but the probabilities that came from that original file. So you end up with a tune that sounds somewhat like the original, but depending on how many notes we use, we can alter how much it sounds like the original, how much it sounds different, and how much variety there is. But the core piece, the quantum piece, is generating those circuits that get measured in the probabilities for the allowed next notes.

Yuval: And what type of music did you use as input? Was it Bach, was it the Beatles? Beethoven? What was going on there?

Eduardo: We use a number of things. We use classical music melodies like Bach, Mozart, whatever. And then it so happened that I had a few media files for film music. And so we had this quirk idea. Let’s use for the paper the tune from the Mission Impossible film. Is it possible to program qubit to sing? Is it a mission impossible? No, it is not. It is possible. The tunes that are mentioned in the paper are from the Mission Impossible theme. And we also use a Bach cello sequence to show that it is generic what we do.

Yuval: Was there a particular type of music that worked better, that was more deterministic, or that was easier to follow?

Eduardo: No, I think because we are dealing with probabilities, a stochastic model, those inputs that are more repetitive will tend to generate music that is more similar to the input, but those pieces of music where the notes vary more, there are no obvious and no beat or a pattern that you identify quite immediately. For me, as a composer, generate more interesting results. What we are doing now is trying to go beyond only generating sequences of notes, because I think now we have the beauty of this core quantum algorithm that generates sequences.

What I’m going to do now is to provide Brian with codes that would represent other things in the composition. For example, dynamics, speed, and maybe more voices. And because we are dealing with speech synthesis, there are things in speech that are quite interesting like intonation, prosody, things that relate quite well with speech research. So I think what we are doing can have a nice connection with quantum natural language processing, for example. I know these works are very symbolic. They are looking just at written words, but I think we have a potential here to add the synthesis part, the actual rendering of the speech, which will then give another dimension to language, which is the inflection of the speech, the emotions, and things like that. So I think this is where music can feed in quite interesting in scientific research.

Yuval: You mentioned speed and dynamics, but the current circuit generates the melody.

Eduardo: Correct.

Yuval: Are you working on generating the harmony as well?

Eduardo: Yes, we are doing that. So, of course, the encoding of these materials will explode the dimensions of what we are dealing with. So at the moment, we have two dimensions, which is pitch and the duration of the note. But if we begin to add all the elements, like what instrument plays this? What’s the loudness of this note? How do we actually play it? Is it a plucked string? Is it a bowed string? Is it a consonant? Is it a vowel? So once you begin to add these other dimensions, then it’s here where I think the quantum computation will become quite powerful.

Yuval: I assume this still needs, or for the foreseeable future, this is still going to need a lot of human supervision. So, for instance, how would the quantum computer know when the piece ends and how to end it? Usually, I mean, you might end it on a dominant note or something else. How would that work, for instance?

Eduardo: I think it’s possible to automate this as well. I think it is, what we say, the cherry on the cake, so to speak. After we’ve got to the core thing working, then I believe that the details can be honed and further incorporated into the formalism. So these are things that I need to liaise very close with Brian so that we keep the beauty of the model, but use what we’ve got and try to scale it up to more number of qubits and so on.

Yuval: There was a book that was quite popular several years ago called Sleeping with Schubert. And it was about this lady who became obsessed. Something got into her, and she started thinking like Schubert. And, of course, she also finished the Unfinished Symphony. How far in the future do you think that you’ll be able to automatically finish the Unfinished?

Eduardo: We more or less know how to do this because there are examples of composers that programed classic AI to do these other things. So we’ve got the Mahler symphony finished. We’ve got the Schubert symphony finished. We have a Beethoven symphony finished. Of course, these outputs are not, I would say, very realistic in the sense that it’s easy to spot that was done by a machine, but they know how to do this. I think it’s already out there. What we need to do is to perfect this. And I believe that because I am so enthusiastic about the quantum computing world, I think it is new ways of thinking about programming AI will lead to these things that you just mentioned. So maybe the dream may be quite realistic, I think, pretty soon.

Yuval: Brian, from a quantum perspective, what did you learn doing this project? Obviously, I’m guessing you learned a lot about music and about composition, but how did you become a better quantum circuit designer with this project?

Brian: I’ll have to go back about two years. Well, I like this story. I was reading a paper a day off of arXiv. I was getting most of the suggestions from Twitter. Somehow I discovered a paper from professor Miranda called Quantum Computer: Hello, Music! And it blew my mind, and we’ve worked together, and it still blows my mind. But the key difference with quantum music, and I would compare it to quantum gaming where you generate some random numbers and you can use it during the game. You can generate some random numbers, and you can generate terrain. But the interesting thing about music is it has to sound good, and that’s very subjective, and I have no musical talent whatsoever. So when we’re playing back these tunes and sometimes it’s catchy, it sounds interesting, the original’s in there, but there’s that little bit of variety that sounds good. It just blew my mind.

And as we were working on it and still as we chat now, I think about all the other possibilities because we started off very simple because professor Miranda said about getting the core working. So we started very simply with that. And then, we switched to notes and pitches, and it was a seamless and easy transition. So it blows my mind further what we can do next with it, and what’s the potential. And I don’t know where I found his paper from. I’m thrilled I found it. I put it in my first book, Dungeons & Qubits.

I had no real expectation that this much adventure would come out of it, but I’d say that’s the key thing I learned. Not only is it possible to create something that’s subjective and good, but we haven’t even started yet. There’s so much more that … I’m waiting for more to come out of professor Miranda’s mind just so I can work with it and see what it can do and then have him play it back and just blow my mind some more.

Yuval: Would a larger quantum computer, better quantum computer help you, or are you perfectly fine with just a handful of qubits?

Brian: Well, a handful of qubits are okay at this point because we worked out a compression system. So when we switched to notes and pitches, it was a nine-bit binary string, but we don’t need nine qubits. We were able to compress that, I think, down to four qubits, five qubits. The advantage of better quality qubits, though, would be in the error handling. So if we have ABC, we can generate D or E. The problem is if noise makes us generate an F, we don’t have a BCF in the transition matrix, and that’s a problem. So there was a lot of error handling that we had to throw at it.

So you’ve got the simulator that is pretty precise, but it’s not as interesting. Then you’ve got real qubits singing far better than I can possibly do, but you get that BCF. We just can’t work with that. I’d really prefer qubits that, if we want 70% and 30%, they’re at least pretty close with some weird quantum fluctuations, some quantum variety, but it still has to follow the rules, or you get error messages. So better qubits are better, but then it’s exponential. So five qubits give us 32 different combinations. So each additional qubit doubles that. So you would need a very long complex tune to really run out of or to need additional qubits.

Yuval: And how long does the process take? So if you take a Bach cello Sonata, take it through the learning phase. How long does it take to start generating music from that?

Brian: Well, the learning phases with the size of the pieces that we use was instantaneous practically. Then the slow part was running on real quantum hardware, but as soon as we were done with it, I started thinking of new ideas of how we can speed that up. So I’m really looking forward to the next round and just overhauling pieces of it like that. We can speed up the quantum part, but most of the classical parts are practically instantaneous.

Eduardo: It’s pretty quick. It depends on what we do with it. For example, I am interested in real-time interaction. Right? So real-time is not possible yet because of the connection to the cloud and blah, blah, blah, blah. But in a hypothetical world, what I wanted to do is to be playing with my instrument and open a window for the quantum machine to listen to what I’m playing and then play responses back to me or accompaniments and so on. So I would like to jam with a quantum machine. So for this purpose, what we’ve got is pitch instantaneous. We would do that in a sec. It is real-time.

Now, the other vein of research would be to train the system with large amounts of information. Let’s say take the record of John Coltrane, put it in the system, and see if the system can generate melodies that are in the style of John Coltrane, for example. So these would then depend on how many files you use to train the system and so on and so forth. But I would say that what we are doing is much faster than, let’s say, in the deep neural networks and things like that.

That’s the beauty of what we’ve got. It’s so neat. It’s so simple and so smart. It’s almost unbelievable when we came up with it. I think Brian was also very surprised that such a simple elegant mode of encoding these transitions in the amplitudes of the quantum states. It’s almost as natural as it can be.

Yuval: Listening to you guys, it sounds like this is just a start, and there’s lots more work and additional ideas that are going on, and hopefully, we’ll be able to speak again in the future with the qubit choir or the qubit orchestra or the qubit band.

Eduardo: Yeah, most definitely. So watch this space.

Yuval: How can people get in touch with both of you to learn more about your work?

Eduardo: The best thing to do is to go to arXiv and have a look at the paper we posted up there. It is called Teaching Qubits to Sing: Mission Impossible? And our emails are there on the paper. So just email us, and we’ll be happy to interact with people there. And most importantly, I think suggestions and ideas, things that we spot in the paper that we had not thought of before, that would be very welcome.

Yuval: Excellent. So Eduardo, Brian, thank you so much for joining me today.

Eduardo: It’s a pleasure.

Yuval Boger is a quantum computing executive. Known as the “Superposition Guy” as well as the original “Qubit Guy,” he most recently served as Chief Marketing Officer for Classiq. He can be reached on LinkedIn or at this email

September 14, 2022