AI is helping write an opera for a N.J. university and … it’s pretty odd

The first thing to know about using artificial intelligence to help compose an opera is that the technology has its limits — at least for now.

“ChatGPT is a horrible lyricist,” said Christopher Herbert, a music professor at William Paterson University.

Undaunted, Herbert and other professors at the public university in Wayne are working with students and AI to help write a three-act opera featuring a chatbot as a key character.

Students enrolled in William Paterson University’s Opera Workshop usually spend a semester staging traditional operas by Mozart, Puccini or other masters. This year, they are working with AI to develop and write five or six scenes of a new opera.

Half of the class is writing lyrics using ChatGPT, a chatbot that uses AI launched in 2022. Meanwhile, the other half of the class is composing music using a range of online music generator programs.

Herbert’s initial experiments with using ChatGPT to write opera didn’t go well last summer, he said.

“Trying to get it to generate something that sounds good to sing, it just took forever. The thread was kind of comical. Finally, I landed on something I liked,” Herbert said, adding that he sent the piece over to another professor for polishing via an electronic music generator.

The AI-generated opera the students are working on is set in 2034. It centers on two friends grappling with access and affordability while pursing their studies in higher education, according to a description provided by William Paterson.

Viviana Kaszubski plays Ariana, a student who works 30 hours per week while taking a full schedule of classes. Ariana is in the midst of losing a cherished friendship and has begun confiding in “Pickles,” an AI chatbot that she now trusts more than people.

She sings “Ariana’s Lament,” composed by two students, Jason Braun and Chelsea McBride, with the help of AI and online music generators.

The students in William Paterson’s open workshop course say they have embraced the AI-inspired concept. They participated in a preliminary performance of the opera on Dec. 5 at the Shea Recital Hall on campus. They expect to finish the opera during the spring semester.

Sofia Lopez-Rodriguez, one of the student composers, offered insights into the process of using AI and music generators while speaking on stage during the Dec. 5 performance.

“I used AI Sound Raw to create the two different melodies that you’re going to be seeing. Then, I asked ChatGBT on the instruments that I should use, which is why the trombone is going to be played,” said Lopez-Rodriguez, who drew laughter from the audience after volunteering a blunt assessment of the questionable result.

“Sometimes, AI is not too creative, at all, as you’ll soon hear,” Lopez-Rodriguez said.

William Paterson’s AI-infused opera is among the examples of how the technology is being used in New Jersey as officials explore the scope of its potential advantages, as well as possible downsides.

The state is setting up a unit for AI research and development at Princeton University, Gov. Phil Murphy and the university’s president announced last month.

New Jersey’s Artificial Intelligence Task Force, established in October, will examine how New Jersey may “continue to foster an environment for innovation while protecting individual and civil rights,” Murphy said at the time.

Payton MacDonald, professor and chair of William Paterson’s music department, said AI’s ability to generate music varies by genre.

“They’re reasonably quite good at producing music that works on a grid. So, especially EDM — electronic dance music — they’re quite good at that,” MacDonald said.

“They struggle with jazz, they struggle with classical music, they struggle with anything that’s off of a grid. Which is understandable, because I think the zeros and ones move around more easily when you’re dealing with grids,” MacDonald said.

With the AI revolution still in its very early stages, what might developing an opera look like in a decade?

“What all the CEOs are saying is that they’re only a year or two into it. Part of it depends on where the tech industry puts its money. It may not be worth their while in investing a lot of money into AI music generators. But maybe there’s enough that, in 10 years, you’re going to see some pretty extraordinary classical music coming out of AI generators,” MacDonald said.

MacDonald, Herbert, and a third music professor, David Weisberg, are working with the students in developing and writing music for their AI-inspired opera.

As part of their coursework in the fall semester, the students also participated in a master class with Timothy Long, artistic and music director of opera at the Eastman School of Music at the University of Rochester. They also attended a performance of “X: The Life and Times of Malcolm X,” at the Metropolitan Opera in New York.

It’s unclear what AI might mean for the students’ future careers.

“It’s not going to put composers out of business, but it will shift what they do. I would guess film composers will be using it in five years as much as they’re using samples now,” MacDonald said.

“Like drum machines did for percussionists, it will shift the work. It’s very easy right now if you own a ‘mom and pop’ business and you wanted to run a local commercial on the radio, you can very easily get a perfectly fine, passable track on an AI music generator as background for a 30-second spot that costs you $5. You don’t need to spend $500 hiring a composer and waiting three weeks to get the track,” MacDonald said.

Herbert said the human touch, no matter what, will remain essential even as AI improves to a degree that might be unimaginable today.

“I find, currently, all of the bots are pretty limited in terms of what they are able to do creatively, because they essentially are just recreative. You can use it currently to get the germ of an idea, but it is really on the human to spit out that idea and make it something fascinating or valuable,” Herbert said.

“In five years, you’re probably going to have a little bit more nuance to the bots, but I don’t think it’s ever going to replace human creativity. Then, there’s always going to be the challenge of performing the music, no matter what, and that’s always going to be a human endeavor,” Herbert said.

Please subscribe now and support the local journalism you rely on and trust.

Rob Jennings may be reached at [email protected].