Inside the headquarters of Dolby Laboratories in San Francisco (USA), some people sit down to watch cinema. But instead of a soda and a bucket of popcorn, in their hands they carry biosensors that detect their heart rate or the electrical resistance of their skin. On their head they sport a cap with multiple electrodes that record their brain activity, while a thermal camera records their body temperature. All this takes place while they relax to enjoy a movie or a television series. The objective of this research by Dolby, as published by The Verge, is to understand the human experience in order to build intelligent technologies capable of provoking emotional responses in what we see or hear.
Dolby’s experiments are an example of how current technologies based on artificial intelligence (AI) are trying to unravel what it is that a blockbuster movie, a musical hit or a best-seller have that allow them to engage millions of people around the world, and how this knowledge could be applied through algorithms towards the creation of works that are sure-fire successes.
Naturally, to create works that we enjoy, the first essential step is to discover what the qualities are of the things that we like. If the answer is that it’s all about emotions, few are as visible as those that make us sing along loudly when we listen to a favourite song. In 2011, two British researchers embarked on a tour of pubs and nightclubs in the north of England to discover which songs were being sung to most frequently. We Are the Champions by Queen and Y.M.C.A. from The Village People topped the list.
Through the use of data mining technologies that analysed more than 1,100 examples, the two researchers identified certain common features in these songs. The study’s director, music psychologist Daniel Müllensiefen from Goldsmiths, University of London, explains to OpenMind that the study “points to the importance of the performer rather than the composer.” In subsequent works, Müllensiefen and his collaborators analysed the musical characteristics of catchy songs and commercial success. But the hits are still reluctant to reveal their secrets: “Even our most sophisticated models can explain little of the variation in terms of commercial success,” says Müllensiefen. “So either compositional features aren’t that important for songs to be commercially successful or we are not using the right features yet.”
In search of the key to success
These common features that differentiate the triumphant works from the rest are the holy grail sought by publishers and producers. They were also the goal of Tijl De Bie, a professor at the University of Ghent (Belgium) and an expert in AI, when in 2011 he analysed all the singles from the top 40 list of the United Kingdom for the last half century in search of the key to success. Using the results, De Bie and his collaborators created an algorithm capable of predicting, from 23 musical traits, the success potential of a song with an accuracy rate of 60%.
De Bie has confidence that discovering the secret of a hit is a goal that will become increasingly affordable with more data and more advanced computer systems. However, at the same time he tells OpenMind that public reception always interposes a factor of uncertainty. “The diversity of music tastes is so large that there is no good single measure of hit potential. What is a hit for you may not be a hit for me.”
Dissecting the keys to a song’s success is also the goal of Canadian music technology company HITLAB. Its AI tool called Music Digital Nuance Analysis (DNA) analyses 84 mathematical parameters to identify new songs that share traits with past hits, with the aim of predicting new hits even in specific genres, languages or regions of the world. Another company working along the same lines, Musiio, has been acquired by the music streaming platform SoundCloud; it would seem that the music industry is betting on the future of these technologies.
Style patterns for successful literature
Another field in which researchers are pursuing the profile of a successful work is the literary one. What separates the best seller on display in the shop window from hundreds of other volumes on the shelves of the bookstore is something that publishers themselves still recognize as the greatest of mysteries. But also in the world of print, modern technology can get closer to the magical formula for success than the most experienced of publishers.
When it comes to backing a particular work, publishers play with the mystery of knowing what kind of stories interest the public. But according to three researchers from Stony Brook University in New York (USA), this does not matter. In 2014, they introduced about 800 books and movie scripts into their machines in search of style patterns, regardless of the plot. “The key findings of our research reveal that there exists distinct linguistic patterns shared among successful literature, at least within the same genre, making it possible to build a model with surprisingly high accuracy (up to 84%) in predicting the success of a novel,” they wrote in their study. Among these patterns, the winning books used more verbs of thought than of action, preferred nouns, adjectives and conjunctions over verbs and, curiously, shied away from clichés like the word “love.”
Among other recent initiatives to formulate the mystery of the novel into mathematics, it is worth highlighting The Bestseller Code: Anatomy of the Blockbuster Novel (St. Martin’s Griffin, 2016). Jodie Archer and Matthew L. Jockers, experts in literature and in the emerging field of the digital humanities, have designed an algorithm that has scrutinized 20,000 novels published over the last 30 years, including not only language patterns but also plots and characters. Their bestseller-o-meter, as they call it, can predict with a minimum of 80% success if a manuscript will climb to the best-seller list of The New York Times.
Works composed by IA
With all these new tools, it is plausible that the creators of an already-existing future will begin to rely not only on their own inspiration, but also on the auguries of these digital crystal balls when it comes to producing their works. For De Bie, “I think it is likely that AI is used as an assistant to composers, an extra tool that can help them create material for a new composition. Such a tool could, of course, be guided by what is more likely to become a hit.”
But one step further would be for the machines themselves to create the songs we listen to, the content we watch or the books we read. “This is already starting to happen today,” says De Bie. In 2012, the project of the University of Malaga (Spain) created with Iamus—the first computer that has learned human musical language—the first great musical work composed entirely by an AI system without imitating human composers. In 2016, the musical Beyond the Fence was released in London, also the product of a machine. The Flow Machines project from the Computer Science Laboratory of Sony in Paris, released two pop songs generated by AI in 2016 and has continued later with new compositions. Algorithms are also taking their first steps in the field of the novel and in that of screenplays.
The next step will be for machines not only to produce the works, but also to apply their knowledge of the anatomy of a hit. “As far as I am aware, the AI techniques used here are not immediately trying to optimise hit potential, but it is conceivable that, one day soon, they will,” says De Bie. The expert does not believe that this phenomenon will kill traditional creation; another question is whether humans will be able to compete commercially with machines. “As soon as AI composition systems exceed humans in speed and quality, the chances are that human composers will continue doing this for fun, but will be unable to do it for commercial gain,” De Bie predicts. “It may become like old crafts that have been replaced by machines. Sewing is not commercially viable in the developed world, but people still do it for personal satisfaction.”
For the time being, far from rejecting the intrusion of AI into music creation, more and more musicians are incorporating these tools as an exploration of compositional possibilities. There are already companies like Audoir that offer composition tools based on AI systems trained to help creators find that magic key that captures the audience’s taste. And even apps like Boomy, which can create a new song to the user’s taste in just 30 seconds, even if the user has never composed music and has no specialised knowledge. Another question, as complicated as it is subjective, is whether all this can really be considered art.