My last post presented some data from the movie biz illustrating the minimal correlation between popularity and excellence. At least two hypotheses might explain the finding that, on average, the highest-grossing films don’t garner correspondingly high reviews. Maybe the ticket-buying populace has bad taste. Maybe reviewers are out of touch, attempting to impose their anti-democratic elitist standards on the people. It’s worth observing that, when popularity is measured by sales receipts, democracy becomes indistinguishable from capitalism. Still, suppose you were a screenwriter: wouldn’t you want to have your screenplay turned into a film; wouldn’t you want everybody and his brother to go see it at the theater; wouldn’t you regard box-office success as validation of your excellence?
In 1936 John Maynard Keynes described the stock market as a beauty contest:
Professional investment may be likened to those newspaper competitions in which the competitors have to pick out the six prettiest faces from a hundred photographs, the prize being awarded to the competitor whose choice most nearly corresponds to the average preferences of the competitors as a whole; so that each competitor has to pick, not those faces which he himself finds prettiest, but those which he thinks likeliest to catch the fancy of the other competitors, all of whom are looking at the problem from the same point of view. It is not a case of choosing those which, to the best of one’s judgement, are really the prettiest, nor even those which average opinion genuinely thinks the prettiest. We have reached the third degree where we devote our intelligences to anticipating what average opinion expects the average opinion to be. And there are some, I believe, who practise the fourth, fifth and higher degrees.
The publishing industry is giving us the third degree and then some. Readers want books that appeal to them. Bookstore owners pick and promote books not that they themselves find the prettiest but the ones they predict will be deemed prettiest by the average reader; publishers pick and promote books they predict will appeal to the average bookstore owner; agents pick and promote books they predict will appeal to the average publisher. It’s a series of nested mind-reading exercises four levels deep, constructed simulations of simulations of simulations of readers. Writers are operating at fifth-degree simulation, crafting books that they predict will appeal to the average agent. Once I came across a thread on an online message board in which writers were role-playing agents, critiquing book proposals submitted by other writers. Writers trying to predict what will appeal not to real agents but to simulated ones – sixth degree losers.
Fiction writers are inveterate simulators. That’s what a short story or a novel is: a simulated reality. Fictional simulations vary in the degree to which they correspond to actually existing realities, and there is no reason to regard realistic fictions as intrinsically better or truer than unrealistic ones. Similarly, it seems unwarranted to dismiss the sixth-degree simulators as dissimulators. One could argue that, in attempting to survive and thrive in the fictional multiverse, the sixth-degree simulators are being more realistic than their purportedly more authentic first-degree counterparts who don’t go out of their way to be liked.
The agents, the publishers, the retailers – they don’t pick the winner in the beauty pageant. They don’t have to. Instead, they pick all of the contestants. There will always be winners; there will always be losers. The middlemen are there to ensure that the pageant wins. The pageant wins when the aggregate of all the contestants taken together – the portfolio – makes a profit for the middlemen who stage the show. Middlemen enhance the likelihood of running a successful pageant by selecting individual contestants who, based on their features and on the tastes of the audience, stand a fair chance of winning. Beauty counts, but so do charisma and a track record in other pageants and a compelling back story and the endorsements of other winners. It is possible that, in staging these competitions, the middlemen are creating environmental conditions that reward beauty, thus increasing the overall level of beauty across all contestants and across each new generation of contests. Another way to assure a successful pageant is for the middlemen actively to shape the tastes of the audience, tastes that are easier to satisfy than more discriminating, more adventurous standards, making the audience more likely to be attracted to all of the contestants paraded in front of them. Contestants don’t become progressively more beautiful; they become more similar to one another.
The self-publisher cuts out some of the middlemen, making an appeal directly to the audience. But now, instead of being part of a portfolio that wins in the aggregate, the self-publisher has to win the pageant as an individual contestant. And without the widespread distribution and publicity apparatus that the intermediaries use to light up the stage and to pull in the crowds, the size of the win is diminished. Ultimately the pageant isn’t changed all that much with self-publishing. There are individual winners and losers, but the real winners are the aggregators, most notably the e-publishing platforms that host the pageants. The stakes are lower for each contest, but the aggregator makes it up in volume…
While writers might disdain the crass commercialism and institutional hubris of the industry, the fact remains that the agents and publishers do impose an accepted standard of excellence on the enormous quantity of writings that come across their computer screens. I don’t know about other writers, but what I would most value from being represented by an agent isn’t the possibility of earning some money, or even of gaining access to readers – being allowed to strut the runway in the pageant. What I want is an imprimatur: the affirmation from a recognized emissary of the fictional multiverse that my writing is worthy. Novels that make it through the pipeline might in the aggregate be predictable, but they will honed to a professional sheen at every level: writerly craft, line editing, copy editing, formatting, packaging. Polished and groomed, each of these professional novels will have been judged beautiful enough to be entered into the pageant. Self-publishers, answerable to no one but themselves, can take risks to be sure, cultivating idiosyncratic standards of beauty rarely seen among contestants in the staged pageants. But the self-publishers can also put out books that are pretty ugly. It’s not that self-published books are necessarily of lower quality than books produced by the industry, although I’m guessing that the average industry book probably is better than the average self-published book. It’s that the range of variability is so much wider across the self-published books. As a self-published writer I can have confidence that I’m putting good stuff out there, but maybe I’m deluding myself, overestimating my competence. It would be nice to have the external validation – in short, to be liked as a writer by someone known for having good taste and high standards…
In Keynes’s capitalistic beauty pageant it’s the professional investors – the stock analysts and floor traders and stockbrokers – who simulate the marketplace’s tastes in stocks and bonds. But the simulators can themselves be simulated. Algorithms and self-learning AIs, capable of building more accurate simulations and running them much faster than the human professionals, now account for the majority of trading on the big exchanges. Meanwhile the ranks of the Wall Street pros have thinned considerably, having been rendered redundant by the robots who, instead of commanding seven-figure compensation packages, do a better job for free. If stock-picking can be automated, why not novel-picking? Online retailers, their search engines rapidly working through the enormous dataset on purchasing patterns at their disposal, can already steer you to individually customized recommendations for what you might want to read next. Why not go even further upstream, using data about manuscripts and their writers to build simulations that predict which books are most likely to be best sellers in various market niches? It’s already being done with pop music and television; surely it can be done with novels.
And in fact it already is being done. Here comes The Bestseller Code, a 2016 release cowritten by Jodie Archer, a former acquisitions specialist at a major publishing house, and Matthew Jockers, an English professor who specializes in digital humanities. Applying the latest AI and machine learning technologies to data extracted from thousands of novels, the authors built a “bestseller-ometer,” a complex algorithm capable of predicting with high reliability the statistical likelihood that a novel will go on to become a bestseller. Soon the professional literary talent spotters will be joining the ranks of the unemployed stock market professionals, the lawyers and accountants and underwriters, the middle managers and clerical workers and technicians, the retail workers and factory workers and farmers and miners, all of them outperformed by robots doing the work at a fraction of the cost…