Bestseller charts are not a recent invention. The New York Times began publishing a weekly chart in the 1930s, the Sunday Times published its first book chart in 1974. The Bookseller’s first weekly chart, Booksellers’ Bestsellers, was introduced on 1st October 1977, a collaboration with the Sunday Times. Inevitably it will differ from publishers’ own records, the magazine sensibly warned at the time (the chart did not include book clubs). The first chart-toppers of that era—J R R Tolkien’s The Silmarillion and Jack Higgins’ Storm Warning—were an indication of how important the term "bestseller" would become and how it could be used to make an author visible.
The Bookseller switched to using BookScan (neé BookTrack and founded by the magazine’s then-parent Whitaker) data in its final issue of 1996, when Longitude and Notes from a Small Island sat at the top of the new chart. At the time Philippa Harrison, c.e.o. of Little, Brown, said she hoped the new charts would "continue informing the public about the books that are selling best at any particular moment in a way which makes sense to them and us". Twenty years on, it seems to me that the industry is falling short of that.
In many ways we have, of course, unparalleled access to data about the market—thanks principally to Nielsen BookScan (which, under Nielsen, has expanded in the UK and internationally) but also via the Publishers Association’s Statistics Yearbook, and those publishers who supply to this magazine each week data about their e-book hits. But just as there was a missing tranche in 1977, so there is today. Earlier this year, the Society of Authors (SoA) raised the issue of "special sales", asking publishers to record such deals alongside an author’s track-record, and also encourage BookScan to measure these, as it does traditional sales. There is also the continued absence of e-book and audiobook data, not released by Amazon or Audible, and therefore not used in the official charts we (and others) publish. No-one pretends otherwise, of course.
Back in the 1990s, Harrison placed the onus on the data supplier to capture as much of the market as was sensible, and both Nielsen and The Bookseller have worked to get around the problem using publisher e-book numbers. But really this is an industry-wide dilemma. As the SoA makes clear, the missing numbers can and do damage an author’s career, as well as the public’s perception of them. They also damage the sector: we look narrower than we are.
The worst of it is this curious delusion that appears to have overtaken the trade, that these missing markets somehow don’t count; and that the authors and books that do well through these routes are somehow second-class.
Today’s bestsellers deserve better—all of them.