When Bill O'Reilly and Al Franken were arguing last month about who had sold more books, online muckraker Matt Drudge stepped in with the final word by citing an inviolable source: Nielsen Bookscan.

The point-of-sale data service, owned by Dutch conglomerate VNU (a competitor of PW parent Reed Elsevier) is on its way to becoming a standard-bearer in an industry starved for hard data. In the space of a few years, it has gone from a sputtering pipe-dream to a recognized tool, one used in everything from settling a celebrity debate to setting the price of a rights sale. Publishers have embraced the system, with Penguin becoming one of the last of the large houses to sign up.

But while the media and the industry increasingly cite Bookscan, questions have cropped up about how representative it truly is—and how it might be misused. Agents in particular have been ambivalent, saying editors too often wield the figures as a blunt instrument in negotiations. As publishers become increasingly likely to use Bookscan as a guide, anecdotes abound of agents who play down the importance of Bookscan numbers.

"It's not bad in and of itself," said Trident Media's Robert Gottlieb. "But some editors, especially less experienced ones, over-rely on the system. Editors shouldn't be using it unless they're supervised by other people in the house with other information." For their part, some publishers are concerned that if the consumer media regularly use Bookscan numbers, they will underreport the success of a title.

So how representative are the service's numbers? An informal survey of the top-selling books of 2003 revealed some surprising information. Bookscan generally claims to represent 70%—75% of retail sales in the industry (Wal-Mart, libraries and some of the supermarket chains are among those who don't report). But a comparison with in-print figures supplied by publishers shows that the numbers are more likely to represent about 65%, even after unsold books and returns are deducted.

Nielsen BookScan's Jim King said the service is striving to add more stores. It currently has about 400 independents; it also recently added Follett. Adding drug stores, King said, can be a problem because they don't always scan by ISBN, but he hopes to also add there. He does not expect Wal-Mart and Sam's stores to change their policy of not sharing data with Nielsen. And King said that figuring out how publishers can best use the data could take time. "We've only been online a couple of years," he said. "There's a learning curve."

The range of coverage is clearly shown by three Simon & Schuster books tracked by Bookscan. The service's figures were very close to the numbers reported by S&S for Benjamin Franklin: An American Life; Bookscan reported sales of 384,000, while S&S said that, with a recent printing, the in-print figure is now up to 550,000. Figures for Living History diverged a bit more, with Bookscan showing 1,085,000 copies sold and S&S reporting 1.8 million copies in print. The biggest discrepancy came for The Ultimate Weight Solution, for which 836,000 copies were reported sold by Bookscan while S&S has an in-print total of 2.5 million.

S&S spokesperson Adam Rothberg said Bookscan's numbers "are very accurate for what they cover, but a large portion of the market is not represented in their figures." He noted that among the outlets not captured by Bookscan are Wal-Mart, Sam's and BJ's, outlets that are more likely to do better with The Ultimate Weight Solution than with Benjamin Franklin.

Bookscan captured about half (2.3 million) of the five million copies in print of Rodale's South Beach Diet, a fact that doesn't bother Rodale trade book publisher Amy Rhodes at all. Rhodes explained that she estimates that for a book like South Beach—which will have strong sales in nontraditional outlets—Bookscan will capture about 65% to 70% of all sales. She fills in some holes with numbers from the warehouse clubs and grocery stores to get what she believes is an accurate sell-through picture. She told PW Rodale has shipped 4.6 million copies of the title and estimates that 3.5 million copies have been sold.

Rhodes said she "couldn't have managed South Beach without Bookscan," noting that seeing the solid sell-through numbers gave her the confidence to keep hitting the reprint button aggressively, especially on the 22nd (and most recent) printing, which was for 1.7 million copies.

Rhodes conceded, however, that as well as South Beach has done, it has been outsold this year by The Purpose-Driven Life. Although Bookscan reports that the title has sold only 1.5 million copies, publisher Zondervan said 11.2 million books were sold last year. The "dramatic underreporting" is due, Zondervan president Bruce Ryskamp said, to the strong sales of Purpose-DrivenLife outside the scope of Bookscan's service—warehouse clubs as well as the Christian bookstore and church markets, all of which have been "wildly successful" with the title.

Another book where Bookscan grabbed about 50% of the title's in-print figure is Who's Looking Out for You? According to the service, 430,000 copies were sold, while Doubleday Broadway had an in-print figure—before a sixth printing last week—of 938,436. Publicist Heather McGuire noted the book has done very well at the warehouse clubs.

Most people in the industry say Bookscan is better than a typical bestseller list or another indicator like the Amazon list, but that it remains a far cry from a royalty statement or from being the definitive gauge that its sister Soundscan has become to the music industry. "The more numbers you have, the more likely you are to find the truth between them," said agent William Clark, implying that the only reliable thing about book sales trackers is that none is fully reliable.