The other day I came across a blog that essentially accused the publishing industry of “corruption” and my own publisher, Tor, of “gaming” the system to produce artificial bestsellers.  While there is truth in the fact that Tor and indeed all publishers do their best to release titles in a way that will maximize the possibility that new releases make various “bestseller” lists, the idea [or at least this particular blogger’s idea] that the “big” publishers are somehow actively corrupting publishing is based on some serious misconceptions about bestseller lists, both in how they’re compiled and how well/poorly they reflect sales.

To begin with, the lists don’t represent total sales or anything close to that.  They represent sales in a given period, based on data from various – and often differing – sources.  Publishers Weekly now bases its lists on Nielsen BookScan numbers, which, in general but not always in specific cases, comprise about 60% to 80% of the major retail sales points, depending on which expert one talks to, but that leaves major gaps, such as Walmart, whose numbers aren’t included in BookScan.  The New York Times does not reveal its specific methodology, but has stated that its survey includes 4,000 bookstores as well as representative wholesalers. USA Today surveys some 3,000 retailers.  In the F&SF area, Locus magazine does its own survey of bookstores and publishes its own monthly F&SF listing, which, according to some editors in the field, may be more representative of the genre, depending on one’s definition of fantasy and science fiction.

But there are other factors to consider as well. While the BookScan sales numbers do represent actual sales, all “pre-orders” or sales made before the release date are tallied for bestseller numbers in the first week’s sales.  That means that a book that’s been listed at, say, 5,000 on the Amazon sales ranking for five months may well rack up higher numbers than a book that’s only been available for two months, and shows more pre-orders at any one time, but for a shorter period.  Then there’s the fact that books have differing readerships, and the audience buying patterns reflect such differences, so that a book that never makes the “bestseller” lists may actually outsell a one-time sensation over the course of a year… or twenty. Add to that the fact that the BookScan numbers don’t reflect sales at some large outlets and at a considerable number of independent bookstores.

There’s no doubt that publishers do try to game the system.  A publisher generally won’t schedule two possible blockbuster books from its own list in the same genre or with the same readership list in the same month or even the same season. They may set up special events just after midnight so that a book comes out in the first minutes of the next reporting week, maximizing possible sales for that one week.  Or they may try to generate huge-pre-orders, as it appears Tor did with the last book of the Wheel of Time.  Sometimes that works; sometimes it doesn’t; and sometimes, even the publisher gets surprised.  No one at Tor, for example, had any idea that Imager’s Battalion would make The New York Times bestseller list, especially since it was released something like two weeks after A Memory of Light [the final WOT book].

Upon occasion, however, individual authors have also gamed the system, like the authors of a business book who bought something like 10,000 copies of their own book in targeted areas across the USA in order to get on the bestseller lists – and did. I don’t have any idea if they made back the likely quarter of a million dollars that cost, but maybe they did.  And so did the author of a book called Leapfrogging, but doing so required getting others [i.e., his business clients] to pre-order some 3,000 books… and that’s not all that different from simply getting 3,000 people to buy your book.  As a very practical matter, major publishers don’t “game” the system this way.  They’ll persuade, talk to book buyers, advertise, create fan groups and buzz, etc., but it’s rather obvious if your purchasing department buys on the market 10,000 copies of a book you just published… not that for high-priority titles they won’t do quite a few other things.

Because Amazon does make weekly BookScan numbers available to authors – just of the sales of their own books – I can compare what BookScan reports to what is actually sold, albeit on a greatly delayed basis, since I get author royalty reports only twice yearly, and those numbers are for a 6 month period ending four months previously. I think it’s fair to say that, while royalty reports may not reflect all books being sold in a period, they’re far more accurate that any bestseller methodology, and they certainly don’t have inflated sales numbers because I can be absolutely certain that my publisher is not going to pay me royalties for books that were not sold.

Based on that comparison, the sales numbers for my books that I get from BookScan through Amazon [sales, not their hourly ranking] only account for roughly 50% of actual sales, suggesting that I have significant sales in smaller retail outlets – and if those outlets are ones surveyed by the Times or USA Today, I’ll rank higher on their lists than on the PW list, which is in fact exactly what has happened on those few occasions when I’ve made the lists [near the bottom, I will confess, but it’s still very nice to be on those lists].

As for reader ratings, they reflect nothing more and nothing less than reader preferences, and have little to do with sales numbers, except that books that sell more copies usually have more reader reviews, but such reviews usually comprise less than half of one percent of the number of copies sold – except in the cases where authors have made a huge campaign to get their readers to write reviews [and in the handful of cases where desperate authors used multiple pseudonyms to pad their reader reviews].

So… the bottom line, so to speak, is that bestseller lists do reflect comparative sales, but only roughly, not inclusively, and only for a defined period of time… and publishers do their best to maximize the appearance of their books on lists… sometimes… but the information is far too scattered and imprecise for the kind of  wholesale “insider corruption” that is periodically suggested, particularly since the bestseller lists compiled by different entities with differing sources and methodologies seem to share a great many titles, if in differently ranked positions.  And while the cost of “gaming” the system by buying your own books can work… it’s far too expensive, even with the “multiplier effect” of being on the list, for most people or publishers to find it cost-effective over the long run.

In short, from what I’ve seen, there’s a fair amount of “gaming” on the part of authors and publishers but not, if you will, list “manipulation” or “corruption” on the part of either publishers or those who publish the various best-seller lists.

Graduation rates, market coverage rates, rate of return, RBIs, batting averages, the Dow Jones or S&P Index, bestseller lists, consumer ratings, ACT , SAT, other school test scores…  We’ve gotten to the point in American society where numbers seem to define everything – especially success.  And if you can’t quantify it, then either it’s not worth doing… or there’s something wrong with your analysis and understanding.

I’ve seen article after article talking about the need to increase retention and graduation rates in education, both from high school and from college.  I can’t recall a single one emphasizing the need to improve students’ ability to think, to write and speak coherently, and to complete tasks well and on time.  The emphasis is all on the numbers – how many and what percentage graduate and with what degrees.  My wife has taught at the same university for twenty years, and today, if one looks at test scores, the students are far brighter than those of twenty years ago.  Yet twenty years ago, students who could not write a coherent essay were a small minority; today, those who can are the minority.

Today’s students cannot only not remember facts critical to their field for more than a few days, but even when given those facts, most cannot create a logical structure with them.  But the numbers say they’re brighter.  So do their grades, as a result of rampant grade inflation, another effect of “numbers” emphasis. Yet regardless of what the numbers say, more practical evaluations indicate that that basic measures of education, like thinking, reading, writing, retaining knowledge and being able to use it effectively, are actually declining. But what’s happening in education is symptomatic of an illness that pervades all of society.

Look at business.  Companies are graded almost exclusively on the numbers, to the point that if a company’s reported earnings drop a few pennies per share for one quarter, the stock price may drop dollars.  And those same companies pursue profits to the extreme, often taking enormous risks to add a few percentage points to their rate of return, even when those rates of return, as in the case of hedge funds and investment banks, were among the highest in finance and industry.  Those “numbers” weren’t enough, and those oh-so-brilliant business types pushed for higher numbers… and crashed the economy.

In sports, athletes are paid on their numbers, and the sports sections are filled with tables and stories dealing with those numbers.  Is it any wonder that there are doping scandals, when only a few percentage points, or seconds, or some other numerical differential that is comparatively small, equates to hundreds of thousands, if not millions, of dollars’ difference in compensation?

As a not-quite-side note, what I also find amazing is that for all this emphasis on numbers, the vast majority of people who emphasize them don’t know what they mean or even whether they’re relevant, not to mention the fact that millions if not billions of dollars are spent, or cut from spending, based on projections of future numbers, projections that are almost always revised and often far too optimistic.

Rate-of- return, return-on-investment – those can mean anything, depending on how the underlying figures are juggled.  Which is more accurate, ROI based on the cash value of the investment at the time it was made… or the present value of that investment, and if it’s the present value, which of a half dozen measures of inflation are you using to determine it?  What does an increased rate of high school graduation mean when the students who are graduating have a lower rate of reading comprehension and mathematical understanding [NOT test scores on those subjects]?  Does it mean that nothing has changed, except that more students have a piece of paper?

The problem with chasing numbers is that the numbers become more important than what they’re supposed to measure. Years ago, I came across an analysis of the results of a college degree, and one of the surprising results, at least to the analysts, was that the college or university from which one graduated was far more predictive of future success than was class ranking.  Likewise, the early and even the present studies of the value of a college degree are missing the point. When the first studies in this area were made, college graduates were in fact a small percentage of the population and an academic and social elite. That these graduates made more money over a lifetime had less to do with their having a piece of paper than their having mastered the range of skills necessary to obtain that piece of paper… and the study about which college trumping comparative class rank actually supports that point of view.  Elite colleges require elite students and demand they achieve more, so much so in some cases that students in the lower levels of an Ivy League-level school may well learn more than all but the very, very few in the majority of state universities.

Test scores don’t measure those differences well, because tests have gotten better and better at measuring innate intelligence, rather than applied intelligence… and it’s applied intelligence in the end that determines success, just as it’s the effectiveness of applied capital and personnel that determines corporate success over time, not this quarter’s or this year’s accountant-adjusted profits.

The best “numbers” usually go to the best statistical liars, and it’s well past time to get back to looking at the qualities and quality [or the lack of it] that the numbers are so good at distorting… or not measuring at all.

One of the most basic problems with a market economy is that it values goods and services essentially on their cost of procurement and not upon their importance to society, or even to survival.  For survival, for example, two of the most basic goods are air and water.  Without air, we would die within minutes, without water, within days.  Yet in any market economy, there is no charge for air, and an ounce of gold, which has no intrinsic value at all, will cost somewhere over $1,500, while an ounce of the most expensive bottled water might cost five cents [fifteen cents in New York], and tap water, in my area, costs about a tenth of a cent a gallon.

The same discrepancies occur in paying for services.  Both society and individuals can exist without hedge fund managers, who are, as a category, about the highest-paid individuals in U.S. society, and who collect salaries/bonuses in the million dollar plus category.  On the other hand, firefighters, police officers, and teachers, who are vital to an organized society, have among the lowest salaries of skilled professionals because they are public employees and considered easily replaceable.  What essentially determines those relative salaries are two factors – how much income an employee generates and what it will cost to replace him or her.  This is the bottom line of the business model so often touted these days by politicians and business people.  And from a profit and economic basis, it makes sense – but only from those bases, and often, only in the short-run and only to the specific organization.

What market economics ignores is the value provided by those “cheap” and “replaceable” people. Without teachers, those high-paid hedge fund managers would be even harder to find, because there wouldn’t be enough educated people to spare any for complicated financial dealings.  Without firefighters, sanitation workers, police officers, and other public infrastructure employees, cities as we know them in the Europe and the western hemisphere would become uninhabitable… or at the very least, warrens of filth, squalor, and crime.

Even within the business community, the heads of companies often do not know the return on their investment in their employees.  A year or so ago, I ran across a study on retail stores that concluded most large retail establishments would be more profitable if they hired more staff, rather than less.  It’s anecdotal, but my wife and I came up with five separate occasions in the last month or two when we went somewhere to purchase a specific item… and left without purchasing it because we couldn’t get service, not even a polite, “I’ll be with you in a minute.”  I’ve watched people walk out of restaurants because they sat there for five or ten minutes without seeing a server.

Now… the latest “buzz” in education is the need for more professionals in the STEM [Science, Technology, Engineering, Mathematics] areas, and politicians have even proposed charging students more for university courses in curricular areas where there are fewer jobs, such as the performing arts, history, and philosophy.  But, of course, I haven’t seen any move to raise tuition or cut faculty salaries in law and business, where we now suddenly have far more graduates than jobs for them.  The fact is that a society needs a wide range of abilities and skills, and the scarcity of skill or a job shouldn’t be the only factor in determining what it pays.

But then, to pay more for people who provide vital functions, in order to get better individuals in those fields…. how shocking… how… uneconomic… how at odds with the business model.

Maybe I’m old-fashioned, but when people talk about rich, I get an image of someone who has a 10,000 square foot mansion or an enormous penthouse in Manhattan, or their own private jet, who drives a car that costs more than our house and whose vacations are often on private islands or on yachts the size of a Navy destroyer.  That’s what it means to be rich.  And I think most people have similar ideas.

That may be why more than a few people I know were incredulous when President Obama declared that couples who make more than $250,000 a year were “rich.”  The general consensus among them was that families that made $250,000 were comfortable, even in some cases fairly well off, and possibly upper-middle class… maybe.  But rich?  No way.

One couple pointed out that while their house was paid for, it had taken 20 years, and that it wasn’t exactly a mansion, but a raised ranch of 3500 square feet with a modest two-car garage on a half-acre lot in a nice neighborhood, but certainly not one filled with mansions or even McMansions… and that she and her husband had only taken one vacation in the past ten years, and that their cars, a four year old GM Denali and a twelve year old RAV, while also paid for, were not exactly luxury vehicles… and that the rest of their possessions were similar… and that what they had saved for retirement might provide them with a comfortable but even more modest lifestyle than what they now enjoy.

Another couple just laughed. They both work in Manhattan, and their combined income, I’d estimate, is probably a little over $200,000. They can’t afford to live in New York itself, not in anything other than a shoebox, and instead own an 1800 square foot co-op apartment – definitely not a luxury one — in the suburbs north of the city and commute to work by train.  Again, they’re certainly not starving or impoverished, but any reasonable person might find it difficult to insist that they are on the borderline of being rich.

I could give example after example of similar instances, of couples making $200,000 – $300,000, who may be well-off, but are anything but rich.  They don’t own a Mercedes or a Rolls-Royce.  They don’t travel first class on the Queen Mary, or have luxury skyboxes at the Superbowl, or millions – or even tens of thousands of dollars – stashed away in Switzerland.  And they don’t squander money on anything.

Now, I know that the IRS statistics say that only something like two percent of the population makes more than $250,000, and only one percent makes more than $450,000, but those numbers can be misleading.  To begin with, there’s a significant difference between income and wealth, and between sources of income.  For example, compare a businessman or a doctor, or a two income family that makes $250,000 a year to an heir or heiress getting the same income from sitting on $20 million in tax-free municipal bonds.  A couple that averages $200,000 in income for 40 years would have to save 25 % of their pre-tax income every year [and closer to a third of their after-tax income] to put away two million dollars… and even with compounding of those assets, they’d likely only provide a retirement income of $50,000- $75,000. That’s certainly comfortable, but rich?  And even with Social Security on top of that, it’s barely middle-class in places like New York, San Francisco, and other high-cost cities.

Rich at $250,000 a year?  I don’t think so, not unless someone can explain how you can have a mansion, a Mercedes, and a private jet on that… and not end up in jail for theft or tax evasion.

Several weeks ago, and perhaps it was longer, one commenter made the astute observation that technology magnifies everything, both the good and the bad. I think that’s definitely true, but, based on reflection and more recent observation, I have the definite feeling that it magnifies some things more than others – and incompetence is one of them.

What’s the basis for this conclusion?  Human nature… and the fact that incompetent actions have a multiplier effect, and when that multiplier effect is magnified by technology, incompetence has a disturbing tendency to spiral into ever greater incompetence because, at least at present, most computer systems are designed to carry out instructions with great efficiency and speed, and if the instructions or the programming are flawed, the magnification of difficulties or ineptness can quickly result in enormous problems.

The great financial meltdown and the ensuing recession from which we still may not have emerged is one very good example. Without computerization and sophisticated software, the development and management [such as it was] of securitized derivatives, CDOs and the like, simply would not have been possible.  Add to that the consolidation of the banking and mortgage industries, and their centralization with decisions being made by a few, again a situation not possible without high technology.  Follow up with sophisticated and high tech profit models, and top off with nano-second securities trading technology.  At that point, we had a highly concentrated and centralized structure where bad decision-making, a lack of competence in understanding what those computer models meant, and a poor understanding, if any understanding, of some basic economics could combine to bring down the entire economy.  And that is exactly what happened.

The lithium-ion battery fiasco with Boeing’s 787 Dreamliner, which could easily have become a disaster, is another example of how technology misused, I venture, can multiply incompetence. Several aviation battery specialists have made the observation that lithium-based batteries have a tendency to overheat if, first, the individual batteries are too large, and, second, if their architecture [i.e., the way in which they are arranged and interconnected] is not well-designed, and, third, if they are overcharged, although so far the third condition does not appear to be a factor.

There are also innumerable simpler forms of technologically multiplied incompetence with often disastrous results, such as the simple combination of cell-phone texting with the operation of an automobile, particularly when the operator’s skills are marginal, in the case of teenage drivers or of tired or distracted drivers.  Or the 80 to 100 car pile-ups that seemly happen routinely anymore because of the combination of incompetent driving [driving too fast for the conditions] in fog or snow. Somehow, I doubt that there were ever hundred-carriage pile-ups.  And how many product recalls have we experienced because of manufacturing or design failures multiplied by the technology of mass production?

Incompetence in data-processing/computer systems is another area where a small incompetence can be multiplied a million-fold.  As I noted in an earlier blog, Delta failed to renew its online website security system, and for two days thousands, if not hundreds of thousands of customers could not make reservations. A software error in the Utah Medicaid system allowed hackers access to personal information of over 700,000 thousand people and the Social Security numbers of more than 280,000 individuals.

Now, the optimists will say, “But look what good technology does.”  I’d agree, but technology’s greatest asset, these days, seems to its efficiency in replacing people, and cutting costs, far more than multiplying benefits.  Yet those people, especially good competent people, are, for lack of a better term, often the circuit-breakers who stop the magnification of incompetence.  So, at present, technological systems are optimized for, if you will, magnification of whatever they do.  That means doing more with fewer people and less oversight and supervision, since one of the areas of employment that’s taken the biggest hit is middle management… and that includes people who can actually solve problems.

Just look at computerized telephone answering systems. If your inquiry doesn’t fit in the proper “box,” it may take what seems like forever to get an answer – if you can at all… and that’s another form of incompetence, magnified by technology.  Now… I may be overcritical, but, all in all, I’m seeing incompetence and outright criminality magnified far more than benefits.

What about you?