Charles Seife knows how to spot a bad number. “Certain obviously wrong mathematical ideas were floated by politicians or even by scientists, and the press swallowed them up without thinking,” said Seife, an NYU associate professor of journalism and author of Proofiness: The Dark Arts of Mathematical Deception. Seife was particularly surprised at numbers that quickly fell apart with the slightest questioning, such as a recent National Institutes of Health-backed HIV vaccine trial in Thailand. “This trial – which they said was a success – was nothing of the sort and in fact was a dud,” he said. “There are entire industries devoted to making numerical mountains out of molehills, making drugs look like they’re performing better than they are.” Below, Seife chats with Zócalo about why we tend to trust numbers, what makes a number bad or good, and how errors can disenfranchise voters or swing an election.

Q. Why do we regard numbers as having this veneer of truth?

A. Numbers in the abstract are about as pure a concept as you can get. They represent absolute truth in a way we rarely get to access. The truths of mathematicians, the proofs they provide, are absolutely correct. There is no argument once the proof has been verified; it simply enters the realm of truth. When mathematicians study numbers, they’re looking at these platonic ideals.

In some ways this idea of purity of numbers has filtered down to the public. But for people who are not mathematicians, numbers as pure forms don’t really have any interest. The only reason we care about numbers is because they’re attached to something in the real world. The moment these numbers get attached to something in the real world, they lose their purity. Real world numbers have lost their purity, but they’re still imbued with that feeling. And that’s why we get deceived. We still think of math as having absolute truth, but the numbers we’re dealing with are admixed with dirty reality.

Q. What makes a good number and what makes a bad number?

A. It’s hard to say there are absolutely good or absolutely bad numbers in the real world. We can say that some numbers are better than others. A number is trustworthy if the measurement that it is based upon is an easy one to make, and easily replicable. If you want to figure out how long an object is, you pull out your tape measure. You come up with a number. Many other people measure it and they’ll come up with pretty much the same figure. That’s a reasonably good number – as are measurements of length, time, weight. They’re reliable and easy to talk about.

But then you get to measurements that are more and more difficult to make – like counting the number of people in the U.S. You can make a pretty good estimate, but there are problems. And then you get to measurements that are dubious. You see an ad for a shampoo that claims to double the silkiness of your hair. How do you measure that? It’s not so obvious.

Then there are numbers that are made up entirely. One that comes to mind is Glenn Beck’s rally on the National Mall. He claimed that 500,000 people showed up, but other counts came to 90,000. That’s a lie by a factor of five or six. Louis Farrakhan did this almost two decades earlier, when he claimed there were a million men at his Million Man March, but there were in fact fewer than half a million. These are what I call Potemkin numbers – made-up to try to fool people who don’t look closely.

Q. Where do these bad numbers get harmful – how are they used in policy?

A. The census I brought up earlier, for instance. It’s not so easy to count the population. The Census Bureau has to use some sophisticated statistical techniques to count people who don’t answer the mail, who disappear when Census workers knock on their door, who move, who don’t speak English. To get a good number, the Census has to figure out whom they’re missing and correct their raw count. Statistical sampling is the best technique for doing that. But the Census was actually barred by the Supreme Court from using the sampling-corrected numbers for apportionment. They’re forced to report two numbers to Congress, one which is pretty accurate and one that is demonstrably inaccurate. The one used to determine who gets power and who doesn’t is the inaccurate one. As a result, there are populations in the U.S. that are undercounted, that do not have the power of representation they are entitled to under the Constitution – these tend to be minorities, immigrants, poor people who rent, and these people tend to vote Democratic. I don’t think there’s anything really ideological going on here – I think if the situation were reversed, and the undercounted were Republicans, Republicans would be trying to get the Census Bureau to count better and Democrats wouldn’t be. But as it is, the Republicans have steadfastly tried to stop any attempt by the Census Bureau to use statistical sampling.

Q. So even the Supreme Court isn’t immune to using bad numbers?

A. The Supreme Court uses hazy numbers all the time. The case in question is United States Department of Commerce v. United States House of Representatives. Republicans in the House sued to stop the Census from using statistical sampling. The winning argument – and it was a five-to-four decision – was that the language in the Constitution, the phrase “actual enumeration” that must be performed every 10 years, does not include statistical correction. That it includes a headcount and nothing else. It was specious reasoning and it became obvious how specious it was in a later case, when Justice Clarence Thomas said that Thomas Jefferson was well aware of all these statistical techniques. But those techniques were invented in the 20th century.

I don’t think that’s the worst misuse of statistics and numbers of the Supreme Court. I think that goes to a death penalty case, Kansas v. Marsh. There was an argument over whether innocent people were getting sent to their death. As you can guess, the liberal justices were on one side and the conservatives on the other. The proofiness there, the really bad number, came from Justice Antonin Scalia. He pulled out an argument that said that juries almost never sent an innocent person to their death. He claimed they were beyond 99 percent accurate – that just a few in a thousand would get wrongly convicted. But if you look at the math behind that number, it falls apart. It ignores that we have to count some felonies differently, that not all people who are guilty of a crime actually go to jury trial – most cases come from a guilty plea, which messes up the statistic. If you do the calculation right, you find that between three and five percent are wrongly convicted. That’s what you expect for human nature and human fallibility. People aren’t perfect, they’re swayed by evidence they shouldn’t be swayed by. They put undue weight on eyewitness testimony, which is unreliable. People do get things wrong.

Q. Is it harder to understand numbers today because they’ve grown more complex, as have the ways to count?

A. It’s a much harder task to count when you have a population that’s so dense and moving around. We’ve got 300 million people, obviously much greater than the Founding Fathers had to deal with. Votes are tough to count, too. When you’re dealing with counts of millions of votes, it’s much harder than the traditional old precincts had to do. You’re more likely to make mistakes. When you’ve got large numbers of votes, small errors that are only a few  fractions of a percent of the vote total start adding up to large blocks of votes. If you look at Bush v. Gore or more recently Franken v. Coleman, these are votes of millions of people – six million in Florida and three million in Minnesota – that were decided by a few hundred ballots, a tiny, tiny, tiny fraction of a percent of the votes cast. These errors, which might throw 20 or 40 or 100 votes this way or that, wound up swamping the elections so much that you really could not tell who won. As a result, the more people you have to count, the more complex the election, the more likely you’ll have errors, and the more likely you’ll end with a really bad situation.

Q. How do we get smarter about numbers – is it about learning math better?

A. Mathematical literacy can help to some extent. People have to be taught that numbers aren’t pure. They can’t be afraid to question a number. The moment they’re able to do that, everything they’re presented with a number, they can think, is this number correct? Do I believe it? Does it make sense? I think most people are scared of numbers. They’re afraid to question them. As a result, I think the influence of bad numbers is very large and growing. You see it everywhere. Every big story seems to have a bogus number in it. In the late spring and over the summer, for instance, the BP oil leak story started with bogus numbers. BP lied about how much oil leaked into the Gulf, while scientists said the number was much larger. BP and even the Coast Guard low-balled – it was a propaganda issue. The smaller the number, the less damaging the leak looked to the public, regardless of what was happening on the ground. That number had a tremendous effect. I think the recognition that numbers are a powerful form of propaganda means they’re going to be used more and more often until people recognize that numbers can be bad.

*Photo of Charles Seife by Sigrid Estrada. Photo of numbers courtesy mollycakes.