Writers’ Block

Decrease Font Size Increase Font Size Text Size Print This Page Send by Email

Why too many journalists get crunched by numbers—and why their stories often don't add up

The Kansas City Star spent more than four years researching the prevalence of AIDS in the priesthood and 18 months interviewing experts and priests, and examining church documents and death certificates to ensure that what it was putting out was accurate journalism. In January 2000, the Star published an 11-article series built around the “fact” that Roman Catholic priests in the United States were dying of AIDS-related illnesses at an alarmingly high rate. The series often referred to a poll sent by the Star to 3,000 U.S. priests.

When the series was published, the lead story contained a paragraph that read, “The actual number of AIDS deaths is difficult to determine. But it appears priests are dying of AIDS at a rate of at least four times that of the general U.S. population, according to estimates from medical experts and priests and an analysis of health statistics.”

Controversy followed. A number of critics and colleagues felt that what The Kansas City Star published was not accurate journalism. The first grievance can be found in an article written by Columbus Dispatch reporter Jennifer Halperin, who described the above statistic as misleading. Mentioned in her article is a senior analyst with the Statistical Assessment Service in Washington, Iain Murray, who pointed out that the article looks as if it says one thing but upon further scrutiny says quite another.

He and other critics called the comparison between priests and the general public invalid. “Priests are adult men,” Murray told Halperin. “The general population includes women and children, and thus is much less likely to be infected with HIV.” Women are said to account for only 20 per cent of new AIDS cases and children make up a relatively small percentage of AIDS victims. “By some calculations the AIDS death rate of adult males is about 4 per 10,000-about the same as the death rate among priests, and four times as high as the general population,” continued Murray.

The Star fought back, quoting an AIDS expert who deemed the comparison to be common and legitimate, also pointing out that the series also made the comparison between priests and all adult males. But Halperin wrote, “Not all reprints in other newspapers included that element.” Also, the comparison between priests and adult males was not published in the second paragraph of the lead story; the comparison between priests and the general public was.

The second grievance concerned the Star‘s survey of 3,000 priests. Only 801 of the 3,000 responded, a number many critics deem unrepresentative. Also, because the poll was anonymous, it is difficult to ascertain where the responses came from, and again if it was representative. The poll, although meant to add credibility to the series, did the opposite.

In an Editor & Publisher magazine article five weeks later, Star editor Mark Zieman was quoted as saying, “We had the story reported at the time we decided to do the poll. But we felt the poll would lend more credibility. We believed we would be criticized for being too anecdotal, not scientific enough. We learned a big lesson. This has been a cautionary tale.”

Numbers saturate every page of our newspapers and magazines, and at times act as the cornerstones of good reporting. They need to be given the same consideration that words do, and this just isn’t happening. Journalists build their stories around facts and statistics, assuming that they’re correct. “There is something so magical about numbers,” says Don Gibb, a reporting instructor at Ryerson University in Toronto. “You come back with numbers and just toss them into a story, or even better than that you toss them into a quote, and somehow that gives it authenticity.” But the result can be the opposite: numerical inaccuracies threaten the credibility of writers and in turn the publications they write for. “There are continual problems with the way reporters handle basic math, everything from issues as simple as percentages to probability theory and news accounts. It’s endemic,” says Jack Hart, managing editor of The Oregonian, a daily paper in Portland, Oregon.

Fear of math pervades the industry and as a result numbers aren’t being questioned. Even more alarming is the fact that numeracy isn’t even being considered by aspiring journalists. In his 1995 Editor & Publisherarticle “Young Journalists Are Terrified by Numbers,” Melvin Mencher, professor emeritus of journalism at Columbia University, wrote that a group of Japanese sixth graders gave more correct answers on a math test than did applicants to the university’s prestigious journalism school.

Innumeracy in journalism is not a new phenomenon. In 1936, Mitchell V. Charnley wrote in Journalism Quarterly about his discovery of habitual numerical errors in newspapers. Almost 70 years later, inaccuracies continue to plague the pages of our dailies.

Numeracy – an acquaintance with the basic principles of mathematics – is not required to enter the industry, nor is it required to enter most journalism schools. As a result, while journalists may generally be good writers, they are generally weak mathematicians.

Consider Christopher “Chip” Scanlan, of the Poynter Institute, “a school dedicated to teaching and inspiring journalists and media leaders,” in St. Petersburg, Florida. When you read his chapter on numbers and the novice journalist in Reporting and Writing: Basics for the 21st Century, you wouldn’t know that Scanlan is math-challenged, except for the fact that he tells you. You wouldn’t know that in his newsroom days he kept the back of a calculator package in his desk drawer because it had instructions for calculating percentages, or that he almost declined my interview, wanting to refer me to a better source. Scanlan knew that he had to include a chapter about numbers in a textbook for journalism students, but writing the chapter failed to give even Scanlan himself confidence with numbers. Instead it gave him confidence regarding where to find information about numbers. He describes the result as “the expanded version of the back of the calculator package.”

While many journalists fear and dislike math, they love a good statistic. As Scanlan says, numbers “come kind of cloaked with this authority and if it’s a really sexy number then journalists love dropping it in there.” He offers the example of the widely used statistic declaring that the average child watches 100,000 acts of violence on television by the time he finishes elementary school – a stat he thinks he may have used himself. He emphasizes the need for reporters to ask why they like that number. “Do you like that number because it fits your preconception, or are you looking at the number and saying, ‘Okay, let me do the math here’?” The statistic originates with a study done by the American Psychological Association, and it is an estimate based on the average American child watching 28 hours of television a week.

A number is not by nature automatically a fact. Obvious mistakes get past reporters and copy editors purely because they aren’t looked at critically. Don Sellar, who runs the Bureau of Accuracy at the Toronto Star, mentioned an article in which the Canadian Press misstated the losses for the wireless firm Look Communications Inc. in the third quarter of 2001. The figure printed was $103,091; the company’s actual net loss was $103 million. The mistake may seem trivial, but it involves powers of 10, understating the loss by over $100 million-a mistake stockholders would deem anything but trivial.

There are obvious mistakes like this one, where percentages are calculated incorrectly or zeros are dropped from a number, but there are also subtle mistakes, which are even more dangerous. These mistakes involve the interpretation of numbers and affect a reporter’s control over a story. Consider this line: “Ottawa estimates up to 16,000 Canadians die prematurely each year from pollution caused by burning fossil fuels.” This appeared in a Toronto Star article concerning the Kyoto Protocol. Contrary to the article’s assertion, the Canadian Health Coalition statement being quoted made no direct claim that all 16,000 lives could be saved if Canada ratifies the agreement to cut greenhouse gas emissions by one-third. A diligent reader informed theStar of the error and a correction followed, but the mistake demonstrates the danger of misinterpreting statistics. Credibility is jeopardized and the entire angle of a story can change.

Information is the heartbeat of journalism, and sources are the blood. Journalists rely on the information sources provide in order to do their jobs. But sources can have their own agendas.

By blindly accepting numbers, journalists may be forfeiting control over their stories to those agendas. “I think reporters are easily snowballed because they’re not comfortable with the idiom,” says The Oregonian‘s Jack Hart. “It’s like trying to order food in a different language, and they’re just happy if something edible comes along.” Usually something edible does comes along, but that doesn’t mean it’s what was ordered. Numbers shouldn’t get through the gate unchecked.

A reporter must understand what the numbers in a story mean before even attempting to explain them to readers. A major inhibitor of this is the comparison of apples to oranges. Comparing annual figures to semi-annual figures, for example, is hardly insightful. An article in the Toronto Star last November about gang violence said that Toronto police laid 4,165 charges for illegal possession of a firearm in 2001, up from 3,565 in 1997. The article also cited a 1996 figure of 809 charges, suggesting a major increase from 1996 to 2001. The 1996 statistic was accurate, but it only represented part of the year.

Many articles about innumeracy in the industry have surfaced over the last couple of years and heightened awareness, but few yield concrete solutions to the problem. Education is critical, but involves time, initiative and money. In-house training programs are surfacing, but not in every newsroom.The Toronto Star, the Kitchener-Waterloo Record and The Oregonian are among those that offer numeracy courses. The St. Petersburg Times in Florida has even implemented a three-phase math-training program.

Ryerson’s Don Gibb wishes there were numeracy police in the newsrooms. “There is always the style person who says, ‘That’s not the way we write daycare. Day care we write as two words, not one word.’ There has always been that person there and there’s always that person who has a fine eye for syntax but I’ve never met anyone who billed himself or herself as a numeracy expert.”

Using outside experts is another solution. University of Toronto mathematics professor Ed Barbeau believes that every newsroom should have a Rolodex of statisticians they trust and can consult. One of his colleagues has been called in by news stations at election time to help deliver coverage of the election race as votes come in. Barbeau also recommends having a statistician come into the classroom or newsroom to talk about how stats can be manipulated. Experts – or even math-proficient colleagues – can act as checks and balances.

On their way to the newsroom, some journalists spend a few years at journalism school. Unfortunately, they aren’t likely to be any more adept with numbers than those already in the business. Although math content in curriculums is increasing at university-level journalism schools in Canada, it’s still not pervasive. Almost all Canadian journalism schools include some sort of math training in their reporting courses, but the concentration varies. The University of Regina is the only one that offers an entire journalism-related math course. Concordia University in Montreal and Ryerson University in Toronto are the only schools that offer a course designed to teach students how to find and analyze information using digital technology.

Several schools also have specific beat-related courses that address numeracy issues. But this is a bit idealistic considering journalists are almost never assigned to their desired beat upon entering the newsroom. Also, those who don’t specialize will still face numbers daily.

The nonexistence of journalism-related math courses isn’t a result of apathetic curriculum committees; it’s more an issue of budgets and balance. It’s not that directors of journalism schools don’t care about numeracy-they actually take it very seriously. It’s that they care more about other things. To add a course you have to drop a course, and most schools aren’t willing to do that. “We don’t believe there is a journalism requirement we would be prepared to drop to add such a course, and we believe that the balance of academic credits is equally important to our students’ education,” says Stephen Kimber, director of journalism at University of King’s College in Halifax. Unless innumeracy in journalism acquires a larger profile, journalism-related math courses will continue to take a back seat to other interests.

Those who are deprived of math lessons in newsrooms or classrooms can certainly learn a few from the staff at the Kitchener-Waterloo Record, who broke a major story simply by taking a closer look at the numbers.

When Record editor-in-chief Lynn Haddrall saw the interest rate of a city park financing agreement, she saw two reasons to launch an investigation. The first: her need to know how the City of Waterloo got such a beneficial arrangement, while other municipalities weren’t enjoying the same good fortune. And the second: her need to know if this deal was in fact too good to be true.

The City of Waterloo joined forces with MFP Financial Services Ltd. of Mississauga to finance the building of a large sports complex called Rim Park. Waterloo said it was repaying the loan over 30 years at an interest rate of approximately 4.7 per cent – two per cent less than the usual rates for this type of loan. Haddrall called in Kevin Crowley-then a business reporter, now the paper’s business editor – to investigate. After unsatisfactory interviews with city officials, Crowley decided to try his hand at calculating the costs of the park himself. The results he came up with were nearly double those given to him by the city.

Crowley knew his way around financial statements. He had taken the Canadian Securities Course-a course for aspiring stockbrokers-at the suggestion of a friend who billed it as great for investigative journalism. Though Crowley calls the course a “confidence booster,” he still didn’t feel comfortable enough to run with his own numbers. He decided to pursue a more tangible angle-the tax breaks in the deal, which conflicted with federal tax laws. A few weeks later, he put everything on hold to take a vacation with his family.

It was shortly before lunch about 12 days later and Crowley had just come in from the beach in Gabriola Island, B.C., where he was vacationing, when his father-in-law told him to call the paper right away. His editor was inquiring about the reliability of a tip he had received claiming that actual costs of the park were twice what city officials had thought. Having done similar calculations himself, Crowley confirmed that it was a possibility. Just like that, he and The Record staff uncovered one of Canada’s largest municipal-government money scandals.

The interest rate of approximately 4.7 per cent turned out to be closer to nine per cent in reality. The actual costs of the park were about $227.7 million, more than double the original estimate of $112.9 million. Waterloo city officials had signed contracts that they didn’t understand and taxpayers almost had to pay the price. The Record published a series of articles detailing the scandal, for which it received the 2001 Michener Award for public service journalism. Since then, Waterloo has reached an out-of-court settlement said to save taxpayers $82 million, a judicial inquiry is currently underway, and city treasurer John Ford has resigned.

Following the scandal in Waterloo, several other MFP clients decided to take a closer look at their deals. Brock University has reached a settlement with the company said to save the school millions of dollars. The municipal governments of Toronto and Hamilton, county officials in Windsor and Essex County, and the Union Water System near Leamington have all filed their own suits against MFP, and an inquiry is also underway in Toronto.

Meanwhile, the Ontario government is developing regulations to guard taxpayers from similar deals, and two finance officials in Windsor and a technology manager in Toronto have lost their jobs. The catalyst that set this all in motion: an editor who looked at the numbers and asked, “Does this make sense?”

Haddrall doesn’t think she learned how to be skeptical; she thinks she was born that way. “I think for some journalists you’re always just always asking, ‘Well, why? Well, why?’ and ‘Show me, show me.'” She urges journalists to never stop asking questions. Especially when it comes to numbers.