Between a rock and a hard place with literacy rate statistics

by Susan Sussman

When it comes to using literacy rate statistics, advocates and policy makers are stuck between a rock and a hard place – we are damned if we don’t and damned if we do use statistics to help us do our jobs.

Damned if we don't

Where dozens of social and economic issues compete every day for media attention, public awareness, political support and a bigger share of public funding, nearly every social, political and economic argument is backed up by statistical research. In the month of August 2003 alone, statistical data supported reports in the Toronto Star and Globe and Mail on baby boomers’ retirement worries, childhood obesity, crime rates, drug trafficking, the economic integration of immigrants in Canada, the effects of family income on university attendance, gender differences in performance on standardized reading tests, homelessness, home- schooling, infant mortality, major depression and other mental illnesses among Canadians, and seniors requiring home care services.

Most advocates find it impossible to move literacy issue
into the spotlight and onto the public policy agenda
without referring to literacy rate statistics.

Journalists, decision makers and corporate leaders with statistics that point to staggering numbers of Canadians with literacy difficulties. Most advocates find it impossible to move literacy issues into the spotlight and onto the public policy agenda without referring to literacy rate statistics. And the influence of literacy rate statistics doesn’t end there; it is seen throughout the policy development process.

Once decision makers have bought into the notion that literacy warrants a public policy response, the issue literacy has to be framed further. What is the specific problem in need of attention? Who suffers from it and how? Who gains from it? What causes it? Population literacy surveys, statistics that derive from them, and interpretations of that data all influence the way literacy is framed as a policy issue. For example, reports on the International Adult Literacy Survey (IALS), issued by the OECD and Statistics Canada, frame literacy as a human capital issue, crucial to the economic performance of industrialized nations in an increasingly competitive global economy.

Forecasting is used in policy development to help decision makers make better decisions. Where literacy is regarded as a public policy concern, literacy rate data is bound to show up in the forecasts. For example, consider the following notes from a presentation on the federal government’s Skills and Learning Agenda (1999).

  • Canada’s demographics mean there will be fewer new workers.
  • Shortages of skilled workers could restrain future growth and innovation.
  • 8 million working age Canadians have low liter- acy skills (IALS levels 1 and 2) by international standards.
  • An additional 6.5 million (IALS level 3) will need continuous upgrading/lifelong learning to participate effectively in the knowledge-based economy.

Goal-setting and decision-making about the allocation of public resources are also essential for policy development. These usually involve a priority-setting process in which problems, goals, services, geographic areas and/or specific population groups are ranked. Data from IALS has been used to make the case for placing adult literacy at the top of the priority lists of industrialized nations, primarily as a labour force development issue. Most Canadian literacy advocates acknowledge that reports from IALS have been instrumental in preserving or increasing funding for literacy.

Once policy goals are determined, options for valuated and selected. Some argue that literacy assessment data from large-scale surveys must be used to inform decisions about how best to address literacy problems.

Without such data it is difficult to determine what types of educational programs are needed and where funding should be channeled. For example, national data can be used to determine where English literacy programs and native language literacy services or bi-lingual literacy services are needed. (Wiley)

graphic - Moving the Markers

Finally, policy outcomes are reviewed and ev development process are commonly the benchmark against which outcomes are measured. When expectations have been expressed in terms of changes in literacy rates, this data becomes a yardstick used to evaluate policy outcomes. For example, the 2001 Throne Speech included the goal of “significantly increasing the proportion of adults with higher-level skills.”

In a nutshell, the issue of adult literacy likely never would have gained the public and political recognition it enjoys in Canada today were it not for the startling statistical results of national literacy surveys like IALS. To a considerable extent, the design of current policies and programs has been influenced by interpretation of literacy rate data.

"Statistics are human beings with the tears wiped off."

The Current Literacy Target
(based on IALS, 1994)

Target When Literacy Rates are
Improved at Some Point in the Future

graphic - chart chart

Damned if we do

Few people, including those Canadians involved in adult literacy work, have ever formally studied the sciences of statistics or research design. Thus few can confidently examine statistical research findings and interpret them independently. People working in the literacy field who do have a research background rarely have the time or mandate to critically consider new research. As far as many people are concerned, statistics are alien and alienating.

Meanwhile, most research experts agree that all existing methods of estimating literacy rates have significant conceptual and technical limitations. For starters, there’s no broad consensus about what it means to be literate. This is a big problem. Logic dictates that a shared understanding of the word literacy is a prerequisite for a shared understanding of how to measure it.

Even those who define literacy in the same way may disagree about if and/or how it should be measured. Words used in a definition are one thing; the phenomenon captured by a particular assessment method is another. Many researchers question the validity of all assessment methods used so far, arguing that they fail to reflect how real people use literacy skills in their real lives.

The science of statistics complicates the problem further when the results of thousands of individual literacy assessments are aggregated into population literacy rate estimates. Population statistics tend to diminish the complex realities of individuals in service of creating mathematical findings. This problem led one pundit to say, “Statistics are human beings with the tears wiped off.”

Different experts can legitimate ly draw different conclusions from the same data. A skilled mathematician who tortures numbers long enough can make them confess to almost any thing. The pliable nature of statistics led Mark Twain to say, “There are three kinds of lies – lies, damned lies and statistics.” Many of our colleagues are wary of statistics believing that numbers can and will be manipulated to prove whatever a researcher wants them to show.

Numbers that quantify Canada’s literacy challenges may engage us, but they may also enrage us. They may reveal a lot, but they may conceal as much.

Thus literacy advocates and policy makers are damned if we do and damned if we don’t use literacy statistics to help us do our jobs. Without them we seem unable to gain and keep public and political support for broad-based strategies to improve literacy levels. While the statistics can help put the issue on the map, they don’t always lead us in the right direction. Thus numbers that quantify Canada’s literacy challenges may engage us, but they may also enrage us. They may reveal a lot, but they may conceal as much. While discourse on literacy is influenced by the numbers at the policy level, literacy rate estimates don’t necessarily reflect literacy problems as learners and practitioners in programs understand them.

We need to understand statistics, and when, how and why they are used.

Tensions between the opportunities and challenges associated with using literacy statistics generate noise and confusion in Canada’s system of literacy initiatives. What get said about literacy in public awareness campaigns (e.g. “22 per cent of adult Canadians have serious difficulty dealing with print”) may be highly relevant to literacy rate statistics but not entirely relevant to what goes on in literacy programs, which tend to be shaped more by realities and needs of individual learners and practitioners. Many of the important gains made in literacy programs may never show up in literacy rate statistics. Thus we run the risk of winning public and political support today because of what the numbers show, only to loose it tomorrow because of what the numbers won’t show.

What to do?

For better or worse, literacy rate statistics will continue to be used wherever literacy policy decisions Canada’s literacy policies cannot simply ignore the tead, we need to understand them and when, how and why they are used. We don’t necessarily need to become statisticians but we do need to know what questions to ask about large scale literacy assessment research, and we need to develop the will and the ability to become critical consumers of literacy rate statistics. Towards these ends, I offer four recommendations.

  1. Given the potential influence of literacy rate data on the development of policies and programs, alternative plausible interpretations of literacy rate data should always be identified and considered before that data is used to inform policy decisions.
  2. Researchers from various professional affiliations and theoretical persuasions should be involved in interpreting literacy rate data as early as possible in data-analysis and policy- development processes.
  3. Consumers of research should be encouraged and supported to reflect critically on literacy rate data and related interpretations.
  4. Training for advocates, policy-makers and other interested parties in how to critically review literacy rate research should be included in initiatives to build Canada’s literacy-related research capacity.

Susan Sussman has worked in literacy since 1993, in the position of Executive Director of the Ontario Literacy Coalition, as the President and board member of the Movement for Canadian Literacy, and as an independent researcher. Sussman holds a master’s degree from Columbia University's Teachers College and a bachelor's degree from the City University of New York.


Skills and Learning Agenda 1999. Notes from a presentation by a representative from Human Resources Development Canada, at a meeting of invited literacy stakeholders convened by Senator Joyce Fairbairn, to gather input on the proposed Skills and Learning Agenda. Ottawa.

Speech from the Throne to Open the First Session of the Thirty-Seventh Parliament of Canada, Ottawa, January 30, 2001. Her Excellency the Right Honourable Adrienne Clarkson.

Wiley, Terrence G. (1994) “Estimating Literacy in the Multicultural United States: Issues and Concerns”. Washington D.C.: ERIC Digests.

View in PDF (96k)

Return to the table of contents

placeholder placeholder placeholder
Valid HTML 4.01! Valid CSS! placeholder Creative Commons License This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 2.5 Canada License