What happened when the long-form census was replaced?

Notes on non-response bias
The Peace Tower is seen in Ottawa, Friday September 25, 2009. Adrian Wyld/TCPI/The Canadian Press

As was expected when the Harper government decided to scrap the long-form census and replace it with a voluntary survey, the new numbers arrive with some caveats.

Stephen Gordon explains the problem.

The National Household Survey findings on “immigration, place of birth, citizenship, ethnic origin, visible minorities, language and religion,” for instance, are here. Included is a note to readers.

The National Household Survey User Guide provides information on the methodology, collection, processing, evaluation and data quality of the National Household Survey (NHS).

Estimates and trends from other data sources suggest that certain population groups may be overestimated or underestimated in the NHS. Information on the quality of NHS data on immigration, place of birth, citizenship, ethnic origin, visible minorities, language and religion as well as explanations of concepts, classifications, questions and comparability with other data sources can be found in the series of reference guides for these topics.

Those reference guides include explanations of data quality. Here is the note on non-response bias for ethnic origin.

Non-response bias is a potential source of error for all surveys including the NHS. This issue arises when the characteristics of those who choose to participate in a survey are different than those who refuse. Statistics Canada adapted its collection and estimation procedures in order to mitigate, to the extent possible, the effect of non-response bias. (For more details please refer to the National Household Survey User Guide, Catalogue no. 99-001-X.)

Several data sources were used to evaluate the NHS estimates for ethnic origin such as: 2006 Census of Population, 2011 Census of Population results for mother tongue, the Longitudinal Immigrant Database (IMDB) and administrative data pertaining to permanent residents and non-permanent residents from Citizenship and Immigration Canada.

It is impossible to definitively determine how much the NHS may be affected by non-response bias. Furthermore, the reporting of ethnicity, and subsequent interpretation of the results, is complex, and poses challenges for comparisons with other data sources. There is evidence of non-response bias for other ethnocultural variables (e.g., Filipino population group could be overestimated in the NHS).

Generally, the risk of error increases for lower levels of geography and for smaller populations. At the same time, the data sources used to evaluate these results are also less reliable making it difficult to certify these smaller counts.

The reference for ethnic origin in the 2006 census is here.

Our complete archive of coverage of the long-form census controversy is here.