6 Comparability

6.1 Historical comparability

The counts for Aboriginal groups, Registered or Treaty Indian status group and First Nation/Indian band membership may change over time for a number of reasons. Part of the change (increase or decrease) in the number of people reporting Aboriginal group, Registered or Treaty Indian status or First Nation/Indian band membership is due to demographic growth, another part is due to changes in reporting patterns between Aboriginal groups and between Aboriginal and non-Aboriginal populations.

Changes in Aboriginal counts over time can also be affected by differences in the wording and in the format of Aboriginal questions; legislative changes; differences in the set of incompletely enumerated reserves; and changes made to the definition of reserves.

As well, the counts for the Aboriginal variables may also have changed as a result of differences in edit and imputation (E & I) procedures, and changes in methodology between the 2011 NHS and the 2006 and 2001 censuses – for example changes related to weighting and calibration, changes to the universe, etc.

6.1.1 Differences in wording and format of Aboriginal questions

There are various ways to define the Aboriginal population based on the four questions asked in the NHS. These questions are Aboriginal ancestry (Question 17 on ethnic origin); Aboriginal group (Question 18); Registered or Treaty Indian status (Question 20); and Membership in a First Nation/Indian band (Question 21).

Although measuring the same concepts, the four NHS questions differ slightly from the Aboriginal questions on the 2006 Census. The question wording was modified to reflect current terminology and ensure ongoing accuracy when measuring the Aboriginal population. For example, in 2011 changes were introduced to the terminology in the Aboriginal group question (Question 18). 'North American Indian' was replaced by 'First Nations (North American Indian)' and 'Inuit (Eskimo)' was replaced by 'Inuk (Inuit)' in both the question and the response categories. An instruction was added 'Note: First Nations (North American Indian) includes Status and Non-Status Indians.' See the earlier Aboriginal concepts section (Section 1.1) for the details about each question.

6.1.2 Legislative changes

Legislative changes such as the amendments to the Indian Act of Canada: Bill C-31 in 1985 and Bill C-3 in 2011,^Footnote1 may affect concepts such as Aboriginal identity and Registered or Treaty Indian status. Such legislative changes can affect how individuals may have responded to the Aboriginal identity and Registered or Treaty Indian status questions in the 2011 NHS. The net effect of these changes cannot be measured. Users should be careful when interpreting the results.

6.1.3 Differences in the list set of incompletely enumerated reserves

In 2011, there were a total of 36 Indian reserves and Indian settlements reported^Footnote2 as 'incompletely enumerated' in the NHS. For 18 reserves or settlements, census enumeration was either not permitted or was interrupted before it could be completed and so the NHS was not administered in those areas. For four (4) reserves or settlements, census enumeration was completed however, data collection for the NHS was not permitted or interrupted and in one reserve or settlement, it was determined that there was no resident population contrary to what was erroneously reported in the census. In the case of 13 reserves in Northern Ontario, enumeration was delayed because of natural events (specifically forest fires) and estimates for these communities are not included in geographic areas that include these communities (e.g., provincial and national estimates).

There are no data for incompletely enumerated Indian reserves and settlements on the NHS database. Higher-level geographic areas containing these areas are identified in the NHS products. Although NHS data are not available for incompletely enumerated Indian reserves and settlements, the areas themselves are included as part of the standard geographic hierarchies on the NHS databases.

The extent of the impact of the exclusion of the incompletely enumerated reserves will depend on the geographic area under study. It is much less for higher geographic areas such as Canada, provinces and territories, census metropolitan areas, and census agglomerations. The impact may be more significant for lower geographic areas, such as census subdivisions.

The issue of incompletely enumerated reserves has the most impact on NHS estimates for the First Nations population living on reserve as well as the Registered or Treaty Indian status population living on reserve. Estimates for other populations will also be affected. Other affected populations include: total Aboriginal identity population, First Nations (North American Indian) identity population, total Aboriginal ancestry population, First Nations (North American Indian) ancestry population, Registered or Treaty Indian status population, population who reported membership in a First Nation/Indian band, and population living on Indian reserves and Indian settlements.

Estimates associated with other variables related to First Nations, such as language and band housing, may also be affected by the incomplete enumeration of certain Indian reserves and Indian settlements in the NHS. This issue does not have an impact on the estimates for the Inuit or Métis populations as they are much less likely to live on reserve.

Top of Page

6.1.4 Changes made to the census subdivision types associated with 'on reserve' population

Statistics Canada uses the definition of 'on reserve' provided by Aboriginal Affairs and Northern Development Canada (AANDC). From time to time, there are changes made to the geographies that define Indian reserves and settlements. In 2006, there were 1,174 'on reserve' census subdivisions (CSD) while in 2011, there were 997. The changes are the result of additions, deletions, changes in geographic coding and amalgamations. See the Census subdivision (CSD) definition in the 2011 Census Dictionary for the reserves associate with each type of change. Because these are geographic changes, adjustments for these are possible.

Users need to request special tabulations for data that have been adjusted for these changes in order to compare populations on the affected geographies (e.g., comparing on- and off-reserve population).

6.1.5 Processing and methodological changes

6.1.5.1 Editing and imputation

The editing and imputation (E & I) of the Aboriginal variables was almost entirely redesigned for 2011, with the primary goal being to streamline the processes and to use, as much as possible, one donor to impute data for a respondent who had provided incomplete or invalid responses on his/her NHS questionnaire. In 2011, the variables of immigration, citizenship, place of birth, ethnic origin/Aboriginal ancestry, population group/visible minority, Aboriginal group, Registered or Treaty Indian status, and First Nation/Indian band membership were processed together, with the interrelations between these variables clearly defined in advance. Donor imputation for missing information within these variables was done with one donor for all variables, as much as possible.

Given the low item non-response and invalid rates, and the correspondingly low imputation rates for Aboriginal identity group, Registered or Treaty Indian status, and First Nation/Indian band membership, the overall effect of E & I on the data quality of these variables is not nearly as significant as other processes, such as the weighting procedures used to deal with non-responding households.

6.1.5.2 Weighting and calibration

The NHS weighting process involves calculating sampling weights, adjusting the weights for the survey's total non-response and calibrating the weights against census totals.

First, an initial sampling weight of about 3 is assigned to each sampled household. However, the sampling fraction varies with the questionnaire delivery mode. For the mail delivery mode, about 3 in 10 households (29%) received a questionnaire. For the enumerator delivery mode, the sampling fraction was 1 in 3 households (33%). Nevertheless, in cases where it was necessary to reach households in remote areas or on Indian reserves, where only the interview response mode was offered, all households were invited to participate in the NHS.

Then the sampling weights are adjusted to reflect the selection of the subsample from the set of households that had not responded to the NHS by mid-July 2011. Next, since a number of households in the subsample were still non-respondents at the end of collection operations, the sampling weight was adjusted for the survey's residual non-response. This was done by transferring the weights of non-respondent households to the nearest-neighbour-respondent households.

Lastly, the weights are calibrated against census totals at the level of geographic calibration areas. Calibration is performed so that the estimates for an NHS calibration area are approximately equal to the census counts for that area, for a set of about 60 characteristics common to the NHS and the census.

Nevertheless, there may be differences between the NHS estimates and the census counts for common characteristics. The smaller the geographic area is, the greater the risk that the NHS estimates will be different from the census counts. This was present with the 2006 Census long form, but it was less common because of the higher response rates and the small variation in these response rates across areas, for both small and large municipalities.

For more information on the methodology of the NHS, refer to the National Household Survey User Guide, Catalogue no. 99-001-X2011001. All efforts are made to reduce errors in estimation and the Census of Population plays a major role in ensuring the reliability in the estimates of the NHS.

Top of Page

6.2 Undercoverage of population in participating Indian reserves and settlements

In 2011, it is estimated that the net undercoverage of the population living in participating Indian reserves and settlements is 10,125 persons or 2.8%, compared to 2.2% net undercoverage for the remaining population in the 2011 Census. For detailed information on issues related to coverage, such as overcoverage and undercoverage, see the earlier coverage section (Section 5.3).

The specific undercoverage of the off-reserve Aboriginal population for 2011 is not available.

6.3 Comparability with other sources

6.3.1 Comparability with the Indian Register

The 2011 National Household Survey (NHS) from Statistics Canada, as well as the censuses of population previous to that, and the Indian Register maintained by Aboriginal Affairs and Northern Development Canada (AANDC) are the principal sources of demographic data on the Registered (or Status) Indian population.

The 2011 NHS, like censuses before 2011, are snapshots of the population that self-identify as Registered or Treaty Indians at a moment in time, while the Indian Register is a continuous statutory administrative file based on the registration of individuals who meet specific criteria as defined by the Indian Act.

The NHS estimate is affected by undercount of Registered Indians living on incompletely enumerated reserves where enumeration was is not permitted, or was interrupted before it could be completed; undercoverage of Registered Indians living on participating reserves and settlements; and undercoverage of Registered Indians living off reserve. As well, the NHS did not collect data on Registered Indians living in institutions (for example, hospitals, senior citizens' homes, jails, shelters, etc.), nor did the NHS collect data on Registered Indians living outside of the country on NHS collection day.

The Indian Register, on the other hand, forms one cluster of AANDC's entire Indian Registration System (IRS). While considered the authoritative source for the number of Registered Indians as per criteria established in the Indian Act, the Indian Register also comes with its own set of limitations. First of all, the main purpose of the Indian Register is to record individual names and a range of non-statutory data in accordance with specific subsections of the Indian Act. It was not designed for statistical analysis and therefore data from the application may not fully meet the requirements of some statistical activities (e.g., demographic projections, migration patterns).

There is a reporting lag between the occurrence of a given life event and its being reported and recorded for updating of the Indian Register. This means that events occurring in a given year may not be reflected in the Indian Register during that year. The following examples taken from the Registered Indian Population by Sex and Residence 2011 report illustrate this concept.

1. Infants entitled to be registered at birth may not be registered by the December 31st reporting date. Parents often do not report the birth until a later year. It is estimated that of all births reported in 2011, approximately 73% had actually occurred prior to 2011.
   
   
2. Individuals can remain on the Indian Register for some time after they are deceased. A certificate of death or a confirmation of presumed death is normally required to remove a name from the Indian Register.
   
   
3. Residency codes are typically updated when a life event is reported, although some bands update them more frequently. Residency remains a voluntary field in that the information is not mandatory to collect. For additional information on the Indian Register, refer to Registered Indian Population by Sex and Residence 2011.

In addition, Registered Indian status does not guarantee Canadian residency. On the Indian Register there are a number of registrants living in other countries. For example, at the 2011 year-end, there were nearly 19,000 registrants whose province of residence was 'Outside of Canada.'

The different purposes, as well as methodological and conceptual differences between the Indian Register and the NHS, result in estimates that are not directly comparable. The estimate of Registered Indians from the NHS, on May 10, 2011, is 697,510 which is 19.7% lower than the estimate from the Indian Register (IR) as of December 31, 2011 (868,206).

Top of Page

6.3.2 Comparability with Demosim projections

Demosim is a microsimulation model developed and maintained at Statistics Canada, and designed for population projections. Using the microdata file from the Canadian Census of Population (20% sample) as its starting point, Demosim produces dynamic population projections at the level of the provinces, territories, census metropolitan areas and selected smaller geographies, based on a number of characteristics such as age, sex, visible minority group, place of birth, generation status, Aboriginal identity, highest level of educational attainment and labour force participation, among others. It does so by simulating events such as births, deaths, migrations and changes in level of education, according to various population growth scenarios.^Footnote3

Different Demosim scenarios were prepared specifically for the purposes of comparing NHS and projection results for the Aboriginal variables: Aboriginal identity and Registered or Treaty Indian status. For Aboriginal identity variable, two scenarios were prepared: one scenario that does not include ethnic mobility^Footnote4 and keeps Aboriginal fertility rates constant; and another scenario that includes ethnic mobility according to the average from 1996 to 2001 and 2001 to 2006, and Aboriginal fertility rates are kept constant. A third scenario does not include ethnic mobility, Aboriginal fertility rates are kept constant and there is no migration on the reserves.

At the Canada level, the 2011 NHS Aboriginal identity results (1,400,690) are very close to those of the Demosim projection ethnic mobility scenario (1,393,775) – a difference of 6,915 or 0.5%. However, the difference increases to 102,580 (7.9%), when comparing the NHS results with those of the scenario without ethnic and/or cultural mobility. Furthermore, the difference between the two sources is more noticeable for the different Aboriginal groups, especially the Métis and the First Nations (North American Indian).

Ethnic and/or cultural mobility is the transference of identity group that has been observed when comparing one census to the next (more so for non-Aboriginal identity to an Aboriginal identity). Its effect on the estimates of First Nations and Métis are noted more than for the estimates of Inuit. Consequently, overall we observe that the differences in estimates between the two sources are smaller between the NHS and the Demosim scenario with ethnic and/or cultural mobility, than between NHS and the Demosim scenario without ethnic and/or cultural mobility.