Introduction

In the spring of 2018, the Office of the Auditor General of Canada (OAG) released a performance audit, which focused on whether “Indigenous Services Canada [ISC or the Department] satisfactorily measured and reported on Canada’s overall progress in closing the socio-economic gaps between on-reserve First Nations people and other Canadians. It also focused on whether the Department made adequate use of data to improve education programs to close the education gap and improve socio-economic well-being.”[1]

In the Message from the Auditor General, in the spring of 2018, the Auditor General (AG) opined that the lack of progress on the Aboriginal file is an “incomprehensible failure.”[2] In response, the Committee wants all Canadians to know that it takes this opinion very seriously; hopefully, the recommendations laid out in this report are steps in the right direction.

Per the OAG, in 2008, “the Department proposed that it would develop a system to capture First Nations’ education program data, which First Nations could access as well.”[3] This commitment was renewed in 2010 but data access by First Nations seems to have been limited[4]

According to the OAG, the “federal government has made numerous commitments to meet Canada’s obligations to First Nations, Inuit, and Métis people. Indigenous and Northern Affairs Canada was the lead government department responsible for meeting such commitments.”[5] In August 2017, the federal government “announced the dissolution of Indigenous and Northern Affairs Canada and the creation of two departments: Indigenous Services Canada, and Crown-Indigenous Relations and Northern Affairs. [The OAG] audited Indigenous and Northern Affairs Canada before August 2017 and, afterward, the newly formed Indigenous Services Canada, which is now the responsible department.”[6] This Committee report uses the new initialism, ISC, to refer to both departments.

According to ISC, in December 2017 there were about 1 million individuals registered under the Indian Act, half of whom lived on reserve or on Crown land. This report uses the term “First Nations people” to refer to First Nations people on reserves.[7]

ISC “requires First Nations to provide extensive data about their on-reserve members. The Department also obtains data from Statistics Canada, other federal government departments, Indigenous organizations, and other sources.”[8]

Additionally, the “Truth and Reconciliation Commission of Canada, created in 2008, had a mandate to inform Canadians about what happened in Indian residential schools. The Commission’s final 2015 report included 94 calls to action,” including those to “identify, measure, and close the gaps in health outcomes between First Nations and other Canadian communities; and prepare and publish annual reports on these and other issues.”[9] 

On 17 October 2018, the House of Commons Standing Committee on Public Accounts (the Committee) held a hearing on this audit.  In attendance, from the OAG were Michael Ferguson, Auditor General of Canada, and Joe Martire, Principal. From ISC were Jean-François Tremblay, Deputy Minister; Paul Thoppil, Chief Finances, Results and Deivery Officer; and, Shelie Laforest, Acting Senior Director, Program Directorate.[10]

Findings and Recommendations

A. Measuring Well-Being on First Nations Reserves

1. Inadequate Measurement of Well-Being

According to the OAG, the Department’s Community Well-Being Index “is its only publicly available overall measure of the socio-economic well-being of First Nations communities in comparison with other Canadian communities.”[11] The index is based on census data and measures four components: education, labour force participation, per capita income and housing (e.g., overcrowding and need for major repairs).

The OAG “found that the Department did not adequately measure well-being for First Nations people on reserves. The Community Well-Being index measured only four components of well-being.” Although these are important aspects of well-being, “the index did not include critical variables such as health, environment, language, and culture. First Nations have identified language and culture, in particular, as critical to their well-being.”[12]

According to the OAG, although the Department “recognized that the index was incomplete, it did not modify the index to make it more comprehensive or establish a more complete measure or set of measures for assessing the well-being of on-reserve First Nations people.”[13]

As Jean-François Tremblay, Deputy Minister, told the Committee, “What we need is a more comprehensive index. We’re working with First Nations and we’re developing it.”[14]

2. Use of Available Data

The OAG found that the “Department could have used the volumes of available data from multiple sources to more comprehensively compare well-being relative to other Canadians and across First Nations communities, but did not. The Department had data and access to data from public sources, other departments, and First Nations, but did not include it in either the Community Well-Being index or another comprehensive measure or set of measures.”[15] These data include the following:

• “the Department’s internal First Nation community profiles;
• Employment and Social Development Canada data on participation in skills training;
• Health Canada information on First Nations health;
• provincial health agency data;
• provincial data on how many Indigenous children were placed in foster homes; and
• the First Nations Regional Health Survey.”[16]

According to the OAG, “the Department should make better use of such data to more comprehensively assess and understand First Nations well-being in comparison with that of other Canadian communities, and to measure whether progress is being made.”[17]

For instance, the OAG used census program data to calculate “on-reserve and overall Canadian high school graduation (or equivalent) levels from 2001 to 2016.”[18] Although the “results for First Nations had improved, the results for all Canadians had improved by a greater amount: The gap was 30 percentage points in 2001 and 33 percentage points in 2016.” [19] Per the OAG, “this is a clearer way to measure and report on education results and would help to provide a more meaningful picture of well-being.”[20]

3. Incomplete Reporting on Well-being

The OAG found that ISC “did not issue a comprehensive report on the overall socio-economic well-being of First Nations people on reserves. In comparison, [the OAG] noted that, since 2009, the Prime Minister of Australia has issued annual reports, called Closing the Gap, with progress on improving the socio-economic well-being of Australian Indigenous people. These reports outline where more work is needed.”[21]

4. Lack of Meaningful Engagement

According to the OAG, ISC “did not work with First Nations to develop the Community Well-Being index, first published in 2004. Since then, [the OAG] found that the Department has not revised the index to include language and culture or developed another comprehensive set of measures to assess First Nations well-being.”[22] The OAG also found that “greater efforts needed to be made to measure and report on the overall well-being of on-reserve First Nations people compared with that of other Canadians.”[23]

As reported by Jean-François Tremblay, the data shows that “the Atlantic and B.C. are doing a bit better, and they are … managed regionally by First Nations on education …. There’s more involvement and engagement.”[24] This suggests that that outcomes are better when First Nations are involved in developing and administering such programs.

Therefore, the OAG made the following recommendation:

Through engagement with First Nations and other partners, Indigenous Services Canada should use relevant data to comprehensively measure and report on the overall socio-economic well-being of First Nations people on reserves compared with that of other Canadians. The Department should also measure and report on those additional aspects of socio-economic well-being that First Nations have identified as unique priorities, such as language and culture, which might not be directly comparable with other Canadians.[25]

In its Detailed Action Plan, ISC stated that it was working on a National Outcome-Based Framework—a draft of which has been distributed to Committee members—which has not yet been validated by First Nations. The Framework will be further developed in cooperation with the Assembly of First Nations and the First Nations Information Governance Centre and a second draft is “anticipated to be completed by March 2019.” The subsequent milestones consist of “engagement with First Nations on [the] co‑developed National Outcome-Based Framework,” which is expected to lead to the final Framework, “to be completed by March 2020;” and a first “reporting baseline of socio-economic gaps between [First Nations] and non-Indigenous Canadians, anticipated to be completed by March 2021.”[26] According to Jean‑François Tremblay, ISC’s goal is “to have co-developed baseline data on socio-economic gaps within three years so that we can start to systematically measure and report on our progress.” [27]

Therefore, the Committee recommends:

Recommendation 1 – Regarding measurement of well-being on First Nations reserves

That Indigenous Services Canada submit to the House of Commons Standing Committee on Public Accounts: 1) a first progress report on consultations and decisions regarding the National Outcome-Based Framework by 30 April 2019. This Framework should be developed in cooperation with First Nations and include measurements of those aspects of socio-economic well-being that First Nations have identified as unique priorities, such as language and culture; 2) a final report on this Framework by 30 April 2020; 3) a reporting baseline on the socio-economic gaps between First Nations and non‑Indigenous Canadians by 30 April 2021; and 4) starting in 2023, an annual report on socio-economic gaps by June 30 of each year.

B. Collecting, Using and Sharing First Nations’ Education Data

1. Not Collecting Relevant Data

According to the OAG, in 2010, ISC “developed the Education Performance Measurement Strategy to measure results for First Nations students and schools and to determine whether its students achieved education results comparable with those of other Canadian students.” [28]

The OAG found that:

• “the Department did not collect the information needed to compare First Nations’ education results with those of other Canadian students;”
• “the Department did not collect important data from First Nations on primary and secondary education programs;”
• “the Department did not have data about post-secondary graduation rates for on-reserve First Nations students. Nor did it collect complete data on the number of First Nations students who wanted to pursue post-secondary education but could not access the funds to do so. Despite commitments by the Department to review these issues in 2004, it was still unable to report on the extent to which its support for First Nations’ post-secondary education improved student results or whether its delivery model ensured that eligible students had equitable access to post-secondary education funding.”[29]

2. Not Assessing Relevant Data

The OAG also found that “the Department did not distinguish between high school diplomas and high school completion certificates. According to the Department, the certificates might not be recognized by post-secondary institutions.” [30]

Per the OAG, “by not distinguishing between official diplomas and completion certificates, the Department and First Nations could not determine how many First Nations graduating students might not be eligible to attend post-secondary institutions.” [31] 

3. Inadequate Use of Data to Improve Education Programs

In 2008, “Indigenous Services Canada identified literacy as a national priority.” [32] However, the OAG found that “despite collecting literacy information from First Nations for on-reserve students who participated in standardized testing, the Department did not analyze students’ literacy levels to determine whether its support improved results.” Furthermore, “the Department did not compare First Nations literacy levels with data from other Canadian jurisdictions and did not use this information to inform its support for literacy programs.”[33]

The OAG also noted “other examples where the Department did not adequately use data to modify its programs,” such as its University and College Entrance Preparation Program and its Elementary and Secondary Education Program; the latter is intended for students ages 4 to 21, but 17% of the students enrolled in the program were over 21.[34] Jean‑François Tremblay noted that ISC did not develop the preparation program “to actually improve the result. It was to make sure that they can have access to those services that were offered by the institution.” [35] In fact, the result “just tells us that more needs to be done.”[36] ISC is currently evaluating its “post-secondary education programs in collaboration with First Nations.”[37]

4. Inadequate Use of Data to Inform Funding Decisions

According to the OAG, through a “combination of core and supplemental programs, Indigenous Services Canada funded eligible on-reserve First Nations students to attend First Nations-operated, federal, private, or provincial schools. In 2000, the Department committed to developing and using comparative cost information.”[38] The OAG also found that, as of 2017, “this work was still not complete, and that the Department was still unable to report how federal funding for on-reserve education compared with funding levels for other education systems across Canada.”[39]

Additionally, the OAG reported that “in arriving at its core funding decisions, it was not clear to what extent the Department accounted for the unique needs of First Nations students.” These needs can influence costs, which are themselves influenced by factors such as a student’s first language (which is not always English or French), the difficulty in attracting teachers, and the geographic isolation of these communities.[40]

The OAG also noted that to “supplement its core education funding budget, the Department received an additional $2.6 billion over five fiscal years, from 2016–17 to 2020–21.” According to the OAG, the “Department’s analysis to support this amount was insufficient.”[41]

5. Limited Access by First Nations to the Department’s Education Information System

The OAG reported that a “December 2015 internal audit of the system found that First Nations had only limited access to reporting tools to compare, for example, how students in their communities were doing relative to those from other regions. Furthermore, First Nations received little training on how to use the system.”[42]

Jean-François Tremblay explained to the Committee that ISC was “going now with pilot projects where First Nations would have direct access to the data [and] we now have more than 45% of the First Nations,”[43] which means that more than half of First Nations do not have access to the data.

6. Heavy Administrative Burden for First Nations

First Nations representatives informed the OAG that “the reporting requirements related to education programs were burdensome. [The OAG] found that, in the 2017–18 fiscal year, the Department’s education reporting requirements included up to 13 forms with 920 data fields.”[44] As well, ISC “identified the need to reduce the reporting burden to First Nations. First Nations were required to provide 20 annual reports, using data gathered from about 1,500 sources.”[45]

On this matter, Jean-François Tremblay told the Committee that ISC was “negotiating regional agreements where we agree with the First Nations on what is the structure and what would be the data that would be collected.”[46]

Therefore, the OAG made the following recommendation:

Through engagement with First Nations and other partners, Indigenous Services Canada should collect, use, and share data with First Nations appropriately to improve education results of First Nations people on reserves.[47]

In his opening remarks, Jean-François Tremblay stated that there “will never be a ‘one size fits all’ solution for First Nations education. That is why [ISC is] working to strengthen regional approaches. A recent and bold example is the tripartite education framework agreement in British Columbia,” which ISC sees as its “mile zero for this kind of collaborative regional work which is more comprehensive.” [48]

Shelie Laforest, Acting Senior Director, gave another example of choosing the best indicators for education:

I can speak to the B.C. agreement, as it's a signed agreement. If you look at the agreement you will find indicators around connectivity and around literacy by grade. They have attendance rates in their framework. They have their teacher-student ratios. In the B.C. agreement there is a list of indicators. I'm going by memory, but those would be examples, for the B.C. context, that they have decided are appropriate measures to demonstrate results, from a B.C. perspective, as an example.[49]

According to ISC’s Detailed Action Plan, the first agreement is to be “completed in December 2018.”[50] Then, ISC expects “an estimated 10-20 Regional Education Agreements … [are] to be initiated by December 2019.”[51] In addition, national “targets co-developed with First Nations for K-12 education [should] be completed by March 2021.”[52] The updated results framework should be approved by June 2021.[53]

Therefore, the Committee recommends:

Recommendation 2 – Regarding the collection, use and sharing of education data concerning First Nations’ peoples living on reserve

That Indigenous Services Canada provide the House of Commons Standing Committee on Public Accounts: 1) a first progress report on regional education agreements that have been reached or are being negotiated, and on their content by 15 June 2019; 2) a second progress report on the same subject by 30 June 2020; and 3) a final report on all of the regional education agreements that have been reached and on national education targets for kindergarten to Grade 12 by 30 June 2021.

C. Reporting on First Nations’ Education Results

1. Incomplete Reporting

According to the OAG, in “2010 and 2014, Indigenous Services Canada developed Education Performance Measurement Strategies that committed to specific performance measures.”[54] The OAG found that “the Department did not report results for most of the measures. In particular, its 2014 strategy contained 23 specific measures, but [the OAG] found that the Department never reported on 17 of them. For example, it did not report on student attendance and First Nations’ language instruction.”[55]

On this matter, Jean-François Tremblay reiterated that the data were not reported because they “are not good quality and [ISC does not] have all of them.”[56] This is the key problem: there is a lack of reliable data for developing programs and evaluating their results. As the AG stated, “to close socio-economic gaps and improve lives on reserves, federal decision-makers and First Nations need information about the socio-economic conditions of First Nations people and program data that are reliable, relevant and up-to-date.”[57]

Additionally, the OAG found that “the Department’s reported graduation rate for on‑reserve First Nations students was inaccurate. It reported a graduation rate that included only students enrolled in their final year of high school. This meant that the reported graduation rate was overstated, because students who dropped out in grades 9, 10, and 11 were excluded from the Department’s calculation.”[58] The rate was 10 to 29 percentage points lower, depending on the year.[59] Notwithstanding the challenges of tracking students within or between jurisdictions, the Committee was concerned with the inaccurate reporting to Parliament.

Therefore, the OAG made the following recommendation:

Indigenous Services Canada’s reporting on First Nations’ education results should be complete and accurate.[60]

According to its Action Plan, ISC will approve the “updated results framework by June 2021”[61] (see its response to the OAG’s Recommendation 5.83). Consequential “adjustments to [the] Education Information System (or successor) [are] to be completed by June 2022.”[62]

The adjustments to the Education Information System depend on the updated results framework being approved by June 2021, and this framework must be submitted to the Committee by no later than 30 June 2021, according to Recommendation 2 of this report.  Therefore, the Committee recommends:

Recommendation 3 – Regarding Reporting on First Nations’ Education Results

That Indigenous Services Canada provide a final report on adjustments to its Education Information System to the House of Commons Standing Committee on Public Accounts by 30 June 2022.

Conclusion

The Committee concludes that Indigenous Services Canada has not satisfactorily measured Canada’s progress in closing the socio-economic gaps between on-reserve First Nations people and other Canadians, that it has not adequately reported on this progress, and that it has not made proper use of data to improve education programs.

The Committee has made three recommendations for Indigenous Services Canada to ensure that the Department can adequately measure the impact of programs on the real lives of Indigenous people by using the proper indicators, rather than focusing on the amount of money spent on the programs. The ultimate goal is to have programs that will reduce—if not eliminate —the socio-economic gaps between First Nations and non-Indigenous Canadians.

Summary of Recommended Actions and Timelines

Table 1—Summary of Recommended Actions and Timelines

Recommendation	Recommended Action	Timeline
Recommendation 1	Indigenous Services Canada (ISC) must submit to the Committee: 1) a first progress report on consultations and decisions regarding the National Outcome-Based Framework; 2) a final report on this Framework; 3) a reporting baseline on the socio-economic gaps between First Nations and non-Indigenous Canadians; and 4) an annual report on socio-economic gaps.	30 April 2019 30 April 2020 30 April 2021 30 June 2023, 2024, etc.
Recommendation 2	ISC must submit to the Committee: 1) a first progress report on regional education agreements that have been reached or are being negotiated, and on their content; 2) a second progress report on the same subject; and 3) a final report on all of the regional education agreements that have been reached and on national education targets for kindergarten to Grade 12.	15 June 2019 30 June 2020 30 June 2021
Recommendation 3	ISC must submit to the Committee a final report on adjustments to its Education Information System.	30 June 2022