:
I now declare this 36th meeting of the Standing Committee on Public Accounts in order.
Colleagues, you will recall that we followed our choice of chapters following the tabling of the spring 2014 report, and we're working our way through that. Today we have the NDP choice for the public hearing, and it is chapter 8, “Meeting Needs for Key Statistical Data—Statistics Canada”, of the spring 2014 report of the Auditor General of Canada.
I am pleased to advise that of course we have our Auditor General here, Mr. Michael Ferguson.
Welcome, sir. It's good to have you here again.
We also have the appropriate government officials here to give their testimony and answer questions.
I'll ask you to introduce yourselves and your delegation when I call upon you to give your opening remarks.
If there are no interventions from colleagues, or questions, or any reason why we should delay—and I'm not hearing any—we will then proceed. We will begin with the opening remarks of the Auditor General.
Mr. Ferguson, you now have the floor, sir.
:
Mr. Chair, thank you for this opportunity to discuss chapter 8, “Meeting Needs for Key Statistical Data”, from our 2014 spring report.
Joining me at the table are John Affleck, principal, and Colin Meredith, director, who were responsible for the audit.
As Canada's national statistical agency, Statistics Canada plays a fundamental role in generating information on Canada's socio-economic conditions. This information is used by governments, researchers, academics, businesses, and non-governmental organizations.
We examined whether Statistics Canada ensures the quality of key socio-economic information it produces and whether it generates this information efficiently and in response to priority user needs. Our examination of quality and the efficient use of resources was focused on four data products that provide important information on Canada's economic and social conditions: the consumer price index; the labour force survey; the national household survey; and the survey of employment, payrolls, and hours.
[Translation]
Overall, we found that Statistics Canada applied its quality assurance framework to ensure the quality of the statistical programs we examined. However, improvements are needed to better meet user needs.
We found that Statistics Canada consults mainly with federal, provincial, ans territorial users, with less attention paid to the private sector, municipalities and non-government organizations.
Without ongoing consultation and feedback mechanisms for the full range of users, the agency's data may become less relevant. For example, detailed information on job vacancies—by both occupation and location—could inform job seekers and policy-makers on which occupations are in demand and where. Policy-makers could also use this information to develop and manage policies and programs.
However, information on job vacancies is not broken down within provinces, so it is not possible to know if Alberta job vacancies are in Fort McMurray, Medicine Hat, or in any other community. Users told us that as a result of the shortcomings, this information is of limited value to them.
[English]
We found that the agency managed the implementation of the national household survey well. Nevertheless, introducing this voluntary survey to replace the mandatory long form questionnaire that was part of the previous census of population had an impact on users. The agency anticipated a drop in response rate as a result of the move to a voluntary survey. It took a number of steps to mitigate this risk, such as increasing the number of households in the initial sample from 3 million to 4.5 million, and targeting its efforts to encourage households to respond. Notwithstanding the steps taken, the actual response rate dropped from 94% in the 2006 census to 69%. While the number of responses to the 2006 census and the 2011 survey were similar, it is the response rate that determines how the data can be used.
Statistics Canada then took additional steps consistent with its quality assurance framework. It decided not to release data, especially for some sparsely populated, remote areas, or rural communities, because of low response rates. The agency estimated that reliable national data was unavailable from the national household survey for 3% of the Canadian population, up from 1% in the 2006 census. The agency also informed users that they should use caution when making comparisons between data from the national household survey and data from previous censuses.
[Translation]
We also examined whether the agency generated data for the four data products efficiently over the audit. We found that Statistics Canada had identified and implemented a number of approaches that resulted in more efficient use of resources. These included using common corporate services and identifying alternate data sources.
In order to maintain the relevance of its work, it is important for Statistics Canada to identify and respond to emerging needs. We examined how the agency assesses and addresses identified gaps, and how it ensures that its approved priorities are implemented.
[English]
We found that Statistics Canada had an adequate process to identify and implement its priorities, including emerging needs. However the agency did not systematically consider the need to consider cost-recovered work when external funding is withdrawn. For example, after work ended on a client-funded survey of older workers, the agency did not assess whether it should have continued to fund the survey out of its internal resources.
[Translation]
Statistics Canada has agreed with our five recommendations and has set deadlines for implementing them, ranging from April 2014 to January 2015.
Mr. Chair, this concludes my opening remarks. We would be pleased to answer any questions the committee may have.
Thank you.
Good afternoon, committee members.
[English]
I have with me at the table today Stéphane Dufour, the assistant chief statistician for corporate services at Statistics Canada.
As chief statistician of Canada, it is my pleasure to speak to you about the Auditor General's report in which he assessed how well Statistics Canada meets the key statistical information needs of Canadians and their institutions. I was pleased that the chapter on Statistics Canada demonstrated that overall, the agency makes efficient use of its resources to produce relevant and timely data of good quality. Statistics Canada agrees with the Auditor General's recommendations, and we have already begun implementation of our action plan.
In my remarks today, I would like to focus on some of the key issues raised in the report. Essentially these issues centre on the challenge of meeting the diverse statistical needs of a multitude of users, while at the same time ensuring a robust level of quality. As a fundamental principle, since Statistics Canada defines quality as fitness for use, Statistics Canada would never sacrifice quality to deal with budgetary constraints. If need be, Statistics Canada limits the scope of its statistical program, rather than compromise the quality of the information it produces.
[Translation]
In addition to the Censuses of Population and Agriculture, which are conducted every five years, the agency also has 350 active surveys and makes use of approximately 500 administrative data sources to inform Canadians on the economic, social and environmental conditions of their country. Statistics Canada is held in high regard internationally for the quality of its data, its methodologies and its efficiency.
[English]
Since the agency cannot accommodate all information needs within its base budget, we undertake statistical work on behalf of external clients, primarily federal and provincial government departments, on a cost recovery basis. This allows us to respond to emerging needs while ensuring that all Canadians benefit from the information that is ultimately produced. Many surveys have been conducted this way over several decades. Normally, this work ends if the client no longer wishes to fund it. On occasion, Statistics Canada will assume responsibility for funding a program that it considers too strategic to allow to be discontinued. The agency's ability to self-fund these programs is, of course, extremely limited.
The Auditor General has noted that this process of considering whether to self-fund a former cost recovery program has not been formalized at Statistics Canada. The agency has now put in place a mechanism, in the course of its annual planning process, to formally review cost recovery programs that are losing funding, in order to determine whether it is necessary to provide, or to try to secure, permanent funding. In the absence of new external funding, maintaining a cost recovery program using existing funding would, of course, mean discontinuing some other programs.
In our rapidly changing world, there is an unquenchable thirst for information. Statistics Canada receives demands not only for new data on a wide variety of topics, but increasingly for data at the local level, and for very small populations. To fully respond to this thirst for data would require many times the budget we presently possess. Even with unlimited funds at our disposal, the issue of the burden that responding to surveys would impose on the Canadian population and businesses would remain intact. It's necessary for Statistics Canada to balance the burden that we impose on Canadians and their businesses in responding to our surveys against the value of the information that would be obtained.
To address the growing need for data for small areas and small populations, I am pleased to report that Statistics Canada is making great strides by developing and exploiting state-of-the-art techniques, such as micro-data simulation and smaller re-estimation, to extract the maximum value from existing datasets. Our growing use of administrative data is also increasing our ability to provide data for small areas and populations. These techniques produce, at affordable cost, estimates at a detail beyond that of a single survey, without imposing an additional response burden on Canadians and their businesses.
[Translation]
In the Auditor General's report, there is some focus on the National Household Survey and the agency's decision not to publish local estimates for more than 1,100 communities, representing 3% of the population, due to quality concerns. Lower response rates relative to those of the mandatory 2006 census long-form, due to the voluntary nature of this survey, were the principal driver behind this decision.
While there has been some erosion of data quality due to lower response rates, the National Household Survey nonetheless produced a massive data base of robust information at all levels of geography and for many small populations. To seek, as some have, to dissuade Canadians from using this rich and powerful data source does them a terrible disservice.
[English]
Relevance is one of the underpinnings of a successful national statistical office. Statistics Canada seeks to align its statistical and analytical programs with the highest priority information needs of the nation. To accomplish this, we have always engaged a wide range of partners, users, and stakeholders in order to understand the revolving information needs and to ground our statistics in a genuine understanding of the phenomena we are trying to measure.
As recommended by the Auditor General, Statistics Canada will in future systematically document this outreach process and demonstrate clearly how it is taken into account in the agency's program planning.
As part of the agency's move to expand its consultative base, the membership of the national statistics council has recently been modified to broaden the number of voices at the table. This council sits at the pinnacle of a system of advisory committees and brings together distinguished Canadians with strong knowledge of Canada's national statistical system and a strong interest in participating in its development.
In addition, we have begun to formalize the renewal process for the agency's numerous advisory committees, to ensure they are properly constituted given their mandates, thus helping the agency remain aware of and responsive to evolving needs and statistical methods.
Let me conclude by saying that I believe our national statistical system is a national asset of great value, which produces returns for Canadians well beyond their investment, by providing them with high quality information about Canada's economy, society, and environment, information that they require to function effectively as citizens and decision-makers in a rapidly evolving world.
The quality of Statistics Canada's programs, the morale of its employees, and its international reputation for excellence are very much intact.
Thank you.
:
While we view Statistics Canada as a national statistical office and, therefore, try to meet the needs of the entire nation, all Canadians, the reality is that with the budget available our focus is primarily on government and primarily on the two most senior levels of government. A lot of our program certainly addresses the needs of federal departments, and the provincial and territorial governments as well. The census particularly addresses the needs of municipal governments.
Most of the rest of the use of our program is really residual use of the information that we've created in response to the needs of those governments, by other organizations. So non-government organizations use our information to, for example, develop positions on government policies, as do businesses. Businesses use our information—for example, the census data—extensively in their marketing efforts. Financial institutions use our information extensively in their analysis of financial markets and adjust their strategies in response to that.
We do meet regularly. I recently met with a group representing service industries to talk about our program and how it meets their needs and where they felt there were gaps that should be addressed.
I also participated recently in a business round table to talk about how our data is used in conjunction with, for example, big data sources in order to meet the needs of the private sector.
Every time we release the national accounts—the gross domestic product, for example—we meet with a table of senior economists from the private sector to talk about their views of the data we've just released, any concerns they have.
We've added three or four members to our national statistical council, who come from the business community, to hear their voices.
In our economic statistics programs and our advisory committees, we have representation from the business sector, and business associations as well. It's a very broad-based effort to make sure we're engaging with them.
:
There's absolutely no doubt that I would have preferred to have a 94% response rate or any response rate higher than 69%, but the fact that the response rate was 69% is not in itself a condemnation of the data.
Response rates really have two impacts when you start talking about data. One is an impact on the statistical variability of the estimates—when you think about polls and when you talk about how this estimate is accurate plus or minus 2%, 95 times out of 100. That's where the 69% comes in. If we hadn't adjusted the size of the sample as we did, that would have resulted in the estimates being of a much poorer quality from that perspective. Because we adjusted the size of the sample and went from a 20% sampling rate to a 30% sampling rate, we actually got, as the Auditor General noted, the same number of responses from households and Canadians—it was actually a slightly higher number—and that took care of that issue. In terms of sampling variability, the estimates from the 2011 national household survey, as we demonstrated in some documentation that we released regarding coefficients of variation, were roughly as good as what we got from the 2006 census.
The second issue is non-response bias. There was a possibility that, because the proportion of people who answered is smaller and significantly different from 100%, those people might be significantly different in terms of their characteristics in the population as a whole. A lot of claims have been made. A lot of people raised concerns about that possibility. We spent a very large amount of time as were were publishing and prior to publishing the data, looking at that issue.
I thank the witnesses for appearing before our committee.
I have an important question on vacant positions, mentioned in point 8.50.
Because of Statistics Canada's incapacity, if I may say, to answer that question, governments are unable to determine if a request to bring in foreign workers is justified or not. Currently, there are more than 100,000 foreign workers who occupy jobs that could be occupied by Canadians. The whole issue is rather concerning. In your reply to point 8.55, you seem to indicate that budgetary reasons are what prevent you from correcting the gap regarding vacant positions.
Would an increase in your budget allow you to resolve that problem, and thus allow us to settle the issue regarding foreign workers in a satisfactory way?
:
It's nice to be back. Folks, it's nice to see you as well.
I will just get right into it. In chapter 8, section 8.53, I was alarmed to hear some of the words being used:
25 percent of geographic areas do not have reliable National Household Survey data available for their use.
That is part of the process of going to the voluntary survey, obviously, when you talk about a 94% to 69% drop, which we've already talked about.
One of the things I find alarming is that, when you look at policies such as, say, the temporary foreign worker program, you have to come up with your labour market opinion in order to do this. I'll use an example that comes from the person I'm replacing, Ms. Yvonne Jones from Labrador. The unemployment rate in her area is around 5% or 6%, but in western Newfoundland it's well above that. It's around 13% to 15%. If you lump all this in together, as a result, you have businesses closing down in Labrador because they cannot find employees. This is a problem because we don't have a true reflection there.
To Mr. Ferguson, I think that's what you're pointing out in the survey. Is that correct?
:
Statistics Canada is very actively engaged internationally in defining all those standards, and defining them in ways that ensure the resulting statistical programs respond to the needs of Canadians.
The industrial classification we use in the survey of employment, payrolls, and hours, for example, is one that's negotiated between us, the Americans, and the Mexicans, to ensure it is well adapted to the structure of Canadian industry and therefore responsive to the needs of the data users.
The classification standard we use for commodities in international trade is also a product that we negotiate jointly with the United States and Mexico, to ensure that it responds effectively to the needs of Canadians.
We're one of the leading countries in the development of international standards and guidelines around consumer price indexes, and we have a very large investment, a five-year investment in improving the consumer price index because of the importance it has in monetary policy and in transfer payments, the management of income tax thresholds, the Canada pension plan, pensions, and so on.
In the case of the labour force, we're very engaged internationally in the definition of standards, guidelines, and methods. Our Australian colleagues are having some difficulty with their labour force survey. They asked us to give them some advice and some assistance in dealing with the issues they're currently struggling with as, in their view, we're the leading country in this area.
The other thing is that in any budget reduction exercise, we have protected all of them. We understand the importance. we understand the impact on Canadians of getting this right. Therefore we have protected these programs from any impact of any budget reductions Statistics Canada needs to make.
Mr. Smith, I want to go back to the calculation of the unemployment rate in first nations communities, because the information you gave me somewhat contradicts other sources of information that come from your services.
I can understand that 400,000 people in Canada may not have a large impact on the national unemployment rate, but this is hugely important for those 400,000 people.
Of the 1,128 census subdivisions on whom data was not published, only 36 were aboriginal communities. Of the data on 608 census subdivisions that included aboriginal communities, only 36 were not viable and were not released.
You tell me that a financial problem is what prevents you from collecting that data, but how is it that all of the other information is good, except the data on the unemployment rates? I would not like us to wind up with a type of segregation through silence. Even if the process goes back to 1944, currently this information is urgently needed to solve social, economic and political problems those communities are experiencing.
:
I think we are talking about two different things.
For the National Household Survey, which did cost some $650 million, we went systematically to visit all of the reserves in Canada that allowed us to do so, because not all of them do. In some cases there were forest fires or flooding that prevented us from going onto the reserves.
On the reserves where we were granted access, for the National Household Survey, we collected data not on a population sample, but on 100% of the population. Those are good data that also include figures on the working population and the May 2011 unemployment rate.
As for the Labour Force Survey, it was carried out on 56,000 households throughout the country. So this represents only a small fraction of all households. That survey does not include all the reserves. If we went to the reserves for that survey, the data would not include all of the reserve households, but only some of them. Consequently, we would never publish data on those reserves. The same thing applies when we go to Calgary, for instance. For the Labour Force Survey, we do not go to every household in town, buy only to some of them to represent all of the households.
So, even if we went to reserves for the Labour Force Survey, we would not have data on every reserve. The impact of that data on the national, provincial or economic regions unemployment rate would be minor and almost imperceptible.
Be that as it may, I quite agree with you. If we could produce monthly data on reserves, that would be extremely useful and desirable, but we would have to allocate $650 million to that every year, which is not realistic.
:
Yes, that is the figure I have.
If we ever want to find solutions for these problems, we will need exact figures. Your participation would be helpful in that regard.
As for the constitution of your committees, your support committees more particularly, the Auditor General indicated that they are too often made up of academics, or of federal, provincial and municipal public servants, exclusively.
Auditor General, would it be pertinent to ensure not only that they come from various fields, but also, as regards the consumer price index, that these people not be economically orientated to satisfying a particular market, that is to say employers rather than unions, workers.
The Auditor General, at paragraph 8.50 of his report, spoke of job vacancy data and of being informed by users that because of “shortcomings, available information on job vacancies is of limited value to them”.
The Auditor General, at paragraph 8.55, made a recommendation that you “should assess the feasibility of more fully addressing user needs for data from small areas and subpopulations”.
Then I read that, in fact, you have begun work on a quarterly job vacancy survey.
I'd like to connect those dots. Is this quarterly job vacancy survey intended to address this issue, and if so, how? I know it focuses on more numbers of employers surveyed, and it focuses on data by economic region. Could you expand on how this will address the issue the Auditor General has raised?
:
Well, they play two roles, or more than two roles, actually. Some of the advisory committees are really committees of experts who are knowledgeable about such things as consumer price indexes or survey methods. We have an advisory committee on statistical methods composed of leading statisticians from around the world, particularly from the United States and Canada. That committee isn't designed to be representative of anything except experts who can advise us on our methods.
We have another committee on consumer price indexes, which is really of the same nature. Its principal purpose is to advise on methods, and it is populated by experts.
Other advisory committees on, for example, social issues are really there to advise us on the design of our programs, the design of specific surveys, and data gaps. That's a case in which, as the Auditor General has pointed out, we should have broad representation not only from academics, but also from the full range of users, including non-government organizations. Even businesses have some level of interest, as do municipal governments again.
That's fair commentary; they're not well represented in our advisory committees.
:
Sorry, Mr. Woodworth, time has expired. Thank you.
Colleagues, that wraps up the rotation of questions we have.
I thank our guests for being here. I don't believe there were any outstanding issues for follow-up information. There are no loose ends that I'm aware of. I think the committee has what it needs to consider a draft report.
Mr. Ferguson and your staff, and Mr. Smith and yours, we thank you both for being here. You gave full and complete answers, and we appreciate that very much.
With that, I will dismiss you and tell you that we look forward to seeing you next time.
Colleagues, I'm proposing that we suspend for a moment, go in camera, and then do some committee business. If we're in agreement on that, we now stand suspended.
[Proceedings continue in camera]