HEAL Committee Meeting

37th PARLIAMENT, 2nd SESSION

Standing Committee on Health

EVIDENCE

CONTENTS

Monday, April 28, 2003

¹

1540

The Chair (Ms. Bonnie Brown (Oakville, Lib.))

Ms. Maria Barrados (Assistant Auditor General, Office of the Auditor General of Canada)

¹

1545

The Chair

Mrs. Carol Skelton (Saskatoon—Rosetown—Biggar, Canadian Alliance)

Ms. Maria Barrados

Mr. Jerome Berthelette (Principal, Office of the Auditor General of Canada)

Mrs. Carol Skelton

Ms. Maria Barrados

Mrs. Carol Skelton

¹

1550

Ms. Maria Barrados

Mrs. Carol Skelton

Ms. Maria Barrados

Mrs. Carol Skelton

The Chair

Ms. Hedy Fry (Vancouver Centre, Lib.)

Ms. Maria Barrados

Ms. Hedy Fry

¹

1555

Ms. Maria Barrados

The Chair

Mr. Jeannot Castonguay (Madawaska—Restigouche, Lib.)

Ms. Maria Barrados

º

1600

Mr. Jerome Berthelette

Mr. Jeannot Castonguay

Ms. Maria Barrados

Mr. Jeannot Castonguay

Ms. Maria Barrados

º

1605

Mr. Jeannot Castonguay

Ms. Maria Barrados

Mr. Jeannot Castonguay

The Chair

Ms. Hélène Scherrer (Louis-Hébert, Lib.)

Ms. Maria Barrados

Ms. Hélène Scherrer

Ms. Maria Barrados

º

1610

Mr. Jerome Berthelette

Ms. Hélène Scherrer

The Chair

Ms. Yolande Thibeault (Saint-Lambert, Lib.)

Ms. Maria Barrados

Ms. Yolande Thibeault

º

1615

The Chair

Ms. Maria Barrados

Mr. Barry Leighton (Principal, Office of the Auditor General of Canada)

º

1620

The Chair

Ms. Maria Barrados

The Chair

Ms. Maria Barrados

º

1625

The Chair

Mr. Barry Leighton

Ms. Maria Barrados

The Chair

Ms. Maria Barrados

The Chair

Ms. Hélène Scherrer

Ms. Maria Barrados

º

1630

Ms. Hélène Scherrer

Ms. Maria Barrados

Ms. Hélène Scherrer

Ms. Maria Barrados

The Chair

Mrs. Carol Skelton

Ms. Maria Barrados

Mrs. Carol Skelton

Ms. Maria Barrados

º

1635

Mrs. Carol Skelton

Ms. Maria Barrados

Mrs. Carol Skelton

Ms. Maria Barrados

Mrs. Carol Skelton

The Chair

Ms. Hedy Fry

Ms. Maria Barrados

Ms. Hedy Fry

Ms. Maria Barrados

Ms. Hedy Fry

Ms. Maria Barrados

The Chair

Ms. Maria Barrados

The Chair

Ms. Maria Barrados

º

1640

The Chair

Ms. Maria Barrados

The Chair

Ms. Maria Barrados

The Chair

º

1645

Ms. Yolande Thibeault

The Chair

Ms. Hélène Scherrer

The Chair

Mrs. Carol Skelton

The Chair

Mrs. Carol Skelton

The Chair

Mrs. Carol Skelton

The Chair

Mrs. Carol Skelton

The Chair

Mrs. Carol Skelton

The Chair

Mrs. Carol Skelton

The Chair

Mrs. Carol Skelton

The Chair

---

CANADA

Standing Committee on Health

---

NUMBER 030

l

2nd SESSION

l

37th PARLIAMENT

---

EVIDENCE

Monday, April 28, 2003

[Recorded by Electronic Apparatus]

¹  (1540)  

[English]

    The Chair (Ms. Bonnie Brown (Oakville, Lib.)): Colleagues, we're meeting with the Auditor General's people to hear what they wrote after their audit of certain parts of the health department. It's supposed to be our preparation meeting prior to our three meetings on the estimates so that we can ask some intelligent questions of the officials when they come.

    I want to welcome the people from the Auditor General's office, particularly the Assistant Auditor General, Ms. Barrados, and the two prinicpals, Mr. Berthelette and Mr. Leighton.

    Please go ahead, Ms. Barrados.

    Ms. Maria Barrados (Assistant Auditor General, Office of the Auditor General of Canada): Thank you, Madam Chair.

    First, an apology. In the opening statement that is distributed, it somehow got through the edit process as “Mr. Chairman”, and I fully understand that we have a very capable Madam Chair here.

    Thank you for this opportunity to discuss two chapters of our December 2002 report--chapter 1, on first nations reporting to federal organizations, and chapter 6, on the management of health statistics quality.

    With me today are Jerome Berthelette and Barry Leighton, the principals responsible for these chapters.

    Our health statistics audit examined the quality of the statistics that Statistics Canada, Health Canada, and the Canadian Institute for Health Information provide for the federal, provincial, and territorial reports on health indicators. These audits resulted from an agreement by first ministers in September 2000 to have independent third-party verification of the accuracy of the health indicators data.

[Translation]

    The Auditor General verified the accuracy of the health indicators reports for Canada, Nunavut, the Northwest Territories, and Yukon. Health Canada's report included measures of health status, health outcomes and quality of health care services provided to the total Canadian population and to specific populations receiving federal health services, such as First Nations.

    We worked closely with our colleagues in the provincial audit offices who carried out the verification of their provinces' reports. In a joint team effort led by the Office of the Auditor General of British Columbia, we worked on statistics produced by the Canadian Institute for Health Information on seven of the health indicators.

    We found that Statistics Canada databases met standards for quality assurance, with the exception of the Hospital Person-oriented Database, which relies on CIHI data. From our review of the two CIHI databases, we concluded that the quality assurance processes were not documented adequately to be able to form an opinion on the quality of the indicators.

    We found that Health Canada's quality assurance systems and practices for its national surveillance systems were inadequate. They did not assure that the data were accurate and therefore we could not form an opinion on their accuracy.

[English]

    These work initiatives were new for the preparer of the reports and the auditors alike. We worked closely together to produce these first verified reports. The 2003 agreement, which builds on the 2000 agreement, confirms the continuing commitment to produce reports on health indicators. We believe legislative auditors can continue to play an important role.

    Health Canada recognized in its report that the collection of health data for first nations populations faces several challenges. We found that quality assurance processes for these indicators were inadequate to ensure the accuracy of the data.

    As we noted in our chapter on first nations reporting, first nations that sign multi-year contribution agreements with Health Canada are required to report annually on programs and services, including mandatory programs such as communicable disease control, environmental health, and treatment services. First nations are also required to prepare several reports on program activities, with detailed information on the numbers and types of health services provided in their community.

    The Health Canada officials we met with in the Saskatchewan region indicated that in many cases they do not rely on non-financial reports, such as annual reports, when dealing with first nations. Notable exceptions to this practice include some targeted programs such as the tuberculosis elimination strategy. That program centralizes control and measures how effective it is.

    The first nations we visited were concerned about the number and frequency of the reports required by Health Canada. They also told us they received no feedback on the reports from Health Canada officials in the Saskatchewan region. Health Canada's lack of attention to many of the non-financial reports it requires of first nations calls into question the purpose of the reports.

    That concludes my opening statement, Madam Chair,. We'd be pleased to answer any questions the committee may have on these issues or on issues related to the estimates.

¹  (1545)  

    The Chair: Thank you, Ms. Barrados.

    We'll begin the questioning with Mr. Merrifield--or Ms. Skelton, fine.

    Mrs. Carol Skelton (Saskatoon—Rosetown—Biggar, Canadian Alliance): I have a couple of questions.

    Being from Saskatchewan, I really want to ask about the concerns of our first nations people, who say they are required to do so much paperwork when a lot of times they feel the paperwork or their reports aren't looked at. The Health Canada officials themselves have told you that they do not reply on the non-financial reports, such as annual reports, when dealing with first nations people.

    Could you further expand on that whole issue? Did they give you any reasons why they weren't using the reports or that somebody was not going through these annual reports?

    Ms. Maria Barrados: Madam Chair, I'll start, and I'll ask Mr. Berthelette to perhaps explain some of the details of what we found in the work we did in Saskatchewan.

    The results of that work were largely done with the cooperation of the first nations people in Saskatchewan. As you know, we don't have an audit mandate on the reserves, and we did this work in a cooperative manner. Our objective was to try to count all the reports required not only by Health Canada but also by other departments. We concluded that it was a very large number of reports that put excessive burden on these first nations communities.

    The problem with the reporting requirements is that if you look at each individual program--if you look at the individual Health Canada programs, for example--they make a lot of sense. You can understand why you would want to have information on what occurred, why you would want to have plans, and why you would want to have results reported. But when you map them all together, you get a picture portraying that it isn't really workable.

    There is a number of specific things with respect to Health Canada. One is that a number of the programs are overlapping, so you have intervention programs that are fairly similar. For instance, you have a program directed to drug and alcohol abuse, or some of those particular areas, and then you have another program that goes to a similar population. Yet each requires a separate reporting requirement.

    So the program structures are a problem. It would be a good area in which to question officials--how many programs they have in a particular area, and how they manage these together--because it forces a number of communities to actually get the money, pool it, and then do reporting that is rather artificial.

    On the Health Canada side, on the officials side, they feel that they don't have the resources to either do the work they would like to do with first nations in order to get the kind of reporting everyone would like or to study these reports.

    Perhaps Mr. Berthelette would like to add to that.

    Mr. Jerome Berthelette (Principal, Office of the Auditor General of Canada): I think that covers it off.

    From the governmental officials' perspective, they receive a lot of reports. Each community, as we noted in the study, would have to produce about 60 reports. So if you have about 50 communities in Saskatchewan and you multiply that by 60 reports for Health Canada, they have to look at a lot of reports.

    As Madam Barrados has said, the officials have told us they don't have the resources to be able to look at them, so they prioritize them. They focus on the financial side and make sure the financial reporting is correct, and they follow up on those.

    Mrs. Carol Skelton: Basically they're tending not to look at the overall health programming, then, but going more towards the audit side, the finances. Is that a correct statement?

    Ms. Maria Barrados: We are concerned, in the work we have done on both the first nations side and the health surveillance side--another area your committee may want to ask officials some questions on--about how good those numbers are on the results, on what you are actually accomplishing.

    For management and administrative purposes, it's important to know moneys, and to know what you're doing in the short term, but that all has to head toward what kind of overall result you're trying to achieve. We raise concerns about the quality of that kind of information, both for general surveillance for public policy and the specifics on the first nations side.

    Mrs. Carol Skelton: You said that our first nations people told you they weren't getting feedback on the reports they were giving. Did Health Canada say that was because of lack of funding? Is that what they said on why they weren't communicating back to our first nations people?

¹  (1550)  

    Ms. Maria Barrados: Officials by and large expressed the concern that they don't have the resources to do this kind of work. However, the other concern they expressed was how useful the reports are.

    You have this situation where you have a requirement to produce reports. You get the reports, but the reports aren't that good because of the capacity people have to fill in the reports. So you're inclined not to make the time to deal with the reports because they aren't good enough.

    You can see what kind of vicious circle it gets to be.

    Mrs. Carol Skelton: I've heard quite often too that the reports are demanded on such short notice from Health Canada that a lot of times there just isn't adequate time to have the reports done properly. As well, because funding is so late in the year, lots of times they scramble to get everything in place.

    Is that something that was mentioned?

    Ms. Maria Barrados: We found instances of that too, particularly in the housing. I don't recall, although perhaps Jerome does, whether we noted that particularly in Health Canada. It's when additional moneys come in at the end of the year.

    But there is this real problem of setting up a system where you can actually measure things in any meaningful way. You can't always tie it to small interventions, and it gets to be very small kinds of interventions.

    Mrs. Carol Skelton: Thanks. That's it for now.

    The Chair: Ms. Fry.

    Ms. Hedy Fry (Vancouver Centre, Lib.): Thank you very much.

    I notice you have concerns about three databases--the notifiable disease reporting system, the national diabetes surveillance system, and the Canadian tuberculosis reporting system. You then went on to say that this is a voluntary system of reporting from the provinces and territories.

    Do you believe you can strengthen...especially with communicable diseases, and especially when we're talking about things like tuberculosis and in the light of SARS and other communicable diseases? Do you believe there is a role here for the federal government to mandate that the provinces report, to change the reporting structure for provinces and territories? Do you believe that would assist in creating a real and solid database?

    There's no way you can track any communicable disease and be able to contain it if you don't have some sort of clear, mandatory reporting, if it's just left up to provinces to make a decision about whether they will or will not. What are your feelings about a national mandate to do these?

    Ms. Maria Barrados: Thank you very much, Madam Chair.

    We have been concerned about the state of national health surveillance systems. We had done an audit in 1999 and we did a status report in September 2002, and we in fact spent some time discussing the role of the federal government in the whole area of public health.

    This is an area where there isn't a piece of legislation, as you well know, and there isn't really a policy that everyone has agreed to. There are drafts, and we're working at it, but there isn't really a policy that articulates what the federal role is.

    When we look at all the players, and there are many players in this area, you can't have one person doing it all, obviously. You have the health care providers, you have the laboratories, you have the municipal levels, you have the provincial levels. They all have a contribution to make here, but who is the coordinator? Who is the one who leads? Who is the one who worries about the agreements, worries about the standards, worries about having a similar set of definitions? We feel that this is a very important role for the federal government, and that's the tenor of our report in September.

    Ms. Hedy Fry: Could I add a second question to that? For many years, a lot of people I know, medical associations, for instance, have called for the creation of a surgeon general's office that is arm's length and completely separate and apolitical, that in fact would be able to mandate these things, to be in charge of keeping this data, and to deal with public health emergencies when they arise. The politics of the thing would not get mixed up with the public health and communicable diseases component of the thing.

    If you look across the border to the United States, their Surgeon General plays an extremely important role in being able to flag, without any political repercussions whatsoever, what are in effect public health issues. Do you see a role there?

    You said that you felt CIHI needed better documentation. What teeth does the Canadian Institute for Health Information have to be able to demand better information? Is it that they don't have any teeth, that they just have to ask and people may or may not decide to, because everything is so voluntary?

    Again, I'm back to this voluntary data provision, and back to the question of having a surgeon general with a clear mandate and with teeth. What is your opinion of that kind of suggestion?

¹  (1555)  

    Ms. Maria Barrados: I'm afraid I can't really venture on the surgeon general, per se, because it's the type of machinery of government argument we tend not to venture into. But I think we are very strongly convinced that there has to be some real and clear leadership from the federal government in the area of public health, however that's done.

    We talk about legislation, or a clear policy framework, and however that then gets implemented.... A surgeon general could be one way that is done.

    On the notifiable diseases, those are required to be reported, but as we said, and as you said, Dr. Fry, there really is difficulty in getting people to report these things in a timely manner. There have been attempts with the sentinel physician system, and there are now attempts through the laboratories, where you've seen some improvements. Our concern has been that these improvements are very slow. They're trying to design a system that shows a lot of promise, so often you will hear descriptions of a future system that is very promising indeed, but in the meantime you're dealing with today, and the issues today in terms of getting the information.

    With regard to CIHI, in fact CIHI is not quite the information system that would answer the questions on surveillance. CIHI is a statistical agency, and they in fact are operating, under the powers of the Statistics Act...as contribution agreements from Statistics Canada. So they are not that kind of body. As a body they collect a series of data sets that Statistics Canada used to collect in the past, but they're doing it as a statistical body.

    The Chair: Mr. Castonguay.

[Translation]

    Mr. Jeannot Castonguay (Madawaska—Restigouche, Lib.): Thank you, Madam Chair.

    I am struck by the vast quantities of reports that are required. If I had to write these reports, I would probably have to work 24 hours a day all year long. We are talking about one department here, and there are a host of departments that require almost the same thing from everyone.

    I think that our biggest challenge is simplifying things, using the KISS principle. These reports are repetitive. We see that in this field. People back home come to see me and tell me that they must fill out all kinds of reports week in and week out and that it makes absolutely no sense. Moreover, we are told that these reports are virtually worthless because there are too many of them and the department does not have the resources necessary to do any follow-up. I wonder if, despite how useful these reports should be, they are actually useful.

    How could we simplify matters and work horizontally? Will the various departments eventually talk to each other instead of each requiring these reports to be filled out? Do you have any practical suggestions for simplifying matters and making the system much more effective?

    Ms. Maria Barrados: Thank you very much.

    It is very important for us to try and simplify the reporting system. For starters, that will require better coordination within the federal government among the various departments and agencies. We must simplify and coordinate. Each program has a legislative or policy basis, and we must put these aspects together to simplify the objectives and programs.

    Mr. Berthelette can tell you about the specific suggestions we made in the audit report.

º  (1600)  

    Mr. Jerome Berthelette: We made three recommendations to the various departments in the report.

    The first is to consult First Nations to determine which reports are the most important and which message, in the reports, is the most important for the government, departments, and First Nations. First nations are currently having trouble with the messages contained in these reports. They are not using the reports, because the department has imposed these without first consulting them.

    The second recommendation is to use new technologies. In Saskatchewan, most of the reports are hand-written. New technology is not being used to improve the situation. That is also a major recommendation.

    The third recommendation is to conduct a true review of programs and services. As Ms. Barrados said, there is not a whole lot of difference between the various programs and services. There are many reports and most concern small populations of less than 500 people. It is not necessary to have 200 reports for a population of 500 people. The department must therefore examine all programs and services and streamline them.

    Mr. Jeannot Castonguay: Sixty reports for a population of 500 is approximately 8 reports per person. To my mind, that is an unacceptable number.

    I am trying to see what could be done practically speaking. Technology could perhaps be used. The name, address and other kinds of information are all the same in many of these reports. Our role will be to see not only how all of that can be simplified at Health Canada, but also how it can be integrated. It would be a very good idea to integrate all of this information in the various departments. Do you think this is an approach we should explore and one that holds promise?

    Your work is to audit Health Canada as well as other departments. Do you see similar difficulties in other departments?

    Ms. Maria Barrados: It is even worse in other departments. Reports relating to health are not the only ones creating a problem in these departments. So there is a very large number of reports. I think it would be a good idea to ask the people from Health Canada about their coordination and the integration of their programs with those of other departments, especially as regards First Nations. There are programs in the other departments, and we hope that the various programs are linked.

    Mr. Jeannot Castonguay: How did Health Canada respond when you made these recommendations?

    Ms. Maria Barrados: We made this recommendation to the government in the chapter dealing with reports. The response was positive in that it was recognized that there was a problem. They promised to try and improve the situation, but that takes time. We will have to push this...

º  (1605)  

    Mr. Jeannot Castonguay: But they were nevertheless open to your recommendations.

    Ms. Maria Barrados: Yes. The Treasury Board and the Department of Indian Affairs were receptive, but in terms of specific programs, changes must be made. That is another story.

    Mr. Jeannot Castonguay: That's another kettle of fish. Thank you.

    The Chair: Ms. Scherrer.

    Ms. Hélène Scherrer (Louis-Hébert, Lib.): I would like to go back to the programs and ask you to expand on those reports. Are those reports used to get a health profile of certain First Nations groups, to see whether the money invested in those programs produced the expected results or to see whether the money was used for its intended purpose? If you are telling me that you used the reports to determine how much money you will invest, some people will say that a given program was not a roaring success and that it is therefore perfectly acceptable to reduce the program funding.

    So is the purpose of these reports to get a health profile of First Nations or to use a program's results to determine the amount to be invested? If that is the case, I can understand why people don't feel like answering or that the reports might sometimes be biased. If the result automatically translates into less financial support, they are not given the choice. They will give the answers the department wants to hear.

    Ms. Maria Barrados: In our report, there is a section that refers to all reports, both financial and non-financial. It includes reports on plans, on support for proposals, on activities, on the census, on assessment and oversight. We made a list of all of the reports to get an idea of how many were required.

    We audited Health Canada's level of support for native programs. The department has agreements with the First Nations whereby they must produce health plans, but we did notice that plans had been renewed without reviewing earlier plans and without Health Canada's input. The purpose of the plan is to have some basis for discussion in order to implement programs. We noticed that insufficient effort had been made to establish those plans.

    As for the subsidies and contributions, First Nations are asked to fill in forms and to meet certain conditions. We also found some gaps in that regard when it came to preparing reports. We also noted that there was no direct link between the programs and the subsidies and contributions. There are very few people for a huge number of programs, especially for children, for example. Independent reports should be made.

    Ms. Hélène Scherrer: My question may be irrelevant, but has Health Canada ever thought of going to the different communities and doing on-site reports, in order to get a better idea of how healthy the people are and to produce reports based on much more accurate information, instead of sending forms to people and waiting for their answers? Everyone knows the answers are incorrect and that the forms will not be filled in. Everyone knows those statistics are useless. I get the feeling it is a totally pointless exercise.

    If people have known for years that it is pointless, why do you keep doing it? Why not send a Health Canada representative to do an on-site check?

    Ms. Maria Barrados: We have noted significant differences between the regions. Mr. Berthelette might want to add something on that point.

º  (1610)  

    Mr. Jerome Berthelette: Health Canada is in the process of renewing the structure of the reports. The department has produced a document entitled First Nations and Inuit Health Program Compendium. It would be a good idea for you to meet with Health Canada officials and ask them how this compendium will change the reports submitted by the communities, as well as the exchange of information among First Nations. For now, as you put it so well, the information is not very good and there is no mention of results in the reports.

    The objective of the compendium is to change this situation. It may be a good idea for you to talk about the issue with officials.

    Ms. Hélène Scherrer: We seem to be caught in a vicious circle. You don't get the right information and when this information finds its way into a budget or elsewhere, people end up complaining that the money is not invested where it should be. The money is sent back and the reports get it wrong. If this goes on, no one will ever be happy on either side.

    The government will think that it is not investing the money where it should be or that it is not getting the expected results and, for their part, First Nations will reply that they are sending reports to the government, but that the government is not putting the money where it is needed, because it is relying on insufficient information with regard to the affected programs.

    I don't think First Nations will be eager to cooperate if we reprimand them and say that from now on, they will have to fill out a report each year, which is a heavy task. Health Canada will have to take the initiative by getting the relevant information itself instead of waiting for reports which never come. The department will have to get the information itself in order to ensure that the money is invested where it is needed.

    When First Nations realize that the money is being invested wisely, they may be more open to partnerships. You get the impression that each side is complaining that the other is wrong. Unless something is done, the situation will never change.

    Thank you.

[English]

    The Chair: Seeing no further hands, if I could ask this question....

    I'm sorry, Madam Thibeault, I didn't see you .

[Translation]

    Ms. Yolande Thibeault (Saint-Lambert, Lib.): Thank you very much.

    In the report entitled "Chapter 1: Streamlining First Nations Reporting to Federal Organizations", in the highlights, it says: "The reports contain information that does not reflect community priorities." What exactly does that mean?

    I really wonder about that. Does it mean that communities are providing inaccurate information? It could be interpreted in other ways. Perhaps the information being requested is not the right information. Is it one or the other, or a bit of both?

    Ms. Maria Barrados: When we ask native communities whether they want to manage the information and whether they need information on health outcomes, the answer is always yes. They absolutely need to have this information. Most communities are willing to share the information they need with federal departments.

    But what they find frustrating is having to put in a request for their own information and its management and to make another request for the federal government. This issue speaks to their frustration because they do not have the information they need to manage their own affairs.

    Ms. Yolande Thibeault: I understand.

    Thank you very much.

º  (1615)  

[English]

    The Chair: Thank you.

    So many reports of late have talked about the need for a good statistical base to make decisions around planning for health care in Canada, whether it's the Romanow report, the Kirby report, or whatever. I think with the establishment of CIHI a few years ago, Canadians feel there is this agency that is doing some of what Statistics Canada used to do, and that is collect all this very relevant health data. That's where I would think the average Canadian's mind is on this. And you're telling us that the information CIHI produces is “inadequate”; I think that's the word you used.

    So if in fact moneys have followed the assignment of tasks from Statistics Canada to CIHI, and if Health Canada also puts other money into CIHI, are Canadians getting their money's worth?

    Ms. Maria Barrados: Madam Chair, that's an interesting set of questions.

    First, CIHI is one of these bodies put together by the federal government and the provincial governments, and is not in the federal sphere. In other words, it was set up in a totally independent manner. It does not report to any minister, and it does not have an audit requirement from any legislative auditor. So we at the Office of the Auditor General have been quite concerned about the accountability requirements of these new bodies.

    CIHI is one of those bodies that is only required to have a financial audit. It is not required to have any of the value-for-money performance audits. That has been a concern to us, because it has been our experience that public funds usually involve a desire on the part of members of Parliament to be scrutinized and to have a form of assurance by the Auditor General. So CIHI is away from us, and is not part of our audit realm.

    However, when the reports were produced...and members may want to talk a little bit about these reports. Last year was the first time these reports were produced as a result of the first ministers' agreement. But this is now the accountability we're going to see coming with the billions that are being spent through the health accords. This is the way it's going to be done.

    CIHI produces some of that information. One particular data series, that came from the abstracting of patient charts, forms numbers into these reports. Because of our role of verifier for these reports, and auditor for Statistics Canada, although we're not the auditors for CIHI--nobody's the auditor for CIHI--CIHI did agree to have us come in and look at only that database. So our comments have to do with that one database and the data series that come from that database.

    So it would be incorrect for me to say there's a quality problem with all of CIHI data. There was a quality problem with that particular data series in that we couldn't get the assurance we expected to see in the statistical agency about the quality of the data.

    There are some questions, though, that arise, and I'll ask Barry Leighton to speak a little more about that.

    CIHI is funded by the federal government, and there are specific contribution agreements given by the federal government, some from Health Canada and some from Statistics Canada. The funding for CIHI flows, I believe, through Health Canada. The first set of money was given on the condition that there be an evaluation. This evaluation didn't occur, and the second set of money was given.

    So we have specific recommendations requiring more of these things and you may...[Technical difficulty--Editor].

    Barry, do you just want to take a minute on the specifics of those?

    Mr. Barry Leighton (Principal, Office of the Auditor General of Canada): Thank you, Madam Chair.

    Yes, there was a contribution agreement where Health Canada provided about $4.7 million in 1994-95 for about a four-year period. Then, as Madam Barrados mentioned, there was an expectation under the terms of the contribution agreement that this would be evaluated, and that was not done before the renewal, which was another $13.4 million for the next four years.

    So we have concerns that the terms of the contribution agreement were not respected, but perhaps equally so that we didn't have any feedback about the effectiveness of the program, which would be produced by an evaluation.

º  (1620)  

    The Chair: The thing that concerns me is that we keep referring to CIHI as a statistical agency. Granted, it's arm's length, etc., and I'm happy they cooperated and let you come in and audit at least one section of their work, but the fact of the matter is, a lot of taxpayer money is flowing to it.

    I forget where I read it, but somewhere in here you say, essentially, they won't stand up for their own statistics, they won't verify their accuracy, and therefore you can't.

    Well, why are we putting all this money into a statistical agency whose statistics nobody will stand up and say are wonderful?

    Ms. Maria Barrados: As we note in the audit report, the 1999 budget gave them $95 million, and the subsequent budget gave them a further amount, about the same amount of money.

    What we expect of statistical agencies is the same kind of thing we expect of Statistics Canada. We expect them to have a quality assurance system and to document that quality assurance system, to do their own reviews to see how they're doing on it.

    A lot of these numbers have a certain amount of judgment, and they're not always as precise as we want them, but we expect the bodies to work with source inputs and report how good that number is. That way, when you're looking at it, you know how reliable it is. What we found in the case of CIHI is that they just didn't have in place, for that one data series we looked at, the kind of quality assurance systems we expected to see.

    Now, they are very professional people, working very hard, but there are a lot of issues around the abstracting that goes out of patient charts, because what you're doing is using patient charts to get person information. Patient charting information, as the medical people on this committee will know, is used for a lot of other purposes, and there are a lot of concerns about the accuracy of this.

    For us, this is an integral part of how you present statistics--how good it is, and whether it matters. They are in the process of doing work on that, but it wasn't good enough for us to say that users of the report didn't need to have some kind of warning on it.

    The Chair: Does it go back to Dr. Fry's question, which is more the voluntary nature of this, or is it more the fact that there are these different provinces and territories with maybe slightly different standards in what has to go into charts, and therefore, trying to draw some kind of a national picture becomes too difficult when the health care provision is at the provincial level? Is it impossible to elicit the statistics in an accurate way because of the split jurisdictions--and I don't mean federal and provincial, I mean province to province to province to territory--or is it the interpretation of a chart?

    Ms. Maria Barrados: Dr. Fry was asking about surveillance systems, which are clearly a worry for Health Canada. They are doing work on that in Health Canada. It just is very slow in taking place.

    On the abstracting from the charts, it's a technical area, because a lot of the charts are hand-written. There is another area your committee may want to pursue, and that's to find out about the money that is being given by the federal government through Health Canada for improved information technology. It's another one of these funds and foundations sitting out there that is receiving a lot of money to improve technology in health care.

    The problem with the chart abstracting is that you have very complex records that are kept manually and that coders have to code. There are inherent difficulties with this, and there are things that could be done. You have to have some thought as to how much you want to put into this, but our concern was that there wasn't enough looking at the front end of that and then documenting what you were doing about it so that you knew how good a number it was.

º  (1625)  

    The Chair: Mr. Leighton, do you want to comment?

    Mr. Barry Leighton: Thank you, Madam Chair.

    Some people have told us that coding these charts is more of an art form. I probably shouldn't say that, but we do have concern about the different raters, the different coders. I watched, with someone from our advisory committee for the study, as they did coding in his hospital in Halifax. It was quite a rigorous process, but what they did there was have some sampling of the coding so they could have someone to check up and see the kind of coding error that was going on and get some assurance, some confidence, in the quality of the coding. We're not sure if that practise is established in many other locations, though.

    I did want to say something about CIHI, if I may. In our audit, as Madam Barrados mentioned, we didn't find much in the way of quality assurance processes. One thing we know we didn't find was documentation; there was next to none.

    That's just a minor clarification that I thought might help. What we expect of any statistical agency is to state the strengths and weaknesses of the data so that the user can be aware of, for their particular use, whether it's suitable for these.

    Ms. Maria Barrados: That's quite correct. Our emphasis was on the documentation. If I left the impression that they weren't concerned about quality, that's not correct. They do have a concern about quality. We just didn't find it documented to the extent that we thought was appropriate.

    The Chair: There's a stark difference in your report between the types of words used to describe Statistics Canada data and CIHI data. There's a very stark difference. Now, is it that Statistics Canada has a longer history and knows exactly what it's going after and how to collect that, and CIHI is newer and is still developing its methodologies?

    Ms. Maria Barrados: We have been involved in doing this kind of audit work in Statistics Canada for a number of years. The first time we went to Statistics Canada they didn't do as well either, and they're professional statisticians. There are professional statisticians at CIHI as well, so they're professionally committed to doing a good job, but they hadn't been systematic, in terms of the steps they followed in their quality procedures, in documenting and sharing the statistical series. When we went back to Statistics Canada in this health area they had made significant improvements, and we were pleased to see how they had done. We had not been to CIHI before, and I would hope that if we went back we would see an improvement.

    I might add that there's some uncertainty now about the further reports that are going to be produced from the new health accord as to who is going to be producing them and if there will be any verification of them, and who will do it.

    The Chair: Yes, this is the whole question we're all concerned about, because by about May 4 this health council is supposed to be formed. It was my understanding that the collection of information, and the evaluation of it, and the comparison of it, was going to be in the hands of a health council. But so far we don't have too much news on that front.

    Madam Scherrer, do you want to speak?

[Translation]

    Ms. Hélène Scherrer: In fact, I don't, but since you are giving me the opportunity to do so, I will take it.

    I would like to specifically talk about the report dealing with Statistic Canada, which talks about the limited application of quality assurance practices with regard to health statistics data. One paragraph in particular made me jump:

However, Health Canada has no systematic procedure for edit checks [...]. As well, the department has not established standards for data quality. Nor has it measured the quality of the data [...]. Therefore, Health Canada cannot determine the accuracy of its data for these three surveillance systems.

    Are you telling us that you have no relevant data with regard to health indicators or that the current data is unreliable?

    Ms. Maria Barrados: As auditors, we have to express an opinion on the reliability of the statistics and we expect systems to be in place. We did another audit of Health Canada's surveillance systems and found problems with all of them. There has been some improvement, but that is not yet reflected in the statistics.

    The issues are very simple and yet very difficult. For example, do we have a standard definition? That's essential. It's at the root of our statistics. And in many cases, there is no standard definition.

    There has to be agreements among various levels of government on a statistical basis, so that the statistics are reliable. I am not saying the statistics are worthless, because that isn't true, but as auditors of data, we had to express an opinion. And that is what we attempted to do.

º  (1630)  

    Ms. Hélène Scherrer: It also says somewhere in the document that even the data coding varies from one province to the next. There isn't even any consistency in your data. Incomparable data are being gathered.

    Ms. Maria Barrados: The department attempts to express the statistics so that they can be compared, and questions must always be asked about the impact of their use. That is the other type of thing we look at. If there are errors, that must be reflected in the reports on statistics.

    Ms. Hélène Scherrer: You make recommendations. Will any of those recommendations provide some consistency in the relatively near future? A council is being set up. Given the recommendations in recent reports and the investments being made, we are told that from now on, the provinces will have to provide reports on health indicators. Will we be able to say to the provinces in the relatively near future that we are able to assess the health profile and that our data are consistent, or is it going to take three, four or five years to implement the recommendations? Is it reasonable to expect something meaningful within a year?

    Ms. Maria Barrados: That would be an excellent question for the departments. In our report, we said there was a reasonable plan that could solve a lot of these issues, but that it would take a long time and there is no clear timetable for its implementation.

[English]

    The Chair: Thank you.

    Ms. Skelton, then Ms. Fry.

    Mrs. Carol Skelton: I'd like to go back to the first nations report. I'm from the province of Saskatchewan, and I realize what's happening there. Our first nations people had to file approximately 202 reports. Do you feel, of did you find in your audit, that enough money was being give to them for their report requirements?

    Ms. Maria Barrados: We note in the audit report that in fact the first nations are given money to prepare the reports. The auditors used in their report the line “data for dollars”. So they are given money to prepare their reports in order to get the money. However, it's not a very productive use of resources or funds if the reports aren't of use to the first nations for their own purposes, or aren't being used by the government. We feel that there could be a much more productive use of that money.

    Mrs. Carol Skelton: I've heard that from some of my communities, that there isn't the money there that should be.

    Are there other non-first nations organizations that have a similar report burden put on them?

    Ms. Maria Barrados: We have done this piece of work only in the first nations area. I have had representations from people in the voluntary sector who've raised similar concerns, particularly after the period at HRDC when there was a concern about the management of grants and contributions. We, in one of our follow-ups on HRDC, observed that in some instances there had been what we call “overcorrection”.

º  (1635)  

    Mrs. Carol Skelton: Do you know yet whether Health Canada has taken any steps in reducing the reporting burden?

    Ms. Maria Barrados: Our understanding, as we have it in the report here, is that there is an initiative, I believe now led by Indian and Northern Affairs, to work at beginning to streamline these programs. There have to be questions fundamentally about why you have to have so many separate programs for young people, and why you have to have so many separate programs for different kinds of interventions.

    Mrs. Carol Skelton: The overlap.

    Ms. Maria Barrados: Yes.

    Mrs. Carol Skelton: Good. Thank you very much.

    The Chair: Ms. Fry.

    Ms. Hedy Fry: I have just a couple of questions.

    With regard to first nations, you talked about the use of available technology as opposed to just writing out hard copy and sending it in. Have you found out why it is first nations do not use technology? Do you believe there needs to be some more assistance to first nations to be able to use technology, or is the ability to do so already there?

    The second question I wanted to ask you goes back to statistics, to managing the quality of statistics and the information. You state on page 13 of the short report, paragraph 6.37, that:

The Council [of statistics] developed a business plan for 1999-2004 that commits to improving national standards in areas such as data definition, collection, and reporting.

And I think that's something that is really needed.

    The question is, it was a plan for 1999-2004, so did you do an interim report to see where it was going and whether it was on the right track?

    Ms. Maria Barrados: Which report?

    Ms. Hedy Fry: The one on chapter 6.

    Ms. Maria Barrados: Okay, that's an easy one: No, we didn't. We just finished that one. If we are auditing the next reports, we would obviously go back and look to see that it was implemented and the work was done.

    Ms. Hedy Fry: Excuse me, but before you finish that, is there a requirement...? For instance, 1999-2004 is a five-year plan. Is there a requirement to check plans midway to see that they're on the right route, or do you have to wait a whole five years to see whether they were doing the right thing or not and then start up a whole other new five years? Is there a requirement to do a midway report of anything by the Auditor General?

    Ms. Maria Barrados: We can. It's really at our discretion. This particular one on the Vital Statistics Council for Canada is actually one where we feel things are in good hands. Vital Statistics has been around for a long time. There are things that could be improved, particularly some of the conversations that get into birthweight and things like this. It's not always as solid as you might think.

    This is an area where we feel that Statistics Canada is working with the Vital Statistics Council. They're developing a plan. I don't see this as being an area where I would say we would have to intervene. We would tend to go back to areas that we think are particularly risky, or if a parliamentary committee is telling us they want us to go back, we would go back. Otherwise, on one like this, we wouldn't.

    On your other question, Dr. Fry, on the ability to use technology, we are concerned about that area. In fact, we have the intention to do further study on that for December 2004. Most of the communities we've visited have computers, but you have to have an ability to have those computers talk to other computers and have the software and the packages. We think we're going to look some more at that, because that might make things easier.

    The Chair: Thank you.

    Seeing no further hands, it's my pleasure to thank you....

    Is there something you want to add?

    Ms. Maria Barrados: Yes, Madam Chair. A number of times you've raised issues about the estimates, and we did in the office produce a document to help parliamentary committees in their review of the estimates. This document has been distributed to members of Parliament. If it hasn't arrived here, we can make sure it does get here.

    The Chair: What is it called, Ms. Barrados?

    Ms. Maria Barrados: It's called Parliamentary Committee Review of the Estimates Documents.

º  (1640)  

    The Chair: It seems to me I saw that. Was it distributed maybe a year ago, or a long time ago?

    Ms. Maria Barrados: We updated it, and just within the last couple of weeks it was produced.

    The Chair: Okay. So people have it.

    Ms. Maria Barrados: What we tried to do is reinforce how important the reviews of the estimates are, and to suggest the kinds of questions members can ask about the estimates documents, on both the report on plans and priorities, setting out the intentions, and the performance reports.

    The interests of the committees have a big impact in departments, particularly if you take a look at the report on plans and priorities of what is intended to be done, and try to get some specifics on that, and then take a look at the performance reports. On your questions about surveillance systems, what exactly are they going to do to improve those things? On the first nations, where are you going to be on streamlining some of those programs? On the accountability reports, where are you going to be on the accountability reports for all of this new flow of money? How are you going to improve your responsibilities in reporting back on the moneys that have gone to CIHI?

    These are the kinds of questions.

    The Chair: Good. Thank you for that tip. We'll all go back and get that document out and have a look at it before we meet with the officials on the estimates.

    This has been most helpful. I want to thank you very much for the work you do all the time for the government, and specifically for coming to us today and enlightening us. Thank you very much.

    Ms. Maria Barrados: Thank you.

    The Chair: Perhaps I can ask the members just to wait a minute.

    As you know, we are trying to juggle several topics, so there are a few process questions. At the last meeting, one of the members suggested we were just getting together to educate each other and be educated. Nothing could be further from the truth. Each of the topics we decided to investigate we had hoped would result in one of two things--one, a letter to the minister, and the other, a report to be tabled in Parliament.

    You'll recall that on the HIV/AIDS status we have done some work. We have had a draft summary of evidence, and you were asked to submit recommendations.

    So that the research staff would not have to write lengthy paragraphs about your recommendations--because there were many--and add it all to the report they've already written, I'm suggesting that tomorrow, after our meeting on insulin, we stay for a few minutes. They have written up just those recommendations, the recommendations that they felt came out of our discussions. They're the recommendations we came up with when we had that informal discussion, and they're also the ones you sent in over the last week or so.

    There's a little package, and I think we could just go through it and say “yea” or “nay”. We could have little votes on it, and then they'll know which ones to build up with the preliminary prose--you know, “This is why the committee thinks so....” It's an awful lot of writing for them if we're going to reject some of these, so I suggest it would be more efficient if we voted on the recommendations before they wrote up the rationales for them.

    Is that agreeable to you? Okay. So tomorrow, after the insulin meeting, we can do that.

    On the insulin meeting tomorrow, I have had a call from one of our previous witnesses. You'll recall that Eli Lilly at that time was producing the synthetic insulin and had some porcine insulin available. The witness phoned today to say that Eli Lilly has just announced they are getting rid of...that they're not going to be selling the porcine any more, which is going to force even more people to use the synthetic.

    So I wanted you to be prepared for that. I don't have it in writing from her, but this is what she told me in a phone call today.

    Now, if in fact we get through these recommendations on HIV/AIDS after the insulin meeting, that will guide the writers of the report. I'm wondering if we could have another meeting next Tuesday, at the same time, to go through the draft report line by line and approve it. We can then send it for publication and translation and then present it in Parliament. Time is beginning to slip away if we're going to report on these various subjects.

    We also have three motions. One is Ms. Bennett's motion to request extension on the study of Bill C-260--that's on the smokeless cigarettes--as the instigator of that topic, Mr. McKay, has asked us to do.

    The second one is about the methodology we'll use for questioning, which I think we agreed upon the last time, that Mr. Ménard is going to bring forward.

    The third one is from Mr. Merrifield, who wants to call the minister in for a discussion on SARS.

    In addition to Mr. Merrifield, I have another letter from my favourite correspondent, Mr. Robinson, who writes me on a regular basis. He also wants to have an urgent meeting with the minister on the issue of SARS, at the earliest opportunity. That goes along with Mr. Merrifield's motion.

    So if we could get a quorum for tomorrow, if you could try to motivate your colleagues....

º  (1645)  

    Is your meeting over at eleven o'clock?

    Ms. Yolande Thibeault: Yes.

    The Chair: If you could come over then, you'd be here for this. Could you do that? Thank you.

    I think that's everything. I think if we can keep a couple of days ahead.... We know what we're doing tomorrow. We're doing two things--questioning Eli Lilly about their provision of insulin and going through the AIDS recommendations.

    Wednesday is another new topic that I'm hoping we can get enough information on in one meeting to have an effect--in other words, a letter or a short report or something. That's on dental health for first nations.

    So that's this week, and I'm hoping to have another meeting a week from tomorrow to go over the final HIV/AIDS report.

    Is everybody straight on all that? Are you happy?

    Ms. Hélène Scherrer: Madam Chair, where can we get the document you had? If the meeting is at 9 o'clock tomorrow, can we get it right away?

    The Chair: I agree with you. It would be very good. You can have it tonight and then be ready, either with questions or with how you want to vote on them. Then we should be able to move through it pretty quickly.

    I thank you very much, people, for staying, and thank you for your succinct questioning. We're in good time.

    Mrs. Carol Skelton: Madam Chair, Mr. Spencer is going to take my place, because I have two meetings tomorrow morning--

    The Chair: Mr. Who?

    Mrs. Carol Skelton: Spencer.

    The Chair: Who is that?

    Mrs. Carol Skelton: Larry Spencer.

    The Chair: I've never heard of him.

    Mrs. Carol Skelton: He's our family critic. He's diabetic, so he wanted to come.

    The Chair: Excellent. Was he here for the first diabetes meeting?

    Mrs. Carol Skelton: I'm not sure. He could have been. At any rate, he is diabetic. But I will try to come as well.

    The Chair: Just brief him on the issues we've heard.

    Mrs. Carol Skelton: That's why I wanted to look at that. I'll do this tonight and give it to him.

    The Chair: No, what you're going to look at is the HIV/AIDS--

    Mrs. Carol Skelton: No, no, that's what I was going to do. And he has all the information.

    The Chair: Okay, great.

    This meeting is adjourned.