PACP Committee Report

INTRODUCTION

A common question that arises among political commentators, parliamentarians, public servants and the Canadian public is “Who audits the Office of the Auditor General (OAG)?” The OAG is well aware of this question and has a number of methods of internal and external review in place.

            The  OAG conducts internal audits of its management and administration.^[1] Internal audits provide the Auditor General with assurance about the extent to which key risk areas within the OAG are being adequately managed. The OAG also conducts internal practice reviews on selected attest and performance audits, special examinations, assessments of performance information, and sustainable development monitoring activities.^[2] These reviews assess audit quality and compliance with OAG policies and professional auditing standards.

            Externally, the OAG’s annual financial statements are audited by an accounting firm—Lévesque Marchand, S.E.N.C., and the audit opinion is included in the OAG’s departmental performance report. The OAG has also requested various external reviews of its audit practice. In 1999, the OAG asked an external audit firm to review its quality management system for financial audits. In 2004, in a first for a national audit office, the OAG requested an international peer review of its performance audit practice.

            In December 2008, the Auditor General of Canada requested a second international peer review, which was completed in May 2010. It was led by the Australian National Audit Office and included officials from the national audit offices of Denmark, the Netherlands, Norway, and Sweden. Rather than focusing on one particular area, this review examined all of the OAG’s audit and assurance practice areas. The objective of the review was, “to provide an independent opinion on whether the OAG’s QMS [Quality Management System] was suitably designed and operating effectively to provide reasonable assurance that the work of the OAG complied with relevant legislative authorities and professional standards.”^[3] The peer review covered the period of September 2008 to October 2009 and examined a sample of OAG performance audits, special examinations, annual audits, components of the audit of the summary public accounts of Canada, and internal practice reviews.

            As the House of Commons Standing Committee on Public Accounts (the Committee) believes that it has an important role to play in holding the OAG to account, it held a hearing on this peer review on 21 September 2010.^[4] The Australian National Audit Office was represented by Ian McPhee, Auditor-General for Australia; Brandon Jarrett, Executive Director, Professional Services Branch; Deborah Jackson, Senior Director, Performance Audit Services Group; and Wayne Jones, Executive Director, Information Technology Audits. The OAG was represented by Sheila Fraser, Auditor General of Canada, and John Wiersema, Deputy Auditor General.

STATUS REPORT

            According to Auditor General for Australia, Ian McPhee, the peer review went to the core of the OAG’s business—audit quality.^[5] The peer review examined the OAG’s Quality Management System (QMS)—the structure and processes used to provide assurance of audit quality—and found that it was suitably designed for performance audit, special examination and annual financial audit practices. Although, there was room to improve the QMS by eliminating some overlapping sub-elements and changing the way criteria were expressed.

            The review concluded that the QMS was operating effectively for performance audit and special examination practices. In these practice areas planning was sound, audit teams obtained sufficient and appropriate evidence to support reported findings, reports were balanced, OAG policies and procedures were complied with, and practice reviews were thorough.

            The review suggested that the OAG could improve performance audit and special examination reports by including more context about the issue examined and an explanation of the impact of the findings. Also, special examination reports could be improved by providing clearer explanations of why certain systems and practices were selected for detailed examination, aligning the findings and conclusions for each system examined, and explaining the term “significant deficiency.”

            For annual financial audit, the review found that the audits were generally compliant with Canadian auditing standards, but there were implementation issues in relation to:

• the completeness of the risk assessment procedures informing the nature and extent of further audit procedures; and
• the sufficiency of audit documentation recording the results of the audit work performed.

That is, for some annual audits, the reviewers found no record of enquiries to the entity’s management regarding the risk of fraud and related matters, and the reviewers had difficulty connecting planned audit procedures with the risk they were designed to address. The reviewers also found examples where the audit working papers did not contain sufficient documentation to support the judgments and conclusions in the audit file.

            The review made two recommendations: that greater emphasis be given to risk assessment in annual audits; and that the OAG reinforce to staff the need for documentation to demonstrate compliance with relevant standards and OAG policy. The review also suggested that the OAG develop a more robust process to ensure that lessons learned are identified, captured, and disseminated; that the OAG develop a system to rank annual audit findings according to risk; and that evidence of the review of annual audit files be kept in the electronic working papers.

            Two issues persisted from the 2004 review. The previous review suggested that the presentation of reports could be improved through the use of graphics and tables to present complex numerical data and footnotes showing the sources of data. The 2010 peer reviewers considered these suggestions to be still valid. The Auditor General of Canada, Sheila Fraser, said that her office has tried to include more graphics and tables in their reports,^[6] but the OAG did not agree with the suggestion to include footnotes, indicating that it was “not useful.”^[7]  The Auditor General of Canada said that it would make the reports too heavy if they started to source all evidence in footnotes. However, the Auditor General for Australia, Ian McPhee, noted that his office uses footnotes judiciously to refer to key sources of evidence. He said that the issue is a matter of discretion.^[8] Nonetheless, the Committee is concerned by the OAG’s continued resistance to a practice that is common amongst other audit offices and could be used to provide more context and information to readers.

             The Auditor General of Canada acknowledged that her office needed to make improvements and told the Committee that the review’s findings were similar to those found during an internal practice review. She said, “The recommendations that came out of the peer review were not a surprise. The issues that were raised were very similar to the same issues we had found in our most recent practice reviews, so we knew we had work to do in this area.”^[9]

            The Auditor General for Australia told the Committee that the peer review’s findings did not indicate that there were errors or omissions in the OAG’s financial audit work. Brandon Jarrett, Executive Director at the Australian National Audit Office, explained that, “In some cases, in looking at the files we couldn't see a clear path between things that were planned to be done or were done. But that doesn't mean that the judgments, at the end of the day, were incorrect. What that means is that in terms of reviewing the file I couldn't see a clear process or clear documentation to be able to support everything that needed to be done in accordance with the auditing standards.”^[10] The Auditor General for Australia concluded, “[T]he issues we have raised in this report are not unique to [the] OAG. It's a message to all audit offices, including my own, about the importance of making sure we have the training and the follow-through to ensure that the systems we put in place are actually followed so that we can produce quality audits for the information of Parliament and the country.”^[11]

            When the peer review report was released in June 2010, the OAG also released its action plan to address the peer review findings, which primarily involves a Renewal of Audit Methodology project—revising and updating audit methodology, the QMS, related audit tools, checklists, and training. The OAG will develop a change management component to ensure that the updated methodology is put into place. As well, the OAG is working to ensure that its senior managers are more involved in key judgments and conclusions resulting from the audit work.

            At the September hearing, the OAG provided the Committee with an update on progress on implementing its action plan, which noted that a number of actions have been completed. However, the Renewal of Audit Methodology project will not be completed until December 2011. As the Committee would like to monitor the OAG’s progress to ensure that it fully addresses the review’s recommendations and suggestions, the Committee recommends:

RECOMMENDATION 1

That the Office of the Auditor General of Canada provide the Public Accounts Committee with a progress report by 31 March 2011 of what actions have been taken to address the recommendations and suggestions of the international peer review.

FUTURE PEER REVIEWS

            As noted earlier, there have been several reviews of the OAG’s audit practice. All of these reviews have been at the request of the OAG. The 2004 review of the OAG’s performance audit practice was the first review of its kind for a national audit office. The Auditor General of Canada explained why she requested the current review just before the end of her term. She said, “I thought it was important that a new Auditor General coming in not have the responsibility of conducting a peer review right away. The issues we have to deal with are known now, and hopefully that Auditor General, before the end of his or her term, will also do a similar peer review to assess the state of affairs of the office.”^[12]

            The Committee commends the Auditor General and the OAG for being leaders in requesting external reviews of their audit practice. It takes courage to request a review that may present unfavourable findings. The Committee appreciates the Auditor General’s commitment to accountability and transparency. However, the Committee notes that these reviews are discretionary and are not based on a specific schedule. On the other hand, an official from the Australian National Audit Office, Deborah Jackson, noted that they have an agreement with the New Zealand audit office. She said, “Every second year we do a mini peer review of each other's offices.”^[13] As the Committee believes that peer reviews are an effective means of providing external assurance to Parliament about the quality of the OAG’s audit work, the Committee recommends:

RECOMMENDATION 2

That the Office of the Auditor General develop a schedule for regular peer reviews of its audit practice.

BEST PRACTICES

            While the peer review did find weaknesses in the OAG’s audit practices and made suggestions for improvement, the review also identified a number of good practices that will be of interest to other national audit offices. According to the review, the OAG:

• actively engages parliamentarians,
• delivers a clear and consistent message in performance audit reports,
• conducts collaborative audits with other organizations, and
• has developed criteria and sub-criteria for special examinations.^[14]

 The Auditor General for Australia also indicated that performing a peer review can be useful for the reviewer, as well as subject of the review. He said, “I would like to recognize that peer reviews are a learning experience for both the review team and the office being reviewed. They facilitate the sharing of ideas and experiences among review team members. As members of the review team, we found the experience to be valuable and informative.”^[15]

            The Committee believes that it is important for audit offices to share best practices, but is concerned that these practices would be discovered through the peer review process rather than other forums. There are clear international standards for financial audits, with the Canadian Institute of Chartered Accountants adopting the new International Standards for Auditing in 2010, but there are no similar international standards for performance audits, which can complicate the completion of peer reviews. While it may be difficult to develop common standards for performance auditing due to differing legislative mandates, the Committee believes that sharing best practices can be an effective means of improving performance audit practices. The Committee recommends:

RECOMMENDATION 3

That the Office of the Auditor General work with other national audit offices to develop a means to share best practices for performance auditing.

CONCLUSION

            By requesting independent peer reviews of the OAG’s audit practice, the Auditor General of Canada has demonstrated a clear commitment to accountability and to continuous improvement. The reviews provide assurance to Parliament and Canadians that the OAG is conducting quality work. The most recent peer review found that the OAG’s Quality Management System was well designed and working well for performance audit and special examination practices. The review also identified a number of good practices that might be of interest to other national audit offices. While the review found areas where improvements are needed in the financial audit practice, the OAG appears to be committed to renewing its audit methodology and providing additional training to its staff. The Committee trusts that the OAG will continue to set and maintain high standards of excellence for its audit practice, and hopes that peer reviews will become a regular practice.