PACP Committee Report

The project with the worst rating by the Office of the Auditor General was the Treasury Board Secretariat’s Expenditure Management Information System (EMIS). This project was designed to support government expenditure management by providing information on the costs of meeting the government’s priorities and achieving particular results. EMIS was originally estimated to cost $16.2 million but this was revised to $53.7 million. At an earlier meeting on the Expenditure Management System, the Secretary to the Treasury Board, Wayne Wouters, suggested that this increase was due to an increased scope for the project. He said, “We modified the overall project to allow us to be able to not only gather the financial information, but also to be able to gather financial and non-financial information and link it to results.”^[12] However, the problems with EMIS went much deeper than that.

The OAG concluded that EMIS met none of the criteria used to assess the large IT projects. The shortcomings of the business case noted above meant that the project team had difficulty determining whether they were achieving their overall objective. In addition, the audit found that the Secretariat lacked the capacity for the new system. The Secretariat never determined the resources needed to complete the project, had high turnover in management positions, and did not have enough experienced and qualified people working on the project for it to progress as originally planned. It was also an example of ineffective project management—it had no effective project plan, no performance measurement framework, and no specific project milestones and deliverables. Risk management for the system was weak and incomplete. The results of an internal audit of the development of EMIS were equally damning. It concluded, “You will note that many of the observations and recommendations read like a textbook on project management. For every project there are key critical success factors. We used these factors as our audit criteria and found that in most cases these factors were either absent or not completely implemented.”^[13]

The poor management of EMIS casts some doubt on its chances for success, but aspects of EMIS began operations in December 2007 and will be used to support the development of the Main Estimates in 2008-09. If EMIS does not perform as expected, then it will pose problems for the government’s attempt to revise the Expenditure Management System. It is not just public funds that are at risk in large IT projects but substantial changes in the way government operates. The Committee’s concerns about EMIS led it to recommend in its report on the Expenditure Management System that the Secretariat provide the Committee with a status report on the implementation of EMIS.

More importantly, the fact that the EMIS project team did not follow the Secretariat’s own Enhanced Management Framework damages the credibility of the Treasury Board Secretariat to provide direction, guidance, and oversight to departments for large IT projects. The Secretariat must lead by example if it expects departments to give any credence to its guidance and directions. In order for the Secretariat to regain its credibility, it must show that it can learn from the EMIS experience and apply those lessons learned in its guidance to departments. Consequently, the Committee recommends that:

Recommendation 4

The Treasury Board Secretariat conduct an evaluation of the development and implementation of the Expenditure Management Information System, including the deficiencies in the business case identified by the Office of the Auditor General, and provide a report to the Public Accounts Committee on how it will apply lessons learned to the Enhanced Management Framework by 31 December 2008.

 

The most expensive project examined by the OAG was the Secure Channel. This project is a key component of the Government-On Line initiative. It provides a secure infrastructure to allow Canadians to use a single window to conduct online transactions with the government safely and effectively. During the planning, design, and field trial period (from 1999 to 2003), the Treasury Board Secretariat had responsibility for oversight, strategic direction, and decision-making for the Secure Channel project. In September 2003, the project received approval to start to use the technology and in December 2003, Public Works and Government Services took over project management. The Secure Channel was initially estimated to cost $96 million, which was revised to $400 million. It also has high ongoing costs of around $100 million per year to operate it. By 31 March 2006, $596 million had been spent on the project, including $196 million for operations.

The audit found that the Secure Channel project had a number of governance issues. Departments and agencies had not agreed on how to sustain the project or what benefits it would provide. The project did not have a program budget reflecting full life-cycle costs. In addition, the business case did not identify a source of funds to sustain the ongoing operations of the project. Treasury Board made five requests for a long-term sustainable business model that would indicate the sources of funds for the operation of the Secure Channel. It has received funding from Treasury Board 11 times and in 2004, funds earmarked for an exit strategy were used to sustain operations. It would appear that the government built the system without a long-term plan of how to sustain the project into the future. The Auditor General told the Committee, “For a project of this size and complexity one would have expected that the need would have been identified and communicated from the client departments rather than building something and then expecting them to use it.”^[14] The Committee is extremely disappointed that the Treasury Board Secretariat and Public Works and Government Services Canada would proceed with a very expensive project without first clearly identifying the need for the project and without a thorough cost-benefit analysis. Officials at these departments must bear some responsibility for the project’s ever increasing costs to the taxpayer.

The Secure Channel is now supposed to move to a cost-recovery model whereby departments pay Public Works and Government Services Canada (PWGSC) for each time Canadians use the Secure Channel to access their programs. The take-up by departments is important because the government must spend approximately $100 million each year for the operations of the Secure Channel. If departments do not use it, then the per transaction cost of operations is considerable and PWGSC must absorb the operating costs. PWGSC’s performance report notes that, “when operating at capacity, Secure Channel service costs are well within the industry norm and, in some cases, slightly lower than industry averages.”^[15] However, the audit found that take-up by departments was far below projected expectations, and the Secure Channel operating at less than 50 percent of its capacity.^[16] The Auditor General told the Committee that “We have become aware that Service Canada has temporarily suspended it. Now, that may be coming back on, but the last time we looked it was still not using the Secure Channel. The Canada Revenue Agency does not use it for income tax.”^[17] 

Published reports indicate that departments are reluctant to sign-on to use the Secure Channel because it is more expensive than other options and difficult to use.^[18] In order to address the problem of long-term sustainability and take-up by departments, the Treasury Board has now made it mandatory for departments to use the Secure Channel. The Committee supports the move towards on-line government services, but there is little point in making an online system mandatory for departments if it is too costly and inhibits Canadians from using those services. Therefore the Committee recommends that:

Recommendation 5

Public Works and Government Services Canada conduct a cost-benefit analysis of the continued use of the Secure Channel and provide it to the Public Accounts Committee by 31 December 2008.

While the Committee learned that a revised business case was recently accepted by the Treasury Board, it believes that ongoing, publicly available information about the Secure Channel is vital to monitor the costs and sustainability of this project. PWGSC did provide information about the Secure Channel in its performance report as a horizontal initiative, but as the project is no longer a horizontal initiative, the information will no longer be available. In addition, the information available was hardly results based. As an example, the 2005-2006 report indicated that the planned result was “client take-up” and the resulted achieved were, “with 42 applications implemented across 26 organizations, client take-up has increased.” This should be an area where meaningful performance information is rather straightforward. Consequently, the Committee recommends that:

           

Recommendation 6

Should the Secure Channel be continued, Public Works and Government Services Canada provide meaningful results-based information in its annual departmental performance report on the Secure Channel, including information on how many transactions it processes, the number of departments using it, its percent capacity use and per transaction cost.

The Committee is not just concerned about these two projects. It is important that all of the government’s large IT projects are well managed. While the Treasury Board Secretariat has committed to improve its review process, the Committee strongly believes that Parliament needs to have access to sufficient information to allow it to function properly as an institution of accountability and to keep track of government spending on these projects, which is often considerable. The Committee notes that it was only through its report on the Canadian Firearms Program that the government provided detailed financial information on the total costs of the Canada Firearms Information System II, which was discontinued after spending $81 million.^[19] In addition, the Committee would never have learned without the Auditor General’s report that Agriculture and Agri-Food Canada discontinued its AgConnex project after spending $14 million. As departments and agencies do not seem to be inclined to provide information proactively on the costs of their large IT projects, the Committee recommends that:

Recommendation 7

In its guidance to departments on the preparation of reports to Parliament, the Treasury Board Secretariat direct departments and agencies to provide financial and performance information on information technology projects expected to cost over $10 million, including original and current estimated total costs, costs incurred to date, the expected completion date, and the intended outcomes of the project.

It is important that the government continues to invest in information technology projects in order to modernize the way it operates. However, it is vital that these projects are well managed. If they are not, there are huge risks to the significant amounts of public funds involved and the anticipated changes in operations. In order for departments to avoid past mistakes, it is essential that they have a strong governance and management framework, beginning first and foremost with a thorough and effective business case. Unfortunately, the Auditor General found that there was limited progress from the last audit of IT projects in 1997. Departments do not always follow the Treasury Board Secretariat’s Enhanced Management Framework and they are also undertaking large projects which have increased risks. The Public Accounts Committee is troubled by the lack of progress and believes that the Secretariat needs to take a stronger role with respect to these projects, especially during the planning phase and the preparation of business cases. While two cases from the audit raise particular concern, the Committee believes that the government needs to be more proactive in providing public information about all large IT projects because Parliamentarians and Canadians need to have assurance that public funds are being spent wisely and prudently.