PACP Committee Report

Large information technology (IT) projects involve incorporating new hardware, software, and systems into government operations. They also can change the way departments do business by introducing new processes and modernizing work processes. The federal government has invested heavily in large IT projects. In the last three years, Treasury Board approved 88 IT projects with costs that totaled $7.1 billion, with an additional $1.2 billion provided to Canada Health Infoway and $431 million invested by the Canada Revenue Agency in its systems.

However, large IT projects are inherently complex, expensive, and risky. They usually involve long planning and development times. Large IT projects have had a history of overspending, delays, performance shortfalls, and abandonment after major investments. Understanding why some projects have succeeded while others have failed would help the government manage IT projects for success.

As the Public Accounts Committee is concerned that some large IT projects may not be well managed thus putting substantial amounts of public funds at risk, the Committee held a hearing on the Auditor General’s November 2006 Report, Chapter 3: Large Information Technology Projects on 18 June 2007. The Committee heard from officials of the Office of the Auditor General: Sheila Fraser, Auditor General of CanadaDouglas Timmins, Assistant Auditor GeneralRichard Brisebois, Principal; from the Treasury Board Secretariat: Ken Cochrane, Chief Information Officer; Jim Alexander, Deputy Chief Information Officer; from Public Works and Government Services Canada: Steven Poole, Chief Executive Officer, Information Technology Services Branch.

           

The Office of the Auditor General (OAG) last examined IT systems under development in government-wide audits in 1995, 1996, and 1997. The audits found inadequate analyses of underlying business issues, inconsistent support from management, a lack of experienced resources on project teams, inconsistent involvement on the part of users, and a lack of effective monitoring.

In May 1996, the Public Accounts Committee released a report on the issue and recommended that the government:

• allocate more resources to monitor large information technology projects;
• submit an annual report to the Committee on the status of departments’ investments in systems that are under development;
• have the Treasury Board Secretariat take steps to ensure that there is complete ownership and accountability for large IT projects; and
• whenever possible, divide projects into smaller, more manageable components.

In 1998, partly in response to the OAG’s recommendations, the Treasury Board Secretariat created an overall guidance document called An Enhanced Framework for the Management of Information Technology Projects—Part II (EMF). The Secretariat is currently undertaking a review of the EMF.

In its most recent audit, the OAG examined seven large IT projects: Global Case Management, Secure Channel, Expenditure Management and Information System, Integrated Revenue Collections, 2006 Census Online, AgConnex, and My Account/My Business Account. The audit examined these projects from four perspectives: governance, business case, organizational capacity, and project management. The audit concluded that overall, the government has made limited progress since the last audit of IT projects in 1997. The audit made six recommendations concerning the overall management of IT projects, all of which were accepted by the government.

The Public Accounts Committee fully endorses the findings and recommendations of this audit. However, there are several areas that are of concern to the Committee and are thus explored in more detail in this report: review by the Treasury Board Secretariat, the poor management of the Expenditure Management Information System and the Secure Channel, and public reporting of IT projects.

Departments must seek Treasury Board approval before starting projects that exceed established departmental project authorities. Often approval must be sought for two phases: preliminary project approval and effective project approval. According to the audit, the Secretariat assists Treasury Board ministers by reviewing departmental submissions to ensure that the proposals comply with applicable policies, to assess the business value of the proposal, and to determine whether projects have the essential project management elements to be successful. The Secretariat describes its role as one of selective oversight based on an assessment of risks and the Secretariat’s available resources and capacity to oversee and monitor projects.

The audit attempted to determine whether the Treasury Board Secretariat adequately fulfilled its challenge and oversight responsibilities. However, the Secretariat denied access to the OAG to most of the information and analysis it collects and prepares, citing Cabinet confidence. Consequently, the audit was unable to assess the rigour and quality of the Secretariat’s analysis. While the Auditor General told the Committee that this issue has been clarified by a new order in council, the Committee is concerned that this issue arose at all, since the Auditor General should have full access to all documentation held by the government, including the Treasury Board Secretariat. Now that the OAG has access to the needed information, the Auditor General may wish to closely examine the oversight role of the Secretariat in a follow-up audit.

In his appearance before the Committee the Chief Information Officer (CIO), Ken Cochrane, further clarified the Treasury Board Secretariat’s role by categorizing it into four components: policy, practices, challenge, and monitoring. Under the authority of the Treasury Board, the Secretariat prepares policies to direct and guide departments when they undertake IT projects. For the second component, the Secretariat establishes best practices, such as the Enhanced Management Framework for IT-enabled projects. For the challenge function, the Secretariat reviews and makes recommendations to ministers on departmental projects. For higher risk projects the CIO branch monitors them on a regular basis.

According Mr. Cochrane, the Secretariat intends to take corrective actions to address the issues identified by the Auditor General. The Secretariat plans to develop new policies on the management of IT projects, implement improvements to the Enhanced Management Framework, such as a new capacity assessment tool, redesign and update its processes for reviewing submissions, and require departments to have independent third-party assessments done at key milestones. These and several other activities were outlined in a very brief “action plan” provided to the Committee.

While the Committee appreciates that the Secretariat is undertaking activities to strengthen its role in the governance and management of IT projects, the Committee is somewhat concerned that problems with large IT projects have existed for years and the Secretariat is only now seeking to improve its practices. It is hard to have confidence in the Secretariat when the so-called “action plan” provided to the Committee is nothing more than a few bullet points and a diagram. When the Auditor General releases an audit, the Committee expects that departments will develop detailed action plans with specific target dates for the implementation of these recommendations. Mr. Cochrane told the Committee that he was, “prepared to speak to the target dates of the action plan in far more detail.^[1] This is not satisfactory because the Committee must then ask numerous questions in order to obtain any detail on the plan. Instead, a detailed plan should be provided to the Committee before the hearing in order to allow Members time to study it and develop questions. The Committee recommends that:

           

Recommendation 1

The Treasury Board Secretariat provide a detailed action plan with specific target dates to the Public Accounts Committee by 30 September 2008 for the implementation of the Auditor General’s recommendations on large information technology projects.

While it may be difficult for the Treasury Board Secretariat to monitor all large IT projects as they proceed, the Secretariat has a vital role to play in deciding whether to recommend approval of a project to the Treasury Board. Without thorough planning and a careful consideration of options, complex and difficult projects are not likely to succeed. According to the audit, a business case is the foundation of every sound investment decision because it explains the rationale for the project and the results that are needed.^[2] However, the Auditor General told the Committee that, “Five projects were allowed to proceed with a business case that was incomplete or out of date or contained information that could not be supported.”^[3] The audit provides more detail on specific cases. The business case for the Expenditure Management Information System (EMIS) had no comprehensive project plan and vague, immeasurable business goals.^[4] Citizenship and Immigration Canada received two budget increases for its Global Case Management System without submitting a revised business case to provide a clearer picture of the project or to redefine objectives.^[5] The Secure Channel project’s business case did not identify a source of funds to sustain the ongoing operating and maintenance costs of the project.^[6] In fact, the Treasury Board made five requests for a long-term sustainable business model indicating the source of funds for the operation of the Secure Channel. The business case prepared by Public Works and Government Services Canada was endorsed by the Secretariat, but it did not meet the sustainability concerns raised by the Auditor General in 2003.^[7]

Ken Cochrane acknowledged that the government needed to do better in the area of business cases and said they will be putting in place a more consistent approach. He told the Committee that the Secretariat will also monitor compliance with the Enhanced Management Framework, and specifically business cases, through the Management Accountability Framework (MAF). He said, “If you're familiar with the management accountability framework, it does have an indicator that focuses very specifically on project management, and business cases are a key component in overall project management.”^[8] However, the MAF exercise is a department-wide assessment and does not examine each and every project. Moreover, the Committee is astounded that the Treasury Board Secretariat would even allow projects to proceed without adequate business cases. When projects are approved with information that is incomplete or out of date, this is more than an issue of just refining the requirements for business cases.

The Auditor General recommended that the Secretariat should improve the requirements for business cases.^[9] The Committee agrees, but would like to go further because if the Secretariat recommends funding to the Treasury Board for projects that have inadequate business cases, then the Secretariat must bear some of the responsibility for any failures that take place due to inadequate proposals. As the Auditor General put it, “So unless there’s a consequence to not giving complete business plans, people will continue to give incomplete business plans.”^[10] Consequently, the Committee recommends that:

Recommendation 2

The Treasury Board Secretariat ensure that all information technology projects have a detailed, complete, accurate, and up-to-date business case prior to being submitted to the Treasury Board for approval.

Research indicates that small IT projects are more likely to succeed than large ones. However, the audit states that departments and agencies are once again undertaking large IT projects.^[11] The Committee notes that the well managed projects examined by the audit, 2006 Census Online and My Account/My Business Account, were very specific and targeted. In its 1996 report on the topic, the Committee recommended that whenever possible, the government should divide IT projects into smaller, more manageable components. While the Committee acknowledges that the government is faced with broad, complex, cross-departmental issues, it continues to believe that every effort should be made to pursue small, targeted projects where possible. The Committee recommends that:

Recommendation 3

The Treasury Board Secretariat require all new information technology project submissions to include an options analysis of the possibility of breaking large projects into smaller, more manageable projects.