Skip to main content

PACP Committee Report

If you have any questions or comments regarding the accessibility of this publication, please contact us at accessible@parl.gc.ca.

PDF

CHAPTER 5—INFORMATION TECHNOLOGY INVESTMENTS—CANADA BORDER SERVICES AGENCY, SPRING 2015 REPORTS OF THE AUDITOR GENERAL OF CANADA

INTRODUCTION

With a workforce of approximately 13,000 employees and an operating budget of over $1.7 billion, the Canada Border Services Agency (CBSA) manages the access of people and goods to Canada. In 2013–2014, the CBSA administered close to 100 million travelers into Canada, cleared more than 14 million commercial shipments, made more than 9,000 drug seizures, and removed almost 14,000 failed refugee claimants and other inadmissible persons. The CBSA also collected $26.9 billion, which is approximately 10% of the Government of Canada’s revenues.[1]

Like other departments and agencies, the CBSA relies upon information technology (IT) to fulfil its mandated services. The CBSA’s portfolio of IT investments, managed by its Information, Science and Technology Branch, is budgeted at more than $1 billion, primarily consisting of the Border Modernization initiative, involving eight projects, and the Beyond the Border Action Plan, which includes 32 government-wide initiatives.[2]

The Beyond the Border Action Plan aims to enhance border security while expediting legitimate cross-border trade and travel. It seeks to “push the border out,” by collecting information earlier and thereby facilitating the flow of legitimate goods and travellers that are low risk, through such programs as NEXUS, Trusted Trader and FAST.[3] The CBSA’s IT projects will help it realize operational benefits, such as improving its lookout system, scrutinizing passenger name record information before inbound flights depart, and analyzing electronic manifest information before commercial import shipments arrive.[4]

In spring 2015, the Office of the Auditor General of Canada (OAG) released a performance audit that examined whether the CBSA had the corporate and management practices in place to enable the delivery of IT investments that align with and support the CBSA’s strategic corporate objectives.[5] The OAG examined five of the CBSA’s IT projects:

  • Entry/Exit Initiative;
  • Field Operations Support System replacement project;
  • Interactive Advance Passenger Information Initiative;
  • Single Window Initiative; and
  • Temporary Resident Biometrics Project.[6] 

The House of Commons Standing Committee on Public Accounts (the Committee) held a hearing on this audit on 27 May 2015.[7] From the OAG, the Committee met with Michael Ferguson, Auditor General of Canada, and Martin Dompierre, principal. The CBSA was represented by Caroline Weber, Vice-President, Corporate Affairs Branch; Louis-Paul Normand, Associate Vice-President, Information, Science and Information Technology Branch; and Chris Bucar, Acting Deputy Chief Financial Officer and Director General Resource Management.

PORTFOLIO MANAGEMENT

In response to an external review, the CBSA implemented, in 2013, the Project Portfolio Management Framework to better manage its portfolio of IT investments. The framework focuses on undertaking the right projects at the right time and overseeing them to ensure that expected benefits are realized.[8] In order to assess the CBSA’s management practices, the OAG examined the CBSA’s portfolio governance, investment planning and annual IT planning, enterprise architecture, and the management of IT portfolio risk.[9]

A. Governance

The OAG found that the Transformation, Innovation and Project Portfolio Committee, which is responsible for project portfolio oversight and alignment, had not been given sufficiently detailed information to exercise its responsibilities, and there was no ongoing assessment of how sub-portfolios were strategically aligned and were within resource capacity.[10] While the CBSA has a Project Management Framework, the OAG noted that projects were allowed to move to subsequent phases of development even though prerequisite conditions under the Framework, such as business cases, risk assessments, detailed project plans and project benefit plans, were not met.[11]

Michael Ferguson, Auditor General of Canada, clarified the OAG’s expectations with respect to applying the framework. He commented:

Going through projects and making sure that each stage is met along the way is critically important to understand that all the requirements as well as the time and the budget are on schedule, to make sure the project is going to deliver what was intended. We would never ask a department to go back and try to redo gates that have already been passed. But we would expect the department, at the point they were putting in place a new framework, to at least look at those projects that were in place and identify if they seem to be on the right path to deliver what they were intended to deliver, regardless of the fact that you can't go back and re-document certain gates, whether they were passed or not passed.[12]

Caroline Weber, Vice-President of the Corporate Affairs Branch at the CBSA, described how the CBSA was working to apply its framework to projects that are in progress. She said:

The challenge for us has been IT projects that were already in progress, had been started, were a long way down the road to being completed when we bring in a new project management framework. The question then was whether we were going to go back in order to check the boxes on these projects, or whether we were going to continue and deliver, especially when we're near the end of a project, knowing that we would leave it undocumented, but thinking perhaps it wasn't the right way to use our resources at that time. On a go-forward basis, all of our projects are being put through the entire framework with no exceptions, and we review that all the time.[13]

B. Investment Plan and Annual IT Plan

The OAG observed that the CBSA’s investment plan, which outlines investments in projects, assets and acquired services, had not been updated since 2011, even though the Treasury Board Policy on Investment Planning—Assets and Acquired Services requires the plan to be updated every three years, or when a significant change occurs. The CBSA’s IT portfolio size has significantly increased since then.[14] Nonetheless, the CBSA had conducted some ad hoc IT investment planning activities. In April 2014, it developed a Five-year Capital Plan, in part to manage the fact that capital funding surpluses had grown even though some projects were delayed due to a lack of funding. In June 2014, the CBSA completed an Annual IT Plan to demonstrate how IT aligns with the CBSA’s strategic objectives, but the plan does not include IT projects. At the request of the Treasury Board Secretariat of Canada, in August 2014, the CBSA prepared an Integrated IT Project Plan that included 14 of the CBSA’s 30 projects.[15]

Ms. Weber told the Committee that the failure to update the investment plan resulted “in some frequent and intensive conversations with our colleagues in the Treasury Board Secretariat.”[16] However, Chris Bucar, Acting Director General of Resource Management at the CBSA, informed the Committee that the CBSA’s “investment plan and annual IT plan … were submitted to Treasury Board, and approved on April 23, 2015.”[17]

C. Enterprise Architecture

The OAG found that the CBSA does not have an overall portfolio enterprise architecture, which can maximize the value of IT investments by promoting reuse, agility and innovation.[18] According to the OAG, the lack of such an architecture resulted in the development of duplicate IT systems. For example, two master data management systems were built, with a future plan to merge them.[19] Additionally, the CBSA ended its Business-to-Business project after completing phase 1, even though several of its IT systems identified the need to develop business-to-business services in order to communicate with the IT systems of other organizations.[20]

Ms. Weber indicated that the CBSA was working “to ensure that enterprise architecture is adhered to by all IT projects through formal gate reviews and approvals.”[21] She also said:

A functional directive covering every domain of the enterprise architecture will be finalized by September, 2015. Also, we've begun to move individual projects toward architecture standards that fully align with Shared Services Canada's directions, and the service life-cycle management framework ensures that enterprise architecture directions are adhered to by all projects.[22]

D. Portfolio Risk

While the CBSA conducted some risk assessments, the OAG found that it did not have an overall risk profile for its IT investment portfolio. Without this profile, the OAG concluded that the CBSA could not determine the level of risk in proportion to the expected benefits of its portfolio of IT projects.[23]

Ms. Weber told the Committee that the CBSA is “going to continue to maintain [its] Beyond the Border project-level risk profile and to provide a full portfolio risk update roll-up at the quarterly Beyond the Border senior project advisory committee.”[24]

In order to encompass issues related to updating the investment plan and annual IT plan, defining and completing target enterprise architecture, and completing an IT project portfolio risk profile, the OAG recommended that the CBSA ensure that all elements of the Project Portfolio Management Framework are implemented.[25] When he released the audit in April, Mr. Ferguson commented, “We were very happy with the framework that had been put in place in the agency and the fact that it was comprehensive. Our concern, again, was that it wasn't at this point in time always being applied in the management and the oversight of the projects.”[26]

In response, Ms. Weber noted, “We did recognize a number of years ago that we needed to improve with regard to IT project management, so we put in place this framework and started implementing it. I can tell you that our IT projects are being delivered on time now and on budget, so the framework that we put in place has achieved that.”[27]

MEETING BUSINESS REQUIREMENTS

Clearly defining IT systems requirements allows organizations to demonstrate that IT projects align with strategic directions, meet business needs, and have appropriate funding and resources.[28] The OAG examined whether the CBSA implemented IT project management practices and whether it demonstrated results, in particular, business requirements and outcomes and benefits.[29]

The OAG noted that the projects examined did not have clear corresponding IT systems requirements before the projects started the execution phase, which led to several challenges:

  • As noted above, there was a duplication of effort in the development of master data management systems;
  • As projects did not have costs broken down by deliverables, the CBSA could not identify the costs of deferring work, leading to projects requesting more funds than could be used in a given fiscal year;
  • Differences of opinions with Citizenship and Immigration Canada (CIC) over system functions for the replacement of the Field Operations Support System (FOSS) led to project delays; and
  • Some projects, such as the Interactive Advance Passenger Information initiative, were at risk of not delivering what was expected.[30]

The Development of CBSA’s IT projects which meet necessary business requirements is particularly challenging because of the CBSA’s dependence upon external agencies in the ecosystem in which it works. The consequence is that not all business requirements are specified up front and that policy change decisions are sometimes made after a project begins.

Examples of external agencies upon which the CBSA is dependent in developing one IT project alone (the Entry-Exit IT project) include:

  • global partners;
  • nine partners within the Government of Canada who request information;
  • ninety acts of Parliament to comply with, including associated regulations, with some still being awaited;
  • privacy requirements; and
  • the airline industry.[31]

With respect to the replacement for FOSS, Louis-Paul Normand, Associate Vice-president of the Information, Science and Technology Branch at the CBSA, informed the Committee that the decision for developing master data management has not been made and was still under consideration.[32] In response to a question about the cost of the delay in replacing FOSS, he said:

We had to agree as to what would be done in GCMS, so in the CIC system, and what had to be done and developed by CBSA. It's not an incremental cost, in that if we hadn't done it—in this case we decided to do it—then CIC would have had to do it. The disagreement had to do with who was to do it, not the cause of an overrun. The $2.3 million was the licence for the FOSS vendor, the legacy vendor. The decision would have been to carry it on anyway because of the rollout plan that we deployed. In other words, there was no way that we would have been able to turn off FOSS as a back-up system in December 2013 as originally planned.[33]

Of the five projects examined, the OAG found that all had defined project outcomes at the planning phase, but four of the projects did not define measureable benefits, which would have enabled the CBSA to demonstrate alignment between project delivery and business needs.[34] Furthermore, in August 2014, the CBSA identified that over 50% of its portfolio projects had “low readiness,” which means that there was minimal information on whether benefits would be realized or aligned with strategic objectives, and the readiness of another 27% of projects was unknown.[35]

The OAG recommended that the CBSA ensure that project requirements are met and measures are defined to assess whether projects deliver expected benefits.[36] Mr. Ferguson emphasized the importance of assessing benefits, stating, “There's more than just making sure systems are delivered on time and on budget. There's also making sure that the systems are delivering what they were intended to deliver. That's why we talk in the report about the need to assess the benefits and assess whether the systems are aligned with the objective of the agency.”[37]

Mr. Normand described the CBSA’s response, commenting, “We've mandated that all active projects go back and revisit their business case, their business requirements, and their project plan to make sure that the benefits are clearly identified, measurable, and harvestable.”[38]

REPORTING PROJECT INFORMATION

Project dashboards, which show the status of a project or portfolio of projects, can be used by senior management to make decisions on whether IT projects require more active monitoring and whether corrective action is needed.[39] The CBSA uses project dashboards that show the health of individual projects, i.e., whether they are meeting schedule milestones, undergoing changes in scope and staying within budget, as well as the enterprise dashboard, which is a consolidation of project dashboards and is provided to the executive committee.[40] The OAG examined whether the information reported to decision makers on IT projects was in accordance with the CBSA’s frameworks and guidelines.[41]

The OAG found that the information provided in project dashboards was inconsistent and incomplete.[42] For example, there were variances that ranged between 6% and 157% with regards to the financial information reported on project dashboards when compared to other CBSA reports.[43] Additionally, project managers had discretion in what they reported on the dashboards, which influenced a project’s health status.[44] The OAG also observed variances in the reporting of project status and project scope between project dashboards and other documentation.[45]

The OAG noted that in May 2014, the CBSA introduced a new process to monitor project performance using value management reporting to mitigate discrepancies in the monthly project dashboards.[46] This method can improve project performance monitoring, but the OAG noted some inconsistencies between value management reports and project dashboards.[47]

Mr. Normand described the benefits of value management reporting at the CBSA, observing:

The first requirement is that you need to have a good plan. Based on that plan, you load up with your resources that you need to deliver on that plan. Then the tracking becomes the planned value. As you go through six months of activities, you should be 50% done on a 12-month project. If your actuals, your financials, are over that or below that, it creates a variance that you track. That's the essence of earned value. It does that for scheduling variance. It does that for cost variance. Now in our dashboard to Treasury Board, the colour for cost and schedule is no longer subjective. It is based on a percentage of variance on cost or scheduled variance.[48]

As the absence of consistent controls over project reporting can raise risks that reporting on project status is ineffective, the OAG recommended that the CBSA establish clear procedures and practices on how the information for project dashboards is collected, reported and enforced.[49]

Ms. Weber acknowledged the issues raised by the OAG, stating:

We did have some discrepancies, as the Auditor General referred to. We knew about those discrepancies, but we were living with them because there were time gaps or time lags between some of the information. We were not including some standard information that would have been easy to add because we knew it was there and it's not the way we usually reflect things internally. However, we are changing that as a result of this audit.[50]

She added that in response, the CBSA is “developing a baseline set of performance benefits indicators and quarterly reporting to the executive cadre on benefits realization status of IT projects; and initiating a formal review process of the procedures and practices of how project dashboard information is collected, reported, and enforced.”[51]

PROGRESS REPORT

The CBSA provided an action plan to the Committee which outlined a number of commitments to improve the management of its portfolio of IT projects. As the latest completion date listed in the action plan is March 2016, the Committee recommends:

RECOMMENDATION

That, by 31 March 2016, the Canada Border Services Agency provide the Standing Committee on Public Accounts with a report outlining its progress in addressing the Office of the Auditor General of Canada’s recommendations contained in Report 5 of the Spring 2015 Reports.

CONCLUSION

The CBSA has a large portfolio of IT projects, many of which are essential to the effective delivery of its strategic objectives, in addition to improving the efficiency of its operations. For example, the Single Window Initiative allows data for several different federal departments to be collected in a single transaction. In order to manage these complex projects, decision-makers at the CBSA need to have access to complete and reliable project information, with clear business requirements.

Overall, the OAG concluded that the CBSA had the necessary corporate and management practices to deliver on its IT investments. However, the CBSA had not put into practice all of the elements of its project portfolio management framework to ensure that its IT projects would support strategic corporate objectives. Also, the committees overseeing project portfolios did not have sufficient information to fully exercise their responsibility to ensure that project requirements and conditions were met at each stage of approval.[52]

The CBSA has recognized that it needs to improve the implementation of its project management framework. The Committee expects that the CBSA will strengthen the governance process for IT investments, clarify IT systems requirements to ensure project benefits are realized, and establish consistent procedures and practices for the collection and reporting of project status information.



[1]              Auditor General of Canada, “Report 5 – Information Technology Investments – Canada Border Services Agency,” 2015 Spring Reports of the Auditor General of Canada, Ottawa, 2015, para. 5.1.

[2]              Ibid., para. 5.9.

[3]              House of Commons Standing Committee on Public Accounts, Evidence, 2nd Session, 41st Parliament, 27 May 2015, Meeting 61, 1615.

[4]              Ibid., 1535.

[5]              Auditor General of Canada, Report 5, para. 5.4.

[6]              Ibid., Exhibit 5.1.

[7]              Meeting 61.

[8]              Auditor General of Canada, Report 5, para. 5.10.

[9]              Ibid., para. 5.16.

[10]           Ibid., para. 5.19.

[11]           Ibid., para. 5.20.

[12]           Meeting 61, 1645.

[13]           Ibid., 1645.

[14]           Auditor General of Canada, Report 5, para. 5.22.

[15]           Ibid., para. 5.23.

[16]           Meeting 61, 1630.

[17]           Ibid., 1610.

[18]           Auditor General of Canada, Report 5, para. 5.25.

[19]           Ibid., para. 5.26.

[20]           Ibid., para. 5.27.

[21]           Meeting 61, 1535.

[22]           Ibid., 1650.

[23]           Auditor General of Canada, Report 5, para. 5.29.

[24]           Meeting 61, 1650.

[25]           Auditor General of Canada, Report 5, para. 5.30.

[26]           House of Commons Standing Committee on Public Accounts, Evidence, 2nd Session, 41st Parliament,
29 April 2015, Meeting 57, 1600.

[27]           Meeting 61, 1605.

[28]           Auditor General of Canada, Report 5, para. 5.33.

[29]           Ibid., para. 5.35.

[30]           Ibid., para. 5.37.

[31]           Meeting 61, 1540.

[32]           Ibid., 1600.

[33]           Ibid., 1610.

[34]           Auditor General of Canada, Report 5, para. 5.40.

[35]           Ibid., para. 5.41.

[36]           Ibid., para. 5.42.

[37]           Meeting 61, 1625.

[38]           Ibid., 1630.

[39]           Auditor General of Canada, Report 5, para. 5.45.

[40]           Ibid., para. 5.49.

[41]           Ibid., para. 5.48.

[42]           Ibid., para. 5.50.

[43]           Ibid., para. 5.53.

[44]           Ibid., para. 5.52.

[45]           Ibid., para. 5.54.

[46]           Ibid., para. 5.57.

[47]           Ibid., para. 5.58.

[48]           Meeting 61, 1650.

[49]           Auditor General of Canada, Report 5, para. 5.61.

[50]           Meeting 61, 1555.

[51]           Ibid., 1535.

[52]           Auditor General of Canada, Report 5, para. 5.62.