INST Committee Report

CHAPTER THREE: EXCELLENCE, RELEVANCE, AND STRATEGIC RESEARCH

The issue of funding priorities for Canada, and their relationship to granting agency programs and selection criteria, came up on a number of occasions during the Committee’s hearings. Most of the witnesses appearing before the Committee stressed that excellence should be the fundamental selection criterion for any peer-review process for the allocation of federal research funds. However, for certain types of research programs, the short-term socio-economic relevance of the proposed research, sometimes in specified target areas, is also very important. The question of what proportion of granting agency funds should be dedicated to supporting research in particular target areas is a contentious issue. Some critics argue that certain research areas of socio‑economic importance to Canada are not receiving adequate funding and that more funds should be channelled to those areas. Other commentators argue that there is already an adequate proportion of granting agency funds directed towards supporting research in target areas, and more funds should not be directed there at the expense of funding for other types of research. The issue of how much funding should be directed to different research areas is clouded, according to come critics, by a lack of structured government policy in this area. These questions are explored in this chapter.

Relevance and “Strategic” Granting Programs

In addition to the excellence of the research proposal and investigators, and the need for funds, other factors are considered for certain programs when deciding on the allocation of research funds. For example, some programs at the federal agencies examine the industrial relevance of the proposed research and whether the project is likely to generate economic spin-offs for the Canadian economy within a reasonable period of time. Other programs focus on the socio-economic relevance of the proposed research in particular target areas of national importance. The objective of these “strategic” programs is to promote research and training in target areas with the ultimate goal of transferring knowledge or technology expertise to Canadian-based organizations that have a capacity to use the results; all three granting agencies offer some form of dedicated, strategic research programs. The research is usually conducted in conjunction with government or industry-based partners.

In fiscal year 2000-01, these strategic programs accounted for approximately 7% of expenditures at NSERC (spending on other non-targeted research with societal or industrial relevance conducted in partnership with industry and government agencies accounted for a further 9% of NSERC expenditures), 15% at SSHRC, and 20% at CIHR (Networks of Centres of Excellence not included in the calculations for the three agencies). Expenditures on strategic research at CIHR is expected to grow to between 30% and 40% of the total budget as the agency’s institutes, which began operating in February 2001, expand. Other research in target areas of national importance to Canada is funded through non-targeted research programs, so the actual expenditures on such research by each agency are higher than the amounts reported here. The proportion of research in target areas funded by the granting agencies has increased over the past few years:

Look at the agencies’ budgets … There has been an evolution over the course of the last 20 years … They have all, without exception, followed this trend. Today, between 20 and 30% of the agencies’ budgets, agencies which up until very recently only funded non-directed, non-strategic fundamental research, [are] already devoted to such programs. [Benoît Godin, Observatoire des sciences et des technologies, 66:09:55]

The percentage of expenditures at SSHRC on targeted research programs will be higher for 2002-03 because of the funds ($100 million over five years) that the government has invested in the Initiative on the New Economy (INE), which are managed by SSHRC. The main goal of this initiative is to help Canada and Canadians adapt to and benefit from the “New Economy.” The INE will focus on four major areas of research (that were selected by the federal government in consultation with a variety of stakeholders): general new economy issues, management and entrepreneurship, education and lifelong learning. A major departure in the evaluation process compared to other SSHRC programs is that before INE applications proceed to the adjudication stage, an interdisciplinary committee of academics and non-academics, with experience in the areas relevant to the four INE themes, screens the applications for relevance to INE objectives. Only applications that pass this “relevancy test” are sent on for traditional peer review.

For strategic research programs, the granting agencies all have mechanisms in place to choose and modify target areas. The methods used and the time periods between reviews of the areas vary among the agencies. At NSERC, the Strategic Projects Program supports research in target and emerging areas of national importance. The Program operates on a five-year cycle; the Program’s most recent evaluation of target areas was in 2000. The target area evaluation begins with a study of the existing national and international material on priority research areas identified by the private and public sectors. The results of this study are then validated with leaders in the academic, industrial and government sectors. In 2000, the strategic areas selected were: biosciences, environment and sustainable development, information and communications technologies, value-added products and processes, and “new directions.” At NSERC, the allocation of funds to various disciplines is also examined and modified every four years in its main Discovery Grants Program, which funds basic and applied research. Funds are shifted among disciplines during the “Reallocations Exercise” to ensure that science and engineering that is important for Canada receives appropriate levels of funding under the system.

At SSHRC, different methods have been used over the past 10 years to select strategic research areas. The selection is usually made following broad consultation with academics, university administrators, government representatives, members of non‑governmental organizations and other stakeholders. An “environmental scan” highlighting knowledge gaps in the social sciences and humanities is produced following the consultation process and is presented to SSHRC’s Governing Council. A short list of theme areas is drawn up, discussed with the community, and then the Council makes final decisions on theme areas to be selected. Reviews of the theme areas are made every three to five years to ensure that the areas selected are still relevant. The most recent set of strategic themes, selected in March 2002, are: culture, citizenship and identity (including issues involving peace and security); environment and sustainable development; texts, visuals and technology; and Aboriginal peoples.

At CIHR, the individual Institutes decide on thematic initiatives based on a rolling three- to five-year strategic “outlook.” The outlooks are developed by the Scientific Director of each Institute, working in partnership with each Institute’s multi-stakeholder Advisory Board. The CIHR Governing Council approves these broad outlooks, and from them the Institutes develop specific plans for research investments in the upcoming fiscal year. These plans are presented to the Research Planning and Priorities Committee (RPPC), which is made up of CIHR’s President, Vice-President (research portfolio) and the 13 Institute Directors, for appraisal and modification prior to launch. Following the publication of CIHR’s first overall strategic outlook,³¹ the RPPC is now considering a number of proposals for major cross-cutting research priorities which would involve a number of Institutes working together, in addition to the priority areas defined by each Institute individually. Examples include such themes as rural and northern health, gene‑environment interactions and disease, and tobacco control.

Other research in target areas is also supported through the granting agencies. The Networks of Centres of Excellence (NCE) Program is a tri-council and Industry Canada initiative that was established in 1989 and made a permanent Program in 1997. The Program promotes partnerships among universities, industry, government and non‑governmental organizations. There are currently 22 networks in four areas: health, human development and biotechnology, information and communication technology, natural resources, and engineering and manufacturing. Its present budget stands at $77.4 million per year.

The identification of research priorities within the NCE Program has been achieved in two ways: (1) through a bottom-up approach in which applicants propose a NCE in a given area and the applications are judged in open competition; and (2) through a top‑down approach in which specific research areas of national importance are identified in advance and proposals invited to address those areas. The first of these targeted competitions was conducted in 1995, the second one in 1999. Target areas are identified through consultations with the research community, with industry and with government. Prior to each competition, the NCE Steering Committee (composed of the presidents of the three federal granting agencies and the Deputy Minister of Industry Canada) decides on the need to target areas for new networks and, if the decision is to target, recommends targeted areas to the Ministers of Industry and Health. The federal government (Cabinet) makes the final decision.

Research in Interdisciplinary and Emerging Areas

Much of today’s research is interdisciplinary, crossing discipline boundaries within agencies and, in an increasing number of cases, crossing the boundaries of granting agency mandates. Other research is in emerging or small areas that does not fit well into the traditional, discipline-based structure of the major “open” granting programs at many agencies. Some researchers contend that research in interdisciplinary, emerging or small fields is disadvantaged under the present system. The Committee heard of such an example from a researcher who works in the area of animal health research:

NSERC has neither a mechanism nor the expertise to review or fund this [population] real-world [animal health] research. NSERC’s perspective on the issue is that there is no problem, since it doesn’t receive proposals for funding of this type of research. The reason they receive no proposals is that investigators will not submit proposals until there is an appropriate grant selection committee in place. [Ian Dohoo, Professor, University of PEI, 58:09:40]

The witness believed that areas not covered by the present committee structure are disadvantaged in the competition for funds. He argued that because of limited funds, committee members are reluctant to recommend funding for research in new areas at the expense of funding for the majority of proposals that are in more traditional areas. For animal health research, the witness suggested that NSERC should follow the example of CIHR in its approach to funding research:

So what’s the solution? Well, in the recent transformation of the Medical Research Council into the Canadian Institutes for Health Research, the need to fund the full spectrum of health research from basic laboratory research to real-world or population- and clinical-based research was clearly recognized, and that’s been built into the four pillars each institute is built upon. Ideally we need the same sort of transformation — the same approach to animal health research — at NSERC. [Ian Dohoo, Professor, University of PEI, 58:09:40]

NSERC contends that it funds both laboratory-based and population studies in animal health, and that a variety of NSERC programs are available to support animal health research. Despite this assurance, the Committee encourages NSERC to review its mechanisms for funding animal health research (in collaboration with other federal departments that work in this area, if necessary) to ensure that this area of research is
receiving proper consideration. Additionally, the Committee notes that Canada’s four veterinary colleges are facing other problems related to outdated or inadequate infrastructure that, if not remedied, could result in the colleges losing their international accreditation.³² Given the importance to Canadians of the research conducted by the veterinary colleges in such areas as food safety and animal disease, the Committee encourages the federal government, granting agencies, veterinary colleges and provinces to work together to quickly resolve these problems. 

For interdisciplinary research and research in emerging areas, the Committee heard that the small pool of reviewers available is a problem in terms of evaluating proposals in these areas:

We have an excellent system for judging incoming proposals. If it has flaws, it’s because the pool of academic reviewers is too small with the result that there is reviewer fatigue. This is particularly true for interdisciplinary panels. The problem is often compounded since the individual who may be familiar with your work often has to withdraw simply because they are in some way part of your small community with the result that the proposal does not receive the detailed review that it should. [J. Adam Holbrook, Associate Director, Centre for Policy Research on Science and Technology, Simon Fraser University, 66:09:10]

An example of the “small reviewer pool” problem was provided to the Committee for the area of polar research. The witness suggested that polar research is disadvantaged by
the system, because of the small research community and consequently a lack of researchers who can provide unbiased reviews of polar research proposals:

The small and shrinking community of polar scientists, in many disciplines, results in the fact that we know each other very well; we’ve usually co-operated on research projects; we’ve frequently co-authored papers with each other; and established partnerships in terms of funding with the granting councils, with government agencies and other people. And, as a consequence, establishing a peer review panel for major research proposals or even proposals within the current granting council system is really quite difficult. [Peter Johnson, Chair, Canadian Polar Commission, 75:09:15]

In the long term, the witness argued that the solution to improving the situation for the evaluation of polar research proposals is to build up the polar research community (for example, by implementing the recommendations of the Task Force on Northern Research³³). In the short term, the witness suggested that agencies should rely more on international reviewers to evaluate proposals in polar research (the agencies already do solicit the help of international reviewers in their review processes).

At CIHR, efforts are being made to ensure that the review process reflects its new, broader, interdisciplinary mandate:

The peer review system at CIHR is undergoing a period of evolution. We’ve introduced a dozen new committees in the past year to deal with increased application pressure and to accommodate CIHR’s broadened mandate, and we’re creating about 15 ad hoc committees to review the strategic or thematic initiative … . To try to find the best fit between a specific proposal and the team of reviewers that evaluates it, we’ll be forming clusters of similar peer review committees, which will meet at the same time, allowing for a flexible committee membership, better tailored to the review of a wide range of proposals. This cluster organization should also stimulate a more rapid evolution of committee mandates to anticipate, rather than react to, changes in the directions of health research. [Mark Bisby, CIHR, 39:15:30]

For their major research grants programs, both NSERC and SSHRC have interdisciplinary committees that evaluate research proposals at the boundary between disciplines or involving several disciplines that cannot easily be reviewed by the traditional discipline-based committees. The committee members represent a variety of disciplines. At NSERC, the interdisciplinary committee often requests reports on proposals from members of other grant selection committees, in addition to reports requested from external experts, in helping it reach a funding recommendation.

High-risk Research

Given that, on average, industry tends to shy away from conducting research with a high level of risk, governments have an important role to play in helping to fund this type of high-risk, high-payoff research. One witness contended that peer review works well for most types of research, including interdisciplinary research, as long as selection committees are appropriately structured. However, the witnessed questioned whether high-risk research receives proper consideration under peer review. According to the witness, the inherently high risk of failure associated with this type of research, the scarcity of funds, and the tendency for committees to reach consensus by making conservative funding recommendations lead to a bias against this type of research:

[P]robably the most serious problem with peer review is how it treats high-risk research. Despite the many federal agency pronouncements on the importance of supporting high-risk, high-payoff research, in reality there are few incentives and motivations for promoting truly high-risk research and there are many disincentives. … Use of committees for performing peer review, especially large committees, which is characteristic of many of the funding agencies, intrinsically leads to conservative judgments. [Ronald N. Kostoff, 88:10:10]

The witness suggested that “top-down” agency mechanisms may be the only way to ensure that this type of research receives appropriate levels of funding:

I don’t really have an answer to [the problem] other than hiring [agency staff] who basically are willing to take these risks and are willing to accept the failures. It becomes a very personal issue. I don’t see how one can in a sense legislate that organizations should be taking risks. [Ronald N. Kostoff, 88:10:35]

In some foreign funding agencies (e.g., the National Science Foundation in the United States), program officers or directors have the authority, in some cases, to allocate a small proportion of the budget envelopes that they manage to fund high-risk research or research in emerging areas. Furthermore, staff can override the funding recommendation of a selection panel if the panel’s decision is considered to be overly conservative. At the Canadian federal granting agencies, staff overseeing the operations of peer review committees does not generally have the authority to make independent funding decisions, alter budget envelopes, or unilaterally override the funding recommendations of the committees. 

The Committee realizes that the major “open” granting programs at the Canadian agencies allow researchers to pursue other, often riskier, lines of research than those detailed in their grant applications. The agencies, as part of program evaluations, are examining the support of high-risk and interdisciplinary research. Some effort (e.g., at CIHR) is being made to provide funds specifically for high-risk research projects. The Committee encourages the agencies to continue to monitor and improve the mechanisms for the support of high-risk research.

Increased Collaboration Among the Granting Agencies

The Committee was generally pleased with the mechanisms in place within each of the granting agencies, especially at the CIHR, to assess and modify the distribution of research funds within strategic programs. It questions, however, whether research in target areas of national importance is receiving adequate support from the overall funds available at NSERC and SSHRC, although it realizes that such research is also being funded through non-targeted programs at those agencies. Additionally, the Committee is concerned about whether support for research in emerging fields and high-risk or interdisciplinary research is receiving adequate consideration under the present system. The Committee is worried that interdisciplinary research, often in strategically important areas, that crosses the boundaries of the three granting agencies’ mandates (other than that supported through the NCE Program) is not receiving adequate consideration and appropriate levels of funding.

The Committee notes that the seven research councils in the United Kingdom have established “The Cross Council Research Forum,” an informal group of individuals whose goal is to improve cross-council operations. One of the topics it has tackled is the review of proposals that are at the “interface” of the six grant-awarding research councils. The Committee encourages the Canadian granting agencies to review the general operating principles³⁴ established by the U.K. councils for reviewing such proposals to see whether there are any novel principles that might be appropriate in the Canadian context. In May 2002, the U.K. government launched Research Councils UK which, working with the Office of Science and Technology, will develop new areas of collaboration in key scientific fields. It will build on areas where research councils are already working together, including the provision of interdisciplinary programs in such areas as genomics, e-science, basic technologies, stem cell research and climate change. Research Councils UK will also strive to harmonize procedures and provide a more efficient service for the councils’ applicants, by, for example, developing a single application route for all research councils.³⁵

The Committee believes that the Canadian federal granting agencies could co‑operate further to support strategic and interdisciplinary research that falls under the purview of all three agencies. It also believes that cross-council operations in general could be improved. The Committee recommends:

RECOMMENDATION 5

That the Government of Canada encourage the granting agencies to collaborate further with each other and with non-academic stakeholders, including the provinces and territories, when deciding on target areas within strategic programs. The agencies should also ensure that formalized review mechanisms and adequate levels of funding are available to support high-calibre interdisciplinary research or research in emerging areas, especially that which crosses the boundaries of the agencies’ mandates. The agencies should consider establishing a formal mechanism to facilitate collaboration on all levels so that their operations are more uniform, allowing for “seamless” access by the research community to federal research funds.

Basic and “Curiosity-driven” Research

Research that is deemed to be of immediate socio-economic importance to Canada is usually easier to justify to the government and public in terms of it being “value for money” than is basic research. The former, however, often depends heavily on advances in areas of basic research that may not have obvious or immediate value in a non-scientific arena:

The advances in biology and human health tomorrow will come from fundamental understandings in physics, in social sciences, in philosophy, etc. [Matthew Spence, President and Chief Executive Officer, Alberta Heritage Foundation for Medical Research, 66:09:30]

In many cases, the socio-economic value of a particular research program is not realized until years down the road when a discovery emanating from that research is applied to a specific problem. The majority of federal granting agency funding in Canada goes to fund research in areas chosen by the investigators themselves (“curiosity-driven” research, which may or may not have immediate industrial or societal relevance), and is not channelled into funding research in target areas selected by the granting agencies or government. According to some witnesses, most scientists support such a system and suggest that it may lead to much more useful research in the longer term than directly supporting research that has immediate and obvious socio-economic value and/or is in a particular target area:

The great thing about our Canadian system is that we have enough autonomy among our faculty and universities that we can have this interest-driven research that often is far more valuable in the long run than research that has an immediate commercial application. [Paul Davenport, President, University of Western Ontario, 51:10:15]

The question of what proportion of federal granting agency funding should be directed towards funding basic, applied and strategic research is a contentious issue that is often raised by scientists, funding agencies and governments around the world:

We ask this question for 50 years now, since government started investing in science and technology: What is the balance between fundamental research and applied research? There’s no magic formula to answer this … if you [look at] the funding council[s], a third of their funding is actually devoted to a kind of strategic research … Is that too much? Scientists say so. Government thinks no ... I think we should not go too far in that direction. [Benoît Godin, Director, Observatoire des sciences et des technologies, 66:10:25 ]

The Committee recognizes that there is no consensus on this issue.

Federal Research Priorities

Along with the issue of what proportion of federal funds should be directed towards supporting research in target areas deemed to be of national importance is the question of who should be deciding on which areas are to be funded. Target areas in strategic programs at the federal granting agencies are generally proposed by multi-stakeholder committees and decided upon by the agencies’ governing councils, or in the case of the NCEs, by Cabinet. Some critics point to the lack of clear research priorities at the federal level to explain the recurring debate that relates to the subject of the allocation of research funds:

I would suggest that one of the causes of this problem is simply that we do not have a clear set of priorities for our country. As a consequence, we try to fund all fields of research and thus cut a small pie into even smaller pieces. To be fair, the government has tried to remedy this situation by setting up special funds, programs and even institutions to focus scarce research funds in specific areas, but at the end of the day we still do not have these priorities. Rather, we have all-encompassing generalities such as economic development or social capital. [J. Adam Holbrook, Associate Director, Centre for Policy Research on Science and Technology, Simon Fraser University, 66:09:10]

Some national governments do channel relatively large (according to some members of the research community) proportions of funds for university-based research to target areas of national importance. For example, in January 2002, the Australian government directed the Australian Research Council (Australia’s main funding agency for basic research) to allocate 33% (about $130 million) of its total funds for the 2003 funding round to research projects in four priority research areas: nano-materials and biomaterials, genomics and gene expression, complex/intelligent systems, and photonics. The directive met with some criticism from the research community, but others feel that for small countries, such national priority setting is necessary:

It’s certainly an issue which is very contentious, but the argument in favour of it in relation to Australia is that we have a very small population. We have a large landmass, but in terms of the population, the spread of our expertise, we can’t possibly afford to cover everything and priority setting at some stage is inevitable. [Fiona Wood, University of New England, Australia, 79:19:55]

In terms of decisions on the allocation of federal research funds in Canada, the Committee shares the concerns expressed by some of the witnesses about the lack of a clear set of federal priorities in this area. The Committee believes that a stronger federal S&T advisory framework is necessary to ensure that the government is receiving appropriate and adequate counsel for setting federal research priorities and policy. Such advice would ensure that the government is providing sufficient funding for research in strategically important areas while at the same time maintaining a strong support base for other research. The Committee was encouraged to see that the issue of decision making and priority setting is mentioned in the federal government’s Innovation Strategy, and that the government will consider establishing a national, arm’s length science organization (the Canadian Academies of Science) to provide independent assessments on science‑based issues of national importance.³⁶ The Committee notes that other industrialized countries have S&T advisory frameworks that include a Chief Scientific Adviser to the executive branch of government who is usually head of a government office of science and technology policy (e.g., in the United Kingdom and the United States). Some countries also have priority-setting programs that identify future S&T directions and needs (e.g., the United Kingdom’s Foresight program). In terms of Canada’s S&T advisory framework, the Committee recommends:

RECOMMENDATION 6

That the Government of Canada establish a more formalized mechanism, in consultation with the provinces and territories, for setting or modifying S&T policy, deciding on funding priorities, and ensuring that they are implemented. Such a framework could include a science advisory body and/or Chief Scientific Adviser that would report directly to Parliament.