Characterizing and quantifying uncertainty in forest pest risk analyses
PARTNERS: North Carolina State University (NCSU) Department of Forestry and Environmental Resources; Landscape Analysis and Applications Section, Great Lakes Forestry Centre, Canadian Forest Service
SUMMARY: Eastern Threat Center scientists conduct research resulting in national-scale databases and reports that address forest health and sustainability, and the development, demonstration, and transfer of new protocols for assessing forest health. Team members have participated in research on special risk analyses, especially spatially explicit risk analyses (i.e., pest risk maps) for individual alien-invasive species affecting forests. The current analytical framework for such analyses typically overlooks the impact of uncertainty in input data and assumptions, and how these uncertainties propagate to map outputs. The research described here highlights methods for incorporating variability and uncertainty into models of forest pest risk. Developing tools for handling uncertainty in forest pest risk assessments will provide important information to the Forest Service, States, and other stakeholders who are faced with making critical and timely decisions on forest management and the best allocation of often-scarce resources. In particular, such tools will enable Forest Service and other scientists to incorporate uncertainty into their risk assessments and thus provide better appraisals of current forest pest threats.
EFETAC's ROLE: This project is supported by Eastern Threat Center funding and collaborative research.
STATUS: Ongoing
PROGRESS: Eastern Threat Center research ecologist Frank Koch worked with Canadian Forest Service scientists Denys Yemshanov and Dan McKenney, along with Marla Downing and Frank Sapio (USDA Forest Service Forest Health Technology Enterprise Team), to adapt a generic bioeconomic modeling framework for examining pest invasions through time for the purpose of mapping risk (i.e., probability of invasion) and associated uncertainties. They chose the sirex woodwasp (Sirex noctilio) as their example pest because it has invaded both Ontario and parts of the northeastern United States during the last few years, offering a relevant opportunity to perform broad-scale, cross-border analyses. Their first study illustrates an integrated risk mapping methodology using stochastic simulation, which is more fully detailed in an article published in the journal Risk Analysis:
- Yemshanov, D.; Koch, F.H.; McKenney, D.W.; Downing, M.C.; Sapio, F. 2009. Mapping invasive species risks with stochastic models: a cross-border US-Canada application for Sirex noctilio Fabricius. Risk Analysis. 29(6): 868-884. (PDF)
Koch was also a co-author on a follow-up article that details common approaches for generating pest risk maps, then uses the example outlined above to highlight the advantages of an integrated approach, including the ability to jointly depict risk and uncertainty in a manner helpful to decision makers. Less technical than the first, this second article is intended for a broader audience of scientists and managers:
- Yemshanov, D.; McKenney, D.W.; Pedlar, J.H.; Koch, F.H.; Cook, D. 2009. Towards an integrated approach to modeling the risks and impacts of invasive species. Environmental Reviews. 17: 163-178. (PDF)
In a third article, again using sirex woodwasp as the test case, Koch and co-authors analyzed several key invasion model parameters, examining at what level of parametric uncertainty the output invasion risk and uncertainty maps become unstable. This approach attempts to ascertain and illustrate at what level(s) of parametric uncertainty output predictions remain reliable and are thus “robust” to uncertainty (see related discussion of “info-gap” analysis, below). The article received a 2009 Best Paper award from the Society for Risk Analysis:
- Koch, F.H.; Yemshanov, D.; McKenney, D.W.; Smith, W.D. 2009. Evaluating critical uncertainty thresholds in a spatial model of forest pest invasion risk. Risk Analysis. 29(9): 1227-1241. (PDF)
Info-gap analysis is a non-probabilistic approach to characterizing uncertainty in risk assessment, which attempts to establish where risk estimates remain reliable (i.e., robust) in the face of severe uncertainty. The concept was advanced by Yakov Ben-Haim from the Techion (Israel) Institute of Technology, who collaborated with Koch, Yemshanov, and Bill Smith (USDA Forest Service, retired) on the application of info-gap decision theory to the selection of the most robust pest detection survey network developed from the above-described sirex woodwasp risk map:
- Yemshanov, D.; Koch, F.H.; Ben-Haim, Y.; Smith, W.D. 2010. Robustness of risk maps and survey networks to knowledge gaps about a new invasive pest. Risk Analysis. 30(2): 261-276. (PDF)
In follow-up work involving info-gap decision theory, the researchers examined the trade-off between robustness to uncertainty and opportuneness, which is the possibility that a certain level of uncertainty in a pest risk model may enable “windfall” success (e.g., unanticipated but timely detections of an invasive pest). This work was published in the Journal of Environmental Management:
- Yemshanov, D.; Koch, F.H.; Ben-Haim, Y.; Smith, W.D. 2010. Detection capacity, information gaps and the design of surveillance programs for invasive forest pests. Journal of Environmental Management. 91:2535-2546. (PDF)
Koch collaborated with Yemshanov, Mark Ducey (University of New Hampshire), Klaus Koehler (Canadian Food Inspection Agency), and Barry Lyons (Canadian Forest Service) on another approach for quantifying uncertainty. The stochastic dominance concept has been applied in financial analysis as a way to identify the best choice out of a set of possible investment portfolios. The financial concepts of estimated “net return” on investment, and the “volatility” in that estimate, can be seen as analogous to the concepts of predicted invasion risk and the uncertainty of that risk prediction. Stochastic dominance is noteworthy in that it integrates uncertainty by comparing entire distributions of predicted outcomes, rather than adopting a more limited frame of reference (such as comparing only average values). The researchers developed a methodology for analyzing the human-assisted spread of the emerald ash borer (Agrilus planipennis), a major forest pest in North America. They used a spatial simulation model to generate distributions of potential invasion outcomes over eastern and central Canada, and then used stochastic dominance criteria to rank all locations in the study region based on these distributions. This work was published in the journal Diversity and Distributions:
- Yemshanov, D.; Koch, F.H.; Lyons, D.B.; Ducey, M.; Koehler, K. 2012. A dominance-based approach to map risks of ecological invasions in the presence of severe uncertainty. Diversity and Distributions 18:33-46. (PDF)
Additionally, Koch co-authored three articles about methods for adapting financial portfolio allocation techniques to combine risk and uncertainty estimates in output map products. The articles emphasize the fact that portfolio-based techniques can facilitate the incorporation of decision-making preferences about uncertainty into the outputs, particularly a decision maker’s possible aversion to uncertainty when determining management priorities (such as selecting the “best” geographic locations for pest surveillance). The articles were published in the journals Diversity and Distributions, NeoBiota, and Ecological Economics:
- Yemshanov, D.; Koch, F.H.; Ducey, M.; Koehler, K. 2013. Mapping ecological risks with a portfolio-based technique: incorporating uncertainty and decision-making preferences. Diversity and Distributions 19: 567-579. (PDF)
- Yemshanov, D.; Koch, F.H.; Ducey, M.J.; Haack, R.A.; Siltanen, M.; Wilson, K. 2013. Quantifying uncertainty in pest risk maps and assessments: adopting a risk-averse decision maker’s perspective. NeoBiota 18: 193-218. (PDF)
- Yemshanov, D.; Koch, F.H.; Lu, B.; Lyons, D.B.; Prestemon, J.P.; Scarr, T.; Koehler, K. 2014. There is no silver bullet: the value of diversification in planning invasive species surveillance. Ecological Economics 104: 61-72. (PDF)
In another line of research, Koch worked with Yemshanov and various co-authors on articles describing analytical techniques for evaluating uncertainties associated with key assumptions of network models, which are used to depict human-mediated dispersal pathways for forest pests. The first, published in the Journal of Environmental Management, focused on uncertainties in a network model of the potential movement of alien forest species in Canada via commercial road transportation and domestic trade:
- Yemshanov, D.; Koch, F.H.; Ducey, M.J.; Siltanen, M.; Wilson, K.; Koehler, K. 2013. Exploring critical uncertainties in pathways assessments of human-assisted introductions of alien forest species in Canada. Journal of Environmental Management 129: 173-182. (PDF)
The second article, published in the journal NeoBiota, focused on uncertainties in a network model of the potential movement of alien forest insects in firewood transported by campers:
- Koch, F.H.; Yemshanov, D.; Haack, R.A. 2013. Representing uncertainty in a spatial model that incorporates human-mediated dispersal. NeoBiota 18: 173-191. (PDF)
Recently, Koch, Yemshanov, and co-authors published three book chapters about their work on the topic of uncertainty in pest risk models and maps. In particular, Koch was the lead author on a chapter that revisited the 2009 Risk Analysis article, “Evaluating critical uncertainty thresholds in a spatial model of forest pest invasion risk”. It provides a detailed instructional guide that will better enable pest risk analysts and other practitioners to apply the described methods to their own work. The two chapters led by Yemshanov refer to their ongoing research regarding uncertainty in the decision-making context. The chapters can be found in two different books, both published by CABI (Centre for Agriculture and Biosciences International):
- Koch, F.H.; Yemshanov, D. 2015. Identifying and assessing critical uncertainty thresholds in a forest pest risk model. Chapter 13 (pp. 189-205) in Venette, R.C. (Ed.) Pest Risk Modeling and Mapping for Invasive Alien Species. Wallingford, UK: CABI Publishing. (PDF)
- Yemshanov, D.; Koch, F.H.; Ducey, M.J. 2015. Making invasion models useful for decisionmakers: incorporating uncertainty, knowledge gaps, and decision-making preferences. Chapter 14 (pp. 206-222) in Venette, R.C. (Ed.) Pest Risk Modeling and Mapping for Invasive Alien Species. Wallingford, UK: CABI Publishing. (PDF)
- Yemshanov, D.; Koch, F.H.; Ducey, M.J.; Haack, R.A. 2015. Towards reliable mapping of biosecurity risk: incorporating uncertainty and decision-makers’ risk aversion. Chapter 12 (pp. 217-237) in Jarrad, F.; Choy, S.L.; Mengerson, K. (Eds.) Biosecurity Surveillance: Quantitative Approaches. Wallingford, UK: CABI Publishing. (PDF)
LINKS:
February 16, 2010: Researchers Receive Top Honors For Risk Analysis Paper
"Risk Analysis Journal Honors Eastern Threat Center Research" (related article from CompassLive)
CONTACT: Frank Koch, Eastern Threat Center Research Ecologist, frank.h.koch@usda.gov or (919) 549-4006
Updated August 2016
