Comparison of scoring systems for invasive pests using ROC analysis and Monte Carlo simulation
Makowski, David & Mittinty, Murthy (2010) Comparison of scoring systems for invasive pests using ROC analysis and Monte Carlo simulation. Risk Analysis, 30(6), pp. 906-915.
Different international plant protection organisations advocate different schemes for conducting pest risk assessments. Most of these schemes use structured questionnaire in which experts are asked to score several items using an ordinal scale. The scores are then combined using a range of procedures, such as simple arithmetic mean, weighted averages, multiplication of scores, and cumulative sums. The most useful schemes will correctly identify harmful pests and identify ones that are not. As the quality of a pest risk assessment can depend on the characteristics of the scoring system used by the risk assessors (i.e., on the number of points of the scale and on the method used for combining the component scores), it is important to assess and compare the performance of different scoring systems. In this article, we proposed a new method for assessing scoring systems. Its principle is to simulate virtual data using a stochastic model and, then, to estimate sensitivity and speciﬁcity values from these data for different scoring systems. The interest of our approach was illustrated in a case study where several scoring systems were compared. Data for this analysis were generated using a probabilistic model describing the pest introduction process. The generated data were then used to simulate the outcome of scoring systems and to assess the accuracy of the decisions about positive and negative introduction. The results showed that ordinal scales with at most 5 or 6 points were sufﬁcient and that the multiplication-based scoring systems performed better than their sum-based counterparts. The proposed method could be used in the future to assess a great diversity of scoring systems.
Impact and interest:
Citation counts are sourced monthly from and citation databases.
These databases contain citations from different subsets of available publications and different time periods and thus the citation count from each is usually different. Some works are not in either database and no count is displayed. Scopus includes citations from articles published in 1996 onwards, and Web of Science® generally from 1980 onwards.
Citations counts from theindexing service can be viewed at the linked Google Scholar™ search.
|Item Type:||Journal Article|
|Keywords:||invasive species, pest risk assessment, ROC, scoring systems, sensitivity|
|Subjects:||Australian and New Zealand Standard Research Classification > MATHEMATICAL SCIENCES (010000) > STATISTICS (010400) > Applied Statistics (010401)
Australian and New Zealand Standard Research Classification > ENVIRONMENTAL SCIENCES (050000) > ECOLOGICAL APPLICATIONS (050100) > Invasive Species Ecology (050103)
Australian and New Zealand Standard Research Classification > AGRICULTURAL AND VETERINARY SCIENCES (070000) > CROP AND PASTURE PRODUCTION (070300) > Crop and Pasture Protection (Pests Diseases and Weeds) (070308)
|Divisions:||Past > QUT Faculties & Divisions > Faculty of Science and Technology
Past > Institutes > Institute for Sustainable Resources
Past > Schools > Mathematical Sciences
|Deposited On:||19 May 2011 06:52|
|Last Modified:||10 Aug 2011 15:52|
Repository Staff Only: item control page