DeepFruits: A fruit detection system using deep neural networks

Sa, Inkyu, Ge, ZongYuan, Dayoub, Feras, Upcroft, Ben, Perez, Tristan, & McCool, Christopher (2016) DeepFruits: A fruit detection system using deep neural networks. Sensors, 16(8), Article Number-1222.

View at publisher (open access)

Abstract

This paper presents a novel approach to fruit detection using deep convolutional neural networks. The aim is to build an accurate, fast and reliable fruit detection system, which is a vital element of an autonomous agricultural robotic platform; it is a key element for fruit yield estimation and automated harvesting. Recent work in deep neural networks has led to the development of a state-of-the-art object detector termed Faster Region-based CNN (Faster R-CNN). We adapt this model, through transfer learning, for the task of fruit detection using imagery obtained from two modalities: colour (RGB) and Near-Infrared (NIR). Early and late fusion methods are explored for combining the multi-modal (RGB and NIR) information. This leads to a novel multi-modal Faster R-CNN model, which achieves state-of-the-art results compared to prior work with the F1 score, which takes into account both precision and recall performances improving from 0.807 to 0.838 for the detection of sweet pepper. In addition to improved accuracy, this approach is also much quicker to deploy for new fruits, as it requires bounding box annotation rather than pixel-level annotation (annotating bounding boxes is approximately an order of magnitude quicker to perform). The model is retrained to perform the detection of seven fruits, with the entire process taking four hours to annotate and train the new model per fruit.

Impact and interest:

0 citations in Scopus
Search Google Scholar™

Citation counts are sourced monthly from Scopus and Web of Science® citation databases.

These databases contain citations from different subsets of available publications and different time periods and thus the citation count from each is usually different. Some works are not in either database and no count is displayed. Scopus includes citations from articles published in 1996 onwards, and Web of Science® generally from 1980 onwards.

Citations counts from the Google Scholar™ indexing service can be viewed at the linked Google Scholar™ search.

Full-text downloads:

13 since deposited on 28 Aug 2016
13 in the past twelve months

Full-text downloads displays the total number of times this work’s files (e.g., a PDF) have been downloaded from QUT ePrints as well as the number of downloads in the previous 365 days. The count includes downloads for all files if a work has more than one.

ID Code: 98421
Item Type: Journal Article
Refereed: Yes
Keywords: visual fruit detection, deep convolutional neural network, multi-modal, rapid training, real-time performance, harvesting robots, horticulture, agricultural robotics
DOI: 10.3390/s16081222
ISSN: 1424-8220
Subjects: Australian and New Zealand Standard Research Classification > ENGINEERING (090000) > ELECTRICAL AND ELECTRONIC ENGINEERING (090600) > Control Systems Robotics and Automation (090602)
Divisions: Current > Research Centres > ARC Centre of Excellence for Robotic Vision
Current > Schools > School of Electrical Engineering & Computer Science
Current > Institutes > Institute for Future Environments
Current > QUT Faculties and Divisions > Science & Engineering Faculty
Copyright Owner: Copyright 2016 The Author(s)
Deposited On: 28 Aug 2016 23:29
Last Modified: 29 Aug 2016 22:36

Export: EndNote | Dublin Core | BibTeX

Repository Staff Only: item control page