Teaching robots generalisable hierarchical tasks through natural language instruction

Suddrey, Gavin, Lehnert, Christopher, Eich, Markus, Maire, Frederic D., & Roberts, Jonathan M. (2016) Teaching robots generalisable hierarchical tasks through natural language instruction. IEEE Robotics and Automation Letters. (In Press)

[img] Accepted Version (PDF 886kB)
Administrators only | Request a copy from author

View at publisher


Natural language provides a convenient means of communicating information, and as such, is an ideal medium for enabling non-expert users to teach robots novel tasks. However in order to take advantage of natural language, a series of challenges must first be overcome. These challenges include the need to

  • a) generalise learnt tasks to novel scenarios without retraining,

  • b) resolve problems encountered during task execution, and

  • c) derive implicit information from knowledge about the domain.

To solve these challenges, this paper presents a novel approach to learning complex hierarchical tasks through natural language instruction, which not only allows learnt tasks to be generalised to novel situations without the need for retraining, but also enables an agent to derive implicit information from domain knowledge. Additionally, the approach presented in this paper enables the agent to infer task properties, such as preconditions and effects, directly from the explanation of the task flow. We validate our approach by demonstrating an implementation of our algorithms both on a simulated agent, as well as a Baxter robot. In each case, the agent is provided with a small set of primitive tasks for manipulating its workspace. From these primitives, we demonstrate the ability to teach the agent increasing complex tasks, including tasks of table cleaning, solely through natural language instructions.

Impact and interest:

Citation counts are sourced monthly from Scopus and Web of Science® citation databases.

These databases contain citations from different subsets of available publications and different time periods and thus the citation count from each is usually different. Some works are not in either database and no count is displayed. Scopus includes citations from articles published in 1996 onwards, and Web of Science® generally from 1980 onwards.

Citations counts from the Google Scholar™ indexing service can be viewed at the linked Google Scholar™ search.

ID Code: 97612
Item Type: Journal Article
Refereed: Yes
Keywords: Human-Robot Interaction, Planning, Scheduling and Coordination, Autonomous Agents
DOI: 10.1109/LRA.2016.2588584
ISSN: 2377-3766
Divisions: Current > Research Centres > ARC Centre of Excellence for Robotic Vision
Current > Schools > School of Electrical Engineering & Computer Science
Current > QUT Faculties and Divisions > Science & Engineering Faculty
Copyright Owner: IEEE (Institute of Electrical and Electronics Engineers)
Deposited On: 01 Aug 2016 01:44
Last Modified: 07 Aug 2016 06:32

Export: EndNote | Dublin Core | BibTeX

Repository Staff Only: item control page