Comparing XML-IR Query Formation Interfaces

Abstract

XML information retrieval (XML-IR)
systems differ from traditional information retrieval
systems by using structure of XML documents to
retrieve more specific units of information than the
documents themselves. Users interact with XML-IR
systems via structured queries that express their
content and structural requirements. Historically,
it has been common belief within the XML-IR
community that structured queries will perform better
than traditional keyword-only queries. However,
recent system-orientated analysis has show that this
assumption may be incorrect when system performance
is averaged over a set of queries. Here, we test this
assumption with users via a simulated work task
experiment. We compare a keyword only interface
with two user friendly XML-IR interfaces: NLPX, a
natural language interface and Bricks, a query-bytemplate
interface. This is the first time that a XML-IR
natural language interface has been tested in user
experiments. We compare the retrieval performance
of all three interfaces and the usability of the two
structured interfaces. Our results correspond to those
of the system-orientated evaluation and indicate that
structured queries do not aid retrieval performance.
They also show that in terms of retrieval performance
and usability the structured interfaces are comparable.