Martin Halvey

Leif Azzopardi, Mateusz Dubiel, Martin Halvey et al.

The conversational search task aims to enable a user to resolve information needs via natural language dialogue with an agent. In this paper, we aim to develop a conceptual framework of the actions and intents of users and agents explaining how these actions enable the user to explore the search space and resolve their information need. We outline the different actions and intents, before discussing key decision points in the conversation where the agent needs to decide how to steer the conversational search process to a successful and/or satisfactory conclusion. Essentially, this paper provides a conceptualization of the conversational search process between an agent and user, which provides a framework and a starting point for research, development and evaluation of conversational search agents.

Mateusz Dubiel, Martin Halvey, Leif Azzopardi

Despite significant improvements in automatic speech recognition and spoken language understanding - human interaction with Virtual Personal Assistants (VPAs) through speech remains irregular and sporadic. According to recent studies, currently the usage of VPAs is constrained to basic tasks such as checking facts, playing music, and obtaining weather updates.In this paper, we present results of a survey (N = 118) that analyses usage of VPAs by frequent and infrequent users. We investigate how usage experience, performance expectations, and privacy concerns differ between these two groups. The results indicate that, compared with infrequent users, frequent users of VPAs are more satisfied with their assistants, more eager to use them in a variety of settings, yet equally concerned about their privacy.

Laura Hasler, Martin Halvey, Robert Villa

In this position paper we argue that certain aspects of relevance assessment in the evaluation of IR systems are oversimplified and that human assessments represented by qrels should be augmented to take account of contextual factors and the subjectivity of the task at hand. We propose enhancing test collections used in evaluation with information related to human assessors and their interpretation of the task. Such augmented collections would provide a more realistic and user-focused evaluation, enabling us to better understand the evaluation process, the performance of systems and user interactions. A first step is to conduct user studies to examine in more detail what people actually do when we ask them to judge the relevance of a document.