Ma, Yuan; Donkers, Tim; Kleemann, Timm; Ziegler, Jürgen:
An Instrument for measuring users’ meta-intents
In: CHIIR '23 : Proceedings of the 2023 Conference on Human Information Interaction and Retrieval / Gwizdka, Jacek; Rieh, Soo Young (Eds.). - 2023 Conference on Human Information Interaction and Retrieval ; CHIIR 2023 ; March 19–23, 2023, Austin, Texas, USA - Washington: ACM, 2023, pp. 290 - 302
2023book article/chapter in ProceedingsClosed access
Computer Science
Title in English:
An Instrument for measuring users’ meta-intents
Author:
Ma, YuanUDE;Donkers, TimUDE;Kleemann, TimmUDE;Ziegler, JürgenUDE
Open Access?:
Closed access
Language of text:
English

Abstract in English:

We propose the concept of meta-intents which represent high-level user preferences related to the interaction and decision-making in conversational recommender systems (CRS) and present a questionnaire instrument for measuring meta-intents. We conducted a two-stage user study, an exploratory study with 212 participants on Prolific, and a confirmatory study with 394 participants on Prolific. We obtained a reliable and stable meta-intents questionnaire with 22 question items, corresponding to seven latent factors (concepts). These seven factors cover important interaction preferences and are closely related to users' decision-making process. For example, the factor dialog-initiative reflects whether users prefer to follow the system's guidance or ask their own questions in a CRS. We conducted statistical analyses of meta-intents in two domains (smartphones and hotels), and a general chatbot scenario. We also investigated the influence of additional factors (demography, decision-making style) on meta-intents through Structural Equation Modeling (SEM). Our results provide preliminary evidence that the proposed meta-intents are domain and demography (gender, age) independent. They can be linked to the general decision-making style and can thus be instrumental in translating general decision-making factors into more concrete design guidance for CRS and their potential personalization. Meta-intents also provide a basis for future analyses of interaction behavior in CRS and the development of a cognitively founded theoretical framework.