Central Library

Normal view MARC view ISBD view

Interactive IR User Study Design, Evaluation, and Reporting [electronic resource] / by Jiqun Liu, Chirag Shah.

By: Liu, Jiqun [author.].
Contributor(s): Shah, Chirag [author.] | SpringerLink (Online service).
Material type: materialTypeLabelBookSeries: Synthesis Lectures on Information Concepts, Retrieval, and Services: Publisher: Cham : Springer International Publishing : Imprint: Springer, 2019Edition: 1st ed. 2019.Description: XVII, 75 p. online resource.Content type: text Media type: computer Carrier type: online resourceISBN: 9783031023194.Subject(s): Computer networks  | Computer Communication NetworksAdditional physical formats: Printed edition:: No title; Printed edition:: No title; Printed edition:: No titleDDC classification: 004.6 Online resources: Click here to access online
Contents:
Preface -- Acknowledgments -- Introduction -- Interactive Information Retrieval -- Methodology: Paper Selection and Coding Scheme -- Faceted Framework of IIR User Studies -- Evaluating IIR User Studies of Different Types -- Implications and Limitations of the Faceted Framework -- Conclusion and Future Directions -- Appendix -- Bibliography -- Authors' Biographies.
In: Springer Nature eBookSummary: Since user study design has been widely applied in search interactions and information retrieval (IR) systems evaluation studies, a deep reflection and meta-evaluation of interactive IR (IIR) user studies is critical for sharpening the instruments of IIR research and improving the reliability and validity of the conclusions drawn from IIR user studies. To this end, we developed a faceted framework for supporting user study design, reporting, and evaluation based on a systematic review of the state-of-the-art IIR research papers recently published in several top IR venues (n=462). Within the framework, we identify three major types of research focuses, extract and summarize facet values from specific cases, and highlight the under-reported user study components which may significantly affect the results of research. Then, we employ the faceted framework in evaluating a series of IIR user studies against their respective research questions and explain the roles and impacts of the underlying connections and "collaborations" among different facet values. Through bridging diverse combinations of facet values with the study design decisions made for addressing research problems, the faceted framework can shed light on IIR user study design, reporting, and evaluation practices and help students and young researchers design and assess their own studies.
    average rating: 0.0 (0 votes)
No physical items for this record

Preface -- Acknowledgments -- Introduction -- Interactive Information Retrieval -- Methodology: Paper Selection and Coding Scheme -- Faceted Framework of IIR User Studies -- Evaluating IIR User Studies of Different Types -- Implications and Limitations of the Faceted Framework -- Conclusion and Future Directions -- Appendix -- Bibliography -- Authors' Biographies.

Since user study design has been widely applied in search interactions and information retrieval (IR) systems evaluation studies, a deep reflection and meta-evaluation of interactive IR (IIR) user studies is critical for sharpening the instruments of IIR research and improving the reliability and validity of the conclusions drawn from IIR user studies. To this end, we developed a faceted framework for supporting user study design, reporting, and evaluation based on a systematic review of the state-of-the-art IIR research papers recently published in several top IR venues (n=462). Within the framework, we identify three major types of research focuses, extract and summarize facet values from specific cases, and highlight the under-reported user study components which may significantly affect the results of research. Then, we employ the faceted framework in evaluating a series of IIR user studies against their respective research questions and explain the roles and impacts of the underlying connections and "collaborations" among different facet values. Through bridging diverse combinations of facet values with the study design decisions made for addressing research problems, the faceted framework can shed light on IIR user study design, reporting, and evaluation practices and help students and young researchers design and assess their own studies.

There are no comments for this item.

Log in to your account to post a comment.