Prompt Interaction: Search Redux

Issue #106

by Gary Marchionini (UNC School of Information & Library Science)

Continued development and application of generative AI (GenAI) depend on significant investments in energy, materials, and data acquisition, but the greatest investments are in human talent to create computational improvements and novel applications to the world’s problems. Information professionals play important roles in GenAI progress (see Marchionini, 2024¹ for a set of roles), one of which is what is popularly known as prompt engineering. I prefer the term prompt interaction because it puts the focus on continuing human involvement and control of GenAI use. Prompt interaction initiates GenAI problem solving and guides the process toward useful outcomes. Several recent papers focus on prompt interaction as a process and as a user interaction design challenge. Here I highlight two different threads of work along these lines, one that addresses prompt interaction for research purposes and one that aims to make prompting easier for casual use by novice populations. I then argue that these and other works in information science are new steps in a long arc of information science research on the search process and search systems.

Chirag Shah’s Communications of ACM June 2025 paper² argues for human-in-the-loop collaborative prompting that is inspired by qualitative research methods that employ multiple coders to develop and assess a code book for data collection and interpretation. The paper notes that both under specification and overspecification (over fitting) of prompts to chatbots lead to biased outcomes. The approach entails a four-phase pipeline that begins with a preliminary prompt, identifying and clarifying criteria for evaluating outputs, iteratively refining the prompts, and validating the overall pipeline. Phase 2 is the key component and involves multiple humans who assess the preliminary outcomes, examine intercoder reliability, discuss the efficacy of their assessments, and execute subsequent multiple runs to sharpen the shared criteria (the “codebook”). Shah and his colleagues tested this approach on two use cases: discovering user intent in search and auditing the effectiveness of LLMs. In both cases, the approach was found to yield trustworthy and reliable results. The author acknowledges that this approach is significantly more expensive in time and resources (at least 4X) than simply using a single human or LLM iteratively but argues that the value of scientific progress demands outputs that are unbiased and trustworthy. This work illustrates the value that information scientists bring to the table when applying GenAI to research.

The second example comes from a recent posting by Ben Shneiderman in Medium³ arguing that designers should work to provide better user interfaces to support GenAI prompting. He argues that faceted prompt interfaces can help increase discoverability, reduce cognitive load, support exploratory prompting, and enhance consistency and guidance. These effects are well-known for faceted search systems that are widely deployed for ecommerce and other search environments (see Tunkelang, 2009⁴ for a treatment of faceted search development and implementation). 

These current examples build upon decades of research on search strategy and online search system design that lie at the core of information science. Our field has a long history of search strategy research and education from the earliest days of online databases. Pioneering research on search strategy by Marcia Bates (e.g., 1979)⁵, and others such as Meadow and Cochrane’s 1981⁶ successive fraction strategy, Markey and Cochrane’s 1981⁷ pearl growing strategy, Hawkins and Wager’s 1982⁸ interactive scanning strategy, and Harter’s 1986⁹ building block strategy set the stage for systematic search before full-text search emerged in the WWW. Throughout the 1980-2000 years, most information schools required online searching instruction as part of the core curriculum. As WWW-based information became the norm, research continued to make full-text and multimedia searching accessible to everyone through well-structured data views (e.g., faceted search) and highly interactive user interfaces (e.g., supporting exploratory and collaborative search). In addition to research on general search problems, support emerged for specialized applications such as systems and strategies tuned to high-recall needs such as legal and patent searching, searching in non-textual databases, for recurring or ongoing searches (e.g., alerting, recommendations) and for complex searches to synthesize literatures (e.g., systematic reviews and meta-analyses), as well as studies of the effects of different system implementations to understand and build next-generation information systems (e.g., Gusenbaur & Haddaway’s 2020¹⁰ systematic comparison of more than two dozen search environments for systematic review applications provides specific criteria for assessing search outcomes). 

Information schools took the lead in this research and education over the decades, and they are taking the lead today to investigate how GenAI might be leveraged to improve search for different needs and by different information seekers. Almost half a century of progress brings us to today’s search engines augmented by GenAI chatbots and synthesized outputs. Results of current studies are reported in information science conferences such as the ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR), the iConference, the ASIST Conference, as well as in information and computer science journals. In addition to the classical user studies and design evaluations, information researchers are investigating novel ways to insert LLMs into the search process, whether to create systematic pipelines like the Shah study described above or comparing large-scale adversarial trials across LLMs and search tasks (e.g., Triem and Ding, 2024¹¹). It continues to be exciting to work in the information field and progress will surely continue to develop in the years ahead.

---

1: Marchionini, G. (2024), Information and library professionals' roles and responsibilities in an AI-augmented world. J Assoc Inf Sci Technol, 75: 865-868. https://doi.org/10.1002/asi.24930

2: Shah, C. 2025. From prompt engineering to prompt science with humans in the loop. Communications of the ACM, 68(6), June 2025, 54-61.

3: Shneiderman, B. 2025. Don’t GenAI designers want a better prompt interface? Faceted UI to support prompt interactions. Medium https://medium.com/@ben.shneiderman/dont-genai-designers-want-a-better-prompt-interface-71c5fa84f85b

4: Tunkelang, D. (2009). Faceted Search. Springer-Nature. https://doi.org/10.1007/978-3-031-02262-3 5: Bates, M. (1979). Information search tactics. Journal of the American Society for Information Science, 30(4), 205-214.

6: Meadow, C.T. & Cochrane, P. (1981). Basics of online searching. New York: John Wiley & Sons.

7: Markey, K. & Cochrane, P. (1981). Online training and practice manual for ERIC data base searchers (2nd Edition). Syracuse, NY: ERIC Clearinghouse on Information Resources.

8: Hawkins, D.T. & Wagers, R. (1982). Online bibliographic search strategy

development. Online, 6(3), 12-19.

9: Harter, S.P. (1986). Online information retrieval: Concepts, principles, and techniques. Orlando, FL: Academic Press.

10: Gusenbauer M, Haddaway NR. Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Res Synth Methods. 2020 Mar;11(2):181-217. doi: 10.1002/jrsm.1378. Epub 2020 Jan 28. PMID: 31614060; PMCID: PMC7079055. https://pmc.ncbi.nlm.nih.gov/articles/PMC7079055/ 

11: Triem, H. and Ding, Y. (2024), “Tipping the Balance”: Human Intervention in Large Language Model Multi-Agent Debate. Proceedings of the Association for Information Science and Technology, 61: 361-373. https://doi.org/10.1002/pra2.1034

---

Feature Stories solely reflect the opinion of the author.