TLDR: CHOIR is an LLM-based chatbot designed to improve organizational memory in university research labs by integrating with chat platforms like Slack. It offers document-grounded Q&A, Q&A sharing, knowledge extraction from conversations, and AI-assisted document updates. A one-month deployment in four labs showed it effectively helped students retrieve information and directors update documents, but also revealed a tension between student privacy in asking questions and directors’ need for awareness of knowledge gaps. The study highlights AI’s role as a mediator, fostering communication and collaborative knowledge management.
University research labs are bustling hubs of innovation, but they often face a common challenge: valuable knowledge shared through chat platforms like Slack or Microsoft Teams can easily get lost in the endless stream of messages. While documentation exists, it requires constant upkeep and can be difficult for students to navigate, leading to professors repeatedly answering the same questions.
To tackle this, researchers designed CHOIR (Chat-based Helper for Organizational Intelligence Repository), an advanced chatbot system powered by Large Language Models (LLMs). Embedded directly within communication platforms like Slack, CHOIR aims to transform how research labs manage and preserve their collective knowledge. For more detailed information, you can refer to the original research paper: CHOIR: A Chatbot-mediated Organizational Memory Leveraging Communication in University Research Labs.
CHOIR’s Four Core Functions
CHOIR is built around four interconnected features designed to make knowledge accessible and maintainable:
1. Document-grounded Q&A: Students can ask CHOIR questions directly in Slack, and the chatbot provides answers drawn from existing lab documents, complete with references to the source material. This helps students quickly find information and become familiar with the lab’s documentation.
2. Q&A Sharing for Follow-up Discussion: If a question isn’t fully answered or requires further discussion, users can easily share the Q&A exchange with peers or the lab manager. This encourages collaborative problem-solving and ensures that unanswered questions get the attention they need.
3. Knowledge Extraction from Conversation: As new knowledge emerges from everyday Slack conversations—perhaps a professor shares a writing tip or students discuss a new procedure—any lab member can summon CHOIR to extract this valuable information and draft a suggested update for the organizational memory.
4. AI-assisted Document Update: Lab managers receive these suggested updates and can review, edit, and approve them with AI assistance. This streamlines the documentation maintenance process, making it easier to keep information current without significant manual effort.
Real-World Deployment and Key Insights
CHOIR was deployed for one month in four university research labs, involving 21 participants (lab directors and students). The study aimed to understand how lab members engaged with and experienced this chatbot-driven system.
Students quickly embraced CHOIR, often perceiving it as a “senior in the lab” or a “librarian” due to its consistent availability and non-judgmental responses. They appreciated its speed and accuracy in retrieving information. Interestingly, lab culture played a significant role in how questions were asked. In one lab with a strong collaborative culture, students frequently asked questions publicly, viewing it as a way to help others and identify documentation gaps. In contrast, students in other labs preferred private direct messages to CHOIR, often out of concern about asking “dumb questions” or bothering peers and professors.
Lab directors also found innovative ways to use CHOIR, sometimes employing it as a personal assistant to proactively share important information or acknowledge that documentation had been updated in response to a student’s query.
While the Q&A sharing feature was designed to foster communication, most questions remained unshared, as students often opted for direct communication with peers or directors. This created a “privacy-awareness tension,” where directors had limited visibility into the questions students were asking privately, thus missing opportunities to identify and address knowledge gaps in the documentation.
The AI-assisted document update feature was highly valued by directors for simplifying maintenance. However, student contributions to documentation updates were limited. Students often lacked confidence in providing universally accurate answers or struggled to generalize their personal experiences into formal documentation, highlighting a psychological barrier to collective knowledge curation.
Also Read:
- Researchers’ Digital Brains: Strategies for Knowledge Management in Obsidian
- Guiding AI Experts: Perspectra’s Forum-Style System Boosts Critical Thinking in Research
Future Directions for AI-Mediated Knowledge Management
Participants offered valuable suggestions for improving CHOIR. They proposed features like aggregated, anonymized summaries of student questions for managers, which would provide awareness of knowledge gaps without compromising individual privacy. There was also a strong desire to expand CHOIR’s knowledge base beyond internal documents to include external websites, Google Docs, and even cross-lab knowledge sharing.
The study’s findings underscore a crucial insight: AI agents like CHOIR can act as powerful mediators, facilitating human interaction and preserving knowledge, rather than simply replacing human communication. By reducing social barriers (through anonymity) and providing shared reference points, CHOIR fosters a more collaborative and efficient knowledge ecosystem in research labs. The research suggests that future systems should embrace a broader definition of “documentable knowledge,” including contextual and case-based information, and continue to explore how AI can enhance, not diminish, human connection and collaboration within organizations.
