Bearc8tBot - Your Personal Assistant for Campus Resources

Rolland Muzeya

Large Language Models (LLMs) have gained widespread popularity, offering solutions to diverse problems for users worldwide. However, the computational demands of high-performing models, such as those with 70 billion parameters, present significant resource constraints. This project aims to address this issue by deploying a smaller (~7 billion parameter) model as a chatbot to assist users with general inquiries about Lander University. Users can interact with the model and swiftly obtain information regarding campus resources, alleviating the cumbersome task of navigating through many links on the university website. Experimentation involved running the Llama2-uncensored and Mistral 7B models offline using Ollama, as well as utilizing AnythingLLM to generate summaries from PDF documents sourced from the Lander website. Notably, this system operated offline on a CPU, showcasing its resource efficiency. The next steps would be to train a lightweight model on publicly available documents from the university's website and use Ollama, Langchain, and Streamlit to develop a Retrieval-Augmented Generation (RAG) application that allows users to interact with the model offline and on their computers. The model’s performance will be assessed using precision and recall scores. This project's implications extend beyond enhancing user experience when searching for Lander online resources. It also paves a way for institutional adoption and further model refinement, enhancing productivity for faculty, staff, and students alike.

• Rolland Muzeya graduated from Mazowe Boys High School in Harare, Zimbabwe. He is a senior computer information systems major with a software development emphasis, and minors in mathematics and cybersecurity. His achievements include working as a computer repair technician on campus, presenting undergraduate research in computer information systems, and he will be joining BlackRock as a software engineer in the fall of 2024.