Author: Chloe Matecki, PharmD
Artificial intelligence (AI) and large language models (LLM) have taken a turn for the ubiquitous, including flooding both clinical practice and academic settings. For those who teach or precept learners, this has become a constant discussion, especially as the topic can be difficult to breach for all parties involved. Many remain hesitant to implement AI fully while others point to the wide-spread implementation across digital spaces like Google Gemini’s top of page search results or Microsoft’s Copilot that is standard in their suite of products. Others still worry about being left behind without learning to use platforms like OpenEvidence, which promises more reliable information based on their unique database. As AI use rises and more evidence comes out about its impact on learners’ information retention and critical thinking skills, it is important to examine what can currently be done to promote conscientious use in the pharmacy school context.
AI tools and LLM are commonly marketed to students and young professionals alike. Grammarly, a tool that started as an online proofreader, began a full push into AI spaces with AI writing services, checkers, and “humanizers” that students can use to generate more human-appearing written pieces. Another less publicized tool is their citation agent, which will attempt to match references to a user’s claim. This tool is not alone in generating something called AI hallucinations but can prove to be quite dangerous if used in backing up a medical recommendation.
AI hallucinations themselves happen due to the very nature of most of the model and LLM designs themselves. These LLMs cannot “think” in the same way humans can, understanding how and why words are connected. A pharmacist can understand that aripiprazole and olanzapine are both second generation antipsychotics as well as each of their nuances. AI and LLM can only associate them with the statistical associations with the words surrounding them. The tools therefore may fill in gaps that are missing in the databases they pull from with hallucinations because they are predicting incorrectly. As educators and preceptors, we are tasked with teaching students how to not only memorize but understand the topics and their nuances to prepare them for clinical practice. The issue that can arise on over-reliance with LLM and AI tools is that they themselves cannot think and have the potential to lead students astray. As these LLMs are also trained to be “yes-bots” and provide answers to the prompts given, hallucinations and the polite, confident tone answers can take can make unvetted use especially problematic for learners and preceptors alike.
AI and LLM tools can also pose threats to protected health information (PHI). Several free-to-use tools like ChatGPT do not sign agreements that assure non-release or prevent breaches of HIPAA compliance laws with their security standards. The prompts that students may use in these LLMs are taken, processed, and then used in the future to better respond to subsequent similar prompts. The issue again is that there are no safeguards for data extraction if HPI is put into prompts. This concern is not just one for students but represents a great counseling point for students and pharmacists alike to mention to patients.
Patients and providers alike may also come to appointments with information from AI sources. Training ourselves while we train students to respond to these potentially false or misunderstood prompt responses can be a joint growing pain as more people use AI in all aspects of their lives. Approaching these encounters with compassion and an open mind, especially with patients, remains key. It is entirely possible that they will bring something to consider in their personalized treatment versus dismissing every AI generated prompt as incorrect. As mentioned previously, a tool like OpenEvidence has gained a lot of trust for its database sources. As an LLM, it has partnered with respected medical journals like the New England Journal of Medicine, JAMA, and Cochrane Review, as well as pulling medication information from FDA drug labels. When integrating the use of something like OpenEvidence, it once again requires nuance, as learners should be reminded to vet the articles as sources it pulls for them.
With the discussions points mentioned in this article, there remains the issue of safely, thoughtfully integrating these tools that learners are likely already using into precepting and academia. I have implemented an AI clause in my rotation syllabus similar to one many may have in their general syllabi. Institutions may have this as a requirement, but it often pays off to be specific in usage limits. I require disclosures for every use, with a general ban on its use past idea generation. Writing- and assessment-heavy assignments, like drug information questions or journal clubs, must be 100% human made. Outlines, source searches, or proofreading prompts are allowed with disclosures. I can recognize this feels restrictive to students, but in a news environment littered with stories of how AI chatbots have failed the youth leading to immense tragedy and worsened psychosis with prolonged use, I remain cautious for both my students and patients’ sake.
I would encourage all those who precept to take a more in-depth look at how the LLMs and AI tools work and decide how best to integrate discussions of their use with learners. It is not going away and having the working knowledge to understand how to explain their uses and risks to learners may lead to more productive conversations for us all.