top of page

Using AI for Health Information? Careful!

Writer: Leanna Coy, FNP-C
Leanna Coy, FNP-C
Oct 26
2 min read
Neon skeletons representing AI health information

AI is quickly becoming a go-to source for information. But these large language models (LLMs) are still in their early stages, and their flaws are still being worked out. Many reported episodes show these bots giving wrong and potentially harmful advice. For example:


  • Telling people to eat rocks as a way to add minerals and vitamins to their diet

  • Advising the addition of glue into pizza sauce to keep cheese from sliding off

  • The National Eating Disorders Association removed its chatbot after the AI repeatedly gave information that could worsen eating disorders

  • Incorrectly listing a Pap smear as a screening test for vaginal cancer


One of the many problems with asking an LLM for health information is that you can mislead the AI with your questions. What do I mean? If you say to an AI, "I think I have diabetes. What can I do to treat it?" The AI isn't going to question whether or not you ACTUALLY have diabetes. It will accept this self-diagnosis you have just given. Because chatbots are trained to be people-pleasers, they're unlikely to disagree with you.


This can lead to potentially harmful recommendations. The LLM can offer advice that is technically correct, but likely inappropriate because it doesn't understand the full context of your question. AIs are trained on exam-style questions and don't understand that when a real person asks a health question, they have legitimate concerns and gray areas.


This also means the AI is just spewing out facts. It cannot diagnose or treat you. They don't know your health history - and you likely shouldn't give it to them. A health professional is trained to understand what you are asking, along with the follow-up questions and tests needed to find an accurate diagnosis and treatment plan. They also know where to draw the line for patients. AI doesn't.


For example, a study found that when an LLM was asked how to perform a medical procedure at home, it warned that it wasn't safe and should only be done by a health professional. But then it went ahead and gave step-by-step instructions, completely ignoring the risks. An LLM can't reason like a human. A doctor or nurse knows when and why not to give out information.


AIs are also known to spread misinformation. This is because LLMs pull information from anywhere and everywhere on the internet. This includes websites you would never consider for health information because they aren't reputable.


You can safely use AI to help interpret health information. Ask it to clarify confusing medical terms, such as "What does triple-negative breast cancer mean in plain language?" Use it to simplify health research into something you can read and understand more easily. Remember to protect your privacy by not sharing your personal information with the AI. Don't give it your full name, birth date, medical records, or insurance details. Most LLMs collect, store, and potentially use the information you type into them.


It's best to stay mindful of your privacy and the AI's limitations. Use it to gather basic information that can help you talk with a health care provider about your concerns.

  • LinkedIn
bottom of page