If you have ever wondered if a local AI LLM can hold their own against the latest frontier models then allow me to offer you a small, real world example, regarding my own medical health.

I have reached that stage of life (50s) where everything is monitored and any slight movement, we get on it. I still have time to make life-style changes to correct anything that comes up prior to it becoming “suck it up and live with it buttercup” sort of thing. This translates to getting blood work done twice a year and seeing what has or has not moved.

This week was that time, and as usual, once they extracted what felt like pints of blood (I still can’t look at the process for fear of being flat on my back) my medical portal lit up with all the results coming back. Downloaded them all, eyeballed them, and one of the usual still markers, blipped. Oh.

My doctor would be calling me soon, telling me to either not panic or start getting my affairs in order. Prior to that, I thought, let me experiment with my local Gemma 4 26B LLM to see what “Marge” had to say, and then I can validate that against what Claudia would say.

Security First

Before tossing my detailed medical records into the AI pit, I desensitized them a little. I took image grabs of each of the PDF pages, and then blacked out the names/addresses of me and the doctor. These would null and void any lurking fields inside of the PDF – though since I had done a print from the browser, chances are it was a binary file anyway.

Dr Claudia (Claude)

First one, was Claude. I prompted with the following, attaching all the images.

After some churning, Claudia returned noting that most of them were “clean or near clean. Four flags, and only two are worth a conversation”. She the proceeded to detail what they were, why, and what the normal levels should be, and the sort of things that would cause that.

Now at this point, not once did Claudia remind me she was AI and not medically qualified to know for sure or to give me recommendations.

Dr Marge (LM Studio running Gemma 26b locally)

I opened up my local LLM and gave it the exact same prompt and files. Off it spun to do its work. Not only was I giving it images to go through and extract out, it was running in a much smaller context due to the fact I have a limited memory on my GPU (Nvidia 5070Ti).

After a few minutes (notably longer than Dr Claudia), she began to render her analysis. The first line was comforting: “Disclaimer: I am an AI, not a doctor. These results must be reviewed by your healthcare provider for a formal medical interpretation.“. Right out of the gate, she is noting this needs oversight, in wonderful bold text.

The results, while formatted differently, was spot on as to what Dr Claudia deduced. It was a good long piece, detailing in language a non-medical person could understand. It ended the piece with suggested questions I should ask my doctor.

I then prompted some more questions, to which it again prefaced “As an AI, I cannot provide medical recommendations or tell you whether or not to undergo a medical procedure. Only a licensed healthcare professional can make that decision based on your complete medical history, lifestyle, and risk factors.“.

This whole exchange, took about 5 minutes to run and used only 50k of context. Color my “im” as pressed.

I repeated the experiment, but this time, I removed the images, and fed Dr Marge, the raw PDF files. Pretty much the same analysis came back (different wording naturally, since AI will never return word-for-word the same thing).

To be clear, my LLM in this instance was not hooked up to any MCP or tools. It was not reaching out to the internet for more information.

Fascinating

Regular readers will know that I am bullish on the usage of local AI LLM for a variety of reasons, with security being one of them (stability and cost being the others). When it comes to medical records, you can’t get more of a personal sensitive document. These are the ones that I frankly do not trust the large AI models to not be training on.

It is easy to conclude that the Dr Marge was a little more empathetic, reminding me she was AI, and even giving me suggested questions to ask my real doctor but I am not going to put too much behind that. It was just a nice touch without being prompted to do so.

The conclusion is that the local LLM, running securely on your own machine (or infrastructure) is more than up for most of the every day tasks we are throwing at the frontier models, but with the added benefit of knowing your data is being retained, and never leaving the building. I was impressed at how well it done on these blood results and how it detailed it was. I am inspired to go through some of my legacy reports and see what Dr Marge says.

Now with that, I have a call to go and make a phone call, to talk to a real medical professional to see what the damage is.

AI Disclaimer: Gemini Nano Banana Pro was used to generate the photo – from the 1974 “Young Frankenstein“ movie.