Self-hosted open-weight models change the PHI math
For a few years, the case for sending healthcare data to a proprietary model API was simple. The best models were proprietary, so you accepted the governance trade-off.
That case is much weaker now. Open-weight models such as Llama’s successors, Mistral, Qwen and DeepSeek have closed much of the gap with proprietary models. More important, they’re good enough for the bounded tasks that matter most in healthcare: summarizing clinical notes, pulling details out of prior auth requests, suggesting ICD codes and drafting discharge instructions.
And you can run them inside your own cloud network.
Healthcare has been paying a quiet price for AI. Each time a clinical note goes to an outside inference API, someone in compliance gets nervous, and for good reason. The usual workarounds are familiar. You de-identify before inference, sign a BAA with the API vendor and write prompts that strip identifiers.
Those work up to a point. De-identification misses things. A BAA is a promise, and you’re relying on the vendor to keep it. Every extra hop in the pipeline is another place data can leak.
The root problem was where the best models lived. They ran on someone else’s infrastructure, so you had to choose between model quality and control of your PHI.
What changes when the model runs in your stack
Open-weight models small enough to run on your own GPUs have improved a lot, thanks to better training data, better tuning and fine-tuning for specific domains. For data teams, that has three practical effects.
Inference inside the warehouse. Snowflake Cortex runs inference inside your data perimeter. With support for open-weight models, your Snowflake account can act as a model endpoint, and no data leaves your cloud account.
Fine-tuning on your own data. Proprietary models are fixed. You can fine-tune an open-weight model on your health system’s clinical language, formulary, payer rules and coding patterns.
Audit trails you own. When inference runs in your stack, every input and output lands in your logs, not a third party’s.
A tiered approach
The pattern I’d recommend sorts AI work into tiers by sensitivity and difficulty.
- General tasks, such as summarizing, reformatting and pulling out low-sensitivity details. Use self-hosted open-weight models, served with Ollama, vLLM on Kubernetes or Snowflake Cortex.
- Clinical tasks, such as clinical NLP, coding help and prior auth classification. Use open-weight models fine-tuned on your data, running on dedicated GPUs inside your network.
- Hard reasoning, used rarely, such as complex multi-step clinical reasoning, research and generating synthetic data. Use proprietary APIs, and only with de-identified or synthetic inputs.
This mirrors how most health systems already tier cloud workloads for cost and compliance. What’s new is that the first two tiers no longer cost you much in quality.
Fine-tuning is the hard part
Here’s where most efforts stall. Getting from “we have an open-weight model” to “we have a useful clinical tool” runs through fine-tuning, and most health systems don’t have the setup for it.
Fine-tuning needs labeled clinical datasets that someone has curated. It needs GPU capacity, which most IT purchasing processes weren’t built to buy. It needs MLOps pipelines to version and evaluate models. And it needs regular retraining as clinical practice changes.
I see three workable routes. You can build an internal ML platform with a dedicated clinical NLP team. You can work with a healthcare-focused partner that offers fine-tuned open-weight models as a service. Or you can use Snowflake ML and Cortex to absorb the infrastructure work and put your own people on curating data.
The model weights are free. The cost is in the work around them.
If you lead a data platform, here’s what I’d do this quarter:
- Map where PHI reaches outside inference APIs today. Build that map before compliance asks for it.
- Compare Snowflake Cortex’s support for open models against the AI workloads you run now. Its cost and compliance profile has changed in the last two quarters.
- Pilot fine-tuning on one bounded, low-risk task. ICD code suggestion from free text is a good choice, because your warehouse already holds the correct codes for past encounters to train and test against.
- Set up model governance early. A model you fine-tune is your model. It needs a model card, version control, bias testing and a retirement plan, and your compliance team probably has no procedures for any of that yet.
If you only do one of these, do the first. You can’t decide which AI work to bring in-house until you know where your patient data goes today.
Discussion