Canada's privacy regulators on AI in health

If you build AI for healthcare, Canada's privacy regulators have set out what they expect from you and from the hospitals that buy from you.
Canada's federal, provincial and territorial privacy regulators adopted a joint resolution in Ottawa on 15 September 2026, and the Office of the Privacy Commissioner announced it on 29 October. It makes recommendations and asks governments to legally require governance and accountability frameworks based on risk assessments.
The part I would read first as a founder is the list for developers: know where your training data came from, share your risk assessment findings with clients, and give users a way to challenge outputs. Buyers are asked to weigh those vendor assessments before adopting a tool.
My view: this reads as a due diligence list even for teams outside Canada. A hospital's procurement team could ask these questions tomorrow, and it helps to have the answers written down first.
AI in health should be:
- privacy protective;
- valid and reliable;
- safe;
- human rights affirming;
- transparent and accountable.
- Adopted in Ottawa, 15 September 2026, by Canada's federal, provincial and territorial privacy regulators
What you are asked to do:
- use de-identified or synthetic data wherever possible;
- confirm that training data is lawfully sourced;
- set the most protective privacy defaults;
- keep human-in-the-loop oversight, and a way to challenge outputs;
- assess privacy, security and AI risks throughout the life cycle.
- Joint resolution, 15 Sep 2026
What you are asked to share with clients:
- evidence of training;
- a general explanation of how the system works;
- risk assessment findings and downstream mitigations;
- a demonstration that you comply with relevant privacy law.
- Joint resolution, 15 Sep 2026
What your buyers are asked to do:
- weigh vendors' assessment results before adopting a tool;
- limit third-party uses in contracts, with audits and no unauthorised commercialisation;
- train staff and prevent shadow AI, meaning use without approval;
- have a human verify outputs.
- Joint resolution, 15 Sep 2026
Could you answer these if a hospital asked?
- Where did your training data come from?
- What did your last risk assessment find?
- How can a user challenge an output?
- Who checks outputs before they are acted on?
- My questions, drawn from the resolution
See the full carousel on Instagram.
Sources
- https://www.priv.gc.ca/en/about-the-opc/what-we-do/provincial-and-territorial-collaboration/joint-resolutions-with-provinces-and-territories/res_20260915_aihealth/
- https://www.priv.gc.ca/en/opc-news/news-and-announcements/2026/nr-c_261029/
Questions about this? Legal Help.