AI over care notes: what it can honestly do, and where it must stop

This is for coordinators and quality leads weighing up AI in a care record — for an NDIS, aged-care or community service — who need to know what the technology actually reads, what it’s allowed to write, and where the privacy obligations sit.

In care software, “AI” usually means one thing: a language model reading text and producing more text. It is not machine learning trained on your outcomes, and it is not predictive analytics. It has no clinical judgement. It cannot diagnose, triage in the medical sense, or decide anything on a participant’s behalf. Anyone selling you an AI that builds your roster, or one that makes clinical decisions, should be able to tell you exactly what the model does mechanically — and in a care record a human decides, always; if they can’t say that plainly, be sceptical.

The safest framing, and the one worth holding a vendor to, is that AI in a care record should be assistive and office-only: it reads what staff have already written, it produces its own separate output, and a human decides what happens next. It should never edit, block or author the actual care documentation — the progress note, the incident report, the support plan — because those are legal and clinical records that a person is accountable for.

The useful version of this technology runs a periodic pass over progress notes — new ones and edited ones, because an edit can change the meaning of what was flagged — and surfaces things a busy coordinator might miss: a mention of a fall, a note that reads like distress, a shift that went sideways. Done properly, each signal should carry:

• a category (health, behavioural, shift issue, client distress, safeguarding, or something else)
• a confidence score, not a certainty
• a verbatim excerpt from the actual note, so you can check it yourself in seconds
• a severity and a record of where it came from

That last point matters more than it sounds. A signal with no traceable excerpt is just a black-box alert you either trust blindly or ignore. A signal you can click through to the exact sentence is something you can act on.

From there the workflow should look like ordinary triage: open, follow-up, acknowledged, cleared, or escalated to a formal incident with the detail already carried across. Nothing should close itself. A well-built system also lets an organisation tune the sensitivity — how far back it looks, which categories it cares about, which note types it scans, how confident it needs to be before it bothers anyone — because what counts as noise for one service is a real flag for another.

A related but separate idea is a documentation quality check: comparing what’s been written against an organisation’s own minimum-note rules, plus a judgement on whether the note actually says enough, not just whether it’s long enough. That’s a genuinely useful nudge for supervision conversations. It is not a performance review tool, and it shouldn’t be treated as one.

First: it reads progress notes. It does not read your document library, your risk assessments, or your uploaded PDFs. If a vendor implies otherwise, ask them directly which record types the model actually opens. Some products in this market do cover documents as well as notes — a real point of difference worth checking, not blurring.

Second: any org-wide summary or “at-risk clients” view that’s built from your data is built from named records, not de-identified ones. If a rollup tells a manager which participants need attention this week, it is doing that by name. Never accept “de-identified” or “anonymised” as a description of that kind of view — it isn’t, and claiming it is creates a false sense of privacy.

The same logic applies to two newer features worth understanding: voice dictation of notes, and recording a meeting to get a summary.

Dictation should only ever append to whatever a carer has already typed — it shouldn’t replace or silently edit anything — and the record should show that speech was involved, so a reviewer isn’t reading a transcript thinking it’s typed prose. A meeting summary tool should work the same way in spirit: it can separate speakers, pull out decisions and actions with owners, but it should be fully editable, and only a deliberate human click should file it into anyone’s record. Nothing should land in a care file automatically because a microphone was on.

This is the one place to be precise, because it’s where a provider’s own privacy obligations kick in.

Speech-to-text at this maturity level is not processed on Australian soil — it’s typically sent to a transcription service offshore and converted to text. Do not assume the audio is then thrown away. For the asynchronous transcription APIs these products actually use, the vendor’s documented default is usually to retain both the audio and the transcript — zero-retention tends to be a separate product tier covering live streaming, not the batch path a care app uses. So the sharper question to put to a vendor is not whether the audio is stored, it is whether their software issues an explicit delete once the text is back, and on which paths. Ask about the one where a recording was abandoned part-way, because that is the path that leaks quietly. The text that comes back is what gets stored, and it should sit in an Australian-resident database that belongs to the organisation, not to whichever transcription vendor did the conversion.

Two separate facts, and they should never be merged into one reassuring sentence:

1. The recording leaves Australia, briefly, to be transcribed.
2. The resulting text is stored in Australia, in the provider’s own instance.

“Your notes are stored in Sydney” is true. “Your audio never leaves Australia” is not, and no provider should let a vendor’s marketing collapse the two. Under APP 8, the disclosure obligation attaches to the fact that data crosses a border for processing — not to where it ends up sitting afterwards. That means this is genuinely your compliance responsibility as the provider, not just the software’s.

Because that obligation is yours, it’s worth having ready-made wording rather than drafting it from scratch under time pressure: a clause for your privacy policy, a clause for your service agreements, and a plain sentence a staff member can say out loud before they hit record on a meeting. All three should name the actual region the processing happens in and your organisation by name — generic wording that could apply to anyone is not a real disclosure.

Be wary of anyone claiming otherwise. This category of feature is new across the sector, not proven over years, and no serious vendor should be quoting accuracy figures for it — a hit rate quoted off a handful of notes is marketing, not evidence, because the sample sizes involved are never large enough to mean anything. Treat every signal, every summary and every dictated note as a first draft that a person checks, not a verdict.

Corella runs this model of AI — office-only, note-reading, human-in-the-loop — as switchable modules included in the one price, not a separate AI tier. It’s honest that dictation and meeting summaries involve sending audio to a transcription step in the European Union before the text lands in your Australian instance, and it’s honest that the residency detail is two facts rather than one. It doesn’t read documents, only progress notes, and it won’t pretend it’s more mature than it is.