Balisage Paper: Clean SOAP

The Risks to Quality

As described earlier, AI Scribe is a rapidly evolving product category that consists of tools that are authorized to listen into a patient/doctor conversation and generate a “Visit Note” in the SOAP format.

SOAP is a loose format for Physician Visit Notes which was developed by Dr. Lawrence Weed in the 1960's at the University of Vermont. Lawrence Weed explained his zeal for organized records like this:

While AI Scribes offer significant advantages in the automation of SOAP note generation, several risks are associated with potential mistakes they could make. These risks can impact patient care, data integrity, legal compliance, and reimbursement. Key risks include:

This paper is focused on the first four issues, although Elation Health takes all seven very seriously. Contracts prevent us from publishing evaluation results for specific Note Generation models, but this paper describes our evaluation mechanism for these factors.

AI Scribe and SOAP Notes

Having established the stakes, let us turn to the problem. Given an audio trasnscript for an input text like this (from the ACI-Bench corpus):

hey brandon you know glad to see you in here today i see on your chart that you're experiencing some neck pain could you tell me a bit about what happened
yeah i was in a car crash
wow okay when was that
well which car crash
okay so multiple car crashes alright so let's see if we can how many let's start
my therapist said well my well actually my mother said i should go see the therapist and the therapist said i should see the lawyer but my neck's hurting
okay so i'm glad that you know you're getting some advice and so let's let's talk about this neck pain how many car crashes have we had recently
well the ones that are my fault or all of them
all of them
i was fine after the second crash although i was in therapy for a few months and then after the third crash i had surgery but i was fine until this crash
okay the most recent crash when was that
that's when i was coming home from the pain clinic because my neck hurt and my back hurt but that was in february
okay alright so we had a car crash in february
what year it was which february it was
okay so let's let's try with this one see what happens hopefully you remember i need you to start writing down these car crashes that this is becoming a thing but you know it's okay so let's let's say maybe you had a
you're not judging me are you
no there's no judgment here whatsoever i want to make sure that i'm giving you the best advice possible and in order to do that i need the most information that you can provide me makes sense
yes
alright so we're gon na say hope maybe that you had a car crash and we can verify this in february of this year and you've been experiencing some neck pain since then right
yes
okay alright on a scale of one to ten what ten is your arm is being cut off by a chainsaw severe how bad is your pain
twelve
okay terrible pain now i know you mentioned you had previous car crashes and you've been to therapy has anyone prescribed you any medication it's you said you went to a pain clinic yes
well they had prescribed it recently i was i was on fentanyl
oh
i haven't gotten a prescription for several weeks
okay alright and so we will be able to check on that when you take your medication so before you take your medication rather like are you able to move like are you experiencing any stiffness
…

We would like an output like this:

CHIEF COMPLAINT
    Neck pain.

HISTORY OF PRESENT ILLNESS
    Brandon Green is a 46-year-old male who presents to the clinic today for the
    evaluation of neck pain. His pain began when he was involved in a motor vehicle
    accident in 02/2022 when he was on his way home from a pain clinic. The patient
    notes that he has been in 4 motor vehicle accidents; however, he notes that he was
    fine after the first two accidents, but the third motor vehicle accident is when his
    neck and back pain began. He states that he was in therapy following the second
    accident and had surgery after his third accident. The patient was seen at a pain
    clinic secondary to neck and back pain. He was prescribed fentanyl; however, he has
    not received a prescription for several weeks. Today, he reports that his pain is a
    12 out of 10. He describes his pain as sharp and incapacitating with stiffness and
    pain. The patient also reports headaches, occasional dizziness. He denies any recent
    visual disturbances. He also reports numbness in his left arm and right leg. The
    patient also reports spasms throughout his body. He states that he has been
    experiencing fatigue since the accident. He notes that he is unable to work with
    this much pain.

REVIEW OF SYSTEMS
    Constitutional: Reports fatigue.
    Eyes: Denies any recent visual disturbances.
    Musculoskeletal: Reports neck and back pain, and occasional swelling and bruising
        of the neck.
    Neurological: Reports headaches, dizziness, spasms, and numbness.
    PHYSICAL EXAM
    SKIN: No lacerations.
    MSK: Examination of the cervical spine: Pain on palpation on the bony process and
        muscle. Moderate ROM. No bruising or edema noted.

RESULTS
    X-rays of the neck reveal no fractures.

ASSESSMENT
    Neck sprain.

PLAN
    After reviewing the patient's examination and radiographic findings today, I have
    had a lengthy discussion with the patient in regards to his current symptoms. I have
    explained to him that his x-rays did not reveal any signs of a fracture. I
    recommended an MRI for further evaluation. I have also prescribed the patient
    Robaxin 1500 mg every 6 to 8 hours to treat his pain. I have also advised him to
    utilize ice, a heating pad, IcyHot, or Biofreeze on his neck as needed. I have also
    provided him with a home exercise program to work on his range of motion. I advised
    the patient that he will not be able to work until we have the MRI results.

INSTRUCTIONS
    The patient will follow up with me after his MRI for results.

Or…depending on the physician’s tastes, perhaps like this:

CHIEF COMPLAINT
    Patient is experiencing neck pain.

HISTORY OF PRESENT ILLNESS
    * Motor Vehicle Accident (MVA) - February 2023 (V43.52XA)
        - Most recent car crash was in February of this year.
        - Patient reports fatigue since the accident.
    * Pain (R52)
        - Patient rates pain as 12 out of 10.
        - Patient experiences stiffness and sharp, incapacitating pain.
        - Patient reports it really hurts to bend neck forward and backward.
    * Headaches (R51)
        - Patient reports headaches, occasional dizziness, and numbness in the left arm
        and right leg.
    * Muscle Spasms (M62.838)
        - Patient reports muscle spasms.

ASSESSMENT AND PLAN
    * Neck Sprain (ICD10: S13.4XXA)
        - Doctor's assessment is neck sprain.
        - Doctor orders an MRI for a more thorough image.
        - Doctor prescribes Robaxin, 1500 mg every 6-8 hours.
        - Doctor suggests using a heat pad or biofreeze if Robaxin is not effective.
        - Doctor plans to refer patient to physical therapy and pain medicine for
        possible local injections.

PAST HISTORY
    Patient was in multiple car crashes.

CURRENT MEDICATIONS

    Patient was previously prescribed fentanyl but has not had a prescription for
        several weeks.

PHYSICAL EXAM
    Neck: Pain on palpation both on the bony process and on the muscle. Patient has
        moderate range of movement in the neck.
    Skin: No bruising, swelling, or lacerations observed during the exam.

DATA REVIEWED
    X-ray shows no fracture.

PATIENT INSTRUCTIONS
    * Doctor advises using ice for bruising and swelling.
    * Doctor provides exercises to improve neck movement.
    * Patient will be given time off work until MRI results are available.

Can an automated system achieve such a task? And can we be confident enough in it to trust it with patient transcripts?

Evaluating Transcription and Transcript Cleanup

Most of these systems operate in two steps: first audio is transcribed into textual transcripts, and then transcripts are converted into SOAP Notes. One concern we had was that the first part of this process might make transcription errors and that the second step would compound the errors.

Instead, we generally found the opposite - the second step would usually correct for most errors in the first step. This is because the note generation process has more time and more context to work than the real-time transcription process.

We also found that it is possible to inject an explicit “cleanup this transcript” step between the audio transcription and the note generation, to improve the output quality even more.

One particular question we had was about bias and unfairness in transcription. We hypothesized that the system might work less effectively for doctors with strong accents or speech impediments. Using co-workers with accents, we did an experiment to determine whether this was a serious issue.

Early indications confirm that the transcription may bias in the form of higher error rates for some voices, but that transcript cleanup step with a generalized Large Language Model model not only corrects for the majority of errors but also significantly narrows the variability in error rates across voice samples.

This table compares transcription accuracy metrics before and after applying LLM (Language Model) cleanup. The table includes multiple metrics: Word Error Rate (WER), Match Error Rate (MER), Word Information Lost (WIL), Word Information Preserved (WIP), and Character Error Rate (CER). Lower scores are better for every value other than WIP.

As you can see on the left, Voice 1 (0.16), a speaker with a standard American accent, had a dramatically lower Word Error Rate in the initial transcription than Voice 4 (0.26), a participant who learned English as a Second Language.

Whereas after LLM cleanup, both of them had a Word Error Rate of 0.08. Voice 1 and Voice 4 had nearly identical error rates and information preservation rates after data cleanup. In this case, AI is a technology which reduces rather than exacerbates inequality.

Although our sample size is small, we have consistently seen (in formal and informal tests) that most errors were generally minor and non-consequential. We have highlighted a couple of consequential ones in the diagram, but the vast majority are extremely minor.

We will watch these metrics closely as we scale up the test corpus volume.

Once we are comfortable with the text transcript, we can move on to evaluating the Note Generation aspect.

Automating Textual Quality Assessment

Our challenge in evaluating Note Generation is that the AI is not the only stochastic factor in the system. Our gold standard notes are written (or at least finalized) by humans, and there is a large subjective factor in their decision about how to structure the information. Both the AI and the human reference documentor have quite a bit of flexibility in terms of what order they state facts and even what section of the SOAP note to put facts. This makes it difficult to use simplistic text-similarity metrics like BLEU, and ROUGE which are common in the NLP field for text translation tasks.

A bigger problem with these tools, however, is that they are not designed to describe – in English – what is wrong with the output. They are designed to provide a score, but not a reason for the score. This is a problem because we want to be able to provide feedback to the AI developers about what is wrong with the output.

Instead of using these sorts of text-similarity metrics, what we chose to do is automate – and scale – tests like:

In general, traditional imperative code is not sophisticated enough to do this, so we need to rely on NLP tools, and Large Language Models in particular. This does raise the question of meta-evaluation: how do we trust the tools that evaluate the evaluator. This will be discussed in the section Meta-Validation.

Our approach to scoring is strongly influenced by and partially implemented with the open source project called DeepEval.^[4]

For example, here is a prompt that can be used to verify if a certain fact is grounded in the transcript:

Given the evaluation steps, return a JSON with two keys:
1) a score key ranging from 0 - 10, with 10 being that it follows the
criteria outlined in the steps and 0 being that it does not, and
2) a reason key, a reason for the given score, but DO NOT QUOTE THE
SCORE in your reason.
Please mention specific information from Input and Transcript in your reason, but
be very concise with it!

Evaluation Steps:
1. Extract the key fact from the input.
2. Search the transcript for any mention of the key fact from the input.
3. If the key fact is found anywhere in the transcript, assign a high score.
4. If the key fact is not found in the transcript, assign a low score.

Input:
Neck Pain: Patient cannot swivel neck side to side.
Transcript: hey brandon you know glad to see you in here today… the full
transcript goes here … okay okay any other questions? Not right now. Alright...

Notice how the prompt uses both a score for quantitative and programmatic purposes and a reason for transparency and human review. Our reports make both parts accessible to human reviewers and downstream processes.

In addition to using the score, DeepEval does some wizardry with the log probabilities of the tokens in the output to judge how confident the Evaluator LLM was in its scoring.

The primary tests that we currently do are:

The output is a 2x2 matrix with different Note Generation System versions and configurations as columns and criteria like these as rows. For example, a system might support different SOAP Note styles and we would like to compare the Generation fidelity across all of them.

None of our testing Transcripts or SOAP notes are from real patients. They are either authored by Elation Health or part of the ACI-Bench Dataset which was designed specifically for doing these kinds of evaluations.

Dealing with Ambiguity

There are subtle, usually subjective, cases where humans might disagree with the evaluator.

For example, the transcript might say:

okay so … let's wait for the mri result… and we'll see what the mri says about
what whether or not we can get you like true local injections

The Note Generator might output:

Follow up after MRI results.

The Evaluator might say that this is not explicitly stated:

Hallucination: Data Reviewed: Follow-up appointment after MRI results are available ❌
Reason: The actual output does not mention the follow-up appointment after MRI results are available.
Score: 6.0%

It is largely correct: the intent to follow up after the MRI is implied, not stated explicitly. The Generator and Evaluator are both “trying” to be helpful, but with different “goals”. The Generator “wants”^[5] to find something to add to the Follow-Up section, even if it is implied rather than stated. The Evaluator “wants” to find facts that are not explicitly grounded.

In a case like this, we would adjudicate that the evaluator is working fine, but that the evaluation dataset needs to be tweaked to allow this particular fact to be accepted without criticism. We are developing a sort of “exceptions” mechanism for this.

This is one of the steps in our process which is the least scalable, at the moment. This exception labeling does require human annotation. Perhaps by this time next year we will have trained a third AI to adjudicate these corner cases!

One interesting note on the topic of Follow Ups. One of the few hallucinations that we do see persistently is “Follow-up in four weeks” and we do in fact need to watch for this and ensure it does not erroneously slip into real notes. Sometimes the follow-up is implied, but very occasionally it really is hallucinated.

Meta-Validation

Using Large Language Models to check the output of other Large Language Models does open up a different can of worms, however. If we cannot trust the LLMs underlying the Note Generation tools, what makes us feel that we can trust the tester LLMs? Quis custodiet ipsos custodes? How do we avoid an infinite regress of validators?

What we do to evaluate the evaluators is create negative examples of tests that should fail, due to hallucinations, missed facts and other similar factors. By running tests of what happens if we inject false facts into SOAP notes (or delete facts from Notes), we can evaluate whether the Evaluator would properly pick up on those facts.

We find that Evaluator LLMs are more reliable than Soap Note Generation Systems for the simple reason that they are doing much simpler work with much more focus. They are generally focused on one, or a small number, of facts at a time. This allows us to build confidence in the Evaluators faster than in the Note Generation Systems themselves.