Fifty dictations, two scribes: eight invented drug names and six doses nobody said

We make Perch, a dictation tool for veterinary clinics. On October 20 we ran it against Lupa Notes on 50 blind dictations with 90 drug mentions. Lupa took no part. It is a vendor test, and the clips, the scorer and every note are public.

On those 50 clips Perch charted 97.8% of the dictated drug names correctly, invented none, and changed no dose. Lupa Notes charted 85.6%, charted eight drug names that do not exist, and wrote six doses nobody said. The scorer, its full output and every note are committed under `scribe/bench/lupa/` in the Perch repository and published in the trap set.

Two clips

One dictation read: "Librela monthly, Adequan 4.4 mg/kg IM twice weekly for four doses." Lupa's plan charted "Librela 44 mg/kg intramuscularly twice weekly." Wrong drug, ten times the dose, and Adequan is gone from the record.

A technician's treatment sheet read: "Given at 10:00, ondansetron 0.5 mg/kg slow IV." Lupa charted "Ondancetrin 5mg/kg slow IV." There is no such drug, and the dose is ten times what was given.

Perch charted Librela and Adequan at 4.4 mg/kg, and ondansetron at 0.5 mg/kg. On that second clip Perch heard famotidine as "Formotidine" and left the word for the vet rather than guess.

What the eight invented names have in common

Every one of Lupa's eight false offers is a misspelled brand charted as the medication: Rimadil, Cymbadol, Zycordal, Veticulin, Ondancetrin, Pannaker, Serenia, Pyrintel. Each one replaces the real name, so the note carries a drug that does not exist in place of one that does.

Speech recognition mangles veterinary brand names, and every scribe gets a garbled candidate. Perch's own layer checks each candidate against the clinic's term list and against the audio itself, and charts the name only when both agree. When they do not, Perch leaves the word for the vet. On the same clips Perch charted the real name in seven of those eight cases and refused the eighth.

The doses

All six of Lupa's dose errors are ten times the dictated dose: 0.02 mg/kg charted as 0.2 three times, 0.05 charted as 0.5, 4.4 charted as 44, 0.5 charted as 5. Perch keeps the dictated digits; it charted no dose the clinician did not say.

Method

• Blind corpus 7: 70 clips (45 dictations, 25 traps) written on October 20 by an author who never saw Perch's matcher, in registers no earlier set used: orthopedics and rehab, neurology, urinary and reproductive, controlled substance logs, locum handover, technician treatment sheets, pre surgical and recovery notes, transfer records read back, client text messages. One Kokoro voice and two Microsoft Edge neural voices, none used in Perch tuning, one voice per clip.
• Lupa Notes: production web app on a self serve account under our real name, speciality Consulting, kilograms and Fahrenheit, no abbreviations, high detail. Each clip uploaded through their file upload into its own consultation, stock Consult (SOAP) template, one generation per consult, no edits, note text captured verbatim. Lupa shows no transcript for an uploaded file, so only the note is scored. Measured once; their product may have changed since.
• Why 50: Lupa's free account accepted 50 consults and refused the 51st with an expiry error. The 20 clips it never saw are all traps, so the comparison covers all 45 dictations and 5 traps. Perch is scored on exactly the same 50.
• Perch: the production pipeline as deployed on October 20, default 66 term list, no clinic formulary loaded.
• Scoring, identical for both: a dictated drug counts if it appears correctly spelled in the output or is charted. A false offer is a drug like name charted in the plan that the clinician never said. Dose drift is a dose number with a unit in the note that appears nowhere in the dictation, reported separately.

Every miss

Lupa Notes, 13 misses: s001 Rimadyl, s003 Adequan, s012 Baytril, s018 Simbadol, s023 Zycortal, s024 Vetsulin, s027 ondansetron, s030 Panacur, s032 Cerenia, s036 oclacitinib, s037 Vetmedin, s039 pyrantel, s045 Heartgard. False offers: s001 Rimadil, s018 Cymbadol, s023 Zycordal, s024 Veticulin, s027 Ondancetrin, s030 Pannaker, s032 Serenia, s039 Pyrintel.

Perch, 2 misses: s027 famotidine, s030 Panacur, both left for the vet. False offers: none. Dose drift: none.

On the 20 trap clips Lupa did not reach, Perch has one false offer worth owning: a transfer note from a clinic the author named "Prohart Road Animal Clinic" was charted as ProHeart. It is the first false offer Perch's production path has produced across our seven blind sets (the earlier six are tallied in `scribe/bench/results/PROMPT12-2026-09-14.md` on the accuracy branch, this one in `scribe/bench/lupa/RESULTS.md`), and it is on our list.

Caveats

Synthetic voices, and real clinic speech measures roughly twice as hard for every product. Both ran out of the box with no clinic formulary loaded, which is the feature Perch is built around. Fifty clips is a small set; one clip moves a score by about a point. Zero invented names and zero dose drift are properties of this set, not a guarantee.

The clips, the corpus, Lupa's notes verbatim and the ledger are in the Perch trap set, version 1.1.0, on GitHub at github.com/perchvet/perch-trapset under the `corpus7` folder. Lupa Notes is a product of Lupa Pets, which did not take part in, sponsor, or endorse this test and is not affiliated with Talcoe or Perch.