Design spec for filling a TreatmentDetail by voice, settled across three grilling sessions (30 decisions, logged in the spec). Key shape: - two-stage pipeline: OpenRouter whisper-1 -> gemini-3.7-flash - the LLM emits *intents*, never FDI codes or ISO dates; pure Jest-tested backend resolvers own quadrant mapping and Jalali conversion - provider registry keyed by locale so fa can diverge from en/nl - review sheet confirms before anything touches the form - audio and transcripts are never persisted Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
710 lines
35 KiB
Markdown
710 lines
35 KiB
Markdown
# Voice treatment entry
|
||
|
||
**Status:** Draft — not started
|
||
**Area:** Treatment workspace (CLINIC orgs)
|
||
**Created:** 2026-08-20
|
||
|
||
Fill a `TreatmentDetail` — including its lab dispatch — by speaking, instead of by
|
||
tapping through the type dropdown, the FDI chart, the prosthesis wizard and the lab
|
||
picker.
|
||
|
||
---
|
||
|
||
## 1. Goal
|
||
|
||
A clinician on the Treatment tab taps a microphone, describes the treatment for the
|
||
already-selected patient in one utterance, and is shown a **review sheet** of what was
|
||
understood. Fields they tick are applied to the open detail chip. Nothing is written to
|
||
the form without confirmation.
|
||
|
||
### In scope
|
||
|
||
One recording produces **exactly one** `TreatmentDetail`, and may fill every field of it:
|
||
|
||
| Field | Source |
|
||
|---|---|
|
||
| `treatmentType` | catalog code, matched against locale labels |
|
||
| `teeth` | FDI codes, via tooth-intent resolver |
|
||
| `toothSelectionGroups` | connected (bridge) / single spans |
|
||
| `comment` | cleaned dictated notes |
|
||
| lab: `prosthesisTypeCode` per tooth | default type + per-tooth overrides |
|
||
| lab: `destinationOrganizationId` | matched against the clinic's linked labs |
|
||
| lab: `dueDate` | via due-date intent resolver |
|
||
|
||
### Out of scope (v1)
|
||
|
||
- Multiple detail chips from one recording.
|
||
- `attachmentIds` — files cannot be dictated.
|
||
- Editing an existing detail by voice ("no, make that 15"). Confirming a recording always
|
||
creates a **new** detail (see §2).
|
||
- Creating the treatment or selecting the patient by voice. A patient is already
|
||
selected; voice only fills the form.
|
||
- Lab-side (`LAB` org) usage. Clinic only.
|
||
|
||
---
|
||
|
||
## 2. User flow and UI integration
|
||
|
||
### The control: Add detail, split
|
||
|
||
The `Add detail` button gains a second segment holding the mic. The halves read as
|
||
siblings — both end in a new detail — but they are **independent actions**:
|
||
|
||
- **Add half — unchanged.** Same `onAddDetail`, same seeding, same `setEntryStep`. It
|
||
gains a neighbour and nothing else. Its logic is not modified, wrapped or made
|
||
conditional.
|
||
- **Mic half** — starts a recording. Nothing is created until confirm (below).
|
||
|
||
The `Add detail` `<Button>` in `TreatmentDetailsEditor` becomes a segmented control built
|
||
exactly like the **detail chip's trash affordance** in the same file
|
||
(`TreatmentDetailsEditor.tsx`, the chip `<div>` + label `<button>` + remove `<button>`):
|
||
|
||
- an `inline-flex items-stretch overflow-hidden rounded-[var(--radius-md)]` wrapper;
|
||
- two raw `<button>` children divided by `border-s`, each with its own `disabled`,
|
||
`title`, `aria-label` and `focus-visible:ring-inset`;
|
||
- styled `primary` (`bg-primary text-white`) to preserve the button's current look;
|
||
- `w-full sm:w-auto` on the wrapper with the Add half `flex-1`, reproducing today's
|
||
`fullWidth` + `sm:w-auto shrink-0` behaviour.
|
||
|
||
The shared `Button` component is **not** reusable for the halves: it hardcodes
|
||
`rounded-[var(--radius-md)]` on each instance and owns auto-pending state, both of which
|
||
fight a segmented control. This is precisely why the chip pattern uses raw `<button>`s,
|
||
and this control follows it.
|
||
|
||
**Side:** the mic is the second flex child with `border-s` — the **logical end**, exactly
|
||
like the trash. Visually right in `en`/`nl`, visually left in `fa`. No physical
|
||
left/right anywhere, per the repo's RTL rule.
|
||
|
||
```
|
||
en / nl (LTR) fa (RTL)
|
||
[ Add detail │ 🎤 ] [ 🎤 │ افزودن ]
|
||
[ ترمیم 14,15 │ 🗑 ] [ 🗑 │ ترمیم 14,15 ]
|
||
↑ same side as the chip's trash, in both directions
|
||
```
|
||
|
||
### Flow
|
||
|
||
```
|
||
[editable day]
|
||
│
|
||
▼ tap 🎤 ← mic swaps to ■ (red); Add half disabled
|
||
recording ─────────── inline bar below the header row:
|
||
│ ● 0:12 / 2:00 ▁▃▇▅▂▆█▄▁ [لغو]
|
||
│ auto-stops at the 2:00 cap
|
||
▼ tap ■
|
||
processing ────────── ● transcribing… ○ extracting [cancel]
|
||
│
|
||
▼
|
||
review sheet
|
||
☑ Type ترمیم
|
||
☑ Teeth [mini FDI chart] 14 15
|
||
☑ Notes حساسیت به سرما
|
||
☐ Lab لابراتوار سینا ⚠ similar name
|
||
│
|
||
▼ [Apply n fields]
|
||
detail created or filled → normal save flow
|
||
```
|
||
|
||
**No layout shift.** The segmented control never changes size; only the mic's icon and
|
||
colour change. The timer and level meter live in a full-width bar inserted between the
|
||
header row and the chip strip — the header is `sm:justify-between`, so growing the button
|
||
mid-recording would shove the row on every start and every stop.
|
||
|
||
**Duration is capped at 2 minutes** (`maxMs`, configurable, v1 default `120_000`). The
|
||
timer shows `elapsed / 2:00` and the recorder auto-stops at the cap. `maxMs: null` means
|
||
uncapped and remains supported, but is not the v1 default.
|
||
|
||
Two minutes is generous against the longest realistic utterance — a full prosthesis
|
||
dictation with type, several teeth, a bridge, prosthesis type, lab and due date — while
|
||
bounding worst-case spend. It also sits comfortably inside the vendor limits (§4): 2
|
||
minutes of `webm/opus` is well under 1 MB against a 25 MB ceiling, and transcribes in a
|
||
few seconds against a 60s processing timeout.
|
||
|
||
Cost per recording is therefore bounded at **$0.012** of ASR (2 min × $0.006) plus
|
||
~$0.0008 of extraction — about **1.3¢ worst case**, against ~0.3¢ for a typical 20s
|
||
utterance.
|
||
|
||
### What confirm does
|
||
|
||
The detail is created **on confirm**, never on tap. Tapping the mic starts a recording and
|
||
nothing else, so cancelling, a vendor failure, a rate-limit, or navigating away leaves the
|
||
chip strip untouched — there is no orphan state to clean up.
|
||
|
||
Confirm **always appends a new detail**. One unconditional rule, no dependence on
|
||
invisible state: append `newDetail()` seeded with `defaultTreatmentTypeForAppointment`,
|
||
make it active, `setEntryStep('treatment')`, then apply the ticked rows on top — so an
|
||
extracted type overrides the seed, and unticking the type row leaves the seeded default in
|
||
place.
|
||
|
||
This reuses `newDetail()` and the same state transitions, but it is a **separate code
|
||
path**. `onAddDetail` is not called and not changed.
|
||
|
||
`setEntryStep('treatment')` matters: `showChrome` is always on, so the control is visible
|
||
during the **Lab** wizard step too. Confirming there returns to the treatment step.
|
||
|
||
> Accepted consequences:
|
||
> - Tapping Add and then 🎤 leaves behind the blank chip that Add created. It carries the
|
||
> usual trash affordance.
|
||
> - Dictating into an existing detail is not supported in v1 — voice always makes a new
|
||
> one.
|
||
|
||
### Render policy
|
||
|
||
Three different reasons for "no", rendered differently:
|
||
|
||
| Reason | Condition | Render |
|
||
|---|---|---|
|
||
| **Technical** | no voice profile for the locale; no `MediaRecorder` | segment **absent** — the control is byte-for-byte today's plain Add button |
|
||
| **Commercial** | `Plan.features.voiceTreatmentEntry` false | segment **absent** — ⚠ *not enforced in v1*, see §8 |
|
||
| **Contextual** | `!canEdit \|\| disabled` — past day, read-only load, no `TAB_TREATMENT_EDIT` | both segments render and **disable together**, like the chip's trash (`disabled:opacity-40 disabled:cursor-not-allowed`) |
|
||
|
||
"This feature isn't yours" and "not right now" are different statements. Absence avoids a
|
||
permanently dead control; disabling avoids the button resizing as the day strip moves.
|
||
|
||
`TAB_TREATMENT_EDIT` is resolved via `common/membership-permissions.ts`, never by reading
|
||
`membership.permissions` directly.
|
||
|
||
### Component API
|
||
|
||
`TreatmentDetailsEditor` gains exactly **one** optional prop. All behaviour —
|
||
`MediaRecorder`, the API call, error state — lives in `lib/voice/useVoiceCapture.ts`
|
||
(hooks belong in `lib/` per `AGENTS.md`) and is owned by `TreatmentWorkspace`. The editor
|
||
stays presentational and renders both the segment and the recording bar with its own
|
||
classes, keeping the segmented styling beside the chip pattern it mirrors.
|
||
|
||
```ts
|
||
/** Omit when voice is unavailable — the Add button then renders unsplit. */
|
||
voice?: {
|
||
phase: 'idle' | 'recording' | 'processing';
|
||
elapsedMs: number;
|
||
level: number; // 0..1, for the meter
|
||
maxMs: number | null; // v1: 120_000 (2 min). null = uncapped, supported but not default
|
||
onStart: () => void;
|
||
onStop: () => void;
|
||
onCancel: () => void;
|
||
};
|
||
```
|
||
|
||
`voice === undefined` **is** the absent state above — availability is expressed by
|
||
presence rather than a separate flag, so the two cannot disagree.
|
||
|
||
### Accessibility
|
||
|
||
- Mic segment carries both `title` and `aria-label`, like the chip's trash, and its label
|
||
changes with `phase`.
|
||
- Announce **phase transitions** via a `role="status" aria-live="polite"` region — the
|
||
same idiom as the existing autosave status line. Do not put `aria-live` on the ticking
|
||
timer.
|
||
- Focus ring is `focus-visible:ring-inset` tinted primary when idle, red while recording.
|
||
|
||
---
|
||
|
||
## 3. Architecture
|
||
|
||
Two sequential stages, both server-side. The vendor API key never reaches the browser.
|
||
|
||
```
|
||
browser ──audio(base64)──► POST /treatments/voice-extract
|
||
│
|
||
┌─────────┴─────────┐
|
||
│ 1. AsrProvider │ audio + locale hint → transcript
|
||
└─────────┬─────────┘
|
||
┌─────────┴─────────┐
|
||
│ 2. ExtractionProv │ transcript + catalog + ctx → VoiceIntent
|
||
└─────────┬─────────┘
|
||
┌─────────┴─────────┐
|
||
│ 3. resolvers │ intents → FDI codes, ISO date
|
||
│ (pure, tested) │
|
||
└─────────┬─────────┘
|
||
▼
|
||
VoiceExtractionResult (resolved values + the intents
|
||
that produced them, for display)
|
||
```
|
||
|
||
**Why intents and not final values:** the model never emits an FDI code and never does
|
||
calendar arithmetic. It emits what it *heard*; deterministic, unit-tested code decides
|
||
what that means. This is what makes the two highest-consequence mappings — quadrant
|
||
mirroring and Jalali conversion — testable instead of hopeful.
|
||
|
||
### Endpoint
|
||
|
||
`POST /treatments/voice-extract`
|
||
|
||
- Guards: `JwtAuthGuard` + `ClinicOrgGuard`.
|
||
- Service-level check of `TAB_TREATMENT_EDIT`. The plan flag is **not** checked in v1
|
||
(§8).
|
||
- Per-user throttle via `@nestjs/throttler` (present in `package.json`, currently wired
|
||
nowhere in `src/` — this is its first use, so the module must be registered in
|
||
`app.module.ts`). Configurable; v1 default **6 requests / 60s per user**
|
||
(`VOICE_THROTTLE_LIMIT`, `VOICE_THROTTLE_TTL`). A human cannot approach that — a
|
||
recording plus processing takes ten seconds at minimum — so it is purely an abuse and
|
||
runaway-loop guard, which is doing more work than usual given v1 is ungated and
|
||
uncapped (open item 14).
|
||
- Body: base64 audio + declared format + `clientTodayIso` + IANA `timeZone` +
|
||
`treatmentDetailId`/`clientId` for context.
|
||
|
||
`clientTodayIso` and `timeZone` come from the client per the existing house rule
|
||
(`AGENTS.md`): never derive the clinic's civil day from `Date#getDay()` on the UTC
|
||
server. Reuse `common/zoned-civil-time.ts`.
|
||
|
||
---
|
||
|
||
## 4. Provider registry
|
||
|
||
ASR and extraction are **separate swappable roles**. They will not come from the same
|
||
vendor for every locale.
|
||
|
||
```ts
|
||
interface AsrProvider {
|
||
transcribe(audio: AudioInput, localeHint: string): Promise<{ text: string; usage: AsrUsage }>;
|
||
}
|
||
|
||
interface ExtractionProvider {
|
||
extract(transcript: string, catalog: CatalogPrompt, ctx: ExtractionContext): Promise<VoiceIntent>;
|
||
}
|
||
```
|
||
|
||
Resolved through a registry keyed by locale.
|
||
|
||
At launch **all three locales use the same profile** — OpenRouter with
|
||
`openai/whisper-1` for ASR. The per-locale indirection is kept anyway, because the
|
||
locale is the axis along which this is most likely to diverge: Persian ASR is the
|
||
weakest link (§11), and swapping only `fa` to a Persian-specialist vendor must not be a
|
||
code change.
|
||
|
||
```
|
||
VOICE_PROFILE_FA = openrouter:openai/whisper-1 | openrouter:<llm-model>
|
||
VOICE_PROFILE_EN = openrouter:openai/whisper-1 | openrouter:<llm-model>
|
||
VOICE_PROFILE_NL = openrouter:openai/whisper-1 | openrouter:<llm-model>
|
||
```
|
||
|
||
The `language` hint is the profile's locale as ISO-639-1 — `fa`, `en`, `nl` — not a
|
||
constant.
|
||
|
||
Reachability is a property of **where you deploy**, not of the code. The same image
|
||
serves an Iran-hosted instance with a domestic `fa` profile and a Europe-hosted instance
|
||
with an OpenRouter profile; only config differs. A locale with no configured profile has
|
||
no microphone button at all (§2).
|
||
|
||
> The frontend must learn which locales are enabled **from the API**, not from a
|
||
> `NEXT_PUBLIC_*` var — those are baked in at build time, so an env-var approach would
|
||
> make enabling a locale require rebuilding and repushing the frontend image.
|
||
|
||
### v1 provider: OpenRouter
|
||
|
||
Both roles, one API key.
|
||
|
||
**ASR** — `POST https://openrouter.ai/api/v1/audio/transcriptions`
|
||
|
||
```json
|
||
{
|
||
"model": "openai/whisper-1",
|
||
"input_audio": { "data": "<base64>", "format": "webm" },
|
||
"language": "fa"
|
||
}
|
||
```
|
||
|
||
- Accepted formats: WAV, MP3, FLAC, M4A, OGG, **WebM**, **AAC**.
|
||
Chrome/Android `webm/opus` and Safari/iPad `mp4/aac` both go through unmodified —
|
||
**no transcode dependency is required**.
|
||
- Limits: 25 MB; 60s upstream *processing* timeout. The 2-minute recording cap (§2) sits
|
||
comfortably inside both.
|
||
- Response: `{ text, usage: { seconds, total_tokens, cost } }`.
|
||
- Price: `openai/whisper-1` is **$0.006/minute, billed to the nearest second** → $0.002 for
|
||
a typical 20s utterance, **$0.012 at the 2-minute cap**. OpenRouter forwards this model
|
||
directly to OpenAI.
|
||
|
||
**Extraction** — OpenRouter chat completions with a JSON schema constraining the output
|
||
to `VoiceIntent`. Model id is config (`VOICE_LLM_MODEL`).
|
||
|
||
**Chosen: `google/gemini-3.7-flash`.**
|
||
|
||
Cost is deliberately *not* the selection axis. The call is ~1,250 input tokens (system
|
||
prompt + catalog labels + JSON schema + linked labs + transcript) and ~200 output, so the
|
||
whole candidate field spans about one cent per recording:
|
||
|
||
| Model | in / out per M tokens | ≈ per call |
|
||
|---|---|---|
|
||
| `google/gemini-3.7-flash` | $0.375 / $1.875 | $0.0008 |
|
||
| `qwen/qwen3.8-27b` | $0.45 / $3.20 | $0.0012 |
|
||
| `qwen/qwen3.8-max` | $2 / $6 | $0.0037 |
|
||
| `openai/gpt-5.6-terra` | $2 / $12 | $0.0049 |
|
||
| `anthropic/claude-opus-5` | $5 / $25 | $0.0113 |
|
||
|
||
Select for Persian comprehension and reliable constrained JSON instead. Gemini Flash wins
|
||
on the reasoning that this task is **not** reasoning-heavy — read a short sentence, pick
|
||
codes from a supplied closed list, emit small JSON — and it is simultaneously the cheapest
|
||
and lowest-latency candidate, which matters on a pipeline already at 3–10s.
|
||
|
||
**Escalation path if Persian comprehension proves weak** on real transcripts (item 1):
|
||
`qwen/qwen3.8-max` (leads current multilingual rankings, ~4.6× the cost) and then
|
||
`anthropic/claude-opus-5` as the accuracy ceiling. Both are a config change —
|
||
`VOICE_LLM_MODEL` — not a code change. Worth knowing when judging that ranking evidence:
|
||
it is marked provisional and was measured on the previous Qwen generation.
|
||
|
||
---
|
||
|
||
## 5. Extraction contract
|
||
|
||
The model returns intents only. Illustrative shape:
|
||
|
||
```ts
|
||
type VoiceIntent = {
|
||
treatmentType: string | null; // catalog code, from the supplied closed list
|
||
teeth: ToothIntent[];
|
||
connectedSpans: { from: ToothIntent; to: ToothIntent }[];
|
||
comment: string | null;
|
||
prosthesis: {
|
||
defaultType: string | null; // catalog code
|
||
overrides: { tooth: ToothIntent; type: string }[];
|
||
} | null;
|
||
labId: string | null; // must be one of the supplied linked-lab ids
|
||
labMatchExact: boolean;
|
||
due: DueIntent | null;
|
||
};
|
||
|
||
type ToothIntent =
|
||
| { kind: 'explicit'; fdi: string; spoken: string }
|
||
| { kind: 'positional'; arch: 'upper' | 'lower';
|
||
side: 'patient_right' | 'patient_left'; position: number; spoken: string };
|
||
|
||
type DueIntent =
|
||
| { kind: 'weekday'; weekday: string; which: 'this' | 'next' }
|
||
| { kind: 'offset'; unit: 'day' | 'week' | 'month'; amount: number }
|
||
| { kind: 'jalali'; jy: number; jm: number; jd: number }
|
||
| { kind: 'gregorian'; y: number; m: number; d: number };
|
||
```
|
||
|
||
Every code-valued field is constrained to a **closed list supplied in the prompt**:
|
||
|
||
- Treatment types and prosthesis types come from `CatalogLabelService` in the actor's
|
||
locale, so the model sees "پروتز" and "زیرکونیا مونولیتیک" as the spoken forms of
|
||
`prosthesis` and `monolithic_zirconia` rather than being asked to translate. Catalog
|
||
entities store a stable `code` and no label — never hardcode a label.
|
||
- Lab candidates are the clinic's linked labs only (`OrganizationLink`), passed as
|
||
`{ id, name }`. The model may return one of those ids or `null`, nothing else.
|
||
|
||
Every unresolved or rejected item is reported, never silently dropped.
|
||
|
||
---
|
||
|
||
## 6. Resolvers
|
||
|
||
Both live in `backend/src/`, pure and Jest-covered. The frontend has **no test runner**
|
||
(no jest/vitest, zero spec files) — putting them there would forfeit the testability
|
||
that justified this whole design.
|
||
|
||
### `resolveToothIntent()`
|
||
|
||
- Owns the patient-right convention in exactly one place: `upper + patient_right` →
|
||
quadrant 1, `upper + patient_left` → 2, `lower + patient_left` → 3,
|
||
`lower + patient_right` → 4. This is the mirroring bug, and it becomes a unit test.
|
||
- Rejects out-of-range positions rather than clamping. Position 9 is unresolved, never 8.
|
||
- Permanent dentition only — FDI 11–18/21–28/31–38/41–48, matching
|
||
`FDI_UPPER_LEFT_TO_RIGHT` / `FDI_LOWER_LEFT_TO_RIGHT`. Deciduous references
|
||
("دندان شیری") must resolve to **unresolved**, never snap to a permanent tooth.
|
||
- **Locale-neutral by construction.** `ToothIntent` carries `arch`/`side`/`position`, not
|
||
words, so the resolver needs no per-locale branches. The locale-specific part is the
|
||
*prompt*: each enabled locale needs its own spoken tooth vocabulary (`شش بالا راست`,
|
||
`upper right six`, `rechtsboven zes`).
|
||
- **English carries a numbering hazard the other locales do not.** A clinician trained
|
||
under Universal numbering says "tooth number 14" and means a different tooth than FDI
|
||
14. `nl` is safe — the Netherlands uses FDI — but `en` is not. The `en` prompt must
|
||
therefore not accept a bare two-digit number as `explicitFdi` without the speaker
|
||
having made the notation explicit; ambiguous English numerals resolve to
|
||
**unresolved**. See §11.
|
||
|
||
### `resolveDueDate()`
|
||
|
||
- Takes `clientTodayIso` + IANA `timeZone`; reuses `common/zoned-civil-time.ts`.
|
||
- Week starts **Saturday** (Iranian week) — one place, tested.
|
||
- Jalali conversion is arithmetic, not inference: port `jalaliToGregorian` and
|
||
`toLatinDigits` from `frontend/src/lib/i18n/persianCalendar.ts` into
|
||
`backend/src/common/jalali.ts` with a spec. It is dependency-free integer math
|
||
(~60 relevant lines) and the calendar does not change, so the duplication is stable.
|
||
|
||
### Group / prosthesis rules
|
||
|
||
- Connected spans validate through the shipped helpers — `areArchNeighbors`, `sameArch`,
|
||
`teethBetweenInclusive`. **Never a 1-tooth connected group.** Anything invalid degrades
|
||
to singles and is flagged on the review sheet.
|
||
- Prosthesis: expand `defaultType` across all teeth, then apply per-tooth overrides.
|
||
- **All-or-nothing.** `assertCompleteToothProsthesisMap` requires every tooth on a
|
||
`prosthesis` detail to carry a `prosthesisTypeCode` or the send throws
|
||
`TREATMENT_TOOTH_PROSTHESIS_INCOMPLETE`. So if even one tooth ends untyped, the
|
||
prosthesis row is marked incomplete and stays unticked — the unshippable state surfaces
|
||
at review, where it is cheap, not at dispatch minutes later on another screen.
|
||
- **Apply order is teeth → groups → prosthesis**, so `pruneToothProsthesisForGroups`
|
||
behaves.
|
||
|
||
---
|
||
|
||
## 7. Review sheet
|
||
|
||
**Presentation:** a **modal** on desktop; a **full-screen overlay** on mobile.
|
||
|
||
> ⚠ On mobile this must be an overlay rendered in place — **not** a Next.js route. A real
|
||
> navigation unmounts `TreatmentWorkspace` and destroys the in-progress draft. This is the
|
||
> same constraint the realtime soft-refresh already lives under: never remount the
|
||
> treatment form, never clear a draft.
|
||
|
||
- Renders the transcript, then one row per extracted field in the app's own vocabulary:
|
||
translated catalog labels, and a mini FDI chart for the teeth rather than a list of
|
||
numbers.
|
||
- Each row has a checkbox. Ticked rows apply; nothing else is touched. Confirm is also
|
||
what creates the new detail — see §2.
|
||
- Rows default to ticked **except**:
|
||
- the lab row when `labMatchExact` is false — shipping to a lab always requires a
|
||
deliberate tick;
|
||
- any row carrying an unresolved item or an incomplete prosthesis map.
|
||
- Unresolved items are shown with what was heard ("دندان شیری — بازشناسی نشد"), so the
|
||
clinician can see what the system did not understand.
|
||
- RTL-safe: logical `text-start` / `text-end` only, never `text-left`/`text-right`.
|
||
Dates via `lib/i18n/format.ts`.
|
||
|
||
---
|
||
|
||
## 8. Gating and configuration
|
||
|
||
Gate = `TAB_TREATMENT_EDIT` ∧ `canEditTreatmentForDay` ∧ configured locale profile ∧
|
||
`Plan.features.voiceTreatmentEntry`. How each failing condition *renders* is in §2.
|
||
|
||
`Plan.features` is a `Json` column that already exists on the `Plan` model and is seeded
|
||
as `{}` for all five plans — and is **read nowhere in `backend/src/`**. Voice is its first
|
||
consumer, so:
|
||
|
||
- No migration is needed.
|
||
- The read should go through a small generic helper, since this establishes the pattern
|
||
for every future flag.
|
||
- Voice is metered vendor spend, which is why the gate is commercial (plan) rather than a
|
||
new `TAB_*` permission — a clinician who can already edit the form gains no capability
|
||
from voice, only speed.
|
||
|
||
The per-user throttle is a separate, non-commercial abuse control.
|
||
|
||
### v1 ships ungated
|
||
|
||
**The plan flag is designed but not enforced in v1** — voice is open to every clinic user
|
||
who can edit treatments, in every configured locale. `Plan.features.voiceTreatmentEntry`
|
||
and the availability API stay documented here as the intended gate, deferred rather than
|
||
dropped, so turning them on later is additive.
|
||
|
||
Consequence to accept deliberately: with no plan gate and no duration cap (§2), the
|
||
per-user throttle is the **only** control on metered vendor spend. See open item 14.
|
||
|
||
---
|
||
|
||
## 9. Errors
|
||
|
||
Per the three-layer contract: a code in `common/errors/error-codes.ts`, the throw site,
|
||
and an `errors.X` key in **all three** of `frontend/messages/{en,fa,nl}.json`. Never a raw
|
||
English Nest exception for a user-facing failure.
|
||
|
||
| Code | When |
|
||
|---|---|
|
||
| `VOICE_MIC_DENIED` | browser permission refused — **client-side only**: needs the `errors.X` key in all three message files, but no `ErrorCode` entry and no throw site |
|
||
| `VOICE_CLIP_TOO_LONG` | over `maxMs` (server-side re-check), or over vendor limits |
|
||
| `VOICE_UNSUPPORTED_FORMAT` | recorder produced a container the profile rejects |
|
||
| `VOICE_ASR_FAILED` | transcription stage failed |
|
||
| `VOICE_EXTRACT_FAILED` | transcript obtained, structuring failed |
|
||
| `VOICE_NOTHING_RECOGNIZED` | empty or unusable transcript |
|
||
| `VOICE_NOT_AVAILABLE` | no profile for locale (v1); plan flag off, once enforced |
|
||
| `VOICE_RATE_LIMITED` | throttle |
|
||
|
||
**Transcript salvage:** when ASR succeeded and only extraction failed, the response still
|
||
carries the transcript and the failure dialog offers *"افزودن به یادداشت"*. That action
|
||
**creates a new detail with only `comment` set to the transcript** — everything else left
|
||
at `newDetail()` defaults. The words were captured and paid for; only the structure was
|
||
lost.
|
||
|
||
This keeps the feature's one invariant intact: **voice never writes into an existing
|
||
detail.** Dictating into an already-filled detail is a separate, later feature with its
|
||
own voice-to-text control scoped to that field (§1, out of scope).
|
||
|
||
It also does not bypass the confirmation rule — the dialog shows the transcript, and the
|
||
dentist taps to accept it. That review matters, because a raw transcript carries ASR
|
||
errors and may contain the patient's spoken name, and `comment` is persisted (§10).
|
||
|
||
---
|
||
|
||
## 10. Data handling
|
||
|
||
- Audio is held **in memory for the request only**. Never written to disk, never a Prisma
|
||
row. Note this is deliberately unlike treatment attachments, which do persist to
|
||
`backend/uploads/treatments`.
|
||
- The transcript goes to the browser for the review sheet and dies with it.
|
||
- The `comment` field persists a cleaned version of what was said — that is legitimate
|
||
clinical record-keeping and is the only durable trace.
|
||
- Telemetry is **structured and patient-free**: clip duration, which fields resolved,
|
||
unresolved count, vendor latency, `usage.cost`, outcome (applied / discarded / failed),
|
||
locale. Never the transcript, never audio, never a patient identifier.
|
||
The **destination is an open question** — this repo has no metrics infrastructure yet
|
||
(open item 9). Emit it as structured log lines in the interim so the fields exist and
|
||
can be routed later without changing call sites.
|
||
|
||
**Cancelling aborts the in-flight vendor call** via `AbortController`, rather than letting
|
||
it settle and discarding the result. Note this reduces spend but does not eliminate it:
|
||
work already performed upstream may still be billed.
|
||
|
||
Recordings are clinical descriptions of identifiable patients leaving the server for a
|
||
third party. Confirm the provider's retention and training policy in writing before
|
||
enabling this for real clinics.
|
||
|
||
---
|
||
|
||
## 11. Open items
|
||
|
||
1. **Persian ASR accuracy on tooth numbers is unmeasured, and it decides the feature.**
|
||
Everything downstream assumes a usable transcript; no design choice above compensates
|
||
for a bad one. Run this before writing feature code.
|
||
|
||
*Why tooth numbers specifically, not general accuracy:* a transcript can score well on
|
||
WER and still be useless here, because the errors land on the digits.
|
||
`چهار` (4) / `چهارده` (14) / `چهل` (40) differ by one syllable. FDI spoken as
|
||
`یک چهار` may return as `۱۴`, `14`, or `یک چهار`. Persian and Latin digit scripts mix
|
||
within one transcript. Jargon is loanwords (`زیرکونیا`, `پرسلن فیوزد تو متال`,
|
||
`اینله`/`آنله`) and clinicians code-switch into English mid-sentence. Suction and
|
||
handpiece run in the background.
|
||
|
||
*Protocol:* ~25 utterances from a dentist reading a script covering explicit FDI,
|
||
quadrant-relative phrasing, bridges, prosthesis types, due dates and notes — recorded
|
||
on the real device, ideally once quiet and once with the operatory running. POST each
|
||
to `/api/v1/audio/transcriptions` with `language: "fa"`. Score **per-tooth-reference
|
||
accuracy** (of every tooth spoken, how many survive recoverably?) and jargon
|
||
recognition separately. Notes accuracy barely matters. ~6 minutes of audio ≈ **$0.04**.
|
||
|
||
*Decision it drives:* if tooth accuracy holds, build. If not, the fix is not prompt
|
||
tuning — it is pointing the `fa` ASR slot at a Persian-specialist vendor while `en`/`nl`
|
||
keep `whisper-1`. That is a config change precisely because of decision 23.
|
||
|
||
*Unverified lead for that fallback:* recent Persian-ASR benchmark work reports
|
||
Qwen3-Omni as the strongest open Persian ASR. Not confirmed as available on
|
||
OpenRouter's transcription endpoint — check before relying on it.
|
||
|
||
*Byproduct:* the recordings become the fixture corpus for item 2.
|
||
2. ~~**`VOICE_LLM_MODEL`**~~ — **resolved:** `google/gemini-3.7-flash` (§4). Still worth
|
||
validating on real whisper output during item 1, since clean transcripts flatter every
|
||
model; the escalation path if Persian comprehension disappoints is documented in §4 and
|
||
is a config change.
|
||
3. ~~**Production reachability**~~ — **resolved:** confirmed reachable from the Iranian
|
||
production host. The per-locale registry is retained regardless (decision 23), so the
|
||
`fa` profile can still be repointed at a domestic vendor if item 1 goes badly.
|
||
4. ~~**Provider retention/training policy**~~ — **resolved:** restricted via OpenRouter's
|
||
account-level privacy/data-policy settings. Note this is an *account* setting, not a
|
||
per-request one: re-verify it if the API key or the OpenRouter account changes, and
|
||
remember `whisper-1` is forwarded to OpenAI, so the effective policy is OpenRouter's
|
||
plus that provider's.
|
||
5. **English tooth numbering is unresolved as a product question.** Enabling `en` means
|
||
deciding what "tooth number 14" means when the speaker's notation is unknown —
|
||
Universal or FDI. The spec's current answer is to refuse ambiguous bare numerals in
|
||
`en`, which is safe but will feel broken to a US-trained clinician. Options are: refuse
|
||
(current), an org-level notation preference, or restricting `en` to quadrant-relative
|
||
phrasing. Decide before `en` ships to a real clinic; `fa` and `nl` are unaffected.
|
||
6. **`nl` and `en` have no spike data.** The Persian spike (item 1) should be repeated per
|
||
locale before that locale's mic is enabled for real users — same protocol, same
|
||
scoring, different speaker.
|
||
|
||
7. ~~**Transcript salvage has no target**~~ — **resolved:** salvage creates a **new
|
||
detail** with only `comment` set to the transcript (§9). Preserves the invariant that
|
||
voice never writes into an existing detail; dictating into a filled detail becomes a
|
||
separate later feature with its own field-scoped control.
|
||
8. ~~**Throttle limits**~~ — **resolved:** configurable, v1 default **6 requests / 60s per
|
||
user** (§3). Unreachable by a human; a pure abuse guard.
|
||
9. **Telemetry has no sink.** §10 defines exactly what to record but not where it goes —
|
||
this repo has no metrics or analytics infrastructure yet. Deliberately deferred until
|
||
it does. Interim: structured log lines, so the fields exist and can be routed later
|
||
without touching call sites.
|
||
10. ~~**Review sheet modality**~~ — **resolved:** modal on desktop, full-screen **overlay**
|
||
on mobile (§7). Not a route — navigating would unmount `TreatmentWorkspace` and destroy
|
||
the draft.
|
||
11. ~~**Cancel and in-flight requests**~~ — **resolved:** abort via `AbortController` to
|
||
limit spend (§10). Upstream work already performed may still be billed.
|
||
12. ~~**Recording duration cap**~~ — **resolved:** **2 minutes**, configurable via `maxMs`
|
||
(§2). Still worth timing a realistic worst-case prosthesis dictation during item 1 to
|
||
confirm 2 minutes is comfortable rather than tight.
|
||
13. ~~**Availability API**~~ — **resolved for v1:** voice ships **open to everyone** with a
|
||
configured locale profile. No plan check, no availability endpoint. The `Plan.features`
|
||
design in §8 is deferred, not dropped.
|
||
|
||
### New
|
||
|
||
14. **Cost exposure is bounded but ungated in v1.** With the 2-minute cap (12) and the
|
||
6/60s throttle (8), worst case is 12 audio-minutes per user-minute ≈ **$0.072/min**, or
|
||
~$4.30 per hour of sustained abuse by one user — bounded, not free. Normal use is far
|
||
below this: a clinician doing 60 recordings a day at ~20s each costs about **$0.18/day**.
|
||
|
||
What remains open is that there is **no per-organization limit at all**, because v1
|
||
ships ungated (13). Worth deciding what to watch and at what number to react.
|
||
Mitigations, already designed and each a config or flag change: enforce the
|
||
`Plan.features` gate, lower `maxMs` or the throttle, or add an org-level monthly minute
|
||
budget.
|
||
|
||
---
|
||
|
||
## 12. Verification
|
||
|
||
- `cd backend && npm test` — new suites for `resolveToothIntent` (quadrant mapping in all
|
||
four quadrants, out-of-range rejection, deciduous → unresolved), `resolveDueDate`
|
||
(Saturday week start, "this" vs "next" weekday, Jalali leap year, month-end), the
|
||
Jalali port, prosthesis expansion + completeness, and connected-span validation.
|
||
- `cd backend && npm run build` — cross-cutting backend gate.
|
||
- `cd frontend && npx tsc --noEmit` — frontend gate.
|
||
- Manual: fa locale, editable day, prosthesis detail with a bridge, dispatch to a linked
|
||
lab. Then specifically:
|
||
- **past day** → both segments disabled, control still split (not absent);
|
||
- **locale with no profile** → control renders unsplit, identical to today;
|
||
- **fa vs en** → mic sits at the logical end in both, on the same side as the chip's
|
||
trash;
|
||
- **cancel mid-recording** → chip strip unchanged, no orphan detail;
|
||
- **confirm** → always appends a new chip, whatever the active detail contains;
|
||
- **Add half** → behaves exactly as it did before this change;
|
||
- **tap 🎤 during the Lab wizard step** → confirm returns to the treatment step;
|
||
- no layout shift in the header row on record start, stop, or the 2:00 auto-stop;
|
||
- **hold past 2:00** → auto-stops and proceeds to processing, not an error;
|
||
- **cancel during processing** → the vendor request is actually aborted;
|
||
- **review sheet on mobile** → full-screen overlay; closing it leaves the draft intact.
|
||
|
||
---
|
||
|
||
## 13. Decision log
|
||
|
||
Settled in a grilling session on 2026-08-20.
|
||
|
||
| # | Question | Decision |
|
||
|---|---|---|
|
||
| 1 | Scope | Everything including lab dispatch |
|
||
| 2 | AI supply chain | Domestic provider originally; OpenRouter for v1, registry keeps both open |
|
||
| 3 | Apply model | Review sheet, then apply |
|
||
| 4 | Speech → FDI | LLM emits intent, code resolves |
|
||
| 5 | Cardinality | One detail per recording |
|
||
| 6 | Lab destination | Closed list of linked labs, explicit confirm, unticked when inexact |
|
||
| 7 | Due date | Intent + deterministic resolver |
|
||
| 8 | Resolver location | Backend, Jalali math ported |
|
||
| 9 | Prosthesis | Default type + overrides, all-or-nothing |
|
||
| 10 | Retention | Discard audio and transcript, non-PHI telemetry only |
|
||
| 11 | Capture | Tap to start/stop, hard cap (see 27) |
|
||
| 12 | Failure UX | Stage-aware codes, transcript salvage |
|
||
| 13 | Locales | Provider registry per locale; all three locales enabled |
|
||
| 14 | Reachability | Registry now, slots filled per deployment |
|
||
| 23 | ASR model | `openai/whisper-1` for **every** locale; registry kept so `fa` can diverge |
|
||
| 24 | Extraction model | **`google/gemini-3.7-flash`**; escalation path documented in §4 |
|
||
| 25 | Salvage target | Creates a new detail with only `comment` set — voice never writes into an existing detail |
|
||
| 26 | Throttle | Configurable; v1 default 6 requests / 60s per user |
|
||
| 27 | Duration cap | **2 minutes**, configurable via `maxMs` |
|
||
| 28 | Review sheet | Modal on desktop, full-screen overlay (not a route) on mobile |
|
||
| 29 | Cancel | Aborts the in-flight vendor call |
|
||
| 30 | v1 gating | Open to everyone; `Plan.features` gate deferred, not dropped |
|
||
| 15 | Gating | `Plan.features` flag — its first consumer |
|
||
|
||
UI placement settled in a second grilling session on 2026-08-20.
|
||
|
||
| # | Question | Decision |
|
||
|---|---|---|
|
||
| 16 | Mic action | Independent record action; the Add half's logic is untouched |
|
||
| 17 | Record UI | Mic segment toggles ▶/■; inline bar below the header row |
|
||
| 18 | Unavailable | Absent for technical + commercial; disabled for contextual |
|
||
| 19 | Component API | One optional `voice` object prop; `undefined` means absent |
|
||
| 20 | Side | Logical end (`border-s`), exactly like the chip's trash |
|
||
| 21 | Creation | On confirm, never on tap |
|
||
| 22 | Creation rule | Confirm **always** appends a new detail — no blank-reuse guard |
|