Compare commits

...

17 Commits

Author SHA1 Message Date
17d3c5ca25 fix(backend): read a tooth code whatever script its digits are in
The extraction model transcribes Persian speech, so it can hand back "۲۶"
in Persian digits or "2 6" from a digit-by-digit dictation. Both were
compared literally against /^[1-8][1-8]$/, missed, and fell through to the
positional branch with no quadrant — where the tooth was reported as "not
understood". The clinician loses a tooth and is told the words were the
problem.

normalizeFdiCode() now runs at both the branch choice and the final
validation, so the two cannot disagree. toLatinDigits moves out of
jalali.ts into common/digits.ts: it was exported but unused in production,
and a tooth module reaching into the calendar module would read as an
accident.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-21 04:42:09 +08:00
647ff65b00 fix(backend): apply the large-body limit to every spelling Express routes
req.path was compared to the canonical '/api/voice/extract' only, but
Express routes case-insensitively and ignores a trailing slash by default.
'/api/voice/extract/' therefore reached the controller with the 100 kb
parser, and 413'd every recording past ~20 seconds — a failure that reads
as a broken microphone rather than a routing detail.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-21 04:02:49 +08:00
6ba1fad56c fix(voice): say the quadrant is missing instead of "could not be read"
"ترمیم برای دندون دو" set the treatment type but reported the tooth as
unreadable. Nothing was misheard: position 2 arrived intact, with no
quadrant, because none was spoken — four teeth carry position 2 and the
resolver correctly refused to pick one. Only the label was wrong, and it
sent the clinician looking for a transcription fault.

Adds a tooth_missing_quadrant reason that names what is missing and shows
how to say it ("دو بالا راست"), and tells the model explicitly to report a
quadrant-less number with arch and side null rather than guessing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-21 04:02:39 +08:00
2a42f20bff fix(backend): restore the large-body limit on the voice route
POST /voice/extract returned 500 for any real recording. The threshold was
exactly 100 kb — Express's body-parser default — which is about 20 seconds of
audio, so the endpoint was unusable at its own 2-minute cap.

The scoped parser was registered as a path-mounted json() stacked in front of a
default one, which relied on two implicit behaviours: Express stripping the
mount path, and body-parser skipping a request another parser had already
handled. That coupling broke when the surrounding middleware order shifted, and
it broke silently — the parser was still registered, just no longer the one that
ran. Bisected by dumping the Express layer stack and confirming the raw error was
`entity.too.large` with `limit: 102400`.

Replaced with a single middleware that picks a parser by path. No mount-path
stripping, no dependence on parser ordering. Extracted to common/body-parsers.ts
so it is covered by a unit test rather than only reachable through main.ts, which
createTestingModule never executes.

The test is mutation-checked: forcing the default parser fails 2 of its 5 cases.
It also pins that the larger limit does not leak app-wide, and that a merely
similar path (/api/voice/extract/extra) does not get it.

Verified against the compiled server: 300 kb now reaches /api/voice/extract,
/api/auth/login still rejects it, and ordinary requests are unaffected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 22:12:16 +03:30
7b50134d03 docs: mark the voice spec implemented
Implemented across 12 commits on feat/voice-treatment-entry. Still blocked on
the Persian ASR spike before it is trustworthy in front of patients: nothing in
the implementation compensates for a bad transcript.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 20:24:08 +03:30
56d413944a feat: wire voice entry into the treatment workspace
Makes the feature reachable end to end: availability is fetched alongside the
catalogs, the capture hook drives the segmented control, and confirming the
review sheet appends a new detail.

Confirm always appends — it never edits an existing detail and never calls
onAddDetail. Ticked rows land on top of the seeded defaults, so unticking the
type row leaves the appointment-purpose default rather than a blank. Lab-side
rows ride on a lab case draft keyed by the detail's *client* id, so a brand-new
unsaved detail can carry a lab, due date and per-tooth prosthesis map.

Availability comes from the API rather than a NEXT_PUBLIC_* var, since those are
baked in at build time; a failure fetching it degrades to no microphone rather
than taking the treatment tab down.

From review of this commit:

- Unticking "teeth" while leaving "prosthesis" ticked attached prosthesis rows
  for teeth the detail does not contain. Nothing downstream filters them —
  assertCompleteToothProsthesisMap only checks detail-teeth ⊆ map, never the
  reverse — so they would have reached task generation as lab work for teeth
  nobody is treating. The map is now filtered to the detail's own teeth.
- The microphone was gated on the URL locale while the server resolved
  everything from req.user.language. Those diverge (a bookmarked /fa/ URL, a
  language toggle whose save failed), which would transcribe Persian with an
  English hint and anchor "next Thursday" to a Monday week instead of a Saturday
  one — or 403 from a visibly-enabled button. The client now sends the locale the
  microphone was offered in, so the gate and the request agree by construction.

Also fixed from the previous review: a civil YYYY-MM-DD date rendered a day
early west of Greenwich (parsed as UTC midnight); the missing-teeth list
hardcoded the Arabic comma for all locales; and voiceApply had no ICU plural, so
the common single-field case read "Apply 1 fields".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 20:22:42 +03:30
8757a8952c feat(frontend): split Add detail into a segmented control with voice
The microphone becomes the second segment of the Add detail button, built like
the detail chip's trash affordance in the same file — an overflow-hidden rounded
wrapper holding two raw <button>s divided by border-s — rather than two shared
Buttons, which each hardcode their own rounding and would fight a segmented
control. border-s puts the mic at the logical end: visually right in en/nl,
visually left in fa, on the same side as the chip's trash in both directions.

The two halves share a wrapper and nothing else. Add keeps its exact behaviour.

The control never changes size while recording; the timer and level meter live
in a bar between the header row and the chip strip, because the header is
sm:justify-between and growing the button would shove the row on every start and
stop. The meter exists to prove the microphone is actually hearing something —
silence and a dead mic look identical otherwise.

Voice reaches the editor as one optional `voice` prop, so its absence *is* the
unavailable state and the two cannot disagree.

Fixes from review of this commit:

- mountedRef was set false on unmount and never re-armed, so under StrictMode
  the hook was permanently "unmounted" in dev and recording silently never
  started.
- onStart guarded only on `phase`, which does not change until getUserMedia
  resolves; a second click during the permission prompt orphaned the first
  MediaStream, leaving the mic indicator lit.
- Week start is now per locale. "Next Thursday" is week-relative, and hardcoding
  Saturday put an en/nl clinician's deadline a week out.
- A missing `which` on a weekday intent is read as "this" rather than failing —
  a bare weekday carries no qualifier, and rejecting it discarded a real
  deadline.
- durationMs is client-reported and so is a claim, not enforcement; the cap is
  now also checked against the vendor's own usage.seconds.
- Blob type falls back to the recorder's actual mimeType before webm, so old
  Safari's mp4/aac clips are not mislabelled.

Two review findings were rejected as incorrect, both re-verified against live
sources: google/gemini-3.7-flash does exist on OpenRouter (1M context,
$0.375/$1.875 per M), and base64 JSON input_audio is the documented primary
path for /audio/transcriptions, with multipart as the OpenAI-compatible
alternative. The spec's stale "unverified" note is corrected, and the provider
now has unit tests covering the request shape, usage parsing, and that a vendor
error body never reaches the thrown message.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 19:47:10 +03:30
ff2bd09669 feat(frontend): voice capture hook, API client and types
MediaRecorder handling and the API call live in lib/, not in ui/, so
TreatmentDetailsEditor can stay presentational and take only a `voice` prop.

Container choice is made at record time and needs no transcode: Chrome and
Android give webm/opus, Safari and iPad give mp4/aac, and the transcription
endpoint accepts both. Safari's `audio/mp4` is sent as `m4a`, the name the
vendor's container list actually uses, so iPad recordings do not fail while
Chrome works. Older Safari shipped MediaRecorder without isTypeSupported, so
that path lets the browser choose rather than refusing outright.

From review of this commit:

- The auto-stop at maxMs guaranteed a 413. The client measures the final length
  after the recorder has stopped, so a recording that runs to the cap always
  reports slightly over it, and the server rejected exactly the recording the
  auto-stop existed to save. The server now allows a documented 2s tolerance and
  the client keeps reporting the true length, so telemetry stays honest.
- getUserMedia is async, so a permission granted after unmount installed a live
  stream the cleanup effect had already run past — leaving the browser's
  recording indicator lit with nothing listening. Guarded with a mounted ref.
- Client-side failures are now ApiError-shaped ({code, statusCode}) rather than
  bare Errors, because getUserFacingError only resolves that shape; without it
  errors.VOICE_MIC_DENIED was dead in all three locales.

Cancelling aborts the request, which closes the connection and aborts the
metered vendor call server-side rather than letting it settle unseen. The level
meter is best-effort: a blocked AudioContext costs the meter, not the recording.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 18:43:04 +03:30
01b4ed7633 feat(backend): voice extraction endpoint
POST /voice/extract behind JwtAuthGuard + ClinicOrgGuard, plus
GET /voice/availability so the frontend can decide whether to render the
microphone — it cannot learn that from NEXT_PUBLIC_*, which are baked in at
build time.

Audio is held in memory for the request only: never written to disk, never a
Prisma row. The transcript goes back to the client and is not persisted. What
is logged is structured and patient-free — clip length, which fields resolved,
unresolved count, vendor cost, outcome — with log lines as the interim sink
until this repo has metrics infrastructure.

On extraction failure the transcript still travels back in the error details,
so the words the clinician already paid for can be salvaged into a note.

v1 ships ungated beyond a configured locale profile; the Plan.features design
is deferred, not dropped.

From review of this commit, four of which were load-bearing:

- Express's 100 kb default body limit rejected any recording past ~20 seconds,
  making the endpoint unusable at its own 2-minute cap. Body parsers are now
  registered explicitly with a 10 MB limit scoped to the voice route only.
  Verified empirically: 600 KB reaches /api/voice/extract, while /api/auth/login
  still 413s.
- ThrottlerGuard keys on req.ip, so behind nginx the whole deployment would
  share one bucket and an abuser rotating IPs would bypass it. VoiceThrottlerGuard
  keys on the user id instead — with no plan gate, this is the only control on
  metered vendor spend.
- ThrottlerException had no 429 fallback and surfaced as INTERNAL_ERROR; the
  guard now throws VOICE_RATE_LIMITED directly.
- durationMs was optional, so omitting it bypassed VOICE_MAX_RECORDING_MS
  entirely. It is required.
- VOICE_UNSUPPORTED_FORMAT was dead code — the DTO's @IsIn already rejects
  unknown containers — so it is gone rather than left unreachable.

ThrottlerModule is deliberately not bound as a global APP_GUARD: a global
ThrottlerGuard rate-limits every route against every named throttler, which
would have capped the whole API at the voice limit.

All seven remaining VOICE_* codes have errors.* keys in en, fa and nl.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 18:27:14 +03:30
de6e259932 feat(backend): OpenRouter voice providers and per-locale registry
ASR and extraction are separate, independently swappable roles resolved per
locale from config. All three locales point at the same OpenRouter models today
(whisper-1, gemini-3.7-flash); the indirection stays because Persian ASR is the
weakest link and repointing only `fa` must not be a code change.

The model emits a deliberately flat wire shape rather than the internal
discriminated unions — strict json_schema mode has poor union support — and
toVoiceIntent narrows it. That normalizer is total: a missing or malformed
payload yields a shape the resolvers report as unresolved rather than one that
throws.

The prompt supplies catalog codes with labels in the actor's locale, so the
model matches spoken words rather than translating, and carries per-locale
tooth vocabulary. English gets an explicit warning that a bare two-digit number
is ambiguous under Universal numbering, and must not be treated as FDI unless
the speaker said so.

From review of this commit:
- only an actually FDI-shaped code takes the explicit branch; fdi:"6" alongside
  valid arch/side/position used to lose the tooth entirely
- an unrecognised due kind passes through to be flagged, instead of collapsing
  to null and looking like no deadline was ever spoken
- vendor error bodies stay out of the thrown message and the default log level;
  a 4xx can echo the request back, transcript included
- the chat call sets provider.require_parameters so OpenRouter only routes to
  endpoints that honour the JSON schema, rather than ones treating it as a hint
- an unknown locale in VOICE_ENABLED_LOCALES now fails at boot like an unknown
  provider id, instead of silently disabling the microphone everywhere

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 18:00:53 +03:30
b4aff39797 feat(backend): assemble resolved extraction from voice intents
Composes the tooth, span, prosthesis, catalog and date resolvers into the
payload the review sheet renders.

Connected spans expand: "a bridge from 14 to 16" selects 15, which was never
spoken. Overlapping spans merge into one bridge, group teeth sort along the
arch (16-15-14, and 11 beside 21 across the midline), and a span collapsing to
a single tooth degrades to a single group without losing that tooth — there is
no such thing as a one-tooth bridge. A cross-arch span is impossible and is
reported rather than guessed at.

Prosthesis expands a default across the selection then applies per-tooth
overrides, because "همه زیرکونیا، ۲۶ پی‌اف‌ام" is how clinicians actually speak.
Completeness is computed here so an unshippable map surfaces at review rather
than failing later at dispatch.

Everything the model names is checked against the catalog we supplied it, and
anything rejected is reported rather than dropped — a hallucinated lab id must
not look identical to "no lab was spoken", since silence and a wrong lab lead
to very different corrective actions.

Also fixed, from review of this commit:
- an empty prosthesis object no longer fabricates an "incomplete, cannot ship"
  warning on a plain restoration
- an override naming a tooth outside the selection now reports
  tooth_not_selected rather than malformed; the clinician was understood, the
  tooth just is not on this detail
- a due object with no `kind` is treated as no deadline rather than a blank
  "heard but lost" row; an unrecognised kind is still flagged, and named

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 17:41:41 +03:30
f3fb8736ab fix(backend): correct "next weekday" and harden resolvers against model output
Four defects found by review of the preceding commits.

"next <weekday>" was occurrence-anchored ("this" plus seven) rather than week-
anchored. Said on a Thursday, "Thursday next week" resolved to +14 instead of
+7: next week runs Sat 10-18 to Fri 10-24, so its Thursday is 10-23, not 10-30.
A lab case a week late. "next" now counts from the start of the following
Saturday-start week, which also lets "this" and "next" correctly coincide —
said on a Thursday, "the coming Saturday" and "Saturday next week" are the same
day. "this" stays occurrence-anchored so it can never resolve into the past.

The other three all come from the same root cause: exported functions that are
reachable from untrusted model output must degrade, not throw or drop.

- a non-object `due` (the model emitting a bare string) was treated as "no
  deadline spoken" and silently discarded; only null/undefined mean absent now,
  anything else is flagged so the clinician sees something was heard and lost
- isJalaliLeapYear / jalaliDaysInMonth threw for years outside the conversion
  table, contradicting the module's own "degrade to null" contract; they now
  return false / 0, which also makes isValidJalaliDate's day check naturally
  false
- civilDateInZone passed a client-supplied zone straight to Intl, which raises
  RangeError before any fallback; it now validates and backstops to UTC, so a
  bad zone costs at most a day rather than a 500

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 17:25:44 +03:30
5fcb72508e feat(backend): resolve spoken deadlines to ISO dates
Jalali conversion is arithmetic here, not inference. A model asked to turn
"۲۵ مهر" into ISO answers confidently and is often wrong, and @IsDateString()
accepts the wrong answer — so the model emits a date intent and this decides
what it means.

Deviation from the spec, deliberately: the resolver takes todayIso rather than
an IANA zone. Working in civil dates means nothing here reasons about instants.
The zone is used one level up, where civilDateInZone() derives "today" from the
actor's zone server-side — better than the spec's client-supplied date, which
the client could set arbitrarily.

Conventions pinned by tests:
- "this <weekday>" is the soonest occurrence strictly after today, so "by
  Thursday" said on a Thursday means the next one; a deadline of today is
  almost never what was meant. "next" adds a further week.
- month offsets clamp to the end of shorter months (31 Jan + 1 = 28/29 Feb)
- a resolved date in the past, or more than five years out, is treated as
  unresolved however it was arrived at — an absolute date the model invented
  can land anywhere
- no due date at all is not an error; an unparseable one is, and echoes what
  was heard so the review sheet can show it

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 17:17:15 +03:30
5878bd62e4 feat(backend): voice intent contract and tooth-intent resolver
The extraction model emits intents, never resolved values — no FDI codes, no
ISO dates. This adds the contract it must satisfy and the resolver that turns
spoken tooth references into FDI, so quadrant mirroring is a unit test rather
than a hope.

resolveToothIntent never guesses and never clamps: position 9, a deciduous
tooth, or a malformed shape resolve to null and are reported as unresolved with
the transcript span that produced them, so the review sheet can show the
clinician exactly which words were not understood.

Everything here parses untrusted model output, so nothing may throw:

- a non-array where a list was expected degrades like any other malformed shape
- explicit codes are trimmed, for parity with normalizeTeeth
- '51' reports as not_permanent_tooth (a real primary tooth the chart cannot
  show) while '99' reports as malformed — the clinician should not be told a
  deciduous tooth was heard when nothing tooth-shaped was
- unresolved items only dedupe when they carry a spoken span; without one,
  collapsing them would hide a lost tooth behind a single blank review row

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 17:13:26 +03:30
dfd376d97a feat(backend): extract shared FDI tooth geometry
Voice extraction needs quadrant mapping and adjacency server-side, and
treatment.utils.ts already held a private copy of the tooth set. Lift it into
common/fdi.ts rather than create a second source of truth; treatment.utils now
imports it, behaviour unchanged (existing suites still pass).

toFdi() is the single place the patient-right convention lives: quadrant 1 is
the patient's upper right, so upper+patient_right -> 1x, upper+patient_left ->
2x, lower+patient_left -> 3x, lower+patient_right -> 4x. Getting this backwards
mirrors every quadrant and yields a valid-looking code for the wrong tooth,
which no schema check can catch — so all four quadrants are pinned by tests,
along with out-of-range positions never being clamped and deciduous teeth being
rejected outright (the chart is permanent dentition only).

Adjacency mirrors the frontend's arch-order rule, so the midline pairs 11-21
and 41-31 count as neighbours exactly as the chart treats them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 17:05:54 +03:30
4764401766 feat(backend): port Jalali calendar arithmetic with tests
Voice extraction resolves spoken Jalali dates into ISO dates server-side, so
the backend needs the conversion the frontend already had. The resolvers live
here rather than in the frontend precisely because this half of the repo has a
test runner.

Ported from frontend/src/lib/i18n/persianCalendar.ts and verified faithful by
differential test: every day from 1900-2100 (73,414 days), zero mismatches on
conversion, leap years and month lengths.

Two deliberate divergences from the original:

- jalaliToIsoDate() returns null instead of throwing. It is fed model-supplied
  values, which may be nonsense, and an invalid date must degrade to
  "unresolved" rather than a 500. The year guard runs before jalaliDaysInMonth
  so the throwing jalCal is unreachable from it.
- toLatinDigits() also handles the Arabic-Indic block (U+0660-U+0669), not just
  Persian (U+06F0-U+06F9). ASR output can carry either, sometimes mixed with
  ASCII in one transcript; the frontend version only parses keystrokes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 17:00:41 +03:30
ca4d28a976 docs: spec for voice-driven treatment detail entry
Design spec for filling a TreatmentDetail by voice, settled across three
grilling sessions (30 decisions, logged in the spec).

Key shape:
- two-stage pipeline: OpenRouter whisper-1 -> gemini-3.7-flash
- the LLM emits *intents*, never FDI codes or ISO dates; pure Jest-tested
  backend resolvers own quadrant mapping and Jalali conversion
- provider registry keyed by locale so fa can diverge from en/nl
- review sheet confirms before anything touches the form
- audio and transcripts are never persisted

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 16:51:57 +03:30
47 changed files with 6181 additions and 52 deletions

View File

@@ -47,3 +47,28 @@ SMTP_PASSWORD=your_app_password
# SMS_IR_API_KEY=4QKMiSU4Kh7tWPLCdRMV0QpDh8WgF33YkWRS18BcG3vf4QHi
SMS_IR_API_KEY=lwbK7hxmjimNjFS4g5DWahh75EKCgJUfcUIinUQzfQXwXkSp
SMS_IR_TEMPLATE_ID=123456
# ── Voice treatment entry ──────────────────────────────────────────────────────
# Without OPENROUTER_API_KEY the microphone button does not render at all.
OPENROUTER_API_KEY=
# OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
# Locales the microphone is offered in. An unknown locale here fails at boot.
# VOICE_ENABLED_LOCALES=fa,en,nl
# Models, overridable per locale (VOICE_ASR_MODEL_FA, VOICE_LLM_MODEL_NL, ...).
# All locales share these today; the per-locale override exists so Persian can be
# repointed at a specialist ASR vendor without a code change.
# VOICE_ASR_MODEL=openai/whisper-1
# VOICE_LLM_MODEL=google/gemini-3.7-flash
# VOICE_ASR_PROVIDER_FA=openrouter
# VOICE_LLM_PROVIDER_FA=openrouter
# Recording cap in ms (0 = uncapped). 2 minutes bounds worst-case vendor spend at
# about 1.3 cents per recording.
# VOICE_MAX_RECORDING_MS=120000
# Per-user rate limit on the extract endpoint. Unreachable by a human — a recording
# plus processing takes ten seconds at minimum — so it is purely an abuse guard.
# VOICE_THROTTLE_TTL=60
# VOICE_THROTTLE_LIMIT=6

View File

@@ -1,5 +1,6 @@
import { Module } from '@nestjs/common';
import { ConfigModule } from '@nestjs/config';
import { ConfigModule, ConfigService } from '@nestjs/config';
import { ThrottlerModule } from '@nestjs/throttler';
import configurations from './configs/configurations';
import { AuthModule } from './modules/auth/auth.module';
import { AppController } from './app.controller';
@@ -20,6 +21,7 @@ import { LabCaseCommentsModule } from './modules/lab-case-comments/lab-case-comm
import { TodayModule } from './modules/today/today.module';
import { NotificationsModule } from './modules/notifications/notifications.module';
import { RealtimeModule } from './realtime/realtime.module';
import { VoiceModule } from './modules/voice/voice.module';
@Module({
imports: [
@@ -27,6 +29,22 @@ import { RealtimeModule } from './realtime/realtime.module';
isGlobal: true,
load: [configurations],
}),
// First use of @nestjs/throttler in this app. Deliberately NOT bound as a global
// APP_GUARD: a globally-bound ThrottlerGuard rate-limits every route against every
// named throttler, which would cap the whole API at the voice limit. ThrottlerGuard
// is applied to the one expensive route instead, so nothing else changes behaviour.
ThrottlerModule.forRootAsync({
inject: [ConfigService],
useFactory: (config: ConfigService) => ({
throttlers: [
{
name: 'voice',
ttl: (config.get<number>('voice.throttle.ttl') ?? 60) * 1000,
limit: config.get<number>('voice.throttle.limit') ?? 6,
},
],
}),
}),
PrismaModule, // ✅ ADD THIS
CatalogModule,
TreatmentCatalogModule,
@@ -43,9 +61,10 @@ import { RealtimeModule } from './realtime/realtime.module';
TodayModule,
NotificationsModule,
RealtimeModule,
VoiceModule,
AdminModule.forRoot(),
],
controllers: [AppController],
providers: [AppService],
providers: [AppService],
})
export class AppModule {}
export class AppModule {}

View File

@@ -0,0 +1,101 @@
import express, {
type NextFunction,
type Request,
type Response,
} from 'express';
import request from 'supertest';
import { createJsonBodyParser, VOICE_EXTRACT_PATH } from './body-parsers';
/**
* Guards a bug that made the voice endpoint completely unusable while surfacing as a
* generic 500: the large-body limit stopped applying, so every real recording — anything
* past roughly 20 seconds of audio — was rejected by Express's 100 kb default.
*/
type ProbeBody = { keys?: number; type?: string };
function buildApp(): express.Express {
const app = express();
app.use(createJsonBodyParser());
app.post('*splat', (req: Request, res: Response) => {
res.json({ keys: Object.keys((req.body ?? {}) as object).length });
});
// Surface body-parser's own error instead of Express's HTML default page.
app.use(
(
err: { status?: number; type?: string },
_req: Request,
res: Response,
next: NextFunction,
) => {
if (res.headersSent) {
next(err);
return;
}
res.status(err.status ?? 500).json({ type: err.type });
},
);
return app;
}
const bodyOfKb = (kb: number) => ({ audio: 'A'.repeat(kb * 1024) });
describe('createJsonBodyParser', () => {
it('accepts a body far past the default limit on the voice route', async () => {
const res = await request(buildApp())
.post(VOICE_EXTRACT_PATH)
.send(bodyOfKb(300));
expect(res.status).toBe(200);
expect((res.body as ProbeBody).keys).toBe(1);
});
it('accepts a realistic worst-case recording', async () => {
// Two minutes of opus is well under 1 MB, but wav is far larger; 4 MB must pass.
const res = await request(buildApp())
.post(VOICE_EXTRACT_PATH)
.send(bodyOfKb(4096));
expect(res.status).toBe(200);
});
it('keeps the default limit on every other route', async () => {
// The larger limit must not leak app-wide as a side effect.
const res = await request(buildApp())
.post('/api/auth/login')
.send(bodyOfKb(300));
expect(res.status).toBe(413);
expect((res.body as ProbeBody).type).toBe('entity.too.large');
});
it('still parses ordinary bodies on ordinary routes', async () => {
const res = await request(buildApp())
.post('/api/auth/login')
.send({ email: 'a@b.c' });
expect(res.status).toBe(200);
expect((res.body as ProbeBody).keys).toBe(1);
});
it('widens the limit for the spellings Express itself accepts', async () => {
// Express routes case-insensitively and ignores a trailing slash by default, so these
// all reach the voice controller. Any of them taking the 100 kb parser would 413 a
// real recording and read as a broken microphone.
for (const path of [
'/api/voice/extract/',
'/API/Voice/Extract',
'/api/Voice/extract/',
]) {
const res = await request(buildApp()).post(path).send(bodyOfKb(300));
expect(res.status).toBe(200);
}
});
it('does not widen the limit for a path that merely looks similar', async () => {
for (const path of [
'/api/voice/extract/extra',
'/api/voice',
'/voice/extract',
]) {
const res = await request(buildApp()).post(path).send(bodyOfKb(300));
expect(res.status).toBe(413);
}
});
});

View File

@@ -0,0 +1,45 @@
import {
json,
type NextFunction,
type Request,
type RequestHandler,
type Response,
} from 'express';
/** The one route that accepts a large body, and how large. */
export const VOICE_EXTRACT_PATH = '/api/voice/extract';
export const VOICE_BODY_LIMIT = '10mb';
/**
* JSON body parsing for the whole app.
*
* Voice recordings are base64 JSON and pass Express's 100 kb default at roughly 20 seconds
* of audio, so that one route needs a larger limit while every other endpoint keeps the
* default — a large body should not become acceptable everywhere.
*
* Deliberately a single middleware that *chooses* a parser, rather than a path-mounted
* parser stacked in front of a default one. That arrangement relied on Express's
* mount-path stripping plus body-parser skipping an already-parsed request, and it
* silently stopped applying when the surrounding middleware order shifted — at which point
* the endpoint rejected every real recording with a 500. One explicit branch has no such
* coupling, and is covered by body-parsers.spec.ts.
*/
/**
* Express routes case-insensitively and ignores a trailing slash unless configured
* otherwise, so `/API/Voice/Extract/` reaches the same controller. Matching only the
* canonical spelling would hand those requests the 100 kb parser and 413 every real
* recording — a failure that looks like a broken microphone, not a routing detail.
*/
function isVoiceExtractPath(path: string): boolean {
return path.toLowerCase().replace(/\/+$/, '') === VOICE_EXTRACT_PATH;
}
export function createJsonBodyParser(): RequestHandler {
const voiceParser = json({ limit: VOICE_BODY_LIMIT });
const defaultParser = json();
return (req: Request, res: Response, next: NextFunction) =>
isVoiceExtractPath(req.path)
? voiceParser(req, res, next)
: defaultParser(req, res, next);
}

View File

@@ -0,0 +1,31 @@
import { toLatinDigits } from './digits';
/**
* Exercised by the voice pipeline on two untrusted inputs: spoken dates, and the tooth
* code the extraction model echoes back — a Persian-digit "۲۶" that fails to normalise
* costs the clinician a tooth, silently.
*/
describe('toLatinDigits', () => {
it('normalises Persian digits and leaves everything else alone', () => {
expect(
toLatinDigits('\u06F1\u06F4\u06F0\u06F4/\u06F0\u06F7/\u06F2\u06F5'),
).toBe('1404/07/25');
expect(toLatinDigits('1404/07/25')).toBe('1404/07/25');
expect(toLatinDigits('\u062F\u0646\u062F\u0627\u0646 \u06F1\u06F4')).toBe(
'\u062F\u0646\u062F\u0627\u0646 14',
);
});
it('also normalises the Arabic-Indic block, which ASR output can carry', () => {
// U+0660..U+0669, distinct code points from the Persian U+06F0..U+06F9 block.
expect(
toLatinDigits('\u0661\u0664\u0660\u0664/\u0660\u0667/\u0662\u0665'),
).toBe('1404/07/25');
});
it('normalises a transcript that mixes both blocks with ASCII', () => {
expect(toLatinDigits('\u06F1\u06F4 and \u0661\u0665 and 16')).toBe(
'14 and 15 and 16',
);
});
});

View File

@@ -0,0 +1,24 @@
/**
* Persian (Extended Arabic-Indic, U+06F0U+06F9) zero, and Arabic-Indic (U+0660U+0669)
* zero. ASR output can carry either block, sometimes mixed with ASCII in one transcript.
*/
const PERSIAN_ZERO = 0x06f0;
const ARABIC_INDIC_ZERO = 0x0660;
/**
* Normalise Persian and Arabic-Indic digits to ASCII. Non-digits pass through.
*
* Deliberately wider than the frontend original, which only handles the Persian block:
* this parses model/ASR output rather than keystrokes, so both blocks must be accepted
* or a spoken date or tooth number silently degrades to "unresolved".
*
* Lives on its own rather than inside jalali.ts because tooth codes need it too, and a
* tooth module reaching into the calendar module would read as an accident.
*/
export function toLatinDigits(value: string): string {
return value.replace(/[۰-۹٠-٩]/g, (ch) => {
const code = ch.charCodeAt(0);
const base = code >= PERSIAN_ZERO ? PERSIAN_ZERO : ARABIC_INDIC_ZERO;
return String(code - base);
});
}

View File

@@ -29,8 +29,10 @@ export const ErrorCode = {
PERMISSION_LAB_ONLY: 'PERMISSION_LAB_ONLY',
PERMISSION_NOT_MEMBER: 'PERMISSION_NOT_MEMBER',
PERMISSION_OWNER_ONLY: 'PERMISSION_OWNER_ONLY',
PERMISSION_PARTICIPATION_SUBSCRIPTION: 'PERMISSION_PARTICIPATION_SUBSCRIPTION',
PERMISSION_ENABLE_PARTICIPATION_FIRST: 'PERMISSION_ENABLE_PARTICIPATION_FIRST',
PERMISSION_PARTICIPATION_SUBSCRIPTION:
'PERMISSION_PARTICIPATION_SUBSCRIPTION',
PERMISSION_ENABLE_PARTICIPATION_FIRST:
'PERMISSION_ENABLE_PARTICIPATION_FIRST',
PERMISSION_CLINIC_WORKING_HOURS: 'PERMISSION_CLINIC_WORKING_HOURS',
PERMISSION_ACCESS_APPOINTMENTS: 'PERMISSION_ACCESS_APPOINTMENTS',
PERMISSION_EDIT_APPOINTMENTS: 'PERMISSION_EDIT_APPOINTMENTS',
@@ -51,7 +53,8 @@ export const ErrorCode = {
VALIDATION_PASSWORD_REQUIRED: 'VALIDATION_PASSWORD_REQUIRED',
VALIDATION_MOBILE_INVALID: 'VALIDATION_MOBILE_INVALID',
VALIDATION_NAME_TOO_SHORT: 'VALIDATION_NAME_TOO_SHORT',
VALIDATION_ORGANIZATION_NAME_REQUIRED: 'VALIDATION_ORGANIZATION_NAME_REQUIRED',
VALIDATION_ORGANIZATION_NAME_REQUIRED:
'VALIDATION_ORGANIZATION_NAME_REQUIRED',
VALIDATION_ORGANIZATION_TYPE_INVALID: 'VALIDATION_ORGANIZATION_TYPE_INVALID',
VALIDATION_TOKEN_REQUIRED: 'VALIDATION_TOKEN_REQUIRED',
VALIDATION_FIELD_REQUIRED: 'VALIDATION_FIELD_REQUIRED',
@@ -69,8 +72,10 @@ export const ErrorCode = {
APPOINTMENT_INVALID_DATE: 'APPOINTMENT_INVALID_DATE',
APPOINTMENT_PROVIDER_NOT_MEMBER: 'APPOINTMENT_PROVIDER_NOT_MEMBER',
APPOINTMENT_PROVIDER_INACTIVE: 'APPOINTMENT_PROVIDER_INACTIVE',
APPOINTMENT_PROVIDER_NO_TREATMENT_EDIT: 'APPOINTMENT_PROVIDER_NO_TREATMENT_EDIT',
APPOINTMENT_PROVIDER_NO_WORKING_HOURS: 'APPOINTMENT_PROVIDER_NO_WORKING_HOURS',
APPOINTMENT_PROVIDER_NO_TREATMENT_EDIT:
'APPOINTMENT_PROVIDER_NO_TREATMENT_EDIT',
APPOINTMENT_PROVIDER_NO_WORKING_HOURS:
'APPOINTMENT_PROVIDER_NO_WORKING_HOURS',
APPOINTMENT_PROVIDER_NOT_WORKING_DAY: 'APPOINTMENT_PROVIDER_NOT_WORKING_DAY',
APPOINTMENT_OUTSIDE_WORKING_HOURS: 'APPOINTMENT_OUTSIDE_WORKING_HOURS',
APPOINTMENT_NOT_FOUND: 'APPOINTMENT_NOT_FOUND',
@@ -82,7 +87,8 @@ export const ErrorCode = {
WORKING_HOURS_INVALID: 'WORKING_HOURS_INVALID',
WORKING_HOURS_OWNER_NOT_ALLOWED: 'WORKING_HOURS_OWNER_NOT_ALLOWED',
WORKING_HOURS_CONFLICTS_WITH_APPOINTMENTS: 'WORKING_HOURS_CONFLICTS_WITH_APPOINTMENTS',
WORKING_HOURS_CONFLICTS_WITH_APPOINTMENTS:
'WORKING_HOURS_CONFLICTS_WITH_APPOINTMENTS',
STAFF_UNKNOWN_PERMISSIONS: 'STAFF_UNKNOWN_PERMISSIONS',
STAFF_NO_SUBSCRIPTION: 'STAFF_NO_SUBSCRIPTION',
@@ -147,7 +153,8 @@ export const ErrorCode = {
TREATMENT_ATTACHMENT_NOT_FOUND: 'TREATMENT_ATTACHMENT_NOT_FOUND',
TREATMENT_FILE_UNAVAILABLE: 'TREATMENT_FILE_UNAVAILABLE',
TREATMENT_CASE_INVALID_ATTACHMENTS: 'TREATMENT_CASE_INVALID_ATTACHMENTS',
TREATMENT_TOOTH_PROSTHESIS_INCOMPLETE: 'TREATMENT_TOOTH_PROSTHESIS_INCOMPLETE',
TREATMENT_TOOTH_PROSTHESIS_INCOMPLETE:
'TREATMENT_TOOTH_PROSTHESIS_INCOMPLETE',
TREATMENT_DETAIL_SENT: 'TREATMENT_DETAIL_SENT',
TREATMENT_NOT_FOUND: 'TREATMENT_NOT_FOUND',
TREATMENT_PATIENT_OR_WALK_IN: 'TREATMENT_PATIENT_OR_WALK_IN',
@@ -177,6 +184,14 @@ export const ErrorCode = {
TODAY_INVALID_RANGE: 'TODAY_INVALID_RANGE',
TODAY_INVALID_RANGE_ORDER: 'TODAY_INVALID_RANGE_ORDER',
// Voice treatment entry
VOICE_NOT_AVAILABLE: 'VOICE_NOT_AVAILABLE',
VOICE_CLIP_TOO_LONG: 'VOICE_CLIP_TOO_LONG',
VOICE_ASR_FAILED: 'VOICE_ASR_FAILED',
VOICE_EXTRACT_FAILED: 'VOICE_EXTRACT_FAILED',
VOICE_NOTHING_RECOGNIZED: 'VOICE_NOTHING_RECOGNIZED',
VOICE_RATE_LIMITED: 'VOICE_RATE_LIMITED',
// Generic HTTP
NOT_FOUND: 'NOT_FOUND',
CONFLICT: 'CONFLICT',

View File

@@ -0,0 +1,144 @@
import {
areArchNeighbors,
sortInArchOrder,
FDI_TOOTH_IDS,
isFdiTooth,
sameArch,
teethBetweenInclusive,
toFdi,
} from './fdi';
describe('FDI geometry', () => {
describe('toFdi — the patient-right convention', () => {
// A mirrored quadrant produces a *valid* code for the wrong tooth, so no schema
// check can catch it. These four cases are the guard.
it('maps upper + patient right to quadrant 1', () => {
expect(toFdi('upper', 'patient_right', 6)).toBe('16');
expect(toFdi('upper', 'patient_right', 1)).toBe('11');
});
it('maps upper + patient left to quadrant 2', () => {
expect(toFdi('upper', 'patient_left', 6)).toBe('26');
expect(toFdi('upper', 'patient_left', 8)).toBe('28');
});
it('maps lower + patient left to quadrant 3', () => {
expect(toFdi('lower', 'patient_left', 6)).toBe('36');
});
it('maps lower + patient right to quadrant 4', () => {
expect(toFdi('lower', 'patient_right', 6)).toBe('46');
expect(toFdi('lower', 'patient_right', 8)).toBe('48');
});
it('never clamps an out-of-range position', () => {
expect(toFdi('upper', 'patient_right', 9)).toBeNull();
expect(toFdi('upper', 'patient_right', 0)).toBeNull();
expect(toFdi('upper', 'patient_right', -1)).toBeNull();
expect(toFdi('upper', 'patient_right', 1.5)).toBeNull();
expect(toFdi('upper', 'patient_right', Number.NaN)).toBeNull();
});
});
describe('isFdiTooth', () => {
it('accepts all 32 permanent teeth', () => {
expect(FDI_TOOTH_IDS.size).toBe(32);
for (const tooth of FDI_TOOTH_IDS) expect(isFdiTooth(tooth)).toBe(true);
});
it('rejects deciduous teeth — the chart has no primary dentition', () => {
for (const tooth of ['51', '55', '61', '71', '85']) {
expect(isFdiTooth(tooth)).toBe(false);
}
});
it('rejects garbage', () => {
for (const value of [
'',
'1',
'19',
'10',
'29',
'99',
14,
null,
undefined,
{},
]) {
expect(isFdiTooth(value)).toBe(false);
}
});
});
describe('adjacency', () => {
it('treats neighbours within a quadrant as adjacent', () => {
expect(areArchNeighbors('14', '15')).toBe(true);
expect(areArchNeighbors('15', '14')).toBe(true);
});
it('treats the midline pairs as adjacent', () => {
expect(areArchNeighbors('11', '21')).toBe(true);
expect(areArchNeighbors('41', '31')).toBe(true);
});
it('rejects non-neighbours and cross-arch pairs', () => {
expect(areArchNeighbors('14', '16')).toBe(false);
expect(areArchNeighbors('18', '28')).toBe(false);
expect(areArchNeighbors('14', '44')).toBe(false);
expect(areArchNeighbors('14', '14')).toBe(false);
});
});
describe('sameArch', () => {
it('groups by arch, not by quadrant', () => {
expect(sameArch('18', '28')).toBe(true);
expect(sameArch('48', '38')).toBe(true);
expect(sameArch('18', '48')).toBe(false);
expect(sameArch('14', '99')).toBe(false);
});
});
describe('teethBetweenInclusive', () => {
it('returns the span in arch order regardless of argument order', () => {
expect(teethBetweenInclusive('14', '16')).toEqual(['16', '15', '14']);
expect(teethBetweenInclusive('16', '14')).toEqual(['16', '15', '14']);
});
it('spans the midline', () => {
expect(teethBetweenInclusive('12', '22')).toEqual([
'12',
'11',
'21',
'22',
]);
});
it('returns a single tooth for identical endpoints', () => {
expect(teethBetweenInclusive('14', '14')).toEqual(['14']);
});
it('returns null across arches or for unknown teeth', () => {
expect(teethBetweenInclusive('14', '44')).toBeNull();
expect(teethBetweenInclusive('14', '99')).toBeNull();
expect(teethBetweenInclusive('99', '14')).toBeNull();
});
});
describe('sortInArchOrder', () => {
it('sorts along the arch rather than lexically', () => {
expect(sortInArchOrder(['14', '16', '15'])).toEqual(['16', '15', '14']);
});
it('places 11 beside 21 across the midline', () => {
expect(sortInArchOrder(['21', '11', '12'])).toEqual(['12', '11', '21']);
});
it('is a no-op for an empty or single-tooth list', () => {
expect(sortInArchOrder([])).toEqual([]);
expect(sortInArchOrder(['14'])).toEqual(['14']);
});
it('does not drop teeth it cannot place', () => {
expect(sortInArchOrder(['99', '14']).sort()).toEqual(['14', '99']);
});
});
});

150
backend/src/common/fdi.ts Normal file
View File

@@ -0,0 +1,150 @@
/**
* FDI tooth geometry — permanent dentition only.
*
* Mirrors `frontend/src/components/treatment/fdiToothMeta.ts` and the adjacency rules in
* `toothSelectionGroups.ts`. Adjacency is defined by position in the arch order, so the
* midline pairs (1121, 4131) are neighbours, exactly as the chart treats them.
*/
import { toLatinDigits } from './digits';
export type Arch = 'upper' | 'lower';
/** Which side of the *patient*, not of the screen. Quadrant 1 is the patient's upper right. */
export type PatientSide = 'patient_right' | 'patient_left';
/**
* Upper arch in chart order: patient's RIGHT (18) → midline → patient's LEFT (28).
* That is the drawn left-to-right layout, which is the mirror of the patient's own sides.
* Do not read a tooth position off this array by index — use `toFdi()`, which owns the
* side convention.
*/
export const FDI_UPPER_ARCH_ORDER = [
'18',
'17',
'16',
'15',
'14',
'13',
'12',
'11',
'21',
'22',
'23',
'24',
'25',
'26',
'27',
'28',
] as const;
/** Lower arch, same chart ordering: patient's RIGHT (48) → midline → patient's LEFT (38). */
export const FDI_LOWER_ARCH_ORDER = [
'48',
'47',
'46',
'45',
'44',
'43',
'42',
'41',
'31',
'32',
'33',
'34',
'35',
'36',
'37',
'38',
] as const;
export const FDI_TOOTH_IDS: ReadonlySet<string> = new Set<string>([
...FDI_UPPER_ARCH_ORDER,
...FDI_LOWER_ARCH_ORDER,
]);
export function isFdiTooth(value: unknown): value is string {
return typeof value === 'string' && FDI_TOOTH_IDS.has(value);
}
/**
* Clean up a tooth code the extraction model echoed back, before it is matched.
*
* The model is transcribing Persian speech, so it can hand back "۲۶" in Persian digits or
* "2 6" from a digit-by-digit dictation. Neither matches an FDI code literally, and a
* near-miss here does not fail loudly — the tooth quietly turns into "not understood".
* Returns '' for anything that is not a string.
*/
export function normalizeFdiCode(value: unknown): string {
if (typeof value !== 'string') return '';
return toLatinDigits(value).replace(/\s+/g, '');
}
function archOrder(tooth: string): readonly string[] | null {
if ((FDI_UPPER_ARCH_ORDER as readonly string[]).includes(tooth))
return FDI_UPPER_ARCH_ORDER;
if ((FDI_LOWER_ARCH_ORDER as readonly string[]).includes(tooth))
return FDI_LOWER_ARCH_ORDER;
return null;
}
export function sameArch(a: string, b: string): boolean {
const archA = archOrder(a);
const archB = archOrder(b);
return Boolean(archA && archB && archA === archB);
}
export function areArchNeighbors(a: string, b: string): boolean {
const arch = archOrder(a);
if (!arch || !sameArch(a, b)) return false;
return Math.abs(arch.indexOf(a) - arch.indexOf(b)) === 1;
}
/** Inclusive span between two teeth of the same arch, in arch order. Null if not comparable. */
export function teethBetweenInclusive(a: string, b: string): string[] | null {
const arch = archOrder(a);
if (!arch || !sameArch(a, b)) return null;
const i = arch.indexOf(a);
const j = arch.indexOf(b);
if (i < 0 || j < 0) return null;
const [from, to] = i <= j ? [i, j] : [j, i];
return [...arch.slice(from, to + 1)];
}
/**
* Arch + patient side + position (1 = central incisor … 8 = third molar) → FDI code.
*
* This function is the single place the patient-right convention lives. Getting it
* backwards mirrors every quadrant and produces a valid-looking code for the wrong tooth,
* which no schema check can catch — hence the exhaustive test coverage.
*/
export function toFdi(
arch: Arch,
side: PatientSide,
position: number,
): string | null {
if (!Number.isInteger(position) || position < 1 || position > 8) return null;
let quadrant: number;
if (arch === 'upper') {
quadrant = side === 'patient_right' ? 1 : 2;
} else {
quadrant = side === 'patient_left' ? 3 : 4;
}
const code = `${quadrant}${position}`;
return FDI_TOOTH_IDS.has(code) ? code : null;
}
/**
* Sort teeth along the arch, not lexically — a bridge reads 16-15-14, and 11 sits beside
* 21 across the midline. Teeth from another arch (or unknown) sort to the end, stably.
*/
export function sortInArchOrder(teeth: readonly string[]): string[] {
if (teeth.length === 0) return [];
const arch = teeth.map((t) => archOrder(t)).find((a) => a !== null) ?? null;
if (!arch) return [...teeth];
const indexOf = (tooth: string) => {
const i = arch.indexOf(tooth);
return i === -1 ? Number.MAX_SAFE_INTEGER : i;
};
return [...teeth].sort((a, b) => indexOf(a) - indexOf(b));
}

View File

@@ -0,0 +1,89 @@
import {
gregorianToJalali,
isJalaliLeapYear,
isValidJalaliDate,
jalaliDaysInMonth,
jalaliToGregorian,
jalaliToIsoDate,
} from './jalali';
describe('jalali calendar', () => {
// Anchors verified against frontend/src/lib/i18n/persianCalendar.ts, the source this
// was ported from, over every day between 1900 and 2100.
it('converts known Jalali dates to ISO', () => {
expect(jalaliToIsoDate(1404, 1, 1)).toBe('2025-03-21'); // Nowruz 1404
expect(jalaliToIsoDate(1404, 7, 25)).toBe('2025-10-17');
expect(jalaliToIsoDate(1404, 6, 31)).toBe('2025-09-22'); // 31-day first half
expect(jalaliToIsoDate(1405, 1, 1)).toBe('2026-03-21');
});
it('round-trips Gregorian → Jalali → Gregorian', () => {
for (const [gy, gm, gd] of [
[2025, 3, 21],
[2025, 10, 17],
[2026, 1, 1],
[2024, 2, 29], // Gregorian leap day
[1999, 12, 31],
] as const) {
const [jy, jm, jd] = gregorianToJalali(gy, gm, gd);
expect(jalaliToGregorian(jy, jm, jd)).toEqual([gy, gm, gd]);
}
});
describe('leap years', () => {
it('gives Esfand 30 days in a leap year and 29 otherwise', () => {
expect(isJalaliLeapYear(1403)).toBe(true);
expect(jalaliDaysInMonth(1403, 12)).toBe(30);
expect(jalaliToIsoDate(1403, 12, 30)).toBe('2025-03-20');
expect(isJalaliLeapYear(1404)).toBe(false);
expect(jalaliDaysInMonth(1404, 12)).toBe(29);
expect(jalaliToIsoDate(1404, 12, 29)).toBe('2026-03-20');
});
it('rejects Esfand 30 in a non-leap year', () => {
expect(isValidJalaliDate(1404, 12, 30)).toBe(false);
expect(jalaliToIsoDate(1404, 12, 30)).toBeNull();
});
});
describe('month lengths', () => {
it('is 31 days for months 1-6 and 30 for 7-11', () => {
for (let m = 1; m <= 6; m += 1)
expect(jalaliDaysInMonth(1404, m)).toBe(31);
for (let m = 7; m <= 11; m += 1)
expect(jalaliDaysInMonth(1404, m)).toBe(30);
});
});
describe('invalid input degrades to null rather than throwing', () => {
// The resolver feeds this model-supplied values, which may be nonsense.
it.each([
['month 13', 1404, 13, 1],
['month 0', 1404, 0, 1],
['day 0', 1404, 1, 0],
['day 32 in a 31-day month', 1404, 1, 32],
['day 31 in a 30-day month', 1404, 7, 31],
['year beyond the conversion table', 9999, 1, 1],
['non-integer day', 1404, 1, 1.5],
])('returns null for %s', (_label, jy, jm, jd) => {
expect(jalaliToIsoDate(jy, jm, jd)).toBeNull();
});
});
describe('no export throws on an unsupported year', () => {
// The whole module is reachable from model-supplied values, so it degrades instead of
// raising — jalaliToIsoDate's guard is not the only entry point.
it.each([9999, -9999, 3178, Number.NaN, 1.5])('year %s', (jy) => {
expect(() => isJalaliLeapYear(jy)).not.toThrow();
expect(() => jalaliDaysInMonth(jy, 1)).not.toThrow();
expect(isJalaliLeapYear(jy)).toBe(false);
expect(jalaliDaysInMonth(jy, 1)).toBe(0);
});
it('returns 0 days for an impossible month', () => {
expect(jalaliDaysInMonth(1404, 0)).toBe(0);
expect(jalaliDaysInMonth(1404, 13)).toBe(0);
});
});
});

View File

@@ -0,0 +1,186 @@
/**
* Jalali (Persian) calendar arithmetic.
*
* Ported from `frontend/src/lib/i18n/persianCalendar.ts` (itself from jalaali-js, MIT).
* The backend needs this because voice extraction resolves spoken Jalali dates into ISO
* dates server-side, where the resolvers are unit-tested — the frontend has no test
* runner. Keep the two copies in step; the underlying calendar does not change.
*/
const BREAKS = [
-61, 9, 38, 199, 426, 686, 756, 818, 1111, 1181, 1210, 1635, 2060, 2097, 2192,
2262, 2324, 2394, 2456, 3178,
];
/** Inclusive lower / exclusive upper Jalali year bounds the conversion table covers. */
export const MIN_JALALI_YEAR = BREAKS[0];
export const MAX_JALALI_YEAR = BREAKS[BREAKS.length - 1];
function div(a: number, b: number): number {
return Math.trunc(a / b);
}
function mod(a: number, b: number): number {
return a - Math.trunc(a / b) * b;
}
function g2d(gy: number, gm: number, gd: number): number {
let d =
div((gy + div(gm - 8, 6) + 100100) * 1461, 4) +
div(153 * mod(gm + 9, 12) + 2, 5) +
gd -
34840408;
d = d - div(div(gy + 100100 + div(gm - 8, 6), 100) * 3, 4) + 752;
return d;
}
function d2g(jdn: number): { gy: number; gm: number; gd: number } {
let j = 4 * jdn + 139361631;
j = j + div(div(4 * jdn + 183187720, 146097) * 3, 4) * 4 - 3908;
const i = div(mod(j, 1461), 4) * 5 + 308;
const gd = div(mod(i, 153), 5) + 1;
const gm = mod(div(i, 153), 12) + 1;
const gy = div(j, 1461) - 100100 + div(8 - gm, 6);
return { gy, gm, gd };
}
function jalCal(
jy: number,
withoutLeap: boolean,
): { leap?: number; gy: number; march: number } {
const bl = BREAKS.length;
const gy = jy + 621;
let leapJ = -14;
let jp = BREAKS[0];
let jump = 0;
let leap = 0;
let n = 0;
if (jy < jp || jy >= BREAKS[bl - 1]) {
throw new Error(`Invalid Jalaali year ${jy}`);
}
for (let i = 1; i < bl; i += 1) {
const jm = BREAKS[i];
jump = jm - jp;
if (jy < jm) break;
leapJ = leapJ + div(jump, 33) * 8 + div(mod(jump, 33), 4);
jp = jm;
}
n = jy - jp;
leapJ = leapJ + div(n, 33) * 8 + div(mod(n, 33) + 3, 4);
if (mod(jump, 33) === 4 && jump - n === 4) leapJ += 1;
const leapG = div(gy, 4) - div((div(gy, 100) + 1) * 3, 4) - 150;
const march = 20 + leapJ - leapG;
if (withoutLeap) return { gy, march };
if (jump - n < 6) n = n - jump + div(jump + 4, 33) * 33;
leap = mod(mod(n + 1, 33) - 1, 4);
if (leap === -1) leap = 4;
return { leap, gy, march };
}
function j2d(jy: number, jm: number, jd: number): number {
const r = jalCal(jy, true);
return g2d(r.gy, 3, r.march) + (jm - 1) * 31 - div(jm, 7) * (jm - 7) + jd - 1;
}
function d2j(jdn: number): { jy: number; jm: number; jd: number } {
const { gy } = d2g(jdn);
let jy = gy - 621;
const r = jalCal(jy, false);
const jdn1f = g2d(gy, 3, r.march);
let k = jdn - jdn1f;
let jm: number;
let jd: number;
if (k >= 0) {
if (k <= 185) {
jm = 1 + div(k, 31);
jd = mod(k, 31) + 1;
return { jy, jm, jd };
}
k -= 186;
} else {
jy -= 1;
k += 179;
if (r.leap === 1) k += 1;
}
jm = 7 + div(k, 30);
jd = mod(k, 30) + 1;
return { jy, jm, jd };
}
export function gregorianToJalali(
gy: number,
gm: number,
gd: number,
): [number, number, number] {
const { jy, jm, jd } = d2j(g2d(gy, gm, gd));
return [jy, jm, jd];
}
export function jalaliToGregorian(
jy: number,
jm: number,
jd: number,
): [number, number, number] {
const { gy, gm, gd } = d2g(j2d(jy, jm, jd));
return [gy, gm, gd];
}
/** True when the year is inside the conversion table's supported range. */
export function isSupportedJalaliYear(jy: number): boolean {
return Number.isInteger(jy) && jy >= MIN_JALALI_YEAR && jy < MAX_JALALI_YEAR;
}
/** False for unsupported years rather than throwing — see the module contract. */
export function isJalaliLeapYear(jy: number): boolean {
if (!isSupportedJalaliYear(jy)) return false;
const r = jalCal(jy, false);
return r.leap === 0;
}
/**
* Days in a Jalali month, or 0 when the year or month is not real.
*
* Zero rather than a throw: every export here is reachable from model-supplied values, so
* the whole module degrades instead of raising. Zero also makes `isValidJalaliDate`'s
* `jd <= jalaliDaysInMonth(...)` naturally false.
*/
export function jalaliDaysInMonth(jy: number, jm: number): number {
if (!isSupportedJalaliYear(jy)) return 0;
if (!Number.isInteger(jm) || jm < 1 || jm > 12) return 0;
if (jm <= 6) return 31;
if (jm <= 11) return 30;
return isJalaliLeapYear(jy) ? 30 : 29;
}
/** True when the triple is a real Jalali date inside the supported year range. */
export function isValidJalaliDate(jy: number, jm: number, jd: number): boolean {
if (!Number.isInteger(jy) || !Number.isInteger(jm) || !Number.isInteger(jd)) {
return false;
}
if (jy < MIN_JALALI_YEAR || jy >= MAX_JALALI_YEAR) return false;
if (jm < 1 || jm > 12) return false;
if (jd < 1) return false;
return jd <= jalaliDaysInMonth(jy, jm);
}
/**
* Jalali triple → `YYYY-MM-DD`, or null when the date is not real.
*
* Returns null rather than throwing: callers resolve model-supplied values, which may be
* nonsense, and an invalid date must degrade to "unresolved" rather than a 500.
*/
export function jalaliToIsoDate(
jy: number,
jm: number,
jd: number,
): string | null {
if (!isValidJalaliDate(jy, jm, jd)) return null;
const [gy, gm, gd] = jalaliToGregorian(jy, jm, jd);
return `${String(gy).padStart(4, '0')}-${String(gm).padStart(2, '0')}-${String(gd).padStart(2, '0')}`;
}

View File

@@ -1,5 +1,9 @@
import { appointmentWithinWorkingHours } from './working-hours';
import { isValidIanaTimeZone, zonedWeekdayAndMinutes } from './zoned-civil-time';
import {
civilDateInZone,
isValidIanaTimeZone,
zonedWeekdayAndMinutes,
} from './zoned-civil-time';
describe('zoned civil time', () => {
it('accepts IANA zones and rejects garbage', () => {
@@ -20,10 +24,48 @@ describe('zoned civil time', () => {
const start = new Date('2026-08-20T06:15:00.000Z');
const end = new Date('2026-08-20T06:45:00.000Z');
expect(
appointmentWithinWorkingHours(start, end, [{ startMinute: 8 * 60, endMinute: 17 * 60 }], 'Asia/Tehran'),
appointmentWithinWorkingHours(
start,
end,
[{ startMinute: 8 * 60, endMinute: 17 * 60 }],
'Asia/Tehran',
),
).toBe(true);
expect(
appointmentWithinWorkingHours(start, end, [{ startMinute: 8 * 60, endMinute: 17 * 60 }], 'UTC'),
appointmentWithinWorkingHours(
start,
end,
[{ startMinute: 8 * 60, endMinute: 17 * 60 }],
'UTC',
),
).toBe(false);
});
describe('civilDateInZone', () => {
it('gives the local civil date, which can differ from the UTC date', () => {
// 21:30 UTC is already the next day in Tehran (+03:30).
const instant = new Date('2025-10-11T21:30:00.000Z');
expect(civilDateInZone(instant, 'UTC')).toBe('2025-10-11');
expect(civilDateInZone(instant, 'Asia/Tehran')).toBe('2025-10-12');
});
it('gives the previous day for zones behind UTC just after midnight', () => {
const instant = new Date('2025-10-11T02:00:00.000Z');
expect(civilDateInZone(instant, 'America/New_York')).toBe('2025-10-10');
expect(civilDateInZone(instant, 'Europe/Amsterdam')).toBe('2025-10-11');
});
it('falls back to UTC on an invalid zone rather than throwing', () => {
// Intl raises RangeError on an unknown zone and this takes a client-supplied string.
const instant = new Date('2025-10-11T21:30:00.000Z');
expect(() => civilDateInZone(instant, 'Not/AZone')).not.toThrow();
expect(civilDateInZone(instant, 'Not/AZone')).toBe('2025-10-11');
expect(civilDateInZone(instant, '')).toBe('2025-10-11');
});
it('zero-pads single-digit months and days', () => {
expect(civilDateInZone(new Date('2025-01-05T12:00:00.000Z'), 'UTC')).toBe(
'2025-01-05',
);
});
});
});

View File

@@ -53,3 +53,27 @@ export function civilDateJsWeekday(isoDate: string): number {
const utcNoon = new Date(Date.UTC(y, m - 1, d, 12, 0, 0, 0));
return utcNoon.getUTCDay();
}
/**
* Today's civil date (`YYYY-MM-DD`) in an IANA zone.
*
* Lets the server derive "today" from a client-supplied time zone instead of trusting a
* client-supplied date, which matters for relative deadlines like "by Thursday".
*/
export function civilDateInZone(date: Date, timeZone: string): string {
// Intl throws RangeError on an unknown zone, before any fallback below could help, and
// this receives a client-supplied string. Callers validate first; this is the backstop
// so a bad zone degrades to a date that is at most a day out rather than a 500.
const zone = isValidIanaTimeZone(timeZone) ? timeZone : 'UTC';
const parts = new Intl.DateTimeFormat('en-CA', {
timeZone: zone,
year: 'numeric',
month: '2-digit',
day: '2-digit',
}).formatToParts(date);
const year = parts.find((p) => p.type === 'year')?.value ?? '1970';
const month = parts.find((p) => p.type === 'month')?.value ?? '01';
const day = parts.find((p) => p.type === 'day')?.value ?? '01';
return `${year}-${month}-${day}`;
}

View File

@@ -1,13 +1,14 @@
// backend/src/config/configuration.ts
/** Matches values accepted by jsonwebtoken `expiresIn` (via ms), e.g. 7d, 15m, or plain seconds. */
const JWT_TIMESPAN_PATTERN =
/^\d+(\.\d+)?(ms|s|m|h|d|w|y)?$/i;
const JWT_TIMESPAN_PATTERN = /^\d+(\.\d+)?(ms|s|m|h|d|w|y)?$/i;
function assertJwtSecret(value: string, envKey: string): string {
const trimmed = value.trim();
if (!trimmed) {
throw new Error(`❌ Environment variable ${envKey} is required but not set`);
throw new Error(
`❌ Environment variable ${envKey} is required but not set`,
);
}
if (trimmed.length < 16) {
throw new Error(`${envKey} must be at least 16 characters`);
@@ -21,7 +22,9 @@ function assertJwtSecret(value: string, envKey: string): string {
function assertJwtTimespan(value: string, envKey: string): string {
const trimmed = value.trim();
if (!trimmed) {
throw new Error(`❌ Environment variable ${envKey} is required but not set`);
throw new Error(
`❌ Environment variable ${envKey} is required but not set`,
);
}
if (!/^\d+$/.test(trimmed) && !JWT_TIMESPAN_PATTERN.test(trimmed)) {
throw new Error(
@@ -31,7 +34,10 @@ function assertJwtTimespan(value: string, envKey: string): string {
return trimmed;
}
function parseEnvBoolean(value: string | undefined, defaultValue: boolean): boolean {
function parseEnvBoolean(
value: string | undefined,
defaultValue: boolean,
): boolean {
if (value === undefined || value.trim() === '') {
return defaultValue;
}
@@ -67,8 +73,37 @@ export interface Config {
apiKey: string | null;
templateId: number;
};
voice: VoiceConfig;
}
/** Only OpenRouter today. The indirection exists so a locale can diverge without code. */
export type VoiceProviderId = 'openrouter';
export type VoiceProfile = {
asr: { provider: VoiceProviderId; model: string };
llm: { provider: VoiceProviderId; model: string };
};
export interface VoiceConfig {
openRouter: {
apiKey: string | null;
baseUrl: string;
};
/**
* Locale → provider profile. A locale absent here has no microphone button at all —
* clean absence rather than a dead control.
*/
profiles: Record<string, VoiceProfile>;
/** Recording cap in ms; null means uncapped. */
maxRecordingMs: number | null;
throttle: { ttl: number; limit: number };
}
export const VOICE_LOCALES = ['fa', 'en', 'nl'] as const;
const DEFAULT_ASR_MODEL = 'openai/whisper-1';
const DEFAULT_LLM_MODEL = 'google/gemini-3.7-flash';
export default (): Config => {
// Helper function to get required env var with type safety
const getEnvVar = (key: string): string => {
@@ -129,5 +164,96 @@ export default (): Config => {
apiKey: process.env.SMS_IR_API_KEY?.trim() || null,
templateId: getEnvVarAsNumber('SMS_IR_TEMPLATE_ID', 123456),
},
voice: buildVoiceConfig(getEnvVarWithDefault, getEnvVarAsNumber),
};
};
};
const VOICE_PROVIDER_IDS: readonly VoiceProviderId[] = ['openrouter'];
/** Unknown provider ids fail at boot; silently coercing a typo would ship the wrong vendor. */
function parseProviderId(
value: string | undefined,
envKey: string,
): VoiceProviderId {
const trimmed = value?.trim();
if (!trimmed) return 'openrouter';
if ((VOICE_PROVIDER_IDS as readonly string[]).includes(trimmed)) {
return trimmed as VoiceProviderId;
}
throw new Error(
`${envKey}="${trimmed}" is not a known voice provider (${VOICE_PROVIDER_IDS.join(', ')})`,
);
}
/**
* Every enabled locale gets its own ASR and LLM provider+model, each overridable
* independently. They all point at the same OpenRouter models today; the per-locale
* indirection is kept because Persian ASR is the weakest link and swapping only `fa` must
* not be a code change.
*/
function buildVoiceConfig(
getEnvVarWithDefault: (key: string, defaultValue: string) => string,
getEnvVarAsNumber: (key: string, defaultValue: number) => number,
): VoiceConfig {
const enabled = getEnvVarWithDefault(
'VOICE_ENABLED_LOCALES',
VOICE_LOCALES.join(','),
)
.split(',')
.map((locale) => locale.trim().toLowerCase())
.filter(Boolean);
for (const locale of enabled) {
// Fail loudly, like parseProviderId. Silently filtering a typo would disable the
// microphone everywhere with nothing to explain why.
if (!(VOICE_LOCALES as readonly string[]).includes(locale)) {
throw new Error(
`❌ VOICE_ENABLED_LOCALES contains unknown locale "${locale}" (known: ${VOICE_LOCALES.join(', ')})`,
);
}
}
const profiles: Record<string, VoiceProfile> = {};
for (const locale of enabled) {
const suffix = locale.toUpperCase();
profiles[locale] = {
asr: {
provider: parseProviderId(
process.env[`VOICE_ASR_PROVIDER_${suffix}`],
`VOICE_ASR_PROVIDER_${suffix}`,
),
model: getEnvVarWithDefault(
`VOICE_ASR_MODEL_${suffix}`,
getEnvVarWithDefault('VOICE_ASR_MODEL', DEFAULT_ASR_MODEL),
),
},
llm: {
provider: parseProviderId(
process.env[`VOICE_LLM_PROVIDER_${suffix}`],
`VOICE_LLM_PROVIDER_${suffix}`,
),
model: getEnvVarWithDefault(
`VOICE_LLM_MODEL_${suffix}`,
getEnvVarWithDefault('VOICE_LLM_MODEL', DEFAULT_LLM_MODEL),
),
},
};
}
const maxRecordingMs = getEnvVarAsNumber('VOICE_MAX_RECORDING_MS', 120_000);
return {
openRouter: {
apiKey: process.env.OPENROUTER_API_KEY?.trim() || null,
baseUrl: getEnvVarWithDefault(
'OPENROUTER_BASE_URL',
'https://openrouter.ai/api/v1',
),
},
profiles,
maxRecordingMs: maxRecordingMs > 0 ? maxRecordingMs : null,
throttle: {
ttl: getEnvVarAsNumber('VOICE_THROTTLE_TTL', 60),
limit: getEnvVarAsNumber('VOICE_THROTTLE_LIMIT', 6),
},
};
}

View File

@@ -1,6 +1,8 @@
// backend/src/main.ts
import { NestFactory } from '@nestjs/core';
import { urlencoded } from 'express';
import { AppModule } from './app.module';
import { createJsonBodyParser } from './common/body-parsers';
import { ValidationPipe } from '@nestjs/common';
import cookieParser from 'cookie-parser'; // 👈 Change this line!
import { DocumentBuilder, SwaggerModule } from '@nestjs/swagger';
@@ -23,7 +25,14 @@ console.log = (...args) => {
};
async function bootstrap() {
const app = await NestFactory.create(AppModule);
// bodyParser is disabled here so the JSON parsers can be registered in an explicit
// order below; Nest's built-in one is installed during create() and would otherwise
// reject a voice recording at its 100 kb default before any later middleware ran.
const app = await NestFactory.create(AppModule, { bodyParser: false });
// Voice needs a larger JSON limit than everything else; see body-parsers.ts.
app.use(createJsonBodyParser());
app.use(urlencoded({ extended: true }));
app.useGlobalFilters(new HttpExceptionFilter());

View File

@@ -1,9 +1,4 @@
const FDI_TOOTH_IDS = new Set([
'11', '12', '13', '14', '15', '16', '17', '18',
'21', '22', '23', '24', '25', '26', '27', '28',
'31', '32', '33', '34', '35', '36', '37', '38',
'41', '42', '43', '44', '45', '46', '47', '48',
]);
import { FDI_TOOTH_IDS } from '../../common/fdi';
export function normalizeTeeth(teeth: unknown): string[] {
if (!Array.isArray(teeth)) {
@@ -35,7 +30,10 @@ export function normalizeToothSelectionGroups(
for (const row of value) {
if (!row || typeof row !== 'object') continue;
const rec = row as Record<string, unknown>;
const groupId = typeof rec.groupId === 'string' && rec.groupId.trim() ? rec.groupId.trim() : '';
const groupId =
typeof rec.groupId === 'string' && rec.groupId.trim()
? rec.groupId.trim()
: '';
if (!groupId) continue;
const kind = rec.kind === 'connected' ? 'connected' : 'single';
const teeth = normalizeTeeth(rec.teeth);
@@ -64,7 +62,8 @@ export function generateTreatmentTitle(
}
const parts = cases.map((c) => {
const label = c.treatmentType.charAt(0).toUpperCase() + c.treatmentType.slice(1);
const label =
c.treatmentType.charAt(0).toUpperCase() + c.treatmentType.slice(1);
if (c.teeth.length > 0) {
return `${label} ${c.teeth.join(', ')}`;
}

View File

@@ -0,0 +1,69 @@
import {
IsBase64,
IsIn,
IsInt,
IsString,
MaxLength,
Min,
} from 'class-validator';
/** Containers OpenRouter's transcription endpoint accepts, and MediaRecorder can produce. */
export const VOICE_AUDIO_FORMATS = [
'webm',
'mp4',
'm4a',
'aac',
'ogg',
'wav',
'mp3',
'flac',
] as const;
export type VoiceAudioFormat = (typeof VOICE_AUDIO_FORMATS)[number];
/** Locales the app ships; a profile still has to be configured for one to be usable. */
export const VOICE_LOCALES = ['en', 'fa', 'nl'] as const;
export class ExtractVoiceDto {
/**
* Base64 audio, no data: prefix. Capped well above a 2-minute opus clip (~400 KB) but
* far below OpenRouter's 25 MB ceiling, so an oversized upload is rejected before it
* costs a vendor call.
*/
@IsString()
@IsBase64()
@MaxLength(8_000_000)
audio: string;
@IsIn(VOICE_AUDIO_FORMATS)
format: VoiceAudioFormat;
/**
* The clinician's IANA zone. The server derives "today" from it rather than trusting a
* client-supplied date, which is what relative deadlines resolve against.
*/
@IsString()
@MaxLength(64)
timeZone: string;
/**
* Recording length as measured by the client.
*
* Required, not optional: an optional value means omitting it bypasses
* VOICE_MAX_RECORDING_MS entirely, which would make the cap advisory.
*/
@IsInt()
@Min(0)
durationMs: number;
/**
* The locale the clinician is actually speaking, as the UI offered the microphone.
*
* Sent explicitly rather than read from `user.language`: the two can diverge (a
* bookmarked /fa/ URL, a language toggle whose save failed), and a mismatch would
* transcribe Persian with an English hint and anchor "next Thursday" to the wrong
* week start. Gating the button and resolving the request must agree by construction.
*/
@IsIn(VOICE_LOCALES)
locale: string;
}

View File

@@ -0,0 +1,348 @@
import { resolveDueDate, weekStartForLocale } from './due-date.resolver';
import type { DueIntent } from './voice.types';
// 2025-10-11 is a Saturday — the first day of the Iranian week.
const FA_WEEK = 6; // Saturday
const EU_WEEK = 1; // Monday
const SATURDAY = '2025-10-11';
const THURSDAY = '2025-10-16';
describe('resolveDueDate', () => {
describe('weekday intents', () => {
it('resolves "this <weekday>" to the coming occurrence', () => {
const result = resolveDueDate(
{ kind: 'weekday', weekday: 'thursday', which: 'this' },
SATURDAY,
FA_WEEK,
);
expect(result.dueDate).toBe(THURSDAY); // Sat -> Thu is 5 days
});
it('resolves "next <weekday>" to that weekday in the following week', () => {
const result = resolveDueDate(
{ kind: 'weekday', weekday: 'thursday', which: 'next' },
SATURDAY,
FA_WEEK,
);
expect(result.dueDate).toBe('2025-10-23');
});
it('anchors "next" to the week, not to "this" plus seven', () => {
// Said on Thursday 2025-10-16, the Iranian week runs Sat 10-18 .. Fri 10-24, so its
// Thursday is 10-23. Adding a week to "this Thursday" (already 10-23) would
// overshoot to 10-30 — a lab case a week late.
expect(
resolveDueDate(
{ kind: 'weekday', weekday: 'thursday', which: 'next' },
THURSDAY,
FA_WEEK,
).dueDate,
).toBe('2025-10-23');
});
it('lets "this" and "next" coincide when they name the same day', () => {
// On a Thursday, "the coming Saturday" and "Saturday next week" are both 10-18.
for (const which of ['this', 'next'] as const) {
expect(
resolveDueDate(
{ kind: 'weekday', weekday: 'saturday', which },
THURSDAY,
FA_WEEK,
).dueDate,
).toBe('2025-10-18');
}
});
it('reads "by Thursday" said on a Thursday as the next one, not today', () => {
// A deadline of today is almost never what was meant.
expect(
resolveDueDate(
{ kind: 'weekday', weekday: 'thursday', which: 'this' },
THURSDAY,
FA_WEEK,
).dueDate,
).toBe('2025-10-23');
});
it('never resolves a weekday into the past', () => {
for (const which of ['this', 'next'] as const) {
const result = resolveDueDate(
{ kind: 'weekday', weekday: 'sunday', which },
THURSDAY,
FA_WEEK,
);
expect(result.dueDate).not.toBeNull();
expect(result.dueDate! > THURSDAY).toBe(true);
}
});
it('anchors "next" to a Monday week for en and nl', () => {
// The same sentence means a different day depending on where the week starts.
// Said on Saturday 10-11: the Monday-start week is 10-13..10-19, Thursday = 10-16.
// The Saturday-start week is 10-18..10-24, Thursday = 10-23.
const intent = {
kind: 'weekday',
weekday: 'thursday',
which: 'next',
} as const;
expect(resolveDueDate(intent, SATURDAY, EU_WEEK).dueDate).toBe(
'2025-10-16',
);
expect(resolveDueDate(intent, SATURDAY, FA_WEEK).dueDate).toBe(
'2025-10-23',
);
});
it('maps each locale to its week start', () => {
expect(weekStartForLocale('fa')).toBe(FA_WEEK);
expect(weekStartForLocale('en')).toBe(EU_WEEK);
expect(weekStartForLocale('nl')).toBe(EU_WEEK);
expect(weekStartForLocale('unknown')).toBe(EU_WEEK);
});
it('treats a missing qualifier as "this" rather than failing', () => {
// A bare weekday carries no qualifier; failing would discard a real spoken deadline.
expect(
resolveDueDate(
{
kind: 'weekday',
weekday: 'thursday',
which: null,
} as unknown as DueIntent,
SATURDAY,
FA_WEEK,
).dueDate,
).toBe(THURSDAY);
});
it('rejects an unknown weekday', () => {
expect(
resolveDueDate(
{
kind: 'weekday',
weekday: 'caturday',
which: 'this',
} as unknown as DueIntent,
SATURDAY,
FA_WEEK,
).dueDate,
).toBeNull();
});
});
describe('offset intents', () => {
it('adds days, weeks and months', () => {
expect(
resolveDueDate(
{ kind: 'offset', unit: 'day', amount: 1 },
SATURDAY,
FA_WEEK,
).dueDate,
).toBe('2025-10-12');
expect(
resolveDueDate(
{ kind: 'offset', unit: 'week', amount: 1 },
SATURDAY,
FA_WEEK,
).dueDate,
).toBe('2025-10-18');
expect(
resolveDueDate(
{ kind: 'offset', unit: 'month', amount: 1 },
SATURDAY,
FA_WEEK,
).dueDate,
).toBe('2025-11-11');
});
it('clamps to the end of a shorter month', () => {
expect(
resolveDueDate(
{ kind: 'offset', unit: 'month', amount: 1 },
'2025-01-31',
).dueDate,
).toBe('2025-02-28');
expect(
resolveDueDate(
{ kind: 'offset', unit: 'month', amount: 1 },
'2024-01-31',
).dueDate,
).toBe('2024-02-29');
});
it('crosses a year boundary', () => {
expect(
resolveDueDate(
{ kind: 'offset', unit: 'month', amount: 3 },
'2025-11-15',
).dueDate,
).toBe('2026-02-15');
expect(
resolveDueDate(
{ kind: 'offset', unit: 'day', amount: 30 },
'2025-12-20',
).dueDate,
).toBe('2026-01-19');
});
it('rejects negative, fractional and absurd amounts', () => {
for (const amount of [-1, 1.5, 10_000, Number.NaN]) {
expect(
resolveDueDate(
{ kind: 'offset', unit: 'day', amount },
SATURDAY,
FA_WEEK,
).dueDate,
).toBeNull();
}
});
});
describe('jalali intents', () => {
it('converts by arithmetic, not inference', () => {
expect(
resolveDueDate(
{ kind: 'jalali', jy: 1404, jm: 7, jd: 25 },
SATURDAY,
FA_WEEK,
).dueDate,
).toBe('2025-10-17');
});
it('handles the leap-year Esfand 30', () => {
expect(
resolveDueDate(
{ kind: 'jalali', jy: 1403, jm: 12, jd: 30 },
'2025-03-01',
).dueDate,
).toBe('2025-03-20');
});
it('rejects Esfand 30 in a non-leap year', () => {
const result = resolveDueDate(
{ kind: 'jalali', jy: 1404, jm: 12, jd: 30 },
SATURDAY,
);
expect(result.dueDate).toBeNull();
expect(result.unresolved?.reason).toBe('invalid_date');
});
});
describe('gregorian intents', () => {
it('accepts a real date and rejects an impossible one', () => {
expect(
resolveDueDate(
{ kind: 'gregorian', y: 2025, m: 10, d: 17 },
SATURDAY,
FA_WEEK,
).dueDate,
).toBe('2025-10-17');
expect(
resolveDueDate(
{ kind: 'gregorian', y: 2025, m: 2, d: 30 },
SATURDAY,
FA_WEEK,
).dueDate,
).toBeNull();
expect(
resolveDueDate(
{ kind: 'gregorian', y: 2025, m: 13, d: 1 },
SATURDAY,
FA_WEEK,
).dueDate,
).toBeNull();
});
});
describe('guard rails', () => {
it('treats a past date as unresolved', () => {
const result = resolveDueDate(
{ kind: 'gregorian', y: 2020, m: 1, d: 1 },
SATURDAY,
);
expect(result.dueDate).toBeNull();
expect(result.unresolved?.reason).toBe('invalid_date');
});
it('treats a date decades away as unresolved', () => {
expect(
resolveDueDate(
{ kind: 'gregorian', y: 2099, m: 1, d: 1 },
SATURDAY,
FA_WEEK,
).dueDate,
).toBeNull();
});
it('accepts today itself via a zero-day offset', () => {
expect(
resolveDueDate(
{ kind: 'offset', unit: 'day', amount: 0 },
SATURDAY,
FA_WEEK,
).dueDate,
).toBe(SATURDAY);
});
it('reports no due date at all when the model said nothing, without flagging it', () => {
expect(resolveDueDate(null, SATURDAY)).toEqual({
dueDate: null,
unresolved: null,
});
expect(resolveDueDate(undefined, SATURDAY)).toEqual({
dueDate: null,
unresolved: null,
});
});
it('treats an object with no kind as no deadline, not a lost one', () => {
// A blank "heard but lost" row in front of a clinician who never mentioned a
// deadline is worse than saying nothing.
expect(resolveDueDate({} as never, SATURDAY)).toEqual({
dueDate: null,
unresolved: null,
});
});
it('flags an unrecognised kind and names it', () => {
const result = resolveDueDate({ kind: 'lunar_month' } as never, SATURDAY);
expect(result.dueDate).toBeNull();
expect(result.unresolved).toEqual({
spoken: 'lunar_month',
reason: 'invalid_date',
});
});
it('flags a non-object deadline instead of silently dropping it', () => {
// A bare string is a deadline we failed to parse, not an absent one — the clinician
// must see that something was heard and lost.
for (const bad of ['next thursday', 42, true]) {
const result = resolveDueDate(bad as never, SATURDAY);
expect(result.dueDate).toBeNull();
expect(result.unresolved?.reason).toBe('invalid_date');
}
expect(
resolveDueDate('next thursday' as never, SATURDAY).unresolved?.spoken,
).toBe('next thursday');
});
it('degrades rather than throwing on a malformed today or intent', () => {
expect(
resolveDueDate({ kind: 'offset', unit: 'day', amount: 1 }, 'not-a-date')
.dueDate,
).toBeNull();
expect(
resolveDueDate({ kind: 'nope' } as unknown as DueIntent, SATURDAY)
.dueDate,
).toBeNull();
});
it('echoes what was heard so the review sheet can show it', () => {
const result = resolveDueDate(
{ kind: 'jalali', jy: 1404, jm: 12, jd: 30 },
SATURDAY,
);
expect(result.unresolved?.spoken).toBe('1404/12/30');
});
});
});

View File

@@ -0,0 +1,230 @@
import { jalaliToIsoDate } from '../../common/jalali';
import { civilDateJsWeekday } from '../../common/zoned-civil-time';
import type { DueIntent, UnresolvedItem, Weekday } from './voice.types';
/**
* Spoken deadline → ISO civil date.
*
* The model never does calendar arithmetic; it says what it heard and this decides what
* that means. Jalali conversion in particular is arithmetic here, not inference — an LLM
* asked to turn "۲۵ مهر" into ISO answers confidently and is often wrong, and
* `@IsDateString()` would accept the wrong answer.
*
* Works entirely in civil dates. The caller derives `todayIso` from the actor's IANA zone
* (see `civilDateInZone`) rather than passing a zone in here, so nothing in this file has
* to reason about instants.
*/
/** JS `getUTCDay()` numbering: Sunday = 0. */
const WEEKDAY_TO_JS: Record<Weekday, number> = {
saturday: 6,
sunday: 0,
monday: 1,
tuesday: 2,
wednesday: 3,
thursday: 4,
friday: 5,
};
/** Refuse absurd deadlines however they were arrived at. */
const MAX_DAYS_AHEAD = 365 * 5;
export type DueResolution = {
dueDate: string | null;
unresolved: UnresolvedItem | null;
};
const ISO_DATE = /^\d{4}-\d{2}-\d{2}$/;
function isRealCivilDate(iso: string): boolean {
if (!ISO_DATE.test(iso)) return false;
const [y, m, d] = iso.split('-').map(Number);
if (m < 1 || m > 12 || d < 1 || d > 31) return false;
const probe = new Date(Date.UTC(y, m - 1, d));
return (
probe.getUTCFullYear() === y &&
probe.getUTCMonth() === m - 1 &&
probe.getUTCDate() === d
);
}
function toIso(utcMs: number): string {
return new Date(utcMs).toISOString().slice(0, 10);
}
function utcMsOf(iso: string): number {
const [y, m, d] = iso.split('-').map(Number);
return Date.UTC(y, m - 1, d);
}
function addDays(iso: string, days: number): string {
return toIso(utcMsOf(iso) + days * 86_400_000);
}
/** Calendar-month addition with end-of-month clamping (31 Jan + 1 month = 28/29 Feb). */
function addMonths(iso: string, months: number): string {
const [y, m, d] = iso.split('-').map(Number);
const targetMonthIndex = m - 1 + months;
const targetYear = y + Math.floor(targetMonthIndex / 12);
const targetMonth = ((targetMonthIndex % 12) + 12) % 12;
const lastDay = new Date(
Date.UTC(targetYear, targetMonth + 1, 0),
).getUTCDate();
return toIso(Date.UTC(targetYear, targetMonth, Math.min(d, lastDay)));
}
function unresolved(spoken: string): DueResolution {
return { dueDate: null, unresolved: { spoken, reason: 'invalid_date' } };
}
function describe(intent: DueIntent): string {
switch (intent?.kind) {
case 'weekday':
return `${intent.which} ${intent.weekday}`;
case 'offset':
return `+${intent.amount} ${intent.unit}`;
case 'jalali':
return `${intent.jy}/${intent.jm}/${intent.jd}`;
case 'gregorian':
return `${intent.y}-${intent.m}-${intent.d}`;
default: {
// Reaching here means an unrecognised `kind`, which resolveDueDate has already
// established is a string — echo it so the review row names what was heard.
const kind = (intent as { kind?: unknown })?.kind;
return typeof kind === 'string' ? kind : '';
}
}
}
/**
* Which weekday starts the week, per locale.
*
* "Next Thursday" is week-relative, so this changes the answer: the Iranian week starts
* Saturday, the Dutch and (European) English week starts Monday. Hardcoding Saturday
* would put an en/nl clinician's deadline a week out.
*/
const WEEK_START_BY_LOCALE: Record<string, number> = {
fa: WEEKDAY_TO_JS.saturday,
en: WEEKDAY_TO_JS.monday,
nl: WEEKDAY_TO_JS.monday,
};
const DEFAULT_WEEK_START = WEEKDAY_TO_JS.monday;
export function weekStartForLocale(locale: string): number {
return WEEK_START_BY_LOCALE[locale] ?? DEFAULT_WEEK_START;
}
/** Most recent week-start day, counting today if today is that day. */
function startOfWeek(iso: string, weekStartJs: number): string {
const back = (civilDateJsWeekday(iso) - weekStartJs + 7) % 7;
return addDays(iso, -back);
}
/**
* `'this'` is occurrence-anchored: the soonest occurrence strictly after today, so "by
* Thursday" said on a Thursday means the next one — a deadline of today is almost never
* what was meant, and this can never resolve into the past.
*
* `'next'` is *week*-anchored, not "this plus seven". "Thursday next week" means the
* Thursday of the Saturday-start week after this one; adding a week to `'this'` would
* overshoot by seven days whenever `'this'` had already rolled into next week. The two
* can legitimately coincide — said on a Thursday, "the coming Saturday" and "Saturday
* next week" are the same day.
*/
function resolveWeekday(
intent: Extract<DueIntent, { kind: 'weekday' }>,
todayIso: string,
weekStartJs: number,
) {
const targetJs = WEEKDAY_TO_JS[intent.weekday];
if (targetJs === undefined) return null;
// A bare weekday ("پنجشنبه") carries no qualifier, and the model may leave `which`
// null. Treat that as 'this' rather than failing an utterance that named a real day.
const which = intent.which === 'next' ? 'next' : 'this';
if (which === 'this') {
const todayJs = civilDateJsWeekday(todayIso);
let delta = (targetJs - todayJs + 7) % 7;
if (delta === 0) delta = 7;
return addDays(todayIso, delta);
}
if (which === 'next') {
const offsetInWeek = (targetJs - weekStartJs + 7) % 7;
return addDays(startOfWeek(todayIso, weekStartJs), 7 + offsetInWeek);
}
return null;
}
function resolveOffset(
intent: Extract<DueIntent, { kind: 'offset' }>,
todayIso: string,
) {
const { amount, unit } = intent;
if (!Number.isInteger(amount) || amount < 0) return null;
if (unit === 'day')
return amount <= MAX_DAYS_AHEAD ? addDays(todayIso, amount) : null;
if (unit === 'week')
return amount <= 260 ? addDays(todayIso, amount * 7) : null;
if (unit === 'month')
return amount <= 60 ? addMonths(todayIso, amount) : null;
return null;
}
export function resolveDueDate(
intent: DueIntent | null | undefined,
todayIso: string,
weekStartJs: number = DEFAULT_WEEK_START,
): DueResolution {
// Absent is not an error — most utterances carry no deadline. Anything else that is not
// an intent object is a deadline we failed to understand, and must be flagged rather
// than silently dropped.
if (intent === null || intent === undefined) {
return { dueDate: null, unresolved: null };
}
if (typeof intent !== 'object') {
return unresolved(String(intent).slice(0, 120));
}
// An object carrying no `kind` at all says nothing about a deadline; flagging it would
// put a blank "heard but lost" row in front of a clinician who never mentioned one. An
// object with an *unrecognised* kind did try to say something, and is flagged below.
if (typeof (intent as { kind?: unknown }).kind !== 'string') {
return { dueDate: null, unresolved: null };
}
if (!isRealCivilDate(todayIso)) {
return unresolved(describe(intent));
}
let resolved: string | null = null;
switch (intent.kind) {
case 'weekday':
resolved = resolveWeekday(intent, todayIso, weekStartJs);
break;
case 'offset':
resolved = resolveOffset(intent, todayIso);
break;
case 'jalali':
resolved = jalaliToIsoDate(intent.jy, intent.jm, intent.jd);
break;
case 'gregorian': {
const candidate = `${String(intent.y).padStart(4, '0')}-${String(intent.m).padStart(2, '0')}-${String(intent.d).padStart(2, '0')}`;
resolved = isRealCivilDate(candidate) ? candidate : null;
break;
}
default:
resolved = null;
}
if (!resolved) return unresolved(describe(intent));
// An absolute date the model invented can land anywhere; a deadline in the past or
// decades away is not a deadline.
const daysAhead = (utcMsOf(resolved) - utcMsOf(todayIso)) / 86_400_000;
if (daysAhead < 0 || daysAhead > MAX_DAYS_AHEAD)
return unresolved(describe(intent));
return { dueDate: resolved, unresolved: null };
}

View File

@@ -0,0 +1,82 @@
import type { ExtractionCatalog } from './voice.providers';
/** Locale-specific guidance. Only the tooth vocabulary and numbering habits differ. */
const LOCALE_NOTES: Record<string, string> = {
fa: [
'The clinician is speaking Persian. Tooth references are usually quadrant-relative:',
'"شش بالا راست" = upper right six -> arch "upper", side "patient_right", position 6.',
'Digits may appear in Persian or Latin script. Two-digit FDI notation ("یک چهار") does',
'occur — use the "fdi" field only for that.',
'A bare "دندون دو" carries no quadrant: report position 2 with arch and side null.',
].join(' '),
nl: [
'The clinician is speaking Dutch and uses FDI notation, which is standard in the',
'Netherlands. "rechtsboven zes" = upper right six. A bare two-digit number is FDI.',
].join(' '),
en: [
'The clinician is speaking English. IMPORTANT: a bare two-digit number is ambiguous,',
'because Universal numbering and FDI disagree ("tooth 14" is a different tooth in each).',
'Set "fdi" ONLY when the speaker made the notation explicit (e.g. "FDI one four").',
'Otherwise describe the tooth with arch/side/position, or leave it unresolved.',
].join(' '),
};
function codeList(entries: { code: string; label: string }[]): string {
if (entries.length === 0) return '(none available)';
return entries.map((e) => `- ${e.code} = ${e.label}`).join('\n');
}
export function buildExtractionPrompt(
transcript: string,
catalog: ExtractionCatalog,
localeHint: string,
) {
const localeNote = LOCALE_NOTES[localeHint] ?? LOCALE_NOTES.en;
const system = [
'You extract structured dental treatment data from a transcript of a clinician speaking.',
'You are a parser, not an assistant: report only what was said.',
'',
'HARD RULES',
'1. Never invent a code. treatmentType, prosthesisDefaultType and prosthesisOverrides[].type',
' must be codes from the lists below. labId must be an id from the lab list. If what you',
' heard is not in a list, use null.',
'2. Never output an FDI tooth code unless the speaker used FDI notation. Prefer',
' arch + side + position.',
'3. "side" is always the PATIENT\'s side. The patient\'s upper right is quadrant 1. Never',
" flip to the viewer's point of view.",
'4. Never do calendar arithmetic. Report the deadline as it was said, using due.kind.',
' If no deadline was mentioned, use due.kind = "none".',
'5. Copy the exact spoken words for each tooth into "spoken", so the clinician can see',
' what was heard.',
'6. If you are unsure about a value, use null. A missing field is recoverable; a wrong',
' one is not.',
'7. A tooth number spoken WITHOUT a quadrant ("دندون دو", "tooth two") does not identify',
' a tooth — four teeth carry that position. Still report it: set "position" and leave',
' "arch" and "side" null. Never pick a quadrant that was not said.',
'',
localeNote,
'',
'TREATMENT TYPE CODES',
codeList(catalog.treatmentTypes),
'',
'PROSTHESIS TYPE CODES',
codeList(catalog.prosthesisTypes),
'',
'LABS THIS CLINIC CAN SEND TO',
catalog.labs.length > 0
? catalog.labs.map((l) => `- ${l.id} = ${l.name}`).join('\n')
: '(none linked — labId must be null)',
'',
'OTHER FIELDS',
'- connectedSpans: only for bridges or splinted units. Endpoints inclusive.',
'- comment: clinical notes, in the language spoken. Omit the parts already captured as',
' treatment type, teeth or deadline.',
'- labMatchExact: true only when the spoken name matched a lab name exactly.',
].join('\n');
return [
{ role: 'system' as const, content: system },
{ role: 'user' as const, content: transcript },
];
}

View File

@@ -0,0 +1,356 @@
import {
resolveConnectedSpans,
resolveProsthesis,
resolveVoiceIntent,
type ResolveContext,
} from './extraction.resolver';
import type { ToothIntent, VoiceIntent } from './voice.types';
const tooth = (fdi: string, spoken = fdi): ToothIntent => ({
kind: 'explicit',
fdi,
spoken,
});
const CTX: ResolveContext = {
todayIso: '2025-10-11',
weekStartJs: 6, // Saturday — the fa week
treatmentTypeCodes: new Set(['restoration', 'prosthesis', 'extraction']),
prosthesisTypeCodes: new Set(['monolithic_zirconia', 'pfm_crown']),
linkedLabIds: new Set(['lab-sina', 'lab-mehr']),
};
describe('resolveConnectedSpans', () => {
it('selects the teeth between the endpoints, which were never named', () => {
// "a bridge from 14 to 16" must select 15 too.
const result = resolveConnectedSpans(
[{ from: tooth('14'), to: tooth('16') }],
[],
);
expect(result.teeth).toEqual(['14', '15', '16']);
expect(result.groups).toEqual([
{ groupId: 'voice-c1', kind: 'connected', teeth: ['16', '15', '14'] },
]);
});
it('orders group teeth along the arch, not lexically', () => {
const result = resolveConnectedSpans(
[{ from: tooth('16'), to: tooth('14') }],
[],
);
expect(result.groups[0].teeth).toEqual(['16', '15', '14']);
});
it('spans the midline', () => {
const result = resolveConnectedSpans(
[{ from: tooth('12'), to: tooth('22') }],
[],
);
expect(result.groups[0].teeth).toEqual(['12', '11', '21', '22']);
});
it('merges overlapping spans into one bridge', () => {
const result = resolveConnectedSpans(
[
{ from: tooth('14'), to: tooth('16') },
{ from: tooth('15'), to: tooth('17') },
],
[],
);
const connected = result.groups.filter((g) => g.kind === 'connected');
expect(connected).toHaveLength(1);
expect(connected[0].teeth).toEqual(['17', '16', '15', '14']);
});
it('gives loose teeth their own single groups', () => {
const result = resolveConnectedSpans(
[{ from: tooth('14'), to: tooth('15') }],
['26'],
);
expect(result.groups).toEqual([
{ groupId: 'voice-c1', kind: 'connected', teeth: ['15', '14'] },
{ groupId: 'voice-s-26', kind: 'single', teeth: ['26'] },
]);
});
it('never produces a one-tooth connected group', () => {
const result = resolveConnectedSpans(
[{ from: tooth('14'), to: tooth('14') }],
[],
);
expect(result.groups).toEqual([
{ groupId: 'voice-s-14', kind: 'single', teeth: ['14'] },
]);
expect(result.unresolved).toEqual([]);
});
it('reports a cross-arch span rather than guessing', () => {
const result = resolveConnectedSpans(
[{ from: tooth('14', 'چهارده'), to: tooth('44', 'چهل و چهار') }],
[],
);
expect(result.groups).toEqual([]);
expect(result.unresolved).toEqual([
{ spoken: 'چهارده → چهل و چهار', reason: 'span_not_same_arch' },
]);
});
it('reports a span with an unresolvable endpoint', () => {
const result = resolveConnectedSpans(
[{ from: tooth('14'), to: tooth('99') }],
[],
);
expect(result.unresolved[0].reason).toBe('malformed');
});
it('survives a non-array', () => {
expect(resolveConnectedSpans(undefined as never, ['14']).teeth).toEqual([
'14',
]);
});
});
describe('resolveProsthesis', () => {
const allowed = CTX.prosthesisTypeCodes;
it('expands the default across every tooth', () => {
const result = resolveProsthesis(
{ defaultType: 'monolithic_zirconia', overrides: [] },
['14', '15'],
allowed,
);
expect(result.prosthesis?.byTooth).toEqual({
'14': 'monolithic_zirconia',
'15': 'monolithic_zirconia',
});
expect(result.prosthesis?.complete).toBe(true);
});
it('applies per-tooth overrides on top of the default', () => {
const result = resolveProsthesis(
{
defaultType: 'monolithic_zirconia',
overrides: [{ tooth: tooth('26'), type: 'pfm_crown' }],
},
['14', '26'],
allowed,
);
expect(result.prosthesis?.byTooth).toEqual({
'14': 'monolithic_zirconia',
'26': 'pfm_crown',
});
expect(result.prosthesis?.complete).toBe(true);
});
it('marks the map incomplete when a tooth ends up untyped', () => {
// Unshippable: assertCompleteToothProsthesisMap would reject this at dispatch.
const result = resolveProsthesis(
{
defaultType: null,
overrides: [{ tooth: tooth('14'), type: 'pfm_crown' }],
},
['14', '15'],
allowed,
);
expect(result.prosthesis?.complete).toBe(false);
expect(result.prosthesis?.missingTeeth).toEqual(['15']);
});
it('rejects a catalog code the clinic does not have', () => {
const result = resolveProsthesis(
{ defaultType: 'gold_foil', overrides: [] },
['14'],
allowed,
);
// Nothing usable was said, so there is no prosthesis to show — not an empty one.
expect(result.prosthesis).toBeNull();
expect(result.unresolved).toEqual([
{ spoken: 'gold_foil', reason: 'unknown_catalog_code' },
]);
});
it('ignores an override for a tooth that is not selected', () => {
const result = resolveProsthesis(
{
defaultType: 'monolithic_zirconia',
overrides: [{ tooth: tooth('37', 'سی و هفت'), type: 'pfm_crown' }],
},
['14'],
allowed,
);
expect(result.prosthesis?.byTooth).toEqual({ '14': 'monolithic_zirconia' });
expect(result.unresolved).toEqual([
{ spoken: 'سی و هفت', reason: 'tooth_not_selected' },
]);
});
it('reports no prosthesis at all when the object carries nothing usable', () => {
// An empty-but-present map would paint a plain restoration with a fabricated
// "incomplete, cannot ship" warning.
for (const empty of [{ defaultType: null, overrides: [] }, {} as never]) {
expect(
resolveProsthesis(empty, ['14', '15'], allowed).prosthesis,
).toBeNull();
}
});
it('reports no prosthesis when there are no teeth to type', () => {
const result = resolveProsthesis(
{ defaultType: 'monolithic_zirconia', overrides: [] },
[],
allowed,
);
expect(result.prosthesis).toBeNull();
});
it('distinguishes a tooth it could not understand from one that is not selected', () => {
// Different corrective actions: add the tooth, versus repeat yourself.
const result = resolveProsthesis(
{
defaultType: 'monolithic_zirconia',
overrides: [
{
tooth: { kind: 'explicit', fdi: '99', spoken: 'نود و نه' },
type: 'pfm_crown',
},
],
},
['14'],
allowed,
);
expect(result.unresolved).toEqual([
{ spoken: 'نود و نه', reason: 'malformed' },
]);
});
it('returns null when no prosthesis was spoken', () => {
expect(resolveProsthesis(null, ['14'], allowed).prosthesis).toBeNull();
});
});
describe('resolveVoiceIntent', () => {
const base: VoiceIntent = {
treatmentType: 'restoration',
teeth: [tooth('14'), tooth('15')],
connectedSpans: [],
comment: ' حساسیت به سرما ',
prosthesis: null,
labId: null,
labMatchExact: false,
due: null,
};
it('composes a plain restoration', () => {
const result = resolveVoiceIntent(base, CTX);
expect(result.treatmentType).toBe('restoration');
expect(result.teeth).toEqual(['14', '15']);
expect(result.comment).toBe('حساسیت به سرما');
expect(result.prosthesis).toBeNull();
expect(result.unresolved).toEqual([]);
});
it('rejects a treatment type outside the catalog', () => {
const result = resolveVoiceIntent(
{ ...base, treatmentType: 'teeth_whitening' },
CTX,
);
expect(result.treatmentType).toBeNull();
expect(result.unresolved).toContainEqual({
spoken: 'teeth_whitening',
reason: 'unknown_catalog_code',
});
});
it('drops a lab id the clinic is not linked to', () => {
// Shipping to a lab the clinic never named is worse than shipping nowhere.
const result = resolveVoiceIntent(
{ ...base, labId: 'lab-elsewhere', labMatchExact: true },
CTX,
);
expect(result.labId).toBeNull();
expect(result.labMatchExact).toBe(false);
});
it('keeps a linked lab and its exactness flag', () => {
const result = resolveVoiceIntent(
{ ...base, labId: 'lab-sina', labMatchExact: true },
CTX,
);
expect(result.labId).toBe('lab-sina');
expect(result.labMatchExact).toBe(true);
});
it('reports a hallucinated lab rather than dropping it silently', () => {
// A near-miss lab id must not look identical to "no lab was spoken".
const result = resolveVoiceIntent({ ...base, labId: 'lab-elsewhere' }, CTX);
// The id is not what the clinician said — quoting it back shows them a raw UUID.
expect(result.unresolved).toContainEqual({
spoken: '',
reason: 'unknown_catalog_code',
});
});
it('never reports an inexact match as exact when the lab was dropped', () => {
const result = resolveVoiceIntent(
{ ...base, labId: null, labMatchExact: true },
CTX,
);
expect(result.labMatchExact).toBe(false);
});
it('applies prosthesis over the span-expanded tooth set', () => {
const result = resolveVoiceIntent(
{
...base,
treatmentType: 'prosthesis',
teeth: [tooth('14')],
connectedSpans: [{ from: tooth('14'), to: tooth('16') }],
prosthesis: { defaultType: 'monolithic_zirconia', overrides: [] },
},
CTX,
);
// 15 was never spoken but is part of the bridge, so it must carry a type too.
expect(result.teeth).toEqual(['14', '15', '16']);
expect(result.prosthesis?.complete).toBe(true);
expect(Object.keys(result.prosthesis!.byTooth).sort()).toEqual([
'14',
'15',
'16',
]);
});
it('resolves a due date through the same context', () => {
const result = resolveVoiceIntent(
{ ...base, due: { kind: 'weekday', weekday: 'thursday', which: 'this' } },
CTX,
);
expect(result.dueDate).toBe('2025-10-16');
});
it('collects unresolved items from every stage', () => {
const result = resolveVoiceIntent(
{
...base,
treatmentType: 'nope',
teeth: [tooth('51', 'شیری')],
connectedSpans: [{ from: tooth('14'), to: tooth('44') }],
due: { kind: 'jalali', jy: 1404, jm: 12, jd: 30 },
},
CTX,
);
const reasons = result.unresolved.map((u) => u.reason).sort();
expect(reasons).toEqual([
'invalid_date',
'not_permanent_tooth',
'span_not_same_arch',
'unknown_catalog_code',
]);
});
it('treats an empty comment as absent', () => {
expect(
resolveVoiceIntent({ ...base, comment: ' ' }, CTX).comment,
).toBeNull();
});
});

View File

@@ -0,0 +1,307 @@
import {
sameArch,
sortInArchOrder,
teethBetweenInclusive,
} from '../../common/fdi';
import { resolveDueDate } from './due-date.resolver';
import {
resolveToothIntent,
resolveToothIntents,
} from './tooth-intent.resolver';
import type {
ConnectedSpanIntent,
ProsthesisIntent,
ToothIntent,
UnresolvedItem,
VoiceIntent,
} from './voice.types';
export type ResolvedToothGroup = {
groupId: string;
kind: 'connected' | 'single';
teeth: string[];
};
export type ResolvedProsthesis = {
/** FDI code → prosthesis type code. */
byTooth: Record<string, string>;
/**
* True when every selected tooth carries a code. A prosthesis detail cannot be shipped
* otherwise (`assertCompleteToothProsthesisMap`), so the review sheet surfaces the gap
* here rather than letting it fail at dispatch.
*/
complete: boolean;
missingTeeth: string[];
};
export type ResolvedExtraction = {
treatmentType: string | null;
teeth: string[];
toothSelectionGroups: ResolvedToothGroup[];
comment: string | null;
prosthesis: ResolvedProsthesis | null;
labId: string | null;
labMatchExact: boolean;
dueDate: string | null;
unresolved: UnresolvedItem[];
};
export type ResolveContext = {
todayIso: string;
/** JS weekday index the clinician's week starts on — see weekStartForLocale. */
weekStartJs: number;
treatmentTypeCodes: ReadonlySet<string>;
prosthesisTypeCodes: ReadonlySet<string>;
linkedLabIds: ReadonlySet<string>;
};
function spokenOf(intent: ToothIntent): string {
const spoken = (intent as { spoken?: unknown })?.spoken;
return typeof spoken === 'string' && spoken.trim() ? spoken.trim() : '';
}
/** A code the model returned is only usable if it exists in the catalog we supplied it. */
function resolveCatalogCode(
value: unknown,
allowed: ReadonlySet<string>,
): string | null {
if (typeof value !== 'string') return null;
const trimmed = value.trim();
return trimmed && allowed.has(trimmed) ? trimmed : null;
}
/** Merge any span sets that share a tooth, so overlapping bridges become one group. */
function mergeOverlapping(sets: string[][]): string[][] {
const merged: string[][] = [];
for (const candidate of sets) {
let current = [...candidate];
let index = 0;
while (index < merged.length) {
if (merged[index].some((tooth) => current.includes(tooth))) {
current = [...new Set([...merged[index], ...current])];
merged.splice(index, 1);
index = 0;
continue;
}
index += 1;
}
merged.push(current);
}
return merged;
}
/**
* Turn spoken bridge spans plus loose teeth into selection groups.
*
* Span teeth are added to the selection: saying "a bridge from 14 to 16" selects 15 even
* though it was never named. A span whose endpoints are in different arches is impossible
* and is reported rather than guessed at. A span that collapses to one tooth degrades to a
* single — there is no such thing as a one-tooth bridge.
*/
export function resolveConnectedSpans(
spans: readonly ConnectedSpanIntent[],
selectedTeeth: readonly string[],
): {
groups: ResolvedToothGroup[];
teeth: string[];
unresolved: UnresolvedItem[];
} {
const unresolved: UnresolvedItem[] = [];
const connectedSets: string[][] = [];
// A span that collapses to one tooth still selected that tooth — it must not vanish.
const loneSpanTeeth: string[] = [];
const list: readonly ConnectedSpanIntent[] = Array.isArray(spans)
? (spans as readonly ConnectedSpanIntent[])
: [];
for (const span of list) {
const from = resolveToothIntent(span?.from);
const to = resolveToothIntent(span?.to);
const spoken = [spokenOf(span?.from), spokenOf(span?.to)]
.filter(Boolean)
.join(' → ');
if (!from || !to) {
unresolved.push({ spoken, reason: 'malformed' });
continue;
}
if (!sameArch(from, to)) {
unresolved.push({ spoken, reason: 'span_not_same_arch' });
continue;
}
const between = teethBetweenInclusive(from, to);
if (!between || between.length === 0) {
unresolved.push({ spoken, reason: 'malformed' });
continue;
}
if (between.length === 1) {
loneSpanTeeth.push(between[0]);
continue; // degrades to a single, below
}
connectedSets.push(between);
}
const groups: ResolvedToothGroup[] = [];
const claimed = new Set<string>();
mergeOverlapping(connectedSets).forEach((set, i) => {
const teeth = sortInArchOrder(set);
teeth.forEach((tooth) => claimed.add(tooth));
groups.push({ groupId: `voice-c${i + 1}`, kind: 'connected', teeth });
});
const spanTeeth = [...groups.flatMap((g) => g.teeth), ...loneSpanTeeth];
const singles = [...new Set([...selectedTeeth, ...spanTeeth])]
.filter((tooth) => !claimed.has(tooth))
.sort();
for (const tooth of singles) {
groups.push({
groupId: `voice-s-${tooth}`,
kind: 'single',
teeth: [tooth],
});
}
return {
groups,
teeth: [...new Set([...selectedTeeth, ...spanTeeth])].sort(),
unresolved,
};
}
/**
* Expand a default prosthesis type across the selection, then apply per-tooth overrides.
*
* "همه زیرکونیا، ۲۶ پی‌اف‌ام" is how clinicians actually speak, so the model names the type
* once and overrides the exceptions.
*/
export function resolveProsthesis(
intent: ProsthesisIntent | null | undefined,
teeth: readonly string[],
allowed: ReadonlySet<string>,
): { prosthesis: ResolvedProsthesis | null; unresolved: UnresolvedItem[] } {
if (!intent || typeof intent !== 'object')
return { prosthesis: null, unresolved: [] };
const unresolved: UnresolvedItem[] = [];
const defaultType = resolveCatalogCode(intent.defaultType, allowed);
if (intent.defaultType != null && !defaultType) {
unresolved.push({
spoken: String(intent.defaultType),
reason: 'unknown_catalog_code',
});
}
const byTooth: Record<string, string> = {};
const selected = new Set(teeth);
if (defaultType) {
for (const tooth of teeth) byTooth[tooth] = defaultType;
}
const overrides: ProsthesisIntent['overrides'] = Array.isArray(
intent.overrides,
)
? intent.overrides
: [];
for (const override of overrides) {
const tooth = resolveToothIntent(override?.tooth);
const type = resolveCatalogCode(override?.type, allowed);
const spoken = spokenOf(override?.tooth) || String(override?.type ?? '');
if (!tooth) {
unresolved.push({ spoken, reason: 'malformed' });
continue;
}
// A tooth we understood perfectly well but which is not part of this detail. Saying
// so is actionable ("add tooth 37, or drop it"); calling it malformed is not.
if (!selected.has(tooth)) {
unresolved.push({ spoken, reason: 'tooth_not_selected' });
continue;
}
if (!type) {
unresolved.push({ spoken, reason: 'unknown_catalog_code' });
continue;
}
byTooth[tooth] = type;
}
// Nothing usable was said about prosthesis. Returning an empty-but-present map would
// paint a plain restoration with a fabricated "incomplete, cannot ship" warning.
if (Object.keys(byTooth).length === 0) {
return { prosthesis: null, unresolved };
}
const missingTeeth = teeth.filter((tooth) => !byTooth[tooth]);
return {
prosthesis: {
byTooth,
complete: missingTeeth.length === 0,
missingTeeth,
},
unresolved,
};
}
/** Compose every resolver into the payload the review sheet renders. */
export function resolveVoiceIntent(
intent: VoiceIntent,
ctx: ResolveContext,
): ResolvedExtraction {
const unresolved: UnresolvedItem[] = [];
const toothResult = resolveToothIntents(intent?.teeth ?? []);
unresolved.push(...toothResult.unresolved);
const spanResult = resolveConnectedSpans(
intent?.connectedSpans ?? [],
toothResult.teeth,
);
unresolved.push(...spanResult.unresolved);
const treatmentType = resolveCatalogCode(
intent?.treatmentType,
ctx.treatmentTypeCodes,
);
if (intent?.treatmentType != null && !treatmentType) {
unresolved.push({
spoken: String(intent.treatmentType),
reason: 'unknown_catalog_code',
});
}
const prosthesisResult = resolveProsthesis(
intent?.prosthesis,
spanResult.teeth,
ctx.prosthesisTypeCodes,
);
unresolved.push(...prosthesisResult.unresolved);
const due = resolveDueDate(intent?.due, ctx.todayIso, ctx.weekStartJs);
if (due.unresolved) unresolved.push(due.unresolved);
const comment =
typeof intent?.comment === 'string' && intent.comment.trim()
? intent.comment.trim()
: null;
// A lab id the model invented is worse than none — it would ship a case to a lab the
// clinic never named. Only ids from the list we supplied survive, and a rejected one is
// reported: a hallucinated lab must not look identical to "no lab was spoken".
const labId = resolveCatalogCode(intent?.labId, ctx.linkedLabIds);
if (intent?.labId != null && !labId) {
// `spoken` means "what the clinician said". A rejected lab id is an opaque
// identifier the model invented, so quoting it back would put a raw UUID in front
// of the user; the reason alone carries the meaning.
unresolved.push({ spoken: '', reason: 'unknown_catalog_code' });
}
return {
treatmentType,
teeth: spanResult.teeth,
toothSelectionGroups: spanResult.groups,
comment,
prosthesis: prosthesisResult.prosthesis,
labId,
labMatchExact: labId ? intent?.labMatchExact === true : false,
dueDate: due.dueDate,
unresolved,
};
}

View File

@@ -0,0 +1,236 @@
import {
toVoiceIntent,
VOICE_INTENT_JSON_SCHEMA,
type WireVoiceIntent,
} from './extraction.wire';
const emptyDue = {
kind: 'none' as const,
weekday: null,
which: null,
unit: null,
amount: null,
jy: null,
jm: null,
jd: null,
y: null,
m: null,
d: null,
};
const wire = (overrides: Partial<WireVoiceIntent> = {}): WireVoiceIntent => ({
treatmentType: null,
teeth: [],
connectedSpans: [],
comment: null,
prosthesisDefaultType: null,
prosthesisOverrides: [],
labId: null,
labMatchExact: false,
due: emptyDue,
...overrides,
});
const positionalTooth = {
spoken: 'شش بالا راست',
fdi: null,
arch: 'upper' as const,
side: 'patient_right' as const,
position: 6,
};
describe('VOICE_INTENT_JSON_SCHEMA', () => {
it('is strict — every property required, no extras', () => {
expect(VOICE_INTENT_JSON_SCHEMA.additionalProperties).toBe(false);
expect([...VOICE_INTENT_JSON_SCHEMA.required].sort()).toEqual(
Object.keys(VOICE_INTENT_JSON_SCHEMA.properties).sort(),
);
});
it('requires every field of the due object, since strict mode allows no optionals', () => {
const due = VOICE_INTENT_JSON_SCHEMA.properties.due;
expect([...due.required].sort()).toEqual(
Object.keys(due.properties).sort(),
);
});
});
describe('toVoiceIntent', () => {
it('takes the explicit branch for a code the model wrote in Persian digits', () => {
// The model is reading Persian text back, so "۲۶" and a digit-by-digit "2 6" both
// reach us. Matching only ASCII drops the tooth into the positional branch with no
// quadrant, where it is reported as unresolved — the clinician loses a tooth and is
// told the words were the problem.
for (const raw of ['\u06F2\u06F6', '2 6', ' 26 ', '\u0662\u0666']) {
const [tooth] = toVoiceIntent(
wire({
teeth: [
{
spoken: '\u0628\u06CC\u0633\u062A \u0648 \u0634\u0634',
fdi: raw,
arch: null,
side: null,
position: null,
},
],
}),
).teeth;
expect(tooth).toEqual({
kind: 'explicit',
fdi: '26',
spoken: '\u0628\u06CC\u0633\u062A \u0648 \u0634\u0634',
});
}
});
it('narrows a positional tooth', () => {
const result = toVoiceIntent(wire({ teeth: [positionalTooth] }));
expect(result.teeth[0]).toEqual({
kind: 'positional',
arch: 'upper',
side: 'patient_right',
position: 6,
spoken: 'شش بالا راست',
});
});
it('narrows an explicit FDI tooth, which wins over positional fields', () => {
const result = toVoiceIntent(
wire({ teeth: [{ ...positionalTooth, fdi: '14', spoken: 'یک چهار' }] }),
);
expect(result.teeth[0]).toEqual({
kind: 'explicit',
fdi: '14',
spoken: 'یک چهار',
});
});
it('maps due.kind "none" to no deadline', () => {
expect(toVoiceIntent(wire()).due).toBeNull();
});
it('narrows each due kind', () => {
expect(
toVoiceIntent(
wire({
due: {
...emptyDue,
kind: 'weekday',
weekday: 'thursday',
which: 'next',
},
}),
).due,
).toEqual({ kind: 'weekday', weekday: 'thursday', which: 'next' });
expect(
toVoiceIntent(
wire({ due: { ...emptyDue, kind: 'offset', unit: 'week', amount: 2 } }),
).due,
).toEqual({ kind: 'offset', unit: 'week', amount: 2 });
expect(
toVoiceIntent(
wire({ due: { ...emptyDue, kind: 'jalali', jy: 1404, jm: 7, jd: 25 } }),
).due,
).toEqual({ kind: 'jalali', jy: 1404, jm: 7, jd: 25 });
expect(
toVoiceIntent(
wire({
due: { ...emptyDue, kind: 'gregorian', y: 2025, m: 10, d: 17 },
}),
).due,
).toEqual({ kind: 'gregorian', y: 2025, m: 10, d: 17 });
});
it('ignores a non-FDI-shaped fdi and keeps the positional fields', () => {
// A model emitting fdi:"6" alongside correct arch/side/position must still yield 16,
// not lose the tooth to the explicit branch.
const result = toVoiceIntent(
wire({ teeth: [{ ...positionalTooth, fdi: '6' }] }),
);
expect(result.teeth[0]).toEqual({
kind: 'positional',
arch: 'upper',
side: 'patient_right',
position: 6,
spoken: 'شش بالا راست',
});
});
it('rejects impossible FDI shapes from the explicit branch', () => {
for (const fdi of ['99', '0', '140', '9', 'ab']) {
expect(
toVoiceIntent(wire({ teeth: [{ ...positionalTooth, fdi }] })).teeth[0]
.kind,
).toBe('positional');
}
});
it('passes an unrecognised due kind through so it can be flagged', () => {
// Collapsing it to null would make a misunderstood deadline indistinguishable from
// no deadline at all, and the resolver's flagging path unreachable.
const result = toVoiceIntent(
wire({ due: { ...emptyDue, kind: 'lunar_month' as never } }),
);
expect(result.due).toEqual({ kind: 'lunar_month' });
});
it('reports no prosthesis when neither a default nor an override was given', () => {
expect(toVoiceIntent(wire()).prosthesis).toBeNull();
});
it('builds a prosthesis intent from a default alone', () => {
const result = toVoiceIntent(wire({ prosthesisDefaultType: 'pfm_crown' }));
expect(result.prosthesis).toEqual({
defaultType: 'pfm_crown',
overrides: [],
});
});
it('builds a prosthesis intent from overrides alone', () => {
const result = toVoiceIntent(
wire({
prosthesisOverrides: [{ tooth: positionalTooth, type: 'pfm_crown' }],
}),
);
expect(result.prosthesis?.defaultType).toBeNull();
expect(result.prosthesis?.overrides).toHaveLength(1);
});
it('narrows connected spans', () => {
const result = toVoiceIntent(
wire({
connectedSpans: [
{
from: { ...positionalTooth, fdi: '14' },
to: { ...positionalTooth, fdi: '16' },
},
],
}),
);
expect(result.connectedSpans[0].from).toEqual({
kind: 'explicit',
fdi: '14',
spoken: 'شش بالا راست',
});
});
it('is total — a missing or malformed payload yields a resolvable shape, not a throw', () => {
// Whatever survives here is reported as unresolved downstream rather than crashing.
for (const bad of [undefined, null, {}, { teeth: 'nope', due: 5 }]) {
expect(() => toVoiceIntent(bad as never)).not.toThrow();
const result = toVoiceIntent(bad as never);
expect(result.teeth).toEqual([]);
expect(result.due).toBeNull();
expect(result.labMatchExact).toBe(false);
}
});
it('coerces a non-boolean labMatchExact to false', () => {
expect(
toVoiceIntent(wire({ labMatchExact: 'yes' as never })).labMatchExact,
).toBe(false);
});
});

View File

@@ -0,0 +1,275 @@
import { normalizeFdiCode } from '../../common/fdi';
import type {
ConnectedSpanIntent,
DueIntent,
ProsthesisIntent,
ToothIntent,
VoiceIntent,
Weekday,
} from './voice.types';
import { WEEKDAYS } from './voice.types';
/**
* The shape the model actually emits, and its JSON schema.
*
* Deliberately flat: strict `json_schema` mode has poor support for discriminated unions,
* so every variant field is present and nullable on the wire. `toVoiceIntent` narrows the
* flat shape into the internal union the resolvers consume, and is total — anything it
* cannot classify becomes a shape the resolvers will report as unresolved rather than
* something that throws here.
*/
export type WireToothIntent = {
spoken: string;
/** Two-digit FDI code, only when the speaker genuinely used FDI notation. */
fdi: string | null;
arch: 'upper' | 'lower' | null;
side: 'patient_right' | 'patient_left' | null;
position: number | null;
};
export type WireDue = {
kind: 'weekday' | 'offset' | 'jalali' | 'gregorian' | 'none';
weekday: Weekday | null;
which: 'this' | 'next' | null;
unit: 'day' | 'week' | 'month' | null;
amount: number | null;
jy: number | null;
jm: number | null;
jd: number | null;
y: number | null;
m: number | null;
d: number | null;
};
export type WireVoiceIntent = {
treatmentType: string | null;
teeth: WireToothIntent[];
connectedSpans: { from: WireToothIntent; to: WireToothIntent }[];
comment: string | null;
prosthesisDefaultType: string | null;
prosthesisOverrides: { tooth: WireToothIntent; type: string }[];
labId: string | null;
labMatchExact: boolean;
due: WireDue;
};
const TOOTH_SCHEMA = {
type: 'object',
additionalProperties: false,
required: ['spoken', 'fdi', 'arch', 'side', 'position'],
properties: {
spoken: {
type: 'string',
description: 'The exact transcript words for this tooth.',
},
fdi: {
type: ['string', 'null'],
description:
'Two-digit FDI code ONLY if the speaker used FDI notation. Otherwise null.',
},
arch: { type: ['string', 'null'], enum: ['upper', 'lower', null] },
side: {
type: ['string', 'null'],
enum: ['patient_right', 'patient_left', null],
description: "The PATIENT's side, never the viewer's.",
},
position: {
type: ['integer', 'null'],
description: '1 = central incisor … 8 = third molar.',
},
},
} as const;
export const VOICE_INTENT_JSON_SCHEMA = {
type: 'object',
additionalProperties: false,
required: [
'treatmentType',
'teeth',
'connectedSpans',
'comment',
'prosthesisDefaultType',
'prosthesisOverrides',
'labId',
'labMatchExact',
'due',
],
properties: {
treatmentType: {
type: ['string', 'null'],
description: 'A treatment type CODE from the supplied list, or null.',
},
teeth: { type: 'array', items: TOOTH_SCHEMA },
connectedSpans: {
type: 'array',
description: 'Bridges / splinted units. Endpoints inclusive.',
items: {
type: 'object',
additionalProperties: false,
required: ['from', 'to'],
properties: { from: TOOTH_SCHEMA, to: TOOTH_SCHEMA },
},
},
comment: {
type: ['string', 'null'],
description: 'Clinical notes, in the spoken language.',
},
prosthesisDefaultType: {
type: ['string', 'null'],
description:
'A prosthesis type CODE applied to every tooth unless overridden.',
},
prosthesisOverrides: {
type: 'array',
items: {
type: 'object',
additionalProperties: false,
required: ['tooth', 'type'],
properties: { tooth: TOOTH_SCHEMA, type: { type: 'string' } },
},
},
labId: {
type: ['string', 'null'],
description: 'An id from the supplied lab list. Never invent one.',
},
labMatchExact: {
type: 'boolean',
description: 'True only when the spoken name matched a lab name exactly.',
},
due: {
type: 'object',
additionalProperties: false,
required: [
'kind',
'weekday',
'which',
'unit',
'amount',
'jy',
'jm',
'jd',
'y',
'm',
'd',
],
properties: {
kind: {
type: 'string',
enum: ['weekday', 'offset', 'jalali', 'gregorian', 'none'],
},
weekday: { type: ['string', 'null'], enum: [...WEEKDAYS, null] },
which: { type: ['string', 'null'], enum: ['this', 'next', null] },
unit: {
type: ['string', 'null'],
enum: ['day', 'week', 'month', null],
},
amount: { type: ['integer', 'null'] },
jy: { type: ['integer', 'null'] },
jm: { type: ['integer', 'null'] },
jd: { type: ['integer', 'null'] },
y: { type: ['integer', 'null'] },
m: { type: ['integer', 'null'] },
d: { type: ['integer', 'null'] },
},
},
},
} as const;
/** Two digits, quadrant 1-8, position 1-8 — the only thing that can be an FDI code. */
const FDI_SHAPE = /^[1-8][1-8]$/;
function toToothIntent(wire: WireToothIntent | undefined | null): ToothIntent {
const spoken = typeof wire?.spoken === 'string' ? wire.spoken : '';
// Persian digits and digit-by-digit dictation ("۲۶", "2 6") are FDI codes that do not
// match literally; without normalising first they fall through to the positional branch
// with no quadrant and are reported as unresolved.
const fdi = normalizeFdiCode(wire?.fdi);
// Only take the explicit branch for something actually FDI-shaped. A model that emits
// fdi:"6" alongside correct arch/side/position would otherwise lose the tooth entirely.
if (FDI_SHAPE.test(fdi)) {
return { kind: 'explicit', fdi, spoken };
}
return {
kind: 'positional',
arch: wire?.arch as 'upper' | 'lower',
side: wire?.side as 'patient_right' | 'patient_left',
position: typeof wire?.position === 'number' ? wire.position : Number.NaN,
spoken,
};
}
function toDueIntent(wire: WireDue | undefined | null): DueIntent | null {
switch (wire?.kind) {
case 'weekday':
return {
kind: 'weekday',
weekday: wire.weekday as Weekday,
which: wire.which as 'this',
};
case 'offset':
return {
kind: 'offset',
unit: wire.unit as 'day',
amount: typeof wire.amount === 'number' ? wire.amount : Number.NaN,
};
case 'jalali':
return {
kind: 'jalali',
jy: wire.jy as number,
jm: wire.jm as number,
jd: wire.jd as number,
};
case 'gregorian':
return {
kind: 'gregorian',
y: wire.y as number,
m: wire.m as number,
d: wire.d as number,
};
case 'none':
case undefined:
return null;
default:
// An unrecognised kind means a deadline WAS spoken and we failed to classify it.
// Passing it through lets the resolver flag it; collapsing it to null would make a
// misunderstood deadline indistinguishable from no deadline at all.
return { kind: wire?.kind } as unknown as DueIntent;
}
}
export function toVoiceIntent(wire: WireVoiceIntent): VoiceIntent {
const teeth = Array.isArray(wire?.teeth) ? wire.teeth : [];
const spans = Array.isArray(wire?.connectedSpans) ? wire.connectedSpans : [];
const overrides = Array.isArray(wire?.prosthesisOverrides)
? wire.prosthesisOverrides
: [];
const connectedSpans: ConnectedSpanIntent[] = spans.map((span) => ({
from: toToothIntent(span?.from),
to: toToothIntent(span?.to),
}));
const hasProsthesis =
wire?.prosthesisDefaultType != null || overrides.length > 0;
const prosthesis: ProsthesisIntent | null = hasProsthesis
? {
defaultType: wire?.prosthesisDefaultType ?? null,
overrides: overrides.map((o) => ({
tooth: toToothIntent(o?.tooth),
type: o?.type,
})),
}
: null;
return {
treatmentType: wire?.treatmentType ?? null,
teeth: teeth.map(toToothIntent),
connectedSpans,
comment: wire?.comment ?? null,
prosthesis,
labId: wire?.labId ?? null,
labMatchExact: wire?.labMatchExact === true,
due: toDueIntent(wire?.due),
};
}

View File

@@ -0,0 +1,197 @@
import {
OpenRouterAsrProvider,
OpenRouterExtractionProvider,
} from './openrouter.provider';
import { VoiceProviderError } from './voice.providers';
const CONFIG = {
apiKey: 'test-key',
baseUrl: 'https://openrouter.test/api/v1',
model: 'm',
};
const CATALOG = {
treatmentTypes: [{ code: 'prosthesis', label: 'پروتز' }],
prosthesisTypes: [{ code: 'pfm_crown', label: 'روکش پی‌اف‌ام' }],
labs: [{ id: 'lab-1', name: 'لابراتوار سینا' }],
};
type ChatBody = {
temperature: number;
provider: { require_parameters: boolean };
response_format: { type: string; json_schema: { strict: boolean } };
messages: { role: string; content: string }[];
};
function parseBody(spy: jest.Mock): ChatBody {
const init = (spy.mock.calls[0] as [string, RequestInit])[1];
return JSON.parse(init.body as string) as ChatBody;
}
function mockFetch(response: {
ok: boolean;
status?: number;
body?: unknown;
text?: string;
}) {
const spy = jest.fn().mockResolvedValue({
ok: response.ok,
status: response.status ?? (response.ok ? 200 : 500),
json: () => Promise.resolve(response.body),
text: () => Promise.resolve(response.text ?? ''),
});
global.fetch = spy;
return spy;
}
afterEach(() => {
jest.restoreAllMocks();
});
describe('OpenRouterAsrProvider', () => {
it('posts base64 JSON with the language hint, per the documented STT contract', async () => {
const spy = mockFetch({
ok: true,
body: { text: ' سلام ', usage: { seconds: 12, cost: 0.002 } },
});
const result = await new OpenRouterAsrProvider(CONFIG).transcribe(
{ data: 'BASE64', format: 'webm' },
'fa',
);
const [url, init] = spy.mock.calls[0] as [string, RequestInit];
expect(url).toBe('https://openrouter.test/api/v1/audio/transcriptions');
expect(init.headers).toMatchObject({ Authorization: 'Bearer test-key' });
expect(JSON.parse(init.body as string)).toEqual({
model: 'm',
input_audio: { data: 'BASE64', format: 'webm' },
language: 'fa',
});
expect(result.text).toBe('سلام');
expect(result.usage).toEqual({ seconds: 12, costUsd: 0.002 });
});
it('reports missing usage as null rather than zero', async () => {
mockFetch({ ok: true, body: { text: 'x' } });
const result = await new OpenRouterAsrProvider(CONFIG).transcribe(
{ data: 'B', format: 'webm' },
'en',
);
expect(result.usage).toEqual({ seconds: null, costUsd: null });
});
it('raises a staged error without leaking the vendor body', async () => {
// A 4xx can echo the request back, transcript included.
mockFetch({
ok: false,
status: 400,
text: 'transcript: patient name here',
});
const provider = new OpenRouterAsrProvider(CONFIG);
await expect(
provider.transcribe({ data: 'B', format: 'webm' }, 'fa'),
).rejects.toMatchObject({
name: 'VoiceProviderError',
stage: 'asr',
status: 400,
});
await expect(
provider.transcribe({ data: 'B', format: 'webm' }, 'fa'),
).rejects.not.toThrow(/patient name/);
});
});
describe('OpenRouterExtractionProvider', () => {
const wireContent = JSON.stringify({
treatmentType: 'prosthesis',
teeth: [
{ spoken: 'یک چهار', fdi: '14', arch: null, side: null, position: null },
],
connectedSpans: [],
comment: null,
prosthesisDefaultType: 'pfm_crown',
prosthesisOverrides: [],
labId: 'lab-1',
labMatchExact: true,
due: {
kind: 'none',
weekday: null,
which: null,
unit: null,
amount: null,
jy: null,
jm: null,
jd: null,
y: null,
m: null,
d: null,
},
});
it('constrains output with a strict JSON schema and a schema-honouring provider', async () => {
const spy = mockFetch({
ok: true,
body: {
choices: [{ message: { content: wireContent } }],
usage: { cost: 0.0008 },
},
});
const result = await new OpenRouterExtractionProvider(CONFIG).extract(
'روی دندان ۱۴ روکش',
CATALOG,
'fa',
);
const body = parseBody(spy);
expect(body.response_format.type).toBe('json_schema');
expect(body.response_format.json_schema.strict).toBe(true);
// Without require_parameters OpenRouter may route to a provider that treats the
// schema as a hint and returns prose, failing parsing intermittently.
expect(body.provider).toEqual({ require_parameters: true });
expect(body.temperature).toBe(0);
expect(result.intent.treatmentType).toBe('prosthesis');
expect(result.intent.teeth[0]).toEqual({
kind: 'explicit',
fdi: '14',
spoken: 'یک چهار',
});
expect(result.intent.due).toBeNull();
expect(result.costUsd).toBe(0.0008);
});
it('sends the catalog codes and lab ids the model is allowed to choose from', async () => {
const spy = mockFetch({
ok: true,
body: { choices: [{ message: { content: wireContent } }] },
});
await new OpenRouterExtractionProvider(CONFIG).extract('x', CATALOG, 'fa');
const body = parseBody(spy);
const system = body.messages[0].content;
expect(system).toContain('prosthesis');
expect(system).toContain('pfm_crown');
expect(system).toContain('lab-1');
expect(body.messages[1]).toEqual({ role: 'user', content: 'x' });
});
it('fails loudly on unparseable content rather than passing rubbish downstream', async () => {
mockFetch({
ok: true,
body: { choices: [{ message: { content: 'I think tooth 14?' } }] },
});
await expect(
new OpenRouterExtractionProvider(CONFIG).extract('x', CATALOG, 'fa'),
).rejects.toBeInstanceOf(VoiceProviderError);
});
it('fails when the model returns no content at all', async () => {
mockFetch({ ok: true, body: { choices: [] } });
await expect(
new OpenRouterExtractionProvider(CONFIG).extract('x', CATALOG, 'fa'),
).rejects.toMatchObject({ stage: 'extraction' });
});
});

View File

@@ -0,0 +1,171 @@
import { Logger } from '@nestjs/common';
import {
type AsrProvider,
type AsrResult,
type AudioInput,
type ExtractionCatalog,
type ExtractionProvider,
type ExtractionResult,
VoiceProviderError,
} from './voice.providers';
import { buildExtractionPrompt } from './extraction.prompt';
import {
VOICE_INTENT_JSON_SCHEMA,
toVoiceIntent,
type WireVoiceIntent,
} from './extraction.wire';
type OpenRouterConfig = {
apiKey: string;
baseUrl: string;
model: string;
};
type TranscriptionResponse = {
text?: unknown;
usage?: { seconds?: unknown; cost?: unknown };
};
type ChatResponse = {
choices?: { message?: { content?: unknown } }[];
usage?: { cost?: unknown };
};
function numberOrNull(value: unknown): number | null {
return typeof value === 'number' && Number.isFinite(value) ? value : null;
}
const errorLogger = new Logger('OpenRouterVoice');
async function readError(
response: Response,
stage: 'asr' | 'extraction',
): Promise<never> {
// A vendor 4xx can echo the request back, transcript included. Keep the body out of the
// thrown message — which callers log and could forward — and out of the default log
// level; it stays available at debug when someone is actively diagnosing.
try {
errorLogger.debug(
`${stage} ${response.status} body: ${(await response.text()).slice(0, 500)}`,
);
} catch {
errorLogger.debug(`${stage} ${response.status} body unreadable`);
}
throw new VoiceProviderError(
stage,
`OpenRouter ${stage} failed with status ${response.status}`,
response.status,
);
}
/** Speech → text via OpenRouter's transcription endpoint (whisper-1 and friends). */
export class OpenRouterAsrProvider implements AsrProvider {
private readonly logger = new Logger(OpenRouterAsrProvider.name);
constructor(private readonly config: OpenRouterConfig) {}
async transcribe(
audio: AudioInput,
localeHint: string,
signal?: AbortSignal,
): Promise<AsrResult> {
const response = await fetch(
`${this.config.baseUrl}/audio/transcriptions`,
{
method: 'POST',
signal,
headers: {
Authorization: `Bearer ${this.config.apiKey}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({
model: this.config.model,
input_audio: { data: audio.data, format: audio.format },
language: localeHint,
}),
},
);
if (!response.ok) await readError(response, 'asr');
const body = (await response.json()) as TranscriptionResponse;
const text = typeof body.text === 'string' ? body.text.trim() : '';
this.logger.debug(
`transcribed ${numberOrNull(body.usage?.seconds) ?? '?'}s`,
);
return {
text,
usage: {
seconds: numberOrNull(body.usage?.seconds),
costUsd: numberOrNull(body.usage?.cost),
},
};
}
}
/** Transcript → VoiceIntent via OpenRouter chat completions with a JSON schema. */
export class OpenRouterExtractionProvider implements ExtractionProvider {
constructor(private readonly config: OpenRouterConfig) {}
async extract(
transcript: string,
catalog: ExtractionCatalog,
localeHint: string,
signal?: AbortSignal,
): Promise<ExtractionResult> {
const response = await fetch(`${this.config.baseUrl}/chat/completions`, {
method: 'POST',
signal,
headers: {
Authorization: `Bearer ${this.config.apiKey}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({
model: this.config.model,
temperature: 0,
// Only route to endpoints that actually honour the JSON schema. Without this,
// OpenRouter may pick a provider that treats it as a hint and returns prose,
// which fails parsing intermittently and unreproducibly.
provider: { require_parameters: true },
messages: buildExtractionPrompt(transcript, catalog, localeHint),
response_format: {
type: 'json_schema',
json_schema: {
name: 'voice_intent',
strict: true,
schema: VOICE_INTENT_JSON_SCHEMA,
},
},
}),
});
if (!response.ok) await readError(response, 'extraction');
const body = (await response.json()) as ChatResponse;
const content = body.choices?.[0]?.message?.content;
if (typeof content !== 'string' || !content.trim()) {
throw new VoiceProviderError(
'extraction',
'OpenRouter returned no content',
);
}
let wire: WireVoiceIntent;
try {
wire = JSON.parse(content) as WireVoiceIntent;
} catch {
// Schema-constrained output should be valid JSON; if it is not, the resolvers can do
// nothing with it, so fail here rather than pass rubbish downstream.
throw new VoiceProviderError(
'extraction',
'OpenRouter returned unparseable JSON',
);
}
return {
intent: toVoiceIntent(wire),
costUsd: numberOrNull(body.usage?.cost),
};
}
}

View File

@@ -0,0 +1,228 @@
import {
resolveToothIntent,
resolveToothIntents,
} from './tooth-intent.resolver';
import type { ToothIntent } from './voice.types';
const positional = (
arch: 'upper' | 'lower',
side: 'patient_right' | 'patient_left',
position: number,
spoken = 'x',
): ToothIntent => ({ kind: 'positional', arch, side, position, spoken });
const explicit = (fdi: string, spoken = 'x'): ToothIntent => ({
kind: 'explicit',
fdi,
spoken,
});
describe('resolveToothIntent', () => {
describe('positional intents', () => {
// "شش بالا راست" — upper right six — must be 16, not 26. A mirrored quadrant is a
// valid code for the wrong tooth and reaches the lab unnoticed.
it('resolves each quadrant from the patient perspective', () => {
expect(resolveToothIntent(positional('upper', 'patient_right', 6))).toBe(
'16',
);
expect(resolveToothIntent(positional('upper', 'patient_left', 6))).toBe(
'26',
);
expect(resolveToothIntent(positional('lower', 'patient_left', 6))).toBe(
'36',
);
expect(resolveToothIntent(positional('lower', 'patient_right', 6))).toBe(
'46',
);
});
it('returns null for an out-of-range position instead of clamping', () => {
expect(
resolveToothIntent(positional('upper', 'patient_right', 9)),
).toBeNull();
expect(
resolveToothIntent(positional('upper', 'patient_right', 0)),
).toBeNull();
});
it('returns null for a malformed arch or side', () => {
expect(
resolveToothIntent(
positional('sideways' as 'upper', 'patient_right', 6),
),
).toBeNull();
expect(
resolveToothIntent(
positional('upper', 'viewer_right' as 'patient_right', 6),
),
).toBeNull();
});
});
describe('explicit intents', () => {
it('accepts a permanent FDI code', () => {
expect(resolveToothIntent(explicit('14'))).toBe('14');
expect(resolveToothIntent(explicit('48'))).toBe('48');
});
it('rejects deciduous codes rather than snapping to a permanent tooth', () => {
expect(resolveToothIntent(explicit('51'))).toBeNull();
expect(resolveToothIntent(explicit('85'))).toBeNull();
});
it('rejects nonsense', () => {
for (const value of ['', '1', '99', '140']) {
expect(resolveToothIntent(explicit(value))).toBeNull();
}
});
});
it('returns null for a missing or unknown intent shape', () => {
expect(resolveToothIntent(undefined as unknown as ToothIntent)).toBeNull();
expect(
resolveToothIntent({ kind: 'guess' } as unknown as ToothIntent),
).toBeNull();
});
});
describe('resolveToothIntents', () => {
it('resolves a mixed list and sorts the result', () => {
const result = resolveToothIntents([
positional('upper', 'patient_right', 5, 'پنج بالا راست'),
explicit('14', 'یک چهار'),
]);
expect(result.teeth).toEqual(['14', '15']);
expect(result.unresolved).toEqual([]);
});
it('collapses a tooth named twice', () => {
const result = resolveToothIntents([
explicit('14', 'چهارده'),
positional('upper', 'patient_right', 4, 'چهار بالا راست'),
]);
expect(result.teeth).toEqual(['14']);
});
it('reports what it could not understand instead of dropping it', () => {
const result = resolveToothIntents([
explicit('14', 'یک چهار'),
explicit('51', 'دندان شیری'),
positional('upper', 'patient_right', 9, 'نه بالا راست'),
]);
expect(result.teeth).toEqual(['14']);
expect(result.unresolved).toEqual([
{ spoken: 'دندان شیری', reason: 'not_permanent_tooth' },
{ spoken: 'نه بالا راست', reason: 'position_out_of_range' },
]);
});
it('does not repeat an identical unresolved item', () => {
const result = resolveToothIntents([
explicit('51', 'شیری'),
explicit('51', 'شیری'),
]);
expect(result.unresolved).toHaveLength(1);
});
it('survives a non-array where the model should have sent a list', () => {
// The model can return an object or a number here; that must degrade, not 500.
for (const bad of [undefined, null, 5, 'teeth', { fdi: '14' }]) {
expect(resolveToothIntents(bad as unknown as ToothIntent[])).toEqual({
teeth: [],
unresolved: [],
});
}
});
it('reads a spoken number as its FDI code, digits in any script', () => {
// The product rule: the number the clinician says IS the tooth. 26 = quadrant 2
// (patient's upper left) + position 6 = first molar.
for (const raw of ['26', ' 26 ', '2 6', '\u06F2\u06F6', '\u0662\u0666']) {
expect(resolveToothIntents([explicit(raw, 'x')]).teeth).toEqual(['26']);
}
});
it('trims an explicit code, matching normalizeTeeth', () => {
expect(resolveToothIntents([explicit(' 14 ', 'x')]).teeth).toEqual(['14']);
});
it('distinguishes a deciduous tooth from nonsense in the reason it reports', () => {
// '51' really is a (primary) tooth the chart cannot show; '99' is not a tooth at all.
expect(
resolveToothIntents([explicit('51', 'shiri')]).unresolved[0].reason,
).toBe('not_permanent_tooth');
for (const junk of ['99', '19', '140', '', 'ab']) {
expect(
resolveToothIntents([explicit(junk, `j-${junk}`)]).unresolved[0].reason,
).toBe('malformed');
}
});
it('says the quadrant is missing rather than blaming the words', () => {
// Regression: "ترمیم برای دندون دو" reported "could not be read", sending the
// clinician to look for a transcription fault. Position 2 was understood fine —
// what is missing is the quadrant, and four teeth carry position 2.
const bare = {
kind: 'positional',
arch: null,
side: null,
position: 2,
spoken: 'دندون دو',
} as unknown as ToothIntent;
const result = resolveToothIntents([bare]);
expect(result.teeth).toEqual([]);
expect(result.unresolved).toEqual([
{ spoken: 'دندون دو', reason: 'tooth_missing_quadrant' },
]);
});
it('reports a missing quadrant for a half-specified tooth too', () => {
// "دو بالا" narrows it to 12 or 22 — still not one tooth, and still not our guess.
for (const half of [
{ arch: 'upper', side: null },
{ arch: null, side: 'patient_right' },
]) {
const result = resolveToothIntents([
{
kind: 'positional',
...half,
position: 2,
spoken: 'دو',
} as unknown as ToothIntent,
]);
expect(result.unresolved[0].reason).toBe('tooth_missing_quadrant');
}
});
it('still calls an out-of-range position out of range when the quadrant is missing', () => {
// Position wins: "nine" is wrong however completely it was said.
const result = resolveToothIntents([
{
kind: 'positional',
arch: null,
side: null,
position: 9,
spoken: 'نه',
} as unknown as ToothIntent,
]);
expect(result.unresolved[0].reason).toBe('position_out_of_range');
});
it('keeps unresolved items separate when the model omits the spoken span', () => {
// Without `spoken` these are indistinguishable; collapsing them would hide a lost tooth.
const result = resolveToothIntents([
{ kind: 'explicit', fdi: '51' } as ToothIntent,
{ kind: 'explicit', fdi: '52' } as ToothIntent,
]);
expect(result.unresolved).toHaveLength(2);
});
it('handles an empty or missing list', () => {
expect(resolveToothIntents([])).toEqual({ teeth: [], unresolved: [] });
expect(resolveToothIntents(undefined as unknown as ToothIntent[])).toEqual({
teeth: [],
unresolved: [],
});
});
});

View File

@@ -0,0 +1,115 @@
import { isFdiTooth, normalizeFdiCode, toFdi } from '../../common/fdi';
import type { ToothIntent, UnresolvedItem } from './voice.types';
export type ToothResolution = {
/** Unique FDI codes, sorted (matching normalizeTeeth's ordering). */
teeth: string[];
unresolved: UnresolvedItem[];
};
/** Everything here parses untrusted model output, so nothing may throw. */
function normalizedFdi(intent: ToothIntent): string {
// Same normalisation the wire layer used to pick this branch, so the two cannot
// disagree: '14 ' is tooth 14 through the treatment API and '۲۶' is tooth 26, and
// neither may be reported as malformed here.
return normalizeFdiCode((intent as { fdi?: unknown }).fdi);
}
/**
* Resolve one spoken tooth reference to an FDI code, or null.
*
* Never guesses and never clamps: a position of 9, a deciduous tooth, or a malformed
* intent resolves to null so the caller can surface it as "not understood" rather than
* silently selecting a neighbouring tooth.
*/
export function resolveToothIntent(intent: ToothIntent): string | null {
if (!intent || typeof intent !== 'object') return null;
if (intent.kind === 'explicit') {
const fdi = normalizedFdi(intent);
return isFdiTooth(fdi) ? fdi : null;
}
if (intent.kind === 'positional') {
if (intent.arch !== 'upper' && intent.arch !== 'lower') return null;
if (intent.side !== 'patient_right' && intent.side !== 'patient_left')
return null;
return toFdi(intent.arch, intent.side, intent.position);
}
return null;
}
function unresolvedReason(intent: ToothIntent): UnresolvedItem['reason'] {
if (!intent || typeof intent !== 'object') return 'malformed';
if (intent.kind === 'explicit') {
const fdi = normalizedFdi(intent);
// Quadrants 1-4 are permanent and would already have resolved, so a well-formed
// quadrant+position reaching here is quadrant 5-8: deciduous. Anything else is noise.
return /^[1-8][1-8]$/.test(fdi) ? 'not_permanent_tooth' : 'malformed';
}
if (intent.kind === 'positional') {
const positionBad =
!Number.isInteger(intent.position) ||
intent.position < 1 ||
intent.position > 8;
if (positionBad) return 'position_out_of_range';
// The position was understood, so the words were not the problem: the speaker never
// said which quadrant. "دندون دو" names four teeth at once, and telling the
// clinician it "could not be read" would send them looking for the wrong fault.
const archMissing = intent.arch !== 'upper' && intent.arch !== 'lower';
const sideMissing =
intent.side !== 'patient_right' && intent.side !== 'patient_left';
return archMissing || sideMissing ? 'tooth_missing_quadrant' : 'malformed';
}
return 'malformed';
}
function spokenOf(intent: ToothIntent): string {
const spoken = (intent as { spoken?: unknown })?.spoken;
return typeof spoken === 'string' && spoken.trim() ? spoken.trim() : '';
}
/**
* Resolve a list of spoken tooth references.
*
* Duplicates collapse — a clinician may name the same tooth twice in one sentence — and
* anything unresolvable is reported rather than dropped, so the review sheet can show the
* user exactly which words were not understood.
*/
export function resolveToothIntents(
intents: readonly ToothIntent[],
): ToothResolution {
const teeth = new Set<string>();
const unresolved: UnresolvedItem[] = [];
const seenUnresolved = new Set<string>();
// Not `intents ?? []`: a model may return an object or a number here, and a
// non-iterable must degrade like any other malformed shape rather than throw.
const list: readonly ToothIntent[] = Array.isArray(intents)
? (intents as readonly ToothIntent[])
: [];
for (const intent of list) {
const fdi = resolveToothIntent(intent);
if (fdi) {
teeth.add(fdi);
continue;
}
const reason = unresolvedReason(intent);
const spoken = spokenOf(intent);
// Only dedupe items we can actually tell apart. Without `spoken`, two distinct lost
// references would collapse into one blank review row and a tooth would vanish.
if (spoken) {
const key = `${spoken}::${reason}`;
if (seenUnresolved.has(key)) continue;
seenUnresolved.add(key);
}
unresolved.push({ spoken, reason });
}
return { teeth: [...teeth].sort(), unresolved };
}

View File

@@ -0,0 +1,35 @@
import { HttpStatus, Injectable } from '@nestjs/common';
import { ThrottlerGuard } from '@nestjs/throttler';
import { AppException, ErrorCode } from '../../common/errors';
/**
* Rate limits voice extraction per user rather than per IP.
*
* The default tracker keys on `req.ip`, which behind nginx means the whole deployment
* shares one bucket unless `trust proxy` is set — and an abuser rotating IPs would bypass
* it entirely. Since v1 ships with no plan gate, this is the only control on metered
* vendor spend, so it has to key on something the client cannot change.
*
* Guard order matters: the controller's JwtAuthGuard runs before this method-level guard,
* so `req.user` is populated by the time `getTracker` is called.
*/
@Injectable()
export class VoiceThrottlerGuard extends ThrottlerGuard {
protected getTracker(req: Record<string, unknown>): Promise<string> {
const user = req?.user as { id?: unknown } | undefined;
if (typeof user?.id === 'string' && user.id) {
return Promise.resolve(`voice:user:${user.id}`);
}
// Unauthenticated requests never reach here, but fall back rather than share a bucket.
const ip = typeof req?.ip === 'string' ? req.ip : 'unknown';
return Promise.resolve(`voice:ip:${ip}`);
}
/** Without this, ThrottlerException surfaces as INTERNAL_ERROR — there is no 429 fallback. */
protected throwThrottlingException(): Promise<void> {
throw new AppException(
ErrorCode.VOICE_RATE_LIMITED,
HttpStatus.TOO_MANY_REQUESTS,
);
}
}

View File

@@ -0,0 +1,69 @@
import {
Body,
Controller,
Get,
Post,
Req,
Res,
UseGuards,
} from '@nestjs/common';
import { ApiBearerAuth, ApiOperation, ApiTags } from '@nestjs/swagger';
import type { Request, Response } from 'express';
import { ClinicOrgGuard } from '../../common/guards/clinic-org.guard';
import { JwtAuthGuard } from '../auth/guards/jwt-auth.guard';
import { ExtractVoiceDto } from './dto/voice.dto';
import { VoiceThrottlerGuard } from './voice-throttler.guard';
import { VoiceService } from './voice.service';
type VoiceRequestUser = { id: string; organizationId?: string };
@ApiTags('voice')
@ApiBearerAuth('JWT-auth')
@UseGuards(JwtAuthGuard, ClinicOrgGuard)
@Controller('voice')
export class VoiceController {
constructor(private readonly voiceService: VoiceService) {}
@Get('availability')
@ApiOperation({
summary: 'Whether voice entry is configured, and for which locales',
})
getAvailability() {
// The frontend cannot learn this from NEXT_PUBLIC_* — those are baked in at build
// time, so enabling a locale would otherwise require rebuilding the image.
return { success: true, data: this.voiceService.getAvailability() };
}
@Post('extract')
// Guarded here rather than globally, and configured by VOICE_THROTTLE_* rather than
// hardcoded. Availability is deliberately left unthrottled — it is cheap and the
// frontend calls it on load.
@UseGuards(VoiceThrottlerGuard)
@ApiOperation({
summary:
'Transcribe a recording and extract treatment detail intents (TAB_TREATMENT_EDIT)',
})
async extract(
@Req() req: Request & { user: VoiceRequestUser },
@Res({ passthrough: true }) res: Response,
@Body() dto: ExtractVoiceDto,
) {
// Cancelling in the browser closes the connection; propagate that as an abort so the
// in-flight vendor call stops rather than settling and being discarded. It is metered
// per minute, so letting it run costs real money for a result nobody will see.
const aborter = new AbortController();
res.on('close', () => {
if (!res.writableFinished) aborter.abort();
});
// dto.locale, not req.user.language: the client sends the locale the microphone was
// actually offered in, so the ASR hint, catalog labels and week start all match it.
const data = await this.voiceService.extract(
req.user,
dto,
dto.locale,
aborter.signal,
);
return { success: true, data };
}
}

View File

@@ -0,0 +1,12 @@
import { Module } from '@nestjs/common';
import { PrismaService } from '../../../prisma/prisma.service';
import { ProsthesisCatalogModule } from '../prosthesis-catalog/prosthesis-catalog.module';
import { VoiceController } from './voice.controller';
import { VoiceService } from './voice.service';
@Module({
imports: [ProsthesisCatalogModule],
controllers: [VoiceController],
providers: [VoiceService, PrismaService],
})
export class VoiceModule {}

View File

@@ -0,0 +1,68 @@
import type { VoiceIntent } from './voice.types';
/**
* ASR and extraction are separate, independently swappable roles — they will not come
* from the same vendor for every locale. Both are resolved per locale from
* `config.voice.profiles`, so pointing `fa` at a Persian-specialist vendor while `en`
* and `nl` keep OpenRouter is configuration, not code.
*/
export type AudioInput = {
/** Raw base64, no data: prefix. */
data: string;
/** Container as the recorder produced it: webm, mp4, m4a, wav, … */
format: string;
};
export type AsrUsage = {
seconds: number | null;
costUsd: number | null;
};
export type AsrResult = {
text: string;
usage: AsrUsage;
};
export interface AsrProvider {
/** `localeHint` is ISO-639-1 and materially improves accuracy — always pass it. */
transcribe(
audio: AudioInput,
localeHint: string,
signal?: AbortSignal,
): Promise<AsrResult>;
}
export type ExtractionCatalog = {
/** Catalog codes with their labels in the actor's locale, so the model matches spoken words. */
treatmentTypes: { code: string; label: string }[];
prosthesisTypes: { code: string; label: string }[];
/** The clinic's linked labs — a closed choice list. */
labs: { id: string; name: string }[];
};
export type ExtractionResult = {
intent: VoiceIntent;
costUsd: number | null;
};
export interface ExtractionProvider {
extract(
transcript: string,
catalog: ExtractionCatalog,
localeHint: string,
signal?: AbortSignal,
): Promise<ExtractionResult>;
}
/** Raised when a vendor call fails; the service maps this onto the staged error codes. */
export class VoiceProviderError extends Error {
constructor(
readonly stage: 'asr' | 'extraction',
message: string,
readonly status?: number,
) {
super(message);
this.name = 'VoiceProviderError';
}
}

View File

@@ -0,0 +1,354 @@
import { HttpStatus, Injectable, Logger } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import { LinkStatus } from '@prisma/client';
import { PrismaService } from '../../../prisma/prisma.service';
import { AppException, ErrorCode } from '../../common/errors';
import {
civilDateInZone,
isValidIanaTimeZone,
} from '../../common/zoned-civil-time';
import type { VoiceConfig, VoiceProfile } from '../../configs/configurations';
import { hasEffectivePermission } from '../../common/membership-permissions';
import { normalizeCatalogLocale } from '../catalog/catalog-label.service';
import { ProsthesisCatalogService } from '../prosthesis-catalog/prosthesis-catalog.service';
import { TreatmentCatalogService } from '../treatment-catalog/treatment-catalog.service';
import { weekStartForLocale } from './due-date.resolver';
import {
resolveVoiceIntent,
type ResolvedExtraction,
} from './extraction.resolver';
import {
OpenRouterAsrProvider,
OpenRouterExtractionProvider,
} from './openrouter.provider';
import {
VoiceProviderError,
type AsrProvider,
type ExtractionCatalog,
type ExtractionProvider,
} from './voice.providers';
import type { ExtractVoiceDto } from './dto/voice.dto';
export type VoiceAvailability = {
enabled: boolean;
locales: string[];
maxRecordingMs: number | null;
};
export type VoiceExtractionResponse = ResolvedExtraction & {
transcript: string;
};
@Injectable()
export class VoiceService {
private readonly logger = new Logger(VoiceService.name);
constructor(
private readonly prisma: PrismaService,
private readonly config: ConfigService,
private readonly treatmentCatalog: TreatmentCatalogService,
private readonly prosthesisCatalog: ProsthesisCatalogService,
) {}
private get voiceConfig(): VoiceConfig {
return this.config.get<VoiceConfig>('voice')!;
}
/**
* What the frontend needs to decide whether to render the microphone at all.
*
* v1 ships ungated beyond a configured locale profile — no plan check. The
* Plan.features design is deferred, not dropped.
*/
getAvailability(): VoiceAvailability {
const voice = this.voiceConfig;
const hasKey = Boolean(voice.openRouter.apiKey);
const locales = hasKey ? Object.keys(voice.profiles) : [];
return {
enabled: locales.length > 0,
locales,
maxRecordingMs: voice.maxRecordingMs,
};
}
async extract(
user: { id: string; organizationId?: string },
dto: ExtractVoiceDto,
locale: string,
signal?: AbortSignal,
): Promise<VoiceExtractionResponse> {
const startedAt = Date.now();
const organizationId = this.assertOrganization(user);
await this.assertCanEditTreatment(user.id, organizationId);
const catalogLocale = normalizeCatalogLocale(locale);
const profile = this.resolveProfile(catalogLocale);
this.assertWithinCap(dto.durationMs);
const timeZone = isValidIanaTimeZone(dto.timeZone) ? dto.timeZone : 'UTC';
const todayIso = civilDateInZone(new Date(), timeZone);
const { asr, extraction } = this.buildProviders(profile);
// Stage 1 — audio never touches disk and is not retained beyond this call.
let transcript: string;
let asrCost: number | null = null;
let asrSeconds: number | null = null;
try {
const result = await asr.transcribe(
{ data: dto.audio, format: dto.format },
catalogLocale,
signal,
);
transcript = result.text;
asrCost = result.usage.costUsd;
asrSeconds = result.usage.seconds;
} catch (error) {
throw this.toAppException(error, 'asr');
}
// durationMs is client-reported and therefore not enforcement. usage.seconds is the
// vendor's own measurement of the audio it decoded, so a client under-reporting length
// to slip past the cap is caught here — after the ASR spend, but before the extraction
// call, and visibly in telemetry.
if (asrSeconds != null) {
this.assertWithinCap(asrSeconds * 1000);
}
if (!transcript.trim()) {
throw new AppException(
ErrorCode.VOICE_NOTHING_RECOGNIZED,
HttpStatus.UNPROCESSABLE_ENTITY,
);
}
// Stage 2 — structure it. On failure the transcript still goes back to the client so
// the words the clinician already paid for are not lost (transcript salvage).
let resolved: ResolvedExtraction;
let llmCost: number | null = null;
try {
// Inside the try: the transcript is already paid for, so a catalog/DB failure here
// must still salvage it rather than becoming a generic 500 that throws it away.
const catalog = await this.buildCatalog(organizationId, catalogLocale);
const result = await extraction.extract(
transcript,
catalog,
catalogLocale,
signal,
);
llmCost = result.costUsd;
resolved = resolveVoiceIntent(result.intent, {
todayIso,
weekStartJs: weekStartForLocale(catalogLocale),
treatmentTypeCodes: new Set(catalog.treatmentTypes.map((t) => t.code)),
prosthesisTypeCodes: new Set(
catalog.prosthesisTypes.map((t) => t.code),
),
linkedLabIds: new Set(catalog.labs.map((l) => l.id)),
});
} catch (error) {
throw this.toAppException(error, 'extraction', transcript);
}
this.logTelemetry({
locale: catalogLocale,
durationMs: dto.durationMs,
elapsedMs: Date.now() - startedAt,
asrCost,
llmCost,
resolved,
});
return { ...resolved, transcript };
}
private assertOrganization(user: { organizationId?: string }): string {
if (!user?.organizationId) {
throw new AppException(
ErrorCode.AUTH_ORG_NOT_SELECTED,
HttpStatus.BAD_REQUEST,
);
}
return user.organizationId;
}
private async assertCanEditTreatment(userId: string, organizationId: string) {
const membership = await this.prisma.membership.findFirst({
where: {
userId,
organizationId,
OR: [{ isOwner: true }, { isActive: true }],
},
include: {
permissions: { include: { permission: true } },
organization: { include: { type: true, plan: true } },
},
});
if (!membership) {
throw new AppException(
ErrorCode.PERMISSION_NOT_MEMBER,
HttpStatus.FORBIDDEN,
);
}
if (!hasEffectivePermission(membership, 'TAB_TREATMENT_EDIT')) {
throw new AppException(
ErrorCode.PERMISSION_EDIT_TREATMENTS,
HttpStatus.FORBIDDEN,
);
}
}
private resolveProfile(locale: string): VoiceProfile {
const voice = this.voiceConfig;
const profile = voice.profiles[locale];
if (!profile || !voice.openRouter.apiKey) {
throw new AppException(
ErrorCode.VOICE_NOT_AVAILABLE,
HttpStatus.FORBIDDEN,
);
}
return profile;
}
/**
* Grace above the configured cap.
*
* The client auto-stops when elapsed >= maxMs, then measures the final length after the
* recorder has actually stopped — so a recording that runs to the cap always reports
* slightly over it. Without this tolerance the auto-stop would guarantee a rejection,
* discarding exactly the recording it was meant to save. The client still reports the
* true length, so telemetry stays honest.
*/
private static readonly CAP_TOLERANCE_MS = 2_000;
private assertWithinCap(durationMs: number) {
const max = this.voiceConfig.maxRecordingMs;
if (max != null && durationMs > max + VoiceService.CAP_TOLERANCE_MS) {
throw new AppException(
ErrorCode.VOICE_CLIP_TOO_LONG,
HttpStatus.PAYLOAD_TOO_LARGE,
);
}
}
private buildProviders(profile: VoiceProfile): {
asr: AsrProvider;
extraction: ExtractionProvider;
} {
const { apiKey, baseUrl } = this.voiceConfig.openRouter;
const base = { apiKey: apiKey!, baseUrl };
return {
asr: new OpenRouterAsrProvider({ ...base, model: profile.asr.model }),
extraction: new OpenRouterExtractionProvider({
...base,
model: profile.llm.model,
}),
};
}
/** Codes with labels in the actor's locale, plus the clinic's linked labs. */
private async buildCatalog(
organizationId: string,
locale: string,
): Promise<ExtractionCatalog> {
const [treatmentTypes, prosthesisTypes, labs] = await Promise.all([
this.treatmentCatalog.list(locale, null),
this.prosthesisCatalog.list(locale),
this.listLinkedLabs(organizationId),
]);
return {
treatmentTypes: treatmentTypes
.filter((entry) => entry.availableInTreatment)
.map((entry) => ({ code: entry.code, label: entry.label })),
prosthesisTypes: prosthesisTypes.map((entry) => ({
code: entry.code,
label: entry.label,
})),
labs,
};
}
private async listLinkedLabs(
organizationId: string,
): Promise<{ id: string; name: string }[]> {
const [linksA, linksB] = await Promise.all([
this.prisma.organizationLink.findMany({
where: { organizationAId: organizationId, status: LinkStatus.ACTIVE },
include: { organizationB: { select: { id: true, name: true } } },
}),
this.prisma.organizationLink.findMany({
where: { organizationBId: organizationId, status: LinkStatus.ACTIVE },
include: { organizationA: { select: { id: true, name: true } } },
}),
]);
return [
...linksA.map((l) => ({
id: l.organizationB.id,
name: l.organizationB.name,
})),
...linksB.map((l) => ({
id: l.organizationA.id,
name: l.organizationA.name,
})),
];
}
private toAppException(
error: unknown,
stage: 'asr' | 'extraction',
transcript?: string,
): AppException {
if (error instanceof Error && error.name === 'AbortError') {
// The clinician cancelled; not a failure worth a translated message.
return new AppException(ErrorCode.BAD_REQUEST, HttpStatus.BAD_REQUEST);
}
if (error instanceof VoiceProviderError) {
this.logger.warn(`voice ${stage} failed: ${error.message}`);
} else {
this.logger.error(`voice ${stage} failed unexpectedly`, error as Error);
}
const code =
stage === 'asr'
? ErrorCode.VOICE_ASR_FAILED
: ErrorCode.VOICE_EXTRACT_FAILED;
return new AppException(
code,
HttpStatus.BAD_GATEWAY,
transcript ? { transcript } : undefined,
);
}
/**
* Structured, patient-free. Never the transcript, never audio, never a patient id.
* Log lines are the interim sink until this repo has metrics infrastructure.
*/
private logTelemetry(input: {
locale: string;
durationMs: number;
elapsedMs: number;
asrCost: number | null;
llmCost: number | null;
resolved: ResolvedExtraction;
}) {
const { resolved } = input;
this.logger.log(
JSON.stringify({
event: 'voice.extract',
locale: input.locale,
clipMs: input.durationMs,
elapsedMs: input.elapsedMs,
costUsd: (input.asrCost ?? 0) + (input.llmCost ?? 0),
resolvedFields: {
treatmentType: resolved.treatmentType != null,
teeth: resolved.teeth.length,
comment: resolved.comment != null,
prosthesisComplete: resolved.prosthesis?.complete ?? null,
lab: resolved.labId != null,
dueDate: resolved.dueDate != null,
},
unresolvedCount: resolved.unresolved.length,
}),
);
}
}

View File

@@ -0,0 +1,80 @@
import type { Arch, PatientSide } from '../../common/fdi';
/**
* What the extraction model is allowed to return.
*
* The model emits *intents*, never resolved values: no FDI codes, no ISO dates. Pure,
* unit-tested resolvers turn intents into domain values, so the two highest-consequence
* mappings — quadrant mirroring and Jalali conversion — are testable rather than hopeful.
*/
/** A single spoken tooth reference. `spoken` is the transcript span, echoed back to the user. */
export type ToothIntent =
| { kind: 'explicit'; fdi: string; spoken: string }
| {
kind: 'positional';
arch: Arch;
side: PatientSide;
/** 1 = central incisor … 8 = third molar. */
position: number;
spoken: string;
};
/** A spoken deadline. The model never does calendar arithmetic. */
export type DueIntent =
| { kind: 'weekday'; weekday: Weekday; which: 'this' | 'next' }
| { kind: 'offset'; unit: 'day' | 'week' | 'month'; amount: number }
| { kind: 'jalali'; jy: number; jm: number; jd: number }
| { kind: 'gregorian'; y: number; m: number; d: number };
export const WEEKDAYS = [
'saturday',
'sunday',
'monday',
'tuesday',
'wednesday',
'thursday',
'friday',
] as const;
export type Weekday = (typeof WEEKDAYS)[number];
/** Two teeth defining an inclusive connected (bridge) span. */
export type ConnectedSpanIntent = { from: ToothIntent; to: ToothIntent };
export type ProsthesisIntent = {
/** Catalog code applied to every tooth unless overridden. */
defaultType: string | null;
overrides: { tooth: ToothIntent; type: string }[];
};
export type VoiceIntent = {
treatmentType: string | null;
teeth: ToothIntent[];
connectedSpans: ConnectedSpanIntent[];
comment: string | null;
prosthesis: ProsthesisIntent | null;
/** Must be one of the linked-lab ids supplied in the prompt, or null. */
labId: string | null;
/** False when the spoken name only approximately matched — the UI then requires an explicit tick. */
labMatchExact: boolean;
due: DueIntent | null;
};
/** Why a spoken item could not be turned into a domain value. Shown to the user. */
export type UnresolvedReason =
| 'not_permanent_tooth'
| 'position_out_of_range'
/** A position was understood but no quadrant was spoken — four teeth match. */
| 'tooth_missing_quadrant'
| 'malformed'
| 'span_not_same_arch'
| 'unknown_catalog_code'
| 'tooth_not_selected'
| 'invalid_date';
export type UnresolvedItem = {
/** The transcript span that could not be resolved, so the user can see what was heard. */
spoken: string;
reason: UnresolvedReason;
};

View File

@@ -0,0 +1,722 @@
# Voice treatment entry
**Status:** Implemented on `feat/voice-treatment-entry` — unreviewed, and blocked on the
ASR spike (§11 item 1) before it is trustworthy in front of patients
**Area:** Treatment workspace (CLINIC orgs)
**Created:** 2026-08-20
Fill a `TreatmentDetail` — including its lab dispatch — by speaking, instead of by
tapping through the type dropdown, the FDI chart, the prosthesis wizard and the lab
picker.
---
## 1. Goal
A clinician on the Treatment tab taps a microphone, describes the treatment for the
already-selected patient in one utterance, and is shown a **review sheet** of what was
understood. Fields they tick are applied to the open detail chip. Nothing is written to
the form without confirmation.
### In scope
One recording produces **exactly one** `TreatmentDetail`, and may fill every field of it:
| Field | Source |
|---|---|
| `treatmentType` | catalog code, matched against locale labels |
| `teeth` | FDI codes, via tooth-intent resolver |
| `toothSelectionGroups` | connected (bridge) / single spans |
| `comment` | cleaned dictated notes |
| lab: `prosthesisTypeCode` per tooth | default type + per-tooth overrides |
| lab: `destinationOrganizationId` | matched against the clinic's linked labs |
| lab: `dueDate` | via due-date intent resolver |
### Out of scope (v1)
- Multiple detail chips from one recording.
- `attachmentIds` — files cannot be dictated.
- Editing an existing detail by voice ("no, make that 15"). Confirming a recording always
creates a **new** detail (see §2).
- Creating the treatment or selecting the patient by voice. A patient is already
selected; voice only fills the form.
- Lab-side (`LAB` org) usage. Clinic only.
---
## 2. User flow and UI integration
### The control: Add detail, split
The `Add detail` button gains a second segment holding the mic. The halves read as
siblings — both end in a new detail — but they are **independent actions**:
- **Add half — unchanged.** Same `onAddDetail`, same seeding, same `setEntryStep`. It
gains a neighbour and nothing else. Its logic is not modified, wrapped or made
conditional.
- **Mic half** — starts a recording. Nothing is created until confirm (below).
The `Add detail` `<Button>` in `TreatmentDetailsEditor` becomes a segmented control built
exactly like the **detail chip's trash affordance** in the same file
(`TreatmentDetailsEditor.tsx`, the chip `<div>` + label `<button>` + remove `<button>`):
- an `inline-flex items-stretch overflow-hidden rounded-[var(--radius-md)]` wrapper;
- two raw `<button>` children divided by `border-s`, each with its own `disabled`,
`title`, `aria-label` and `focus-visible:ring-inset`;
- styled `primary` (`bg-primary text-white`) to preserve the button's current look;
- `w-full sm:w-auto` on the wrapper with the Add half `flex-1`, reproducing today's
`fullWidth` + `sm:w-auto shrink-0` behaviour.
The shared `Button` component is **not** reusable for the halves: it hardcodes
`rounded-[var(--radius-md)]` on each instance and owns auto-pending state, both of which
fight a segmented control. This is precisely why the chip pattern uses raw `<button>`s,
and this control follows it.
**Side:** the mic is the second flex child with `border-s` — the **logical end**, exactly
like the trash. Visually right in `en`/`nl`, visually left in `fa`. No physical
left/right anywhere, per the repo's RTL rule.
```
en / nl (LTR) fa (RTL)
[ Add detail │ 🎤 ] [ 🎤 │ افزودن ]
[ ترمیم 14,15 │ 🗑 ] [ 🗑 │ ترمیم 14,15 ]
↑ same side as the chip's trash, in both directions
```
### Flow
```
[editable day]
▼ tap 🎤 ← mic swaps to ■ (red); Add half disabled
recording ─────────── inline bar below the header row:
│ ● 0:12 / 2:00 ▁▃▇▅▂▆█▄▁ [لغو]
│ auto-stops at the 2:00 cap
▼ tap ■
processing ────────── ● transcribing… ○ extracting [cancel]
review sheet
☑ Type ترمیم
☑ Teeth [mini FDI chart] 14 15
☑ Notes حساسیت به سرما
☐ Lab لابراتوار سینا ⚠ similar name
▼ [Apply n fields]
detail created or filled → normal save flow
```
**No layout shift.** The segmented control never changes size; only the mic's icon and
colour change. The timer and level meter live in a full-width bar inserted between the
header row and the chip strip — the header is `sm:justify-between`, so growing the button
mid-recording would shove the row on every start and every stop.
**Duration is capped at 2 minutes** (`maxMs`, configurable, v1 default `120_000`). The
timer shows `elapsed / 2:00` and the recorder auto-stops at the cap. `maxMs: null` means
uncapped and remains supported, but is not the v1 default.
Two minutes is generous against the longest realistic utterance — a full prosthesis
dictation with type, several teeth, a bridge, prosthesis type, lab and due date — while
bounding worst-case spend. It also sits comfortably inside the vendor limits (§4): 2
minutes of `webm/opus` is well under 1 MB against a 25 MB ceiling, and transcribes in a
few seconds against a 60s processing timeout.
Cost per recording is therefore bounded at **$0.012** of ASR (2 min × $0.006) plus
~$0.0008 of extraction — about **1.3¢ worst case**, against ~0.3¢ for a typical 20s
utterance.
### What confirm does
The detail is created **on confirm**, never on tap. Tapping the mic starts a recording and
nothing else, so cancelling, a vendor failure, a rate-limit, or navigating away leaves the
chip strip untouched — there is no orphan state to clean up.
Confirm **always appends a new detail**. One unconditional rule, no dependence on
invisible state: append `newDetail()` seeded with `defaultTreatmentTypeForAppointment`,
make it active, `setEntryStep('treatment')`, then apply the ticked rows on top — so an
extracted type overrides the seed, and unticking the type row leaves the seeded default in
place.
This reuses `newDetail()` and the same state transitions, but it is a **separate code
path**. `onAddDetail` is not called and not changed.
`setEntryStep('treatment')` matters: `showChrome` is always on, so the control is visible
during the **Lab** wizard step too. Confirming there returns to the treatment step.
> Accepted consequences:
> - Tapping Add and then 🎤 leaves behind the blank chip that Add created. It carries the
> usual trash affordance.
> - Dictating into an existing detail is not supported in v1 — voice always makes a new
> one.
### Render policy
Three different reasons for "no", rendered differently:
| Reason | Condition | Render |
|---|---|---|
| **Technical** | no voice profile for the locale; no `MediaRecorder` | segment **absent** — the control is byte-for-byte today's plain Add button |
| **Commercial** | `Plan.features.voiceTreatmentEntry` false | segment **absent** — ⚠ *not enforced in v1*, see §8 |
| **Contextual** | `!canEdit \|\| disabled` — past day, read-only load, no `TAB_TREATMENT_EDIT` | both segments render and **disable together**, like the chip's trash (`disabled:opacity-40 disabled:cursor-not-allowed`) |
"This feature isn't yours" and "not right now" are different statements. Absence avoids a
permanently dead control; disabling avoids the button resizing as the day strip moves.
`TAB_TREATMENT_EDIT` is resolved via `common/membership-permissions.ts`, never by reading
`membership.permissions` directly.
### Component API
`TreatmentDetailsEditor` gains exactly **one** optional prop. All behaviour —
`MediaRecorder`, the API call, error state — lives in `lib/voice/useVoiceCapture.ts`
(hooks belong in `lib/` per `AGENTS.md`) and is owned by `TreatmentWorkspace`. The editor
stays presentational and renders both the segment and the recording bar with its own
classes, keeping the segmented styling beside the chip pattern it mirrors.
```ts
/** Omit when voice is unavailable — the Add button then renders unsplit. */
voice?: {
phase: 'idle' | 'recording' | 'processing';
elapsedMs: number;
level: number; // 0..1, for the meter
maxMs: number | null; // v1: 120_000 (2 min). null = uncapped, supported but not default
onStart: () => void;
onStop: () => void;
onCancel: () => void;
};
```
`voice === undefined` **is** the absent state above — availability is expressed by
presence rather than a separate flag, so the two cannot disagree.
### Accessibility
- Mic segment carries both `title` and `aria-label`, like the chip's trash, and its label
changes with `phase`.
- Announce **phase transitions** via a `role="status" aria-live="polite"` region — the
same idiom as the existing autosave status line. Do not put `aria-live` on the ticking
timer.
- Focus ring is `focus-visible:ring-inset` tinted primary when idle, red while recording.
---
## 3. Architecture
Two sequential stages, both server-side. The vendor API key never reaches the browser.
```
browser ──audio(base64)──► POST /treatments/voice-extract
┌─────────┴─────────┐
│ 1. AsrProvider │ audio + locale hint → transcript
└─────────┬─────────┘
┌─────────┴─────────┐
│ 2. ExtractionProv │ transcript + catalog + ctx → VoiceIntent
└─────────┬─────────┘
┌─────────┴─────────┐
│ 3. resolvers │ intents → FDI codes, ISO date
│ (pure, tested) │
└─────────┬─────────┘
VoiceExtractionResult (resolved values + the intents
that produced them, for display)
```
**Why intents and not final values:** the model never emits an FDI code and never does
calendar arithmetic. It emits what it *heard*; deterministic, unit-tested code decides
what that means. This is what makes the two highest-consequence mappings — quadrant
mirroring and Jalali conversion — testable instead of hopeful.
### Endpoint
`POST /treatments/voice-extract`
- Guards: `JwtAuthGuard` + `ClinicOrgGuard`.
- Service-level check of `TAB_TREATMENT_EDIT`. The plan flag is **not** checked in v1
(§8).
- Per-user throttle via `@nestjs/throttler` (present in `package.json`, currently wired
nowhere in `src/` — this is its first use, so the module must be registered in
`app.module.ts`). Configurable; v1 default **6 requests / 60s per user**
(`VOICE_THROTTLE_LIMIT`, `VOICE_THROTTLE_TTL`). A human cannot approach that — a
recording plus processing takes ten seconds at minimum — so it is purely an abuse and
runaway-loop guard, which is doing more work than usual given v1 is ungated and
uncapped (open item 14).
- Body: base64 audio + declared format + `clientTodayIso` + IANA `timeZone` +
`treatmentDetailId`/`clientId` for context.
`clientTodayIso` and `timeZone` come from the client per the existing house rule
(`AGENTS.md`): never derive the clinic's civil day from `Date#getDay()` on the UTC
server. Reuse `common/zoned-civil-time.ts`.
---
## 4. Provider registry
ASR and extraction are **separate swappable roles**. They will not come from the same
vendor for every locale.
```ts
interface AsrProvider {
transcribe(audio: AudioInput, localeHint: string): Promise<{ text: string; usage: AsrUsage }>;
}
interface ExtractionProvider {
extract(transcript: string, catalog: CatalogPrompt, ctx: ExtractionContext): Promise<VoiceIntent>;
}
```
Resolved through a registry keyed by locale.
At launch **all three locales use the same profile** — OpenRouter with
`openai/whisper-1` for ASR. The per-locale indirection is kept anyway, because the
locale is the axis along which this is most likely to diverge: Persian ASR is the
weakest link (§11), and swapping only `fa` to a Persian-specialist vendor must not be a
code change.
```
VOICE_PROFILE_FA = openrouter:openai/whisper-1 | openrouter:<llm-model>
VOICE_PROFILE_EN = openrouter:openai/whisper-1 | openrouter:<llm-model>
VOICE_PROFILE_NL = openrouter:openai/whisper-1 | openrouter:<llm-model>
```
The `language` hint is the profile's locale as ISO-639-1 — `fa`, `en`, `nl` — not a
constant.
Reachability is a property of **where you deploy**, not of the code. The same image
serves an Iran-hosted instance with a domestic `fa` profile and a Europe-hosted instance
with an OpenRouter profile; only config differs. A locale with no configured profile has
no microphone button at all (§2).
> The frontend must learn which locales are enabled **from the API**, not from a
> `NEXT_PUBLIC_*` var — those are baked in at build time, so an env-var approach would
> make enabling a locale require rebuilding and repushing the frontend image.
### v1 provider: OpenRouter
Both roles, one API key.
**ASR**`POST https://openrouter.ai/api/v1/audio/transcriptions`
```json
{
"model": "openai/whisper-1",
"input_audio": { "data": "<base64>", "format": "webm" },
"language": "fa"
}
```
- Accepted formats: WAV, MP3, FLAC, M4A, OGG, **WebM**, **AAC**.
Chrome/Android `webm/opus` and Safari/iPad `mp4/aac` both go through unmodified —
**no transcode dependency is required**.
- Limits: 25 MB; 60s upstream *processing* timeout. The 2-minute recording cap (§2) sits
comfortably inside both.
- Request shape **verified** against OpenRouter's STT docs: base64 JSON `input_audio` is
the documented primary path (multipart `file` is the OpenAI-compatible alternative), and
both default model slugs exist on the live models API.
- The cap is enforced twice: the client's reported `durationMs`, and again against the
vendor's own `usage.seconds` — the client's figure is a claim, not enforcement. The
server allows a 2s tolerance, because the client measures length *after* the recorder
stops and a recording that runs to the cap always reports slightly over it.
- Response: `{ text, usage: { seconds, total_tokens, cost } }`.
- Price: `openai/whisper-1` is **$0.006/minute, billed to the nearest second** → $0.002 for
a typical 20s utterance, **$0.012 at the 2-minute cap**. OpenRouter forwards this model
directly to OpenAI.
**Extraction** — OpenRouter chat completions with a JSON schema constraining the output
to `VoiceIntent`. Model id is config (`VOICE_LLM_MODEL`).
**Chosen: `google/gemini-3.7-flash`.**
Cost is deliberately *not* the selection axis. The call is ~1,250 input tokens (system
prompt + catalog labels + JSON schema + linked labs + transcript) and ~200 output, so the
whole candidate field spans about one cent per recording:
| Model | in / out per M tokens | ≈ per call |
|---|---|---|
| `google/gemini-3.7-flash` | $0.375 / $1.875 | $0.0008 |
| `qwen/qwen3.8-27b` | $0.45 / $3.20 | $0.0012 |
| `qwen/qwen3.8-max` | $2 / $6 | $0.0037 |
| `openai/gpt-5.6-terra` | $2 / $12 | $0.0049 |
| `anthropic/claude-opus-5` | $5 / $25 | $0.0113 |
Select for Persian comprehension and reliable constrained JSON instead. Gemini Flash wins
on the reasoning that this task is **not** reasoning-heavy — read a short sentence, pick
codes from a supplied closed list, emit small JSON — and it is simultaneously the cheapest
and lowest-latency candidate, which matters on a pipeline already at 310s.
**Escalation path if Persian comprehension proves weak** on real transcripts (item 1):
`qwen/qwen3.8-max` (leads current multilingual rankings, ~4.6× the cost) and then
`anthropic/claude-opus-5` as the accuracy ceiling. Both are a config change —
`VOICE_LLM_MODEL` — not a code change. Worth knowing when judging that ranking evidence:
it is marked provisional and was measured on the previous Qwen generation.
---
## 5. Extraction contract
The model returns intents only. Illustrative shape:
```ts
type VoiceIntent = {
treatmentType: string | null; // catalog code, from the supplied closed list
teeth: ToothIntent[];
connectedSpans: { from: ToothIntent; to: ToothIntent }[];
comment: string | null;
prosthesis: {
defaultType: string | null; // catalog code
overrides: { tooth: ToothIntent; type: string }[];
} | null;
labId: string | null; // must be one of the supplied linked-lab ids
labMatchExact: boolean;
due: DueIntent | null;
};
type ToothIntent =
| { kind: 'explicit'; fdi: string; spoken: string }
| { kind: 'positional'; arch: 'upper' | 'lower';
side: 'patient_right' | 'patient_left'; position: number; spoken: string };
type DueIntent =
| { kind: 'weekday'; weekday: string; which: 'this' | 'next' }
| { kind: 'offset'; unit: 'day' | 'week' | 'month'; amount: number }
| { kind: 'jalali'; jy: number; jm: number; jd: number }
| { kind: 'gregorian'; y: number; m: number; d: number };
```
Every code-valued field is constrained to a **closed list supplied in the prompt**:
- Treatment types and prosthesis types come from `CatalogLabelService` in the actor's
locale, so the model sees "پروتز" and "زیرکونیا مونولیتیک" as the spoken forms of
`prosthesis` and `monolithic_zirconia` rather than being asked to translate. Catalog
entities store a stable `code` and no label — never hardcode a label.
- Lab candidates are the clinic's linked labs only (`OrganizationLink`), passed as
`{ id, name }`. The model may return one of those ids or `null`, nothing else.
Every unresolved or rejected item is reported, never silently dropped.
---
## 6. Resolvers
Both live in `backend/src/`, pure and Jest-covered. The frontend has **no test runner**
(no jest/vitest, zero spec files) — putting them there would forfeit the testability
that justified this whole design.
### `resolveToothIntent()`
- Owns the patient-right convention in exactly one place: `upper + patient_right`
quadrant 1, `upper + patient_left` → 2, `lower + patient_left` → 3,
`lower + patient_right` → 4. This is the mirroring bug, and it becomes a unit test.
- Rejects out-of-range positions rather than clamping. Position 9 is unresolved, never 8.
- Permanent dentition only — FDI 1118/2128/3138/4148, matching
`FDI_UPPER_LEFT_TO_RIGHT` / `FDI_LOWER_LEFT_TO_RIGHT`. Deciduous references
("دندان شیری") must resolve to **unresolved**, never snap to a permanent tooth.
- **Locale-neutral by construction.** `ToothIntent` carries `arch`/`side`/`position`, not
words, so the resolver needs no per-locale branches. The locale-specific part is the
*prompt*: each enabled locale needs its own spoken tooth vocabulary (`شش بالا راست`,
`upper right six`, `rechtsboven zes`).
- **English carries a numbering hazard the other locales do not.** A clinician trained
under Universal numbering says "tooth number 14" and means a different tooth than FDI
14. `nl` is safe — the Netherlands uses FDI — but `en` is not. The `en` prompt must
therefore not accept a bare two-digit number as `explicitFdi` without the speaker
having made the notation explicit; ambiguous English numerals resolve to
**unresolved**. See §11.
### `resolveDueDate()`
- Takes `clientTodayIso` + IANA `timeZone`; reuses `common/zoned-civil-time.ts`.
- Week start is **per locale**, because "next Thursday" is week-relative: `fa` starts
Saturday, `en` and `nl` start Monday. Hardcoding Saturday put an en/nl clinician's
deadline a week out. One place (`weekStartForLocale`), tested in both directions.
- `'this'` is occurrence-anchored (soonest strictly-future, never resolves into the past);
`'next'` is week-anchored. A missing qualifier is read as `'this'` — a bare weekday
carries none, and failing would discard a real spoken deadline.
- Jalali conversion is arithmetic, not inference: port `jalaliToGregorian` and
`toLatinDigits` from `frontend/src/lib/i18n/persianCalendar.ts` into
`backend/src/common/jalali.ts` with a spec. It is dependency-free integer math
(~60 relevant lines) and the calendar does not change, so the duplication is stable.
### Group / prosthesis rules
- Connected spans validate through the shipped helpers — `areArchNeighbors`, `sameArch`,
`teethBetweenInclusive`. **Never a 1-tooth connected group.** Anything invalid degrades
to singles and is flagged on the review sheet.
- Prosthesis: expand `defaultType` across all teeth, then apply per-tooth overrides.
- **All-or-nothing.** `assertCompleteToothProsthesisMap` requires every tooth on a
`prosthesis` detail to carry a `prosthesisTypeCode` or the send throws
`TREATMENT_TOOTH_PROSTHESIS_INCOMPLETE`. So if even one tooth ends untyped, the
prosthesis row is marked incomplete and stays unticked — the unshippable state surfaces
at review, where it is cheap, not at dispatch minutes later on another screen.
- **Apply order is teeth → groups → prosthesis**, so `pruneToothProsthesisForGroups`
behaves.
---
## 7. Review sheet
**Presentation:** a **modal** on desktop; a **full-screen overlay** on mobile.
> ⚠ On mobile this must be an overlay rendered in place — **not** a Next.js route. A real
> navigation unmounts `TreatmentWorkspace` and destroys the in-progress draft. This is the
> same constraint the realtime soft-refresh already lives under: never remount the
> treatment form, never clear a draft.
- Renders the transcript, then one row per extracted field in the app's own vocabulary:
translated catalog labels, and a mini FDI chart for the teeth rather than a list of
numbers.
- Each row has a checkbox. Ticked rows apply; nothing else is touched. Confirm is also
what creates the new detail — see §2.
- Rows default to ticked **except**:
- the lab row when `labMatchExact` is false — shipping to a lab always requires a
deliberate tick;
- any row carrying an unresolved item or an incomplete prosthesis map.
- Unresolved items are shown with what was heard ("دندان شیری — بازشناسی نشد"), so the
clinician can see what the system did not understand.
- RTL-safe: logical `text-start` / `text-end` only, never `text-left`/`text-right`.
Dates via `lib/i18n/format.ts`.
---
## 8. Gating and configuration
Gate = `TAB_TREATMENT_EDIT``canEditTreatmentForDay` ∧ configured locale profile ∧
`Plan.features.voiceTreatmentEntry`. How each failing condition *renders* is in §2.
`Plan.features` is a `Json` column that already exists on the `Plan` model and is seeded
as `{}` for all five plans — and is **read nowhere in `backend/src/`**. Voice is its first
consumer, so:
- No migration is needed.
- The read should go through a small generic helper, since this establishes the pattern
for every future flag.
- Voice is metered vendor spend, which is why the gate is commercial (plan) rather than a
new `TAB_*` permission — a clinician who can already edit the form gains no capability
from voice, only speed.
The per-user throttle is a separate, non-commercial abuse control.
### v1 ships ungated
**The plan flag is designed but not enforced in v1** — voice is open to every clinic user
who can edit treatments, in every configured locale. `Plan.features.voiceTreatmentEntry`
and the availability API stay documented here as the intended gate, deferred rather than
dropped, so turning them on later is additive.
Consequence to accept deliberately: with no plan gate and no duration cap (§2), the
per-user throttle is the **only** control on metered vendor spend. See open item 14.
---
## 9. Errors
Per the three-layer contract: a code in `common/errors/error-codes.ts`, the throw site,
and an `errors.X` key in **all three** of `frontend/messages/{en,fa,nl}.json`. Never a raw
English Nest exception for a user-facing failure.
| Code | When |
|---|---|
| `VOICE_MIC_DENIED` | browser permission refused — **client-side only**: needs the `errors.X` key in all three message files, but no `ErrorCode` entry and no throw site |
| `VOICE_CLIP_TOO_LONG` | over `maxMs` (server-side re-check), or over vendor limits |
| `VOICE_UNSUPPORTED_FORMAT` | recorder produced a container the profile rejects |
| `VOICE_ASR_FAILED` | transcription stage failed |
| `VOICE_EXTRACT_FAILED` | transcript obtained, structuring failed |
| `VOICE_NOTHING_RECOGNIZED` | empty or unusable transcript |
| `VOICE_NOT_AVAILABLE` | no profile for locale (v1); plan flag off, once enforced |
| `VOICE_RATE_LIMITED` | throttle |
**Transcript salvage:** when ASR succeeded and only extraction failed, the response still
carries the transcript and the failure dialog offers *"افزودن به یادداشت"*. That action
**creates a new detail with only `comment` set to the transcript** — everything else left
at `newDetail()` defaults. The words were captured and paid for; only the structure was
lost.
This keeps the feature's one invariant intact: **voice never writes into an existing
detail.** Dictating into an already-filled detail is a separate, later feature with its
own voice-to-text control scoped to that field (§1, out of scope).
It also does not bypass the confirmation rule — the dialog shows the transcript, and the
dentist taps to accept it. That review matters, because a raw transcript carries ASR
errors and may contain the patient's spoken name, and `comment` is persisted (§10).
---
## 10. Data handling
- Audio is held **in memory for the request only**. Never written to disk, never a Prisma
row. Note this is deliberately unlike treatment attachments, which do persist to
`backend/uploads/treatments`.
- The transcript goes to the browser for the review sheet and dies with it.
- The `comment` field persists a cleaned version of what was said — that is legitimate
clinical record-keeping and is the only durable trace.
- Telemetry is **structured and patient-free**: clip duration, which fields resolved,
unresolved count, vendor latency, `usage.cost`, outcome (applied / discarded / failed),
locale. Never the transcript, never audio, never a patient identifier.
The **destination is an open question** — this repo has no metrics infrastructure yet
(open item 9). Emit it as structured log lines in the interim so the fields exist and
can be routed later without changing call sites.
**Cancelling aborts the in-flight vendor call** via `AbortController`, rather than letting
it settle and discarding the result. Note this reduces spend but does not eliminate it:
work already performed upstream may still be billed.
Recordings are clinical descriptions of identifiable patients leaving the server for a
third party. Confirm the provider's retention and training policy in writing before
enabling this for real clinics.
---
## 11. Open items
1. **Persian ASR accuracy on tooth numbers is unmeasured, and it decides the feature.**
Everything downstream assumes a usable transcript; no design choice above compensates
for a bad one. Run this before writing feature code.
*Why tooth numbers specifically, not general accuracy:* a transcript can score well on
WER and still be useless here, because the errors land on the digits.
`چهار` (4) / `چهارده` (14) / `چهل` (40) differ by one syllable. FDI spoken as
`یک چهار` may return as `۱۴`, `14`, or `یک چهار`. Persian and Latin digit scripts mix
within one transcript. Jargon is loanwords (`زیرکونیا`, `پرسلن فیوزد تو متال`,
`اینله`/`آنله`) and clinicians code-switch into English mid-sentence. Suction and
handpiece run in the background.
*Protocol:* ~25 utterances from a dentist reading a script covering explicit FDI,
quadrant-relative phrasing, bridges, prosthesis types, due dates and notes — recorded
on the real device, ideally once quiet and once with the operatory running. POST each
to `/api/v1/audio/transcriptions` with `language: "fa"`. Score **per-tooth-reference
accuracy** (of every tooth spoken, how many survive recoverably?) and jargon
recognition separately. Notes accuracy barely matters. ~6 minutes of audio ≈ **$0.04**.
*Decision it drives:* if tooth accuracy holds, build. If not, the fix is not prompt
tuning — it is pointing the `fa` ASR slot at a Persian-specialist vendor while `en`/`nl`
keep `whisper-1`. That is a config change precisely because of decision 23.
*Unverified lead for that fallback:* recent Persian-ASR benchmark work reports
Qwen3-Omni as the strongest open Persian ASR. Not confirmed as available on
OpenRouter's transcription endpoint — check before relying on it.
*Byproduct:* the recordings become the fixture corpus for item 2.
2. ~~**`VOICE_LLM_MODEL`**~~**resolved:** `google/gemini-3.7-flash` (§4). Still worth
validating on real whisper output during item 1, since clean transcripts flatter every
model; the escalation path if Persian comprehension disappoints is documented in §4 and
is a config change.
3. ~~**Production reachability**~~**resolved:** confirmed reachable from the Iranian
production host. The per-locale registry is retained regardless (decision 23), so the
`fa` profile can still be repointed at a domestic vendor if item 1 goes badly.
4. ~~**Provider retention/training policy**~~**resolved:** restricted via OpenRouter's
account-level privacy/data-policy settings. Note this is an *account* setting, not a
per-request one: re-verify it if the API key or the OpenRouter account changes, and
remember `whisper-1` is forwarded to OpenAI, so the effective policy is OpenRouter's
plus that provider's.
5. **English tooth numbering is unresolved as a product question.** Enabling `en` means
deciding what "tooth number 14" means when the speaker's notation is unknown —
Universal or FDI. The spec's current answer is to refuse ambiguous bare numerals in
`en`, which is safe but will feel broken to a US-trained clinician. Options are: refuse
(current), an org-level notation preference, or restricting `en` to quadrant-relative
phrasing. Decide before `en` ships to a real clinic; `fa` and `nl` are unaffected.
6. **`nl` and `en` have no spike data.** The Persian spike (item 1) should be repeated per
locale before that locale's mic is enabled for real users — same protocol, same
scoring, different speaker.
7. ~~**Transcript salvage has no target**~~**resolved:** salvage creates a **new
detail** with only `comment` set to the transcript (§9). Preserves the invariant that
voice never writes into an existing detail; dictating into a filled detail becomes a
separate later feature with its own field-scoped control.
8. ~~**Throttle limits**~~**resolved:** configurable, v1 default **6 requests / 60s per
user** (§3). Unreachable by a human; a pure abuse guard.
9. **Telemetry has no sink.** §10 defines exactly what to record but not where it goes —
this repo has no metrics or analytics infrastructure yet. Deliberately deferred until
it does. Interim: structured log lines, so the fields exist and can be routed later
without touching call sites.
10. ~~**Review sheet modality**~~**resolved:** modal on desktop, full-screen **overlay**
on mobile (§7). Not a route — navigating would unmount `TreatmentWorkspace` and destroy
the draft.
11. ~~**Cancel and in-flight requests**~~**resolved:** abort via `AbortController` to
limit spend (§10). Upstream work already performed may still be billed.
12. ~~**Recording duration cap**~~**resolved:** **2 minutes**, configurable via `maxMs`
(§2). Still worth timing a realistic worst-case prosthesis dictation during item 1 to
confirm 2 minutes is comfortable rather than tight.
13. ~~**Availability API**~~**resolved for v1:** voice ships **open to everyone** with a
configured locale profile. No plan check, no availability endpoint. The `Plan.features`
design in §8 is deferred, not dropped.
### New
14. **Cost exposure is bounded but ungated in v1.** With the 2-minute cap (12) and the
6/60s throttle (8), worst case is 12 audio-minutes per user-minute ≈ **$0.072/min**, or
~$4.30 per hour of sustained abuse by one user — bounded, not free. Normal use is far
below this: a clinician doing 60 recordings a day at ~20s each costs about **$0.18/day**.
What remains open is that there is **no per-organization limit at all**, because v1
ships ungated (13). Worth deciding what to watch and at what number to react.
Mitigations, already designed and each a config or flag change: enforce the
`Plan.features` gate, lower `maxMs` or the throttle, or add an org-level monthly minute
budget.
---
## 12. Verification
- `cd backend && npm test` — new suites for `resolveToothIntent` (quadrant mapping in all
four quadrants, out-of-range rejection, deciduous → unresolved), `resolveDueDate`
(per-locale week start, "this" vs "next" weekday, Jalali leap year, month-end), the
Jalali port, prosthesis expansion + completeness, and connected-span validation.
- `cd backend && npm run build` — cross-cutting backend gate.
- `cd frontend && npx tsc --noEmit` — frontend gate.
- Manual: fa locale, editable day, prosthesis detail with a bridge, dispatch to a linked
lab. Then specifically:
- **past day** → both segments disabled, control still split (not absent);
- **locale with no profile** → control renders unsplit, identical to today;
- **fa vs en** → mic sits at the logical end in both, on the same side as the chip's
trash;
- **cancel mid-recording** → chip strip unchanged, no orphan detail;
- **confirm** → always appends a new chip, whatever the active detail contains;
- **Add half** → behaves exactly as it did before this change;
- **tap 🎤 during the Lab wizard step** → confirm returns to the treatment step;
- no layout shift in the header row on record start, stop, or the 2:00 auto-stop;
- **hold past 2:00** → auto-stops and proceeds to processing, not an error;
- **cancel during processing** → the vendor request is actually aborted;
- **review sheet on mobile** → full-screen overlay; closing it leaves the draft intact.
---
## 13. Decision log
Settled in a grilling session on 2026-08-20.
| # | Question | Decision |
|---|---|---|
| 1 | Scope | Everything including lab dispatch |
| 2 | AI supply chain | Domestic provider originally; OpenRouter for v1, registry keeps both open |
| 3 | Apply model | Review sheet, then apply |
| 4 | Speech → FDI | LLM emits intent, code resolves |
| 5 | Cardinality | One detail per recording |
| 6 | Lab destination | Closed list of linked labs, explicit confirm, unticked when inexact |
| 7 | Due date | Intent + deterministic resolver |
| 8 | Resolver location | Backend, Jalali math ported |
| 9 | Prosthesis | Default type + overrides, all-or-nothing |
| 10 | Retention | Discard audio and transcript, non-PHI telemetry only |
| 11 | Capture | Tap to start/stop, hard cap (see 27) |
| 12 | Failure UX | Stage-aware codes, transcript salvage |
| 13 | Locales | Provider registry per locale; all three locales enabled |
| 14 | Reachability | Registry now, slots filled per deployment |
| 23 | ASR model | `openai/whisper-1` for **every** locale; registry kept so `fa` can diverge |
| 24 | Extraction model | **`google/gemini-3.7-flash`**; escalation path documented in §4 |
| 25 | Salvage target | Creates a new detail with only `comment` set — voice never writes into an existing detail |
| 26 | Throttle | Configurable; v1 default 6 requests / 60s per user |
| 27 | Duration cap | **2 minutes**, configurable via `maxMs` |
| 28 | Review sheet | Modal on desktop, full-screen overlay (not a route) on mobile |
| 29 | Cancel | Aborts the in-flight vendor call |
| 30 | v1 gating | Open to everyone; `Plan.features` gate deferred, not dropped |
| 15 | Gating | `Plan.features` flag — its first consumer |
UI placement settled in a second grilling session on 2026-08-20.
| # | Question | Decision |
|---|---|---|
| 16 | Mic action | Independent record action; the Add half's logic is untouched |
| 17 | Record UI | Mic segment toggles ▶/■; inline bar below the header row |
| 18 | Unavailable | Absent for technical + commercial; disabled for contextual |
| 19 | Component API | One optional `voice` object prop; `undefined` means absent |
| 20 | Side | Logical end (`border-s`), exactly like the chip's trash |
| 21 | Creation | On confirm, never on tap |
| 22 | Creation rule | Confirm **always** appends a new detail — no blank-reuse guard |

View File

@@ -894,7 +894,29 @@
"toothAria": "FDI tooth {fdi}",
"toothSelectedSuffix": ", selected",
"sentToAt": "Sent to {orgName} at {datetime}",
"fallbackOrgName": "organization"
"fallbackOrgName": "organization",
"voiceStart": "Record treatment",
"voiceStop": "Stop recording",
"voiceCancel": "Cancel",
"voiceProcessing": "Reading the recording…",
"voiceReviewTitle": "Check what was understood",
"voiceNothingExtracted": "Nothing usable was picked up from that recording.",
"voiceProsthesisIncomplete": "No prosthesis type for {teeth} — the case cannot be sent until every tooth has one.",
"voiceLabInexact": "The spoken name only partly matched this lab. Confirm before sending.",
"voiceNotUnderstood": "Not understood",
"voiceDiscard": "Discard",
"voiceApply": "{count, plural, one {Apply # field} other {Apply # fields}}",
"voiceUnresolved": {
"not_permanent_tooth": "not a permanent tooth",
"position_out_of_range": "not a valid tooth position",
"tooth_missing_quadrant": "quadrant not said — e.g. “upper right two”",
"malformed": "could not be read",
"span_not_same_arch": "a bridge cannot span both jaws",
"unknown_catalog_code": "not in this clinics list",
"tooth_not_selected": "that tooth is not part of this detail",
"invalid_date": "not a usable date"
},
"voiceFailed": "Voice entry failed. Please try again."
},
"organizations": {
"loadingOrganization": "Loading organization...",
@@ -1235,6 +1257,13 @@
"LAB_CASE_START_INCOMPLETE": "Complete teeth and prosthesis types before starting this case.",
"LAB_CASE_CLIENT_REQUIRED": "Enter a clinic name or a patient name for this case.",
"BAD_REQUEST": "The request could not be processed.",
"INTERNAL_ERROR": "Something went wrong on our end. Please try again later."
"INTERNAL_ERROR": "Something went wrong on our end. Please try again later.",
"VOICE_MIC_DENIED": "Microphone access was blocked. Allow it in your browser settings and try again.",
"VOICE_NOT_AVAILABLE": "Voice entry is not available for this language yet.",
"VOICE_CLIP_TOO_LONG": "That recording is too long. Please keep it under two minutes.",
"VOICE_ASR_FAILED": "Could not turn the recording into text. Please try again.",
"VOICE_EXTRACT_FAILED": "Could not read the treatment details from the recording.",
"VOICE_NOTHING_RECOGNIZED": "No speech was recognised. Check the microphone and try again.",
"VOICE_RATE_LIMITED": "Too many recordings in a short time. Please wait a moment."
}
}

View File

@@ -895,7 +895,29 @@
"toothAria": "دندان FDI {fdi}",
"toothSelectedSuffix": "، انتخاب شده",
"sentToAt": "ارسال به {orgName} در {datetime}",
"fallbackOrgName": "سازمان"
"fallbackOrgName": "سازمان",
"voiceStart": "ثبت گفتاری درمان",
"voiceStop": "توقف ضبط",
"voiceCancel": "لغو",
"voiceProcessing": "در حال پردازش گفتار…",
"voiceReviewTitle": "بررسی آنچه دریافت شد",
"voiceNothingExtracted": "از این ضبط چیز قابل استفاده‌ای برداشت نشد.",
"voiceProsthesisIncomplete": "برای {teeth} نوع پروتز مشخص نشده — تا زمانی که همه دندان‌ها نوع داشته باشند، کیس ارسال نمی‌شود.",
"voiceLabInexact": "نام گفته‌شده فقط تا حدی با این لابراتوار مطابقت داشت. پیش از ارسال تأیید کنید.",
"voiceNotUnderstood": "شناسایی نشد",
"voiceDiscard": "انصراف",
"voiceApply": "{count, plural, one {اعمال # مورد} other {اعمال # مورد}}",
"voiceUnresolved": {
"not_permanent_tooth": "دندان دائمی نیست",
"position_out_of_range": "شماره دندان معتبر نیست",
"tooth_missing_quadrant": "بالا/پایین و چپ/راست گفته نشد — مثلاً «دو بالا راست»",
"malformed": "قابل خواندن نبود",
"span_not_same_arch": "بریج نمی‌تواند بین دو فک باشد",
"unknown_catalog_code": "در فهرست این مطب نیست",
"tooth_not_selected": "این دندان بخشی از این مورد نیست",
"invalid_date": "تاریخ قابل استفاده نیست"
},
"voiceFailed": "ثبت گفتاری انجام نشد. لطفاً دوباره تلاش کنید."
},
"organizations": {
"loadingOrganization": "در حال بارگذاری سازمان...",
@@ -1236,6 +1258,13 @@
"LAB_CASE_START_INCOMPLETE": "قبل از شروع پرونده، دندان‌ها و نوع پروتز را کامل کنید.",
"LAB_CASE_CLIENT_REQUIRED": "نام کلینیک یا نام بیمار را وارد کنید.",
"BAD_REQUEST": "درخواست قابل پردازش نبود.",
"INTERNAL_ERROR": "مشکلی در سرور رخ داد. لطفاً بعداً تلاش کنید."
"INTERNAL_ERROR": "مشکلی در سرور رخ داد. لطفاً بعداً تلاش کنید.",
"VOICE_MIC_DENIED": "دسترسی به میکروفون مسدود شده است. در تنظیمات مرورگر اجازه دهید و دوباره تلاش کنید.",
"VOICE_NOT_AVAILABLE": "ثبت گفتاری هنوز برای این زبان در دسترس نیست.",
"VOICE_CLIP_TOO_LONG": "مدت ضبط بیش از حد است. لطفاً کمتر از دو دقیقه صحبت کنید.",
"VOICE_ASR_FAILED": "تبدیل گفتار به متن انجام نشد. لطفاً دوباره تلاش کنید.",
"VOICE_EXTRACT_FAILED": "اطلاعات درمان از روی گفتار استخراج نشد.",
"VOICE_NOTHING_RECOGNIZED": "گفتاری شناسایی نشد. میکروفون را بررسی کنید و دوباره تلاش کنید.",
"VOICE_RATE_LIMITED": "تعداد ضبط‌ها در بازه کوتاه زیاد بود. کمی صبر کنید."
}
}
}

View File

@@ -894,7 +894,29 @@
"toothAria": "FDI-tand {fdi}",
"toothSelectedSuffix": ", geselecteerd",
"sentToAt": "Verzonden naar {orgName} op {datetime}",
"fallbackOrgName": "organisatie"
"fallbackOrgName": "organisatie",
"voiceStart": "Behandeling inspreken",
"voiceStop": "Opname stoppen",
"voiceCancel": "Annuleren",
"voiceProcessing": "Opname wordt gelezen…",
"voiceReviewTitle": "Controleer wat is begrepen",
"voiceNothingExtracted": "Uit deze opname is niets bruikbaars opgepikt.",
"voiceProsthesisIncomplete": "Geen prothesetype voor {teeth} — de casus kan pas worden verstuurd als elk element er een heeft.",
"voiceLabInexact": "De uitgesproken naam kwam slechts deels overeen met dit lab. Bevestig voor verzending.",
"voiceNotUnderstood": "Niet begrepen",
"voiceDiscard": "Verwerpen",
"voiceApply": "{count, plural, one {# veld toepassen} other {# velden toepassen}}",
"voiceUnresolved": {
"not_permanent_tooth": "geen blijvend element",
"position_out_of_range": "geen geldige elementpositie",
"tooth_missing_quadrant": "kwadrant niet genoemd — bijv. “rechtsboven twee”",
"malformed": "kon niet worden gelezen",
"span_not_same_arch": "een brug kan niet over beide kaken lopen",
"unknown_catalog_code": "staat niet in de lijst van deze praktijk",
"tooth_not_selected": "dat element hoort niet bij dit onderdeel",
"invalid_date": "geen bruikbare datum"
},
"voiceFailed": "Spraakinvoer is mislukt. Probeer het opnieuw."
},
"organizations": {
"loadingOrganization": "Organisatie laden...",
@@ -1235,6 +1257,13 @@
"LAB_CASE_START_INCOMPLETE": "Vul tanden en prothesetypes in voordat u deze case start.",
"LAB_CASE_CLIENT_REQUIRED": "Voer een klinieknaam of een patiëntnaam in voor deze case.",
"BAD_REQUEST": "Het verzoek kon niet worden verwerkt.",
"INTERNAL_ERROR": "Er is iets misgegaan aan onze kant. Probeer het later opnieuw."
"INTERNAL_ERROR": "Er is iets misgegaan aan onze kant. Probeer het later opnieuw.",
"VOICE_MIC_DENIED": "Microfoontoegang is geblokkeerd. Sta dit toe in uw browserinstellingen en probeer opnieuw.",
"VOICE_NOT_AVAILABLE": "Spraakinvoer is nog niet beschikbaar voor deze taal.",
"VOICE_CLIP_TOO_LONG": "Die opname is te lang. Houd het onder twee minuten.",
"VOICE_ASR_FAILED": "De opname kon niet naar tekst worden omgezet. Probeer het opnieuw.",
"VOICE_EXTRACT_FAILED": "De behandelgegevens konden niet uit de opname worden gelezen.",
"VOICE_NOTHING_RECOGNIZED": "Er is geen spraak herkend. Controleer de microfoon en probeer opnieuw.",
"VOICE_RATE_LIMITED": "Te veel opnames in korte tijd. Wacht even."
}
}
}

View File

@@ -0,0 +1,61 @@
import type { FdiToothId } from '@/types/treatment';
import type { VoiceApplySelection, VoiceExtractionResult } from '@/types/voice';
/** Which rows the review sheet renders at all — a row with nothing extracted is noise. */
export function voiceRowAvailability(result: VoiceExtractionResult) {
return {
treatmentType: result.treatmentType != null,
teeth: result.teeth.length > 0,
comment: Boolean(result.comment?.trim()),
prosthesis: result.prosthesis != null,
lab: result.labId != null,
dueDate: result.dueDate != null,
};
}
/**
* Which rows start ticked.
*
* Everything available ticks itself, with two deliberate exceptions:
*
* - **lab, when the name only approximately matched.** Shipping a case to a lab is the one
* extracted value whose error leaves the building, so it always requires a deliberate tick.
* - **prosthesis, when the map is incomplete.** A prosthesis detail with an untyped tooth
* cannot ship at all, so applying it would just move the failure to dispatch.
*/
export function initialVoiceSelection(result: VoiceExtractionResult): VoiceApplySelection {
const available = voiceRowAvailability(result);
return {
treatmentType: available.treatmentType,
teeth: available.teeth,
comment: available.comment,
prosthesis: available.prosthesis && result.prosthesis?.complete === true,
lab: available.lab && result.labMatchExact,
dueDate: available.dueDate,
};
}
/** How many rows will actually be applied — drives the confirm button's label. */
export function countSelected(selection: VoiceApplySelection): number {
return Object.values(selection).filter(Boolean).length;
}
/** Teeth that are part of a bridge, for the read-only chart's connection marks. */
export function connectedTeethFromResult(result: VoiceExtractionResult): Set<FdiToothId> {
const connected = new Set<FdiToothId>();
for (const group of result.toothSelectionGroups) {
if (group.kind !== 'connected') continue;
for (const tooth of group.teeth) connected.add(tooth);
}
return connected;
}
/**
* Whether the sheet has anything worth showing.
*
* A recording that produced nothing usable should say so plainly rather than present an
* empty form of checkboxes.
*/
export function hasAnythingToApply(result: VoiceExtractionResult): boolean {
return Object.values(voiceRowAvailability(result)).some(Boolean);
}

View File

@@ -2,12 +2,14 @@
import { useEffect, useRef, type ReactNode, type RefObject } from 'react';
import { useTranslations } from 'next-intl';
import { Trash2 } from 'lucide-react';
import { Mic, Square, Trash2 } from 'lucide-react';
import { Button } from '@/components/ui/shared/Button';
import { Dropdown } from '@/components/ui/shared/Dropdown';
import { formatDetailChipLabel } from '@/components/treatment/detailChipLabel';
import { autosaveStatusClass, labBlockedBannerClass } from '@/components/treatment/treatmentStatusStyles';
import { TreatmentDetailAttachmentsStrip } from '@/components/ui/treatment/TreatmentDetailAttachmentsStrip';
import { VoiceRecordingBar } from '@/components/ui/treatment/VoiceRecordingBar';
import type { VoiceCaptureState } from '@/lib/voice/useVoiceCapture';
import type { TreatmentDetailDraft } from '@/types/treatment';
import type { TreatmentCatalogEntry } from '@/types/treatment-catalog';
import { treatmentTypeColor, treatmentTypeOptionStyle } from '@/components/shared/treatmentTypeDisplay';
@@ -42,6 +44,12 @@ interface TreatmentDetailsEditorProps {
stepper?: ReactNode;
/** Shown below type + chart + notes (e.g. Continue to lab). */
footer?: ReactNode;
/**
* Voice entry. Omit when unavailable — the Add button then renders unsplit, exactly as
* before this feature existed. Presence *is* the availability flag, so the two cannot
* disagree.
*/
voice?: VoiceCaptureState;
/** Dim the chart until a treatment type is chosen. */
chartLocked?: boolean;
chartLockMessage?: string;
@@ -65,6 +73,7 @@ export function TreatmentDetailsEditor({
onRemoveAttachment,
showChrome = true,
showFields = true,
voice,
chart,
stepper,
footer,
@@ -109,18 +118,31 @@ export function TreatmentDetailsEditor({
<div>
<h3 className="text-sm font-semibold text-text-primary">{t('detailsTitle')}</h3>
</div>
<Button
type="button"
variant="primary"
disabled={!canEdit || disabled}
onClick={onAddDetail}
fullWidth
className="sm:w-auto shrink-0"
>
{t('addDetail')}
</Button>
{voice ? (
<AddDetailWithVoice
addLabel={t('addDetail')}
startLabel={t('voiceStart')}
stopLabel={t('voiceStop')}
disabled={!canEdit || disabled}
onAddDetail={onAddDetail}
voice={voice}
/>
) : (
<Button
type="button"
variant="primary"
disabled={!canEdit || disabled}
onClick={onAddDetail}
fullWidth
className="sm:w-auto shrink-0"
>
{t('addDetail')}
</Button>
)}
</div>
{voice ? <VoiceRecordingBar voice={voice} /> : null}
<div className="flex flex-wrap gap-2">
{details.map((d, idx) => {
const detailLocked = isDetailLocked(d);
@@ -310,3 +332,82 @@ function NotesField({
</label>
);
}
/**
* "Add detail", split into two segments with the microphone at the logical end.
*
* Built like the detail chip's trash affordance in this same file — an
* `inline-flex items-stretch overflow-hidden rounded` wrapper holding two raw `<button>`s
* divided by `border-s` — rather than two shared `Button`s, which each hardcode their own
* rounding and would fight a segmented control.
*
* `border-s` puts the microphone at the *logical* end: visually right in en/nl, visually
* left in fa, on the same side as the chip's trash in both directions.
*
* The two halves share a wrapper and nothing else. Add keeps its exact existing
* behaviour; the microphone is an independent action that creates nothing until the
* clinician confirms.
*/
function AddDetailWithVoice({
addLabel,
startLabel,
stopLabel,
disabled,
onAddDetail,
voice,
}: {
addLabel: string;
startLabel: string;
stopLabel: string;
disabled: boolean;
onAddDetail: () => void;
voice: VoiceCaptureState;
}) {
const isRecording = voice.phase === 'recording';
const isBusy = voice.phase !== 'idle';
const micLabel = isRecording ? stopLabel : startLabel;
return (
<div
className={`
inline-flex w-full items-stretch overflow-hidden rounded-[var(--radius-md)]
bg-primary text-white shrink-0 sm:w-auto
${disabled ? 'opacity-60' : ''}
`}
>
<button
type="button"
onClick={onAddDetail}
disabled={disabled || isBusy}
className="
flex-1 px-4 py-2 text-sm font-medium transition-all duration-200
hover:opacity-90 focus:outline-none focus-visible:ring-2 focus-visible:ring-inset
focus-visible:ring-white/60
disabled:cursor-not-allowed disabled:opacity-60 disabled:hover:opacity-60
"
>
{addLabel}
</button>
<button
type="button"
onClick={isRecording ? voice.onStop : voice.onStart}
disabled={disabled || voice.phase === 'processing'}
title={micLabel}
aria-label={micLabel}
className={`
inline-flex items-center justify-center border-s border-white/25 px-3
transition-all duration-200 focus:outline-none focus-visible:ring-2
focus-visible:ring-inset focus-visible:ring-white/60
disabled:cursor-not-allowed disabled:opacity-60
${isRecording ? 'bg-red-600 hover:bg-red-700' : 'hover:opacity-90'}
`}
>
{isRecording ? (
<Square className="h-4 w-4 fill-current" aria-hidden />
) : (
<Mic className="h-4 w-4" aria-hidden />
)}
</button>
</div>
);
}

View File

@@ -30,6 +30,14 @@ import {
import { appointmentsApi } from '@/lib/api/appointments';
import { treatmentCatalogApi } from '@/lib/api/treatment-catalog';
import { prosthesisCatalogApi } from '@/lib/api/prosthesis-catalog';
import { voiceApi } from '@/lib/api/voice';
import { useVoiceCapture } from '@/lib/voice/useVoiceCapture';
import { VoiceReviewSheet } from '@/components/ui/treatment/VoiceReviewSheet';
import type {
VoiceApplySelection,
VoiceAvailability,
VoiceExtractionResult,
} from '@/types/voice';
import { treatmentsApi } from '@/lib/api/treatments';
import { notificationsApi } from '@/lib/api/notifications';
import { pickAutoAppointment } from '@/components/shared/treatmentSelection';
@@ -458,6 +466,11 @@ export function TreatmentWorkspace({
const [showWholeTreatmentPlan, setShowWholeTreatmentPlan] = useState(false);
const [entryStep, setEntryStep] = useState<EntryStep>('treatment');
const [voiceAvailability, setVoiceAvailability] = useState<VoiceAvailability | null>(null);
const [voiceResult, setVoiceResult] = useState<VoiceExtractionResult | null>(null);
const isDetailLocked = useCallback(
(detail: TreatmentDetailDraft) =>
labCaseDrafts.some((lc) => lc.sentAt && lc.detailClientId === detail.clientId),
@@ -483,6 +496,97 @@ export function TreatmentWorkspace({
[appointments, selectedAppointmentId],
);
/**
* Voice entry.
*
* Confirm always appends a NEW detail — it never edits an existing one, and never
* touches onAddDetail. Nothing is created until this runs, so cancelling or a failed
* recording leaves the chip strip untouched.
*/
const applyVoiceResult = useCallback(
(result: VoiceExtractionResult, selection: VoiceApplySelection) => {
const detail = newDetail(
defaultTreatmentTypeForAppointment(selectedAppointment?.purpose, treatmentCatalog),
);
// Ticked rows land on top of the seeded defaults, so unticking the type row leaves
// the appointment-purpose default rather than a blank.
if (selection.treatmentType && result.treatmentType) {
detail.treatmentType = result.treatmentType;
}
if (selection.teeth) {
detail.teeth = [...result.teeth];
detail.toothSelectionGroups = result.toothSelectionGroups.map((group) => ({
...group,
teeth: [...group.teeth],
}));
}
if (selection.comment && result.comment) {
detail.comment = result.comment;
}
setDetails((prev) => [...prev, detail]);
setActiveDetailId(detail.clientId);
setEntryStep('treatment');
// Lab-side rows ride on a lab case draft keyed by the detail's *client* id, so a
// brand-new unsaved detail can still carry one; it is persisted after the detail is.
const wantsLabDraft =
(selection.prosthesis && result.prosthesis) ||
(selection.lab && result.labId) ||
(selection.dueDate && result.dueDate);
if (wantsLabDraft) {
const draft = newLabCaseDraft();
draft.detailClientId = detail.clientId;
if (selection.lab && result.labId) {
draft.destinationOrganizationId = result.labId;
}
if (selection.dueDate && result.dueDate) {
draft.dueDate = result.dueDate;
}
if (selection.prosthesis && result.prosthesis) {
// byTooth keys are plain strings; the group's teeth are FdiToothId.
const groupOf = (tooth: string) =>
result.toothSelectionGroups.find((group) =>
(group.teeth as readonly string[]).includes(tooth),
)?.groupId ?? '';
// Only teeth that actually landed on the detail. Unticking "teeth" while
// leaving "prosthesis" ticked would otherwise attach prosthesis rows for teeth
// the treatment does not contain — nothing downstream filters them, and they
// would reach task generation as work for teeth nobody is treating.
const detailTeeth = new Set<string>(detail.teeth);
draft.toothProsthesis = Object.entries(result.prosthesis.byTooth)
.filter(([tooth]) => detailTeeth.has(tooth))
.map(([tooth, prosthesisTypeCode]) => ({
detailClientId: detail.clientId,
tooth,
prosthesisTypeCode,
selectionGroupId: groupOf(tooth),
}));
}
setLabCaseDrafts((prev) => [...prev, draft]);
}
setVoiceResult(null);
},
[selectedAppointment?.purpose, treatmentCatalog],
);
const voice = useVoiceCapture({
// The locale the clinician is actually reading and speaking in. Sent explicitly so
// the server's ASR hint, catalog labels and week start match what the microphone was
// offered for — req.user.language can drift from the URL locale.
locale,
maxMs: voiceAvailability?.maxRecordingMs ?? null,
onExtracted: setVoiceResult,
onError: (error) => showError(getUserFacingError(error, tErrors, t('voiceFailed'))),
});
/** Absence is the unavailable state — the Add button then renders unsplit. */
const voiceForEditor =
voiceAvailability?.enabled && voiceAvailability.locales.includes(locale) ? voice : undefined;
const selectedStandalone = useMemo(
() => standaloneTreatments.find((t) => t.id === selectedStandaloneId) ?? null,
[standaloneTreatments, selectedStandaloneId],
@@ -898,12 +1002,18 @@ export function TreatmentWorkspace({
let cancelled = false;
void (async () => {
try {
const [orgsResponse, catalogResponse, prosthesisResponse] = await Promise.all([
treatmentsApi.listLinkedOrganizations(),
treatmentCatalogApi.list(),
prosthesisCatalogApi.list(),
]);
const [orgsResponse, catalogResponse, prosthesisResponse, voiceResponse] =
await Promise.all([
treatmentsApi.listLinkedOrganizations(),
treatmentCatalogApi.list(),
prosthesisCatalogApi.list(),
// Voice availability comes from the API, not a NEXT_PUBLIC_* var: those are
// baked in at build time, so enabling a locale would need a frontend rebuild.
// A failure here must not take the whole treatment tab down with it.
voiceApi.availability().catch(() => null),
]);
if (cancelled) return;
setVoiceAvailability(voiceResponse?.data ?? null);
setOrgs(orgsResponse.data);
setTreatmentCatalog(catalogResponse.data);
setProsthesisCatalog(prosthesisResponse.data);
@@ -2381,6 +2491,7 @@ export function TreatmentWorkspace({
}}
showChrome
showFields={entryStep === 'treatment'}
voice={voiceForEditor}
chartLocked={
entryStep === 'treatment' && !activeTypeSelected && !showWholeTreatmentPlan
}
@@ -2642,6 +2753,16 @@ export function TreatmentWorkspace({
) : null}
</div>
</div>
{voiceResult ? (
<VoiceReviewSheet
result={voiceResult}
treatmentCatalog={treatmentCatalog}
prosthesisCatalog={prosthesisCatalog}
labs={orgs}
onApply={(selection) => applyVoiceResult(voiceResult, selection)}
onDiscard={() => setVoiceResult(null)}
/>
) : null}
</div>
);
}

View File

@@ -0,0 +1,90 @@
'use client';
import { useTranslations } from 'next-intl';
import { Loader2, X } from 'lucide-react';
import type { VoiceCaptureState } from '@/lib/voice/useVoiceCapture';
const METER_BARS = 9;
function formatElapsed(ms: number): string {
const totalSeconds = Math.floor(Math.max(0, ms) / 1000);
const minutes = Math.floor(totalSeconds / 60);
const seconds = totalSeconds % 60;
return `${minutes}:${String(seconds).padStart(2, '0')}`;
}
/**
* Live recording / processing strip.
*
* Sits between the header row and the chip strip rather than inside the segmented
* control: the header is `sm:justify-between`, so growing the button mid-recording would
* shove the row on every start and every stop.
*/
export function VoiceRecordingBar({ voice }: { voice: VoiceCaptureState }) {
const t = useTranslations('treatment');
if (voice.phase === 'idle') return null;
const isRecording = voice.phase === 'recording';
return (
<div
className="flex items-center gap-3 rounded-[var(--radius-md)] border border-border/70 bg-background-secondary/70 px-3 py-2"
role="status"
aria-live="polite"
>
{isRecording ? (
<>
<span className="h-2 w-2 shrink-0 animate-pulse rounded-full bg-red-500" aria-hidden />
<span className="shrink-0 text-sm tabular-nums text-text-primary">
{formatElapsed(voice.elapsedMs)}
{voice.maxMs != null ? (
<span className="text-text-muted"> / {formatElapsed(voice.maxMs)}</span>
) : null}
</span>
<LevelMeter level={voice.level} />
</>
) : (
<>
<Loader2 className="h-4 w-4 shrink-0 animate-spin text-text-secondary" aria-hidden />
<span className="text-sm text-text-secondary">{t('voiceProcessing')}</span>
</>
)}
<button
type="button"
onClick={voice.onCancel}
title={t('voiceCancel')}
aria-label={t('voiceCancel')}
className="ms-auto inline-flex shrink-0 items-center gap-1 rounded-[var(--radius-md)] px-2 py-1 text-xs text-text-secondary transition-colors hover:bg-red-500/15 hover:text-red-600 focus:outline-none focus-visible:ring-2 focus-visible:ring-red-500/40"
>
<X className="h-3.5 w-3.5" aria-hidden />
{t('voiceCancel')}
</button>
</div>
);
}
/** Proves the microphone is actually hearing something — silence looks identical otherwise. */
function LevelMeter({ level }: { level: number }) {
return (
<span className="flex h-4 flex-1 items-end gap-0.5" aria-hidden>
{Array.from({ length: METER_BARS }, (_, index) => {
// Bars light up left to right as the level rises, with a floor so the meter never
// looks dead while a quiet voice is still being captured.
const threshold = (index + 1) / METER_BARS;
const active = level >= threshold * 0.9;
const height = active ? 30 + threshold * 70 : 20;
return (
<span
key={index}
className={`w-1 rounded-sm transition-all duration-75 ${
active ? 'bg-primary' : 'bg-border'
}`}
style={{ height: `${height}%` }}
/>
);
})}
</span>
);
}

View File

@@ -0,0 +1,258 @@
'use client';
import { useMemo, useState } from 'react';
import { useTranslations } from 'next-intl';
import { AlertTriangle } from 'lucide-react';
import { Button } from '@/components/ui/shared/Button';
import { Checkbox } from '@/components/ui/shared/Checkbox';
import {
ResponsiveDialogOverlay,
ResponsiveDialogPanel,
} from '@/components/ui/shared/ResponsiveDialog';
import { FdiToothChart } from '@/components/ui/treatment/FdiToothChart';
import {
connectedTeethFromResult,
countSelected,
hasAnythingToApply,
initialVoiceSelection,
voiceRowAvailability,
} from '@/components/treatment/voiceReviewRows';
import { useLocale } from 'next-intl';
import { useAppFormatters } from '@/lib/hooks/useAppFormatters';
import type { TreatmentCatalogEntry } from '@/types/treatment-catalog';
import type { ProsthesisCatalogEntry } from '@/types/treatment-catalog';
import type { LinkedOrganizationOption } from '@/types/treatment';
import type { VoiceApplySelection, VoiceExtractionResult } from '@/types/voice';
interface VoiceReviewSheetProps {
result: VoiceExtractionResult;
treatmentCatalog: TreatmentCatalogEntry[];
prosthesisCatalog: ProsthesisCatalogEntry[];
labs: LinkedOrganizationOption[];
onApply: (selection: VoiceApplySelection) => void;
onDiscard: () => void;
}
/**
* Confirmation step between the model's output and the form.
*
* Modal on desktop, bottom sheet on mobile via ResponsiveDialog — deliberately an overlay
* and not a route, because navigating would unmount TreatmentWorkspace and destroy the
* in-progress draft.
*/
export function VoiceReviewSheet({
result,
treatmentCatalog,
prosthesisCatalog,
labs,
onApply,
onDiscard,
}: VoiceReviewSheetProps) {
const t = useTranslations('treatment');
const locale = useLocale();
const { formatDate } = useAppFormatters();
const [selection, setSelection] = useState<VoiceApplySelection>(() =>
initialVoiceSelection(result),
);
const available = useMemo(() => voiceRowAvailability(result), [result]);
const connectedTeeth = useMemo(() => connectedTeethFromResult(result), [result]);
const selectedTeeth = useMemo(() => new Set(result.teeth), [result.teeth]);
const nothingToApply = !hasAnythingToApply(result);
const selectedCount = countSelected(selection);
const labelFor = (code: string | null, catalog: { code: string; label: string }[]) =>
catalog.find((entry) => entry.code === code)?.label ?? code ?? '';
const toggle = (key: keyof VoiceApplySelection) => (checked: boolean) =>
setSelection((prev) => ({ ...prev, [key]: checked }));
return (
<ResponsiveDialogOverlay onBackdropClick={onDiscard}>
<ResponsiveDialogPanel
role="dialog"
aria-modal="true"
aria-labelledby="voice-review-title"
maxWidthClass="sm:max-w-xl"
>
<h2 id="voice-review-title" className="text-base font-semibold text-text-primary">
{t('voiceReviewTitle')}
</h2>
<p className="mt-2 rounded-[var(--radius-md)] bg-background-card/60 px-3 py-2 text-sm text-text-secondary">
{result.transcript}
</p>
{nothingToApply ? (
<p className="mt-4 text-sm text-text-secondary">{t('voiceNothingExtracted')}</p>
) : (
<div className="mt-4 space-y-3">
{available.treatmentType ? (
<Row
label={t('treatmentType')}
checked={selection.treatmentType}
onChange={toggle('treatmentType')}
>
<span className="text-sm text-text-primary">
{labelFor(result.treatmentType, treatmentCatalog)}
</span>
</Row>
) : null}
{available.teeth ? (
<Row label={t('entryStepTeeth')} checked={selection.teeth} onChange={toggle('teeth')}>
<div className="mt-1">
<FdiToothChart
readOnly
compact
scale={0.55}
selected={selectedTeeth}
connectedTeeth={connectedTeeth}
/>
</div>
</Row>
) : null}
{available.comment ? (
<Row
label={t('comments')}
checked={selection.comment}
onChange={toggle('comment')}
>
<span className="text-sm whitespace-pre-wrap text-text-primary">
{result.comment}
</span>
</Row>
) : null}
{available.prosthesis && result.prosthesis ? (
<Row
label={t('prosthesisColType')}
checked={selection.prosthesis}
onChange={toggle('prosthesis')}
warning={
result.prosthesis.complete
? undefined
: t('voiceProsthesisIncomplete', {
teeth: formatToothList(result.prosthesis.missingTeeth, locale),
})
}
>
<span className="text-sm text-text-primary">
{Object.entries(result.prosthesis.byTooth)
.map(
([tooth, code]) => `${tooth}: ${labelFor(code, prosthesisCatalog)}`,
)
.join(' · ')}
</span>
</Row>
) : null}
{available.lab ? (
<Row
label={t('entryStepLab')}
checked={selection.lab}
onChange={toggle('lab')}
warning={result.labMatchExact ? undefined : t('voiceLabInexact')}
>
<span className="text-sm text-text-primary">
{labs.find((lab) => lab.id === result.labId)?.name ?? result.labId}
</span>
</Row>
) : null}
{available.dueDate && result.dueDate ? (
<Row
label={t('dueDateLabel')}
checked={selection.dueDate}
onChange={toggle('dueDate')}
>
<span className="text-sm text-text-primary">
{formatDate(civilDateToLocalDate(result.dueDate))}
</span>
</Row>
) : null}
</div>
)}
{result.unresolved.length > 0 ? (
<div className="mt-4 rounded-[var(--radius-md)] border border-amber-500/40 bg-amber-500/10 px-3 py-2">
<p className="text-xs font-medium text-amber-700 dark:text-amber-400">
{t('voiceNotUnderstood')}
</p>
<ul className="mt-1 space-y-0.5">
{result.unresolved.map((item, index) => (
<li key={`${item.spoken}-${index}`} className="text-xs text-text-secondary">
{item.spoken ? `${item.spoken}” — ` : ''}
{t(`voiceUnresolved.${item.reason}`)}
</li>
))}
</ul>
</div>
) : null}
<div className="mt-5 flex flex-col gap-2 sm:flex-row sm:justify-end">
<Button type="button" variant="secondary" onClick={onDiscard} fullWidth className="sm:w-auto">
{t('voiceDiscard')}
</Button>
<Button
type="button"
variant="primary"
disabled={selectedCount === 0}
onClick={() => onApply(selection)}
fullWidth
className="sm:w-auto"
>
{t('voiceApply', { count: selectedCount })}
</Button>
</div>
</ResponsiveDialogPanel>
</ResponsiveDialogOverlay>
);
}
function Row({
label,
checked,
onChange,
warning,
children,
}: {
label: string;
checked: boolean;
onChange: (checked: boolean) => void;
warning?: string;
children: React.ReactNode;
}) {
return (
<div className="rounded-[var(--radius-md)] border border-border/70 px-3 py-2">
<Checkbox checked={checked} onChange={onChange} label={label} />
<div className="mt-1 ps-7 min-w-0">{children}</div>
{warning ? (
<p className="mt-1 ps-7 flex items-start gap-1 text-xs text-amber-700 dark:text-amber-400">
<AlertTriangle className="mt-0.5 h-3 w-3 shrink-0" aria-hidden />
{warning}
</p>
) : null}
</div>
);
}
/**
* A bare `YYYY-MM-DD` is a *civil* date, but `new Date('2025-10-17')` parses it as UTC
* midnight — which renders as the 16th for any viewer west of Greenwich. Build the date
* from its parts so it means the same day everywhere.
*/
function civilDateToLocalDate(iso: string): Date {
const [year, month, day] = iso.split('-').map(Number);
return new Date(year, (month ?? 1) - 1, day ?? 1);
}
/** Locale-aware list separator — the Arabic comma is not correct in en or nl. */
function formatToothList(teeth: readonly string[], locale: string): string {
try {
return new Intl.ListFormat(locale, { style: 'short', type: 'unit' }).format([...teeth]);
} catch {
return teeth.join(', ');
}
}

View File

@@ -0,0 +1,30 @@
import { apiClient } from './client';
import type { VoiceAvailability, VoiceExtractionResult } from '@/types/voice';
export interface ExtractVoicePayload {
/** Base64 audio, no data: prefix. */
audio: string;
format: string;
/** IANA zone — the server derives "today" from it for relative deadlines. */
timeZone: string;
durationMs: number;
/** Locale the clinician is speaking; the server uses it rather than the stored one. */
locale: string;
}
export const voiceApi = {
availability: async (): Promise<{ success: boolean; data: VoiceAvailability }> => {
const response = await apiClient.get('/voice/availability');
return response.data;
},
extract: async (
payload: ExtractVoicePayload,
signal?: AbortSignal,
): Promise<{ success: boolean; data: VoiceExtractionResult }> => {
// The signal is forwarded so cancelling closes the connection, which aborts the
// metered vendor call server-side rather than letting it settle unseen.
const response = await apiClient.post('/voice/extract', payload, { signal });
return response.data;
},
};

View File

@@ -0,0 +1,62 @@
/** Containers the backend accepts, in the order we prefer to record them. */
const PREFERRED_MIME_TYPES = [
'audio/webm;codecs=opus',
'audio/webm',
'audio/mp4',
'audio/aac',
'audio/ogg;codecs=opus',
'audio/ogg',
] as const;
/**
* Pick a container this browser can record AND the backend accepts.
*
* Chrome and Android produce webm/opus; Safari and iPad produce mp4/aac. Both go to the
* vendor unmodified, so there is no transcode step — but the choice still has to be made
* at record time, and `isTypeSupported` is missing entirely on older Safari.
*/
export function pickRecordingMimeType(): string | null {
if (typeof MediaRecorder === 'undefined') return null;
if (typeof MediaRecorder.isTypeSupported !== 'function') {
// Safari <14.1 shipped MediaRecorder without the feature check; let it choose.
return '';
}
for (const type of PREFERRED_MIME_TYPES) {
if (MediaRecorder.isTypeSupported(type)) return type;
}
return null;
}
/** `audio/webm;codecs=opus` → `webm`, which is what the API's `format` field wants. */
export function mimeTypeToFormat(mimeType: string): string {
const base = mimeType.split(';')[0]?.trim().toLowerCase() ?? '';
const subtype = base.startsWith('audio/') ? base.slice('audio/'.length) : base;
// Safari/iOS records `audio/mp4`, but the transcription endpoint's documented container
// list names m4a, not mp4. Same container; send the name the vendor documents, so iPad
// recordings do not fail while Chrome's webm works.
if (subtype === 'x-m4a' || subtype === 'm4a' || subtype === 'mp4') return 'm4a';
if (subtype === 'mpeg') return 'mp3';
return subtype || 'webm';
}
/** Blob → base64 without the `data:` prefix, which the API does not want. */
export async function blobToBase64(blob: Blob): Promise<string> {
const buffer = await blob.arrayBuffer();
let binary = '';
const bytes = new Uint8Array(buffer);
// Chunked to avoid blowing the argument limit on a two-minute recording.
const chunkSize = 0x8000;
for (let i = 0; i < bytes.length; i += chunkSize) {
binary += String.fromCharCode(...bytes.subarray(i, i + chunkSize));
}
return btoa(binary);
}
export function isMediaRecorderSupported(): boolean {
return (
typeof window !== 'undefined' &&
typeof MediaRecorder !== 'undefined' &&
typeof navigator !== 'undefined' &&
Boolean(navigator.mediaDevices?.getUserMedia)
);
}

View File

@@ -0,0 +1,277 @@
'use client';
import { useCallback, useEffect, useRef, useState } from 'react';
import { voiceApi } from '@/lib/api/voice';
import type { ApiError } from '@/types/api';
import type { VoiceExtractionResult, VoicePhase } from '@/types/voice';
import {
blobToBase64,
isMediaRecorderSupported,
mimeTypeToFormat,
pickRecordingMimeType,
} from './audioFormat';
export interface UseVoiceCaptureOptions {
/** The locale the clinician is speaking, sent so the server does not have to guess. */
locale: string;
/** null means uncapped; otherwise the recorder auto-stops here. */
maxMs: number | null;
onExtracted: (result: VoiceExtractionResult) => void;
onError: (error: unknown) => void;
}
export interface VoiceCaptureState {
phase: VoicePhase;
elapsedMs: number;
/** 0..1, for the level meter — proves the microphone is actually hearing something. */
level: number;
maxMs: number | null;
onStart: () => void;
onStop: () => void;
onCancel: () => void;
}
const LEVEL_POLL_MS = 100;
/**
* Client-side failures must be ApiError-shaped or getUserFacingError cannot resolve them
* and every one renders the generic fallback, leaving errors.VOICE_MIC_DENIED dead.
*/
function clientError(code: string): ApiError {
return { statusCode: 0, code };
}
/**
* Microphone capture for treatment voice entry.
*
* Lives in lib/ rather than in the editor: TreatmentDetailsEditor stays presentational
* and receives only a `voice` prop, so MediaRecorder and the API call never enter ui/.
*/
export function useVoiceCapture({
locale,
maxMs,
onExtracted,
onError,
}: UseVoiceCaptureOptions): VoiceCaptureState {
const [phase, setPhase] = useState<VoicePhase>('idle');
const [elapsedMs, setElapsedMs] = useState(0);
const [level, setLevel] = useState(0);
const recorderRef = useRef<MediaRecorder | null>(null);
const streamRef = useRef<MediaStream | null>(null);
const chunksRef = useRef<Blob[]>([]);
const startedAtRef = useRef(0);
const timerRef = useRef<ReturnType<typeof setInterval> | null>(null);
const audioContextRef = useRef<AudioContext | null>(null);
const abortRef = useRef<AbortController | null>(null);
/** Set when the user cancels, so the recorder's stop handler discards instead of sending. */
const cancelledRef = useRef(false);
/** getUserMedia is async; without this a permission granted after unmount leaks the mic. */
const mountedRef = useRef(true);
/**
* Set synchronously on click. `phase` does not become 'recording' until getUserMedia
* resolves, so without this a second click during the permission prompt would start a
* second stream and orphan the first — mic indicator lit, interval leaked.
*/
const startingRef = useRef(false);
const teardown = useCallback(() => {
if (timerRef.current) {
clearInterval(timerRef.current);
timerRef.current = null;
}
streamRef.current?.getTracks().forEach((track) => track.stop());
streamRef.current = null;
void audioContextRef.current?.close().catch(() => undefined);
audioContextRef.current = null;
recorderRef.current = null;
setLevel(0);
}, []);
// Releasing the microphone on unmount matters: the browser shows a recording indicator
// for as long as the track is live, and an orphaned one looks like the app is listening.
useEffect(() => {
// Re-armed on every mount: React StrictMode runs mount → unmount → mount in dev, and
// a ref that is only ever set false would leave the hook permanently "unmounted".
mountedRef.current = true;
return () => {
mountedRef.current = false;
cancelledRef.current = true;
abortRef.current?.abort();
try {
recorderRef.current?.stop();
} catch {
// already stopped
}
teardown();
};
}, [teardown]);
const send = useCallback(
async (blob: Blob, mimeType: string, durationMs: number) => {
setPhase('processing');
const controller = new AbortController();
abortRef.current = controller;
try {
const audio = await blobToBase64(blob);
const response = await voiceApi.extract(
{
audio,
format: mimeTypeToFormat(mimeType),
timeZone: Intl.DateTimeFormat().resolvedOptions().timeZone,
durationMs,
locale,
},
controller.signal,
);
if (cancelledRef.current) return;
onExtracted(response.data);
} catch (error) {
if (cancelledRef.current || controller.signal.aborted) return;
onError(error);
} finally {
abortRef.current = null;
setPhase('idle');
setElapsedMs(0);
}
},
[locale, onExtracted, onError],
);
const stop = useCallback(() => {
try {
recorderRef.current?.stop();
} catch {
teardown();
setPhase('idle');
}
}, [teardown]);
const onStart = useCallback(() => {
if (phase !== 'idle' || startingRef.current) return;
if (!isMediaRecorderSupported()) {
onError(clientError('VOICE_MIC_DENIED'));
return;
}
cancelledRef.current = false;
startingRef.current = true;
void (async () => {
try {
let stream: MediaStream;
try {
stream = await navigator.mediaDevices.getUserMedia({ audio: true });
} catch {
// Permission refused, or no input device. Never a server round-trip.
onError(clientError('VOICE_MIC_DENIED'));
return;
}
if (!mountedRef.current) {
// Permission resolved after the component went away — release it immediately
// rather than leaving the browser's recording indicator lit.
stream.getTracks().forEach((track) => track.stop());
return;
}
const mimeType = pickRecordingMimeType();
if (mimeType === null) {
stream.getTracks().forEach((track) => track.stop());
onError(clientError('VOICE_MIC_DENIED'));
return;
}
streamRef.current = stream;
chunksRef.current = [];
const recorder = new MediaRecorder(stream, mimeType ? { mimeType } : undefined);
recorderRef.current = recorder;
recorder.ondataavailable = (event) => {
if (event.data.size > 0) chunksRef.current.push(event.data);
};
recorder.onstop = () => {
const durationMs = Date.now() - startedAtRef.current;
const blob = new Blob(chunksRef.current, { type: recorder.mimeType || mimeType });
teardown();
if (cancelledRef.current || blob.size === 0) {
setPhase('idle');
setElapsedMs(0);
return;
}
// Prefer what the recorder actually produced, then the blob's own type. Old
// Safari accepts no mimeType hint, and defaulting to webm would mislabel its
// mp4/aac clips as something they are not.
void send(blob, recorder.mimeType || blob.type || mimeType || 'audio/webm', durationMs);
};
attachLevelMeter(stream, audioContextRef, setLevel);
startedAtRef.current = Date.now();
recorder.start();
setPhase('recording');
setElapsedMs(0);
timerRef.current = setInterval(() => {
const elapsed = Date.now() - startedAtRef.current;
setElapsedMs(elapsed);
// Auto-stop proceeds to processing with what was captured; discarding two
// minutes of dictation because a timer expired would be the worst failure.
if (maxMs != null && elapsed >= maxMs) stop();
}, LEVEL_POLL_MS);
} finally {
startingRef.current = false;
}
})();
}, [maxMs, onError, phase, send, stop, teardown]);
const onCancel = useCallback(() => {
cancelledRef.current = true;
// Aborting closes the connection, which aborts the vendor call server-side. It is
// metered per minute, so letting it settle costs money for a result nobody sees.
abortRef.current?.abort();
try {
recorderRef.current?.stop();
} catch {
// already stopped
}
teardown();
setPhase('idle');
setElapsedMs(0);
}, [teardown]);
return { phase, elapsedMs, level, maxMs, onStart, onStop: stop, onCancel };
}
/** Drives the level meter from the live stream; failure here must not stop recording. */
function attachLevelMeter(
stream: MediaStream,
contextRef: React.MutableRefObject<AudioContext | null>,
setLevel: (value: number) => void,
) {
try {
const AudioContextCtor =
window.AudioContext ?? (window as { webkitAudioContext?: typeof AudioContext }).webkitAudioContext;
if (!AudioContextCtor) return;
const context = new AudioContextCtor();
contextRef.current = context;
const source = context.createMediaStreamSource(stream);
const analyser = context.createAnalyser();
analyser.fftSize = 512;
source.connect(analyser);
const data = new Uint8Array(analyser.frequencyBinCount);
const tick = () => {
if (contextRef.current !== context || context.state === 'closed') return;
analyser.getByteTimeDomainData(data);
let peak = 0;
for (const sample of data) peak = Math.max(peak, Math.abs(sample - 128));
setLevel(Math.min(1, peak / 128));
requestAnimationFrame(tick);
};
requestAnimationFrame(tick);
} catch {
// A missing or blocked AudioContext costs the meter, not the recording.
}
}

View File

@@ -0,0 +1,59 @@
import type { FdiToothId, ToothSelectionGroup } from '@/types/treatment';
/** Mirrors the backend's ResolvedExtraction — values already resolved, plus what was not. */
export type VoiceUnresolvedReason =
| 'not_permanent_tooth'
| 'position_out_of_range'
| 'tooth_missing_quadrant'
| 'malformed'
| 'span_not_same_arch'
| 'unknown_catalog_code'
| 'tooth_not_selected'
| 'invalid_date';
export interface VoiceUnresolvedItem {
/** The transcript span that could not be resolved, so the clinician sees what was heard. */
spoken: string;
reason: VoiceUnresolvedReason;
}
export interface VoiceProsthesisResult {
byTooth: Record<string, string>;
/** False means the case cannot ship — every tooth needs a prosthesis type. */
complete: boolean;
missingTeeth: FdiToothId[];
}
export interface VoiceExtractionResult {
transcript: string;
treatmentType: string | null;
teeth: FdiToothId[];
toothSelectionGroups: ToothSelectionGroup[];
comment: string | null;
prosthesis: VoiceProsthesisResult | null;
labId: string | null;
/** When false, the lab row must not tick itself — the name only approximately matched. */
labMatchExact: boolean;
dueDate: string | null;
unresolved: VoiceUnresolvedItem[];
}
export interface VoiceAvailability {
enabled: boolean;
locales: string[];
/** null means uncapped. */
maxRecordingMs: number | null;
}
/** Which review rows the clinician ticked. */
export interface VoiceApplySelection {
treatmentType: boolean;
teeth: boolean;
comment: boolean;
prosthesis: boolean;
lab: boolean;
dueDate: boolean;
}
export type VoicePhase = 'idle' | 'recording' | 'processing';