feat(backend): voice extraction endpoint
POST /voice/extract behind JwtAuthGuard + ClinicOrgGuard, plus
GET /voice/availability so the frontend can decide whether to render the
microphone — it cannot learn that from NEXT_PUBLIC_*, which are baked in at
build time.
Audio is held in memory for the request only: never written to disk, never a
Prisma row. The transcript goes back to the client and is not persisted. What
is logged is structured and patient-free — clip length, which fields resolved,
unresolved count, vendor cost, outcome — with log lines as the interim sink
until this repo has metrics infrastructure.
On extraction failure the transcript still travels back in the error details,
so the words the clinician already paid for can be salvaged into a note.
v1 ships ungated beyond a configured locale profile; the Plan.features design
is deferred, not dropped.
From review of this commit, four of which were load-bearing:
- Express's 100 kb default body limit rejected any recording past ~20 seconds,
making the endpoint unusable at its own 2-minute cap. Body parsers are now
registered explicitly with a 10 MB limit scoped to the voice route only.
Verified empirically: 600 KB reaches /api/voice/extract, while /api/auth/login
still 413s.
- ThrottlerGuard keys on req.ip, so behind nginx the whole deployment would
share one bucket and an abuser rotating IPs would bypass it. VoiceThrottlerGuard
keys on the user id instead — with no plan gate, this is the only control on
metered vendor spend.
- ThrottlerException had no 429 fallback and surfaced as INTERNAL_ERROR; the
guard now throws VOICE_RATE_LIMITED directly.
- durationMs was optional, so omitting it bypassed VOICE_MAX_RECORDING_MS
entirely. It is required.
- VOICE_UNSUPPORTED_FORMAT was dead code — the DTO's @IsIn already rejects
unknown containers — so it is gone rather than left unreachable.
ThrottlerModule is deliberately not bound as a global APP_GUARD: a global
ThrottlerGuard rate-limits every route against every named throttler, which
would have capped the whole API at the voice limit.
All seven remaining VOICE_* codes have errors.* keys in en, fa and nl.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 18:27:14 +03:30
|
|
|
import {
|
|
|
|
|
IsBase64,
|
|
|
|
|
IsIn,
|
|
|
|
|
IsInt,
|
|
|
|
|
IsString,
|
|
|
|
|
MaxLength,
|
|
|
|
|
Min,
|
|
|
|
|
} from 'class-validator';
|
2026-08-21 05:04:30 +08:00
|
|
|
import { ErrorCode } from '../../../common/errors/error-codes';
|
feat(backend): voice extraction endpoint
POST /voice/extract behind JwtAuthGuard + ClinicOrgGuard, plus
GET /voice/availability so the frontend can decide whether to render the
microphone — it cannot learn that from NEXT_PUBLIC_*, which are baked in at
build time.
Audio is held in memory for the request only: never written to disk, never a
Prisma row. The transcript goes back to the client and is not persisted. What
is logged is structured and patient-free — clip length, which fields resolved,
unresolved count, vendor cost, outcome — with log lines as the interim sink
until this repo has metrics infrastructure.
On extraction failure the transcript still travels back in the error details,
so the words the clinician already paid for can be salvaged into a note.
v1 ships ungated beyond a configured locale profile; the Plan.features design
is deferred, not dropped.
From review of this commit, four of which were load-bearing:
- Express's 100 kb default body limit rejected any recording past ~20 seconds,
making the endpoint unusable at its own 2-minute cap. Body parsers are now
registered explicitly with a 10 MB limit scoped to the voice route only.
Verified empirically: 600 KB reaches /api/voice/extract, while /api/auth/login
still 413s.
- ThrottlerGuard keys on req.ip, so behind nginx the whole deployment would
share one bucket and an abuser rotating IPs would bypass it. VoiceThrottlerGuard
keys on the user id instead — with no plan gate, this is the only control on
metered vendor spend.
- ThrottlerException had no 429 fallback and surfaced as INTERNAL_ERROR; the
guard now throws VOICE_RATE_LIMITED directly.
- durationMs was optional, so omitting it bypassed VOICE_MAX_RECORDING_MS
entirely. It is required.
- VOICE_UNSUPPORTED_FORMAT was dead code — the DTO's @IsIn already rejects
unknown containers — so it is gone rather than left unreachable.
ThrottlerModule is deliberately not bound as a global APP_GUARD: a global
ThrottlerGuard rate-limits every route against every named throttler, which
would have capped the whole API at the voice limit.
All seven remaining VOICE_* codes have errors.* keys in en, fa and nl.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 18:27:14 +03:30
|
|
|
|
|
|
|
|
/** Containers OpenRouter's transcription endpoint accepts, and MediaRecorder can produce. */
|
|
|
|
|
export const VOICE_AUDIO_FORMATS = [
|
|
|
|
|
'webm',
|
|
|
|
|
'mp4',
|
|
|
|
|
'm4a',
|
|
|
|
|
'aac',
|
|
|
|
|
'ogg',
|
|
|
|
|
'wav',
|
|
|
|
|
'mp3',
|
|
|
|
|
'flac',
|
|
|
|
|
] as const;
|
|
|
|
|
|
|
|
|
|
export type VoiceAudioFormat = (typeof VOICE_AUDIO_FORMATS)[number];
|
|
|
|
|
|
feat: wire voice entry into the treatment workspace
Makes the feature reachable end to end: availability is fetched alongside the
catalogs, the capture hook drives the segmented control, and confirming the
review sheet appends a new detail.
Confirm always appends — it never edits an existing detail and never calls
onAddDetail. Ticked rows land on top of the seeded defaults, so unticking the
type row leaves the appointment-purpose default rather than a blank. Lab-side
rows ride on a lab case draft keyed by the detail's *client* id, so a brand-new
unsaved detail can carry a lab, due date and per-tooth prosthesis map.
Availability comes from the API rather than a NEXT_PUBLIC_* var, since those are
baked in at build time; a failure fetching it degrades to no microphone rather
than taking the treatment tab down.
From review of this commit:
- Unticking "teeth" while leaving "prosthesis" ticked attached prosthesis rows
for teeth the detail does not contain. Nothing downstream filters them —
assertCompleteToothProsthesisMap only checks detail-teeth ⊆ map, never the
reverse — so they would have reached task generation as lab work for teeth
nobody is treating. The map is now filtered to the detail's own teeth.
- The microphone was gated on the URL locale while the server resolved
everything from req.user.language. Those diverge (a bookmarked /fa/ URL, a
language toggle whose save failed), which would transcribe Persian with an
English hint and anchor "next Thursday" to a Monday week instead of a Saturday
one — or 403 from a visibly-enabled button. The client now sends the locale the
microphone was offered in, so the gate and the request agree by construction.
Also fixed from the previous review: a civil YYYY-MM-DD date rendered a day
early west of Greenwich (parsed as UTC midnight); the missing-teeth list
hardcoded the Arabic comma for all locales; and voiceApply had no ICU plural, so
the common single-field case read "Apply 1 fields".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 20:22:42 +03:30
|
|
|
/** Locales the app ships; a profile still has to be configured for one to be usable. */
|
|
|
|
|
export const VOICE_LOCALES = ['en', 'fa', 'nl'] as const;
|
|
|
|
|
|
feat(backend): voice extraction endpoint
POST /voice/extract behind JwtAuthGuard + ClinicOrgGuard, plus
GET /voice/availability so the frontend can decide whether to render the
microphone — it cannot learn that from NEXT_PUBLIC_*, which are baked in at
build time.
Audio is held in memory for the request only: never written to disk, never a
Prisma row. The transcript goes back to the client and is not persisted. What
is logged is structured and patient-free — clip length, which fields resolved,
unresolved count, vendor cost, outcome — with log lines as the interim sink
until this repo has metrics infrastructure.
On extraction failure the transcript still travels back in the error details,
so the words the clinician already paid for can be salvaged into a note.
v1 ships ungated beyond a configured locale profile; the Plan.features design
is deferred, not dropped.
From review of this commit, four of which were load-bearing:
- Express's 100 kb default body limit rejected any recording past ~20 seconds,
making the endpoint unusable at its own 2-minute cap. Body parsers are now
registered explicitly with a 10 MB limit scoped to the voice route only.
Verified empirically: 600 KB reaches /api/voice/extract, while /api/auth/login
still 413s.
- ThrottlerGuard keys on req.ip, so behind nginx the whole deployment would
share one bucket and an abuser rotating IPs would bypass it. VoiceThrottlerGuard
keys on the user id instead — with no plan gate, this is the only control on
metered vendor spend.
- ThrottlerException had no 429 fallback and surfaced as INTERNAL_ERROR; the
guard now throws VOICE_RATE_LIMITED directly.
- durationMs was optional, so omitting it bypassed VOICE_MAX_RECORDING_MS
entirely. It is required.
- VOICE_UNSUPPORTED_FORMAT was dead code — the DTO's @IsIn already rejects
unknown containers — so it is gone rather than left unreachable.
ThrottlerModule is deliberately not bound as a global APP_GUARD: a global
ThrottlerGuard rate-limits every route against every named throttler, which
would have capped the whole API at the voice limit.
All seven remaining VOICE_* codes have errors.* keys in en, fa and nl.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 18:27:14 +03:30
|
|
|
export class ExtractVoiceDto {
|
|
|
|
|
/**
|
|
|
|
|
* Base64 audio, no data: prefix. Capped well above a 2-minute opus clip (~400 KB) but
|
|
|
|
|
* far below OpenRouter's 25 MB ceiling, so an oversized upload is rejected before it
|
|
|
|
|
* costs a vendor call.
|
|
|
|
|
*/
|
|
|
|
|
@IsString()
|
|
|
|
|
@IsBase64()
|
2026-08-21 05:04:30 +08:00
|
|
|
// Both constraints name their own code. Left to the default mapping, `maxLength` falls
|
|
|
|
|
// through to VALIDATION_FIELD_REQUIRED and `isIn` resolves to
|
|
|
|
|
// VALIDATION_LANGUAGE_INVALID — so an oversized recording told the clinician a field
|
|
|
|
|
// was missing, and an unsupported container told them their language was invalid.
|
|
|
|
|
@MaxLength(8_000_000, { message: ErrorCode.VOICE_CLIP_TOO_LONG })
|
feat(backend): voice extraction endpoint
POST /voice/extract behind JwtAuthGuard + ClinicOrgGuard, plus
GET /voice/availability so the frontend can decide whether to render the
microphone — it cannot learn that from NEXT_PUBLIC_*, which are baked in at
build time.
Audio is held in memory for the request only: never written to disk, never a
Prisma row. The transcript goes back to the client and is not persisted. What
is logged is structured and patient-free — clip length, which fields resolved,
unresolved count, vendor cost, outcome — with log lines as the interim sink
until this repo has metrics infrastructure.
On extraction failure the transcript still travels back in the error details,
so the words the clinician already paid for can be salvaged into a note.
v1 ships ungated beyond a configured locale profile; the Plan.features design
is deferred, not dropped.
From review of this commit, four of which were load-bearing:
- Express's 100 kb default body limit rejected any recording past ~20 seconds,
making the endpoint unusable at its own 2-minute cap. Body parsers are now
registered explicitly with a 10 MB limit scoped to the voice route only.
Verified empirically: 600 KB reaches /api/voice/extract, while /api/auth/login
still 413s.
- ThrottlerGuard keys on req.ip, so behind nginx the whole deployment would
share one bucket and an abuser rotating IPs would bypass it. VoiceThrottlerGuard
keys on the user id instead — with no plan gate, this is the only control on
metered vendor spend.
- ThrottlerException had no 429 fallback and surfaced as INTERNAL_ERROR; the
guard now throws VOICE_RATE_LIMITED directly.
- durationMs was optional, so omitting it bypassed VOICE_MAX_RECORDING_MS
entirely. It is required.
- VOICE_UNSUPPORTED_FORMAT was dead code — the DTO's @IsIn already rejects
unknown containers — so it is gone rather than left unreachable.
ThrottlerModule is deliberately not bound as a global APP_GUARD: a global
ThrottlerGuard rate-limits every route against every named throttler, which
would have capped the whole API at the voice limit.
All seven remaining VOICE_* codes have errors.* keys in en, fa and nl.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 18:27:14 +03:30
|
|
|
audio: string;
|
|
|
|
|
|
2026-08-21 05:04:30 +08:00
|
|
|
@IsIn(VOICE_AUDIO_FORMATS, { message: ErrorCode.VOICE_UNSUPPORTED_FORMAT })
|
feat(backend): voice extraction endpoint
POST /voice/extract behind JwtAuthGuard + ClinicOrgGuard, plus
GET /voice/availability so the frontend can decide whether to render the
microphone — it cannot learn that from NEXT_PUBLIC_*, which are baked in at
build time.
Audio is held in memory for the request only: never written to disk, never a
Prisma row. The transcript goes back to the client and is not persisted. What
is logged is structured and patient-free — clip length, which fields resolved,
unresolved count, vendor cost, outcome — with log lines as the interim sink
until this repo has metrics infrastructure.
On extraction failure the transcript still travels back in the error details,
so the words the clinician already paid for can be salvaged into a note.
v1 ships ungated beyond a configured locale profile; the Plan.features design
is deferred, not dropped.
From review of this commit, four of which were load-bearing:
- Express's 100 kb default body limit rejected any recording past ~20 seconds,
making the endpoint unusable at its own 2-minute cap. Body parsers are now
registered explicitly with a 10 MB limit scoped to the voice route only.
Verified empirically: 600 KB reaches /api/voice/extract, while /api/auth/login
still 413s.
- ThrottlerGuard keys on req.ip, so behind nginx the whole deployment would
share one bucket and an abuser rotating IPs would bypass it. VoiceThrottlerGuard
keys on the user id instead — with no plan gate, this is the only control on
metered vendor spend.
- ThrottlerException had no 429 fallback and surfaced as INTERNAL_ERROR; the
guard now throws VOICE_RATE_LIMITED directly.
- durationMs was optional, so omitting it bypassed VOICE_MAX_RECORDING_MS
entirely. It is required.
- VOICE_UNSUPPORTED_FORMAT was dead code — the DTO's @IsIn already rejects
unknown containers — so it is gone rather than left unreachable.
ThrottlerModule is deliberately not bound as a global APP_GUARD: a global
ThrottlerGuard rate-limits every route against every named throttler, which
would have capped the whole API at the voice limit.
All seven remaining VOICE_* codes have errors.* keys in en, fa and nl.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 18:27:14 +03:30
|
|
|
format: VoiceAudioFormat;
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* The clinician's IANA zone. The server derives "today" from it rather than trusting a
|
|
|
|
|
* client-supplied date, which is what relative deadlines resolve against.
|
|
|
|
|
*/
|
|
|
|
|
@IsString()
|
|
|
|
|
@MaxLength(64)
|
|
|
|
|
timeZone: string;
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* Recording length as measured by the client.
|
|
|
|
|
*
|
|
|
|
|
* Required, not optional: an optional value means omitting it bypasses
|
|
|
|
|
* VOICE_MAX_RECORDING_MS entirely, which would make the cap advisory.
|
|
|
|
|
*/
|
|
|
|
|
@IsInt()
|
|
|
|
|
@Min(0)
|
|
|
|
|
durationMs: number;
|
feat: wire voice entry into the treatment workspace
Makes the feature reachable end to end: availability is fetched alongside the
catalogs, the capture hook drives the segmented control, and confirming the
review sheet appends a new detail.
Confirm always appends — it never edits an existing detail and never calls
onAddDetail. Ticked rows land on top of the seeded defaults, so unticking the
type row leaves the appointment-purpose default rather than a blank. Lab-side
rows ride on a lab case draft keyed by the detail's *client* id, so a brand-new
unsaved detail can carry a lab, due date and per-tooth prosthesis map.
Availability comes from the API rather than a NEXT_PUBLIC_* var, since those are
baked in at build time; a failure fetching it degrades to no microphone rather
than taking the treatment tab down.
From review of this commit:
- Unticking "teeth" while leaving "prosthesis" ticked attached prosthesis rows
for teeth the detail does not contain. Nothing downstream filters them —
assertCompleteToothProsthesisMap only checks detail-teeth ⊆ map, never the
reverse — so they would have reached task generation as lab work for teeth
nobody is treating. The map is now filtered to the detail's own teeth.
- The microphone was gated on the URL locale while the server resolved
everything from req.user.language. Those diverge (a bookmarked /fa/ URL, a
language toggle whose save failed), which would transcribe Persian with an
English hint and anchor "next Thursday" to a Monday week instead of a Saturday
one — or 403 from a visibly-enabled button. The client now sends the locale the
microphone was offered in, so the gate and the request agree by construction.
Also fixed from the previous review: a civil YYYY-MM-DD date rendered a day
early west of Greenwich (parsed as UTC midnight); the missing-teeth list
hardcoded the Arabic comma for all locales; and voiceApply had no ICU plural, so
the common single-field case read "Apply 1 fields".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 20:22:42 +03:30
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* The locale the clinician is actually speaking, as the UI offered the microphone.
|
|
|
|
|
*
|
|
|
|
|
* Sent explicitly rather than read from `user.language`: the two can diverge (a
|
|
|
|
|
* bookmarked /fa/ URL, a language toggle whose save failed), and a mismatch would
|
|
|
|
|
* transcribe Persian with an English hint and anchor "next Thursday" to the wrong
|
|
|
|
|
* week start. Gating the button and resolving the request must agree by construction.
|
|
|
|
|
*/
|
|
|
|
|
@IsIn(VOICE_LOCALES)
|
|
|
|
|
locale: string;
|
feat(backend): voice extraction endpoint
POST /voice/extract behind JwtAuthGuard + ClinicOrgGuard, plus
GET /voice/availability so the frontend can decide whether to render the
microphone — it cannot learn that from NEXT_PUBLIC_*, which are baked in at
build time.
Audio is held in memory for the request only: never written to disk, never a
Prisma row. The transcript goes back to the client and is not persisted. What
is logged is structured and patient-free — clip length, which fields resolved,
unresolved count, vendor cost, outcome — with log lines as the interim sink
until this repo has metrics infrastructure.
On extraction failure the transcript still travels back in the error details,
so the words the clinician already paid for can be salvaged into a note.
v1 ships ungated beyond a configured locale profile; the Plan.features design
is deferred, not dropped.
From review of this commit, four of which were load-bearing:
- Express's 100 kb default body limit rejected any recording past ~20 seconds,
making the endpoint unusable at its own 2-minute cap. Body parsers are now
registered explicitly with a 10 MB limit scoped to the voice route only.
Verified empirically: 600 KB reaches /api/voice/extract, while /api/auth/login
still 413s.
- ThrottlerGuard keys on req.ip, so behind nginx the whole deployment would
share one bucket and an abuser rotating IPs would bypass it. VoiceThrottlerGuard
keys on the user id instead — with no plan gate, this is the only control on
metered vendor spend.
- ThrottlerException had no 429 fallback and surfaced as INTERNAL_ERROR; the
guard now throws VOICE_RATE_LIMITED directly.
- durationMs was optional, so omitting it bypassed VOICE_MAX_RECORDING_MS
entirely. It is required.
- VOICE_UNSUPPORTED_FORMAT was dead code — the DTO's @IsIn already rejects
unknown containers — so it is gone rather than left unreachable.
ThrottlerModule is deliberately not bound as a global APP_GUARD: a global
ThrottlerGuard rate-limits every route against every named throttler, which
would have capped the whole API at the voice limit.
All seven remaining VOICE_* codes have errors.* keys in en, fa and nl.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 18:27:14 +03:30
|
|
|
}
|