Files
dyolink/frontend/src/lib/voice/useVoiceCapture.ts

274 lines
9.1 KiB
TypeScript
Raw Normal View History

feat(frontend): voice capture hook, API client and types MediaRecorder handling and the API call live in lib/, not in ui/, so TreatmentDetailsEditor can stay presentational and take only a `voice` prop. Container choice is made at record time and needs no transcode: Chrome and Android give webm/opus, Safari and iPad give mp4/aac, and the transcription endpoint accepts both. Safari's `audio/mp4` is sent as `m4a`, the name the vendor's container list actually uses, so iPad recordings do not fail while Chrome works. Older Safari shipped MediaRecorder without isTypeSupported, so that path lets the browser choose rather than refusing outright. From review of this commit: - The auto-stop at maxMs guaranteed a 413. The client measures the final length after the recorder has stopped, so a recording that runs to the cap always reports slightly over it, and the server rejected exactly the recording the auto-stop existed to save. The server now allows a documented 2s tolerance and the client keeps reporting the true length, so telemetry stays honest. - getUserMedia is async, so a permission granted after unmount installed a live stream the cleanup effect had already run past — leaving the browser's recording indicator lit with nothing listening. Guarded with a mounted ref. - Client-side failures are now ApiError-shaped ({code, statusCode}) rather than bare Errors, because getUserFacingError only resolves that shape; without it errors.VOICE_MIC_DENIED was dead in all three locales. Cancelling aborts the request, which closes the connection and aborts the metered vendor call server-side rather than letting it settle unseen. The level meter is best-effort: a blocked AudioContext costs the meter, not the recording. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 18:43:04 +03:30
'use client';
import { useCallback, useEffect, useRef, useState } from 'react';
import { voiceApi } from '@/lib/api/voice';
import type { ApiError } from '@/types/api';
import type { VoiceExtractionResult, VoicePhase } from '@/types/voice';
import {
blobToBase64,
isMediaRecorderSupported,
mimeTypeToFormat,
pickRecordingMimeType,
} from './audioFormat';
export interface UseVoiceCaptureOptions {
/** null means uncapped; otherwise the recorder auto-stops here. */
maxMs: number | null;
onExtracted: (result: VoiceExtractionResult) => void;
onError: (error: unknown) => void;
}
export interface VoiceCaptureState {
phase: VoicePhase;
elapsedMs: number;
/** 0..1, for the level meter — proves the microphone is actually hearing something. */
level: number;
maxMs: number | null;
onStart: () => void;
onStop: () => void;
onCancel: () => void;
}
const LEVEL_POLL_MS = 100;
/**
* Client-side failures must be ApiError-shaped or getUserFacingError cannot resolve them
* and every one renders the generic fallback, leaving errors.VOICE_MIC_DENIED dead.
*/
function clientError(code: string): ApiError {
return { statusCode: 0, code };
}
/**
* Microphone capture for treatment voice entry.
*
* Lives in lib/ rather than in the editor: TreatmentDetailsEditor stays presentational
* and receives only a `voice` prop, so MediaRecorder and the API call never enter ui/.
*/
export function useVoiceCapture({
maxMs,
onExtracted,
onError,
}: UseVoiceCaptureOptions): VoiceCaptureState {
const [phase, setPhase] = useState<VoicePhase>('idle');
const [elapsedMs, setElapsedMs] = useState(0);
const [level, setLevel] = useState(0);
const recorderRef = useRef<MediaRecorder | null>(null);
const streamRef = useRef<MediaStream | null>(null);
const chunksRef = useRef<Blob[]>([]);
const startedAtRef = useRef(0);
const timerRef = useRef<ReturnType<typeof setInterval> | null>(null);
const audioContextRef = useRef<AudioContext | null>(null);
const abortRef = useRef<AbortController | null>(null);
/** Set when the user cancels, so the recorder's stop handler discards instead of sending. */
const cancelledRef = useRef(false);
/** getUserMedia is async; without this a permission granted after unmount leaks the mic. */
const mountedRef = useRef(true);
feat(frontend): split Add detail into a segmented control with voice The microphone becomes the second segment of the Add detail button, built like the detail chip's trash affordance in the same file — an overflow-hidden rounded wrapper holding two raw <button>s divided by border-s — rather than two shared Buttons, which each hardcode their own rounding and would fight a segmented control. border-s puts the mic at the logical end: visually right in en/nl, visually left in fa, on the same side as the chip's trash in both directions. The two halves share a wrapper and nothing else. Add keeps its exact behaviour. The control never changes size while recording; the timer and level meter live in a bar between the header row and the chip strip, because the header is sm:justify-between and growing the button would shove the row on every start and stop. The meter exists to prove the microphone is actually hearing something — silence and a dead mic look identical otherwise. Voice reaches the editor as one optional `voice` prop, so its absence *is* the unavailable state and the two cannot disagree. Fixes from review of this commit: - mountedRef was set false on unmount and never re-armed, so under StrictMode the hook was permanently "unmounted" in dev and recording silently never started. - onStart guarded only on `phase`, which does not change until getUserMedia resolves; a second click during the permission prompt orphaned the first MediaStream, leaving the mic indicator lit. - Week start is now per locale. "Next Thursday" is week-relative, and hardcoding Saturday put an en/nl clinician's deadline a week out. - A missing `which` on a weekday intent is read as "this" rather than failing — a bare weekday carries no qualifier, and rejecting it discarded a real deadline. - durationMs is client-reported and so is a claim, not enforcement; the cap is now also checked against the vendor's own usage.seconds. - Blob type falls back to the recorder's actual mimeType before webm, so old Safari's mp4/aac clips are not mislabelled. Two review findings were rejected as incorrect, both re-verified against live sources: google/gemini-3.7-flash does exist on OpenRouter (1M context, $0.375/$1.875 per M), and base64 JSON input_audio is the documented primary path for /audio/transcriptions, with multipart as the OpenAI-compatible alternative. The spec's stale "unverified" note is corrected, and the provider now has unit tests covering the request shape, usage parsing, and that a vendor error body never reaches the thrown message. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 19:47:10 +03:30
/**
* Set synchronously on click. `phase` does not become 'recording' until getUserMedia
* resolves, so without this a second click during the permission prompt would start a
* second stream and orphan the first mic indicator lit, interval leaked.
*/
const startingRef = useRef(false);
feat(frontend): voice capture hook, API client and types MediaRecorder handling and the API call live in lib/, not in ui/, so TreatmentDetailsEditor can stay presentational and take only a `voice` prop. Container choice is made at record time and needs no transcode: Chrome and Android give webm/opus, Safari and iPad give mp4/aac, and the transcription endpoint accepts both. Safari's `audio/mp4` is sent as `m4a`, the name the vendor's container list actually uses, so iPad recordings do not fail while Chrome works. Older Safari shipped MediaRecorder without isTypeSupported, so that path lets the browser choose rather than refusing outright. From review of this commit: - The auto-stop at maxMs guaranteed a 413. The client measures the final length after the recorder has stopped, so a recording that runs to the cap always reports slightly over it, and the server rejected exactly the recording the auto-stop existed to save. The server now allows a documented 2s tolerance and the client keeps reporting the true length, so telemetry stays honest. - getUserMedia is async, so a permission granted after unmount installed a live stream the cleanup effect had already run past — leaving the browser's recording indicator lit with nothing listening. Guarded with a mounted ref. - Client-side failures are now ApiError-shaped ({code, statusCode}) rather than bare Errors, because getUserFacingError only resolves that shape; without it errors.VOICE_MIC_DENIED was dead in all three locales. Cancelling aborts the request, which closes the connection and aborts the metered vendor call server-side rather than letting it settle unseen. The level meter is best-effort: a blocked AudioContext costs the meter, not the recording. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 18:43:04 +03:30
const teardown = useCallback(() => {
if (timerRef.current) {
clearInterval(timerRef.current);
timerRef.current = null;
}
streamRef.current?.getTracks().forEach((track) => track.stop());
streamRef.current = null;
void audioContextRef.current?.close().catch(() => undefined);
audioContextRef.current = null;
recorderRef.current = null;
setLevel(0);
}, []);
// Releasing the microphone on unmount matters: the browser shows a recording indicator
// for as long as the track is live, and an orphaned one looks like the app is listening.
feat(frontend): split Add detail into a segmented control with voice The microphone becomes the second segment of the Add detail button, built like the detail chip's trash affordance in the same file — an overflow-hidden rounded wrapper holding two raw <button>s divided by border-s — rather than two shared Buttons, which each hardcode their own rounding and would fight a segmented control. border-s puts the mic at the logical end: visually right in en/nl, visually left in fa, on the same side as the chip's trash in both directions. The two halves share a wrapper and nothing else. Add keeps its exact behaviour. The control never changes size while recording; the timer and level meter live in a bar between the header row and the chip strip, because the header is sm:justify-between and growing the button would shove the row on every start and stop. The meter exists to prove the microphone is actually hearing something — silence and a dead mic look identical otherwise. Voice reaches the editor as one optional `voice` prop, so its absence *is* the unavailable state and the two cannot disagree. Fixes from review of this commit: - mountedRef was set false on unmount and never re-armed, so under StrictMode the hook was permanently "unmounted" in dev and recording silently never started. - onStart guarded only on `phase`, which does not change until getUserMedia resolves; a second click during the permission prompt orphaned the first MediaStream, leaving the mic indicator lit. - Week start is now per locale. "Next Thursday" is week-relative, and hardcoding Saturday put an en/nl clinician's deadline a week out. - A missing `which` on a weekday intent is read as "this" rather than failing — a bare weekday carries no qualifier, and rejecting it discarded a real deadline. - durationMs is client-reported and so is a claim, not enforcement; the cap is now also checked against the vendor's own usage.seconds. - Blob type falls back to the recorder's actual mimeType before webm, so old Safari's mp4/aac clips are not mislabelled. Two review findings were rejected as incorrect, both re-verified against live sources: google/gemini-3.7-flash does exist on OpenRouter (1M context, $0.375/$1.875 per M), and base64 JSON input_audio is the documented primary path for /audio/transcriptions, with multipart as the OpenAI-compatible alternative. The spec's stale "unverified" note is corrected, and the provider now has unit tests covering the request shape, usage parsing, and that a vendor error body never reaches the thrown message. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 19:47:10 +03:30
useEffect(() => {
// Re-armed on every mount: React StrictMode runs mount → unmount → mount in dev, and
// a ref that is only ever set false would leave the hook permanently "unmounted".
mountedRef.current = true;
return () => {
mountedRef.current = false;
cancelledRef.current = true;
abortRef.current?.abort();
try {
recorderRef.current?.stop();
} catch {
// already stopped
}
teardown();
};
feat(frontend): voice capture hook, API client and types MediaRecorder handling and the API call live in lib/, not in ui/, so TreatmentDetailsEditor can stay presentational and take only a `voice` prop. Container choice is made at record time and needs no transcode: Chrome and Android give webm/opus, Safari and iPad give mp4/aac, and the transcription endpoint accepts both. Safari's `audio/mp4` is sent as `m4a`, the name the vendor's container list actually uses, so iPad recordings do not fail while Chrome works. Older Safari shipped MediaRecorder without isTypeSupported, so that path lets the browser choose rather than refusing outright. From review of this commit: - The auto-stop at maxMs guaranteed a 413. The client measures the final length after the recorder has stopped, so a recording that runs to the cap always reports slightly over it, and the server rejected exactly the recording the auto-stop existed to save. The server now allows a documented 2s tolerance and the client keeps reporting the true length, so telemetry stays honest. - getUserMedia is async, so a permission granted after unmount installed a live stream the cleanup effect had already run past — leaving the browser's recording indicator lit with nothing listening. Guarded with a mounted ref. - Client-side failures are now ApiError-shaped ({code, statusCode}) rather than bare Errors, because getUserFacingError only resolves that shape; without it errors.VOICE_MIC_DENIED was dead in all three locales. Cancelling aborts the request, which closes the connection and aborts the metered vendor call server-side rather than letting it settle unseen. The level meter is best-effort: a blocked AudioContext costs the meter, not the recording. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 18:43:04 +03:30
}, [teardown]);
const send = useCallback(
async (blob: Blob, mimeType: string, durationMs: number) => {
setPhase('processing');
const controller = new AbortController();
abortRef.current = controller;
try {
const audio = await blobToBase64(blob);
const response = await voiceApi.extract(
{
audio,
format: mimeTypeToFormat(mimeType),
timeZone: Intl.DateTimeFormat().resolvedOptions().timeZone,
durationMs,
},
controller.signal,
);
if (cancelledRef.current) return;
onExtracted(response.data);
} catch (error) {
if (cancelledRef.current || controller.signal.aborted) return;
onError(error);
} finally {
abortRef.current = null;
setPhase('idle');
setElapsedMs(0);
}
},
[onExtracted, onError],
);
const stop = useCallback(() => {
try {
recorderRef.current?.stop();
} catch {
teardown();
setPhase('idle');
}
}, [teardown]);
const onStart = useCallback(() => {
feat(frontend): split Add detail into a segmented control with voice The microphone becomes the second segment of the Add detail button, built like the detail chip's trash affordance in the same file — an overflow-hidden rounded wrapper holding two raw <button>s divided by border-s — rather than two shared Buttons, which each hardcode their own rounding and would fight a segmented control. border-s puts the mic at the logical end: visually right in en/nl, visually left in fa, on the same side as the chip's trash in both directions. The two halves share a wrapper and nothing else. Add keeps its exact behaviour. The control never changes size while recording; the timer and level meter live in a bar between the header row and the chip strip, because the header is sm:justify-between and growing the button would shove the row on every start and stop. The meter exists to prove the microphone is actually hearing something — silence and a dead mic look identical otherwise. Voice reaches the editor as one optional `voice` prop, so its absence *is* the unavailable state and the two cannot disagree. Fixes from review of this commit: - mountedRef was set false on unmount and never re-armed, so under StrictMode the hook was permanently "unmounted" in dev and recording silently never started. - onStart guarded only on `phase`, which does not change until getUserMedia resolves; a second click during the permission prompt orphaned the first MediaStream, leaving the mic indicator lit. - Week start is now per locale. "Next Thursday" is week-relative, and hardcoding Saturday put an en/nl clinician's deadline a week out. - A missing `which` on a weekday intent is read as "this" rather than failing — a bare weekday carries no qualifier, and rejecting it discarded a real deadline. - durationMs is client-reported and so is a claim, not enforcement; the cap is now also checked against the vendor's own usage.seconds. - Blob type falls back to the recorder's actual mimeType before webm, so old Safari's mp4/aac clips are not mislabelled. Two review findings were rejected as incorrect, both re-verified against live sources: google/gemini-3.7-flash does exist on OpenRouter (1M context, $0.375/$1.875 per M), and base64 JSON input_audio is the documented primary path for /audio/transcriptions, with multipart as the OpenAI-compatible alternative. The spec's stale "unverified" note is corrected, and the provider now has unit tests covering the request shape, usage parsing, and that a vendor error body never reaches the thrown message. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 19:47:10 +03:30
if (phase !== 'idle' || startingRef.current) return;
feat(frontend): voice capture hook, API client and types MediaRecorder handling and the API call live in lib/, not in ui/, so TreatmentDetailsEditor can stay presentational and take only a `voice` prop. Container choice is made at record time and needs no transcode: Chrome and Android give webm/opus, Safari and iPad give mp4/aac, and the transcription endpoint accepts both. Safari's `audio/mp4` is sent as `m4a`, the name the vendor's container list actually uses, so iPad recordings do not fail while Chrome works. Older Safari shipped MediaRecorder without isTypeSupported, so that path lets the browser choose rather than refusing outright. From review of this commit: - The auto-stop at maxMs guaranteed a 413. The client measures the final length after the recorder has stopped, so a recording that runs to the cap always reports slightly over it, and the server rejected exactly the recording the auto-stop existed to save. The server now allows a documented 2s tolerance and the client keeps reporting the true length, so telemetry stays honest. - getUserMedia is async, so a permission granted after unmount installed a live stream the cleanup effect had already run past — leaving the browser's recording indicator lit with nothing listening. Guarded with a mounted ref. - Client-side failures are now ApiError-shaped ({code, statusCode}) rather than bare Errors, because getUserFacingError only resolves that shape; without it errors.VOICE_MIC_DENIED was dead in all three locales. Cancelling aborts the request, which closes the connection and aborts the metered vendor call server-side rather than letting it settle unseen. The level meter is best-effort: a blocked AudioContext costs the meter, not the recording. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 18:43:04 +03:30
if (!isMediaRecorderSupported()) {
onError(clientError('VOICE_MIC_DENIED'));
return;
}
cancelledRef.current = false;
feat(frontend): split Add detail into a segmented control with voice The microphone becomes the second segment of the Add detail button, built like the detail chip's trash affordance in the same file — an overflow-hidden rounded wrapper holding two raw <button>s divided by border-s — rather than two shared Buttons, which each hardcode their own rounding and would fight a segmented control. border-s puts the mic at the logical end: visually right in en/nl, visually left in fa, on the same side as the chip's trash in both directions. The two halves share a wrapper and nothing else. Add keeps its exact behaviour. The control never changes size while recording; the timer and level meter live in a bar between the header row and the chip strip, because the header is sm:justify-between and growing the button would shove the row on every start and stop. The meter exists to prove the microphone is actually hearing something — silence and a dead mic look identical otherwise. Voice reaches the editor as one optional `voice` prop, so its absence *is* the unavailable state and the two cannot disagree. Fixes from review of this commit: - mountedRef was set false on unmount and never re-armed, so under StrictMode the hook was permanently "unmounted" in dev and recording silently never started. - onStart guarded only on `phase`, which does not change until getUserMedia resolves; a second click during the permission prompt orphaned the first MediaStream, leaving the mic indicator lit. - Week start is now per locale. "Next Thursday" is week-relative, and hardcoding Saturday put an en/nl clinician's deadline a week out. - A missing `which` on a weekday intent is read as "this" rather than failing — a bare weekday carries no qualifier, and rejecting it discarded a real deadline. - durationMs is client-reported and so is a claim, not enforcement; the cap is now also checked against the vendor's own usage.seconds. - Blob type falls back to the recorder's actual mimeType before webm, so old Safari's mp4/aac clips are not mislabelled. Two review findings were rejected as incorrect, both re-verified against live sources: google/gemini-3.7-flash does exist on OpenRouter (1M context, $0.375/$1.875 per M), and base64 JSON input_audio is the documented primary path for /audio/transcriptions, with multipart as the OpenAI-compatible alternative. The spec's stale "unverified" note is corrected, and the provider now has unit tests covering the request shape, usage parsing, and that a vendor error body never reaches the thrown message. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 19:47:10 +03:30
startingRef.current = true;
feat(frontend): voice capture hook, API client and types MediaRecorder handling and the API call live in lib/, not in ui/, so TreatmentDetailsEditor can stay presentational and take only a `voice` prop. Container choice is made at record time and needs no transcode: Chrome and Android give webm/opus, Safari and iPad give mp4/aac, and the transcription endpoint accepts both. Safari's `audio/mp4` is sent as `m4a`, the name the vendor's container list actually uses, so iPad recordings do not fail while Chrome works. Older Safari shipped MediaRecorder without isTypeSupported, so that path lets the browser choose rather than refusing outright. From review of this commit: - The auto-stop at maxMs guaranteed a 413. The client measures the final length after the recorder has stopped, so a recording that runs to the cap always reports slightly over it, and the server rejected exactly the recording the auto-stop existed to save. The server now allows a documented 2s tolerance and the client keeps reporting the true length, so telemetry stays honest. - getUserMedia is async, so a permission granted after unmount installed a live stream the cleanup effect had already run past — leaving the browser's recording indicator lit with nothing listening. Guarded with a mounted ref. - Client-side failures are now ApiError-shaped ({code, statusCode}) rather than bare Errors, because getUserFacingError only resolves that shape; without it errors.VOICE_MIC_DENIED was dead in all three locales. Cancelling aborts the request, which closes the connection and aborts the metered vendor call server-side rather than letting it settle unseen. The level meter is best-effort: a blocked AudioContext costs the meter, not the recording. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 18:43:04 +03:30
void (async () => {
try {
feat(frontend): split Add detail into a segmented control with voice The microphone becomes the second segment of the Add detail button, built like the detail chip's trash affordance in the same file — an overflow-hidden rounded wrapper holding two raw <button>s divided by border-s — rather than two shared Buttons, which each hardcode their own rounding and would fight a segmented control. border-s puts the mic at the logical end: visually right in en/nl, visually left in fa, on the same side as the chip's trash in both directions. The two halves share a wrapper and nothing else. Add keeps its exact behaviour. The control never changes size while recording; the timer and level meter live in a bar between the header row and the chip strip, because the header is sm:justify-between and growing the button would shove the row on every start and stop. The meter exists to prove the microphone is actually hearing something — silence and a dead mic look identical otherwise. Voice reaches the editor as one optional `voice` prop, so its absence *is* the unavailable state and the two cannot disagree. Fixes from review of this commit: - mountedRef was set false on unmount and never re-armed, so under StrictMode the hook was permanently "unmounted" in dev and recording silently never started. - onStart guarded only on `phase`, which does not change until getUserMedia resolves; a second click during the permission prompt orphaned the first MediaStream, leaving the mic indicator lit. - Week start is now per locale. "Next Thursday" is week-relative, and hardcoding Saturday put an en/nl clinician's deadline a week out. - A missing `which` on a weekday intent is read as "this" rather than failing — a bare weekday carries no qualifier, and rejecting it discarded a real deadline. - durationMs is client-reported and so is a claim, not enforcement; the cap is now also checked against the vendor's own usage.seconds. - Blob type falls back to the recorder's actual mimeType before webm, so old Safari's mp4/aac clips are not mislabelled. Two review findings were rejected as incorrect, both re-verified against live sources: google/gemini-3.7-flash does exist on OpenRouter (1M context, $0.375/$1.875 per M), and base64 JSON input_audio is the documented primary path for /audio/transcriptions, with multipart as the OpenAI-compatible alternative. The spec's stale "unverified" note is corrected, and the provider now has unit tests covering the request shape, usage parsing, and that a vendor error body never reaches the thrown message. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 19:47:10 +03:30
let stream: MediaStream;
try {
stream = await navigator.mediaDevices.getUserMedia({ audio: true });
} catch {
// Permission refused, or no input device. Never a server round-trip.
onError(clientError('VOICE_MIC_DENIED'));
feat(frontend): voice capture hook, API client and types MediaRecorder handling and the API call live in lib/, not in ui/, so TreatmentDetailsEditor can stay presentational and take only a `voice` prop. Container choice is made at record time and needs no transcode: Chrome and Android give webm/opus, Safari and iPad give mp4/aac, and the transcription endpoint accepts both. Safari's `audio/mp4` is sent as `m4a`, the name the vendor's container list actually uses, so iPad recordings do not fail while Chrome works. Older Safari shipped MediaRecorder without isTypeSupported, so that path lets the browser choose rather than refusing outright. From review of this commit: - The auto-stop at maxMs guaranteed a 413. The client measures the final length after the recorder has stopped, so a recording that runs to the cap always reports slightly over it, and the server rejected exactly the recording the auto-stop existed to save. The server now allows a documented 2s tolerance and the client keeps reporting the true length, so telemetry stays honest. - getUserMedia is async, so a permission granted after unmount installed a live stream the cleanup effect had already run past — leaving the browser's recording indicator lit with nothing listening. Guarded with a mounted ref. - Client-side failures are now ApiError-shaped ({code, statusCode}) rather than bare Errors, because getUserFacingError only resolves that shape; without it errors.VOICE_MIC_DENIED was dead in all three locales. Cancelling aborts the request, which closes the connection and aborts the metered vendor call server-side rather than letting it settle unseen. The level meter is best-effort: a blocked AudioContext costs the meter, not the recording. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 18:43:04 +03:30
return;
}
feat(frontend): split Add detail into a segmented control with voice The microphone becomes the second segment of the Add detail button, built like the detail chip's trash affordance in the same file — an overflow-hidden rounded wrapper holding two raw <button>s divided by border-s — rather than two shared Buttons, which each hardcode their own rounding and would fight a segmented control. border-s puts the mic at the logical end: visually right in en/nl, visually left in fa, on the same side as the chip's trash in both directions. The two halves share a wrapper and nothing else. Add keeps its exact behaviour. The control never changes size while recording; the timer and level meter live in a bar between the header row and the chip strip, because the header is sm:justify-between and growing the button would shove the row on every start and stop. The meter exists to prove the microphone is actually hearing something — silence and a dead mic look identical otherwise. Voice reaches the editor as one optional `voice` prop, so its absence *is* the unavailable state and the two cannot disagree. Fixes from review of this commit: - mountedRef was set false on unmount and never re-armed, so under StrictMode the hook was permanently "unmounted" in dev and recording silently never started. - onStart guarded only on `phase`, which does not change until getUserMedia resolves; a second click during the permission prompt orphaned the first MediaStream, leaving the mic indicator lit. - Week start is now per locale. "Next Thursday" is week-relative, and hardcoding Saturday put an en/nl clinician's deadline a week out. - A missing `which` on a weekday intent is read as "this" rather than failing — a bare weekday carries no qualifier, and rejecting it discarded a real deadline. - durationMs is client-reported and so is a claim, not enforcement; the cap is now also checked against the vendor's own usage.seconds. - Blob type falls back to the recorder's actual mimeType before webm, so old Safari's mp4/aac clips are not mislabelled. Two review findings were rejected as incorrect, both re-verified against live sources: google/gemini-3.7-flash does exist on OpenRouter (1M context, $0.375/$1.875 per M), and base64 JSON input_audio is the documented primary path for /audio/transcriptions, with multipart as the OpenAI-compatible alternative. The spec's stale "unverified" note is corrected, and the provider now has unit tests covering the request shape, usage parsing, and that a vendor error body never reaches the thrown message. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 19:47:10 +03:30
if (!mountedRef.current) {
// Permission resolved after the component went away — release it immediately
// rather than leaving the browser's recording indicator lit.
stream.getTracks().forEach((track) => track.stop());
return;
}
feat(frontend): voice capture hook, API client and types MediaRecorder handling and the API call live in lib/, not in ui/, so TreatmentDetailsEditor can stay presentational and take only a `voice` prop. Container choice is made at record time and needs no transcode: Chrome and Android give webm/opus, Safari and iPad give mp4/aac, and the transcription endpoint accepts both. Safari's `audio/mp4` is sent as `m4a`, the name the vendor's container list actually uses, so iPad recordings do not fail while Chrome works. Older Safari shipped MediaRecorder without isTypeSupported, so that path lets the browser choose rather than refusing outright. From review of this commit: - The auto-stop at maxMs guaranteed a 413. The client measures the final length after the recorder has stopped, so a recording that runs to the cap always reports slightly over it, and the server rejected exactly the recording the auto-stop existed to save. The server now allows a documented 2s tolerance and the client keeps reporting the true length, so telemetry stays honest. - getUserMedia is async, so a permission granted after unmount installed a live stream the cleanup effect had already run past — leaving the browser's recording indicator lit with nothing listening. Guarded with a mounted ref. - Client-side failures are now ApiError-shaped ({code, statusCode}) rather than bare Errors, because getUserFacingError only resolves that shape; without it errors.VOICE_MIC_DENIED was dead in all three locales. Cancelling aborts the request, which closes the connection and aborts the metered vendor call server-side rather than letting it settle unseen. The level meter is best-effort: a blocked AudioContext costs the meter, not the recording. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 18:43:04 +03:30
feat(frontend): split Add detail into a segmented control with voice The microphone becomes the second segment of the Add detail button, built like the detail chip's trash affordance in the same file — an overflow-hidden rounded wrapper holding two raw <button>s divided by border-s — rather than two shared Buttons, which each hardcode their own rounding and would fight a segmented control. border-s puts the mic at the logical end: visually right in en/nl, visually left in fa, on the same side as the chip's trash in both directions. The two halves share a wrapper and nothing else. Add keeps its exact behaviour. The control never changes size while recording; the timer and level meter live in a bar between the header row and the chip strip, because the header is sm:justify-between and growing the button would shove the row on every start and stop. The meter exists to prove the microphone is actually hearing something — silence and a dead mic look identical otherwise. Voice reaches the editor as one optional `voice` prop, so its absence *is* the unavailable state and the two cannot disagree. Fixes from review of this commit: - mountedRef was set false on unmount and never re-armed, so under StrictMode the hook was permanently "unmounted" in dev and recording silently never started. - onStart guarded only on `phase`, which does not change until getUserMedia resolves; a second click during the permission prompt orphaned the first MediaStream, leaving the mic indicator lit. - Week start is now per locale. "Next Thursday" is week-relative, and hardcoding Saturday put an en/nl clinician's deadline a week out. - A missing `which` on a weekday intent is read as "this" rather than failing — a bare weekday carries no qualifier, and rejecting it discarded a real deadline. - durationMs is client-reported and so is a claim, not enforcement; the cap is now also checked against the vendor's own usage.seconds. - Blob type falls back to the recorder's actual mimeType before webm, so old Safari's mp4/aac clips are not mislabelled. Two review findings were rejected as incorrect, both re-verified against live sources: google/gemini-3.7-flash does exist on OpenRouter (1M context, $0.375/$1.875 per M), and base64 JSON input_audio is the documented primary path for /audio/transcriptions, with multipart as the OpenAI-compatible alternative. The spec's stale "unverified" note is corrected, and the provider now has unit tests covering the request shape, usage parsing, and that a vendor error body never reaches the thrown message. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 19:47:10 +03:30
const mimeType = pickRecordingMimeType();
if (mimeType === null) {
stream.getTracks().forEach((track) => track.stop());
onError(clientError('VOICE_MIC_DENIED'));
return;
}
feat(frontend): voice capture hook, API client and types MediaRecorder handling and the API call live in lib/, not in ui/, so TreatmentDetailsEditor can stay presentational and take only a `voice` prop. Container choice is made at record time and needs no transcode: Chrome and Android give webm/opus, Safari and iPad give mp4/aac, and the transcription endpoint accepts both. Safari's `audio/mp4` is sent as `m4a`, the name the vendor's container list actually uses, so iPad recordings do not fail while Chrome works. Older Safari shipped MediaRecorder without isTypeSupported, so that path lets the browser choose rather than refusing outright. From review of this commit: - The auto-stop at maxMs guaranteed a 413. The client measures the final length after the recorder has stopped, so a recording that runs to the cap always reports slightly over it, and the server rejected exactly the recording the auto-stop existed to save. The server now allows a documented 2s tolerance and the client keeps reporting the true length, so telemetry stays honest. - getUserMedia is async, so a permission granted after unmount installed a live stream the cleanup effect had already run past — leaving the browser's recording indicator lit with nothing listening. Guarded with a mounted ref. - Client-side failures are now ApiError-shaped ({code, statusCode}) rather than bare Errors, because getUserFacingError only resolves that shape; without it errors.VOICE_MIC_DENIED was dead in all three locales. Cancelling aborts the request, which closes the connection and aborts the metered vendor call server-side rather than letting it settle unseen. The level meter is best-effort: a blocked AudioContext costs the meter, not the recording. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 18:43:04 +03:30
feat(frontend): split Add detail into a segmented control with voice The microphone becomes the second segment of the Add detail button, built like the detail chip's trash affordance in the same file — an overflow-hidden rounded wrapper holding two raw <button>s divided by border-s — rather than two shared Buttons, which each hardcode their own rounding and would fight a segmented control. border-s puts the mic at the logical end: visually right in en/nl, visually left in fa, on the same side as the chip's trash in both directions. The two halves share a wrapper and nothing else. Add keeps its exact behaviour. The control never changes size while recording; the timer and level meter live in a bar between the header row and the chip strip, because the header is sm:justify-between and growing the button would shove the row on every start and stop. The meter exists to prove the microphone is actually hearing something — silence and a dead mic look identical otherwise. Voice reaches the editor as one optional `voice` prop, so its absence *is* the unavailable state and the two cannot disagree. Fixes from review of this commit: - mountedRef was set false on unmount and never re-armed, so under StrictMode the hook was permanently "unmounted" in dev and recording silently never started. - onStart guarded only on `phase`, which does not change until getUserMedia resolves; a second click during the permission prompt orphaned the first MediaStream, leaving the mic indicator lit. - Week start is now per locale. "Next Thursday" is week-relative, and hardcoding Saturday put an en/nl clinician's deadline a week out. - A missing `which` on a weekday intent is read as "this" rather than failing — a bare weekday carries no qualifier, and rejecting it discarded a real deadline. - durationMs is client-reported and so is a claim, not enforcement; the cap is now also checked against the vendor's own usage.seconds. - Blob type falls back to the recorder's actual mimeType before webm, so old Safari's mp4/aac clips are not mislabelled. Two review findings were rejected as incorrect, both re-verified against live sources: google/gemini-3.7-flash does exist on OpenRouter (1M context, $0.375/$1.875 per M), and base64 JSON input_audio is the documented primary path for /audio/transcriptions, with multipart as the OpenAI-compatible alternative. The spec's stale "unverified" note is corrected, and the provider now has unit tests covering the request shape, usage parsing, and that a vendor error body never reaches the thrown message. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 19:47:10 +03:30
streamRef.current = stream;
chunksRef.current = [];
const recorder = new MediaRecorder(stream, mimeType ? { mimeType } : undefined);
recorderRef.current = recorder;
recorder.ondataavailable = (event) => {
if (event.data.size > 0) chunksRef.current.push(event.data);
};
recorder.onstop = () => {
const durationMs = Date.now() - startedAtRef.current;
const blob = new Blob(chunksRef.current, { type: recorder.mimeType || mimeType });
teardown();
if (cancelledRef.current || blob.size === 0) {
setPhase('idle');
setElapsedMs(0);
return;
}
// Prefer what the recorder actually produced, then the blob's own type. Old
// Safari accepts no mimeType hint, and defaulting to webm would mislabel its
// mp4/aac clips as something they are not.
void send(blob, recorder.mimeType || blob.type || mimeType || 'audio/webm', durationMs);
};
attachLevelMeter(stream, audioContextRef, setLevel);
startedAtRef.current = Date.now();
recorder.start();
setPhase('recording');
setElapsedMs(0);
timerRef.current = setInterval(() => {
const elapsed = Date.now() - startedAtRef.current;
setElapsedMs(elapsed);
// Auto-stop proceeds to processing with what was captured; discarding two
// minutes of dictation because a timer expired would be the worst failure.
if (maxMs != null && elapsed >= maxMs) stop();
}, LEVEL_POLL_MS);
} finally {
startingRef.current = false;
}
feat(frontend): voice capture hook, API client and types MediaRecorder handling and the API call live in lib/, not in ui/, so TreatmentDetailsEditor can stay presentational and take only a `voice` prop. Container choice is made at record time and needs no transcode: Chrome and Android give webm/opus, Safari and iPad give mp4/aac, and the transcription endpoint accepts both. Safari's `audio/mp4` is sent as `m4a`, the name the vendor's container list actually uses, so iPad recordings do not fail while Chrome works. Older Safari shipped MediaRecorder without isTypeSupported, so that path lets the browser choose rather than refusing outright. From review of this commit: - The auto-stop at maxMs guaranteed a 413. The client measures the final length after the recorder has stopped, so a recording that runs to the cap always reports slightly over it, and the server rejected exactly the recording the auto-stop existed to save. The server now allows a documented 2s tolerance and the client keeps reporting the true length, so telemetry stays honest. - getUserMedia is async, so a permission granted after unmount installed a live stream the cleanup effect had already run past — leaving the browser's recording indicator lit with nothing listening. Guarded with a mounted ref. - Client-side failures are now ApiError-shaped ({code, statusCode}) rather than bare Errors, because getUserFacingError only resolves that shape; without it errors.VOICE_MIC_DENIED was dead in all three locales. Cancelling aborts the request, which closes the connection and aborts the metered vendor call server-side rather than letting it settle unseen. The level meter is best-effort: a blocked AudioContext costs the meter, not the recording. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 18:43:04 +03:30
})();
}, [maxMs, onError, phase, send, stop, teardown]);
const onCancel = useCallback(() => {
cancelledRef.current = true;
// Aborting closes the connection, which aborts the vendor call server-side. It is
// metered per minute, so letting it settle costs money for a result nobody sees.
abortRef.current?.abort();
try {
recorderRef.current?.stop();
} catch {
// already stopped
}
teardown();
setPhase('idle');
setElapsedMs(0);
}, [teardown]);
return { phase, elapsedMs, level, maxMs, onStart, onStop: stop, onCancel };
}
/** Drives the level meter from the live stream; failure here must not stop recording. */
function attachLevelMeter(
stream: MediaStream,
contextRef: React.MutableRefObject<AudioContext | null>,
setLevel: (value: number) => void,
) {
try {
const AudioContextCtor =
window.AudioContext ?? (window as { webkitAudioContext?: typeof AudioContext }).webkitAudioContext;
if (!AudioContextCtor) return;
const context = new AudioContextCtor();
contextRef.current = context;
const source = context.createMediaStreamSource(stream);
const analyser = context.createAnalyser();
analyser.fftSize = 512;
source.connect(analyser);
const data = new Uint8Array(analyser.frequencyBinCount);
const tick = () => {
if (contextRef.current !== context || context.state === 'closed') return;
analyser.getByteTimeDomainData(data);
let peak = 0;
for (const sample of data) peak = Math.max(peak, Math.abs(sample - 128));
setLevel(Math.min(1, peak / 128));
requestAnimationFrame(tick);
};
requestAnimationFrame(tick);
} catch {
// A missing or blocked AudioContext costs the meter, not the recording.
}
}