Files
dyolink/backend/src/modules/voice/extraction.resolver.ts

308 lines
9.2 KiB
TypeScript
Raw Normal View History

feat(backend): assemble resolved extraction from voice intents Composes the tooth, span, prosthesis, catalog and date resolvers into the payload the review sheet renders. Connected spans expand: "a bridge from 14 to 16" selects 15, which was never spoken. Overlapping spans merge into one bridge, group teeth sort along the arch (16-15-14, and 11 beside 21 across the midline), and a span collapsing to a single tooth degrades to a single group without losing that tooth — there is no such thing as a one-tooth bridge. A cross-arch span is impossible and is reported rather than guessed at. Prosthesis expands a default across the selection then applies per-tooth overrides, because "همه زیرکونیا، ۲۶ پی‌اف‌ام" is how clinicians actually speak. Completeness is computed here so an unshippable map surfaces at review rather than failing later at dispatch. Everything the model names is checked against the catalog we supplied it, and anything rejected is reported rather than dropped — a hallucinated lab id must not look identical to "no lab was spoken", since silence and a wrong lab lead to very different corrective actions. Also fixed, from review of this commit: - an empty prosthesis object no longer fabricates an "incomplete, cannot ship" warning on a plain restoration - an override naming a tooth outside the selection now reports tooth_not_selected rather than malformed; the clinician was understood, the tooth just is not on this detail - a due object with no `kind` is treated as no deadline rather than a blank "heard but lost" row; an unrecognised kind is still flagged, and named Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 17:41:41 +03:30
import {
sameArch,
sortInArchOrder,
teethBetweenInclusive,
} from '../../common/fdi';
import { resolveDueDate } from './due-date.resolver';
import {
resolveToothIntent,
resolveToothIntents,
} from './tooth-intent.resolver';
import type {
ConnectedSpanIntent,
ProsthesisIntent,
ToothIntent,
UnresolvedItem,
VoiceIntent,
} from './voice.types';
export type ResolvedToothGroup = {
groupId: string;
kind: 'connected' | 'single';
teeth: string[];
};
export type ResolvedProsthesis = {
/** FDI code → prosthesis type code. */
byTooth: Record<string, string>;
/**
* True when every selected tooth carries a code. A prosthesis detail cannot be shipped
* otherwise (`assertCompleteToothProsthesisMap`), so the review sheet surfaces the gap
* here rather than letting it fail at dispatch.
*/
complete: boolean;
missingTeeth: string[];
};
export type ResolvedExtraction = {
treatmentType: string | null;
teeth: string[];
toothSelectionGroups: ResolvedToothGroup[];
comment: string | null;
prosthesis: ResolvedProsthesis | null;
labId: string | null;
labMatchExact: boolean;
dueDate: string | null;
unresolved: UnresolvedItem[];
};
export type ResolveContext = {
todayIso: string;
feat(frontend): split Add detail into a segmented control with voice The microphone becomes the second segment of the Add detail button, built like the detail chip's trash affordance in the same file — an overflow-hidden rounded wrapper holding two raw <button>s divided by border-s — rather than two shared Buttons, which each hardcode their own rounding and would fight a segmented control. border-s puts the mic at the logical end: visually right in en/nl, visually left in fa, on the same side as the chip's trash in both directions. The two halves share a wrapper and nothing else. Add keeps its exact behaviour. The control never changes size while recording; the timer and level meter live in a bar between the header row and the chip strip, because the header is sm:justify-between and growing the button would shove the row on every start and stop. The meter exists to prove the microphone is actually hearing something — silence and a dead mic look identical otherwise. Voice reaches the editor as one optional `voice` prop, so its absence *is* the unavailable state and the two cannot disagree. Fixes from review of this commit: - mountedRef was set false on unmount and never re-armed, so under StrictMode the hook was permanently "unmounted" in dev and recording silently never started. - onStart guarded only on `phase`, which does not change until getUserMedia resolves; a second click during the permission prompt orphaned the first MediaStream, leaving the mic indicator lit. - Week start is now per locale. "Next Thursday" is week-relative, and hardcoding Saturday put an en/nl clinician's deadline a week out. - A missing `which` on a weekday intent is read as "this" rather than failing — a bare weekday carries no qualifier, and rejecting it discarded a real deadline. - durationMs is client-reported and so is a claim, not enforcement; the cap is now also checked against the vendor's own usage.seconds. - Blob type falls back to the recorder's actual mimeType before webm, so old Safari's mp4/aac clips are not mislabelled. Two review findings were rejected as incorrect, both re-verified against live sources: google/gemini-3.7-flash does exist on OpenRouter (1M context, $0.375/$1.875 per M), and base64 JSON input_audio is the documented primary path for /audio/transcriptions, with multipart as the OpenAI-compatible alternative. The spec's stale "unverified" note is corrected, and the provider now has unit tests covering the request shape, usage parsing, and that a vendor error body never reaches the thrown message. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 19:47:10 +03:30
/** JS weekday index the clinician's week starts on — see weekStartForLocale. */
weekStartJs: number;
feat(backend): assemble resolved extraction from voice intents Composes the tooth, span, prosthesis, catalog and date resolvers into the payload the review sheet renders. Connected spans expand: "a bridge from 14 to 16" selects 15, which was never spoken. Overlapping spans merge into one bridge, group teeth sort along the arch (16-15-14, and 11 beside 21 across the midline), and a span collapsing to a single tooth degrades to a single group without losing that tooth — there is no such thing as a one-tooth bridge. A cross-arch span is impossible and is reported rather than guessed at. Prosthesis expands a default across the selection then applies per-tooth overrides, because "همه زیرکونیا، ۲۶ پی‌اف‌ام" is how clinicians actually speak. Completeness is computed here so an unshippable map surfaces at review rather than failing later at dispatch. Everything the model names is checked against the catalog we supplied it, and anything rejected is reported rather than dropped — a hallucinated lab id must not look identical to "no lab was spoken", since silence and a wrong lab lead to very different corrective actions. Also fixed, from review of this commit: - an empty prosthesis object no longer fabricates an "incomplete, cannot ship" warning on a plain restoration - an override naming a tooth outside the selection now reports tooth_not_selected rather than malformed; the clinician was understood, the tooth just is not on this detail - a due object with no `kind` is treated as no deadline rather than a blank "heard but lost" row; an unrecognised kind is still flagged, and named Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 17:41:41 +03:30
treatmentTypeCodes: ReadonlySet<string>;
prosthesisTypeCodes: ReadonlySet<string>;
linkedLabIds: ReadonlySet<string>;
};
function spokenOf(intent: ToothIntent): string {
const spoken = (intent as { spoken?: unknown })?.spoken;
return typeof spoken === 'string' && spoken.trim() ? spoken.trim() : '';
}
/** A code the model returned is only usable if it exists in the catalog we supplied it. */
function resolveCatalogCode(
value: unknown,
allowed: ReadonlySet<string>,
): string | null {
if (typeof value !== 'string') return null;
const trimmed = value.trim();
return trimmed && allowed.has(trimmed) ? trimmed : null;
}
/** Merge any span sets that share a tooth, so overlapping bridges become one group. */
function mergeOverlapping(sets: string[][]): string[][] {
const merged: string[][] = [];
for (const candidate of sets) {
let current = [...candidate];
let index = 0;
while (index < merged.length) {
if (merged[index].some((tooth) => current.includes(tooth))) {
current = [...new Set([...merged[index], ...current])];
merged.splice(index, 1);
index = 0;
continue;
}
index += 1;
}
merged.push(current);
}
return merged;
}
/**
* Turn spoken bridge spans plus loose teeth into selection groups.
*
* Span teeth are added to the selection: saying "a bridge from 14 to 16" selects 15 even
* though it was never named. A span whose endpoints are in different arches is impossible
* and is reported rather than guessed at. A span that collapses to one tooth degrades to a
* single there is no such thing as a one-tooth bridge.
*/
export function resolveConnectedSpans(
spans: readonly ConnectedSpanIntent[],
selectedTeeth: readonly string[],
): {
groups: ResolvedToothGroup[];
teeth: string[];
unresolved: UnresolvedItem[];
} {
const unresolved: UnresolvedItem[] = [];
const connectedSets: string[][] = [];
// A span that collapses to one tooth still selected that tooth — it must not vanish.
const loneSpanTeeth: string[] = [];
const list: readonly ConnectedSpanIntent[] = Array.isArray(spans)
? (spans as readonly ConnectedSpanIntent[])
: [];
for (const span of list) {
const from = resolveToothIntent(span?.from);
const to = resolveToothIntent(span?.to);
const spoken = [spokenOf(span?.from), spokenOf(span?.to)]
.filter(Boolean)
.join(' → ');
if (!from || !to) {
unresolved.push({ spoken, reason: 'malformed' });
continue;
}
if (!sameArch(from, to)) {
unresolved.push({ spoken, reason: 'span_not_same_arch' });
continue;
}
const between = teethBetweenInclusive(from, to);
if (!between || between.length === 0) {
unresolved.push({ spoken, reason: 'malformed' });
continue;
}
if (between.length === 1) {
loneSpanTeeth.push(between[0]);
continue; // degrades to a single, below
}
connectedSets.push(between);
}
const groups: ResolvedToothGroup[] = [];
const claimed = new Set<string>();
mergeOverlapping(connectedSets).forEach((set, i) => {
const teeth = sortInArchOrder(set);
teeth.forEach((tooth) => claimed.add(tooth));
groups.push({ groupId: `voice-c${i + 1}`, kind: 'connected', teeth });
});
const spanTeeth = [...groups.flatMap((g) => g.teeth), ...loneSpanTeeth];
const singles = [...new Set([...selectedTeeth, ...spanTeeth])]
.filter((tooth) => !claimed.has(tooth))
.sort();
for (const tooth of singles) {
groups.push({
groupId: `voice-s-${tooth}`,
kind: 'single',
teeth: [tooth],
});
}
return {
groups,
teeth: [...new Set([...selectedTeeth, ...spanTeeth])].sort(),
unresolved,
};
}
/**
* Expand a default prosthesis type across the selection, then apply per-tooth overrides.
*
* "همه زیرکونیا، ۲۶ پی‌اف‌ام" is how clinicians actually speak, so the model names the type
* once and overrides the exceptions.
*/
export function resolveProsthesis(
intent: ProsthesisIntent | null | undefined,
teeth: readonly string[],
allowed: ReadonlySet<string>,
): { prosthesis: ResolvedProsthesis | null; unresolved: UnresolvedItem[] } {
if (!intent || typeof intent !== 'object')
return { prosthesis: null, unresolved: [] };
const unresolved: UnresolvedItem[] = [];
const defaultType = resolveCatalogCode(intent.defaultType, allowed);
if (intent.defaultType != null && !defaultType) {
unresolved.push({
spoken: String(intent.defaultType),
reason: 'unknown_catalog_code',
});
}
const byTooth: Record<string, string> = {};
const selected = new Set(teeth);
if (defaultType) {
for (const tooth of teeth) byTooth[tooth] = defaultType;
}
const overrides: ProsthesisIntent['overrides'] = Array.isArray(
intent.overrides,
)
? intent.overrides
: [];
for (const override of overrides) {
const tooth = resolveToothIntent(override?.tooth);
const type = resolveCatalogCode(override?.type, allowed);
const spoken = spokenOf(override?.tooth) || String(override?.type ?? '');
if (!tooth) {
unresolved.push({ spoken, reason: 'malformed' });
continue;
}
// A tooth we understood perfectly well but which is not part of this detail. Saying
// so is actionable ("add tooth 37, or drop it"); calling it malformed is not.
if (!selected.has(tooth)) {
unresolved.push({ spoken, reason: 'tooth_not_selected' });
continue;
}
if (!type) {
unresolved.push({ spoken, reason: 'unknown_catalog_code' });
continue;
}
byTooth[tooth] = type;
}
// Nothing usable was said about prosthesis. Returning an empty-but-present map would
// paint a plain restoration with a fabricated "incomplete, cannot ship" warning.
if (Object.keys(byTooth).length === 0) {
return { prosthesis: null, unresolved };
}
const missingTeeth = teeth.filter((tooth) => !byTooth[tooth]);
return {
prosthesis: {
byTooth,
complete: missingTeeth.length === 0,
missingTeeth,
},
unresolved,
};
}
/** Compose every resolver into the payload the review sheet renders. */
export function resolveVoiceIntent(
intent: VoiceIntent,
ctx: ResolveContext,
): ResolvedExtraction {
const unresolved: UnresolvedItem[] = [];
const toothResult = resolveToothIntents(intent?.teeth ?? []);
unresolved.push(...toothResult.unresolved);
const spanResult = resolveConnectedSpans(
intent?.connectedSpans ?? [],
toothResult.teeth,
);
unresolved.push(...spanResult.unresolved);
const treatmentType = resolveCatalogCode(
intent?.treatmentType,
ctx.treatmentTypeCodes,
);
if (intent?.treatmentType != null && !treatmentType) {
unresolved.push({
spoken: String(intent.treatmentType),
reason: 'unknown_catalog_code',
});
}
const prosthesisResult = resolveProsthesis(
intent?.prosthesis,
spanResult.teeth,
ctx.prosthesisTypeCodes,
);
unresolved.push(...prosthesisResult.unresolved);
feat(frontend): split Add detail into a segmented control with voice The microphone becomes the second segment of the Add detail button, built like the detail chip's trash affordance in the same file — an overflow-hidden rounded wrapper holding two raw <button>s divided by border-s — rather than two shared Buttons, which each hardcode their own rounding and would fight a segmented control. border-s puts the mic at the logical end: visually right in en/nl, visually left in fa, on the same side as the chip's trash in both directions. The two halves share a wrapper and nothing else. Add keeps its exact behaviour. The control never changes size while recording; the timer and level meter live in a bar between the header row and the chip strip, because the header is sm:justify-between and growing the button would shove the row on every start and stop. The meter exists to prove the microphone is actually hearing something — silence and a dead mic look identical otherwise. Voice reaches the editor as one optional `voice` prop, so its absence *is* the unavailable state and the two cannot disagree. Fixes from review of this commit: - mountedRef was set false on unmount and never re-armed, so under StrictMode the hook was permanently "unmounted" in dev and recording silently never started. - onStart guarded only on `phase`, which does not change until getUserMedia resolves; a second click during the permission prompt orphaned the first MediaStream, leaving the mic indicator lit. - Week start is now per locale. "Next Thursday" is week-relative, and hardcoding Saturday put an en/nl clinician's deadline a week out. - A missing `which` on a weekday intent is read as "this" rather than failing — a bare weekday carries no qualifier, and rejecting it discarded a real deadline. - durationMs is client-reported and so is a claim, not enforcement; the cap is now also checked against the vendor's own usage.seconds. - Blob type falls back to the recorder's actual mimeType before webm, so old Safari's mp4/aac clips are not mislabelled. Two review findings were rejected as incorrect, both re-verified against live sources: google/gemini-3.7-flash does exist on OpenRouter (1M context, $0.375/$1.875 per M), and base64 JSON input_audio is the documented primary path for /audio/transcriptions, with multipart as the OpenAI-compatible alternative. The spec's stale "unverified" note is corrected, and the provider now has unit tests covering the request shape, usage parsing, and that a vendor error body never reaches the thrown message. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 19:47:10 +03:30
const due = resolveDueDate(intent?.due, ctx.todayIso, ctx.weekStartJs);
feat(backend): assemble resolved extraction from voice intents Composes the tooth, span, prosthesis, catalog and date resolvers into the payload the review sheet renders. Connected spans expand: "a bridge from 14 to 16" selects 15, which was never spoken. Overlapping spans merge into one bridge, group teeth sort along the arch (16-15-14, and 11 beside 21 across the midline), and a span collapsing to a single tooth degrades to a single group without losing that tooth — there is no such thing as a one-tooth bridge. A cross-arch span is impossible and is reported rather than guessed at. Prosthesis expands a default across the selection then applies per-tooth overrides, because "همه زیرکونیا، ۲۶ پی‌اف‌ام" is how clinicians actually speak. Completeness is computed here so an unshippable map surfaces at review rather than failing later at dispatch. Everything the model names is checked against the catalog we supplied it, and anything rejected is reported rather than dropped — a hallucinated lab id must not look identical to "no lab was spoken", since silence and a wrong lab lead to very different corrective actions. Also fixed, from review of this commit: - an empty prosthesis object no longer fabricates an "incomplete, cannot ship" warning on a plain restoration - an override naming a tooth outside the selection now reports tooth_not_selected rather than malformed; the clinician was understood, the tooth just is not on this detail - a due object with no `kind` is treated as no deadline rather than a blank "heard but lost" row; an unrecognised kind is still flagged, and named Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 17:41:41 +03:30
if (due.unresolved) unresolved.push(due.unresolved);
const comment =
typeof intent?.comment === 'string' && intent.comment.trim()
? intent.comment.trim()
: null;
// A lab id the model invented is worse than none — it would ship a case to a lab the
// clinic never named. Only ids from the list we supplied survive, and a rejected one is
// reported: a hallucinated lab must not look identical to "no lab was spoken".
const labId = resolveCatalogCode(intent?.labId, ctx.linkedLabIds);
if (intent?.labId != null && !labId) {
unresolved.push({
spoken: String(intent.labId),
reason: 'unknown_catalog_code',
});
}
return {
treatmentType,
teeth: spanResult.teeth,
toothSelectionGroups: spanResult.groups,
comment,
prosthesis: prosthesisResult.prosthesis,
labId,
labMatchExact: labId ? intent?.labMatchExact === true : false,
dueDate: due.dueDate,
unresolved,
};
}