Overview
your app @amphivox/react AmphiVox iframe voice agent
──────── ─────────────── ─────────────── ───────────
form state, its rules, ──► page prop / reportPage ──► relays the page ──► asks, checks,
Next and submit controls (open, mute…) and your controls writes values
◄── onField, onAdvancePage, ◄── relays the agent ◄──
onSessionEnd, …
| Your app owns | The agent owns |
|---|---|
| The form, its fields and how they look | Asking the questions, in the visitor's language |
| Validation rules the schema can't express | Checking answers against the schema's rules |
| Going to the next page | Asking the visitor before it moves on |
Submitting, after an ending of complete | Telling you how the session ended |
An integration makes three choices, and they are independent of each other:
| Choice | Options | It decides |
|---|---|---|
| The widget's mode | 'form-linked' or 'developer-defined' | Where the schema (the list of fields the agent asks for) lives. Set when you create the widget key. |
| Our UI or yours | Our dock (<VoiceWidget> (opens in a new tab)), or your own UI (useVoiceWidget() (opens in a new tab) with hidden (opens in a new tab)) | Who draws the voice controls: start and stop, mute, language and voice, the transcript, and the speaking and thinking indicators. It changes nothing about your form. |
| Who fills your inputs | Always your app, through onField (opens in a new tab): React owns your form state | How an answer gets from the agent into your form. (The vanilla widget.js also has autofill, which fills plain HTML inputs for you; a React app doesn't need it.) |
Every widget key is created in the Widgets area of the AmphiVox dashboard, in one of two modes. The mode you pass must match the key:
mode | Where the schema lives | Best for |
|---|---|---|
'form-linked' | In AmphiVox: a form you built or imported there, or one of your published forms. Your app mirrors its fields and pages. | You already have the form in AmphiVox (you built it, imported it, or have been collecting responses with it), and now want the same form, and the same voice agent, in your own app. |
'developer-defined' | In your code: you pass the whole schema (opens in a new tab) as a prop. Your code supplies the schema for each session. | Your app already has its own form, and you want to add the voice agent to it without building anything in AmphiVox. |
Each part of this guide that differs between the two carries a Form-linked / Developer-defined switch: pick yours on any of them and every such part follows.
Install
npm install @amphivox/react
Peer dependency: react >= 16.8 (tested on 18 and 19). Ships ESM, CJS, and TypeScript types.
Add your app's origin to the widget key's allowed websites in the Widgets area, or connections are rejected.
Serve your app over HTTPS. Microphone access requires a secure context. http://localhost is exempt, so local development works, but any other http:// origin will fail to get the mic.
Quick start
Render <VoiceWidget> (opens in a new tab) with the page on screen and its values. It renders nothing visible (the floating mic lives in the injected iframe) and reports the page to the agent whenever page changes.
Your form lives in AmphiVox; your page mirrors it.
import { useState } from 'react'
import { VoiceWidget, isSkip } from '@amphivox/react'
// Every field id, by page: copy it from the widget's page in the Widgets area.
const PAGE_FIELDS = [['full_name', 'email'], ['goal']]
function ApplicationForm() {
const [page, setPage] = useState(0) // the page number on screen: 0, 1, 2…
const [values, setValues] = useState<Record<string, unknown>>({})
const pageValues = Object.fromEntries(PAGE_FIELDS[page].map(id => [id, values[id] ?? null]))
return (
<form>
{/* …your inputs for this page, bound to `values`… */}
<VoiceWidget
mode="form-linked"
widgetKey="YOUR_WIDGET_KEY"
page={{ page, values: pageValues }}
onField={(fieldId, value) => {
if (isSkip(value)) return // a skipped optional field: not an answer
setValues(v => ({ ...v, [fieldId]: value }))
}}
onAdvancePage={() => { // your own Next
if (!pageIsValid(page)) return false // didn't move: the agent is told
setPage(p => p + 1)
}}
onSessionEnd={(ending) => {
if (ending.reason === 'complete') submitMyForm(values) // your own validation + submit
}}
/>
</form>
)
}
There is no schema (opens in a new tab) prop: the widget loads your form by key. The widget key and every field id are on the widget's page in the Widgets area, whose React snippet fills in PAGE_FIELDS for you.
You pass the schema in code; nothing is stored.
import { useState } from 'react'
import { VoiceWidget, isSkip, type WidgetSchema } from '@amphivox/react'
const schema: WidgetSchema = {
fields: [
{ id: 'full_name', label: 'Full name', type: 'str', required: true, page: 0 },
{ id: 'goal', label: 'What do you want to achieve?', type: 'long', page: 1 },
],
}
function ApplicationForm() {
const [page, setPage] = useState(0) // the page number on screen: 0, 1, 2…
const [values, setValues] = useState<Record<string, unknown>>({})
const pageValues = Object.fromEntries(
schema.fields.filter(f => f.page === page).map(f => [f.id, values[f.id] ?? null]),
)
return (
<form>
{/* …your inputs for this page, bound to `values`… */}
<VoiceWidget
mode="developer-defined"
widgetKey="YOUR_WIDGET_KEY"
schema={schema}
page={{ page, values: pageValues }}
onField={(fieldId, value) => {
if (isSkip(value)) return // a skipped optional field: not an answer
setValues(v => ({ ...v, [fieldId]: value }))
}}
onAdvancePage={() => { // your own Next
if (!pageIsValid(page)) return false // didn't move: the agent is told
setPage(p => p + 1)
}}
onSessionEnd={(ending) => {
if (ending.reason === 'complete') submitMyForm(values) // your own validation + submit
}}
/>
</form>
)
}
schema (opens in a new tab) lists the fields the agent asks for (its keys and limits are in the API reference (opens in a new tab)); a page is just the number its fields carry. The widget key is on the widget's page in the Widgets area; the field ids are your own.
page.values names exactly that page's field ids, null for no answer. The component compares page by content, so rebuilding the same object every render sends nothing. On a one-page form onAdvancePage (opens in a new tab) never fires; keep it anyway, so adding a page later doesn't strand your visitors on page 0.
When your Next refuses, return false from onAdvancePage. The page didn't change, so the component has nothing new to report; false is how the agent learns the move was blocked (opens in a new tab). Without it, the agent waits ~15 seconds before treating the move as blocked.
Component or hook?
The package has two entry points. <VoiceWidget> (opens in a new tab) is exactly useVoiceWidget() (opens in a new tab) plus one effect that reports its page prop.
<VoiceWidget> | useVoiceWidget() | |
|---|---|---|
| Reporting the page | Automatic, from the page prop, whenever its content changes | You call reportPage() (opens in a new tab): before open() (opens in a new tab), after every change, on every page change |
| Controls you get | None | open, close (opens in a new tab), mute (opens in a new tab), unmute (opens in a new tab), setLanguage (opens in a new tab), setVoiceGender (opens in a new tab), reportPage, reportWriteResult (opens in a new tab), reportAdvanceBlocked (opens in a new tab) |
| Refusing a write | Return a refusal from onField (opens in a new tab) | Return it, or call reportWriteResult later (after an async check) |
| A Next that didn't move | Return false from onAdvancePage (opens in a new tab) | Return false, or call reportAdvanceBlocked() |
| Choose it when | You use our dock | You build your own UI, or refuse a write after an async check |
From here on, the snippets spread widgetOptions: the mode and key (and schema) from your quick start, in one constant:
Your form lives in AmphiVox; your page mirrors it.
const widgetOptions = { mode: 'form-linked', widgetKey: 'YOUR_WIDGET_KEY' } as const
You pass the schema in code; nothing is stored.
const widgetOptions = { mode: 'developer-defined', widgetKey: 'YOUR_WIDGET_KEY', schema } as const
The same integration, both ways:
// The component: the page is a prop.
<VoiceWidget {...widgetOptions} page={{ page, values: pageValues }} onField={applyValue} />
// The hook: the page is your job.
const widget = useVoiceWidget({ ...widgetOptions, onField: applyValue })
const snapshot = JSON.stringify({ page, values: pageValues })
useEffect(() => { widget.reportPage(JSON.parse(snapshot)) }, [widget, snapshot])
What the hook makes you responsible for, and what happens if you skip it:
- Report before the first
open(). Without a report,open()fails withwidget_page_required. - Report after every change. A missing report leaves the agent working from stale values: it may ask again for an answer the visitor just typed.
- Report on every page change. The agent only asks about the page you last reported.
- Mount the hook once. Each mounted hook injects its own iframe and dock.
A session, step by step
One conversation on a two-page form, showing what your app does at each step:
1. Form renders page={{ page: 0, values }} (or reportPage)
2. Visitor taps mic onSessionStart()
3. Visitor answers onField('email', 'a@b.com', 'w_17')
→ set it in state; the new `page` reports it
4. Your rule says no onField returns { ok: false, cause: 'validation', reason }
→ the agent says why and asks again; after two
failed attempts it leaves the field to the visitor
5. Page 0 is done the agent asks: "Anything to change before I move on?"
the visitor agrees → onAdvancePage()
→ Next moved: setPage(1); the new page reports itself
→ Next failed: return false from onAdvancePage:
the agent goes back to an unanswered field,
or else ends the session (hands_off)
6. Visitor is silent the agent nudges, warns, then
onSessionEnd({ reason: 'timeout', detail: { timeout: 'idle' } })
7. Last page done the agent reads the answers back, then
onSessionEnd({ reason: 'complete' }) → validate and submit
Each step in the contract: Writes (opens in a new tab), Refusals (opens in a new tab), Page turns (opens in a new tab), Silence and time limits (opens in a new tab), Endings (opens in a new tab).
Build your own UI
Our dock is the floating mic button and its panel. To use your own components instead, call useVoiceWidget() (opens in a new tab) with hidden: true (opens in a new tab) and build each part from these:
| The dock's part | What you use |
|---|---|
| Start and end a session | open() / close() (opens in a new tab); onSessionStart (opens in a new tab) and onSessionEnd (opens in a new tab) tell you when it happens |
| Mute | mute() / unmute() (opens in a new tab); onMuteChange(muted) (opens in a new tab) |
| Language and voice | setLanguage(code) / setVoiceGender(gender) (opens in a new tab), for the next session. The codes, and a live JSON list of them, are in Languages and voices (opens in a new tab) |
| Live transcript | onTranscript(speaker, text, index) (opens in a new tab): lines grow as speech goes on, so replace the line at index, don't append |
| Speaking and thinking | onSpeaking(who, active) (opens in a new tab) and onThinking(active) (opens in a new tab): on or off, not an audio level |
| Problems | onError (opens in a new tab) |
function VoiceButton({ page, values }: { page: number; values: Record<string, unknown> }) {
const [live, setLive] = useState(false)
const [botTalking, setBotTalking] = useState(false)
const [muted, setMuted] = useState(false)
const [lines, setLines] = useState<{ speaker: string; text: string }[]>([])
const widget = useVoiceWidget({
...widgetOptions,
hidden: true,
onField: applyValue,
onSessionStart: () => { setLive(true); setLines([]) },
onSessionEnd: () => setLive(false),
onSpeaking: (who, active) => { if (who === 'bot') setBotTalking(active) },
onMuteChange: setMuted,
onTranscript: (speaker, text, index) =>
setLines(l => { const next = [...l]; next[index] = { speaker, text }; return next }),
})
const snapshot = JSON.stringify({ page, values })
useEffect(() => { widget.reportPage(JSON.parse(snapshot)) }, [widget, snapshot])
return (
<>
<button type="button" onClick={live ? widget.close : widget.open}>{live ? 'Stop' : 'Answer by voice'}</button>
<button type="button" onClick={muted ? widget.unmute : widget.mute} disabled={!live}>{muted ? 'Unmute' : 'Mute'}</button>
<select onChange={e => widget.setLanguage(e.target.value)} disabled={live}>
<option value="en">English</option>
<option value="hi">हिन्दी</option>
</select>
{botTalking && <span>Speaking…</span>}
<ul>{lines.map((line, i) => <li key={i}>{line.speaker}: {line.text}</li>)}</ul>
</>
)
}
Your form works exactly as before: the hook still needs reportPage() (opens in a new tab) (see Component or hook?), and answers still arrive in onField.
- The microphone prompt still comes from the AmphiVox origin.
hidden, not your own CSS. The iframe must stay rendered to keep the session alive: browsers slow down hidden cross-origin frames until they look disconnected.hiddenparks the frame as a 1×1 transparent element instead ofdisplay: none. Toggling it never restarts a live session.- AI disclosure is required. Visitors must be told they are talking to an AI before they start. Our dock shows that line; when you hide it, your own UI must show it, next to whatever starts the call: You're speaking with an AI assistant, not a person.
Recipes
Each recipe is one task, end to end. The options they don't show are the ones from your quick start, and widgetOptions is the constant from Component or hook?.
Handle a blocked Next
Your own Next decides whether the page moves. When it doesn't, return false from onAdvancePage (opens in a new tab) (or resolve to false, when your check is async):
<VoiceWidget
/* …the rest of your props… */
page={{ page, values: pageValues }}
onAdvancePage={async () => {
const errors = await validatePage(page)
if (errors.length > 0) return false // the unchanged page is already reported
setPage(p => p + 1) // the new page reports itself
}}
/>
A rejected promise counts as "didn't move" too. With the hook you can also call reportAdvanceBlocked() (opens in a new tab) yourself instead; use one or the other, not both.
A blocked Next with an unanswered field on the page sends the agent back to that field. Otherwise it ends the session (opens in a new tab) as hands_off: a block the agent can't see the reason for can't be fixed by voice. Most blocked Nexts come from a rule the agent didn't know about. Put it in the schema (required, min, maxLength…) so the agent enforces it while asking, or refuse the value as it arrives (next recipe), so the page is valid by the time the agent asks to move on.
Refuse an answer
A refusal is one thing: you tell the agent an answer didn't land. It carries a cause, which decides what the agent does next:
cause | Means | The agent |
|---|---|---|
'validation' | The answer is well-formed, but your own rule rejects it: an email that's already registered, an expired coupon | Tells the visitor your reason and asks for a different answer |
'write' (the default) | Your control couldn't take the value | Resends the same value once, without telling the visitor |
Either way, after two failed attempts the agent leaves the field for the visitor to fill. In a React app the value lands in your state, so in practice you only send validation.
You can refuse straight away, or after an async check, whichever your rule needs:
| Your check | Refuse by | Works with |
|---|---|---|
| Synchronous | Returning { ok: false, cause, reason } from onField (opens in a new tab) (false alone is short for { ok: false, cause: 'write' }) | The component and the hook |
| Async, such as a server round trip | Returning nothing from onField, then calling reportWriteResult({ fieldId, writeId, ok: false, cause, reason }) with the same writeId | The hook only: the component returns no controls |
Straight away:
onField={(fieldId, value) => {
if (isSkip(value)) return
if (fieldId === 'email' && registeredEmails.has(value as string)) {
return { ok: false, cause: 'validation', reason: 'That email already has an account.' }
}
setValues(v => ({ ...v, [fieldId]: value }))
}}
After an async check:
const widget = useVoiceWidget({
...widgetOptions,
onField: (fieldId, value, writeId) => {
if (isSkip(value)) return
if (fieldId !== 'coupon') { setValues(v => ({ ...v, [fieldId]: value })); return }
checkCoupon(value as string).then((valid) => {
if (valid) setValues(v => ({ ...v, coupon: value }))
else widget.reportWriteResult({
fieldId, writeId: writeId!, ok: false, cause: 'validation', reason: 'That coupon has expired.',
})
})
},
})
Keep reporting what your form really holds afterwards, the rejected value included if you kept it. The agent never picks a refused value up from a page report. The reason is spoken aloud and kept in AmphiVox's session logs: keep it short and never put the value in it.
Remember a skip across sessions
A skip arrives in onField (opens in a new tab) as SKIP (opens in a new tab) (test it with isSkip (opens in a new tab)) with a null write id. Keep the field id, and pass the marker back in initialValues (opens in a new tab) so the next session doesn't ask again:
import { SKIP, isSkip } from '@amphivox/react'
const [skipped, setSkipped] = useState<string[]>([])
const initialValues = Object.fromEntries(skipped.map(id => [id, SKIP]))
<VoiceWidget
/* …the rest of your props… */
initialValues={initialValues}
onField={(fieldId, value) => {
if (isSkip(value)) { // not an answer: leave the field as it is
setSkipped(s => (s.includes(fieldId) ? s : [...s, fieldId]))
return
}
setSkipped(s => s.filter(id => id !== fieldId))
setValues(v => ({ ...v, [fieldId]: value }))
}}
/>
The widget keeps the marker while later page reports still show the field empty (null, blank text, or an empty array). A real answer the visitor types afterwards replaces it. To forget a skip, drop its key from initialValues. Sending a skip for a required field changes nothing.
Accept a file upload
The agent never asks for a file by voice; it tells the visitor to attach it in your form. Check the file against the field's maxSizeMb and extensions, store it yourself, then report the field as uploaded. Never report the File or its name:
<input
type="file"
onChange={async (e) => {
const file = e.target.files?.[0]
if (!file || !acceptsFile(file)) return // your own size and type check
await uploadToYourServer(file)
setValues(v => ({ ...v, cv: { uploaded: true } })) // `null` until then
}}
/>
Start with answers you already know
For a returning visitor, pass the answers you already hold, from any page, so the agent doesn't ask for them again:
<VoiceWidget
/* …the rest of your props… */
initialValues={{ full_name: user.name, email: user.email }}
/>
These are known answers (opens in a new tab): the same shapes as page reports, checked the same way, so an invalid one is dropped and the agent asks for that field as usual. A key that is not one of your field ids rejects the whole set (widget_page_invalid). The prop is compared by content and applies to the next session; null clears it.
Handle every ending
onSessionEnd={(ending) => {
switch (ending.reason) {
case 'complete': // every answer is in: validate and submit
submitMyForm(values)
break
case 'hands_off': // the visitor must finish some fields
highlightFields(ending.detail?.outstanding ?? []) // their labels
break
case 'timeout': // 'idle' or 'max_duration'
case 'manual': // the visitor stopped, or you called close()
break // nothing to do: their answers are in your state
case 'error': // a live session failed; answers already written stay valid
setNotice('Voice stopped working. You can finish the form by hand.')
break
}
}}
Never submit on anything but complete. The dock itself shows the visitor a short notice for hands_off and timeout, then asks them to rate the voice session.
Handle errors
onError (opens in a new tab) reports anything that stopped a session before it started, and messages from your app that the widget rejected. Switch on the code; show the message, which is safe to display:
import type { WidgetError } from '@amphivox/react'
function handleVoiceError({ code, message }: WidgetError) {
switch (code) {
case 'origin_not_allowed': // add this site to the key's allowed websites
case 'invalid_widget_key':
case 'widget_mode_mismatch':
console.error('[voice setup]', code, message)
break
case 'widget_page_invalid': // your page values and the schema disagree
console.error('[voice page]', message)
break
case 'session_limit_exceeded':
case 'session_busy':
setNotice('Voice is busy right now. Try again in a minute.')
break
default: // the list can grow: always keep a default
setNotice(message)
}
}
Every code and what it means: Errors (opens in a new tab).
Use react-hook-form
When a form library owns your values and validation, connect each part of it to the widget:
import { useEffect, useState } from 'react'
import { useForm } from 'react-hook-form'
import { useVoiceWidget, isSkip } from '@amphivox/react'
function ApplicationForm() {
const [page, setPage] = useState(0)
const { register, setValue, watch, trigger, getFieldState, handleSubmit } = useForm()
const pageIds = PAGE_FIELDS[page] // this page's field ids
const widget = useVoiceWidget({
...widgetOptions,
onField: (fieldId, value, writeId) => {
if (isSkip(value)) return
setValue(fieldId, value, { shouldDirty: true })
// Your library's rules decide; a failed rule is a validation refusal.
trigger(fieldId).then((ok) => {
if (!ok && writeId) widget.reportWriteResult({
fieldId, writeId, ok: false, cause: 'validation',
reason: getFieldState(fieldId).error?.message ?? 'That answer is not accepted.',
})
})
},
onAdvancePage: async () => {
if (!(await trigger(pageIds))) return false // blocked: the agent is told
setPage(p => p + 1)
},
onSessionEnd: (ending) => {
if (ending.reason === 'complete') handleSubmit(submitMyForm)()
},
})
// The page report comes from the library's own values.
const all = watch()
const snapshot = JSON.stringify({ page, values: Object.fromEntries(pageIds.map(id => [id, all[id] ?? null])) })
useEffect(() => { widget.reportPage(JSON.parse(snapshot)) }, [widget, snapshot])
return <form>{/* …inputs registered with `register`… */}</form>
}
PAGE_FIELDS lists your field ids page by page, as in the form-linked quick start; with a developer-defined schema, build it from schema.fields. The same pattern fits other form libraries: the write goes through the library's setter, the page report is built from its values, a failed rule becomes a validation refusal, its page check decides the Next, and its submit runs only on complete.
Use it in Next.js
All DOM work happens inside effects, so the package never runs on the server. In the App Router, add the client directive to the file that imports it:
'use client'
import { VoiceWidget } from '@amphivox/react'
The hook supports StrictMode's development-only effect replay on first mount, and ordinary prop changes keep an active call running. A real unmount and remount does not keep the call. Changing widgetKey (opens in a new tab) or mode replaces the iframe, and unmounting removes it and every listener; either one ends the call.
Run two widgets on one page
Each hook ignores any message that didn't come from its own iframe, so several widgets on one page each receive only their own events. Each injects its own dock, so separate them:
// Each key's own mode and key (and schema), like `widgetOptions`:
<VoiceWidget {...salesOptions} page={salesPage} position="bottom-right" />
<VoiceWidget {...supportOptions} page={supportPage} position="bottom-left" />
Content Security Policy
If your app sends a Content-Security-Policy header, the widget needs one directive:
Content-Security-Policy: frame-src https://amphivox.com;
You don't need script-src for AmphiVox (the package is bundled into your JavaScript), nor connect-src or media-src: the package opens no connections of its own. The voice session runs inside the iframe under AmphiVox's CSP, and its audio over WebRTC.
Troubleshooting
widget_page_required. Pass page (or call reportPage (opens in a new tab)) before the session opens.
widget_page_invalid. page.values named a field the page doesn't have, or missed one it does.
Every multi-page session ends on page 0. You aren't handling onAdvancePage (opens in a new tab), so each page turn times out as blocked. See Handle a blocked Next.
origin_not_allowed even though your domain is on the list. Check whether the page sends Referrer-Policy: no-referrer or same-origin. Both hide your page's origin from the documents it frames. Use the default strict-origin-when-cross-origin (or origin, strict-origin, no-referrer-when-downgrade). (Why.)
Mic blocked. Check HTTPS (or localhost), and that your server doesn't send Permissions-Policy: microphone=().
Screen sleeps during a session. Only a Permissions-Policy: screen-wake-lock=() header on your page overrides the iframe's own request: drop it, or include the widget's origin.
setLanguage (opens in a new tab) / setVoiceGender (opens in a new tab) did nothing mid-call. By design: both apply to the next session.
The agent asks for a field my form doesn't have. The form was edited in AmphiVox and your app hasn't caught up. Update your form's fields, ids and pages to match.
A new schema (opens in a new tab) did nothing mid-call. By design: the schema is fixed once a call connects, and a change during the call is refused (widget_schema_locked) and discarded, not kept for later. Change the prop only while no session is running.
The session drops when I hide the widget myself. Hiding the iframe with display: none makes the browser slow it down until it looks disconnected. Use the hidden (opens in a new tab) prop.