Voice Asset Lifecycle

Status: Running today. Voice creation (clone + design) and the VoiceAsset runtime-managed object are shipped under the K-VOICE-* authority surface.

Voice in Nimi is a runtime first-class capability. Voice creation (clone + design) and voice asset lifecycle are owned by Runtime under K-VOICE-001..K-VOICE-018. This page covers the asset side — what VoiceAsset is, how it is created, how it is referenced, and how it differs from one-off speech synthesis.

Voice Creation Scope

Voice creation covers two scenarios:

Scenario	What it does
voice_clone	voice / audio → voice (sample-based)
voice_design	text → voice (description-based)

Both are submitted through a unified Scenario abstraction:

• SubmitScenarioJob with scenario_type=VOICE_CLONE
• SubmitScenarioJob with scenario_type=VOICE_DESIGN

Provider-private parameters do not pass through freely; they go through namespaced ScenarioExtension and are bound by extension registry rules.

VoiceAsset: Runtime-Owned Object Truth

VoiceAsset is the runtime-managed voice resource. Minimum required fields:

Field	Meaning
voice_asset_id	Runtime-owned asset id
app_id	Owning app context
subject_user_id	Owning user (tenant scope)
workflow_type	voice_clone / voice_design
persistence	Logical lifecycle (persistence_types)
status	Asset status (asset_statuses)

Both persistence and status enums live in tables/voice-enums.yaml.

persistence describes logical lifecycle and handle policy. It does not automatically promise a durable local substrate; until durable local substrate is admitted separately, locally-generated VoiceAsset may remain a session-local orchestration object.

Asset vs Reference vs Synthesis

This is the most important distinction on this page:

Concept	What it is	Owner
VoiceAsset	Durable runtime-owned voice resource	Runtime
VoiceReference	The handle a synthesis call uses to identify which voice to speak	Runtime
Provider-native handle	Internal provider identity that never enters ordinary App input	Provider / Runtime Driver
Voice synthesis	One-off tts_synthesize invocation that uses a VoiceReference to produce audio	Runtime (RPC)

Runtime must not promote a provider-native voice handle to a public primary key. When a provider returns one, Runtime preserves it internally and exposes only the Runtime-owned VoiceAsset and admitted VoiceReference.

VoiceReference Boundary

The ordinary synthesis entry point goes through two public reference kinds:

Kind	Use
preset_voice_id	System-preset voice
voice_asset_id	User-created VoiceAsset

The reference kinds enum lives in tables/voice-enums.yaml (reference_kinds).

VoiceReference may be embedded inside the runtime-owned AgentPresentationProfile as the agent's default voice binding. That embedding does not transfer voice workflow / discovery / asset ownership outside K-VOICE-*.

Discovery: Two Separate Channels

Voice discovery splits cleanly into two channels:

Discovery API	What it returns
ListPresetVoices	System-preset voice catalog
ListVoiceAssets	User's VoiceAsset records

Callers must not depend on a single mixed API. When a provider supports both global presets and user assets, both channels stay available — but the runtime never returns mixed-stream results.

The voice.discovery_mode catalog setting binds discovery responsibility:

• static_catalog → preset discovery only via ListPresetVoices
• dynamic_user_scoped → user asset discovery via ListVoiceAssets
• mixed → both channels active, callers invoke separately

ScenarioJob Lifecycle

Voice creation is asynchronous. ScenarioJob lifecycle (state machine + event stream) aligns with K-JOB-002. Voice does not duplicate a parallel job state table.

The four lifecycle RPCs:

• SubmitScenarioJob — start a clone or design job
• GetScenarioJob — poll status
• CancelScenarioJob — cancel in flight
• SubscribeScenarioJobEvents — stream events

Provider-native multi-step workflows (e.g., preview → create) must be encapsulated inside one ScenarioJob lifecycle. Provider internal steps do not become extra public RPCs.

Tenant Isolation

VoiceAsset is user-scoped by default. Cross-app_id or cross-subject_user_id access fails closed. There is no implicit cross-tenant voice asset reuse.

Voice Handle Policy

Workflow-capable voice families must declare voice_handle_policy once admitted. Minimum fields:

Field	Source
persistence	tables/voice-enums.yaml persistence_types
scope	tables/voice-enums.yaml handle_scopes
default_ttl	per-family
delete_semantics	tables/voice-enums.yaml delete_semantics
runtime_reconciliation_required	per-family

Workflow-capable families without an admitted voice_handle_policy may not be admitted.

Reader Scenario: User Clones Their Voice

A user wants to clone their own voice for an agent.

1. Submit clone job. App calls SubmitScenarioJob({ scenario_type: VOICE_CLONE, ... }) with the audio sample inputs.
2. Job lifecycle. Runtime executes through admitted ScenarioJob state machine; events stream via SubscribeScenarioJobEvents.
3. Provider native preview / create wraps. The provider's internal preview-then-create steps are encapsulated inside the one ScenarioJob — the app sees one lifecycle.
4. Result. Runtime emits a VoiceAsset with admitted fields and a VoiceReference{ kind: voice_asset_id }.
5. Bind to agent. App embeds the VoiceReference in AgentPresentationProfile. The agent now speaks with the cloned voice by default.

Reader Scenario: Synthesis With A Preset Voice

App wants the agent to speak a one-off line using a preset voice.

1. Discover. ListPresetVoices returns the catalog with preset_voice_ids.
2. Build VoiceReference. { kind: preset_voice_id, value: ... }.
3. Synthesize. Runtime tts_synthesize call uses the reference; no VoiceAsset is involved (no user asset is created).
4. Audio bytes return. Runtime owns the artifact bytes; Avatar's audio pipeline consumes via runtime.artifacts.readBytes.

Synthesis is transient. The VoiceAsset lifecycle is for durable voice resources — a user's cloned voice, a designed voice, etc.

Reader Scenario: Discovery Honors The Channel Split

App wants to surface the user's available voices in a settings UI.

1. Two calls. App calls ListPresetVoices for system voices and ListVoiceAssets for user-created voices.
2. Render in two sections. UI shows them as separate sections per the discovery_mode boundary.
3. No mixed API. App does not look for a single "list everything" call — there isn't one, and the channel split is contractual.

Lifecycle Boundaries

• It is not a one-off synthesis call. tts_synthesize is transient; VoiceAsset is durable resource truth.
• It does not mix preset and user-asset discovery into one stream.
• It does not let providers escape into the public asset surface; provider-native handles stay inside Runtime and its Driver.
• It does not silently cross tenants. User scope is fail-closed.
• It does not let workflow-capable TTS families substitute for STT; audio.transcribe admits independently per K-VOICE-016 family-level boundary.

Boundary Summary

Concern	Owner
Voice creation workflows (clone + design)	Runtime (K-VOICE-001..002)
VoiceAsset object truth	Runtime (K-VOICE-004)
VoiceReference synthesis entry	Runtime (K-VOICE-003)
Provider-native voice handle	Provider (provider_voice_ref)
Tenant isolation	Runtime (K-VOICE-006)
Discovery channel split	Runtime (K-VOICE-009, K-VOICE-013)
Voice handle policy	Runtime (K-VOICE-015)
Family-level workflow validation boundary	Runtime (K-VOICE-016)

Source Basis

• .nimi/spec/runtime/model-catalog.authority.yaml
• config/runtime-voice-enums.yaml
• .nimi/spec/runtime/agent-participation.authority.yaml