Aether/docs/api/format-conversion-audit.md

Format Conversion Audit

Last audited: 2026-06-03

This audit tracks source format -> Canonical -> target format behavior. It is intentionally stricter than historical best-effort conversion.

Statuses:

• native: emitted as a target-native field without semantic change.
• mapped: converted through canonical/provider-specific mapping.
• extension-preserved: preserved in same-format canonical roundtrip or target-approved extension namespace.
• transport-only: audited client transport metadata intentionally omitted when the target has no compatible transport channel.
• unaudited: rejected because the source field is not in the audited provider schema inventory for cross-format conversion.
• unsupported: rejected because the request/field shape is outside the supported conversion surface, independent of schema drift.
• lossy-blocked: conversion fails closed.
• invalid-enum: conversion fails closed because a provider enum value is not valid for the target mapping.

Full schema field coverage is tracked in docs/api/format-field-coverage-matrix.md. That matrix is generated from the schema inventory in docs/api/provider-interface-definitions.md by python3 docs/api/generate_format_field_coverage.py and gives every documented OpenAI, Claude, and Gemini schema field a handling status. “Handled” means mapped, same-format/native preserved, extension-preserved, blocked with a structured error, or explicitly marked outside the canonical conversion surface.

Provider schema refresh is not a runtime dependency. Same-format runtime paths do not use this matrix; they bypass canonical conversion. Same-format canonical roundtrip must preserve unrecognized provider fields through provider extension namespaces. Cross-format conversion is capability-based: only explicitly mapped fields are emitted, and newly discovered or unknown provider fields fail closed with UnauditedField until a lossless mapping is audited.

Implemented Boundary Changes

Area	Current behavior
Pure conversion API	convert_request_pure and response equivalents do not apply model override or stream policy.
Legacy conversion API	convert_request / convert_response are retained for migration and may still use legacy context behavior.
Same-format provider path	Bypasses canonical conversion and copies the parsed JSON object before transport edits.
Cross-format same-format-provider path	Uses convert_request_pure, then applies model/body/stream edits in transport.
Conversion errors	Added UnauditedField, UnsupportedField, InvalidEnumValue, LossyConversionBlocked, and InvalidTargetField.
Reporting	Added ConversionReport with field statuses. Runtime reports remain conversion-operation oriented; exhaustive nested schema coverage is enforced by format-field-coverage-matrix.md.
Source schema coverage	Cross-format request conversion rejects unknown source root fields before emit. Every documented schema field is covered by the field coverage matrix.
Schema drift handling	Official schema changes are detected by regenerating the inventory/matrix. Runtime same-format remains passthrough; cross-format unknowns return UnauditedField until deliberately mapped.
Tool schema roundtrip	Claude input_schema and Gemini functionDeclarations.parameters preserve raw same-format schema through provider-specific extensions.
Tool result ids	Chat tool_call_id, Responses call_id, Claude tool_use_id, and Gemini functionResponse.id are mapped through canonical tool IDs.

OpenAI Chat -> OpenAI Responses

Chat field	Canonical handling	Responses output	Status
model	request identity	model	native
messages	canonical messages/instructions	input, instructions	mapped
max_tokens	generation max tokens	max_output_tokens	mapped
max_completion_tokens	generation max tokens	max_output_tokens	mapped
temperature	generation	temperature	native
top_p	generation	top_p	native
top_logprobs	generation	top_logprobs	native
n	generation but no Responses equivalent	none	lossy-blocked
stop	generation but no Responses equivalent	none	lossy-blocked
presence_penalty	generation but no Responses equivalent	none	lossy-blocked
frequency_penalty	generation but no Responses equivalent	none	lossy-blocked
seed	generation but no Responses equivalent	none	lossy-blocked
logprobs	generation but no Responses equivalent	none	lossy-blocked
stream	OpenAI extension	stream	mapped if explicit
stream_options	Chat-specific extension	none	lossy-blocked
tools[].function.name	canonical tool	tools[].name	mapped
tools[].function.description	canonical tool	tools[].description	mapped
tools[].function.parameters	canonical tool	tools[].parameters	mapped
tools[].function.strict	canonical tool strict	tools[].strict	mapped, implemented
assistant tool_calls[].id	canonical tool use id	function_call.call_id	mapped, implemented
tool message tool_call_id	canonical tool result id	function_call_output.call_id	mapped, implemented
tool_choice	canonical tool choice	tool_choice	mapped
parallel_tool_calls	canonical bool	parallel_tool_calls	native
metadata	canonical metadata	metadata	native
response_format	canonical response format	text.format	mapped
reasoning_effort	OpenAI enum	reasoning.effort	mapped; invalid enum blocked
verbosity	OpenAI extension	text.verbosity	mapped
store	OpenAI extension	store	extension-preserved
service_tier	OpenAI extension	service_tier	extension-preserved
safety_identifier	OpenAI extension	safety_identifier	extension-preserved
prompt_cache_key	OpenAI extension	prompt_cache_key	extension-preserved
user	legacy Chat user field	none	lossy-blocked
unknown top-level fields	source schema guard	none	unaudited

OpenAI Responses -> OpenAI Chat

Responses field	Canonical handling	Chat output	Status
model	request identity	model	native
input	canonical messages/content/tool I/O	messages	mapped
instructions	canonical instruction/system	messages system/developer	mapped
max_output_tokens	generation max tokens	max_completion_tokens	mapped
temperature	generation	temperature	native
top_p	generation	top_p	native
top_logprobs	generation	top_logprobs	native
metadata	canonical metadata	metadata	native
client_metadata	Responses client transport metadata	none	transport-only; omitted
parallel_tool_calls	canonical bool	parallel_tool_calls	native
text.format	canonical response format	response_format	mapped
text.verbosity	Responses extension	verbosity	mapped
tools[].type=function	canonical tool	tools[].type=function	mapped
tools[].name	canonical tool	tools[].function.name	mapped
tools[].parameters	canonical tool	tools[].function.parameters	mapped
tools[].strict	canonical tool strict	tools[].function.strict	mapped, implemented
function_call.call_id	canonical tool use id	tool_calls[].id	mapped, implemented
function_call_output.call_id	canonical tool result id	tool message tool_call_id	mapped, implemented
tools[].type=custom	raw Responses tool	none	lossy-blocked to Chat
tools[].type=web_search*	raw Responses tool	none	lossy-blocked to Chat
tool_choice	canonical tool choice	tool_choice	mapped
reasoning.effort	OpenAI enum	reasoning_effort	mapped; invalid enum blocked
reasoning.summary	Responses-only	none	lossy-blocked
reasoning.budget_tokens	Responses-only	none	lossy-blocked
stream	Responses request transport policy	none	lossy-blocked; target stream policy is transport-owned
include	Responses-only	none	lossy-blocked; legacy emitter no longer leaks
previous_response_id	Responses-only	none	lossy-blocked; legacy emitter no longer leaks
truncation	Responses-only	none	lossy-blocked
prompt	Responses-only	none	lossy-blocked
conversation	Responses-only	none	lossy-blocked
background	Responses-only	none	lossy-blocked
max_tool_calls	Responses-only	none	lossy-blocked
unknown top-level fields	source schema guard	none	unaudited

Claude Messages <-> OpenAI Chat / Responses

Claude to OpenAI Chat, Claude to OpenAI Responses, and the reverse directions are included in the field coverage matrix. Runtime strict guards cover request root fields, provider extension namespaces, thinking/cache/tool-result hazards, and target generation-field gaps. Fields without a lossless target equivalent fail closed instead of being dropped.

High-risk fields:

Claude field	OpenAI target risk	Required status
system with cache blocks	Chat/Responses system instructions	same-format preserved; cross-format cache_control loss is blocked
thinking	OpenAI reasoning	Claude request-level thinking config maps to OpenAI reasoning; message-level thinking blocks are blocked for Responses
cache_control	OpenAI content/tool extensions	same-format preserved; cross-format blocked when no target equivalent exists
tools[].input_schema	OpenAI tool parameters	mapped; raw same-format schema preservation implemented
tool_choice.disable_parallel_tool_use	OpenAI parallel_tool_calls	mapped, implemented
tool_result multi-block content	OpenAI tool output/content	same-format preserved; cross-format to Chat/Responses is lossy-blocked
metadata	OpenAI metadata	mapped when the target has metadata
container, inference_geo, service_tier	OpenAI target has no audited equivalent	lossy-blocked unless a target-approved mapping is added

Gemini GenerateContent <-> OpenAI Chat / Responses / Claude

Gemini to OpenAI Chat, Gemini to OpenAI Responses, Gemini to Claude, and reverse generation paths are included in the field coverage matrix. Gemini-only request fields are preserved same-format and blocked cross-format unless the target mapping is explicitly audited.

High-risk fields:

Gemini field	Target risk	Required status
contents[].parts[].thoughtSignature	OpenAI/Claude thinking	Chat/Claude preserve; Responses cross-format is lossy-blocked
tools[].functionDeclarations	OpenAI/Claude tool schema	mapped; raw same-format parameters preservation implemented
toolConfig.functionCallingConfig.allowedFunctionNames	OpenAI/Claude tool choice	single-name mapping implemented; multi-name input is lossy-blocked
toolConfig.functionCallingConfig.mode	OpenAI/Claude tool choice enum	valid enum required; invalid values fail with InvalidEnumValue
generationConfig.thinkingConfig.thinkingLevel	OpenAI/Claude reasoning effort	low/medium/high mapping implemented; invalid values fail closed
safetySettings	OpenAI/Claude no direct equivalent	lossy-blocked
cachedContent	OpenAI/Claude no direct equivalent	lossy-blocked
codeExecution	OpenAI/Claude tool/builtin mismatch	lossy-blocked
generationConfig.responseModalities	OpenAI/Claude modality mismatch	lossy-blocked
functionResponse.id	tool result id	conversion preserves id; Gemini upstream cleanup is transport-layer edit only

Embedding And Rerank

Embedding and rerank request parse/emit capability and strict target guards are implemented. Provider schema fields outside these canonical conversion surfaces are marked not-in-conversion-surface in the field coverage matrix instead of being left implicit.

Embedding source capability:

Source format	Parsed request shape	Canonical fields	Status
OpenAI Embedding	model, input, encoding_format, dimensions, user, parameters, task	OpenAI-like embedding	mapped
Jina Embedding	OpenAI-like plus provider extension namespace	OpenAI-like embedding	mapped
Doubao Embedding	OpenAI-like model + text input	OpenAI-like embedding	mapped
Gemini Embedding	single content.parts[].text or batch requests[]	text input, dimensions, task	mapped
Aliyun Multimodal Embedding	input.contents[], parameters.dimension	text/multimodal input, dimensions, parameters	mapped

Embedding target guards:

Target format	Accepted canonical fields	Blocked fields/cases	Status
OpenAI Embedding	text or token input, encoding_format, dimensions, user	multimodal input, task, generic parameters	lossy-blocked
Jina Embedding	text input, dimensions, task, parameters	token/multimodal input, encoding_format, user	lossy-blocked
Gemini Embedding	text input, dimensions, valid taskType	token/multimodal input, encoding_format, user, generic parameters, invalid taskType	lossy-blocked / invalid-enum
Doubao Embedding	text input, dimensions	token/multimodal input, encoding_format, user, task, generic parameters	lossy-blocked
Aliyun Multimodal Embedding	text or multimodal input, dimensions, parameters	token input, encoding_format, user, task	lossy-blocked

Cross-format embedding invariants:

• Embedding formats can only convert to embedding formats.
• Unknown provider-specific embedding extension namespaces are blocked cross-format unless the namespace matches the target.
• Aliyun parameters.dimension maps to canonical dimensions and is not treated as generic parameters.
• Gemini batch embedding parse requires every batch item to share the same model, dimensions, and task.

Rerank first pass:

Area	Current behavior	Status
Source formats	OpenAI Rerank and Jina Rerank parse OpenAI-like model, query, documents, top_n, return_documents	mapped
Target formats	OpenAI Rerank and Jina Rerank emit OpenAI-like rerank bodies	mapped
Boundary	Rerank formats can only convert to rerank formats	lossy-blocked
Validation	Empty query/documents and top_n=0 fail closed	invalid-target-field
Extensions	Unknown provider-specific rerank extension namespaces are blocked cross-format	unsupported

Sync Response Conversion

Cross-format sync response conversion now validates source stop/finish/status enums before emitting a target body. Same-format runtime response passthrough is still outside canonical conversion.

Source field	Target risk	Current behavior	Status
Same-format response raw stop/status fields	canonical emitters would otherwise normalize unknown enum/status to default target stop values	raw OpenAI Chat finish_reason, OpenAI Responses status, Claude stop_reason/stop_sequence, and Gemini finishReason are preserved through provider extension metadata	extension-preserved
OpenAI Chat choices[].finish_reason	unknown value would otherwise emit as target normal stop	valid Chat enum required; unknown values fail with InvalidEnumValue	invalid-enum
OpenAI Responses status	queued, in_progress, and cancelled have no sync target equivalent	non-terminal valid states fail with LossyConversionBlocked; invalid states fail with InvalidEnumValue	lossy-blocked / invalid-enum
OpenAI Responses incomplete_details.reason=content_filter	previously mapped to max tokens/length	maps to canonical content filter and emits Chat content_filter / Claude content_filtered / Gemini SAFETY	mapped
Claude stop_reason	unknown value would otherwise emit as target normal stop	valid known stop enum required for cross-format conversion	invalid-enum
Gemini candidates[].finishReason	known-but-unmappable reasons would otherwise emit as target normal stop	mappable safety/max/stop reasons convert; known unmappable values such as OTHER, MALFORMED_FUNCTION_CALL, UNEXPECTED_TOOL_CALL, MISSING_THOUGHT_SIGNATURE, and MALFORMED_RESPONSE fail with LossyConversionBlocked; future unknown values fail with InvalidEnumValue	lossy-blocked / invalid-enum
Canonical Unknown stop reason	target emitters default to normal stop values	cross-format response conversion blocks canonical unknown stop reasons	lossy-blocked

Stream Conversion

Sixth batch first pass is implemented for unknown event handling and runtime same-format boundaries. Sync response finish/status parity has a first strict pass; stream finish-reason guardrails are implemented for unknown/unmappable terminal reasons. Stream event schema fields are covered in the field coverage matrix; provider-by-provider fixtures cover the runtime event behavior.

Current stream behavior:

Area	Current behavior	Status
Provider parsers	OpenAI Chat, OpenAI Responses, Claude, and Gemini unknown stream payloads become CanonicalStreamEvent::UnknownEvent	mapped
Cross-format stream matrix	Unknown canonical stream events emit a target-format error SSE with unsupported_stream_event and terminate conversion	lossy-blocked
Stream finish reason guard	Unknown OpenAI finish reasons, unknown Claude stop_reason, and Gemini known-but-unmappable finishReason values such as OTHER are preserved as raw canonical finish strings, then blocked by the matrix with unsupported_finish_reason	lossy-blocked
OpenAI Responses stream target	Canonical length and content_filter terminal reasons emit response.incomplete with incomplete_details.reason=max_output_tokens or content_filter instead of response.completed	mapped
Terminal observer	Unknown provider stream events increment unknown_event_count; OpenAI Responses failed events mark terminal error state	mapped
Stream -> sync aggregate	Unknown OpenAI Chat, OpenAI Responses, Claude, and Gemini stream events make the runtime finalize checked path return an error and block body_json fallback; legacy public aggregate helpers keep Option compatibility	lossy-blocked
Runtime strict fallback guard	UnauditedField, InvalidEnumValue, UnsupportedField, LossyConversionBlocked, and InvalidTargetField from registry response conversion are not allowed to fall through legacy conversion helpers	lossy-blocked
Runtime same-format stream	Same-format stream passthrough remains outside canonical conversion; stream policy edits are transport-layer only	native

Stream fixture coverage:

Provider stream	Covered fixture areas
OpenAI Chat	sync aggregation for text, tool call IDs/names/argument deltas, finish reason, and usage; cross-format unknown finish/event blocking
OpenAI Responses	text snapshot de-duplication, multi-part messages, reasoning/items, function calls, image generation calls, same-family stream sync, unknown event blocking
Claude Messages	thinking signatures, tool input deltas, cache/usage aggregation, media emission, unknown stop/event blocking
Gemini GenerateContent	text/media/signature aggregation, function calls/results, safety finish mapping, unknown parts/events, and unmappable finish reason blocking

Matrix-level interception remains the authoritative runtime path for cross-format unknown events; direct client emitters are covered only as provider/client building blocks.