Skip to content

Main Navigation Guide Packages

Appearance

Sidebar Navigation

User Guide

Introduction

Getting Started

The Intent Panel

Configuration

⚠️ Read Before Running

Concepts & Design

Motivation & Workflow

Prompt Lowering

Frontend for Agents

Developers

Developing pdum_aiui

Packages

aiui

Overview

The Agent's Browser

Remote Development

API Reference

Variables

name

aiui-build-config

Overview

API Reference

Interfaces

PackageDeps

Variables

SOLID_TEST_CONDITIONS

solidTestDeps

Functions

externalizeDeps

aiui-cf-creds

Overview

Getting Started with @habemus-papadum/aiui-cf-creds

API Reference

index

Interfaces

BrokerAwsManagerOptions

BrokerOptions

Functions

brokerAwsManager

brokerRoute

mosaic

Interfaces

BrokerConnectorOptions

Type Aliases

InstallMode

Functions

brokerConnector

oracle

Interfaces

FederatedMintOptions

MintClient

Functions

federatedKeySource

stt

Interfaces

ScribeConnectOptions

TranscriptionKeySourceOptions

Variables

DEFAULT_TRANSCRIPTION_MODEL

Functions

scribeConnectUrl

transcriptionKeySource

aiui-claude-channel

Overview

The aiui Claude channel — architecture

Prompt rendering reference

WebSocket Protocol

API Reference

index

Classes

PageToolDirectory

SessionHub

Interfaces

ChannelClient

ChannelClientOptions

ChannelConnection

ChannelConnectionOptions

ChannelErrorMessage

ChannelFormat

ChannelResponse

ChannelServerOptions

ChannelThread

ChromeDevtoolsInfo

DecodedFrame

Envelope

FrameLogEntry

HelloMeta

LaunchInfo

MessageMeta

MountedSidecar

OpenAiSpeakerOptions

PageToolCall

PageToolDescriptor

PageToolDirectoryOptions

PageToolRegistration

PageToolSummary

PayloadCodec

PeersResponse

PublishResult

RealtimeCallbacks

RealtimeResult

RealtimeSession

RealtimeSessionOptions

RealtimeSocket

RealtimeSocketHandlers

ReloadSummary

SendPromptWsOptions

SendResult

SessionHubOptions

SessionPeerInfo

SessionSummary

Sidecar

SidecarContext

SourceInfo

Speaker

SpeakInput

SpeechResult

StreamProcessor

ThreadContext

TranscriptWord

WebServer

WebServerOptions

WsSendResult

Type Aliases

ChannelReload

ChunkDescriptor

EnvelopeKind

FetchLike

FormatLoader

FormatRegistry

FrameLogSink

PageToolSend

PromptHandler

PushMessage

RealtimeSocketFactory

SendPrompt

ServerToClientMessage

SessionClientMessage

SessionServerMessage

StreamProcessorFactory

TabInfo

VendorKeyStatus

Variables

jsonCodec

name

PROTOCOL_VERSION

rawCodec

textConcatFormat

Functions

augmentTextPrompt

channelCliPath

connectChannelClient

createChannelConnection

createChannelServer

decodeFrame

defaultFormatLoader

defaultFormats

encodeFrame

openaiSpeaker

openRealtimeSession

pushError

sendPromptByTag

sendPromptWsByTag

startWebServer

internal

Interfaces

AgentsStatus

ChannelListing

EnrichedChannel

ListOptions

LoweredPromptMessage

ReadFileResult

RegistryEntry

SpeechMessage

Type Aliases

RunningServer

Variables

GREP_TOOL_OPENAI

LIST_FILES_TOOL_OPENAI

LOCAL_READ_TOOLS

READ_FILE_TOOL_OPENAI

REALTIME_VOICE_RATE

Functions

absentKeyPhrase

executeGrep

executeListFiles

executeReadFile

listChannels

listMcpServers

projectCacheDir

selectMcpServer

vendorKey

vendorKeySkipped

aiui-console

Overview

Getting Started with @habemus-papadum/aiui-console

API Reference

index

sidecar

Interfaces

ConsoleSidecarOptions

Variables

CONSOLE_PREFIX

Functions

consoleSidecar

aiui-intent-client

Overview

Getting Started with @habemus-papadum/aiui-intent-client

API Reference

index

Interfaces

CaptureSource

ClaimLaneOptions

FakeBus

HeldStream

IntentClient

IntentClientConfig

IntentContext

IntentHost

IntentLanes

PageTransport

PanelShot

RingState

SurfaceTargeting

Type Aliases

PageCapability

PageEvent

Functions

createIntentClient

fakeBus

sidecar

Interfaces

IntentSidecarOptions

Variables

PAGE_GLOBAL

Functions

intentSidecar

pageBundle

aiui-intent-runtime

Overview

Getting Started with @habemus-papadum/aiui-intent-runtime

API Reference

index

Interfaces

Ack

ErrorMessage

IntentErrorInput

IntentSocket

IntentThread

LoweredPromptMessage

OpenThreadOptions

ServerMessage

WebSocketLike

Type Aliases

AttachmentChunk

AudioChunk

JsonChunk

TabInfo

WebSocketFactory

Variables

PROTOCOL_VERSION

Functions

connectIntentSocket

isErrorMessage

instrumentation

Interfaces

ClientMeta

CollectClientMetaOptions

PageInstrumentation

Type Aliases

PageTabRecord

Variables

ACTOR_STORAGE_KEY

Functions

collectClientMeta

getInstrumentation

pageTabRecord

locator

Functions

locateComponents

selection

Interfaces

SelectionRect

SelectionSnapshot

SelectionWatcher

SelectionWatcherOptions

Functions

installSelectionWatcher

talk

Classes

SpeechPlayer

WorkletPcmSource

Interfaces

PcmSource

SpeechClip

Talk

TalkDeps

Transcriber

Variables

PCM_WORKLET_SOURCE

REALTIME_PCM_MIME

REALTIME_PCM_RATE

Functions

createTalk

mockTranscriber

thread

Interfaces

OpenIntentThreadOptions

Functions

openIntentThread

video

Classes

VideoSampler

Interfaces

SampledFrame

VideoSamplerDeps

Variables

VIDEO_FRAME_MIME

VIDEO_JPEG_QUALITY

VIDEO_MAX_WIDTH

VIDEO_SAMPLE_INTERVAL_MS

Functions

sampleDimensions

wire

Interfaces

Wire

WireDeps

Type Aliases

ThreadSocketState

Functions

createWire

aiui-lowering-pipeline

Overview

Getting Started with @habemus-papadum/aiui-lowering-pipeline

API Reference

index

Classes

Engine

Interfaces

AppSelection

CodeSelection

ComposedIntent

ComposedItem

ComposeOptions

DiffRun

IntentPipelineConfig

LocatedCell

LocatedComponent

Rect

ShotShare

TabRecord

TranscriptWord

Type Aliases

EngineListener

IntentEvent

LinterVendor

LintTurnAction

Mode

PromptSpan

TabInfo

VideoCaptureMode

Variables

DEFAULT_INTENT_CONFIG

LINT_TURN_ACTIONS

LINTER_VENDORS

Functions

applyPatch

composeIntent

expandTier

renderAppSelection

renderCodeSelection

renderPrompt

renderShotMetadata

renderTabRecord

wordDiff

trace-stages

Interfaces

AudioOutOfOrderData

ComposedSpeculativeData

ConditionData

IntentConfigData

RealtimeCommitData

RealtimeDiscardData

SpeechData

SttPartialData

Type Aliases

FrameChunkKind

LinterStageLabel

ParsedStage

StageTag

SttDiagnosticKind

Variables

stageLabel

Functions

parseStageLabel

aiui-oracle

Overview

Getting Started with @habemus-papadum/aiui-oracle

The Oracle

API Reference

index

Classes

OracleSession

Interfaces

ControlSurfaceToolsOptions

FirstReplyGuard

InputTranscription

KeySource

NoiseReduction

OracleAudio

OracleAudioInput

OracleAudioOutput

OracleConfig

OracleCredential

OracleReasoning

OracleSessionOptions

OracleState

OracleTool

OracleTransport

ParamSpec

PromptContext

PromptSlots

RegistryToolsOptions

SemanticVad

ServerVad

StandardKeySourcesOptions

TransportCapabilities

TransportConnectOptions

TransportHandle

UsageTotals

WebRtcTransportOptions

Type Aliases

Greeting

LedgerBody

LedgerEntry

OracleStatus

ParamWhen

ReasoningEffort

Resolved

TurnDetection

Usage

WeaveOptions

Variables

CONSTRAINT_PARAMS

DEFAULT_FIRST_REPLY_MAX_MS

DEFAULT_FIRST_REPLY_PAD_MS

DEFAULT_INPUT_TRANSCRIPTION_MODEL

DEFAULT_ORACLE_MODEL

DEFAULT_ORACLE_VOICE

DEFAULT_PARK_AFTER_IDLE_SECONDS

OPENAI_BASE_URL

ORACLE_BASE_PERSONA

PASTED_KEY_STORAGE_KEY

SESSION_PARAMS

TURN_DETECTION

TURN_DETECTION_TYPE

WEBRTC_CAPABILITIES

Functions

abbreviateTokens

cachingKeySource

chainKeySource

controlValueSchema

devKeySource

formatUsd

getPath

mintClientSecret

mintingKeySource

onControlSurfaceChange

opensTurn

pasteKeySource

priceRealtimeUsage

pruneTurnDetection

rowDrifts

setPath

specApplies

standardKeySources

staticKeySource

toolsFromAiuiRegistry

toolsFromControlSurface

usageFromRealtimeResponse

weaveInstructions

webRtcTransport

widgets

Interfaces

OracleControlProps

OracleKeyProps

OracleParkBannerProps

OracleRealtimeParamsProps

OracleUsageProps

OracleViewerProps

OracleWebRtcParamsProps

ParamRowProps

TurnGroup

Type Aliases

LedgerCategory

Variables

ALL_CATEGORIES

ORACLE_WIDGET_STYLES

Functions

activityLine

categoryOf

defaultCategories

entryDetail

entryLine

groupTurns

OracleControl

OracleKey

OracleMind

OracleParkBanner

OracleRealtimeParams

OracleUsage

OracleViewer

OracleWebRtcParams

ParamRow

summarizeTurn

usageChips

useOracleState

aiui-pencil

Overview

Getting Started with @habemus-papadum/aiui-pencil

API Reference

client

Interfaces

PencilRemoteAppOptions

PencilStripProps

PenInputDeps

PenPreview

PenSink

PlaneBox

PlaneTracker

RemoteViewProps

ResolvedPresentation

SessionPickerProps

Type Aliases

Phase

Variables

FULL_PRESENTATION

REMOTE_APP_CSS

Functions

bindPenInput

createPlaneTracker

PencilRemoteApp

PencilStrip

RemoteView

resolvePresentation

SessionPicker

index

Classes

ClientSession

HostSession

PencilSurface

RemoteClient

RemoteHost

Interfaces

ClientSessionOptions

CuspConfig

Dab

DensifyConfig

HostSessionOptions

HostSessionStatus

InkSignals

InkSource

InkState

InkStroke

InputReport

LinkStats

OneEuroConfig

PencilParams

PencilSurfaceOptions

PenSample

PenSupport

PointerLike

Range

Rect

RemoteClientOptions

RemoteHostOptions

RemotePresentation

SessionInfo

Signal

StrokeContext

StrokeEnd

StrokeOverrides

StrokePlan

Surface

Telemetry

Vec

VideoStatus

WirePoint

Type Aliases

CaptureState

ClientToRelay

HostToRelay

InkEvent

InkIntent

PencilMode

PenKind

PointerKind

Ramp

RelayToClient

RelayToHost

TiltVerdict

Tool

WireMessage

Variables

name

NEW_STROKE

PREVIEW_FADE_MS

SKETCH

WRITE

Functions

clientRelayUrl

dabAt

decode

densify

detectCusps

emptyTelemetry

encode

fadeWindowMs

filterSamples

fromNorm

hostRelayUrl

inkSignals

inputReport

isInkIntent

observe

penSample

penSupport

planStroke

polygonArea

polylineLength

resampleByArcLength

resolveParams

tiltVerdict

toNorm

aiui-remote-bar

Overview

API Reference

Classes

BarClient

BarHost

Interfaces

BarClientOptions

BarHostOptions

BarSource

BarState

BarTransport

BarTransportHandlers

BindRemoteBarOptions

BoundRemoteBar

RemoteBarClient

RemoteBarClientOptions

RemoteBarProps

RemoteCommand

SessionInfo

WireCap

Type Aliases

BarTransportFactory

ClientToRelay

ConnectionStatus

HostToRelay

RelayToClient

RelayToHost

WireMessage

Variables

PROTOCOL_VERSION

REMOTE_BAR_STYLES

websocketTransport

Functions

bindRemoteBar

createRemoteBarClient

decode

defaultBarUrl

encode

isRemoteCommand

RemoteBar

aiui-room-relay

Overview

Getting Started with @habemus-papadum/aiui-room-relay

API Reference

Interfaces

ClientMessageContext

HostMessageContext

RoomRelayBackend

RoomRelayDelegate

RoomRelayOptions

RoomSessionInfo

Type Aliases

Assignable

ChannelResolver

RoomServerFrame

Functions

createRoomRelayBackend

aiui-slides

Overview

API Reference

Interfaces

DeckFrame

DeckKeyState

DeckModel

SlideDef

SlideHandle

TouchIntent

TouchIntentOptions

WheelIntent

WheelIntentOptions

Type Aliases

DeckCommand

GestureStep

Variables

LensLayer

Functions

bindDeckToPath

createDeckModel

createTouchIntent

createWheelIntent

Deck

deckBase

DeckHud

deckKeyLayers

DeckNav

Lens

pathForSlide

ScrollCue

slideFromPath

Step

StepDots

useSlide

aiui-source-processor

Overview

Getting Started with @habemus-papadum/aiui-source-processor

API Reference

Interfaces

AiuiPluginOptions

FactorySpec

SourceLocatorOptions

SourceLocatorViteOptions

Functions

aiui

defaultFactories

devKeysSeed

sourceLocatorBabel

sourceLocatorVite

aiui-stt

Overview

Getting Started with @habemus-papadum/aiui-stt

API Reference

index

Interfaces

AudioSource

CreateSttOptions

OpenaiTranscriptionTransportOptions

ScribeTransportOptions

SttCallbacks

SttCapabilities

SttConnection

SttFinal

SttHandle

SttSignals

SttSocket

SttSocketHandlers

SttTransport

SttWord

Type Aliases

SttDiagnostic

SttEvent

SttSegmentFinal

SttSocketFactory

SttStatus

Variables

browserSocketFactory

DEFAULT_DRAIN_TIMEOUT_MS

DEFAULT_OPENAI_TRANSCRIPTION_MODEL

DEFAULT_SCRIBE_MODEL

OPENAI_CAPABILITIES

OPENAI_REALTIME_URL

SCRIBE_CAPABILITIES

SCRIBE_COMMIT_FLOOR_MS

SCRIBE_KEEPALIVE_MS

SCRIBE_SAMPLE_RATE

Functions

createStt

openaiTranscriptionSession

openaiTranscriptionTransport

scribeTransport

sttSignals

wordsFromTokenLogprobs

mic

Interfaces

MicAudioSourceOptions

MicPickerProps

MicrophoneInfo

Variables

DEFAULT_MIC_FRAME_MS

MIC_SAMPLE_RATE

MIC_WORKLET_SOURCE

Functions

floatToPcm16

listMicrophones

micAudioSource

MicPicker

resampleLinear

aiui-test-app

Overview

aiui-trace-ui

Overview

Getting Started with @habemus-papadum/aiui-trace-ui

API Reference

Classes

TracesPane

TraceView

Interfaces

JsonTreeOptions

LiveTrace

MountDebugPageOptions

TracePollOptions

TracePollResult

TracesPaneOptions

TraceStageLike

TraceViewConfig

Type Aliases

PreviewUrl

Functions

createTracePoll

mountDebugPage

renderJsonTree

aiui-util

Overview

API Reference

index

Classes

VaultUnavailableError

Interfaces

BrowserAutoOpenConfig

BrowserAutoOpenFlags

CacheDirOptions

ResolvedVendorKey

ResolveVendorKeysOptions

SecretInputOptions

SelfProvenanceOptions

SessionBrowser

ToolRunResult

VaultOptions

VendorKeySpec

Type Aliases

BrowserAction

ChromeChannel

KeyDecision

KeyDecisions

KeySource

ResolvedVendorKeys

ToolRunner

VaultProbe

VendorProvider

Variables

CHROME_CHANNELS

DEFAULT_VAULT_SERVICE

FORCE_INSTALLED_ENV

USERDATA_NAMESPACE

VENDOR_KEYS

Functions

cacheDir

decideBrowserAction

discoverSessionBrowser

discoverSessionBrowserInProfiles

headlessReason

isCi

isHeadless

isSsh

launchSessionBrowser

loadUnpackedExtension

openInSessionBrowser

ownPackageRoot

packageFromSource

packageRoot

probeVault

readKeyDecisions

readSecret

rehostSocketUrl

resolveVendorKeys

runningFromSource

sessionBrowserBinary

vaultDelete

vaultLabel

vaultLookup

vaultStore

vendorKeysMode

vendorKeysModeSelf

vendorKeySpec

web-surface

Interfaces

ClientSurface

ServeClientSurfaceOptions

Functions

serveClientSurface

aiui-viz

Overview

Attribution: from a gesture to source

DuckDB in the Browser: Parquet, Wasm, Quack, and Mosaic

The embedding bridge — Apple's Embedding Atlas as a Mosaic citizen

Frontend design choices

Frontend: hard-won details

Frontend: the playbook

Frontend: style guide

Frontend: user guide

Getting Started with @habemus-papadum/aiui-viz

API Reference

duckdb

Functions

fetchWithProgress

instantiateDuckDB

embedding-view

Interfaces

EmbeddingSelectionLike

Functions

EmbeddingView

embeddingViewSupported

index

Interfaces

ActionSpec

AgentTool

AgentToolkit

AgentToolkitHandle

AiuiGlobal

AiuiPageTool

AiuiToolsRegistry

BridgeEffectOptions

BridgeEntry

Cell

CellContext

CellOptions

ControlBox

ControlMeta

ControlSpec

DemoCard

DependencyEdge

DependencyRead

DurableCanvas

HotContext

MosaicPlotLike

MosaicProducerEntry

RegisteredAction

Scope

SignalBox

SitePage

SolidModeEngine

SolidModeEngineConfig

StandardToolsOptions

SurfaceView

ThrottledBox

Type Aliases

Adoption

CellCompute

CellState

ControlSurfaceEntry

MosaicProducerKind

Release

WorkerCancel

WorkerReply

WorkerRequest

WorkerRun

Functions

action

actionByName

adopt

agentToolkit

bridgeByName

bridgeEffect

bridgeRegistry

cell

cellGraph

cellRegistry

CellView

control

controlByName

ControlSlider

controlSurface

ControlToggle

dependencyEdges

disposeDurable

Dropdown

durable

durableCanvas

durableSignal

ensureAiuiGlobal

fromWorker

hotCellGraph

introspectInteractor

mosaicProducerByName

mosaicProducers

onMosaicProducersChange

PageBoundary

registerMosaicInput

registerMosaicPlot

registerStandardTools

scope

setSitePageActive

settledOnly

solidModeEngine

subscribeControlSurface

surfaceViewFor

throttled

workerStream

modal

Classes

LiveDiffText

Interfaces

BarInputs

BarRow

BarTreeNode

CapSpec

CapView

ChoiceRegion

ClaimsHandle

ClaimsOptions

ClaimSpec

ClaimStatus

DiffRun

DiffRunClasses

DispatchEvent

ExcludeRule

FocusTracker

GuardOptions

InstallKeysOptions

KeyBinding

KeyHint

KeyLayer

LadderRegion

LiveDiffTextOptions

ModeEngine

ModeEngineOptions

ModeEngineSpec

ModeSpec

ModeTable

ReconcilerOptions

SurfaceRule

ToggleRegion

WidgetSpec

WidgetView

Type Aliases

BarItem

BarNode

ClaimPhase

ClaimSpecs

CommandFn

EngineEvent

EngineState

EventBinding

GuardedOutcome

KeyClaim

RegionSpec

RegionValue

StatePatch

Functions

barModel

barTree

blurExitTarget

choice

createClaims

createFocusTracker

createModeEngine

createReconciler

escTarget

guardedEffect

installKeys

keyHints

ladder

resolveKey

runTransition

toggle

wordDiff

mosaic

Interfaces

MosaicCoordinator

Type Aliases

Directive

Functions

MosaicView

mosaic-selection

Interfaces

ClearedSelection

IntervalDimSpec

IntervalFacetBinding

PairFacetBinding

PointDimSpec

PointFacetBinding

SelectionClauseLike

SelectionDim

SelectionDimSurfaceEntry

SelectionDimTarget

SelectionLike

SelectionSignal

Type Aliases

FacetBinding

IntervalValue

PointValue

SelectionDimSpec

Functions

bindSelectionComponents

categorySelection

clearAllSelectionDims

clearSelectionFor

registerClearSelection

resetSelectionDimTargets

selectionDim

selectionDimByName

selectionDimReport

selectionDimSurface

selectionPredicateSql

selectionSignal

plot

Functions

PlotFigure

selection-inspector

Interfaces

InspectorCapabilityRow

InspectorClauseRow

SelectionInspectorModel

Functions

formatInspectorValue

SelectionInspector

selectionInspectorModel

selection-views

Interfaces

SavedSelectionView

SelectionViewsStore

Functions

selectionViews

SelectionViewsBar

site

Interfaces

SiteNavItem

SiteNavProps

Variables

pathname

Functions

navigateTo

SiteNav

TeX

TocRail

color-mode

Type Aliases

Variables

Functions

testing

Interfaces

CellHarness

CommitRecorder

WaitOptions

Functions

cellHarness

recordCommits

resetControlSurface

tick

whenReady

whenState

aiui-vscode

Overview

Getting Started with @habemus-papadum/aiui-vscode

VS Code Integration

API Reference

Interfaces

EditorSelection

ListChannelsOptions

PeersResponse

PublishResult

SelectionContribution

SessionPeer

Type Aliases

ChannelEntry

Variables

SESSION_CONTRIBUTION_TOPIC

Functions

channelLabel

fetchPeers

isProcessAlive

listChannels

publishSelection

registryDir

selectionLineCount

selectionLoc

selectionToContribution

create-aiui

Overview

API Reference

Type Aliases

TargetState

Variables

VERSION

Functions

appNameFrom

classifyTarget

dependencyRange

initGitRepo

packageManager

scaffoldApp

templateRoot

On this page

Variable: OPENAI_CAPABILITIES

const OPENAI_CAPABILITIES: SttCapabilities

Defined in: openai.ts:55

Pager

Previous pageDEFAULT_SCRIBE_MODEL

Next pageOPENAI_REALTIME_URL