> built 2026-10-14 05:27 UTC from e6ef523 (main) · hearing 0.0.7. Details: build_info.json
# index.html.md
# hearing
Pluggable meeting transcription & context-connected AI agents — batch today, live next.
`hearing` captures, transcribes, and reasons over meeting audio as four small,
**swappable** concerns wired by composition, not baked together:
```default
capture ──audio──▶ STT ──segments──▶ diarization ──labeled──▶ agents
(mic + system, (local or (who spoke (summaries, actions,
separate channels) cloud, pluggable) when / "me vs them") live notes, research)
```
The one-liner just works; everything else is an optional keyword argument.
```python
from hearing import transcribe
transcript = transcribe("meeting.wav") # -> Transcript (list of segments)
print(transcript.formatted())
```
## Install
```bash
pip install hearing # core: data model, channel split, CLI
pip install 'hearing[whisper]' # + default local STT engine (faster-whisper)
pip install 'hearing[agents]' # + Claude-backed agent layer
pip install 'hearing[all]' # whisper + agents + capture + http + storage (dol)
pip install 'hearing[diarize]' # + pyannote (separate individual remote speakers)
```
### STT engines
One `STTEngine` facade, many engines; pick one with `--engine` (CLI) or `get_engine(name)` (Python). Nothing downstream changes.
| `--engine` | Engine | Install | Needs |
|---------------------|------------------------------------------------------------------|---------------------|-------------------------------------------------------------------------------|
| `whisper` (default) | faster-whisper, local, any CPU/GPU | `hearing[whisper]` | — |
| `openai` | OpenAI transcription API (cloud) | `hearing[openai]` | `OPENAI_API_KEY` |
| `deepgram` | Deepgram Nova-3 (cloud; `--model nova-2` etc. swaps it) | `hearing[deepgram]` | `DEEPGRAM_API_KEY` |
| `mlx` | Whisper on Apple Silicon via mlx-whisper | `hearing[mlx]` | macOS on M-series |
| `parakeet` | NVIDIA Parakeet-TDT on Apple Silicon via parakeet-mlx | `hearing[mlx]` | macOS on M-series, plus `ffmpeg` on PATH (parakeet-mlx decodes audio with it) |
| `whisperx` | WhisperX: aligned word timestamps, optional pyannote diarization | `hearing[whisperx]` | `HF_TOKEN` (diarization only) |
The live path (`hearing live`, `stream_transcribe`) works with every engine: engines without native streaming are fed VAD-segmented utterances. `--model` overrides the engine’s default model (a whisper size such as `small` for whisper/mlx/whisperx, a model name for cloud engines).
## What works today (the batch milestone)
Give it a recording and get back a **speaker-labelled transcript** plus optional
**AI meeting notes**. If the file has the mic on one channel and the system audio
(everyone else) on another, you get “me vs them” labelling *for free* — no
diarization model, no voice enrollment — because the channel itself tells you
who is local.
```bash
hearing transcribe meeting.wav # diarized transcript to stdout
hearing transcribe meeting.wav --model small --out notes.json
hearing transcribe meeting.wav --engine openai # cloud STT (OpenAI); needs OPENAI_API_KEY + hearing[openai]
hearing transcribe meeting.wav --engine deepgram # cloud STT (Deepgram Nova-3); needs DEEPGRAM_API_KEY + hearing[deepgram]
hearing transcribe meeting.wav --engine mlx # local, Apple Silicon (hearing[mlx]); also: parakeet, whisperx
hearing transcribe meeting.wav --save ./meetings # persist the transcript (then `hearing meetings ./meetings`)
hearing summarize meeting.wav # transcribe + AI notes (Claude)
hearing summarize meeting.wav --agent extractive # offline, no API key needed
hearing summarize meeting.wav --context-dir ./project_notes # context-connected: RAG over your docs
hearing summarize meeting.wav --web-search # also pull Wikipedia fact context (key-free)
hearing live meeting.wav # STREAM it: finalized segments as utterances complete
hearing meetings ./meetings # list saved transcripts (--show ID to print one)
hearing info # what's installed / available
```
**Context-connected agents.** Point `--context-dir` at a folder of `.md`/`.txt`
notes (prior meetings’ takeaways, project docs); the agent does RAG over them and
brings in facts that aren’t in the transcript (who owns what, prior decisions,
budgets, deadlines). The retriever and a web-search hook are pluggable
Protocols (`hearing/context.py`): `KeywordRetriever` (TF-IDF, dependency-free,
the default) or `EmbeddingRetriever` (OpenAI semantic recall) — choose with
`--retriever keyword|embedding`. Swap in any vector backend behind the same
`Retriever` interface.
### Live / streaming (milestone 2 — core working)
The same pipeline runs as a low-latency loop: VAD finds utterance boundaries,
each finalized utterance is transcribed and emitted as it completes, and the
channel split still labels “me vs them” — the `STTEngine` and agent interfaces
are unchanged from batch.
```python
import asyncio
from hearing import live_transcribe
from hearing.capture import (
StreamingFileCapture,
) # or DeviceCapture for a live mic+system device
async def main():
async for seg in live_transcribe(source=StreamingFileCapture("meeting.wav")):
print(
seg.speaker, seg.text
) # finalized (seg.meta['final']) segments, as they land
asyncio.run(main())
```
Capturing from a live **device** (`DeviceCapture`, an Aggregate Device with mic
+ BlackHole) is implemented but needs that hardware to verify; streaming a file
works anywhere.
**Command palette (⌘K).** Press ⌘K (or click the button) for a command palette —
export the transcript (markdown/SRT/JSON), focus search, clear. Each command is
defined once (`frontend/src/commands.ts`, the acture pattern) and also projects to
an AI/MCP tool schema via `toAITools()`, so the same definition can drive the
palette and an AI assistant. Run the frontend’s unit tests with `npm test`.
Example transcript + Claude summary (from `examples/demo_meeting.py`, which
synthesizes a two-speaker meeting with macOS `say` — no mic needed):
```default
[00:00:00] me: Hi everyone, thanks for joining the planning sync. I'll own the
API design and send a draft by Friday. What is our deadline...
[00:00:08] them: Thanks. The launch deadline is the end of the month. Let's
decide on the database. I think we should use Postgres...
## Decisions
- PostgreSQL selected as the database for the project.
## Action items
- [me] Own the API design and send a draft by Friday.
## Open questions / follow-ups
- Confirm the exact end-of-month launch deadline.
```
### Composition (dependency injection)
Every concern is a `typing.Protocol`; swap one by changing a single argument.
```python
from hearing import transcribe
from hearing.stt import FasterWhisperSTT
from hearing.diarize import ChannelTrickDiarizer, PyannoteDiarizer
from hearing.agents import ClaudeAgent, ExtractiveAgent
from hearing.stt import OpenAISTT # or DeepgramSTT, MLXSTT, WhisperXSTT
transcribe(
"meeting.wav",
engine=OpenAISTT(), # cloud STT — one line vs FasterWhisperSTT()
diarizer=PyannoteDiarizer(), # upgrade from the channel trick
agent=ClaudeAgent(context="prior meeting takeaways..."), # or ExtractiveAgent()
)
```
The shared data spine is `hearing.types.TranscriptSegment` — a frozen, standoff
interval annotation (time is **integer milliseconds**, never float). See
`hearing/interfaces.py` for the four facade Protocols (`CaptureSource`,
`STTEngine`, `Diarizer`, `AgentConsumer`).
## Capturing the audio (macOS)
`hearing` reads multi-channel audio files and splits the channels. To get a
recording with mic on one channel and system audio (the other participants) on
another, use **BlackHole + an Aggregate Device** or **Core Audio taps** — the
mechanics, the channel layout, and the iOS limits are documented in the
`hearing-audio-capture` agent skill (`.claude/skills/hearing-audio-capture/`).
Live device capture is the next milestone; today you bring a recording.
## Not yet (designed, on the roadmap)
- **Native streaming engines** (Deepgram websocket, OpenAI Realtime,
WhisperLive) behind the same `STTEngine` facade — today every engine streams
through VAD-segmented batch calls — and **semantic turn detection** — see
`hearing-live-pipeline`.
- **Live device capture verified on hardware** (`DeviceCapture` is written; needs
a BlackHole + Aggregate Device) — see `hearing-audio-capture`.
- **TypeScript frontend** (schema-driven transcript viewer + copilot overlay
with zodal/acture) — see `hearing-frontend`.
Full status in [`misc/docs/ROADMAP.md`]().
## Web UI (HTTP API + React frontend)
A FastAPI layer wraps the same facades, and a schema-driven React frontend
(zodal + Zod) renders the diarized transcript (me/them lanes) and the AI notes.
```bash
pip install 'hearing[http,whisper,agents]'
hearing serve # API on http://127.0.0.1:8000 (docs at /docs)
cd frontend && npm install && npm run dev # UI on http://localhost:5173 (proxies /api)
```
Then drop an audio file on the page → diarized transcript + AI meeting notes.
Tick **Live stream** to watch finalized segments append in real time as the live
pipeline produces them (`POST /api/transcribe/stream`, NDJSON). The
TypeScript data shapes live in `frontend/src/schema.ts` as zodal collections
(one Zod schema per surface) and double as the API response validator, so the
Python backend and the UI share one contract. See the `hearing-frontend` skill.
## Architecture & development
This project is developed with a suite of agent **skills** under
`.claude/skills/` — start with the `hearing` skill (the project-manager /
orchestrator), which routes to the per-concern skills (`hearing-architecture`,
`hearing-audio-capture`, `hearing-stt`, `hearing-diarization`, `hearing-agents`,
`hearing-live-pipeline`, `hearing-frontend`). The two research reports the design
is grounded in are in `misc/docs/`.
## License
MIT
This documentation as a single file: hearing.md (Markdown, for agents).
# _autosummary/hearing.agents.html.md
# hearing.agents
The agent / “copilot” layer — the actual point of the hearing project.
An agent consumes a (speaker-labelled) transcript and produces feedback. The
*same* [`AgentConsumer`](_autosummary/hearing.interfaces.html.md#hearing.interfaces.AgentConsumer) interface serves both modes;
only *when* and *what granularity* it consumes differs:
* **batch** — `on_window(whole_transcript)` -> summary, action items,
decisions, open questions, follow-up research (post-meeting).
* **live** — `on_segment(finalized_segment)` -> running notes, suggested
questions, surfaced docs, fact-checks (as the meeting unfolds).
Agents are meant to be *context-connected*: bound to a meeting/project and able
to draw on accumulated knowledge (prior takeaways, the codebase, research
reports) and live web search. The `context` field is that hook.
The default backend is Claude (latest/most capable) but stays pluggable — see
the `claude-api` skill for model ids, pricing, tool use, and structured
output, and the `hearing-agents` skill for the queue-consumer pattern, RAG,
and the batch≡live unification.
### Module Attributes
| [`DEFAULT_CLAUDE_MODEL`](_autosummary/hearing.agents.html.md#hearing.agents.DEFAULT_CLAUDE_MODEL) | A sensible, current default Claude model for meeting analysis. |
|-------------------------------------------------------------------------|------------------------------------------------------------------|
### Functions
| [`build_default_agent`](_autosummary/hearing.agents.html.md#hearing.agents.build_default_agent)(\*[, context, model, ...]) | Pick the best available agent. |
|-----------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------|
| [`summarize_transcript`](_autosummary/hearing.agents.html.md#hearing.agents.summarize_transcript)(transcript, \*[, agent, ...]) | Synchronous facade: run a (batch) agent over a transcript, return notes. |
### Classes
| [`ClaudeAgent`](_autosummary/hearing.agents.html.md#hearing.agents.ClaudeAgent)([model, context, system_prompt, ...]) | Context-connected meeting agent backed by Claude (the default). |
|------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------|
| [`ExtractiveAgent`](_autosummary/hearing.agents.html.md#hearing.agents.ExtractiveAgent)([max_key_points, action_cues]) | Deterministic, dependency-free fallback agent (no API, no network). |
### *class* hearing.agents.ClaudeAgent(model='claude-sonnet-4-6', context=None, system_prompt='You are a meeting assistant. Given a (speaker-labelled) transcript, produce a concise, well-structured analysis with these sections, omitting any that don\\\\'t apply: \\\\n\\\\n## Summary\\\\n2-4 sentences capturing what the meeting was about and its outcome.\\\\n\\\\n## Decisions\\\\nBullet points of decisions that were made.\\\\n\\\\n## Action items\\\\nBullet points as "- [owner] action" when an owner is identifiable, else "- action".\\\\n\\\\n## Open questions / follow-ups\\\\nBullet points of unresolved questions or things worth researching afterwards.\\\\n\\\\nBe faithful to the transcript; do not invent facts. Speakers labelled "me" are the local user; "them"/"spk_\*" are remote participants.', max_tokens=1024, api_key=None, retriever=None, web_search=None, retrieve_k=4, web_k=3)
Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)
Context-connected meeting agent backed by Claude (the default).
* **Parameters:**
* **model** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – Claude model id (see the `claude-api` skill).
* **context** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – optional context to connect the agent to a meeting/project —
prior takeaways, project notes, etc. Prepended to the prompt.
* **system_prompt** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – override the default meeting-assistant instructions.
* **max_tokens** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – output cap.
#### *async* on_segment(segment)
Live per-segment hook (milestone 2). No-op in the batch milestone.
* **Return type:**
[`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]
#### *async* on_window(window)
Analyze a whole transcript (batch) and return structured notes.
* **Return type:**
[`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]
### hearing.agents.DEFAULT_CLAUDE_MODEL *: [str](https://docs.python.org/3/builtins/stdtypes.html#str)* *= 'claude-sonnet-4-6'*
A sensible, current default Claude model for meeting analysis. Override via
`model=` (e.g. “claude-opus-4-8” for max capability). See claude-api skill.
### *class* hearing.agents.ExtractiveAgent(max_key_points=5, action_cues=("let's", 'we should', "i'll", "we'll", 'need to', 'action', 'todo', 'to do', 'follow up', 'next step', 'assign', 'by friday', 'by monday', 'deadline'))
Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)
Deterministic, dependency-free fallback agent (no API, no network).
Produces a usable summary by extraction/heuristics so the app always does
*something* offline and tests stay deterministic. Detects naive action
items (imperative / commitment cues) and open questions (`?`).
#### action_cues *: [tuple](https://docs.python.org/3/builtins/stdtypes.html#tuple)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), ...]* *= ("let's", 'we should', "i'll", "we'll", 'need to', 'action', 'todo', 'to do', 'follow up', 'next step', 'assign', 'by friday', 'by monday', 'deadline')*
cues that hint a segment is an action item / commitment
#### *async* on_segment(segment)
Live per-segment hook: surface questions and action-item cues.
* **Return type:**
[`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]
#### *async* on_window(window)
Return a deterministic extractive summary of the window.
* **Return type:**
[`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]
### hearing.agents.build_default_agent(, context=None, model=None, prefer='auto', retriever=None, web_search=None)
Pick the best available agent.
`prefer="claude"` forces Claude; `"extractive"` forces the offline
fallback; `"auto"` (default) uses Claude when the anthropic SDK and an API
key are available, otherwise the dependency-free [`ExtractiveAgent`](_autosummary/hearing.agents.html.md#hearing.agents.ExtractiveAgent).
A `retriever`/`web_search` make the Claude agent context-connected.
### hearing.agents.summarize_transcript(transcript, , agent=None, context=None, model=None, retriever=None, web_search=None)
Synchronous facade: run a (batch) agent over a transcript, return notes.
Builds the default agent when none is injected (context-connected when a
`retriever`/`web_search` is supplied). Usable on a transcript with no
audio at all.
* **Return type:**
[`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]
# _autosummary/hearing.backends.html.md
# hearing.backends
Extra scribed backends that hearing contributes: Apple-Silicon MLX and WhisperX.
hearing’s STT runs through scribed (see [`hearing.stt`](_autosummary/hearing.stt.html.md#module-hearing.stt)). scribed already ships
the cloud engines (OpenAI, Deepgram, AssemblyAI, …) and faster-whisper; the
engines below are not in scribed yet, so hearing provides them as \*\*scribed
plugins\*\*, registered at runtime with `scribed.register_backend()`:
- `mlx-whisper` — Whisper on Apple Silicon via MLX (`pip install 'hearing[mlx]'`)
- `parakeet-mlx` — NVIDIA Parakeet-TDT on Apple Silicon via MLX (same extra)
- `whisperx` — WhisperX: batched Whisper + word alignment + optional pyannote
diarization (`pip install 'hearing[whisperx]'`)
Each backend is a subpackage laid out exactly like a `scribed.backends.`
package (`config.py` with `BACKEND_CONFIG`, `adapter.py` with `Adapter`),
so it can move into scribed verbatim. Registration is lazy and non-clobbering:
[`register_backends()`](_autosummary/hearing.backends.html.md#hearing.backends.register_backends) only adds an id scribed does not already provide, so
once scribed ships one of these, scribed’s own implementation wins.
Nothing here imports an engine SDK at import time; the adapters import it on first
transcription and raise an informative error (naming the extra) when it is absent,
or — for the MLX engines — when not running on Apple Silicon.
### Module Attributes
| [`BACKEND_MODULES`](_autosummary/hearing.backends.html.md#hearing.backends.BACKEND_MODULES) | Subpackages of this package that are scribed backends (backend id -> module). |
|--------------------------------------------------------------------|---------------------------------------------------------------------------------|
### Functions
| [`backend_config`](_autosummary/hearing.backends.html.md#hearing.backends.backend_config)(backend_id) | The `BACKEND_CONFIG` of one of hearing's backends (no SDK import). |
|-------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------|
| [`is_apple_silicon`](_autosummary/hearing.backends.html.md#hearing.backends.is_apple_silicon)(\*[, system, machine]) | True on macOS running on an ARM (M-series) CPU — where MLX works. |
| [`logprob_to_confidence`](_autosummary/hearing.backends.html.md#hearing.backends.logprob_to_confidence)(avg_logprob) | Map Whisper's average log-probability to a rough `[0, 1]` confidence. |
| [`register_backends`](_autosummary/hearing.backends.html.md#hearing.backends.register_backends)([backend_ids, registry]) | Register hearing's backends with scribed's registry; return the ids added. |
| [`require_apple_silicon`](_autosummary/hearing.backends.html.md#hearing.backends.require_apple_silicon)(engine, \*[, check]) | Raise an informative `RuntimeError` unless running on Apple Silicon. |
| [`require_module`](_autosummary/hearing.backends.html.md#hearing.backends.require_module)(module_name, \*, engine, extra) | Import `module_name` or raise an `ImportError` that names the extra. |
| [`to_waveform_16k`](_autosummary/hearing.backends.html.md#hearing.backends.to_waveform_16k)(audio) | Decode any scribed audio input to a float32 mono 16 kHz numpy array. |
| [`whisper_result_to_segments`](_autosummary/hearing.backends.html.md#hearing.backends.whisper_result_to_segments)(result) | Build scribed `Segment`s from an openai-whisper-style result dict. |
### hearing.backends.BACKEND_MODULES *: [Dict](https://docs.python.org/3/library/typing.html#typing.Dict)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), [str](https://docs.python.org/3/builtins/stdtypes.html#str)]* *= {'mlx-whisper': 'hearing.backends.mlx_whisper', 'parakeet-mlx': 'hearing.backends.parakeet_mlx', 'whisperx': 'hearing.backends.whisperx'}*
Subpackages of this package that are scribed backends (backend id -> module).
### hearing.backends.backend_config(backend_id)
The `BACKEND_CONFIG` of one of hearing’s backends (no SDK import).
* **Return type:**
[`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict)
### hearing.backends.is_apple_silicon(\*, system=, machine=)
True on macOS running on an ARM (M-series) CPU — where MLX works.
* **Return type:**
[`bool`](https://docs.python.org/3/builtins/functions.html#bool)
### hearing.backends.logprob_to_confidence(avg_logprob)
Map Whisper’s average log-probability to a rough `[0, 1]` confidence.
* **Return type:**
[`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`float`](https://docs.python.org/3/builtins/functions.html#float)]
### hearing.backends.register_backends(backend_ids=None, , registry=None)
Register hearing’s backends with scribed’s registry; return the ids added.
Idempotent and non-clobbering: an id scribed already knows is left alone.
The adapter itself is loaded lazily by scribed (from `module_path`).
* **Parameters:**
* **backend_ids** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`Iterable`](https://docs.python.org/3/library/typing.html#typing.Iterable)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]]) – which backends to register (default: all of them).
* **registry** ([`Any`](https://docs.python.org/3/library/typing.html#typing.Any)) – the scribed registry module (injectable for tests).
* **Return type:**
[`List`](https://docs.python.org/3/library/typing.html#typing.List)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]
### hearing.backends.require_apple_silicon(engine, , check=None)
Raise an informative `RuntimeError` unless running on Apple Silicon.
* **Return type:**
[`None`](https://docs.python.org/3/builtins/constants.html#None)
### hearing.backends.require_module(module_name, , engine, extra)
Import `module_name` or raise an `ImportError` that names the extra.
* **Return type:**
[`Any`](https://docs.python.org/3/library/typing.html#typing.Any)
### hearing.backends.to_waveform_16k(audio)
Decode any scribed audio input to a float32 mono 16 kHz numpy array.
Whisper-family engines accept such an array directly, which avoids their
ffmpeg-based file loading (hearing hands utterances over as WAV bytes).
* **Return type:**
[`Any`](https://docs.python.org/3/library/typing.html#typing.Any)
### hearing.backends.whisper_result_to_segments(result)
Build scribed `Segment`s from an openai-whisper-style result dict.
Handles both mlx-whisper (word `probability`) and WhisperX (word `score`,
optional `speaker` on segments and words). Times are float seconds.
* **Return type:**
[`list`](https://docs.python.org/3/builtins/stdtypes.html#list)
```pycon
>>> segs = whisper_result_to_segments({"segments": [
... {"start": 0.0, "end": 1.5, "text": " hi there",
... "words": [{"word": " hi", "start": 0.0, "end": 0.4, "probability": 0.9}]}
... ]})
>>> segs[0].text, segs[0].span.end_ms, segs[0].words[0].text
('hi there', 1500, 'hi')
```
### Modules
| [`mlx_whisper`](_autosummary/hearing.backends.mlx_whisper.html.md#module-hearing.backends.mlx_whisper) | mlx-whisper (Whisper on Apple Silicon via MLX) backend for scribed, from hearing. |
|----------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------|
| [`parakeet_mlx`](_autosummary/hearing.backends.parakeet_mlx.html.md#module-hearing.backends.parakeet_mlx) | parakeet-mlx (NVIDIA Parakeet-TDT on Apple Silicon via MLX) backend for scribed, from hearing. |
| [`whisperx`](_autosummary/hearing.backends.whisperx.html.md#module-hearing.backends.whisperx) | WhisperX (batched Whisper + alignment + optional diarization) backend for scribed, from hearing. |
# _autosummary/hearing.backends.mlx_whisper.adapter.html.md
# hearing.backends.mlx_whisper.adapter
Adapter for the mlx-whisper backend (Whisper on Apple Silicon via MLX).
Decodes the input to a float32 mono 16 kHz array (no ffmpeg needed), calls
`mlx_whisper.transcribe(array, path_or_hf_repo=...)` and maps its
openai-whisper-shaped result dict onto a `scribed.base.Transcript`.
`mlx_whisper` is imported on first use, after the Apple-Silicon check, so this
module imports anywhere.
### Module Attributes
| [`MODEL_REPO_TEMPLATE`](_autosummary/hearing.backends.mlx_whisper.adapter.html.md#hearing.backends.mlx_whisper.adapter.MODEL_REPO_TEMPLATE) | How a bare Whisper size maps to an `mlx-community` repo id. |
|------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------|
| [`MODEL_REPO_OVERRIDES`](_autosummary/hearing.backends.mlx_whisper.adapter.html.md#hearing.backends.mlx_whisper.adapter.MODEL_REPO_OVERRIDES) | Sizes whose mlx-community repo does not follow [`MODEL_REPO_TEMPLATE`](_autosummary/hearing.backends.mlx_whisper.adapter.html.md#hearing.backends.mlx_whisper.adapter.MODEL_REPO_TEMPLATE). |
### Functions
| [`resolve_model_repo`](_autosummary/hearing.backends.mlx_whisper.adapter.html.md#hearing.backends.mlx_whisper.adapter.resolve_model_repo)(model) | Resolve a model name to a Hugging Face repo id for `mlx_whisper`. |
|------------------------------------------------------------------------------|---------------------------------------------------------------------|
### Classes
| [`Adapter`](_autosummary/hearing.backends.mlx_whisper.adapter.html.md#hearing.backends.mlx_whisper.adapter.Adapter)(config) | mlx-whisper adapter (Apple Silicon only). |
|--------------------------------------------------------------------|---------------------------------------------|
### *class* hearing.backends.mlx_whisper.adapter.Adapter(config)
Bases: `BaseTranscriberAdapter`
mlx-whisper adapter (Apple Silicon only).
#### module_name *= 'mlx_whisper'*
Engine module name — the seam tests use to inject a fake backend.
### hearing.backends.mlx_whisper.adapter.MODEL_REPO_OVERRIDES *= {'large-v3-turbo': 'mlx-community/whisper-large-v3-turbo', 'turbo': 'mlx-community/whisper-large-v3-turbo'}*
Sizes whose mlx-community repo does not follow [`MODEL_REPO_TEMPLATE`](_autosummary/hearing.backends.mlx_whisper.adapter.html.md#hearing.backends.mlx_whisper.adapter.MODEL_REPO_TEMPLATE).
### hearing.backends.mlx_whisper.adapter.MODEL_REPO_TEMPLATE *= 'mlx-community/whisper-{model}-mlx'*
How a bare Whisper size maps to an `mlx-community` repo id.
### hearing.backends.mlx_whisper.adapter.resolve_model_repo(model)
Resolve a model name to a Hugging Face repo id for `mlx_whisper`.
* **Return type:**
[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)
```pycon
>>> resolve_model_repo("base")
'mlx-community/whisper-base-mlx'
>>> resolve_model_repo("turbo")
'mlx-community/whisper-large-v3-turbo'
>>> resolve_model_repo("someone/custom-whisper")
'someone/custom-whisper'
```
# _autosummary/hearing.backends.mlx_whisper.config.html.md
# hearing.backends.mlx_whisper.config
Configuration for the mlx-whisper backend (Whisper on Apple Silicon via MLX).
[mlx-whisper](https://pypi.org/project/mlx-whisper/) is Apple’s (ml-explore)
port of OpenAI Whisper to MLX: it runs on the M-series GPU with unified memory, and
returns openai-whisper-shaped results (segments, optional word timestamps).
macOS on Apple Silicon only.
`model` is either a Hugging Face repo id (`"mlx-community/whisper-large-v3-turbo"`,
the default) or a bare Whisper size (`"tiny"`, `"base"`, `"small"`,
`"medium"`, `"large-v3"`, `"turbo"`), resolved by
[`hearing.backends.mlx_whisper.adapter.resolve_model_repo()`](_autosummary/hearing.backends.mlx_whisper.adapter.html.md#hearing.backends.mlx_whisper.adapter.resolve_model_repo).
### Module Attributes
| [`DEFAULT_MODEL`](_autosummary/hearing.backends.mlx_whisper.config.html.md#hearing.backends.mlx_whisper.config.DEFAULT_MODEL) | Default MLX Whisper weights (fast and accurate on M2+). |
|------------------------------------------------------------------|-----------------------------------------------------------|
### hearing.backends.mlx_whisper.config.DEFAULT_MODEL *= 'mlx-community/whisper-large-v3-turbo'*
Default MLX Whisper weights (fast and accurate on M2+).
# _autosummary/hearing.backends.mlx_whisper.html.md
# hearing.backends.mlx_whisper
mlx-whisper (Whisper on Apple Silicon via MLX) backend for scribed, from hearing.
### Modules
| [`adapter`](_autosummary/hearing.backends.mlx_whisper.adapter.html.md#module-hearing.backends.mlx_whisper.adapter) | Adapter for the mlx-whisper backend (Whisper on Apple Silicon via MLX). |
|--------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------|
| [`config`](_autosummary/hearing.backends.mlx_whisper.config.html.md#module-hearing.backends.mlx_whisper.config) | Configuration for the mlx-whisper backend (Whisper on Apple Silicon via MLX). |
# _autosummary/hearing.backends.parakeet_mlx.adapter.html.md
# hearing.backends.parakeet_mlx.adapter
Adapter for the parakeet-mlx backend (NVIDIA Parakeet-TDT on Apple Silicon).
Loads a model with `parakeet_mlx.from_pretrained(repo)` (cached per repo),
transcribes a temporary audio file, and maps the returned `AlignedResult`’s
sentences (text + float-second `start`/`end`) onto scribed segments.
Parakeet’s tokens are sub-word pieces, so no word list is emitted.
`parakeet_mlx` is imported on first use, after the Apple-Silicon check.
### Classes
| [`Adapter`](_autosummary/hearing.backends.parakeet_mlx.adapter.html.md#hearing.backends.parakeet_mlx.adapter.Adapter)(config) | parakeet-mlx adapter (Apple Silicon only; caches one model per repo). |
|--------------------------------------------------------------------|-------------------------------------------------------------------------|
### *class* hearing.backends.parakeet_mlx.adapter.Adapter(config)
Bases: `BaseTranscriberAdapter`
parakeet-mlx adapter (Apple Silicon only; caches one model per repo).
#### module_name *= 'parakeet_mlx'*
Engine module name — the seam tests use to inject a fake backend.
# _autosummary/hearing.backends.parakeet_mlx.config.html.md
# hearing.backends.parakeet_mlx.config
Configuration for the parakeet-mlx backend (NVIDIA Parakeet-TDT on Apple Silicon).
[parakeet-mlx](https://pypi.org/project/parakeet-mlx/) runs NVIDIA’s
Parakeet-TDT models on Apple Silicon via MLX: among the fastest accurate local
English (v2) / 25-European-language (v3) engines. macOS on Apple Silicon only.
The language is detected by the model itself (no `language` argument).
### Module Attributes
| [`DEFAULT_MODEL`](_autosummary/hearing.backends.parakeet_mlx.config.html.md#hearing.backends.parakeet_mlx.config.DEFAULT_MODEL) | multilingual, 25 European languages). |
|------------------------------------------------------------------|-----------------------------------------|
### hearing.backends.parakeet_mlx.config.DEFAULT_MODEL *= 'mlx-community/parakeet-tdt-0.6b-v3'*
multilingual, 25 European languages).
* **Type:**
Default Parakeet weights (v3
# _autosummary/hearing.backends.parakeet_mlx.html.md
# hearing.backends.parakeet_mlx
parakeet-mlx (NVIDIA Parakeet-TDT on Apple Silicon via MLX) backend for scribed, from hearing.
### Modules
| [`adapter`](_autosummary/hearing.backends.parakeet_mlx.adapter.html.md#module-hearing.backends.parakeet_mlx.adapter) | Adapter for the parakeet-mlx backend (NVIDIA Parakeet-TDT on Apple Silicon). |
|---------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------|
| [`config`](_autosummary/hearing.backends.parakeet_mlx.config.html.md#module-hearing.backends.parakeet_mlx.config) | Configuration for the parakeet-mlx backend (NVIDIA Parakeet-TDT on Apple Silicon). |
# _autosummary/hearing.backends.whisperx.adapter.html.md
# hearing.backends.whisperx.adapter
Adapter for the WhisperX backend (transcribe -> align -> optionally diarize).
Pipeline, on a float32 mono 16 kHz array (no ffmpeg needed):
1. `whisperx.load_model(model, device, compute_type=...)` (cached) then
`.transcribe(audio, batch_size=..., language=...)`;
2. if `align`: `whisperx.load_align_model` + `whisperx.align` for accurate
word timestamps;
3. if `diarize`: pyannote via WhisperX’s `DiarizationPipeline` +
`whisperx.assign_word_speakers` (needs a Hugging Face token, env `HF_TOKEN`).
The result is openai-whisper-shaped, mapped by
[`hearing.backends.whisper_result_to_segments()`](_autosummary/hearing.backends.html.md#hearing.backends.whisper_result_to_segments). `whisperx` (and `torch`,
for device auto-detection) are imported on first use.
### Module Attributes
| [`DEFAULT_COMPUTE_TYPES`](_autosummary/hearing.backends.whisperx.adapter.html.md#hearing.backends.whisperx.adapter.DEFAULT_COMPUTE_TYPES) | `compute_type` used when none is given, by device. |
|--------------------------------------------------------------------------|------------------------------------------------------|
### Classes
| [`Adapter`](_autosummary/hearing.backends.whisperx.adapter.html.md#hearing.backends.whisperx.adapter.Adapter)(config) | WhisperX adapter (caches ASR, alignment and diarization models). |
|--------------------------------------------------------------------|--------------------------------------------------------------------|
### *class* hearing.backends.whisperx.adapter.Adapter(config)
Bases: `BaseTranscriberAdapter`
WhisperX adapter (caches ASR, alignment and diarization models).
#### module_name *= 'whisperx'*
Engine module name — the seam tests use to inject a fake backend.
### hearing.backends.whisperx.adapter.DEFAULT_COMPUTE_TYPES *= {'cpu': 'int8', 'cuda': 'float16'}*
`compute_type` used when none is given, by device.
# _autosummary/hearing.backends.whisperx.config.html.md
# hearing.backends.whisperx.config
Configuration for the WhisperX backend.
[WhisperX](https://github.com/m-bain/whisperX) = batched faster-whisper +
wav2vec2 forced alignment (accurate word timestamps) + optional pyannote speaker
diarization, in one pass — the sweet spot for multi-speaker meetings.
Diarization is off by default (in hearing, speakers are a separate concern — see
the `hearing-diarization` skill); pass `diarize=True` to have WhisperX label
speakers. It needs a Hugging Face token (env `HF_TOKEN`) that has accepted the
pyannote model terms. `device`/`compute_type` default to CUDA + float16 when a
GPU is visible, else CPU + int8.
### Module Attributes
| [`DEFAULT_MODEL`](_autosummary/hearing.backends.whisperx.config.html.md#hearing.backends.whisperx.config.DEFAULT_MODEL) | Default Whisper model size for WhisperX. |
|--------------------------------------------------------------------|-----------------------------------------------------------------------|
| [`HF_TOKEN_ENV_VARS`](_autosummary/hearing.backends.whisperx.config.html.md#hearing.backends.whisperx.config.HF_TOKEN_ENV_VARS) | Env var(s) holding the Hugging Face token pyannote diarization needs. |
### hearing.backends.whisperx.config.DEFAULT_MODEL *= 'large-v3'*
Default Whisper model size for WhisperX.
### hearing.backends.whisperx.config.HF_TOKEN_ENV_VARS *= ('HF_TOKEN', 'HUGGING_FACE_HUB_TOKEN')*
Env var(s) holding the Hugging Face token pyannote diarization needs.
# _autosummary/hearing.backends.whisperx.html.md
# hearing.backends.whisperx
WhisperX (batched Whisper + alignment + optional diarization) backend for scribed, from hearing.
### Modules
| [`adapter`](_autosummary/hearing.backends.whisperx.adapter.html.md#module-hearing.backends.whisperx.adapter) | Adapter for the WhisperX backend (transcribe -> align -> optionally diarize). |
|-----------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------|
| [`config`](_autosummary/hearing.backends.whisperx.config.html.md#module-hearing.backends.whisperx.config) | Configuration for the WhisperX backend. |
# _autosummary/hearing.capture.html.md
# hearing.capture
Audio capture and channel handling for the hearing pipeline.
This module owns turning an audio *source* into per-channel float arrays, and the
all-important **channel split**: mic and system audio are kept on separate channels
so that “who is local” is known for free (see the `hearing-diarization` skill for
what we do with that, and `hearing-audio-capture` for the macOS mechanics —
BlackHole + Aggregate Device, or Core Audio taps).
The generic, channel-agnostic conditioning (decode, down-mix, resample, energy) now
lives in `scribed.audio` and is re-exported here for backward compatibility —
hearing owns only the multi-channel / “me vs them” concerns.
### Functions
| [`split_channels`](_autosummary/hearing.capture.html.md#hearing.capture.split_channels)(data, \*[, mic_channels, ...]) | Split a multi-channel array into per-`Channel` mono arrays. |
|--------------------------------------------------------------------------------------------------|---------------------------------------------------------------|
### Classes
| [`ChannelSplitFileCapture`](_autosummary/hearing.capture.html.md#hearing.capture.ChannelSplitFileCapture)(path[, ...]) | A batch [`CaptureSource`](_autosummary/hearing.interfaces.html.md#hearing.interfaces.CaptureSource) over an audio file. |
|----------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------|
| [`DeviceCapture`](_autosummary/hearing.capture.html.md#hearing.capture.DeviceCapture)([device, block_ms, ...]) | Live macOS capture from an audio *device* (the milestone-2 capture source). |
| [`StreamingFileCapture`](_autosummary/hearing.capture.html.md#hearing.capture.StreamingFileCapture)(path[, block_ms, ...]) | A *streaming* [`CaptureSource`](_autosummary/hearing.interfaces.html.md#hearing.interfaces.CaptureSource) over a file. |
### *class* hearing.capture.ChannelSplitFileCapture(path, mic_channels=(0,), system_channels=(1,))
Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)
A batch [`CaptureSource`](_autosummary/hearing.interfaces.html.md#hearing.interfaces.CaptureSource) over an audio file.
Reads the file once and yields `(channel, mono_frames)` for each channel
present. The live counterpart (device/tap) implements the same interface;
swapping it in is the *only* change needed to go live — see the
`hearing-live-pipeline` skill.
Typical use (needs a real audio file, so it is not run as a doctest):
```default
ChannelSplitFileCapture("meeting.wav").frames()
```
#### *async* astream()
Stream the file’s channels in one shot (whole clip per channel).
Provided so a file satisfies the async `CaptureSource` too; for a
realistic block-by-block live simulation use [`StreamingFileCapture`](_autosummary/hearing.capture.html.md#hearing.capture.StreamingFileCapture).
* **Return type:**
[`AsyncIterator`](https://docs.python.org/3/library/typing.html#typing.AsyncIterator)[[`tuple`](https://docs.python.org/3/builtins/stdtypes.html#tuple)[[`Channel`](_autosummary/hearing.types.html.md#hearing.types.Channel), `ndarray`]]
#### channels()
The per-channel mono arrays at the file’s native sample rate.
* **Return type:**
[`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict)[[`Channel`](_autosummary/hearing.types.html.md#hearing.types.Channel), `ndarray`]
#### frames()
Yield `(channel, mono_frames)` once per channel (whole clip).
* **Return type:**
[`Iterator`](https://docs.python.org/3/library/typing.html#typing.Iterator)[[`tuple`](https://docs.python.org/3/builtins/stdtypes.html#tuple)[[`Channel`](_autosummary/hearing.types.html.md#hearing.types.Channel), `ndarray`]]
### *class* hearing.capture.DeviceCapture(device=None, block_ms=200, mic_channels=(0, 1), system_channels=(2, 3), sample_rate=48000, channels_count=4)
Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)
Live macOS capture from an audio *device* (the milestone-2 capture source).
Point this at an **Aggregate Device** that combines your microphone and
BlackHole (so mic and system audio land on separate, non-mixed channels —
see the `hearing-audio-capture` skill for the Audio MIDI Setup). Reads
multi-channel blocks and splits them into mic/system channels.
Requires `pip install 'hearing[capture]'` (sounddevice). Needs real audio
hardware to exercise; constructing it is cheap (the device opens in
`astream`). For a hardware-free run of the live loop, use
[`StreamingFileCapture`](_autosummary/hearing.capture.html.md#hearing.capture.StreamingFileCapture).
#### *async* astream()
Open the device and yield `(channel, block)` blocks until cancelled.
* **Return type:**
[`AsyncIterator`](https://docs.python.org/3/library/typing.html#typing.AsyncIterator)[[`tuple`](https://docs.python.org/3/builtins/stdtypes.html#tuple)[[`Channel`](_autosummary/hearing.types.html.md#hearing.types.Channel), `ndarray`]]
#### *static* list_devices()
Return available audio devices (helps pick the Aggregate Device).
### *class* hearing.capture.StreamingFileCapture(path, block_ms=200, realtime=False, mic_channels=(0,), system_channels=(1,))
Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)
A *streaming* [`CaptureSource`](_autosummary/hearing.interfaces.html.md#hearing.interfaces.CaptureSource) over a file.
Yields `(channel, block)` in `block_ms` chunks, time-ordered across
channels, so the live pipeline can be exercised end-to-end without a mic or
BlackHole. With `realtime=True` it paces playback in real time (for demos);
the default streams as fast as possible (for tests).
#### *async* astream()
Yield `(channel, block)` blocks, all channels per time step.
* **Return type:**
[`AsyncIterator`](https://docs.python.org/3/library/typing.html#typing.AsyncIterator)[[`tuple`](https://docs.python.org/3/builtins/stdtypes.html#tuple)[[`Channel`](_autosummary/hearing.types.html.md#hearing.types.Channel), `ndarray`]]
#### channels()
The per-channel mono arrays at the file’s native sample rate.
* **Return type:**
[`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict)[[`Channel`](_autosummary/hearing.types.html.md#hearing.types.Channel), `ndarray`]
#### frames()
Batch view: whole clip per channel.
* **Return type:**
[`Iterator`](https://docs.python.org/3/library/typing.html#typing.Iterator)[[`tuple`](https://docs.python.org/3/builtins/stdtypes.html#tuple)[[`Channel`](_autosummary/hearing.types.html.md#hearing.types.Channel), `ndarray`]]
### hearing.capture.split_channels(data, , mic_channels=(0,), system_channels=(1,))
Split a multi-channel array into per-`Channel` mono arrays.
An aggregate device *does not mix* — mic lands on one set of channels and
system audio on another — so we can slice the columns apart. A mono file
(or single-column) has no split; it is returned as `{Channel.MIXED: ...}`.
* **Parameters:**
* **data** (`ndarray`) – shape `(n_samples,)` or `(n_samples, n_channels)`.
* **mic_channels** ([`Sequence`](https://docs.python.org/3/library/typing.html#typing.Sequence)[[`int`](https://docs.python.org/3/builtins/functions.html#int)]) – column indices carrying the local microphone.
* **system_channels** ([`Sequence`](https://docs.python.org/3/library/typing.html#typing.Sequence)[[`int`](https://docs.python.org/3/builtins/functions.html#int)]) – column indices carrying captured system audio.
* **Return type:**
[`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict)[[`Channel`](_autosummary/hearing.types.html.md#hearing.types.Channel), `ndarray`]
* **Returns:**
Mapping from `Channel` to a mono float32 array. Channels whose
indices are out of range are silently omitted.
# _autosummary/hearing.cli.html.md
# hearing.cli
Command-line interface for hearing, dispatched with `cw`.
Surfaces the Python facades (`transcribe`, `summarize`) as subcommands — it does
not reimplement them (the facades are the SSOT; see the `python-dispatching`
and `hearing-architecture` skills). Installed as the `hearing` console
script:
```default
hearing transcribe meeting.wav
hearing transcribe meeting.wav --model small --out notes.json
hearing transcribe meeting.wav --engine deepgram # or openai / mlx / parakeet / whisperx
hearing summarize meeting.wav # transcribe + AI meeting notes
hearing info # check what's installed/available
```
### Functions
| [`info`](_autosummary/hearing.cli.html.md#hearing.cli.info)() | Report which optional components are installed and what's missing. |
|----------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------|
| [`live`](_autosummary/hearing.cli.html.md#hearing.cli.live)([path, engine, model, device, ...]) | Run the LIVE streaming pipeline, printing segments as utterances finalize. |
| [`main`](_autosummary/hearing.cli.html.md#hearing.cli.main)([argv]) | Entry point for the `hearing` console script. |
| [`meetings`](_autosummary/hearing.cli.html.md#hearing.cli.meetings)(store, \*[, show]) | List transcripts saved in a store (or print one with --show ID). |
| [`serve`](_autosummary/hearing.cli.html.md#hearing.cli.serve)(\*[, host, port, reload]) | Serve the HTTP API (FastAPI) the frontend talks to. |
| [`summarize`](_autosummary/hearing.cli.html.md#hearing.cli.summarize)(path, \*[, agent, model, context, ...]) | Transcribe a meeting and produce AI notes (summary, actions, questions). |
| [`transcribe`](_autosummary/hearing.cli.html.md#hearing.cli.transcribe)(path, \*[, engine, model, ...]) | Transcribe an audio file to text or JSON. |
### hearing.cli.info()
Report which optional components are installed and what’s missing.
* **Return type:**
[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)
### hearing.cli.live(path=None, , engine='whisper', model=None, device=None, block_ms=200, realtime=False)
Run the LIVE streaming pipeline, printing segments as utterances finalize.
* **Parameters:**
* **path** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – stream this audio FILE through the live loop (great for a demo /
no hardware). Omit to capture from an audio DEVICE instead.
* **engine** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – STT engine (same choices as `transcribe --engine`); engines
without native streaming are fed VAD-segmented utterances.
* **model** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – STT model override (omit for the engine’s default).
* **device** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – audio device index/name for live capture (an Aggregate Device
with mic + BlackHole; see the hearing-audio-capture skill).
* **block_ms** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – streaming block size in milliseconds.
* **realtime** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – when streaming a file, pace it in real time (else as-fast-as).
* **Return type:**
[`None`](https://docs.python.org/3/builtins/constants.html#None)
### hearing.cli.main(argv=None)
Entry point for the `hearing` console script.
`cw.run` *returns* the exit code (argh’s `parser.dispatch` exited by
itself), so the `SystemExit` here is what makes `hearing no-such-command`
exit 2.
* **Return type:**
[`None`](https://docs.python.org/3/builtins/constants.html#None)
### hearing.cli.meetings(store, , show=None)
List transcripts saved in a store (or print one with –show ID).
* **Parameters:**
* **store** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – the store/folder a transcript was saved to (`transcribe --save`).
* **show** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – a meeting id to print (formatted transcript) instead of listing.
* **Return type:**
[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)
### hearing.cli.serve(, host='127.0.0.1', port=8000, reload=False)
Serve the HTTP API (FastAPI) the frontend talks to.
* **Parameters:**
* **host** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – bind address.
* **port** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – bind port.
* **reload** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – auto-reload on code changes (dev).
* **Return type:**
[`None`](https://docs.python.org/3/builtins/constants.html#None)
### hearing.cli.summarize(path, , agent='auto', model=None, context=None, context_dir=None, retriever='keyword', web_search=False, engine='whisper', transcribe_model=None, split=True)
Transcribe a meeting and produce AI notes (summary, actions, questions).
* **Parameters:**
* **path** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – audio file to transcribe and analyze.
* **agent** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – “auto” (Claude if available, else offline extractive), “claude”,
or “extractive” (deterministic, no API key needed).
* **model** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – Claude model id (only used by the claude/auto agent).
* **context** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – optional literal context string to connect the agent.
* **context_dir** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – a folder/file of .txt/.md context docs (prior takeaways,
project notes); the agent does RAG over it (context-connected).
* **retriever** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – “keyword” (TF-IDF, offline) or “embedding” (OpenAI semantic;
needs hearing[openai] + OPENAI_API_KEY). Only used with context_dir.
* **web_search** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – also bring in Wikipedia fact context (key-free) for the agent.
* **engine** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – STT engine (same choices as `transcribe --engine`).
* **transcribe_model** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – STT model override (omit for the engine’s default).
* **split** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – split mic/system channels (me vs them).
* **Return type:**
[`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]
### hearing.cli.transcribe(path, , engine='whisper', model=None, language=None, split=True, diarize=True, out=None, fmt='text', save=None)
Transcribe an audio file to text or JSON.
* **Parameters:**
* **path** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – audio file (wav/flac/aiff/…; convert mp3/m4a with ffmpeg first).
* **engine** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – STT engine — “whisper” (local faster-whisper, default), “openai”
(cloud), “deepgram” (cloud, Nova-3), “mlx” (Whisper on Apple Silicon),
“parakeet” (Parakeet-TDT on Apple Silicon), or “whisperx” (aligned words).
* **model** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – engine model override — a whisper size (tiny/base/small/large-v3/…)
for whisper/mlx/whisperx, a model name for cloud engines (e.g.
“nova-3”); omit for the engine’s default.
* **language** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – force a language code (e.g. “en”); omit to auto-detect.
* **split** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – split mic/system channels for multi-channel files (me vs them).
* **diarize** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – apply the channel-trick “me vs them” labelling.
* **out** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – write to this path (.json -> JSON, else formatted text); else stdout.
* **fmt** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – stdout/text format — “text” or “json”.
* **save** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – persist the transcript to this store/folder (keyed by the file stem),
so post-meeting agents can load the full transcript later.
* **Return type:**
[`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]
# _autosummary/hearing.context.html.md
# hearing.context
Context stores and retrieval (RAG) for context-connected agents.
An agent with no context is just a chatbot — the point of `hearing` is agents
*bound to a meeting and a project*, drawing on accumulated knowledge (prior
takeaways, the codebase, research reports). This module provides the retrieval
side behind a tiny [`Retriever`](_autosummary/hearing.context.html.md#hearing.context.Retriever) Protocol so the backend is swappable and
testable with a fake (per the `hearing-agents` skill).
The default [`KeywordRetriever`](_autosummary/hearing.context.html.md#hearing.context.KeywordRetriever) is a dependency-free TF-IDF retriever —
good enough to wire context-connection end-to-end and fully deterministic for
tests. Swap in an embedding/vector backend (Qdrant + LlamaIndex, or the user’s
own local store — check `my-packages` / `python-storage`) behind the same
Protocol when you need semantic recall.
### Module Attributes
| [`EmbedFn`](_autosummary/hearing.context.html.md#hearing.context.EmbedFn) | maps texts to equal-length float vectors. |
|------------------------------------------------------------|---------------------------------------------|
### Functions
| [`build_embedding_retriever`](_autosummary/hearing.context.html.md#hearing.context.build_embedding_retriever)(source, \*[, embed]) | Build an [`EmbeddingRetriever`](_autosummary/hearing.context.html.md#hearing.context.EmbeddingRetriever) (default embedder: OpenAI) from a dict, a list, or a folder of `.md`/`.txt` docs (same source forms as [`build_retriever()`](_autosummary/hearing.context.html.md#hearing.context.build_retriever)). |
|---------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [`build_retriever`](_autosummary/hearing.context.html.md#hearing.context.build_retriever)(source) | Build a [`KeywordRetriever`](_autosummary/hearing.context.html.md#hearing.context.KeywordRetriever) from a dict, a list, or a folder/file. |
| [`openai_embedder`](_autosummary/hearing.context.html.md#hearing.context.openai_embedder)(\*[, model, api_key, client]) | Return an [`EmbedFn`](_autosummary/hearing.context.html.md#hearing.context.EmbedFn) backed by the OpenAI embeddings API. |
### Classes
| [`CallableWebSearch`](_autosummary/hearing.context.html.md#hearing.context.CallableWebSearch)(fn) | Adapt any callable into a [`WebSearch`](_autosummary/hearing.context.html.md#hearing.context.WebSearch) (for custom backends/tests). |
|------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------|
| [`Chunk`](_autosummary/hearing.context.html.md#hearing.context.Chunk)(text[, title, source, score]) | A retrieved piece of context: the text plus where it came from. |
| [`EmbeddingRetriever`](_autosummary/hearing.context.html.md#hearing.context.EmbeddingRetriever)(docs, \*, embed) | Semantic retriever: ranks chunks by cosine similarity of embeddings. |
| [`KeywordRetriever`](_autosummary/hearing.context.html.md#hearing.context.KeywordRetriever)(docs) | Dependency-free TF-IDF retriever over a small in-memory corpus. |
| [`Retriever`](_autosummary/hearing.context.html.md#hearing.context.Retriever)(\*args, \*\*kwargs) | Retrieves project/meeting context relevant to a query (RAG). |
| [`WebSearch`](_autosummary/hearing.context.html.md#hearing.context.WebSearch)(\*args, \*\*kwargs) | Brings outside information in — one targeted query, `k` results. |
| [`WikipediaSearch`](_autosummary/hearing.context.html.md#hearing.context.WikipediaSearch)([lang, timeout, fetch]) | Key-free, dependency-free [`WebSearch`](_autosummary/hearing.context.html.md#hearing.context.WebSearch) over Wikipedia (stdlib urllib). |
### *class* hearing.context.CallableWebSearch(fn)
Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)
Adapt any callable into a [`WebSearch`](_autosummary/hearing.context.html.md#hearing.context.WebSearch) (for custom backends/tests).
`fn(query, k)` may return Chunks, strings, or dicts (coerced), and may be
sync or async.
### *class* hearing.context.Chunk(text, title='', source='', score=0.0)
Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)
A retrieved piece of context: the text plus where it came from.
### hearing.context.EmbedFn
maps texts to equal-length float vectors.
* **Type:**
A batch embedder
alias of `Callable`[[[`Sequence`](https://docs.python.org/3/library/typing.html#typing.Sequence)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]], [`Sequence`](https://docs.python.org/3/library/typing.html#typing.Sequence)[[`Sequence`](https://docs.python.org/3/library/typing.html#typing.Sequence)[[`float`](https://docs.python.org/3/builtins/functions.html#float)]]]
### *class* hearing.context.EmbeddingRetriever(docs, , embed)
Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)
Semantic retriever: ranks chunks by cosine similarity of embeddings.
The corpus is embedded once at construction; `search` embeds the query and
returns the nearest chunks. The embedder is injected (`embed=`) so it is
backend-agnostic and testable with a deterministic fake — use
[`openai_embedder()`](_autosummary/hearing.context.html.md#hearing.context.openai_embedder) for real semantic recall.
```pycon
>>> import asyncio
>>> fake = lambda texts: [[t.lower().count("db"), t.lower().count("ui")] for t in texts]
>>> r = EmbeddingRetriever({"a": "db db", "b": "ui ui"}, embed=fake)
>>> asyncio.run(r.search("db"))[0].title
'a'
```
#### *async* search(query, , k=4)
Return the top-`k` chunks by cosine similarity to `query`.
* **Return type:**
[`list`](https://docs.python.org/3/builtins/stdtypes.html#list)[[`Chunk`](_autosummary/hearing.context.html.md#hearing.context.Chunk)]
### *class* hearing.context.KeywordRetriever(docs)
Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)
Dependency-free TF-IDF retriever over a small in-memory corpus.
```pycon
>>> import asyncio
>>> r = KeywordRetriever({"db": "We will use PostgreSQL.", "ui": "React frontend."})
>>> hits = asyncio.run(r.search("postgresql migration"))
>>> hits[0].title
'db'
```
Matching is purely lexical, so a query sharing no token with the corpus
retrieves nothing — use [`EmbeddingRetriever`](_autosummary/hearing.context.html.md#hearing.context.EmbeddingRetriever) for semantic recall.
```pycon
>>> asyncio.run(r.search("which database?"))
[]
```
#### *async* search(query, , k=4)
Return the top-`k` chunks by TF-IDF overlap with `query`.
* **Return type:**
[`list`](https://docs.python.org/3/builtins/stdtypes.html#list)[[`Chunk`](_autosummary/hearing.context.html.md#hearing.context.Chunk)]
### *class* hearing.context.Retriever(\*args, \*\*kwargs)
Bases: [`Protocol`](https://docs.python.org/3/library/typing.html#typing.Protocol)
Retrieves project/meeting context relevant to a query (RAG).
#### *async* search(query, , k=4)
Return up to `k` chunks most relevant to `query`.
* **Return type:**
[`Sequence`](https://docs.python.org/3/library/typing.html#typing.Sequence)[[`Chunk`](_autosummary/hearing.context.html.md#hearing.context.Chunk)]
### *class* hearing.context.WebSearch(\*args, \*\*kwargs)
Bases: [`Protocol`](https://docs.python.org/3/library/typing.html#typing.Protocol)
Brings outside information in — one targeted query, `k` results.
#### *async* search(query, , k=3)
Return up to `k` web results as chunks (title/body/url).
* **Return type:**
[`Sequence`](https://docs.python.org/3/library/typing.html#typing.Sequence)[[`Chunk`](_autosummary/hearing.context.html.md#hearing.context.Chunk)]
### *class* hearing.context.WikipediaSearch(lang='en', timeout=8.0, fetch=None)
Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)
Key-free, dependency-free [`WebSearch`](_autosummary/hearing.context.html.md#hearing.context.WebSearch) over Wikipedia (stdlib urllib).
Ideal for the live fact-check / “who is X / what is Y” use: one targeted
query, a few result snippets with source URLs. The HTTP `fetch` is
injectable so tests need no network.
#### *async* search(query, , k=3)
Return up to `k` Wikipedia search hits as chunks (title/snippet/url).
* **Return type:**
[`list`](https://docs.python.org/3/builtins/stdtypes.html#list)[[`Chunk`](_autosummary/hearing.context.html.md#hearing.context.Chunk)]
### hearing.context.build_embedding_retriever(source, , embed=None)
Build an [`EmbeddingRetriever`](_autosummary/hearing.context.html.md#hearing.context.EmbeddingRetriever) (default embedder: OpenAI) from a
dict, a list, or a folder of `.md`/`.txt` docs (same source forms as
[`build_retriever()`](_autosummary/hearing.context.html.md#hearing.context.build_retriever)).
* **Return type:**
[`EmbeddingRetriever`](_autosummary/hearing.context.html.md#hearing.context.EmbeddingRetriever)
### hearing.context.build_retriever(source)
Build a [`KeywordRetriever`](_autosummary/hearing.context.html.md#hearing.context.KeywordRetriever) from a dict, a list, or a folder/file.
A path to a directory indexes its `.txt`/`.md` files (keyed by stem); a
path to a single file indexes that file; a Mapping/Sequence is used directly.
* **Return type:**
[`KeywordRetriever`](_autosummary/hearing.context.html.md#hearing.context.KeywordRetriever)
### hearing.context.openai_embedder(, model='text-embedding-3-small', api_key=None, client=None)
Return an [`EmbedFn`](_autosummary/hearing.context.html.md#hearing.context.EmbedFn) backed by the OpenAI embeddings API.
Needs `pip install 'hearing[openai]'` and `OPENAI_API_KEY` (or pass
`api_key=` / inject `client=` for tests).
* **Return type:**
[`Callable`](https://docs.python.org/3/library/typing.html#typing.Callable)[[[`Sequence`](https://docs.python.org/3/library/typing.html#typing.Sequence)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]], [`Sequence`](https://docs.python.org/3/library/typing.html#typing.Sequence)[[`Sequence`](https://docs.python.org/3/library/typing.html#typing.Sequence)[[`float`](https://docs.python.org/3/builtins/functions.html#float)]]]
# _autosummary/hearing.diarize.html.md
# hearing.diarize
Diarization and speaker labelling for the hearing pipeline.
Two ideas, one interface ([`Diarizer`](_autosummary/hearing.interfaces.html.md#hearing.interfaces.Diarizer)):
1. **The channel trick (default, free).** Because capture keeps the mic and the
system audio on *separate channels*, we already know who is local: the
`MIC` channel is “me”, the `SYSTEM` channel is the remote participants.
[`ChannelTrickDiarizer`](_autosummary/hearing.diarize.html.md#hearing.diarize.ChannelTrickDiarizer) just maps channel -> speaker. No model, no
> audio, no error from overlap on the local side. This beats voice
> fingerprinting for the “is it me?” question.
2. **Real diarization (the upgrade).** To separate *multiple remote* speakers
you need acoustic diarization. [`PyannoteDiarizer`](_autosummary/hearing.diarize.html.md#hearing.diarize.PyannoteDiarizer) wraps
`pyannote.audio` and runs only on the system channel. A diarized transcript
is standoff interval annotation with a speaker tier — see the
`annotation-systems` skill.
See the `hearing-diarization` skill for the full landscape (pyannote vs NeMo
vs WhisperX vs SpeechBrain, DER numbers, enrollment).
### Classes
| [`ChannelTrickDiarizer`](_autosummary/hearing.diarize.html.md#hearing.diarize.ChannelTrickDiarizer)([me_label, them_label]) | Label speakers from the capture channel — the free, reliable default. |
|-------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------|
| [`PyannoteDiarizer`](_autosummary/hearing.diarize.html.md#hearing.diarize.PyannoteDiarizer)([model, hf_token, ...]) | Acoustic diarization via `pyannote.audio` (the upgrade path). |
### *class* hearing.diarize.ChannelTrickDiarizer(me_label='me', them_label='them')
Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)
Label speakers from the capture channel — the free, reliable default.
`MIC` -> `me_label` (the local user), `SYSTEM` -> `them_label` (a
remote participant). Segments already carrying a speaker, or coming from a
`MIXED` channel, are passed through unchanged (there is nothing to infer).
This is the architecture’s default diarizer whenever channel information is
present. To distinguish *individual* remote speakers, compose this with (or
replace it by) [`PyannoteDiarizer`](_autosummary/hearing.diarize.html.md#hearing.diarize.PyannoteDiarizer).
#### assign_speakers(segments, , audio=None, sample_rate=None)
Yield segments with `.speaker` set from their `.channel`.
* **Return type:**
[`Iterable`](https://docs.python.org/3/library/typing.html#typing.Iterable)[[`TranscriptSegment`](_autosummary/hearing.types.html.md#hearing.types.TranscriptSegment)]
### *class* hearing.diarize.PyannoteDiarizer(model='pyannote/speaker-diarization-3.1', hf_token=None, speaker_prefix='spk_')
Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)
Acoustic diarization via `pyannote.audio` (the upgrade path).
Runs the pretrained pipeline over the audio, then assigns each segment the
speaker whose diarized turn overlaps it most. Intended to run on the
*system* channel only (the mic channel is already known to be “me”).
Requires `pip install hearing[diarize]` and a Hugging Face token with the
pyannote model licenses accepted. This implementation is provided for the
diarization milestone; it is import-guarded so the package works without it.
#### assign_speakers(segments, , audio=None, sample_rate=None)
Yield segments labelled with the most-overlapping diarized speaker.
* **Return type:**
[`Iterable`](https://docs.python.org/3/library/typing.html#typing.Iterable)[[`TranscriptSegment`](_autosummary/hearing.types.html.md#hearing.types.TranscriptSegment)]
# _autosummary/hearing.html.md
# hearing
hearing — pluggable meeting transcription & context-aware AI agents.
A small, composable toolkit for capturing, transcribing, and reasoning over
meeting audio. Four swappable concerns compose into one pipeline by dependency
injection: audio **capture** (mic + system audio on separate channels) ->
**STT** engine facade (local or cloud) -> optional **diarization** / speaker-id
(the mic-vs-system channel split gives “me vs them” for free) -> an **agent**
layer that consumes the transcript (batch or live).
Progressive disclosure — the one-liner just works:
```default
from hearing import transcribe
transcript = transcribe("meeting.wav")
print(transcript.formatted())
```
…while engine selection, diarization, agents, channel routing, and the live
loop are all optional keyword-only arguments.
The architecture and the per-concern guidance live in the project’s agent skills
(`.claude/skills/hearing*`); start with the `hearing` skill.
### Functions
| [`transcribe`](_autosummary/hearing.html.md#hearing.transcribe)(source, \*[, engine, diarizer, ...]) | Transcribe an audio source to a [`Transcript`](_autosummary/hearing.types.html.md#hearing.types.Transcript). |
|----------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------|
| [`live_transcribe`](_autosummary/hearing.html.md#hearing.live_transcribe)(\*, source[, engine, ...]) | Stream FINALIZED segments as a meeting unfolds (the live path). |
| [`summarize`](_autosummary/hearing.html.md#hearing.summarize)(source, \*[, agent, context, model]) | Transcribe (if needed) then run an agent to produce meeting notes. |
### Classes
| [`Transcript`](_autosummary/hearing.html.md#hearing.Transcript)([segments, sample_rate, meta]) | A finished transcript: an ordered collection of segments plus metadata. |
|------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|
| [`TranscriptSegment`](_autosummary/hearing.html.md#hearing.TranscriptSegment)(text, span[, channel, ...]) | The data spine — a standoff interval annotation over the audio. |
| [`TimeSpan`](_autosummary/hearing.html.md#hearing.TimeSpan)(start_ms, end_ms) | A half-open interval `[start_ms, end_ms)` in integer milliseconds. |
| [`Word`](_autosummary/hearing.html.md#hearing.Word)(text, span[, confidence]) | Optional word-level timing (WhisperX / `word_timestamps=True`). |
| [`Channel`](_autosummary/hearing.html.md#hearing.Channel)(\*values) | Which capture channel a segment came from. |
| [`SpeakerLabel`](_autosummary/hearing.html.md#hearing.SpeakerLabel) | |
### *class* hearing.Channel(\*values)
Bases: [`str`](https://docs.python.org/3/builtins/stdtypes.html#str), [`Enum`](https://docs.python.org/3/library/enum.html#enum.Enum)
Which capture channel a segment came from.
The channel *is* the “me vs them” signal, for free: `MIC` is the local
user, `SYSTEM` is everyone else (audio that came out of the speakers).
`MIXED` is a single-channel / unknown source.
### hearing.SpeakerLabel
alias of [`str`](https://docs.python.org/3/builtins/stdtypes.html#str)
### *class* hearing.TimeSpan(start_ms, end_ms)
Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)
A half-open interval `[start_ms, end_ms)` in integer milliseconds.
Integer time, never float seconds — accumulation-safe and hashable.
```pycon
>>> TimeSpan(0, 1500).duration_ms
1500
```
#### *property* duration_ms *: [int](https://docs.python.org/3/builtins/functions.html#int)*
Length of the interval in milliseconds.
#### *classmethod* from_seconds(start, end)
Build a span from float seconds (e.g. from an STT engine).
* **Return type:**
[`TimeSpan`](_autosummary/hearing.types.html.md#hearing.types.TimeSpan)
### *class* hearing.Transcript(segments=(), sample_rate=None, meta=)
Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)
A finished transcript: an ordered collection of segments plus metadata.
Iterable and `len`-able, so it stands in for a `Sequence` of segments
wherever the architecture’s facades return “segments”. Adds convenience
accessors the CLI/agents need.
```pycon
>>> t = Transcript((TranscriptSegment("a", TimeSpan(0, 1000), speaker=ME),
... TranscriptSegment("b", TimeSpan(1000, 2000), speaker=THEM)))
>>> t.text
'a b'
>>> len(t)
2
>>> sorted(t.speakers)
['me', 'them']
```
#### *property* duration_ms *: [int](https://docs.python.org/3/builtins/functions.html#int)*
End of the last segment, or 0 if empty.
#### formatted(, speakers=True, timestamps=True)
Render as a human-readable transcript, one line per segment.
* **Return type:**
[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)
#### *property* speakers *: [set](https://docs.python.org/3/builtins/stdtypes.html#set)[[str](https://docs.python.org/3/builtins/stdtypes.html#str)]*
The distinct speaker labels present (ignoring `None`).
#### *property* text *: [str](https://docs.python.org/3/builtins/stdtypes.html#str)*
All segment texts joined by a single space.
#### to_jsonable()
A plain `list[dict]` ready for `json.dumps` (segments only).
* **Return type:**
[`list`](https://docs.python.org/3/builtins/stdtypes.html#list)[[`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict)]
#### with_meta(\*\*kw)
Return a copy with extra metadata merged in.
* **Return type:**
[`Transcript`](_autosummary/hearing.types.html.md#hearing.types.Transcript)
### *class* hearing.TranscriptSegment(text, span, channel=Channel.MIXED, speaker=None, confidence=None, words=(), meta=)
Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)
The data spine — a standoff interval annotation over the audio.
Required fields are `text` and `span`. Everything else is optional
metadata that a concern *enriches*: STT sets `text`/`span`/
`confidence`; capture sets `channel`; diarization sets `speaker`.
Segments are frozen — enrich by copying (`with_speaker`/`with_channel`),
never mutate.
```pycon
>>> seg = TranscriptSegment("hello", TimeSpan(0, 500))
>>> seg.with_speaker(ME).speaker
'me'
>>> seg.text
'hello'
```
#### with_channel(channel)
Return a copy carrying a channel label (frozen -> copy, don’t mutate).
* **Return type:**
[`TranscriptSegment`](_autosummary/hearing.types.html.md#hearing.types.TranscriptSegment)
#### with_speaker(speaker)
Return a copy carrying a speaker label (frozen -> copy, don’t mutate).
* **Return type:**
[`TranscriptSegment`](_autosummary/hearing.types.html.md#hearing.types.TranscriptSegment)
### *class* hearing.Word(text, span, confidence=None)
Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)
Optional word-level timing (WhisperX / `word_timestamps=True`).
Lets an agent trigger on a keyword mid-utterance instead of waiting for the
end of a turn. Purely optional metadata on a [`TranscriptSegment`](_autosummary/hearing.html.md#hearing.TranscriptSegment).
### *async* hearing.live_transcribe(, source, engine=None, diarizer=None, agent=None, audio_queue_max=50, segment_queue_max=100)
Stream FINALIZED segments as a meeting unfolds (the live path).
Same components as [`transcribe()`](_autosummary/hearing.html.md#hearing.transcribe) — only the *source* (a streaming
[`CaptureSource`](_autosummary/hearing.interfaces.html.md#hearing.interfaces.CaptureSource) instead of a file) and the trigger
cadence (VAD utterance turn-ends instead of run-once) change. The
`STTEngine` and `AgentConsumer` interfaces are unchanged.
Each channel is demuxed onto its own bounded queue and driven through
`engine.stream_transcribe` as an independent async task, so a slow agent
never stalls capture (backpressure) and the channel “me vs them” label rides
every segment. Diarization defaults to the free channel trick; the agent’s
`on_segment` is fired fire-and-forget. Yields segments as utterances finalize.
See the `hearing-live-pipeline` skill.
### hearing.summarize(source, , agent=None, context=None, model=None, \*\*transcribe_kwargs)
Transcribe (if needed) then run an agent to produce meeting notes.
Accepts an audio source or an already-built [`Transcript`](_autosummary/hearing.html.md#hearing.Transcript). Builds the
default agent (Claude if available, else the offline extractive fallback)
when none is injected.
* **Return type:**
[`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]
### hearing.transcribe(source, , engine=None, diarizer=None, agent=None, language=None, split=True, mic_channels=(0,), system_channels=(1,), record=None)
Transcribe an audio source to a [`Transcript`](_autosummary/hearing.types.html.md#hearing.types.Transcript).
The one-liner `transcribe("meeting.wav")` works with all defaults: the
default local engine (faster-whisper), automatic channel splitting when the
file is multi-channel, and the free “me vs them” channel-trick diarizer.
* **Parameters:**
* **source** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str) | [`Path`](https://docs.python.org/3/library/pathlib.html#pathlib.Path) | [`CaptureSource`](_autosummary/hearing.interfaces.html.md#hearing.interfaces.CaptureSource)) – a file path or any [`CaptureSource`](_autosummary/hearing.interfaces.html.md#hearing.interfaces.CaptureSource).
* **engine** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`STTEngine`](_autosummary/hearing.interfaces.html.md#hearing.interfaces.STTEngine)]) – STT engine (default: faster-whisper). Inject any `STTEngine`.
* **diarizer** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`Diarizer`](_autosummary/hearing.interfaces.html.md#hearing.interfaces.Diarizer)]) – speaker labeller. `None` -> the channel trick is applied
automatically when channel info is present (mic=”me”, system=”them”).
Inject a `PyannoteDiarizer` to separate individual remote speakers.
* **agent** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`AgentConsumer`](_autosummary/hearing.interfaces.html.md#hearing.interfaces.AgentConsumer)]) – optional `AgentConsumer` run over the finished transcript
(batch). Its output is stored under `transcript.meta['insight']`.
* **language** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – force a language code (e.g. “en”); `None` = auto-detect.
* **split** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – split mic/system channels when the source is multi-channel.
* **system_channels** ([`Sequence`](https://docs.python.org/3/library/typing.html#typing.Sequence)[[`int`](https://docs.python.org/3/builtins/functions.html#int)]) – column indices for the channel split.
* **record** – optional `MutableMapping` store (e.g. a dol store) to persist
the transcript JSON under key `".transcript.json"`.
* **Return type:**
[`Transcript`](_autosummary/hearing.types.html.md#hearing.types.Transcript)
* **Returns:**
A [`Transcript`](_autosummary/hearing.html.md#hearing.Transcript) (iterable/len-able, so it stands in for a sequence
of segments). Segments carry text, span, channel, and speaker.
### Modules
| [`agents`](_autosummary/hearing.agents.html.md#module-hearing.agents) | The agent / "copilot" layer — the actual point of the hearing project. |
|---------------------------------------------------------------------------------------|----------------------------------------------------------------------------------|
| [`backends`](_autosummary/hearing.backends.html.md#module-hearing.backends) | Extra scribed backends that hearing contributes: Apple-Silicon MLX and WhisperX. |
| [`capture`](_autosummary/hearing.capture.html.md#module-hearing.capture) | Audio capture and channel handling for the hearing pipeline. |
| [`cli`](_autosummary/hearing.cli.html.md#module-hearing.cli) | Command-line interface for hearing, dispatched with `cw`. |
| [`context`](_autosummary/hearing.context.html.md#module-hearing.context) | Context stores and retrieval (RAG) for context-connected agents. |
| [`diarize`](_autosummary/hearing.diarize.html.md#module-hearing.diarize) | Diarization and speaker labelling for the hearing pipeline. |
| [`interfaces`](_autosummary/hearing.interfaces.html.md#module-hearing.interfaces) | Facade Protocols for the four pipeline concerns. |
| [`pipeline`](_autosummary/hearing.pipeline.html.md#module-hearing.pipeline) | Pipeline assembly — compose the four concerns by dependency injection. |
| [`storage`](_autosummary/hearing.storage.html.md#module-hearing.storage) | Transcript persistence via key-value stores (python-storage / `dol`). |
| [`stt`](_autosummary/hearing.stt.html.md#module-hearing.stt) | Speech-to-text for hearing — now backed by the scribed transcription façade. |
| [`types`](_autosummary/hearing.types.html.md#module-hearing.types) | Shared data model for the hearing pipeline. |
| [`vad`](_autosummary/hearing.vad.html.md#module-hearing.vad) | Voice-activity detection and utterance segmentation — re-exported from scribed. |
# _autosummary/hearing.interfaces.html.md
# hearing.interfaces
Facade Protocols for the four pipeline concerns.
Every engine/component is dependency-injected as one of these; the pipeline
depends only on these interfaces, never on a concrete engine. Implementers
satisfy them *structurally* (`typing.Protocol`) — they do not subclass.
This is the contract the `hearing-architecture` skill defines and that every
other `hearing-*` concern implements against.
### Classes
| [`AgentConsumer`](_autosummary/hearing.interfaces.html.md#hearing.interfaces.AgentConsumer)(\*args, \*\*kwargs) | Consumes the (labeled) transcript stream and does things. |
|--------------------------------------------------------------------------------------|--------------------------------------------------------------------|
| [`CaptureSource`](_autosummary/hearing.interfaces.html.md#hearing.interfaces.CaptureSource)(\*args, \*\*kwargs) | Audio source — the ONLY thing that differs between batch and live. |
| [`Diarizer`](_autosummary/hearing.interfaces.html.md#hearing.interfaces.Diarizer)(\*args, \*\*kwargs) | Labels who spoke — enriches segments with `.speaker`. |
| [`STTEngine`](_autosummary/hearing.interfaces.html.md#hearing.interfaces.STTEngine)(\*args, \*\*kwargs) | Speech-to-text. |
### *class* hearing.interfaces.AgentConsumer(\*args, \*\*kwargs)
Bases: [`Protocol`](https://docs.python.org/3/library/typing.html#typing.Protocol)
Consumes the (labeled) transcript stream and does things.
Running notes, suggested questions, RAG over related docs, post-meeting
summaries. Defaults to Claude but stays pluggable (see the `claude-api`
skill for model ids / pricing / tool use).
#### *async* on_segment(segment)
Called per finalized segment (live). Return an optional insight.
* **Return type:**
[`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]
#### *async* on_window(window)
Called on a window/turn or the whole transcript (batch).
Return an optional insight (summary, action items, a suggested
question, a surfaced document, …).
* **Return type:**
[`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]
### *class* hearing.interfaces.CaptureSource(\*args, \*\*kwargs)
Bases: [`Protocol`](https://docs.python.org/3/library/typing.html#typing.Protocol)
Audio source — the ONLY thing that differs between batch and live.
Yields `(channel, frames)` so mic and system audio stay on separate
channels all the way down. A batch implementation reads a file; a live
implementation reads a device/tap.
#### astream()
Live / async: yield `(channel, frames)` blocks until stopped.
* **Return type:**
[`AsyncIterator`](https://docs.python.org/3/library/typing.html#typing.AsyncIterator)[[`tuple`](https://docs.python.org/3/builtins/stdtypes.html#tuple)[[`Channel`](_autosummary/hearing.types.html.md#hearing.types.Channel), [`Any`](https://docs.python.org/3/library/typing.html#typing.Any)]]
#### frames()
Batch / sync: yield `(channel, frames[n_samples])` blocks to EOF.
* **Return type:**
[`Iterator`](https://docs.python.org/3/library/typing.html#typing.Iterator)[[`tuple`](https://docs.python.org/3/builtins/stdtypes.html#tuple)[[`Channel`](_autosummary/hearing.types.html.md#hearing.types.Channel), [`Any`](https://docs.python.org/3/library/typing.html#typing.Any)]]
### *class* hearing.interfaces.Diarizer(\*args, \*\*kwargs)
Bases: [`Protocol`](https://docs.python.org/3/library/typing.html#typing.Protocol)
Labels who spoke — enriches segments with `.speaker`.
May use the audio (pyannote) or just the channel (the channel trick).
Identity resolution (`spk_0` -> `"Alice"`) is an optional further step
behind the same interface.
#### assign_speakers(segments, , audio=None, sample_rate=None)
Yield segments with `.speaker` populated (enrich-by-copy).
* **Return type:**
[`Iterable`](https://docs.python.org/3/library/typing.html#typing.Iterable)[[`TranscriptSegment`](_autosummary/hearing.types.html.md#hearing.types.TranscriptSegment)]
### *class* hearing.interfaces.STTEngine(\*args, \*\*kwargs)
Bases: [`Protocol`](https://docs.python.org/3/library/typing.html#typing.Protocol)
Speech-to-text.
`transcribe` is the batch one-liner; `stream_transcribe` is the live
variant. Both yield [`TranscriptSegment`](_autosummary/hearing.types.html.md#hearing.types.TranscriptSegment) — same shape,
same spine. The rest of the architecture never names a concrete engine.
#### *async* stream_transcribe(frames, , sample_rate)
Live: yield interim then finalized segments as audio arrives.
Mark interim vs final via `meta` (e.g. `meta['final']`). See the
`hearing-live-pipeline` skill — this is the milestone-2 entry point.
* **Return type:**
[`AsyncIterator`](https://docs.python.org/3/library/typing.html#typing.AsyncIterator)[[`TranscriptSegment`](_autosummary/hearing.types.html.md#hearing.types.TranscriptSegment)]
#### transcribe(audio, , sample_rate, language=None)
Batch: whole clip in (float32 mono ndarray), segments out.
* **Return type:**
[`Sequence`](https://docs.python.org/3/library/typing.html#typing.Sequence)[[`TranscriptSegment`](_autosummary/hearing.types.html.md#hearing.types.TranscriptSegment)]
# _autosummary/hearing.pipeline.html.md
# hearing.pipeline
Pipeline assembly — compose the four concerns by dependency injection.
This is where progressive disclosure lives: `transcribe("meeting.wav")` is a
one-liner with smart defaults, while the engine, diarizer, agent, channel
routing, and persistence are all optional keyword-only arguments. The same
shape powers the live path (`live_transcribe`); only the *source* and the
*trigger cadence* differ (milestone 2).
See the `hearing-architecture` skill — this module is the thin orchestrator it
describes; swapping any one component touches only the assembly call.
### Functions
| [`live_transcribe`](_autosummary/hearing.pipeline.html.md#hearing.pipeline.live_transcribe)(\*, source[, engine, ...]) | Stream FINALIZED segments as a meeting unfolds (the live path). |
|--------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| [`summarize`](_autosummary/hearing.pipeline.html.md#hearing.pipeline.summarize)(source, \*[, agent, context, model]) | Transcribe (if needed) then run an agent to produce meeting notes. |
| [`transcribe`](_autosummary/hearing.pipeline.html.md#hearing.pipeline.transcribe)(source, \*[, engine, diarizer, ...]) | Transcribe an audio source to a [`Transcript`](_autosummary/hearing.types.html.md#hearing.types.Transcript). |
### *async* hearing.pipeline.live_transcribe(, source, engine=None, diarizer=None, agent=None, audio_queue_max=50, segment_queue_max=100)
Stream FINALIZED segments as a meeting unfolds (the live path).
Same components as [`transcribe()`](_autosummary/hearing.pipeline.html.md#hearing.pipeline.transcribe) — only the *source* (a streaming
[`CaptureSource`](_autosummary/hearing.interfaces.html.md#hearing.interfaces.CaptureSource) instead of a file) and the trigger
cadence (VAD utterance turn-ends instead of run-once) change. The
`STTEngine` and `AgentConsumer` interfaces are unchanged.
Each channel is demuxed onto its own bounded queue and driven through
`engine.stream_transcribe` as an independent async task, so a slow agent
never stalls capture (backpressure) and the channel “me vs them” label rides
every segment. Diarization defaults to the free channel trick; the agent’s
`on_segment` is fired fire-and-forget. Yields segments as utterances finalize.
See the `hearing-live-pipeline` skill.
### hearing.pipeline.summarize(source, , agent=None, context=None, model=None, \*\*transcribe_kwargs)
Transcribe (if needed) then run an agent to produce meeting notes.
Accepts an audio source or an already-built `Transcript`. Builds the
default agent (Claude if available, else the offline extractive fallback)
when none is injected.
* **Return type:**
[`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]
### hearing.pipeline.transcribe(source, , engine=None, diarizer=None, agent=None, language=None, split=True, mic_channels=(0,), system_channels=(1,), record=None)
Transcribe an audio source to a [`Transcript`](_autosummary/hearing.types.html.md#hearing.types.Transcript).
The one-liner `transcribe("meeting.wav")` works with all defaults: the
default local engine (faster-whisper), automatic channel splitting when the
file is multi-channel, and the free “me vs them” channel-trick diarizer.
* **Parameters:**
* **source** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str) | [`Path`](https://docs.python.org/3/library/pathlib.html#pathlib.Path) | [`CaptureSource`](_autosummary/hearing.interfaces.html.md#hearing.interfaces.CaptureSource)) – a file path or any [`CaptureSource`](_autosummary/hearing.interfaces.html.md#hearing.interfaces.CaptureSource).
* **engine** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`STTEngine`](_autosummary/hearing.interfaces.html.md#hearing.interfaces.STTEngine)]) – STT engine (default: faster-whisper). Inject any `STTEngine`.
* **diarizer** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`Diarizer`](_autosummary/hearing.interfaces.html.md#hearing.interfaces.Diarizer)]) – speaker labeller. `None` -> the channel trick is applied
automatically when channel info is present (mic=”me”, system=”them”).
Inject a `PyannoteDiarizer` to separate individual remote speakers.
* **agent** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`AgentConsumer`](_autosummary/hearing.interfaces.html.md#hearing.interfaces.AgentConsumer)]) – optional `AgentConsumer` run over the finished transcript
(batch). Its output is stored under `transcript.meta['insight']`.
* **language** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – force a language code (e.g. “en”); `None` = auto-detect.
* **split** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – split mic/system channels when the source is multi-channel.
* **system_channels** ([`Sequence`](https://docs.python.org/3/library/typing.html#typing.Sequence)[[`int`](https://docs.python.org/3/builtins/functions.html#int)]) – column indices for the channel split.
* **record** – optional `MutableMapping` store (e.g. a dol store) to persist
the transcript JSON under key `".transcript.json"`.
* **Return type:**
[`Transcript`](_autosummary/hearing.types.html.md#hearing.types.Transcript)
* **Returns:**
A `Transcript` (iterable/len-able, so it stands in for a sequence
of segments). Segments carry text, span, channel, and speaker.
# _autosummary/hearing.storage.html.md
# hearing.storage
Transcript persistence via key-value stores (python-storage / `dol`).
A transcript is standoff annotation data — persist the *whole* transcript (not
just a rolling window) so post-meeting agents see everything (see the
`hearing-agents` and `annotation-systems` skills). [`MeetingStore`](_autosummary/hearing.storage.html.md#hearing.storage.MeetingStore)
saves/loads transcripts as JSON keyed by meeting id, over any
`MutableMapping` — a plain dict (tests), a `dol` store (e.g. `TextFiles`),
or the dependency-free filesystem fallback here.
### Functions
| [`get_store`](_autosummary/hearing.storage.html.md#hearing.storage.get_store)([path]) | Return a string-valued `MutableMapping` for transcripts. |
|---------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------|
| [`transcript_from_json`](_autosummary/hearing.storage.html.md#hearing.storage.transcript_from_json)(s) | Reconstruct a `Transcript` from [`transcript_to_json()`](_autosummary/hearing.storage.html.md#hearing.storage.transcript_to_json) output. |
| [`transcript_to_json`](_autosummary/hearing.storage.html.md#hearing.storage.transcript_to_json)(transcript) | Serialize a `Transcript` to a JSON string (round-trippable). |
### Classes
| [`MeetingStore`](_autosummary/hearing.storage.html.md#hearing.storage.MeetingStore)([store]) | Save/load transcripts keyed by meeting id, over any text MutableMapping. |
|--------------------------------------------------------------------------|----------------------------------------------------------------------------|
### *class* hearing.storage.MeetingStore(store=None)
Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)
Save/load transcripts keyed by meeting id, over any text MutableMapping.
```pycon
>>> store = MeetingStore({}) # in-memory
>>> from hearing.types import Transcript, TranscriptSegment, TimeSpan
>>> _ = store.save("m1", Transcript([TranscriptSegment("hi", TimeSpan(0, 500))]))
>>> store.load("m1").text
'hi'
>>> list(store)
['m1']
```
#### load(meeting_id)
Load the transcript stored under `meeting_id`.
* **Return type:**
[`Transcript`](_autosummary/hearing.types.html.md#hearing.types.Transcript)
#### meeting_ids()
Sorted ids of stored meetings.
* **Return type:**
[`list`](https://docs.python.org/3/builtins/stdtypes.html#list)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]
#### save(meeting_id, transcript)
Persist `transcript` under `meeting_id`; return the storage key.
* **Return type:**
[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)
### hearing.storage.get_store(path=None)
Return a string-valued `MutableMapping` for transcripts.
`None` -> an in-memory dict. A path -> a `dol.TextFiles` store if `dol`
is installed (`pip install 'hearing[storage]'`), else the dependency-free
`_FileTextStore` over that directory.
* **Return type:**
[`MutableMapping`](https://docs.python.org/3/library/collections.abc.html#collections.abc.MutableMapping)
### hearing.storage.transcript_from_json(s)
Reconstruct a `Transcript` from [`transcript_to_json()`](_autosummary/hearing.storage.html.md#hearing.storage.transcript_to_json) output.
* **Return type:**
[`Transcript`](_autosummary/hearing.types.html.md#hearing.types.Transcript)
### hearing.storage.transcript_to_json(transcript)
Serialize a `Transcript` to a JSON string (round-trippable).
* **Return type:**
[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)
# _autosummary/hearing.stt.html.md
# hearing.stt
Speech-to-text for hearing — now backed by the scribed transcription façade.
hearing no longer implements its own STT engines. It consumes **scribed** (one
façade over many engines) and adapts scribed’s normalized `Transcript`/`Segment`
spine to hearing’s [`TranscriptSegment`](_autosummary/hearing.types.html.md#hearing.types.TranscriptSegment) data model. The rest of
the architecture still sees only the [`STTEngine`](_autosummary/hearing.interfaces.html.md#hearing.interfaces.STTEngine) Protocol:
`transcribe(audio) -> segments` and `stream_transcribe(frames) -> segments`.
The default engine is scribed’s faster-whisper backend (local, fast). Swap engines by
changing the scribed `backend=` (e.g. `"openai"`) — nothing downstream changes.
See scribed’s `scribed-choose-backend` skill for the full engine landscape, pricing,
and the local-vs-cloud decision.
Engines by friendly name ([`ENGINE_ALIASES`](_autosummary/hearing.stt.html.md#hearing.stt.ENGINE_ALIASES), used by [`get_engine()`](_autosummary/hearing.stt.html.md#hearing.stt.get_engine) and the
CLI’s `--engine`):
- `whisper` — faster-whisper, local, any CPU/GPU (default; `hearing[whisper]`)
- `openai` — OpenAI transcription API (`hearing[openai]`, `OPENAI_API_KEY`)
- `deepgram` — Deepgram Nova-3 (`hearing[deepgram]`, `DEEPGRAM_API_KEY`)
- `mlx` / `parakeet` — Whisper / Parakeet-TDT on Apple Silicon (`hearing[mlx]`)
- `whisperx` — WhisperX, aligned words + optional diarization (`hearing[whisperx]`)
The last three are scribed backends that hearing itself contributes
([`hearing.backends`](_autosummary/hearing.backends.html.md#module-hearing.backends)); they are registered with scribed on first use.
### Module Attributes
| [`DEFAULT_BACKEND`](_autosummary/hearing.stt.html.md#hearing.stt.DEFAULT_BACKEND) | Default scribed backend id for local transcription. |
|-------------------------------------------------------------------------|---------------------------------------------------------------------------------|
| [`ENGINE_ALIASES`](_autosummary/hearing.stt.html.md#hearing.stt.ENGINE_ALIASES) | Friendly engine name -> scribed backend id (the SSOT for `get_engine`/the CLI). |
| [`WHISPER_SIZED_BACKENDS`](_autosummary/hearing.stt.html.md#hearing.stt.WHISPER_SIZED_BACKENDS) | Backends whose `model` accepts a bare whisper size ("base", "small", ...). |
| [`ENGINE_NAMES`](_autosummary/hearing.stt.html.md#hearing.stt.ENGINE_NAMES) | Engine names shown in help and error messages (one per backend). |
### Functions
| [`DeepgramSTT`](_autosummary/hearing.stt.html.md#hearing.stt.DeepgramSTT)(\*[, model]) | Deepgram (cloud) engine — Nova-3 by default; `model="nova-2"` etc. |
|-------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------|
| [`FasterWhisperSTT`](_autosummary/hearing.stt.html.md#hearing.stt.FasterWhisperSTT)(\*[, model_size]) | Backward-compatible constructor for the local engine (returns a `ScribedSTT`). |
| [`MLXSTT`](_autosummary/hearing.stt.html.md#hearing.stt.MLXSTT)(\*[, model, mlx_backend]) | Apple-Silicon (MLX) engine: mlx-whisper by default, or `mlx_backend="parakeet-mlx"`. |
| [`OpenAISTT`](_autosummary/hearing.stt.html.md#hearing.stt.OpenAISTT)(\*\*kwargs) | Backward-compatible constructor for the OpenAI engine (returns a `ScribedSTT`). |
| [`WhisperXSTT`](_autosummary/hearing.stt.html.md#hearing.stt.WhisperXSTT)(\*[, model, diarize]) | WhisperX engine: aligned word timestamps, plus pyannote speakers if `diarize`. |
| [`default_engine`](_autosummary/hearing.stt.html.md#hearing.stt.default_engine)(\*\*kwargs) | Return the project's default STT engine (scribed faster-whisper). |
| [`get_engine`](_autosummary/hearing.stt.html.md#hearing.stt.get_engine)([name]) | Build a scribed-backed STT engine by friendly name (see [`ENGINE_ALIASES`](_autosummary/hearing.stt.html.md#hearing.stt.ENGINE_ALIASES)). |
### Classes
| [`ScribedSTT`](_autosummary/hearing.stt.html.md#hearing.stt.ScribedSTT)([backend, model_size, language, ...]) | An `STTEngine` backed by scribed — the single engine hearing now uses. |
|-----------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------|
### hearing.stt.DEFAULT_BACKEND *: [str](https://docs.python.org/3/builtins/stdtypes.html#str)* *= 'faster-whisper'*
Default scribed backend id for local transcription.
### hearing.stt.DeepgramSTT(, model=None, \*\*kwargs)
Deepgram (cloud) engine — Nova-3 by default; `model="nova-2"` etc. swaps it.
Batch goes to Deepgram’s pre-recorded REST API (via scribed’s deepgram backend);
the live path is scribed’s VAD-segmented stream over that batch call. Needs
`pip install 'hearing[deepgram]'` and `DEEPGRAM_API_KEY`.
* **Return type:**
[`ScribedSTT`](_autosummary/hearing.stt.html.md#hearing.stt.ScribedSTT)
### hearing.stt.ENGINE_ALIASES *: [Dict](https://docs.python.org/3/library/typing.html#typing.Dict)[[str](https://docs.python.org/3/builtins/stdtypes.html#str), [str](https://docs.python.org/3/builtins/stdtypes.html#str)]* *= {'cloud': 'openai', 'deepgram': 'deepgram', 'faster-whisper': 'faster-whisper', 'local': 'faster-whisper', 'mlx': 'mlx-whisper', 'mlx-whisper': 'mlx-whisper', 'nova': 'deepgram', 'openai': 'openai', 'parakeet': 'parakeet-mlx', 'parakeet-mlx': 'parakeet-mlx', 'whisper': 'faster-whisper', 'whisperx': 'whisperx'}*
Friendly engine name -> scribed backend id (the SSOT for `get_engine`/the CLI).
Any other registered scribed backend id is accepted as-is.
### hearing.stt.ENGINE_NAMES *= ('whisper', 'openai', 'deepgram', 'mlx', 'parakeet', 'whisperx')*
Engine names shown in help and error messages (one per backend).
### hearing.stt.FasterWhisperSTT(, model_size=None, \*\*kwargs)
Backward-compatible constructor for the local engine (returns a `ScribedSTT`).
Retained so existing call sites (`FasterWhisperSTT(model_size="tiny")`) keep
working now that the engines are centralized in scribed.
* **Return type:**
[`ScribedSTT`](_autosummary/hearing.stt.html.md#hearing.stt.ScribedSTT)
### hearing.stt.MLXSTT(, model=None, mlx_backend='mlx-whisper', \*\*kwargs)
Apple-Silicon (MLX) engine: mlx-whisper by default, or `mlx_backend="parakeet-mlx"`.
`model` is a Hugging Face repo id or (mlx-whisper) a bare Whisper size such as
`"small"`. Needs macOS on Apple Silicon and `pip install 'hearing[mlx]'`.
* **Return type:**
[`ScribedSTT`](_autosummary/hearing.stt.html.md#hearing.stt.ScribedSTT)
### hearing.stt.OpenAISTT(\*\*kwargs)
Backward-compatible constructor for the OpenAI engine (returns a `ScribedSTT`).
* **Return type:**
[`ScribedSTT`](_autosummary/hearing.stt.html.md#hearing.stt.ScribedSTT)
### *class* hearing.stt.ScribedSTT(backend='faster-whisper', model_size=None, language=None, vad=None, options=)
Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)
An `STTEngine` backed by scribed — the single engine hearing now uses.
* **Parameters:**
* **backend** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – scribed backend id (`"faster-whisper"` default, `"openai"`, …).
* **model_size** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – optional model override forwarded to the backend (e.g. `"tiny"`).
* **language** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – force a language code; `None` => auto-detect.
* **vad** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`VAD`](_autosummary/hearing.vad.html.md#hearing.vad.VAD)]) – utterance segmenter for `stream_transcribe` (default: scribed EnergyVAD).
* **options** ([`Mapping`](https://docs.python.org/3/library/typing.html#typing.Mapping)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str), [`Any`](https://docs.python.org/3/library/typing.html#typing.Any)]) – extra backend kwargs forwarded on every call (e.g.
`{"diarize": True}` for WhisperX/Deepgram); see the backend’s
`param_map` for what it accepts.
#### *async* stream_transcribe(frames, , sample_rate)
Stream finalized segments as utterances complete (via scribed’s live path).
* **Return type:**
[`AsyncIterator`](https://docs.python.org/3/library/typing.html#typing.AsyncIterator)[[`TranscriptSegment`](_autosummary/hearing.types.html.md#hearing.types.TranscriptSegment)]
#### transcribe(audio, , sample_rate, language=None)
Transcribe a whole mono clip via scribed; return ordered hearing segments.
* **Return type:**
[`Sequence`](https://docs.python.org/3/library/typing.html#typing.Sequence)[[`TranscriptSegment`](_autosummary/hearing.types.html.md#hearing.types.TranscriptSegment)]
### hearing.stt.WHISPER_SIZED_BACKENDS *= frozenset({'faster-whisper', 'mlx-whisper', 'whisperx'})*
Backends whose `model` accepts a bare whisper size (“base”, “small”, …).
### hearing.stt.WhisperXSTT(, model=None, diarize=False, \*\*kwargs)
WhisperX engine: aligned word timestamps, plus pyannote speakers if `diarize`.
Needs `pip install 'hearing[whisperx]'`; `diarize=True` also needs a
Hugging Face token in `HF_TOKEN`.
* **Return type:**
[`ScribedSTT`](_autosummary/hearing.stt.html.md#hearing.stt.ScribedSTT)
### hearing.stt.default_engine(\*\*kwargs)
Return the project’s default STT engine (scribed faster-whisper).
Tests and quick runs typically pass `model_size="tiny"`.
* **Return type:**
[`ScribedSTT`](_autosummary/hearing.stt.html.md#hearing.stt.ScribedSTT)
### hearing.stt.get_engine(name='whisper', \*\*kwargs)
Build a scribed-backed STT engine by friendly name (see [`ENGINE_ALIASES`](_autosummary/hearing.stt.html.md#hearing.stt.ENGINE_ALIASES)).
`"whisper"` -> faster-whisper, `"openai"`, `"deepgram"`, `"mlx"` ->
mlx-whisper, `"parakeet"` -> parakeet-mlx, `"whisperx"`. Any other
registered scribed backend id works too. `kwargs` go to [`ScribedSTT`](_autosummary/hearing.stt.html.md#hearing.stt.ScribedSTT).
* **Return type:**
[`ScribedSTT`](_autosummary/hearing.stt.html.md#hearing.stt.ScribedSTT)
```pycon
>>> get_engine("mlx").backend
'mlx-whisper'
```
# _autosummary/hearing.types.html.md
# hearing.types
Shared data model for the hearing pipeline.
This is the SSOT data spine that every concern (capture, STT, diarization,
agents) speaks. A transcript is *standoff interval annotation* over the audio:
segments reference the audio by time (a [`TimeSpan`](_autosummary/hearing.types.html.md#hearing.types.TimeSpan)) and never carry the
samples themselves. Time is integer milliseconds — never bare float seconds —
so it is accumulation-safe, hashable, and round-trippable.
See the `hearing-architecture` skill for the rationale behind every choice
here, and the `annotation-systems` skill for the standoff-annotation theory.
### Functions
| [`merge_segments`](_autosummary/hearing.types.html.md#hearing.types.merge_segments)(\*segment_groups) | Merge several segment sequences into one list ordered by start time. |
|-------------------------------------------------------------------------------------|------------------------------------------------------------------------|
### Classes
| [`Channel`](_autosummary/hearing.types.html.md#hearing.types.Channel)(\*values) | Which capture channel a segment came from. |
|------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------|
| [`TimeSpan`](_autosummary/hearing.types.html.md#hearing.types.TimeSpan)(start_ms, end_ms) | A half-open interval `[start_ms, end_ms)` in integer milliseconds. |
| [`Transcript`](_autosummary/hearing.types.html.md#hearing.types.Transcript)([segments, sample_rate, meta]) | A finished transcript: an ordered collection of segments plus metadata. |
| [`TranscriptSegment`](_autosummary/hearing.types.html.md#hearing.types.TranscriptSegment)(text, span[, channel, ...]) | The data spine — a standoff interval annotation over the audio. |
| [`Word`](_autosummary/hearing.types.html.md#hearing.types.Word)(text, span[, confidence]) | Optional word-level timing (WhisperX / `word_timestamps=True`). |
### *class* hearing.types.Channel(\*values)
Bases: [`str`](https://docs.python.org/3/builtins/stdtypes.html#str), [`Enum`](https://docs.python.org/3/library/enum.html#enum.Enum)
Which capture channel a segment came from.
The channel *is* the “me vs them” signal, for free: `MIC` is the local
user, `SYSTEM` is everyone else (audio that came out of the speakers).
`MIXED` is a single-channel / unknown source.
### *class* hearing.types.TimeSpan(start_ms, end_ms)
Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)
A half-open interval `[start_ms, end_ms)` in integer milliseconds.
Integer time, never float seconds — accumulation-safe and hashable.
```pycon
>>> TimeSpan(0, 1500).duration_ms
1500
```
#### *property* duration_ms *: [int](https://docs.python.org/3/builtins/functions.html#int)*
Length of the interval in milliseconds.
#### *classmethod* from_seconds(start, end)
Build a span from float seconds (e.g. from an STT engine).
* **Return type:**
[`TimeSpan`](_autosummary/hearing.types.html.md#hearing.types.TimeSpan)
### *class* hearing.types.Transcript(segments=(), sample_rate=None, meta=)
Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)
A finished transcript: an ordered collection of segments plus metadata.
Iterable and `len`-able, so it stands in for a `Sequence` of segments
wherever the architecture’s facades return “segments”. Adds convenience
accessors the CLI/agents need.
```pycon
>>> t = Transcript((TranscriptSegment("a", TimeSpan(0, 1000), speaker=ME),
... TranscriptSegment("b", TimeSpan(1000, 2000), speaker=THEM)))
>>> t.text
'a b'
>>> len(t)
2
>>> sorted(t.speakers)
['me', 'them']
```
#### *property* duration_ms *: [int](https://docs.python.org/3/builtins/functions.html#int)*
End of the last segment, or 0 if empty.
#### formatted(, speakers=True, timestamps=True)
Render as a human-readable transcript, one line per segment.
* **Return type:**
[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)
#### *property* speakers *: [set](https://docs.python.org/3/builtins/stdtypes.html#set)[[str](https://docs.python.org/3/builtins/stdtypes.html#str)]*
The distinct speaker labels present (ignoring `None`).
#### *property* text *: [str](https://docs.python.org/3/builtins/stdtypes.html#str)*
All segment texts joined by a single space.
#### to_jsonable()
A plain `list[dict]` ready for `json.dumps` (segments only).
* **Return type:**
[`list`](https://docs.python.org/3/builtins/stdtypes.html#list)[[`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict)]
#### with_meta(\*\*kw)
Return a copy with extra metadata merged in.
* **Return type:**
[`Transcript`](_autosummary/hearing.types.html.md#hearing.types.Transcript)
### *class* hearing.types.TranscriptSegment(text, span, channel=Channel.MIXED, speaker=None, confidence=None, words=(), meta=)
Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)
The data spine — a standoff interval annotation over the audio.
Required fields are `text` and `span`. Everything else is optional
metadata that a concern *enriches*: STT sets `text`/`span`/
`confidence`; capture sets `channel`; diarization sets `speaker`.
Segments are frozen — enrich by copying (`with_speaker`/`with_channel`),
never mutate.
```pycon
>>> seg = TranscriptSegment("hello", TimeSpan(0, 500))
>>> seg.with_speaker(ME).speaker
'me'
>>> seg.text
'hello'
```
#### with_channel(channel)
Return a copy carrying a channel label (frozen -> copy, don’t mutate).
* **Return type:**
[`TranscriptSegment`](_autosummary/hearing.types.html.md#hearing.types.TranscriptSegment)
#### with_speaker(speaker)
Return a copy carrying a speaker label (frozen -> copy, don’t mutate).
* **Return type:**
[`TranscriptSegment`](_autosummary/hearing.types.html.md#hearing.types.TranscriptSegment)
### *class* hearing.types.Word(text, span, confidence=None)
Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)
Optional word-level timing (WhisperX / `word_timestamps=True`).
Lets an agent trigger on a keyword mid-utterance instead of waiting for the
end of a turn. Purely optional metadata on a [`TranscriptSegment`](_autosummary/hearing.types.html.md#hearing.types.TranscriptSegment).
### hearing.types.merge_segments(\*segment_groups)
Merge several segment sequences into one list ordered by start time.
Used when transcribing the mic and system channels separately and then
interleaving them back into a single chronological transcript.
* **Return type:**
[`list`](https://docs.python.org/3/builtins/stdtypes.html#list)[[`TranscriptSegment`](_autosummary/hearing.types.html.md#hearing.types.TranscriptSegment)]
# _autosummary/hearing.vad.html.md
# hearing.vad
Voice-activity detection and utterance segmentation — re-exported from scribed.
The VAD logic (energy/neural detectors + utterance segmentation) is now owned by
`scribed.vad` so it can be shared across every scribed consumer. hearing keeps
this module as the stable import path; the names below are scribed’s.
See the `hearing-live-pipeline` skill for the rationale and the latency budget.
### Functions
| [`segment_utterances`](_autosummary/hearing.vad.html.md#hearing.vad.segment_utterances)(frames, \*, sample_rate[, ...]) | Group an async stream of audio blocks into utterances. |
|-------------------------------------------------------------------------------------------------------|----------------------------------------------------------|
### Classes
| [`VAD`](_autosummary/hearing.vad.html.md#hearing.vad.VAD)(\*args, \*\*kwargs) | Decides whether a block of audio contains speech. |
|-------------------------------------------------------------------------------------|---------------------------------------------------------|
| [`EnergyVAD`](_autosummary/hearing.vad.html.md#hearing.vad.EnergyVAD)([threshold]) | RMS-energy threshold VAD — the dependency-free default. |
| [`SileroVAD`](_autosummary/hearing.vad.html.md#hearing.vad.SileroVAD)([speech_prob_threshold]) | Neural VAD via silero-vad (optional, lazy). |
### *class* hearing.vad.EnergyVAD(threshold=0.01)
Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)
RMS-energy threshold VAD — the dependency-free default.
A block counts as speech when its RMS energy exceeds `threshold` (in the same
units as the float audio, roughly `[-1, 1]`). Simple and deterministic; for
noisy real-world audio prefer [`SileroVAD`](_autosummary/hearing.vad.html.md#hearing.vad.SileroVAD).
#### is_speech(block, sample_rate)
True if the block’s RMS energy is above `threshold`.
* **Return type:**
[`bool`](https://docs.python.org/3/builtins/functions.html#bool)
### *class* hearing.vad.SileroVAD(speech_prob_threshold=0.5)
Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object)
Neural VAD via silero-vad (optional, lazy). Accurate on noisy audio.
Requires `pip install silero-vad` (or torch.hub). Raises an informative error
if unavailable; [`EnergyVAD`](_autosummary/hearing.vad.html.md#hearing.vad.EnergyVAD) needs no dependencies.
#### is_speech(block, sample_rate)
True if silero’s speech probability for the block exceeds the threshold.
* **Return type:**
[`bool`](https://docs.python.org/3/builtins/functions.html#bool)
### *class* hearing.vad.VAD(\*args, \*\*kwargs)
Bases: [`Protocol`](https://docs.python.org/3/library/typing.html#typing.Protocol)
Decides whether a block of audio contains speech.
#### is_speech(block, sample_rate)
True if `block` (mono float32) contains speech.
* **Return type:**
[`bool`](https://docs.python.org/3/builtins/functions.html#bool)
### *async* hearing.vad.segment_utterances(frames, , sample_rate, vad=None, silence_ms=700, min_speech_ms=200, max_utterance_ms=30000)
Group an async stream of audio blocks into utterances.
Yields `(utterance_samples, start_ms)` once per finalized utterance: speech
accumulates into a buffer; after `silence_ms` of trailing silence (or
`max_utterance_ms` of continuous speech) the buffer is emitted with its start
offset (integer ms from the beginning of the stream). Utterances with less than
`min_speech_ms` of actual speech are dropped as noise. `sample_rate` is the
rate of the incoming blocks (used only for the ms arithmetic).
* **Return type:**
[`AsyncIterator`](https://docs.python.org/3/library/typing.html#typing.AsyncIterator)[[`Tuple`](https://docs.python.org/3/library/typing.html#typing.Tuple)[`ndarray`, [`int`](https://docs.python.org/3/builtins/functions.html#int)]]
# about-this-build.html.md
# About this build
This documentation was built on **2026-10-14 05:27 UTC** from commit e6ef523 on branch main, for **hearing 0.0.7** (from pyproject.toml).
#### NOTE
Nothing suggests a mismatch: the tree was clean at the commit above, and the documented version is the one on PyPI.
## Source
| | |
|---------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------|
| Commit | e6ef5233da0d2217bf626bdba9d05c3831b07e0a |
| Branch | main |
| Tags at this commit | none |
| Working tree | clean |
| Remote | https://github.com/thorwhalen/hearing |
## Continuous integration
| | |
|--------------|--------------------------------------------------------------------------------------------|
| Repository | thorwhalen/hearing |
| Run | 38516604881 |
| Ref | refs/heads/main |
| Event commit | e6ef5233da0d2217bf626bdba9d05c3831b07e0a (in the history of the built commit) |
## Tools
| | |
|----------|---------|
| epythet | 0.2.17 |
| Sphinx | 9.1.0 |
| docutils | 0.22.4 |
| Python | 3.12.15 |
## Configuration as resolved
| | |
|---------------|------------------------------------------------------------------|
| theme | auto (Sphinx theme furo) |
| accent | #4d4b9e |
| api_generator | autosummary |
| ignore | tests/, scrap/, examples/ |
| agent_outputs | true |
| aggregates | md |
| ai_artifacts | true |
## Package on PyPI
Latest release: 0.0.7, the same as the documented version.
## Reproduce
```bash
git clone https://github.com/thorwhalen/hearing && cd hearing
git checkout e6ef5233da0d2217bf626bdba9d05c3831b07e0a
pip install "epythet==0.2.17"
epythet quickstart . --ignore tests/ scrap/ examples/
```
The same data, for machines: build_info.json (schema version 1).
# ai-agents.html.md
# For AI agents
`hearing` ships artifacts for coding agents alongside its code. This page lists
them, says where each lives in the repository, and points at the
machine-readable copies of this documentation.
## Skills
Skills are folders holding a `SKILL.md` (the [Agent Skills](https://agentskills.io) format): a description that tells an agent when to use it and a body with the procedure. Install one into your agent with `gh skill` (any host: `--agent claude-code`, `copilot`, `cursor`, `codex`, `gemini`), or use the copy bundled in the wheel.
### `hearing`
The project-manager / orchestrator ENTRY POINT for the “hearing” project (thorwhalen/hearing — the pluggable macOS meeting-transcription + context-connected AI-agent library). Owns the vision, the milestone roadmap (batch first, live additive), the GitHub-as-memory workflow, and project status, and it ROUTES every task to the right hearing-\* sub-skill or global skill. Triggers on PM / orchestration / routing questions: “which skill owns this”, “which part do I work on”, “route this task”, “how does the hearing project fit together”, “what should I build next / where do I start”, the roadmap / milestones / batch-vs-live ordering, “project status / where things stand”, scaffold the package / project structure, set up the repo’s issues / discussions / ROADMAP, and any top-level “work on the hearing project” framing. For DEEP design contracts — the four swappable layers wiring, the shared segment data model, Protocol seams, batch/live unification — route to [[hearing-architecture]] instead. Routes to the hearing-\* sub-skills (hearing-architecture, hearing-audio-capture, hearing-stt, hearing-diarization, hearing-agents, hearing-live-pipeline, hearing-frontend) and to the relevant global skills rather than duplicating them.
Source: [`.claude/skills/hearing`](https://github.com/thorwhalen/hearing/tree/HEAD/.claude/skills/hearing).
### `hearing-agents`
Use when building the agent / “copilot” layer of the hearing meeting-transcription project — the thing that consumes a transcript and produces feedback (summaries, action items, decisions, follow-up research in batch; running notes, suggested questions, relevant docs, fact-checks live). Triggers on agent layer, meeting copilot, post-meeting summary, action items, decision extraction, follow-up research, live notes, suggested questions, real-time RAG, fact-check, queue consumer, asyncio.Queue of utterances, finalized segments, FrameProcessor, Pipecat, LiveKit Agents, RealTimeAgentPipeline, non-blocking transcript intercept, backpressure / decoupling the LLM from capture, feeding me/them + diarization labels into the prompt, vector store over meeting context, LlamaIndex, Qdrant, web search during a meeting, deep-research follow-up, SimpleLLMAgent, pluggable LLM backend, batch-vs-live unification. The agent defaults to Claude but stays pluggable.
Source: [`.claude/skills/hearing-agents`](https://github.com/thorwhalen/hearing/tree/HEAD/.claude/skills/hearing-agents).
### `hearing-architecture`
Use when designing, wiring, or modifying the hearing project’s pipeline architecture — the SSOT for HOW the four layers (audio capture, STT, diarization/speaker-ID, agents) compose by dependency injection. Triggers on capture source, channel split, mic vs system audio, STT engine facade, transcribe(audio)->segments, streaming transcription, diarization interface, speaker label, TranscriptSegment, agent consumer, pipeline assembly, batch vs live, recipe-style composition, swap/substitute a component, Protocol/dataclass contracts, asyncio.Queue between stages, backpressure/decoupling, progressive disclosure of the pipeline, and any “how do the hearing-\* skills fit together” or “what’s the shared data model” question. This is the contract every other hearing-\* skill (hearing-audio-capture, hearing-stt, hearing-diarization, hearing-agents) implements against.
Source: [`.claude/skills/hearing-architecture`](https://github.com/thorwhalen/hearing/tree/HEAD/.claude/skills/hearing-architecture).
### `hearing-audio-capture`
Use when capturing audio for meeting transcription on macOS (and the iOS limits) — getting BOTH the microphone AND system audio (the other participants coming out of your speakers) into a Python pipeline. Triggers on system audio capture, loopback, BlackHole, Aggregate Device, Multi-Output Device, Audio MIDI Setup, Core Audio taps, AudioHardwareCreateProcessTap, AudioTee, ScreenCaptureKit, SCStream, PyObjC SCK -3805, sounddevice InputStream, soundcard, PyAudio, ffmpeg avfoundation, channel slicing / mapping, “record what I hear”, “capture the meeting audio”, separate mic vs system channels, the “me vs them” channel trick, virtual audio driver, sample-rate / clock / drift mismatch, screen-recording permission for audio, and iOS/iPhone call/app audio recording restrictions (ReplayKit, iOS 18.1 call recording).
Source: [`.claude/skills/hearing-audio-capture`](https://github.com/thorwhalen/hearing/tree/HEAD/.claude/skills/hearing-audio-capture).
### `hearing-diarization`
Use when working on speaker attribution in the “hearing” meeting-transcription library — turning a transcript into a who-spoke-when, speaker-labeled (diarized) transcript, separating remote participants, and deciding which library to run. Triggers on diarization, speaker diarization, who spoke when, speaker labels, Speaker 1/Speaker 2, speaker turns, speaker segmentation, speaker attribution, “me vs them”, local vs remote speaker, channel energy / channel trick, dual VAD, speaker identification, voice fingerprinting, voice enrollment, speaker embedding, pyannote.audio, pyannote community-1, speaker-diarization-3.1, NVIDIA NeMo Sortformer/MSDD, WhisperX diarize, SpeechBrain ECAPA/x-vector, DER (diarization error rate), or feeding speaker labels into the agent prompt. Distinguish from raw transcription (that is hearing-stt) and from audio capture/channel layout (that is hearing-audio-capture).
Source: [`.claude/skills/hearing-diarization`](https://github.com/thorwhalen/hearing/tree/HEAD/.claude/skills/hearing-diarization).
### `hearing-frontend`
Use when building or designing the TypeScript/React frontend for the “hearing” meeting-transcription project — the transcript viewer, the live AI-copilot overlay, the agent-feedback panel (notes, suggested questions, surfaced docs, fact-checks), or the meeting/project-context UI. Covers schema-driven UI with zodal (defineCollection, Zod v4, content-metadata bifurcation routing transcript metadata to a DB and audio/recording content to object storage), command-dispatch with acture (one command → palette + hotkeys + AI tool + MCP + tests), how the Python backend (argh CLI + HTTP layer) feeds the UI, and how live finalized utterances stream to the client. Triggers on transcript view, copilot overlay, agent panel, feedback panel, suggested questions, surfaced docs, fact-check, meeting collection, segment schema, speaker/me-vs-them labels, zodal collection, acture command, bifurcation, lk.transcription, SSE/WebSocket transcript stream, novice-friendly frontend, declarative/schema UI for hearing. Explain JS/TS decisions for a frontend novice.
Source: [`.claude/skills/hearing-frontend`](https://github.com/thorwhalen/hearing/tree/HEAD/.claude/skills/hearing-frontend).
### `hearing-live-pipeline`
Use when building the streaming / live / real-time milestone of the hearing meeting-transcription library — turning the batch (record-then-transcribe) path into a low-latency loop that emits transcript segments as people speak. Triggers on streaming STT, real-time transcription, live captions, RealtimeSTT, whisper_streaming, SimulStreaming, WhisperLive, OpenAI Realtime API (gpt-realtime-whisper), Pipecat, LiveKit Agents, VAD / voice activity detection, silero-vad, webrtcvad, utterance / turn finalization, partial vs final segments, word-level timestamps for keyword triggers, feed_audio, asyncio.Queue decoupling, backpressure between capture/STT/diarization/agent stages, latency budget (VAD + STT + LLM TTFT + tool/RAG + transport), semantic turn detection, mid-sentence pause handling, and any task that adds a streaming capture/sink without changing the STT or agent interfaces.
Source: [`.claude/skills/hearing-live-pipeline`](https://github.com/thorwhalen/hearing/tree/HEAD/.claude/skills/hearing-live-pipeline).
### `hearing-stt`
Use when choosing, wiring, swapping, or benchmarking a speech-to-text (STT/ASR) engine behind the hearing project’s transcription facade — the pluggable layer that turns audio into segments so the rest of the architecture never names a specific engine. Triggers on transcribe(audio)->segments, ASR engine selection, Whisper / whisper.cpp / faster-whisper / WhisperX / distil-whisper, NVIDIA Parakeet-TDT / Canary, Moonshine, Apple-Silicon MLX paths (lightning-whisper-mlx, mlx-audio, Lightning-SimulWhisper), cloud STT APIs (OpenAI gpt-4o-transcribe / -mini / gpt-realtime-whisper / gpt-4o-transcribe-diarize, Deepgram Nova-3, AssemblyAI Universal, Google Chirp, Azure Speech, Speechmatics, Rev), per-minute STT pricing, local-vs-cloud / on-device decision, WER tolerance, concurrency, self-hosting break-even (~100k min/month), CTranslate2 / int8 quantization, model-name versioning and the ~June-2026 OpenAI transcribe retirements, and “which engine should I use for meeting transcription”. Diarization (who-spoke-when) is a SEPARATE concern — see hearing-diarization.
Source: [`.claude/skills/hearing-stt`](https://github.com/thorwhalen/hearing/tree/HEAD/.claude/skills/hearing-stt).
## Machine-readable documentation
This site publishes the same documentation in forms that fit an agent’s context window:
- [`llms.txt`](https://thorwhalen.github.io/hearing/llms.txt): an index of every page with a one-line description ([llms.txt](https://llmstxt.org) format)
- [`hearing.md`](https://thorwhalen.github.io/hearing/hearing.md): the whole documentation as one Markdown file
- `.html.md`: a rendered Markdown twin of every page, advertised from each page’s `` with ``
- [`objects.inv`](https://thorwhalen.github.io/hearing/objects.inv): the Sphinx inventory: a symbol-to-URL index (`sphobjinv convert plain objects.inv -`)
# api.html.md
# API reference
| [`hearing`](_autosummary/hearing.html.md#module-hearing) | hearing — pluggable meeting transcription & context-aware AI agents. |
|---------------------------------------------------------------------------|------------------------------------------------------------------------|