foley.sources.base

The bulk-corpus source contract — a downloaded corpus as an ingestable stream.

foley’s SOURCE stage has two shapes. This module defines the bulk-corpus shape used by foley.bootstrap.bootstrap() to seed the library from corpora the user has already downloaded to local disk (FSD50K, Clotho, FoleySet, …). It is deliberately narrower than the live/HTTP SourceAdapter (search / get / download / generate) that subtask #5 introduces: a bulk corpus only has to enumerate its clips and say what each clip’s license is. Everything else — decode, QC, tag, embed, dedup, the by-value/by-reference storage gate — is reused verbatim from foley.index.ingest.ingest_one(); adapters never touch the library.

The two responsibilities of an adapter:

CorpusAdapter.iter_clips() — yield a ClipSpec per audio file,

CorpusAdapter.resolve_license() — map a clip’s corpus metadata to a fully-derived LicenseRecord (via the licensing SSOT, never hand-set flags), setting rights_verified True only for an authoritatively-recognized license (fail-closed otherwise).

Concrete adapters register themselves in CORPUS_REGISTRY via register_corpus(); corpora_in_rings() / select_corpora() drive the ring policy in the bootstrap orchestrator. The registry is a plain dict — no auto-discovery / SOURCE_CONFIG (that is #5’s concern).

This module is also the home of the live-source contracts (auto-discovered by foley.sources.registry, not registered here): the retrieve SourceAdapter (search / get / download — Freesound, #5) and its sibling generate GenerateAdapter (generate → GeneratedClip — Stable Audio Open, ElevenLabs, #6). All three adapter kinds share the license SSOT builders — bulk_license() (bulk), api_license() (retrieve), and generated_license() (generate) — thin wrappers over _build_license() so the derived permission flags stay single-sourced.

foley.sources.base.CORPUS_REGISTRY: dict[str, CorpusAdapter] = {'bbc_remarc': UniformCorpus(name='bbc_remarc', ring=2, default_license_id='RemArc', source='bbc_remarc', rights_verified=True, tag_hints_from_path=False), 'clotho': ClothoEvalCorpus(name='clotho', ring=0, default_license_id='CC-BY-4.0', source='clotho', rights_verified=True, tag_hints_from_path=False), 'foleyset': UniformCorpus(name='foleyset', ring=0, default_license_id='CC-BY-4.0', source='foleyset', rights_verified=True, tag_hints_from_path=True), 'fsd50k': <foley.sources.fsd50k.Fsd50kCorpus object>, 'sonniss': UniformCorpus(name='sonniss', ring=2, default_license_id='Sonniss-GDC', source='sonniss', rights_verified=True, tag_hints_from_path=False)}: Registry of concrete bulk-corpus adapters, keyed by adapter.name.

class foley.sources.base.ClipSpec(path: str, source_id: str, meta: dict = <factory>)[source]

One clip inside a local bulk corpus, described but not yet ingested.

path

Absolute path to the audio file on disk.

Type:: str

source_id

The corpus-native id (e.g. an FSD50K fname); used only for reporting/provenance — the stored SoundRecord.id is still the content hash minted by the ingest pipeline.

Type:: str

meta

Free-form per-clip metadata the adapter carries to CorpusAdapter.resolve_license() and (optionally) to caption/tag hints — e.g. {"license_id", "creator_name", "source_url", "caption", "tag_hints"}.

Type:: dict

class foley.sources.base.CorpusAdapter(*args, **kwargs)[source]

A downloaded bulk corpus presented as an ingestable stream of clips.

Implementations are plain objects (usually a small @dataclass) carrying three class-level facts (name / ring / default_license_id) and the two methods below. They perform no embedding, storage, or library access.

corpus_dir(data_dir: str) → str[source]: The on-disk root for this corpus under data_dir (data_dir/name).

default_license_id: str: The corpus compilation license id (a per-clip license may still override).

iter_clips(root: str) → Iterator[ClipSpec][source]: Yield a ClipSpec for every ingestable clip under root.

name: str: Registry key / CLI name — 'fsd50k' | 'clotho' | 'foleyset' | …

resolve_license(spec: ClipSpec) → LicenseRecord[source]: Return the fully-derived rights record for spec (licensing SSOT).

ring: int

0 ship-in-repo, 1 fetch, 2 opt-in/quarantined.

Type:: Bootstrap ring

class foley.sources.base.GenerateAdapter(*args, **kwargs)[source]

The generation source contract (report 10 §4.2) — a SIBLING of SourceAdapter.

A generate adapter (Stable Audio Open local, ElevenLabs Sound Effects hosted; #6) synthesizes a sound from a prompt. It is a deliberate sibling of the retrieve SourceAdapter rather than an extra method on it, so that SourceAdapter stays runtime_checkable for the retrieve trio (search / get / download) alone.

Like a retrieve adapter, a generate adapter performs no storage or library access: it maps foley’s unified GENERATION_AFFORDANCES (prompt, duration, prompt_influence, negative_prompt, steps, seed, loop, output_format) to its backend’s native params (via SOURCE_CONFIG['param_map'], warning-and-dropping unsupported ones), builds the audio + a generated LicenseRecord, and returns a GeneratedClip. The foley.sources.generate.generate() façade converges it on the shared ingest_one pipeline (by-value, operator-consented).

generate(prompt: str, **affordances) → GeneratedClip[source]: Synthesize a sound for prompt; return its bytes + provisional candidate.

name: str: Registry / façade key — 'stable_audio' | 'elevenlabs' | …

class foley.sources.base.GeneratedClip(audio_bytes: bytes, candidate: Candidate, notes: list = <factory>)[source]

One freshly-generated sound: its transient bytes + a provisional candidate.

The return type of a GenerateAdapter’s generate — the envelope that reconciles report 10 §4.2 (generate -> Candidate: retrieval and generation return the same shape) with foley’s two non-negotiables:

adapters never touch storage — so the bytes ride with the candidate in an explicit field rather than being written anywhere, and
``ingest_one`` is never forked — the foley.sources.generate.generate() façade hands audio_bytes to the one shared pipeline (by-value), exactly as foley.sources.pull.add_from() does for a retrieved download.

Generation is retrieval’s search -> Candidate and download -> bytes fused into one call, because there is no server to re-fetch the bytes from.

Provisional/canonical split — on candidate.sound only license / caption / tags are authoritative. id (a discarded "<source>:pending" placeholder), uri, content_sha256, storage_mode, qc, duration_s, sample_rate, channels and the embedding_* fields are all minted downstream by ingest_one() (the façade passes sound_id=None so the stored id is the decoded-PCM content hash). audio_bytes lives ONLY here and is never serialized — a SoundRecord holds a uri, never bytes.

audio_bytes

The generated audio as encoded container bytes (WAV/FLAC/…), transient and in-memory only — consumed once by the ingest pipeline.

Type:: bytes

candidate

The report-10 Candidate (origin=CandidateOrigin.generated) carrying the authoritative LicenseRecord (is_ai_generated=True + the generation-provenance block) plus the prompt caption + seed tags.

Type:: Candidate

notes

Generation-time messages (on_unsupported_param='warn' drops, output-format fallbacks, …). The foley.sources.generate.generate() façade folds these into the stored IngestResult‘s notes so they surface in the run report.

Type:: list

class foley.sources.base.SourceAdapter(*args, **kwargs)[source]

The live/HTTP source contract (report 10 §4.2): search + get + download.

A retrieve adapter (Freesound, #5) fetches existing sounds from a service: search returns ranked Candidates, get resolves one id to a SoundRecord, and download returns the (transient, TOS-permitting) bytes to embed. It returns the SAME Candidate / SoundRecord shapes as the retrieval index, so callers see one uniform interface whether audio is remote, cached, or local.

Distinct from (and complementary to) the narrow bulk-corpus CorpusAdapter: a live adapter does NOT re-implement enrichment or storage — it converges on the same foley.index.ingest.ingest_one() pipeline via foley.sources.pull.add_from() (it wraps the corpus machinery, it does not fork it). Generation adapters (#6: Stable Audio Open, ElevenLabs) will add a sibling generate surface; it is intentionally out of scope here so this Protocol stays runtime-checkable for retrieve adapters.

download(source_id: str) → bytes[source]: Return a sound’s bytes (honoring cache_bytes_ok at the storage gate).

get(source_id: str) → SoundRecord[source]: Resolve one source id to a metadata SoundRecord.

search(query: str, **kw) → list[Candidate][source]: Return ranked candidates for query (license filter pushed native).

class foley.sources.base.UniformCorpus(name: str, ring: int, default_license_id: str, source: str, rights_verified: bool = True, tag_hints_from_path: bool = False)[source]

A bulk corpus where every clip carries the same license.

Covers FoleySet (CC-BY), Sonniss (Sonniss-GDC), BBC RemArc (RemArc) and any other single-license drop: it walks the audio tree and stamps one license. rights_verified is a constructor field so a corpus whose blanket license is authoritatively known passes the gate, while a merely-assumed one can stay fail-closed. Subclass to enrich per-clip meta (see ClothoEvalCorpus).

corpus_dir(data_dir: str) → str[source]: data_dir/<name> — the conventional on-disk root for this corpus.

iter_clips(root: str) → Iterator[ClipSpec][source]

Yield one ClipSpec per audio file under root.

When tag_hints_from_path is set, the clip’s parent folder names (relative to root) are carried in meta['tag_hints'] — corpora like FoleySet encode a Foley taxonomy in their directory structure.

resolve_license(spec: ClipSpec) → LicenseRecord[source]: Stamp the corpus’s uniform license (derived via the licensing SSOT).

foley.sources.base.api_license(*, source: str, license_id: str, rights_verified: bool, overrides: dict | None = None, source_id: str | None = None, source_url: str | None = None, license_url: str | None = None, creator_name: str | None = None, attribution_text: str | None = None) → LicenseRecord[source]

Build an API-acquisition LicenseRecord, flags derived.

The live-source sibling of bulk_license() (acquisition_method=api), used by HTTP SourceAdapters (Freesound, …). It exposes the overrides seam so an adapter can flip an operational flag on top of the per-item copyright license without minting a new license_id — the Freesound case: keep the sound’s own CC id (CC0-1.0 / CC-BY-4.0 / …) but pass overrides={'cache_bytes_ok': False} because the API TOS forbids caching the bytes even for CC0. redistribute_standalone_ok (copyright) and cache_bytes_ok (TOS) are distinct; only the latter is flipped.

Parameters:

source – Provenance source tag (e.g. 'freesound').
license_id – The per-item normalized license id (a key of LICENSE_FLAGS).
rights_verified – True only for an authoritatively-recognized license (fail-closed gate input); MUST be True for foley.keep() to admit the sound.
overrides – Per-source flag overrides applied on top of license_id’s row (e.g. {'cache_bytes_ok': False}). Keys must be LicenseFlags fields (validated by derive_license_flags()).
source_id – The source-native id (e.g. a Freesound numeric id), for provenance.
source_url – A human-resolvable URL for the item (attribution + the stable by-reference re-fetch handle).
license_url – The license URL/label exactly as the source served it.
creator_name – The uploader/creator (required for CC-BY attribution).
attribution_text – A ready-made attribution string, if supplied.

Returns:

A populated LicenseRecord with its derived flags (+ overrides) applied.

foley.sources.base.bulk_license(*, source: str, license_id: str, rights_verified: bool, source_id: str | None = None, source_url: str | None = None, creator_name: str | None = None, attribution_text: str | None = None) → LicenseRecord[source]

Build a bulk-acquisition LicenseRecord, flags derived.

The SSOT license builder every bulk-corpus adapter routes through (acquisition_method=bulk); a thin wrapper over _build_license(). Its signature is unchanged from #4 — no overrides (a downloaded corpus is cacheable by-value), so every existing corpus adapter keeps working verbatim.

Parameters:

source – Provenance source tag (e.g. 'fsd50k', 'foleyset').
license_id – The normalized license id (a key of LICENSE_FLAGS, else it falls back to the all-False unknown flags).
rights_verified – Whether the license is authoritatively known (fail-closed gate input — pass False for unrecognized/ambiguous licenses).
source_id – The corpus-native clip id, for provenance.
source_url – A human-resolvable URL for the clip (attribution/credits).
creator_name – The uploader/creator (required for CC-BY attribution).
attribution_text – A ready-made attribution string, if the corpus supplies one.

Returns:

A populated LicenseRecord with its derived flags applied.

foley.sources.base.corpora_in_rings(rings: tuple[int, ...]) → list[CorpusAdapter][source]: Registered adapters whose ring is in rings (sorted by name).

foley.sources.base.generated_license(*, source: str, license_id: str, generator_model: str, generation_prompt: str, rights_verified: bool = True, generator_version: str | None = None, generation_seed: int | None = None, generation_params: dict | None = None, disclosure_recommended: bool = True, watermark: dict | None = None, c2pa_manifest_ref: str | None = None, overrides: dict | None = None, source_id: str | None = None, source_url: str | None = None, license_url: str | None = None, creator_name: str | None = None, attribution_text: str | None = None) → LicenseRecord[source]

Build an AI-generated LicenseRecord, flags derived + provenance stamped.

The generation sibling of bulk_license() / api_license() (acquisition_method=generated): it routes through the shared _build_license() core — so the eight permission flags are DERIVED from license_id by apply_license_flags(), never hand-set — and then stamps the generation-provenance block in exactly one place, so every generate adapter (Stable Audio Open, ElevenLabs, …) records provenance identically (open-closed).

The consequences flow automatically from the two generator rows in LICENSE_FLAGS: cache_bytes_ok=True (⇒ by-value storage), ai_training_ok=False (the record keeps this restriction — the generate façade consents to embed+persist via allow_ai_training_forbidden but never flips the flag, so foley.keep() still rejects the sound for any IntendedUse(will_train=True)), and — for Stability-Community — revenue_cap_usd=1_000_000 (enforced by foley.keep() at select time).

Parameters:

source – The generator tag (e.g. 'stable_audio' / 'elevenlabs').
license_id – The generator license id ('Stability-Community' / 'ElevenLabs-SFX' — a key of LICENSE_FLAGS).
generator_model – The model identifier (e.g. 'stable-audio-open-1.0', 'eleven_text_to_sound_v2').
generation_prompt – The user prompt, verbatim.
rights_verified – True (a generator license is authoritatively known); MUST be True or foley.keep() rejects the sound.
generator_version – Optional model/version string.
generation_seed – The reproducibility seed (an int for a seeded Stable-Audio-Open run; None for a non-deterministic backend).
generation_params – The RESOLVED NATIVE params actually sent to the backend (e.g. guidance_scale — not the unified prompt_influence — audio_end_in_s, …), for reproducibility + audit.
disclosure_recommended – EU AI Act Art. 50 hint (default True); makes the credits AI-disclosure line render immediately (#9a already reads it).
watermark – Pass-through carrier for #9b (AudioSeal); None until then.
c2pa_manifest_ref – Pass-through carrier for #9b (C2PA); None until then.
overrides – Optional per-source flag overrides (rare for generation).
source_id – Optional source-native id, for provenance.
source_url – Optional human-resolvable URL.
license_url – Optional license URL/label.
creator_name – Optional creator (usually unset for generation).
attribution_text – Optional ready-made attribution string.

Returns:

A populated LicenseRecord with derived flags applied AND is_ai_generated=True plus the full generation-provenance block.

foley.sources.base.register_corpus(adapter: CorpusAdapter) → CorpusAdapter[source]

Register adapter in CORPUS_REGISTRY (idempotent) and return it.

Raises:: ValueError – If a different adapter is already registered under the name.

foley.sources.base.ring_of(name: str) → int[source]: Return the ring of the registered corpus name (raises KeyError).

foley.sources.base.select_corpora(*, rings: tuple[int, ...] = (0, 1), corpora: list[str] | None = None) → list[CorpusAdapter][source]

Resolve the adapters a bootstrap run should touch.

An explicit corpora allowlist (by name) wins over rings; otherwise every registered adapter in the given rings is selected. Ring 2 is never in the default rings — it is opt-in only.

Raises:: KeyError – If a name in corpora is not registered.