> built 2026-09-22 14:36 UTC from d634c6b (main) · hfdol 0.1.20. Details: build_info.json # index.html.md # hfdol Simple Mapping interface to HuggingFace. (Note – was [hf](https://pypi.org/project/hf/0.0.14/) but realeased the name to Huggingface itself for their tool.) To install: `pip install hfdol` You’ll also need a Hugginface token. See [more about this here](https://huggingface.co/docs/huggingface_hub/en/quick-start). ## Motivation The Python packages [`datasets`](https://github.com/huggingface/datasets) and [`huggingface_hub`](https://github.com/huggingface/huggingface_hub) provide a remarkably clean, well-documented, and comprehensive API for accessing datasets, models, spaces, and papers hosted on [Hugging Face](https://huggingface.co).
\\\\ Yet, as elegant as these APIs are, they remain *their own language*. Every library—no matter how intuitive—inevitably carries its own conventions, abstractions, and domain-specific semantics. When working with one or two APIs, this diversity is harmless, even stimulating. But when juggling dozens or hundreds of them, the cognitive overhead accumulates. Despite their differences, most APIs share a small set of universal primitives — *retrieve something by key, list what’s available, check existence, store, update, delete*.
\\\\ In Python, these operations are embodied by the `Mapping` interface, the conceptual model behind dictionaries. It’s a minimal, ubiquitous, and instantly recognizable abstraction. This package offers such a `Mapping`-based façade to Hugging Face datasets and models, allowing you to browse, query, and access them as if they were simple Python dictionaries. The goal isn’t to replace the original API, but to provide a thin, ergonomic layer for the most common operations — so you can spend less time remembering syntax, and more time working with data. ## Examples This package provides four ready-to-use singleton instances, each offering a dictionary-like interface to different types of HuggingFace resources: ```python import hfdol ``` ### Working with Datasets The `hfdol.datasets` singleton provides a `Mapping` (i.e. read-only-dictionary-like) interface to HuggingFace datasets: #### List Local Datasets As with dictionaries, `hfdol.datasets` is an iterable. An iterable of keys. The keys are repository ids for those datasets you’ve downloaded. See what datasets you already have cached locally like this: ```python list(hfdol.datasets) # Lists locally cached datasets # ['stingning/ultrachat', 'allenai/WildChat-1M', 'google-research-datasets/go_emotions'] ``` #### Access Local Datasets The values of `hfdol.datasets` are the `DatasetDict` (from Huggingface’s `datasets` package) instances that give you access to the dataset. If you already have the dataset downloaded locally, it will load it from there, if not it will download it, then give it to you (and it will be cached locally for the next time you access it). ```python data = hfdol.datasets["stingning/ultrachat"] # Loads the dataset print(data) # Shows dataset information and structure ``` #### Search for Remote Datasets `hfdol.datasets` also offers a search functionality, so you can search “remote” repositories: ```python # Search for music-related datasets search_results = hfdol.datasets.search("music", gated=False) print(f"search_results is a {type(search_results).__name__}") # It's a generator # Get the first result (it will be a `DatasetInfo` instance contain information on the dataset) result = next(search_results) print(f"Dataset ID: {result.id}") print(f"Description: {result.description[:80]}...") # Download and use it directly data = hfdol.datasets[result] # You can pass the DatasetInfo object directly ``` Note that the `gated=False` was to make sure you get models that you have access to. For more search options, see the [HuggingFace Hub documentation](https://huggingface.co/docs/huggingface_hub/package_reference/hf_api#huggingface_hub.HfApi.list_datasets). #### A useful recipe: Get a table of result infos You can use this to get a dataframe of the first/next `n` results of the results iterable: ```py def table_of_results(results, n=10): import itertools, operator, pandas as pd results_table = pd.DataFrame( # make a table with map( operator.attrgetter("__dict__"), # the attributes dicts itertools.islice(results, n), # ... of the first 10 search results ) ) return results_table ``` Example: ```py results_table = table_of_results(search_results) results_table ``` ```none id author sha ... 0 Genius-Society/hoyoMusic Genius-Society 4f7e5120c0e8e26213d4bb3b52bcce76e69dfce4 ... 1 Genius-Society/emo163 Genius-Society 6b8c3526b66940ddaedf15602d01083d24eb370c ... 2 ccmusic-database/acapella ccmusic-database 4cb8a4d4cb58cc55f30cb8c7a180fee1b5576dc5 ... 3 ccmusic-database/pianos ccmusic-database db2b3f74c4c989b4fbda4b309e6bc925bfd8f5d1 ... ... ``` ### Working with Models The `hfdol.models` singleton provides the same dictionary-like interface for models: #### Search for Models Find models by keywords: ```python model_search_results = hfdol.models.search("embeddings", gated=False) model_result = next(model_search_results) print(f"Model: {model_result.id}") ``` #### Download Models Get the local path to a model (downloads if not cached): ```python model_path = hfdol.models[model_result] print(f"Model downloaded to: {model_path}") ``` #### List Local Models See what models you have cached: ```python list(hfdol.models) # Lists all locally cached models ``` ### Working with Spaces The `hfdol.spaces` singleton provides access to HuggingFace Spaces (interactive ML demos and applications): #### Search for Spaces Find interesting Spaces by keywords: ```python space_search_results = hfdol.spaces.search("gradio", limit=5) space_result = next(space_search_results) print(f"Space: {space_result.id}") ``` #### Access Space Information Get detailed information about a Space: ```python space_info = hfdol.spaces[space_result] print(f"Space info: {space_info}") ``` #### List Local Spaces See what spaces you have cached locally: ```python list(hfdol.spaces) # Lists all locally cached spaces ``` ### Working with Papers The `hfdol.papers` singleton provides access to research papers hosted on HuggingFace: #### Search for Papers Find research papers by topic: ```python paper_search_results = hfdol.papers.search("transformer", limit=5) paper_result = next(paper_search_results) print(f"Paper: {paper_result.id}") ``` #### Access Paper Information Get detailed information about a paper: ```python paper_info = hfdol.papers[paper_result] print(f"Paper title: {paper_info.title}") print(f"Abstract: {paper_info.summary[:100]}...") ``` Note: Papers are metadata objects only—they contain information about research papers but don’t have downloadable files like datasets or models. ### Getting Repository Sizes You can check the size of any repository before downloading using the `get_size` function. The `repo_type` parameter is required to avoid ambiguity when repositories exist as multiple types: ```python from hfdol import get_size # Get size of a dataset (specify repo_type explicitly) dataset_size = get_size("ccmusic-database/music_genre", repo_type="dataset") print(f"Dataset size: {dataset_size:.2f} GiB") # Get size of a model model_size = get_size("ccmusic-database/music_genre", repo_type="model") print(f"Model size: {model_size:.2f} GiB") # Using RepoType enum for type safety from hfdol.base import RepoType size_with_enum = get_size("some-repo", repo_type=RepoType.DATASET) # Get size in different units (e.g., bytes) size_in_bytes = get_size("some-repo", repo_type="dataset", unit_bytes=1) ``` **Pro tip**: Use the singleton instances for automatic repo_type handling: ```python # These automatically know their repo_type dataset_size = hfdol.datasets.get_size("ccmusic-database/music_genre") model_size = hfdol.models.get_size("ccmusic-database/music_genre") ``` ### Unified Interface The beauty of this approach is that whether you’re working with datasets, models, spaces, or papers, the interface remains familiar and consistent—just like working with Python dictionaries. All four singleton instances support the same core operations: - **Dictionary-style access**: `resource = hfdol.datasets[key]`, `model_path = hfdol.models[key]` - **Local listing**: `list(hfdol.datasets)`, `list(hfdol.models)` - **Remote searching**: `hfdol.datasets.search(query)`, `hfdol.models.search(query)` - **Existence checking**: `key in hfdol.datasets`, `key in hfdol.models` This unified interface means you can switch between different types of HuggingFace resources without learning new APIs—it’s all just dictionaries! And since they’re singleton instances, they’re always ready to use without any setup. ## Design & Architecture ### Design Philosophy This package is designed as a **thin façade** over the excellent [`huggingface_hub`](https://github.com/huggingface/huggingface_hub) and [`datasets`](https://github.com/huggingface/datasets) libraries. Rather than reinventing functionality, it provides a unified `Mapping` interface that wraps the most common operations, making them feel like native Python dictionary operations. The design balances two sometimes-competing goals: 1. **Simplicity**: Keep the codebase small, readable, and maintainable 2. **Single Source of Truth (SSOT)**: Minimize hardcoded knowledge about the underlying APIs Ideally, this interface would be *entirely* auto-generated through static analysis of the wrapped packages. While we achieve this partially, practical constraints require some manual intervention—but we’ve minimized it as much as possible. ### Key Architectural Patterns #### 1. Configuration-Driven Design (SSOT) The `repo_type_helpers` dictionary serves as the **single source of truth** for all repo-type-specific behavior: ```python repo_type_helpers = dict( dataset=dict( loader_func=load_dataset, search_func=list_datasets, ), model=dict( loader_func=snapshot_download, search_func=list_models, ), # ... etc ) ``` This declarative approach means: - Adding a new repo type requires only updating this configuration - No duplication of logic across different repo types - Clear visibility of how each type differs #### 2. Dynamic Signature Injection Rather than manually replicating the signatures of wrapped functions (which would violate SSOT), we use **signature extraction and injection** via the `sign_kwargs_with` decorator: ```python @sign_kwargs_with(search_func) def search(self, filter, **kwargs): return self.search_func(filter=filter, **kwargs) ``` This means: - Each `.search()` method automatically inherits the correct signature from its underlying function - IDEs and type checkers see the actual parameters available - When HuggingFace updates their APIs, our signatures update automatically - Documentation stays accurate without manual synchronization **Note**: The `list_papers` function required special handling (`_list_papers` wrapper) because it uses `query` instead of `filter` as its parameter name. This is the type of pragmatic compromise we make—we normalize the interface rather than exposing the inconsistency. #### 3. Separation of Concerns The architecture cleanly separates: - **Configuration** (`repo_type_helpers`): What differs between types - **Base functionality** (`HfMapping`): Shared behavior for all types - **Type-specific classes** (`HfDatasets`, `HfModels`, etc.): Minimal subclasses that mainly provide: - Clear, discoverable class names - Type-specific documentation - Future extensibility points - **Convenience layer** (module-level singletons): Zero-setup access for users #### 4. Module-Level Singletons The pre-instantiated `datasets`, `models`, `spaces`, and `papers` instances follow Python’s **convenience instance pattern** (seen in `sys.stdout`, `np.random`, etc.): ```python # Ready to use immediately datasets = HfDatasets() models = HfModels() ``` This works because these instances: - Have no mutable state - Require no configuration for basic use - Represent logical singletons (“the datasets mapping”) #### 5. Progressive Disclosure The API supports multiple levels of sophistication: ```python # Simplest: Use pre-configured singletons data = hfdol.datasets["some/dataset"] # Advanced: Create custom instances with configuration my_datasets = HfDatasets() # Power user: Parameterized mapping for dynamic repo types custom = HfMapping(RepoType.DATASET) ``` ### Design Compromises Several compromises were made for pragmatism: 1. **Manual wrappers**: `_list_papers` normalizes the papers API to match others 2. **Enum + string hybrid**: `RepoType(str, Enum)` allows both type safety and string convenience 3. **Explicit repo_type in get_size**: Required parameter to avoid ambiguity when repos exist as multiple types 4. **Signature injection limitations**: Works well for keyword arguments but can’t handle complex overloads ### Contributing Guidelines When contributing to this package, please maintain these principles: **✅ DO:** - Add configuration to `repo_type_helpers` rather than creating new methods - Use signature extraction (`sign_kwargs_with`) when wrapping functions with many parameters - Keep `HfMapping` generic and push specialization to configuration - Document *why* special cases exist (like `_list_papers`) - Test against actual HuggingFace APIs to catch signature drift **❌ AVOID:** - Duplicating knowledge about wrapped APIs - Hardcoding parameter lists or types that could be extracted - Adding stateful behavior to mapping instances - Creating wrapper methods that simply pass through to underlying functions **When in doubt:** - Ask “Could this be driven by configuration?” - Prefer declarative patterns over imperative logic - Keep the codebase small and the configuration visible The goal is a package where 80% of the code is just wiring and configuration, and the HuggingFace packages do the actual work. This maximizes maintainability and minimizes drift as those packages evolve.

This documentation as a single file: hfdol.md (Markdown, for agents).

# _autosummary/hfdol.base.html.md # hfdol.base Base functionality for hfdol. This module provides Mapping-based interfaces to HuggingFace datasets, models, spaces, and papers, allowing you to interact with them using familiar Python dictionary operations. Key Classes: - HfDatasets: A Mapping interface for browsing and loading HuggingFace datasets - HfModels: A Mapping interface for browsing and downloading HuggingFace models - HfSpaces: A Mapping interface for browsing and accessing HuggingFace Spaces - HfPapers: A Mapping interface for browsing and accessing HuggingFace Papers All classes provide a unified API for local cached items (via iteration and key access) and remote searching/downloading capabilities. ### Functions | `ensure_dir`(dirpath) | | |---------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------| | `ensure_id`(obj) | | | [`get_size`](_autosummary/hfdol.base.html.md#hfdol.base.get_size)(repo_id, \*[, unit_bytes]) | Calculates the total size of a Hugging Face repository (model, dataset, or space) in GiB. | | [`list_local_repos`](_autosummary/hfdol.base.html.md#hfdol.base.list_local_repos)(repo_type) | Dynamically list locally cached repositories of a given type using scan_cache_dir. | | [`sign_kwargs_with`](_autosummary/hfdol.base.html.md#hfdol.base.sign_kwargs_with)(source_func[, first_arg_index]) | Minimal decorator to replace \*\*kwargs with parameters from source_func. | ### Classes | [`HfDatasets`](_autosummary/hfdol.base.html.md#hfdol.base.HfDatasets)([repo_type]) | A Mapping interface to HuggingFace datasets. | |----------------------------------------------------------------------------|------------------------------------------------| | [`HfMapping`](_autosummary/hfdol.base.html.md#hfdol.base.HfMapping)([repo_type]) | Abstract base class for HuggingFace Mappings. | | [`HfModels`](_autosummary/hfdol.base.html.md#hfdol.base.HfModels)([repo_type]) | A Mapping interface to HuggingFace models. | | [`HfPapers`](_autosummary/hfdol.base.html.md#hfdol.base.HfPapers)([repo_type]) | A Mapping interface to HuggingFace Papers. | | [`HfSpaces`](_autosummary/hfdol.base.html.md#hfdol.base.HfSpaces)([repo_type]) | A Mapping interface to HuggingFace Spaces. | | [`RepoType`](_autosummary/hfdol.base.html.md#hfdol.base.RepoType)(\*values) | Valid HuggingFace repository types. | ### *class* hfdol.base.HfDatasets(repo_type=None) Bases: [`HfMapping`](_autosummary/hfdol.base.html.md#hfdol.base.HfMapping) A Mapping interface to HuggingFace datasets. Provides dictionary-like access to locally cached datasets and seamless downloading of remote datasets. Keys are dataset repository IDs (e.g., ‘stingning/ultrachat’). Values are loaded dataset objects from the datasets library. ### Examples ```pycon >>> d = HfDatasets() ``` List locally cached datasets: ```pycon >>> list_of_dataset_repo_ids = list(d) ``` Search remote datasets ```pycon >>> results = d.search('music', gated=False) ``` Load or download a dataset ```pycon >>> data = d['some/dataset'] ``` ### *class* hfdol.base.HfMapping(repo_type=None) Bases: [`Mapping`](https://docs.python.org/3/library/collections.abc.html#collections.abc.Mapping) Abstract base class for HuggingFace Mappings. This class provides common functionality for datasets, models, spaces, and papers, parameterized by repo_type which determines loader_func and search_func. Can be used in two ways: 1. Via subclasses (HfDatasets, HfModels, etc.) for common types - best UX 2. Via direct parameterization for less common types or dynamic use #### get_size(key, , unit_bytes=1073741824) Get size (by default, in GiB) of an item from it’s key (repo ID) * **Return type:** [`float`](https://docs.python.org/3/builtins/functions.html#float) ### *class* hfdol.base.HfModels(repo_type=None) Bases: [`HfMapping`](_autosummary/hfdol.base.html.md#hfdol.base.HfMapping) A Mapping interface to HuggingFace models. Provides dictionary-like access to locally cached models and seamless downloading of remote models. Keys are model repository IDs (e.g., ‘sentence-transformers/all-MiniLM-L6-v2’). Values are file paths to the downloaded model directories. ### Examples ```pycon >>> m = HfModels() ``` List locally cached models ```pycon >>> list_of_model_repo_ids = list(m) ``` Search remote models ```pycon >>> results = m.search('embeddings', gated=False) ``` Download or get path to a model ```pycon >>> model_path = m['some/model'] ``` ### *class* hfdol.base.HfPapers(repo_type=None) Bases: [`HfMapping`](_autosummary/hfdol.base.html.md#hfdol.base.HfMapping) A Mapping interface to HuggingFace Papers. Provides dictionary-like access to paper information. Keys are paper IDs. Values are PaperInfo objects containing paper metadata, abstracts, and links. #### NOTE Papers are metadata objects only - they don’t have downloadable files or sizes. ### Examples ```pycon >>> p = HfPapers() ``` Search papers ```pycon >>> results = p.search('transformer') ``` Get information about a paper ```pycon >>> paper_info = p['2017.12345'] ``` ### *class* hfdol.base.HfSpaces(repo_type=None) Bases: [`HfMapping`](_autosummary/hfdol.base.html.md#hfdol.base.HfMapping) A Mapping interface to HuggingFace Spaces. Provides dictionary-like access to locally cached spaces and seamless retrieval of remote space information. Keys are space repository IDs (e.g., ‘gradio/chatbot’). Values are SpaceInfo objects containing space metadata and configuration. ### Examples ```pycon >>> s = HfSpaces() ``` List locally cached spaces ```pycon >>> list_of_space_repo_ids = list(s) ``` Search remote spaces ```pycon >>> results = s.search('gradio', gated=False) ``` Get information about a space ```pycon >>> space_info = s['gradio/chatbot'] ``` ### *class* hfdol.base.RepoType(\*values) Bases: [`str`](https://docs.python.org/3/builtins/stdtypes.html#str), [`Enum`](https://docs.python.org/3/library/enum.html#enum.Enum) Valid HuggingFace repository types. ### hfdol.base.get_size(repo_id, , unit_bytes=1073741824, repo_type) Calculates the total size of a Hugging Face repository (model, dataset, or space) in GiB. * **Parameters:** * **repo_id** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – The ID of the repository (e.g., “bert-base-uncased”, “MMMU/MMMU”, or “spaces/gradio/chatbot”). * **unit_bytes** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Number of bytes in the desired unit. Default is 1024\*\*3 for GiB. For bytes, enter 1. * **repo_type** ([`RepoType`](_autosummary/hfdol.base.html.md#hfdol.base.RepoType)) – Type of repository (“model”, “dataset”, “space”). Required parameter. Papers don’t have file sizes, so they’re not supported. * **Returns:** The total repository size in the specified unit. * **Return type:** [`float`](https://docs.python.org/3/builtins/functions.html#float) * **Raises:** [**ValueError**](https://docs.python.org/3/builtins/exceptions.html#ValueError) – If repo_type is “paper” (papers don’t have file sizes) or if repo_type is invalid. ### hfdol.base.list_local_repos(repo_type) Dynamically list locally cached repositories of a given type using scan_cache_dir. * **Parameters:** **repo_type** ([*str*](https://docs.python.org/3/builtins/stdtypes.html#str)) – The type of repository (“dataset”, “model”, “space”, or “paper”). * **Returns:** A list of repo IDs for repositories of the specified type. * **Return type:** [*list*](https://docs.python.org/3/builtins/stdtypes.html#list) ### hfdol.base.sign_kwargs_with(source_func, first_arg_index=0) Minimal decorator to replace \*\*kwargs with parameters from source_func. * **Parameters:** * **source_func** – Function whose parameters will be injected * **first_arg_index** – Index of first parameter to include from source (default 0) ```pycon >>> def source(x=0, y=1, z=2): ... >>> @sign_kwargs_with(source) ... def target(some, thing=None, **kwargs): ... pass >>> str(signature(target)) '(some, thing=None, *, x=0, y=1, z=2)' ``` ```pycon >>> @sign_kwargs_with(source, first_arg_index=1) ... def target2(some, thing=None, **kwargs): ... pass >>> str(signature(target2)) '(some, thing=None, *, y=1, z=2)' ``` # _autosummary/hfdol.deploy.html.md # hfdol.deploy Deploy webapps to HuggingFace Spaces. This module wraps `huggingface_hub` with opinionated, idempotent helpers for publishing a Python package’s webapp to a Docker-SDK Space. The primary entry point is [`deploy_webapp()`](_autosummary/hfdol.deploy.html.md#hfdol.deploy.deploy_webapp), which composes the smaller helpers into a full create-or-update + restart cycle. Functions are designed to be composable: each does one thing, fails loudly, and prints what it’s about to do. The module assumes you already have a package on PyPI (or a Dockerfile that knows how to install it). Quick reference: ```pycon >>> from hfdol.deploy import deploy_webapp >>> deploy_webapp( ... repo_id="thorwhalen/typola", ... source_dir="/path/to/staging", ... ) ``` For ad-hoc operations: ```pycon >>> from hfdol.deploy import factory_reboot, wait_for_build >>> factory_reboot("thorwhalen/typola") >>> wait_for_build("thorwhalen/typola") ``` Token resolution: Operations need a write-scoped HF token. By default, [`ensure_write_token()`](_autosummary/hfdol.deploy.html.md#hfdol.deploy.ensure_write_token) reads `HF_WRITE_TOKEN` from the environment, then falls back to sourcing `~/.keys` (a shell file that exports it). If you keep your write token elsewhere, pass `token=...` to any function explicitly. ### Functions | [`create_or_update_space`](_autosummary/hfdol.deploy.html.md#hfdol.deploy.create_or_update_space)(repo_id, \*[, sdk, ...]) | Create an HF Space if it doesn't exist; otherwise return its info. | |-----------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------| | [`deploy_webapp`](_autosummary/hfdol.deploy.html.md#hfdol.deploy.deploy_webapp)(repo_id, source_dir, \*[, sdk, ...]) | End-to-end: create-if-missing → upload → factory reboot → wait for build. | | [`ensure_write_token`](_autosummary/hfdol.deploy.html.md#hfdol.deploy.ensure_write_token)(\*[, env_var, keys_file]) | Return a write-scoped HF token; source `~/.keys` if needed. | | [`factory_reboot`](_autosummary/hfdol.deploy.html.md#hfdol.deploy.factory_reboot)(repo_id, \*[, token]) | Trigger a from-scratch rebuild of the Space (busts Docker layer cache). | | [`render_pypi_webapp_dockerfile`](_autosummary/hfdol.deploy.html.md#hfdol.deploy.render_pypi_webapp_dockerfile)(package, \*[, ...]) | Render a Dockerfile for the canonical "PyPI package + FastAPI + React" pattern. | | [`render_space_readme`](_autosummary/hfdol.deploy.html.md#hfdol.deploy.render_space_readme)(title, \*[, color_from, ...]) | Render an HF Space README with the YAML frontmatter HF requires. | | [`stage_webapp`](_autosummary/hfdol.deploy.html.md#hfdol.deploy.stage_webapp)(\*, staging_dir, api_src, ...[, ...]) | Lay out the contents of a Space repo in a local staging directory. | | [`upload_app_dir`](_autosummary/hfdol.deploy.html.md#hfdol.deploy.upload_app_dir)(repo_id, folder_path, \*[, ...]) | Upload a local folder to an HF Space, replacing the live files. | | [`wait_for_build`](_autosummary/hfdol.deploy.html.md#hfdol.deploy.wait_for_build)(repo_id, \*[, token, timeout, ...]) | Poll Space status until it lands in a terminal state or times out. | ### Classes | [`BuildResult`](_autosummary/hfdol.deploy.html.md#hfdol.deploy.BuildResult)(final_stage, elapsed_seconds[, ...]) | Outcome of waiting for a Space build. | |-----------------------------------------------------------------------------------------------------|-----------------------------------------| ### *class* hfdol.deploy.BuildResult(final_stage, elapsed_seconds, timed_out=False, stages_seen=) Bases: [`object`](https://docs.python.org/3/builtins/functions.html#object) Outcome of waiting for a Space build. ### hfdol.deploy.create_or_update_space(repo_id, , sdk='docker', private=False, token=None, exist_ok=True) Create an HF Space if it doesn’t exist; otherwise return its info. Idempotent. Safe to call before every deploy. * **Parameters:** * **repo_id** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – `owner/space-name`. * **sdk** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – Space SDK — `docker` (recommended for custom apps), `gradio`, `streamlit`, `static`. * **private** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – If True, create as private (only matters on first creation). * **token** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – Write token; defaults to [`ensure_write_token()`](_autosummary/hfdol.deploy.html.md#hfdol.deploy.ensure_write_token). * **exist_ok** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – If True (default), an already-existing repo is fine. * **Returns:** `SpaceInfo` for the (now-existing) Space. ### hfdol.deploy.deploy_webapp(repo_id, source_dir, , sdk='docker', private=False, token=None, message='Deploy webapp', ignore_patterns=('node_modules/\*\*', '_\_pycache_\_/\*\*', '\*.pyc', '.DS_Store', '.git/\*\*'), delete_patterns=None, rebuild=True, wait=True, timeout=900) End-to-end: create-if-missing → upload → factory reboot → wait for build. The most common deploy entry point. `source_dir` should contain everything that goes on the Space repo: `Dockerfile`, `README.md` (with HF YAML frontmatter), and your app source (e.g., `webapp/api/`, `webapp/ui/dist/`). * **Parameters:** * **repo_id** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – `owner/space-name`. * **source_dir** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str) | [`Path`](https://docs.python.org/3/library/pathlib.html#pathlib.Path)) – Local staging directory. * **sdk** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – Space SDK (only matters on first creation). * **private** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – Create as private (only matters on first creation). * **token** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – Write token; defaults to [`ensure_write_token()`](_autosummary/hfdol.deploy.html.md#hfdol.deploy.ensure_write_token). * **message** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – Commit message. * **ignore_patterns** ([`Iterable`](https://docs.python.org/3/library/typing.html#typing.Iterable)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – Glob patterns to skip during upload. * **delete_patterns** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`Iterable`](https://docs.python.org/3/library/typing.html#typing.Iterable)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]]) – Glob patterns to delete from the Space before uploading (e.g., to clear stale build artifacts). * **rebuild** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – If True (default), trigger a factory reboot after upload. Set False to skip — useful when only README/metadata changed. * **wait** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – If True (default), poll until the build settles. * **timeout** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Max seconds to wait for the build. * **Return type:** [`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`BuildResult`](_autosummary/hfdol.deploy.html.md#hfdol.deploy.BuildResult)] * **Returns:** [`BuildResult`](_autosummary/hfdol.deploy.html.md#hfdol.deploy.BuildResult) if `wait=True`, else `None`. ### hfdol.deploy.ensure_write_token(, env_var='HF_WRITE_TOKEN', keys_file=PosixPath('/home/runner/.keys')) Return a write-scoped HF token; source `~/.keys` if needed. HF Spaces operations (create, upload, restart) need a write token. The default `HF_TOKEN` is often read-only. Convention: keep the write token in `~/.keys` as `export HF_WRITE_TOKEN=hf_...`. * **Parameters:** * **env_var** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – Environment variable name to read first. * **keys_file** ([`Path`](https://docs.python.org/3/library/pathlib.html#pathlib.Path)) – Shell file to source as fallback (must export `env_var`). * **Return type:** [`str`](https://docs.python.org/3/builtins/stdtypes.html#str) * **Returns:** The token string. * **Raises:** [**SystemExit**](https://docs.python.org/3/builtins/exceptions.html#SystemExit) – If no token can be resolved. ### hfdol.deploy.factory_reboot(repo_id, , token=None) Trigger a from-scratch rebuild of the Space (busts Docker layer cache). A normal restart reuses cached Docker layers. A factory reboot blows the cache away — required when `pip install` should re-fetch a new package version, or when system deps in the Dockerfile changed. * **Parameters:** * **repo_id** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – `owner/space-name`. * **token** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – Write token; defaults to [`ensure_write_token()`](_autosummary/hfdol.deploy.html.md#hfdol.deploy.ensure_write_token). * **Return type:** [`None`](https://docs.python.org/3/builtins/constants.html#None) ### hfdol.deploy.render_pypi_webapp_dockerfile(package, , extras='web', version_spec='', python_version='3.12', api_module='webapp.api.main:app', port=7860, api_dir='webapp/api', ui_dist_dir='webapp/ui/dist', extra_run_lines=()) Render a Dockerfile for the canonical “PyPI package + FastAPI + React” pattern. Assumes: - Your Python package is on PyPI (with optional `[extras]`). - Your FastAPI app is in `api_dir/` (will be COPYed into the image). - Your React UI is pre-built in `ui_dist_dir/` (will be COPYed in). Why pre-build the UI locally instead of in the Dockerfile? Because `package-lock.json` may have `file:` deps to a local monorepo that doesn’t exist on the build machine. Pre-building sidesteps the issue. * **Parameters:** * **package** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – PyPI package name. * **extras** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – Extras spec (without brackets), e.g. `"web"` → `package[web]`. Pass empty string to omit. * **version_spec** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – PEP 440 version specifier, e.g. `"~=0.1.1"`. Empty string means “latest”. * **python_version** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – Python base image tag (`"3.12"` → `python:3.12-slim`). * **api_module** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – `module.path:app_attr` for uvicorn. * **port** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Port to expose (HF Spaces uses 7860 for Docker SDK). * **api_dir** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – Local path of the FastAPI source, relative to the Space repo root. * **ui_dist_dir** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – Local path of the pre-built React dist, relative to the Space repo root. * **extra_run_lines** ([`Iterable`](https://docs.python.org/3/library/typing.html#typing.Iterable)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – Additional `RUN` lines to insert (e.g., to pre-cache datasets so cold start is sub-second). * **Return type:** [`str`](https://docs.python.org/3/builtins/stdtypes.html#str) * **Returns:** The Dockerfile contents as a string. ### hfdol.deploy.render_space_readme(title, , color_from='blue', color_to='green', sdk='docker', port=7860, pinned=False, body='') Render an HF Space README with the YAML frontmatter HF requires. The frontmatter sets the Space’s metadata (sdk, port, etc.) — required even for Docker SDK Spaces. Body is appended after. * **Parameters:** * **title** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – Display title for the Space. * **color_from** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – Gradient start color (HF picks from a small palette). * **color_to** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – Gradient end color. * **sdk** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – `"docker"`, `"gradio"`, `"streamlit"`, `"static"`. * **port** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – `app_port` (Docker SDK uses this; ignored otherwise). * **pinned** ([`bool`](https://docs.python.org/3/builtins/functions.html#bool)) – Whether to pin to the user’s HF profile. * **body** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – Markdown content to append after the frontmatter. * **Return type:** [`str`](https://docs.python.org/3/builtins/stdtypes.html#str) ### hfdol.deploy.stage_webapp(, staging_dir, api_src, ui_dist_src, api_dir_in_space='webapp/api', ui_dist_dir_in_space='webapp/ui/dist', dockerfile_text=None, readme_text=None, extra_files=None) Lay out the contents of a Space repo in a local staging directory. Idempotent: re-running replaces the API and UI dirs but reuses what’s already there for things you didn’t pass (e.g., an existing Dockerfile). * **Parameters:** * **staging_dir** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str) | [`Path`](https://docs.python.org/3/library/pathlib.html#pathlib.Path)) – Where to assemble the Space contents. * **api_src** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str) | [`Path`](https://docs.python.org/3/library/pathlib.html#pathlib.Path)) – Local source directory for the FastAPI app. * **ui_dist_src** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str) | [`Path`](https://docs.python.org/3/library/pathlib.html#pathlib.Path)) – Local source directory for the pre-built React dist. * **api_dir_in_space** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – Path inside the Space repo for the API. * **ui_dist_dir_in_space** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – Path inside the Space repo for the UI dist. * **dockerfile_text** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – Dockerfile contents (use [`render_pypi_webapp_dockerfile()`](_autosummary/hfdol.deploy.html.md#hfdol.deploy.render_pypi_webapp_dockerfile)). If None, only writes one if the staging dir doesn’t already have it. * **readme_text** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – README contents (use [`render_space_readme()`](_autosummary/hfdol.deploy.html.md#hfdol.deploy.render_space_readme)). Same conditional behavior as `dockerfile_text`. * **extra_files** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`dict`](https://docs.python.org/3/builtins/stdtypes.html#dict)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str), [`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]]) – `{relative_path: contents}` extras (e.g., `.gitignore`). * **Return type:** [`Path`](https://docs.python.org/3/library/pathlib.html#pathlib.Path) * **Returns:** The staging directory path. ### hfdol.deploy.upload_app_dir(repo_id, folder_path, , message='Update app contents', token=None, ignore_patterns=('node_modules/\*\*', '_\_pycache_\_/\*\*', '\*.pyc', '.DS_Store', '.git/\*\*'), delete_patterns=None) Upload a local folder to an HF Space, replacing the live files. Wraps `huggingface_hub.HfApi.upload_folder()` with sensible ignore patterns and a printed plan. Existing files on the Space that are not in the local folder are NOT deleted unless you pass `delete_patterns`. * **Parameters:** * **repo_id** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – `owner/space-name`. * **folder_path** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str) | [`Path`](https://docs.python.org/3/library/pathlib.html#pathlib.Path)) – Local directory to upload. * **message** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – Commit message on the Space repo. * **token** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – Write token; defaults to [`ensure_write_token()`](_autosummary/hfdol.deploy.html.md#hfdol.deploy.ensure_write_token). * **ignore_patterns** ([`Iterable`](https://docs.python.org/3/library/typing.html#typing.Iterable)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – Glob patterns to skip. * **delete_patterns** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`Iterable`](https://docs.python.org/3/library/typing.html#typing.Iterable)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]]) – Optional glob patterns of remote files to delete before upload (e.g., `["webapp/ui/dist/**"]` to ensure stale build artifacts are removed). * **Return type:** [`None`](https://docs.python.org/3/builtins/constants.html#None) ### hfdol.deploy.wait_for_build(repo_id, , token=None, timeout=900, poll_interval=15, on_stage_change=None) Poll Space status until it lands in a terminal state or times out. Terminal states: `RUNNING`, `RUNNING_BUILDING` (success); `BUILD_ERROR`, `RUNTIME_ERROR` (failure). Times out after `timeout` seconds. * **Parameters:** * **repo_id** ([`str`](https://docs.python.org/3/builtins/stdtypes.html#str)) – `owner/space-name`. * **token** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str)]) – Write token (read access also fine); defaults to env. * **timeout** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Max seconds to wait. * **poll_interval** ([`int`](https://docs.python.org/3/builtins/functions.html#int)) – Seconds between polls. * **on_stage_change** ([`Optional`](https://docs.python.org/3/library/typing.html#typing.Optional)[[`Callable`](https://docs.python.org/3/library/typing.html#typing.Callable)[[[`str`](https://docs.python.org/3/builtins/stdtypes.html#str), [`int`](https://docs.python.org/3/builtins/functions.html#int)], [`None`](https://docs.python.org/3/builtins/constants.html#None)]]) – Callback `(stage, elapsed_seconds) -> None` invoked each time the stage transitions. Default prints to stdout. * **Return type:** [`BuildResult`](_autosummary/hfdol.deploy.html.md#hfdol.deploy.BuildResult) * **Returns:** [`BuildResult`](_autosummary/hfdol.deploy.html.md#hfdol.deploy.BuildResult). # _autosummary/hfdol.html.md # hfdol Huggingface interface ### Modules | [`base`](_autosummary/hfdol.base.html.md#module-hfdol.base) | Base functionality for hfdol. | |-----------------------------------------------------------------------------|---------------------------------------| | [`deploy`](_autosummary/hfdol.deploy.html.md#module-hfdol.deploy) | Deploy webapps to HuggingFace Spaces. | # about-this-build.html.md # About this build This documentation was built on **2026-09-22 14:36 UTC** from commit d634c6b on branch main, for **hfdol 0.1.20** (from pyproject.toml). #### WARNING The documentation and the package may be misaligned: - The documented version (0.1.20) is behind the latest release on PyPI (0.1.21): `pip install hfdol` gives newer code than these docs describe. ## Source | | | |---------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------| | Commit | d634c6bb261f8d7a3702e769bcade2c86c8d08a6 | | Branch | main | | Tags at this commit | none | | Working tree | clean | | Remote | https://github.com/thorwhalen/hfdol | ## Continuous integration | | | |--------------|--------------------------------------------------------------------------------------------| | Repository | thorwhalen/hfdol | | Run | 35741246649 | | Ref | refs/heads/main | | Event commit | d634c6bb261f8d7a3702e769bcade2c86c8d08a6 (in the history of the built commit) | ## Tools | | | |----------|---------| | epythet | 0.2.12 | | Sphinx | 9.1.0 | | docutils | 0.22.4 | | Python | 3.12.14 | ## Configuration as resolved | | | |---------------|-------------------------------------------------------------------| | theme | auto (Sphinx theme sphinxawesome_theme) | | accent | #536000 | | api_generator | autosummary | | ignore | tests/, scrap/, examples/ | | agent_outputs | true | | aggregates | md | | ai_artifacts | true | ## Package on PyPI Latest release: 0.1.21, newer than the documented version (0.1.20). ## Reproduce ```bash git clone https://github.com/thorwhalen/hfdol && cd hfdol git checkout d634c6bb261f8d7a3702e769bcade2c86c8d08a6 pip install "epythet==0.2.12" epythet quickstart . --ignore tests/ scrap/ examples/ ``` The same data, for machines: build_info.json (schema version 1). # ai-agents.html.md # For AI agents `hfdol` ships artifacts for coding agents alongside its code. This page lists them, says where each lives in the repository, and points at the machine-readable copies of this documentation. ## Skills Skills are folders holding a `SKILL.md` (the [Agent Skills](https://agentskills.io) format): a description that tells an agent when to use it and a body with the procedure. Install one into your agent with `gh skill` (any host: `--agent claude-code`, `copilot`, `cursor`, `codex`, `gemini`), or use the copy bundled in the wheel. ### `hf-space-deploy` Deploy a Python webapp (FastAPI backend + frontend) to a Hugging Face Space using the Docker SDK. Use when the user wants to “publish to HF”, “put this on Hugging Face Spaces”, “ship a Space”, “host the app on HF”, “make a Space for X”, or “deploy a Gradio/FastAPI/Streamlit app to Hugging Face.” Also triggers on rebuild/refresh requests for an existing Space, factory reboots, swapping out a Space’s PyPI version pin, or fixing build failures. The skill assumes the package is on PyPI; if it isn’t yet, suggest publishing first (or ship source via the Dockerfile). Source: [`.claude/skills/hf-space-deploy`](https://github.com/thorwhalen/hfdol/tree/HEAD/.claude/skills/hf-space-deploy). ## Machine-readable documentation This site publishes the same documentation in forms that fit an agent’s context window: - [`llms.txt`](https://thorwhalen.github.io/hfdol/llms.txt): an index of every page with a one-line description ([llms.txt](https://llmstxt.org) format) - [`hfdol.md`](https://thorwhalen.github.io/hfdol/hfdol.md): the whole documentation as one Markdown file - `.html.md`: a rendered Markdown twin of every page, advertised from each page’s `` with `` - [`objects.inv`](https://thorwhalen.github.io/hfdol/objects.inv): the Sphinx inventory: a symbol-to-URL index (`sphobjinv convert plain objects.inv -`) # api.html.md # API reference | [`hfdol`](_autosummary/hfdol.html.md#module-hfdol) | Huggingface interface | |-----------------------------------------------------------------------|-------------------------|