Skip to content

aimbat.utils

Miscellaneous helpers for AIMBAT.

Covers five areas:

  • Maths — mean/SEM over numeric and pd.Timedelta values (mean_and_sem, mean_and_sem_timedelta).
  • Errors — exception message plus attached notes, for display (exception_message).
  • Pydantic — validation-error formatting, title-map lookup, and the serialisation shared by the core.dump_*_table functions (format_validation_error, get_title_map, dump_models).
  • Sample data — download and delete the bundled sample dataset (download_sampledata, delete_sampledata).
  • UUIDs / formatting — short-UUID lookup, rel() for typed relationships, and the fmt_* display formatters (utils.formatters).

Modules:

Name Description
formatters

Formatters for displaying values in tables and panels.

Functions:

Name Description
check_field_name_flags

Check the field-naming flags the dump_*_table functions take.

delete_sampledata

Delete the sample data directory (Settings.sampledata_dir) and its contents.

download_sampledata

Download and extract the AIMBAT sample dataset.

dump_models

Serialise records to a JSON-compatible list of dicts.

exception_message

Return an exception's message together with any notes attached to it.

format_validation_error

Format a ValidationError as a short, semicolon-joined message.

get_title_map

Build a mapping from field names to their display titles.

mean_and_sem

Return the mean and standard error of the mean (SEM) for a list of numeric values, ignoring None values.

mean_and_sem_timedelta

Return (mean, sem) for a list of pd.Timedelta values.

rel

Cast a SQLModel relationship attribute to QueryableAttribute for use with selectinload.

string_to_uuid

Determine a UUID from a string containing the first few characters.

uuid_shortener

Return the shortest unique prefix for a UUID, formatted with dashes.

check_field_name_flags

check_field_name_flags(
    by_alias: bool,
    by_title: bool,
    from_read_model: bool | None = None,
) -> None

Check the field-naming flags the dump_*_table functions take.

Parameters:

Name Type Description Default
by_alias bool

Whether serialisation aliases are to be used for the field names.

required
by_title bool

Whether field title metadata is to be used for the field names.

required
from_read_model bool | None

Whether the caller is dumping from its read model, for the functions that offer the choice. None for the ones that don't.

None

Raises:

Type Description
ValueError

If both by_alias and by_title are True.

ValueError

If by_title is True but from_read_model is False.

Source code in src/aimbat/utils/_pydantic.py
def check_field_name_flags(
    by_alias: bool, by_title: bool, from_read_model: bool | None = None
) -> None:
    """Check the field-naming flags the `dump_*_table` functions take.

    Args:
        by_alias: Whether serialisation aliases are to be used for the field names.
        by_title: Whether field title metadata is to be used for the field names.
        from_read_model: Whether the caller is dumping from its read model, for
            the functions that offer the choice. `None` for the ones that don't.

    Raises:
        ValueError: If both `by_alias` and `by_title` are True.
        ValueError: If `by_title` is True but `from_read_model` is False.
    """
    if by_alias and by_title:
        raise ValueError("Arguments 'by_alias' and 'by_title' are mutually exclusive.")

    if from_read_model is False and by_title:
        raise ValueError("'by_title' is only supported when 'from_read_model' is True.")

delete_sampledata

delete_sampledata() -> None

Delete the sample data directory (Settings.sampledata_dir) and its contents.

Raises:

Type Description
ValueError

If Settings.sampledata_dir looks unsafe to delete (see _check_safe_to_delete).

Source code in src/aimbat/utils/_sampledata.py
def delete_sampledata() -> None:
    """Delete the sample data directory (`Settings.sampledata_dir`) and its contents.

    Raises:
        ValueError: If `Settings.sampledata_dir` looks unsafe to delete (see
            `_check_safe_to_delete`).
    """

    _check_safe_to_delete(settings.sampledata_dir)

    logger.info(f"Deleting sample data in {settings.sampledata_dir}.")

    shutil.rmtree(settings.sampledata_dir)

download_sampledata

download_sampledata(force: bool = False) -> None

Download and extract the AIMBAT sample dataset.

Downloads the sample data archive and extracts it into Settings.sampledata_dir.

Parameters:

Name Type Description Default
force bool

Delete and replace an existing non-empty sample data directory instead of raising an error.

False

Raises:

Type Description
FileExistsError

If the sample data directory already exists and is non-empty, and force is False.

Source code in src/aimbat/utils/_sampledata.py
def download_sampledata(force: bool = False) -> None:
    """Download and extract the AIMBAT sample dataset.

    Downloads the sample data archive and extracts it into
    `Settings.sampledata_dir`.

    Args:
        force: Delete and replace an existing non-empty sample data
            directory instead of raising an error.

    Raises:
        FileExistsError: If the sample data directory already exists and is
            non-empty, and `force` is False.
    """

    logger.info(
        f"Downloading sample data from {_SAMPLEDATA_SRC} to {settings.sampledata_dir}."
    )

    if (
        settings.sampledata_dir.exists()
        and len(os.listdir(settings.sampledata_dir)) != 0
    ):
        if force is True:
            delete_sampledata()
        else:
            raise FileExistsError(
                f"The directory {settings.sampledata_dir} already exists and is "
                + "non-empty."
            )

    with urlopen(_SAMPLEDATA_SRC, timeout=_DOWNLOAD_TIMEOUT_SECONDS) as zipresp:
        with tempfile.NamedTemporaryFile() as tmpfile:
            shutil.copyfileobj(zipresp, tmpfile)
            tmpfile.flush()
            tmpfile.seek(0)
            logger.debug(f"Extracting sample data to {settings.sampledata_dir}.")
            with ZipFile(tmpfile) as zfile:
                _extract_regular_files(zfile, settings.sampledata_dir)

    logger.info("Sample data downloaded and extracted successfully.")

dump_models

dump_models(
    records: Sequence[T],
    model_class: type[T],
    *,
    by_alias: bool = False,
    by_title: bool = False,
    exclude: set[str] | None = None,
) -> list[dict[str, Any]]

Serialise records to a JSON-compatible list of dicts.

The shared tail of the core.dump_*_table functions: each builds its own query, and its read models where it has them, then hands the result here so field naming and exclusions are applied the same way everywhere.

Parameters:

Name Type Description Default
records Sequence[T]

Records to serialise, all instances of model_class.

required
model_class type[T]

Class of the records, used for both the type adapter and the title map.

required
by_alias bool

Whether to use serialisation aliases for the field names.

False
by_title bool

Whether to use the field title metadata for the field names. Mutually exclusive with by_alias.

False
exclude set[str] | None

Field names to exclude from every record.

None

Returns:

Type Description
list[dict[str, Any]]

One dict per record.

Raises:

Type Description
ValueError

If both by_alias and by_title are True.

Source code in src/aimbat/utils/_pydantic.py
def dump_models[T: BaseModel](
    records: Sequence[T],
    model_class: type[T],
    *,
    by_alias: bool = False,
    by_title: bool = False,
    exclude: set[str] | None = None,
) -> list[dict[str, Any]]:
    """Serialise records to a JSON-compatible list of dicts.

    The shared tail of the `core.dump_*_table` functions: each builds its own
    query, and its read models where it has them, then hands the result here
    so field naming and exclusions are applied the same way everywhere.

    Args:
        records: Records to serialise, all instances of `model_class`.
        model_class: Class of the records, used for both the type adapter and
            the title map.
        by_alias: Whether to use serialisation aliases for the field names.
        by_title: Whether to use the field title metadata for the field names.
            Mutually exclusive with `by_alias`.
        exclude: Field names to exclude from every record.

    Returns:
        One dict per record.

    Raises:
        ValueError: If both `by_alias` and `by_title` are True.
    """
    check_field_name_flags(by_alias, by_title)

    exclude_spec: dict[str, set[str]] | None = {"__all__": exclude} if exclude else None

    adapter: TypeAdapter[Sequence[T]] = TypeAdapter(Sequence[model_class])  # type: ignore[valid-type]
    data: list[dict[str, Any]] = adapter.dump_python(
        records, mode="json", by_alias=by_alias, exclude=exclude_spec
    )

    if by_title:
        title_map = get_title_map(model_class)
        return [{title_map.get(k, k): v for k, v in row.items()} for row in data]

    return data

exception_message

exception_message(exc: BaseException) -> str

Return an exception's message together with any notes attached to it.

Code that knows context the raising site does not - which data source was being read, say - attaches it with BaseException.add_note instead of re-raising a different exception type. Notes only show up in a traceback, so anything that reports an exception without one has to add them back.

Parameters:

Name Type Description Default
exc BaseException

The exception to describe.

required

Returns:

Type Description
str

The exception's message, followed by one line per note.

Source code in src/aimbat/utils/_errors.py
def exception_message(exc: BaseException) -> str:
    """Return an exception's message together with any notes attached to it.

    Code that knows context the raising site does not - which data source was
    being read, say - attaches it with `BaseException.add_note` instead of
    re-raising a different exception type. Notes only show up in a traceback,
    so anything that reports an exception without one has to add them back.

    Args:
        exc: The exception to describe.

    Returns:
        The exception's message, followed by one line per note.
    """
    return "\n".join([str(exc), *getattr(exc, "__notes__", [])])

format_validation_error

format_validation_error(exc: ValidationError) -> str

Format a ValidationError as a short, semicolon-joined message.

Strips the redundant "Value error, " prefix Pydantic adds to custom validator messages, so the result is fit for display without leaking implementation detail.

Parameters:

Name Type Description Default
exc ValidationError

The validation error to format.

required

Returns:

Type Description
str

A single-line, semicolon-joined summary of all validation errors.

Source code in src/aimbat/utils/_pydantic.py
def format_validation_error(exc: ValidationError) -> str:
    """Format a `ValidationError` as a short, semicolon-joined message.

    Strips the redundant `"Value error, "` prefix Pydantic adds to custom
    validator messages, so the result is fit for display without leaking
    implementation detail.

    Args:
        exc: The validation error to format.

    Returns:
        A single-line, semicolon-joined summary of all validation errors.
    """
    return "; ".join(e["msg"].removeprefix("Value error, ") for e in exc.errors())

get_title_map cached

get_title_map(
    model_class: type[BaseModel],
) -> dict[str, str]

Build a mapping from field names to their display titles.

Includes both regular and computed fields. A field without a title set falls back to its name with underscores replaced by spaces.

Parameters:

Name Type Description Default
model_class type[BaseModel]

Pydantic model class to inspect.

required

Returns:

Type Description
dict[str, str]

Mapping of field name to display title.

Source code in src/aimbat/utils/_pydantic.py
@cache
def get_title_map(model_class: type[BaseModel]) -> dict[str, str]:
    """Build a mapping from field names to their display titles.

    Includes both regular and computed fields. A field without a `title` set
    falls back to its name with underscores replaced by spaces.

    Args:
        model_class: Pydantic model class to inspect.

    Returns:
        Mapping of field name to display title.
    """
    mapping: dict[str, str] = {}

    for name, info in model_class.model_fields.items():
        mapping[name] = info.title or name.replace("_", " ")

    computed_fields: dict[str, Any] = getattr(
        model_class, "__pydantic_computed_fields__", {}
    )
    for name, info in computed_fields.items():
        title = getattr(info, "title", None)
        mapping[name] = title or name.replace("_", " ")

    return mapping

mean_and_sem

mean_and_sem(
    data: Sequence[float | None],
) -> tuple[float | None, float | None]

Return the mean and standard error of the mean (SEM) for a list of numeric values, ignoring None values.

Parameters:

Name Type Description Default
data Sequence[float | None]

List of numeric values (float or int) or None.

required

Returns:

Type Description
tuple[float | None, float | None]

A tuple containing the mean and SEM of the input data, both as floats or None if not computable.

Source code in src/aimbat/utils/_maths.py
def mean_and_sem(
    data: Sequence[float | None],
) -> tuple[float | None, float | None]:
    """Return the mean and standard error of the mean (SEM) for a list of numeric values, ignoring None values.

    Args:
        data: List of numeric values (float or int) or None.

    Returns:
        A tuple containing the mean and SEM of the input data, both as floats or None if not computable.
    """
    return _mean_and_sem_tuple(tuple(data))

mean_and_sem_timedelta

mean_and_sem_timedelta(
    values: Sequence[Timedelta],
) -> tuple[Timedelta | None, Timedelta | None]

Return (mean, sem) for a list of pd.Timedelta values.

Parameters:

Name Type Description Default
values Sequence[Timedelta]

List of pd.Timedelta values.

required

Returns:

Type Description
tuple[Timedelta | None, Timedelta | None]

(None, None) when empty. SEM is None for fewer than two values.

Source code in src/aimbat/utils/_maths.py
def mean_and_sem_timedelta(
    values: Sequence[pd.Timedelta],
) -> tuple[pd.Timedelta | None, pd.Timedelta | None]:
    """Return (mean, sem) for a list of pd.Timedelta values.

    Args:
        values: List of pd.Timedelta values.

    Returns:
        `(None, None)` when empty. SEM is `None` for fewer than two values.
    """
    if not values:
        return None, None
    ns_vals = [float(td.value) for td in values]
    mean_ns, sem_ns = mean_and_sem(ns_vals)
    return (
        pd.Timedelta(int(mean_ns), unit="ns") if mean_ns is not None else None,
        pd.Timedelta(int(sem_ns), unit="ns") if sem_ns is not None else None,
    )

rel

rel(attr: Any) -> QueryableAttribute[Any]

Cast a SQLModel relationship attribute to QueryableAttribute for use with selectinload.

SQLModel types relationship fields as their Python collection type (e.g. list[Foo]), but SQLAlchemy's selectinload expects a QueryableAttribute. This helper performs the cast to satisfy mypy without requiring per-call # type: ignore comments.

Source code in src/aimbat/utils/_sqlalchemy.py
def rel(attr: Any) -> QueryableAttribute[Any]:
    """Cast a SQLModel relationship attribute to `QueryableAttribute` for use with `selectinload`.

    SQLModel types relationship fields as their Python collection type (e.g. `list[Foo]`),
    but SQLAlchemy's `selectinload` expects a `QueryableAttribute`. This helper performs
    the cast to satisfy mypy without requiring per-call `# type: ignore` comments.
    """
    return cast(QueryableAttribute[Any], attr)

string_to_uuid

string_to_uuid(
    session: Session,
    id: str,
    aimbat_class: type[AimbatTypes],
    custom_error: str | None = None,
) -> UUID

Determine a UUID from a string containing the first few characters.

Parameters:

Name Type Description Default
session Session

Database session.

required
id str

Input string to find UUID for.

required
aimbat_class type[AimbatTypes]

Aimbat class to use to find UUID.

required
custom_error str | None

Overrides the default error message.

None

Returns:

Type Description
UUID

The full UUID.

Raises:

Type Description
ValueError

If no record matches id, or more than one record does.

Source code in src/aimbat/utils/_uuid.py
def string_to_uuid(
    session: Session,
    id: str,
    aimbat_class: type[AimbatTypes],
    custom_error: str | None = None,
) -> UUID:
    """Determine a UUID from a string containing the first few characters.

    Args:
        session: Database session.
        id: Input string to find UUID for.
        aimbat_class: Aimbat class to use to find UUID.
        custom_error: Overrides the default error message.

    Returns:
        The full UUID.

    Raises:
        ValueError: If no record matches `id`, or more than one record does.
    """
    statement = select(aimbat_class.id).where(
        func.replace(cast(aimbat_class.id, String), "-", "").like(
            f"{_escape_like(id.replace('-', ''))}%", escape=_LIKE_ESCAPE_CHAR
        )
    )
    uuid_set = set(session.exec(statement).all())
    if len(uuid_set) == 1:
        resolved = uuid_set.pop()
        logger.debug(f"Resolved {id} to UUID: {resolved}")
        return resolved
    if len(uuid_set) == 0:
        raise ValueError(
            custom_error or f"Unable to find {aimbat_class.__name__} using id: {id}."
        )
    raise ValueError(f"Found more than one {aimbat_class.__name__} using id: {id}")

uuid_shortener

uuid_shortener(
    session: Session,
    aimbat_obj: T | type[T],
    min_length: int | None = None,
    str_uuid: str | None = None,
) -> str

Return the shortest unique prefix for a UUID, formatted with dashes.

The table's ids are fetched once per session and cached there, so calling this once per row (or once per cell) stays a single query - see _id_pool for what that caching does and does not guarantee.

Parameters:

Name Type Description Default
session Session

An active SQLModel/SQLAlchemy session.

required
aimbat_obj T | type[T]

Either an instance of a SQLModel or the SQLModel class itself.

required
min_length int | None

The starting character length for the shortened ID. Defaults to Settings.min_id_length.

None
str_uuid str | None

The full UUID string. Required only if aimbat_obj is a class.

None

Returns:

Type Description
str

The shortest unique prefix string, including hyphens where applicable.

Raises:

Type Description
ValueError

If aimbat_obj is a class and str_uuid is not provided, or if the resolved UUID has no matching record in the table.

Source code in src/aimbat/utils/_uuid.py
def uuid_shortener[T: AimbatTypes](
    session: Session,
    aimbat_obj: T | type[T],
    min_length: int | None = None,
    str_uuid: str | None = None,
) -> str:
    """Return the shortest unique prefix for a UUID, formatted with dashes.

    The table's ids are fetched once per session and cached there, so calling
    this once per row (or once per cell) stays a single query - see
    `_id_pool` for what that caching does and does not guarantee.

    Args:
        session: An active SQLModel/SQLAlchemy session.
        aimbat_obj: Either an instance of a SQLModel or the SQLModel class itself.
        min_length: The starting character length for the shortened ID.
            Defaults to `Settings.min_id_length`.
        str_uuid: The full UUID string. Required only if `aimbat_obj` is a class.

    Returns:
        The shortest unique prefix string, including hyphens where applicable.

    Raises:
        ValueError: If `aimbat_obj` is a class and `str_uuid` is not
            provided, or if the resolved UUID has no matching record in the
            table.
    """

    if min_length is None:
        min_length = settings.min_id_length

    if isinstance(aimbat_obj, type):
        model_class = aimbat_obj
        if str_uuid is None:
            raise ValueError("str_uuid must be provided when aimbat_obj is a class.")
        target_full = str(UUID(str_uuid))
    else:
        model_class = type(aimbat_obj)
        target_full = str(aimbat_obj.id)

    # Compare on the dash-free form throughout.
    target_clean = target_full.replace("-", "")

    pool = _id_pool(session, model_class, target_clean)
    if not _contains(pool, target_clean):
        raise ValueError(f"ID {target_full} not found in table {model_class.__name__}")

    length = min(
        max(min_length, _unique_prefix_length(pool, target_clean)), len(target_clean)
    )
    candidate = _dashed_prefix(target_full, length)
    logger.debug(f"Shortened {target_full} to: {candidate}")
    return candidate

formatters

Formatters for displaying values in tables and panels.

Functions:

Name Description
fmt_bool

Format a boolean as ✓ (True) or empty string (False/None).

fmt_depth_km

Format a depth value in metres as kilometres with one decimal place.

fmt_flip

Format a boolean flip flag as ↕ (True) or empty string (False).

fmt_float

Format a float to 3 decimal places, or — for a null value (None/NaN/NaT/NA).

fmt_float_sem

Format a float with an optional SEM as value ± sem, or — if v is None.

fmt_timedelta

Format a Timedelta as seconds, or — if None.

fmt_timedelta_sem

Format a Timedelta mean with an optional SEM as mean ± sem (in seconds).

fmt_timestamp

Format a timestamp as YYYY-MM-DD HH:MM:SS, or — for missing values.

fmt_bool

fmt_bool(val: bool | object) -> str

Format a boolean as ✓ (True) or empty string (False/None).

Source code in src/aimbat/utils/formatters.py
def fmt_bool(val: bool | object) -> str:
    """Format a boolean as `✓` (True) or empty string (False/None)."""
    return "✓" if isinstance(val, (bool, np.bool_)) and val else ""

fmt_depth_km

fmt_depth_km(val: int | float | object) -> str

Format a depth value in metres as kilometres with one decimal place.

Source code in src/aimbat/utils/formatters.py
def fmt_depth_km(val: int | float | object) -> str:
    """Format a depth value in metres as kilometres with one decimal place."""
    if isinstance(val, (int, float)) and not isinstance(val, bool):
        return f"{val / 1000:.1f}"
    return str(val)

fmt_flip

fmt_flip(val: bool | object) -> str

Format a boolean flip flag as ↕ (True) or empty string (False).

Source code in src/aimbat/utils/formatters.py
def fmt_flip(val: bool | object) -> str:
    """Format a boolean flip flag as `↕` (True) or empty string (False)."""
    if isinstance(val, (bool, np.bool_)):
        return "↕" if val else ""
    return str(val)

fmt_float

fmt_float(val: float | object) -> str

Format a float to 3 decimal places, or — for a null value (None/NaN/NaT/NA).

Source code in src/aimbat/utils/formatters.py
def fmt_float(val: float | object) -> str:
    """Format a float to 3 decimal places, or ` — ` for a null value (None/NaN/NaT/NA)."""
    if isnull(cast(Any, val)):
        return _MISSING_MARKER
    if isinstance(val, float):
        return f"{val:.3f}"
    return str(val)

fmt_float_sem

fmt_float_sem(
    v: float | None, sem: float | None, decimals: int = 4
) -> str

Format a float with an optional SEM as value ± sem, or — if v is None.

Source code in src/aimbat/utils/formatters.py
def fmt_float_sem(v: float | None, sem: float | None, decimals: int = 4) -> str:
    """Format a float with an optional SEM as `value ± sem`, or `—` if `v` is `None`."""
    if v is None:
        return _MISSING
    if sem is not None:
        return f"{v:.{decimals}f} ± {sem:.{decimals}f}"
    return f"{v:.{decimals}f}"

fmt_timedelta

fmt_timedelta(
    td: Timedelta | None, decimals: int = 4
) -> str

Format a Timedelta as seconds, or — if None.

Source code in src/aimbat/utils/formatters.py
def fmt_timedelta(td: Timedelta | None, decimals: int = 4) -> str:
    """Format a `Timedelta` as seconds, or `—` if `None`."""
    if td is None:
        return _MISSING
    return f"{td.total_seconds():.{decimals}f} s"

fmt_timedelta_sem

fmt_timedelta_sem(
    mean: Timedelta | None,
    sem: Timedelta | None,
    decimals: int = 4,
) -> str

Format a Timedelta mean with an optional SEM as mean ± sem (in seconds).

Returns — if mean is null (None/NaT/NA).

Source code in src/aimbat/utils/formatters.py
def fmt_timedelta_sem(
    mean: Timedelta | None, sem: Timedelta | None, decimals: int = 4
) -> str:
    """Format a `Timedelta` mean with an optional SEM as `mean ± sem` (in seconds).

    Returns `—` if `mean` is null (None/NaT/NA).
    """
    if isnull(cast(Any, mean)):
        return _MISSING
    assert mean is not None
    s = mean.total_seconds()
    if sem is not None:
        return f"{s:.{decimals}f} ± {sem.total_seconds():.{decimals}f} s"
    return f"{s:.{decimals}f} s"

fmt_timestamp

fmt_timestamp(val: Any) -> str

Format a timestamp as YYYY-MM-DD HH:MM:SS, or — for missing values.

Source code in src/aimbat/utils/formatters.py
def fmt_timestamp(val: Any) -> str:
    """Format a timestamp as `YYYY-MM-DD HH:MM:SS`, or ` — ` for missing values."""
    if isinstance(val, str) and val.strip():
        try:
            val = to_datetime(val)
        except (ValueError, TypeError):
            return str(val)
    if val is None or val is NaT or (isinstance(val, str) and val.strip() == ""):
        return _MISSING_MARKER
    if hasattr(val, "strftime"):
        return val.strftime("%Y-%m-%d %H:%M:%S")
    return str(val)