snoozeml.models — Model Metadata & Lifecycle Management#

Model metadata, operational state representations, and lifecycle management for SnoozeML.

class snoozeml.models.ModelState(*values)[source]#

Bases: str, Enum

Operational lifecycle state of a registered model within SnoozeML.

IDLE = 'idle'#

Configuration registered, but weights are deallocated (scaled to zero).

LOADING = 'loading'#

Model weights are currently being loaded into host RAM or device VRAM.

READY = 'ready'#

Weights are resident in memory and ready for immediate inference.

BUSY = 'busy'#

Inference forward pass is actively executing.

classmethod from_str(value)[source]#

Parses a case-insensitive state string into a ModelState.

Parameters:

value (str)

Return type:

ModelState

class snoozeml.models.ModelInfo(id, state, onnx_path, idle_timeout_secs=300, max_batch_size=16, max_batch_delay_ms=10, last_accessed_secs_ago=0, description=None, pinned=False, memory_bytes=0)[source]#

Bases: object

Metadata and live operational state for a registered model in SnoozeML.

Parameters:

  • id (str)

  • state (ModelState)

  • onnx_path (str)

  • idle_timeout_secs (int)

  • max_batch_size (int)

  • max_batch_delay_ms (int)

  • last_accessed_secs_ago (int)

  • description (str | None)

  • pinned (bool)

  • memory_bytes (int)

id#

Unique model identifier.

Type:

str

state#

Operational status (ModelState).

Type:

snoozeml.models.ModelState

onnx_path#

Filesystem or URI path to model weights.

Type:

str

idle_timeout_secs#

Inactivity threshold in seconds before automatic unloading.

Type:

int

max_batch_size#

Maximum batch size for dynamic collation along Axis(0).

Type:

int

max_batch_delay_ms#

Maximum latency budget in milliseconds for batch accumulation.

Type:

int

last_accessed_secs_ago#

Seconds elapsed since the model last served an inference request.

Type:

int

description#

Optional human-readable description of the model.

Type:

str | None

pinned#

Whether the model is pinned in memory and exempt from automatic eviction.

Type:

bool

memory_bytes#

Optional estimated memory footprint in bytes (0 if not reported).

Type:

int

id: str#
state: ModelState#
onnx_path: str#
idle_timeout_secs: int = 300#
max_batch_size: int = 16#
max_batch_delay_ms: int = 10#
last_accessed_secs_ago: int = 0#
description: str | None = None#
pinned: bool = False#
memory_bytes: int = 0#
property is_loaded: bool#

Returns True if model weights are currently resident in memory (ready or busy).

property path: str#

Alias for onnx_path.

classmethod from_dict(data)[source]#

Constructs a ModelInfo from a JSON dictionary returned by GET /v1/models.

Parameters:

data (Mapping[str, Any]) – Dictionary parsed from the SnoozeML REST API.

Returns:

Populated ModelInfo instance.

Return type:

ModelInfo

to_dict()[source]#

Serializes this ModelInfo back into a plain Python dictionary.

Return type:

Dict[str, Any]

class snoozeml.models.ModelsManager(client)[source]#

Bases: object

Synchronous manager for querying and controlling model lifecycles on a SnoozeML daemon.

Accessed via snoozeml.SnoozeClient.models.

Parameters:

client (SnoozeClient)

list()[source]#

Lists all registered models and their current lifecycle telemetry.

Returns:

List of ModelInfo objects.

Raises:

SnoozeError – On authentication, permission, or connection errors.

Return type:

List[ModelInfo]

get(model_id)[source]#

Retrieves metadata and operational state for a specific model.

Parameters:

model_id (str) – Unique identifier of the model to inspect.

Returns:

Matching ModelInfo instance.

Raises:
Return type:

ModelInfo

load(model_id, *, pin=False)[source]#

Eagerly loads a model’s weights into memory ahead of traffic.

Parameters:

  • model_id (str) – Target model identifier.

  • pin (bool) – If True, also pins the model in memory so it is exempt from automatic scale-to-zero and LRU eviction.

Returns:

Status dictionary returned by the server.

Return type:

Dict[str, Any]

unload(model_id)[source]#

Explicitly unloads (deallocates) a model from memory (scale-to-zero).

Parameters:

model_id (str) – Target model identifier.

Returns:

Status dictionary returned by the server.

Return type:

Dict[str, Any]

evict(model_id)[source]#

Alias for unload() — explicitly evicts/snoozes a model from memory.

Parameters:

model_id (str) – Target model identifier.

Returns:

Status dictionary returned by the server.

Return type:

Dict[str, Any]

pin(model_id)[source]#

Pins a model in memory so it is eagerly loaded and exempt from automatic eviction.

Parameters:

model_id (str) – Target model identifier.

Returns:

Status dictionary returned by the server (e.g. {"model_id": ..., "status": "pinned", "pinned": True}).

Return type:

Dict[str, Any]

unpin(model_id)[source]#

Unpins a model so it returns to normal scale-to-zero and LRU lifecycle management.

Parameters:

model_id (str) – Target model identifier.

Returns:

Status dictionary returned by the server (e.g. {"model_id": ..., "status": "unpinned", "pinned": False}).

Return type:

Dict[str, Any]

get_stats(model_id)[source]#

Retrieves cumulative inference and batch execution statistics for a registered model.

Queries the OIP v2 statistics endpoint (GET /v2/models/{model_id}/stats).

Parameters:

model_id (str) – Target model identifier.

Returns:

ModelStats containing request counts, latencies, and execution metrics.

Raises:
Return type:

ModelStats

stats()[source]#

Retrieves cumulative execution statistics across all registered models.

Queries the OIP v2 repository statistics endpoint (GET /v2/models/stats).

Returns:

List of ModelStats instances.

Raises:

SnoozeError – On network or server failure.

Return type:

List[ModelStats]

refresh()[source]#

Triggers runtime discovery and refresh of models on the SnoozeML daemon via HTTP.

Scans the daemon’s configured models directory for newly added or modified models.

Returns:

Updated list of ModelInfo objects registered on the server.

Return type:

List[ModelInfo]

upload(file, *, filename=None, model_id=None, description=None, pinned=False, overwrite=False, max_batch_size=None, max_batch_delay_ms=None, idle_timeout_secs=None, min_residency_secs=None, autoscale_queue_threshold=None, max_replicas=None, routing_strategy=None, device=None, provider=None)[source]#

Uploads a model file directly to the SnoozeML daemon via HTTP.

Supports passing custom runtime configuration (pinned memory, replica limits, batch sizes, idle timeouts, overwrite) without needing to mount local filesystem volumes.

Parameters:

  • file (str | Path | bytes | Any) – Filesystem path (str or Path), raw bytes, or readable binary stream.

  • filename (str | None) – Optional explicit filename (e.g. “iris_classifier.onnx”).

  • model_id (str | None) – Optional unique identifier for the model (defaults to file stem).

  • description (str | None) – Optional custom human-readable description.

  • pinned (bool) – If True, eagerly loads model into memory and exempts it from scale-to-zero.

  • overwrite (bool) – If True, replaces an existing model weight file on the server if present.

  • max_batch_size (int | None) – Maximum batch size for dynamic Axis(0) micro-batching.

  • max_batch_delay_ms (int | None) – Maximum latency budget in ms before executing micro-batch.

  • idle_timeout_secs (int | None) – Inactivity threshold in seconds before scale-to-zero unload.

  • min_residency_secs (int | None) – Minimum duration in seconds model stays resident before LRU eviction.

  • autoscale_queue_threshold (int | None) – Queue backlog threshold to trigger replica autoscale.

  • max_replicas (int | None) – Upper limit on parallel replica instances.

  • routing_strategy (str | None) – Replica routing strategy (“least_queue”, “round_robin”, “random”).

  • device (str | None) – Hardware execution provider (“cpu”, “cuda”, “auto”).

  • provider (str | None)

Returns:

ModelInfo representing the newly registered model.

Return type:

ModelInfo

wait_for_ready(model_id, *, timeout=30.0, poll_interval=0.2, trigger_load=False)[source]#

Polls model metadata until the model reaches a loaded state (ModelState.READY or ModelState.BUSY).

Parameters:

  • model_id (str) – Target model identifier.

  • timeout (float) – Maximum time in seconds to wait before raising InferenceTimeoutError.

  • poll_interval (float) – Interval in seconds between status polls.

  • trigger_load (bool) – If True and the model is currently idle, triggers load() first.

Returns:

Loaded ModelInfo instance.

Return type:

ModelInfo

deploy_sklearn(model, *, model_id, initial_types=None, sample_input=None, input_name='float_input', output_name='output', opset_version=17, overwrite=True, pinned=False, description=None, max_batch_size=None, max_batch_delay_ms=None, idle_timeout_secs=None, min_residency_secs=None, autoscale_queue_threshold=None, max_replicas=None, routing_strategy=None, device=None, provider=None)[source]#

Converts a fitted scikit-learn estimator or Pipeline to ONNX and deploys it to SnoozeML in one call.

Parameters:

  • model (Any)

  • model_id (str)

  • initial_types (Sequence[Tuple[str, Any]] | None)

  • sample_input (Any | None)

  • input_name (str)

  • output_name (str)

  • opset_version (int)

  • overwrite (bool)

  • pinned (bool)

  • description (str | None)

  • max_batch_size (int | None)

  • max_batch_delay_ms (int | None)

  • idle_timeout_secs (int | None)

  • min_residency_secs (int | None)

  • autoscale_queue_threshold (int | None)

  • max_replicas (int | None)

  • routing_strategy (str | None)

  • device (str | None)

  • provider (str | None)

Return type:

ModelInfo

deploy_torch(model, sample_input, *, model_id, input_names=None, output_names=None, dynamic_axes=None, opset_version=17, overwrite=True, pinned=False, description=None, max_batch_size=None, max_batch_delay_ms=None, idle_timeout_secs=None, min_residency_secs=None, autoscale_queue_threshold=None, max_replicas=None, routing_strategy=None, device=None, provider=None)[source]#

Exports a PyTorch nn.Module to ONNX and deploys it to SnoozeML in one call.

Parameters:

  • model (Any)

  • sample_input (Any)

  • model_id (str)

  • input_names (Sequence[str] | None)

  • output_names (Sequence[str] | None)

  • dynamic_axes (Mapping[str, Mapping[int, str]] | None)

  • opset_version (int)

  • overwrite (bool)

  • pinned (bool)

  • description (str | None)

  • max_batch_size (int | None)

  • max_batch_delay_ms (int | None)

  • idle_timeout_secs (int | None)

  • min_residency_secs (int | None)

  • autoscale_queue_threshold (int | None)

  • max_replicas (int | None)

  • routing_strategy (str | None)

  • device (str | None)

  • provider (str | None)

Return type:

ModelInfo

deploy_xgboost(model, *, model_id, overwrite=True, pinned=False, description=None, max_batch_size=None, max_batch_delay_ms=None, idle_timeout_secs=None, min_residency_secs=None, autoscale_queue_threshold=None, max_replicas=None, routing_strategy=None, device=None, provider=None)[source]#

Exports an XGBoost model to .xgb.json and deploys it to SnoozeML in one call.

Parameters:

  • model (Any)

  • model_id (str)

  • overwrite (bool)

  • pinned (bool)

  • description (str | None)

  • max_batch_size (int | None)

  • max_batch_delay_ms (int | None)

  • idle_timeout_secs (int | None)

  • min_residency_secs (int | None)

  • autoscale_queue_threshold (int | None)

  • max_replicas (int | None)

  • routing_strategy (str | None)

  • device (str | None)

  • provider (str | None)

Return type:

ModelInfo

class snoozeml.models.AsyncModelsManager(client)[source]#

Bases: object

Asynchronous manager for querying and controlling model lifecycles on a SnoozeML daemon.

Accessed via snoozeml.AsyncSnoozeClient.models.

Parameters:

client (AsyncSnoozeClient)

async list()[source]#

Asynchronously lists all registered models and their current telemetry.

Return type:

List[ModelInfo]

async get(model_id)[source]#

Asynchronously retrieves metadata and operational state for a specific model.

Parameters:

model_id (str)

Return type:

ModelInfo

async load(model_id, *, pin=False)[source]#

Asynchronously loads a model’s weights into memory.

Parameters:
Return type:

Dict[str, Any]

async unload(model_id)[source]#

Asynchronously unloads a model from memory.

Parameters:

model_id (str)

Return type:

Dict[str, Any]

async evict(model_id)[source]#

Alias for unload() — asynchronously evicts/snoozes a model from memory.

Parameters:

model_id (str)

Return type:

Dict[str, Any]

async pin(model_id)[source]#

Asynchronously pins a model in memory.

Parameters:

model_id (str)

Return type:

Dict[str, Any]

async unpin(model_id)[source]#

Asynchronously unpins a model.

Parameters:

model_id (str)

Return type:

Dict[str, Any]

async get_stats(model_id)[source]#

Asynchronously retrieves cumulative inference and batch execution statistics for a registered model.

Queries the OIP v2 statistics endpoint (GET /v2/models/{model_id}/stats).

Parameters:

model_id (str) – Target model identifier.

Returns:

ModelStats containing request counts, latencies, and execution metrics.

Raises:
Return type:

ModelStats

async stats()[source]#

Asynchronously retrieves cumulative execution statistics across all registered models.

Queries the OIP v2 repository statistics endpoint (GET /v2/models/stats).

Returns:

List of ModelStats instances.

Raises:

SnoozeError – On network or server failure.

Return type:

List[ModelStats]

async refresh()[source]#

Asynchronously triggers runtime discovery and refresh of models on the daemon.

Scans the daemon’s configured models directory for newly added or modified models.

Returns:

Updated list of ModelInfo objects registered on the server.

Return type:

List[ModelInfo]

async upload(file, *, filename=None, model_id=None, description=None, pinned=False, overwrite=False, max_batch_size=None, max_batch_delay_ms=None, idle_timeout_secs=None, min_residency_secs=None, autoscale_queue_threshold=None, max_replicas=None, routing_strategy=None, device=None, provider=None)[source]#

Asynchronously uploads a model file directly to the daemon via HTTP.

Supports passing custom runtime configuration (pinned memory, replica limits, batch sizes, idle timeouts, overwrite) without needing to mount local filesystem volumes.

Parameters:

  • file (str | Path | bytes | Any) – Filesystem path (str or Path), raw bytes, or readable binary stream.

  • filename (str | None) – Optional explicit filename (e.g. “iris_classifier.onnx”).

  • model_id (str | None) – Optional unique identifier for the model (defaults to file stem).

  • description (str | None) – Optional custom human-readable description.

  • pinned (bool) – If True, eagerly loads model into memory and exempts it from scale-to-zero.

  • overwrite (bool) – If True, replaces an existing model weight file on the server if present.

  • max_batch_size (int | None) – Maximum batch size for dynamic Axis(0) micro-batching.

  • max_batch_delay_ms (int | None) – Maximum latency budget in ms before executing micro-batch.

  • idle_timeout_secs (int | None) – Inactivity threshold in seconds before scale-to-zero unload.

  • min_residency_secs (int | None) – Minimum duration in seconds model stays resident before LRU eviction.

  • autoscale_queue_threshold (int | None) – Queue backlog threshold to trigger replica autoscale.

  • max_replicas (int | None) – Upper limit on parallel replica instances.

  • routing_strategy (str | None) – Replica routing strategy (“least_queue”, “round_robin”, “random”).

  • device (str | None) – Hardware execution provider (“cpu”, “cuda”, “auto”).

  • provider (str | None)

Returns:

ModelInfo representing the newly registered model.

Return type:

ModelInfo

async wait_for_ready(model_id, *, timeout=30.0, poll_interval=0.2, trigger_load=False)[source]#

Asynchronously polls model metadata until the model reaches a loaded state.

Parameters:
Return type:

ModelInfo

async deploy_sklearn(model, *, model_id, initial_types=None, sample_input=None, input_name='float_input', output_name='output', opset_version=17, overwrite=True, pinned=False, description=None, max_batch_size=None, max_batch_delay_ms=None, idle_timeout_secs=None, min_residency_secs=None, autoscale_queue_threshold=None, max_replicas=None, routing_strategy=None, device=None, provider=None)[source]#

Asynchronously converts a fitted scikit-learn estimator or Pipeline and deploys it.

Parameters:

  • model (Any)

  • model_id (str)

  • initial_types (Sequence[Tuple[str, Any]] | None)

  • sample_input (Any | None)

  • input_name (str)

  • output_name (str)

  • opset_version (int)

  • overwrite (bool)

  • pinned (bool)

  • description (str | None)

  • max_batch_size (int | None)

  • max_batch_delay_ms (int | None)

  • idle_timeout_secs (int | None)

  • min_residency_secs (int | None)

  • autoscale_queue_threshold (int | None)

  • max_replicas (int | None)

  • routing_strategy (str | None)

  • device (str | None)

  • provider (str | None)

Return type:

ModelInfo

async deploy_torch(model, sample_input, *, model_id, input_names=None, output_names=None, dynamic_axes=None, opset_version=17, overwrite=True, pinned=False, description=None, max_batch_size=None, max_batch_delay_ms=None, idle_timeout_secs=None, min_residency_secs=None, autoscale_queue_threshold=None, max_replicas=None, routing_strategy=None, device=None, provider=None)[source]#

Asynchronously exports a PyTorch nn.Module to ONNX and deploys it.

Parameters:

  • model (Any)

  • sample_input (Any)

  • model_id (str)

  • input_names (Sequence[str] | None)

  • output_names (Sequence[str] | None)

  • dynamic_axes (Mapping[str, Mapping[int, str]] | None)

  • opset_version (int)

  • overwrite (bool)

  • pinned (bool)

  • description (str | None)

  • max_batch_size (int | None)

  • max_batch_delay_ms (int | None)

  • idle_timeout_secs (int | None)

  • min_residency_secs (int | None)

  • autoscale_queue_threshold (int | None)

  • max_replicas (int | None)

  • routing_strategy (str | None)

  • device (str | None)

  • provider (str | None)

Return type:

ModelInfo

async deploy_xgboost(model, *, model_id, overwrite=True, pinned=False, description=None, max_batch_size=None, max_batch_delay_ms=None, idle_timeout_secs=None, min_residency_secs=None, autoscale_queue_threshold=None, max_replicas=None, routing_strategy=None, device=None, provider=None)[source]#

Asynchronously exports an XGBoost model to .xgb.json and deploys it.

Parameters:

  • model (Any)

  • model_id (str)

  • overwrite (bool)

  • pinned (bool)

  • description (str | None)

  • max_batch_size (int | None)

  • max_batch_delay_ms (int | None)

  • idle_timeout_secs (int | None)

  • min_residency_secs (int | None)

  • autoscale_queue_threshold (int | None)

  • max_replicas (int | None)

  • routing_strategy (str | None)

  • device (str | None)

  • provider (str | None)

Return type:

ModelInfo