snoozeml.inference — Tensor Serialization & Responses

Tensor payload validation, wire serialization, and inference response types for SnoozeML.

class snoozeml.inference.TensorDataType(*values)

Bases: str, Enum

Supported tensor element data types.

FP32 = 'fp32'

class snoozeml.inference.TensorInput(name, shape, data)

Bases: object

Validated multi-dimensional input tensor ready for REST or gRPC serialization.

Parameters:

• name (str)

• shape (List[int])

• data (Any)

name

Binding identifier of the input tensor node (e.g. "input", "input_ids").

Type:

str

shape

List of positive integer dimensions (e.g. [1, 3, 224, 224]).

Type:

List[int]

data

Contiguous float32 NumPy array (or fallback array) matching shape.

Type:

Any

name: str

shape: List[int]

data: Any

classmethod from_array(name, array, shape=None)

Constructs a TensorInput from a NumPy array, PyTorch Tensor, or nested sequence.

Parameters:

• name (str) – Binding name of the tensor node.

• array (Any) – NumPy array, PyTorch tensor (auto-detached to CPU), or nested sequence of numeric values.

• shape (Sequence[int] | None) – Optional explicit shape override. If omitted, inferred from array.

Returns:

Validated TensorInput.

Raises:

InvalidInputError – If the shape is empty, contains non-positive dimensions, or does not match the element count.

Return type:

TensorInput

classmethod from_dict(item)

Constructs a TensorInput from a dictionary {"name": ..., "shape": ..., "data": ...}.

Parameters:

item (Mapping[str, Any])

Return type:

TensorInput

to_rest_dict()

Serializes the tensor into the JSON dictionary format expected by POST /v1/models/:id/infer.

Return type:

Dict[str, Any]

to_proto()

Serializes the tensor into a Protobuf TensorData message with raw little-endian FP32 bytes.

Return type:

Any

class snoozeml.inference.TensorOutput(name, shape, data)

Bases: object

Multi-dimensional output tensor returned from model inference.

Parameters:

• name (str)

• shape (List[int])

• data (Any)

name

Binding identifier of the output tensor node (e.g. "output", "logits").

Type:

str

shape

Dimensions of the output tensor.

Type:

List[int]

data

NumPy float32 array (or fallback array) reshaped to shape.

Type:

Any

name: str

shape: List[int]

data: Any

to_numpy()

Returns the output tensor as a reshaped np.ndarray of dtype float32.

Return type:

Any

as_numpy()

Alias for to_numpy().

Return type:

Any

to_torch(device=None)

Returns the output tensor as a PyTorch torch.Tensor (dtype=torch.float32).

Parameters:

device (str | None) – Optional PyTorch device string (e.g. "cpu", "cuda", "mps").

Returns:

torch.Tensor matching shape.

Raises:

InvalidInputError – If torch is not installed.

Return type:

Any

to_list()

Returns the flattened output tensor elements as a Python list[float].

Return type:

List[float]

to_dict()

Returns a dictionary {"name": ..., "shape": ..., "data": ...} matching the REST format.

Return type:

Dict[str, Any]

classmethod from_rest_dict(item)

Constructs a TensorOutput from a REST JSON output tensor dictionary.

Parameters:

item (Mapping[str, Any])

Return type:

TensorOutput

classmethod from_proto(tensor_proto)

Constructs a TensorOutput from a Protobuf TensorData message.

Parameters:

tensor_proto (Any)

Return type:

TensorOutput

class snoozeml.inference.InferenceResponse(model_id, outputs, duration_ms)

Bases: object

Structured result returned by snoozeml.SnoozeClient.infer().

Parameters:

• model_id (str)

• outputs (List[TensorOutput])

• duration_ms (float)

model_id

Identifier of the executed model.

Type:

str

outputs

Ordered list of TensorOutput objects (supports name indexing).

Type:

List[snoozeml.inference.TensorOutput]

duration_ms

Server-side execution duration in milliseconds.

Type:

float

model_id: str

outputs: List[TensorOutput]

duration_ms: float

get_tensor(name=None)

Retrieves a TensorOutput by binding name (or the first output if name is None).

Parameters:

name (str | None) – Optional name of the output tensor node (e.g. "logits", "output"). If None, returns the first output tensor.

Returns:

Matching TensorOutput.

Raises:

KeyError – If no output tensors exist or name is not found.

Return type:

TensorOutput

get_numpy(name=None)

Returns the output tensor as a reshaped np.ndarray (dtype=np.float32).

Parameters:

name (str | None) – Optional name of the output tensor node (e.g. "logits"). If None, returns the first output tensor’s NumPy array.

Returns:

Multi-dimensional np.ndarray matching the output tensor’s shape.

Raises:

KeyError – If name is not present in the response outputs.

Return type:

ndarray

get_torch(name=None, device=None)

Returns the output tensor as a reshaped PyTorch torch.Tensor (dtype=torch.float32).

Parameters:

• name (str | None) – Optional name of the output tensor node (if None, uses first output).

• device (str | None) – Optional PyTorch device string (e.g. "cpu", "cuda").

Returns:

Multi-dimensional torch.Tensor matching the output tensor’s shape.

Return type:

Any

to_dict()

Returns a dictionary mapping each output tensor name to its np.ndarray.

Return type:

Dict[str, ndarray]

classmethod from_rest_dict(data, default_model_id='')

Constructs an InferenceResponse from a REST JSON payload.

Parameters:

• data (Mapping[str, Any])

• default_model_id (str)

Return type:

InferenceResponse

classmethod from_proto(proto_resp)

Constructs an InferenceResponse from a Protobuf ModelInferResponse.

Parameters:

proto_resp (ModelInferResponse)

Return type:

InferenceResponse

snoozeml.inference.normalize_inputs(inputs, default_input_name='input')

Normalizes flexible user input representations into a validated List[TensorInput].

Supported formats for inputs:

1. Dict[str, np.ndarray | Sequence | TensorInput] — e.g. {"input_ids": arr}

2. List[TensorInput | Dict[str, Any]] — e.g. [{"name": "input", "shape": [1, 3], "data": [1, 2, 3]}]

3. Single TensorInput

4. Single np.ndarray (assigned binding name default_input_name, default "input")

Parameters:

• inputs (Mapping[str, Any | Sequence[Any] | TensorInput] | Sequence[TensorInput | Mapping[str, Any]] | TensorInput | Any) – Input tensors in any supported format.

• default_input_name (str) – Binding name used when a single bare np.ndarray is passed.

Returns:

Non-empty list of validated TensorInput objects.

Raises:

InvalidInputError – If inputs is empty or any tensor fails validation.

Return type:

List[TensorInput]

snoozeml.inference.slice_inputs_batch(inputs, batch_size, default_input_name='input')

Splits a dataset of input tensors along Axis(0) into chunks of batch_size.

Parameters:

• inputs (Mapping[str, Any | Sequence[Any] | TensorInput] | Sequence[TensorInput | Mapping[str, Any]] | TensorInput | Any) – Input tensors in any supported format.

• batch_size (int) – Maximum batch size along Axis(0) per chunk.

• default_input_name (str) – Binding name when a bare array is passed.

Returns:

List of List[TensorInput] chunks ready for sequential or concurrent infer() calls.

Raises:

InvalidInputError – If batch_size <= 0 or input tensors have mismatched Axis(0) lengths.

Return type:

List[List[TensorInput]]

snoozeml.inference.concat_inference_responses(responses)

Concatenates a sequence of chunked InferenceResponse objects along Axis(0).

Parameters:

responses (Sequence[InferenceResponse])

Return type:

InferenceResponse