osculari.models

This module readouts features from pretrained networks and implements linear probe layers.

osculari.models.model_utils

A set of utility functions to for PyTorch models.

osculari.models.model_utils.check_input_size(architecture: str, img_size: int) → None[source]

Check if the input image size is compatible with the specified neural network architecture.

Parameters:

architecture (str) – The name of the neural network architecture.
img_size (int) – The input image size.

Raises:

RuntimeError – If the input image size is incompatible with the specified architecture.

osculari.models.model_utils.generic_features_size(model: Module, img_size: int) → Tuple[int][source]

Compute the output size of a neural network model given an input image size.

Parameters:

model (nn.Module) – The neural network model.
img_size (int) – The input image size (assuming square images).

Returns:

The computed output size of the model.

Return type:

Tuple[int]

osculari.models.model_utils.register_model_hooks(model: Module, architecture: str, layers: List[str])[source]

Register hooks for capturing activation for specific layers in the model.

Parameters:

model (nn.Module) – PyTorch model.
architecture (str) – Model architecture name.
layers (List[str]) – List of layer names for which to register hooks.

Raises:

RuntimeError – If the specified layer is not supported for the given architecture.

Returns:

Dictionaries containing activation values and registered forward hooks.

Return type:

(Dict, Dict)

osculari.models.pretrained_layers

Extracting features from different layers of a pretrained model.

osculari.models.pretrained_layers.available_layers(architecture: str) → List[str][source]

Returning a list of supported layers for each architecture.

Parameters:: architecture (str) – The name of the architecture.
Returns:: A list of supported layers for the specified architecture.
Return type:: List[str]
Raises:: RuntimeError – If the specified architecture is not supported.

osculari.models.pretrained_models

A wrapper around publicly available pretrained models in PyTorch.

osculari.models.pretrained_models.available_models(flatten: Optional[bool] = False) → Union[Dict, List][source]

List of supported models.

Parameters:: flatten (bool, optional) – If True, returns a flattened list of all supported models. If False, returns a dictionary of model categories.
Returns:: List of supported models or a dictionary of model categories.
Return type:: Union[Dict, List]

osculari.models.readout

A set of wrapper classes to access pretrained networks for different purposes.

class osculari.models.readout.ActivationLoader(architecture: str, weights: str, layers: Union[str, List[str]])[source]

Bases: BackboneNet

Loading activation of a network.

forward(x: Tensor) → Dict[source]

Forward pass to load the activation of the network.

Parameters:: x (torch.Tensor) – The input tensor.
Returns:: The activation dict.
Return type:: Dict

training: bool

class osculari.models.readout.Classifier2AFC(merge_paradigm: Literal['diff', 'cat'], **kwargs: Any)[source]

Bases: ProbeNet

Classifier for 2AFC (Two-Alternative Forced Choice) task.

forward(x0: Tensor, x1: Tensor) → Tensor[source]

Forward pass for Classifier2AFC.

Parameters:

x0 (torch.Tensor) – The first input tensor.
x1 (torch.Tensor) – The second input tensor.

Returns:

The output tensor.

Return type:

torch.Tensor

static loss_function(output: Tensor, target: Tensor) → Tensor[source]

Compute the loss for Classifier2AFC.

Parameters:

output (torch.Tensor) – The model output.
target (torch.Tensor) – The target labels.

Returns:

The computed loss.

Return type:

torch.Tensor

serialisation_params() → Dict[source]

Get serializable parameters for Classifier2AFC.

Returns:: The serializable parameters.
Return type:: Dict

training: bool

class osculari.models.readout.FeatureExtractor(architecture: str, weights: str, layers: Union[str, List[str]], pooling: Optional[str] = None)[source]

Bases: ReadOutNet

Extracting features from a pretrained network.

forward(x: Tensor) → Tensor[source]

Forward pass to extract features from the pretrained network.

Parameters:: x (torch.Tensor) – The input tensor.
Returns:: The extracted features.
Return type:: torch.Tensor

training: bool

class osculari.models.readout.OddOneOutNet(input_nodes: int, merge_paradigm: Literal['diff', 'cat'], **kwargs: Any)[source]

Bases: ProbeNet

Classifier for Odd-One-Out task.

forward(*xs: Tensor) → Tensor[source]

Forward pass for OddOneOutNet.

Parameters:: *xs (torch.Tensor) – Input tensors.
Returns:: The output tensor.
Return type:: torch.Tensor

static loss_function(output: Tensor, target: Tensor) → Tensor[source]

Compute the categorical cross entropy loss for OddOneOutNet.

Parameters:

output (torch.Tensor) – The model output.
target (torch.Tensor) – The target labels.

Returns:

The computed loss.

Return type:

torch.Tensor

serialisation_params() → Dict[source]

Get serializable parameters for OddOneOutNet for an easy future load.

Returns:: The serializable parameters.
Return type:: Dict

training: bool

class osculari.models.readout.ProbeNet(input_nodes: int, num_classes: int, img_size: int, probe_layer: Optional[str] = 'nn', **kwargs: Any)[source]

Bases: ReadOutNet

Adding a linear layer on top of readout features.

do_features(x: Tensor) → Tensor[source]

Extract features from the input tensor.

Parameters:: x (torch.Tensor) – The input tensor.
Returns:: The extracted features.
Return type:: torch.Tensor

do_probe_layer(x: Tensor) → Tensor[source]

Apply the probe layer to the input tensor.

Parameters:: x (torch.Tensor) – The input tensor.
Returns:: The output tensor.
Return type:: torch.Tensor

forward(*_args: Any)[source]

Forward pass. Not implemented in ProbeNet.

Raises:: NotImplementedError – The forward method is not implemented.

serialisation_params() → Dict[source]

Get serializable parameters for the ProbeNet.

Returns:: The serializable parameters.
Return type:: Dict

training: bool

osculari.models.readout.load_paradigm_2afc(checkpoint: Any) → Classifier2AFC[source]

Load a pre-trained 2AFC (Two-Alternative Forced Choice) Classifier from a checkpoint.

Parameters:: checkpoint (str or Dict) – The checkpoint containing the network information. Either the path to the networks checkpoint or the loaded checkpoint.
Returns:: An instance of the Classifier2AFC class.
Return type:: Classifier2AFC

osculari.models.readout.load_paradigm_ooo(checkpoint: Any) → OddOneOutNet[source]

Load a pre-trained Odd-One-Out Classifier from a checkpoint.

Parameters:: checkpoint (str or Dict) – The checkpoint containing the network information. Either the path to the networks checkpoint or the loaded checkpoint.
Returns:: An instance of the OddOneOutNet class.
Return type:: OddOneOutNet

osculari.models.readout.paradigm_2afc_merge_concatenate(**kwargs: Any) → Classifier2AFC[source]

Create a 2AFC (Two-Alternative Forced Choice) Classifier with merging paradigm ‘concatenate’.

Parameters:

**kwargs – Additional keyword arguments for Classifier2AFC.

Keyword Arguments:

architecture (str) – The name of the pretrained neural network architecture.
weights (str) – The weights of the pretrained network to load into the model.
layers (Union[str, List[str]]) – The layers of the pretrained network from which to read out features.
img_size (int) – The size of the input image.
pooling (Optional[str]) – The type of pooling to apply (e.g., ‘max_2_2’ or ‘avg_2_2’).
probe_layer (Optional[str]) – The type of probe layer (‘nn’ for linear layer).

Returns:

An instance of the Classifier2AFC class.

Return type:

Classifier2AFC

osculari.models.readout.paradigm_2afc_merge_difference(**kwargs: Any) → Classifier2AFC[source]

Create a 2AFC (Two-Alternative Forced Choice) Classifier with merging paradigm ‘difference’.

Parameters:

**kwargs – Additional keyword arguments for Classifier2AFC.

Keyword Arguments:

architecture (str) – The name of the pretrained neural network architecture.
weights (str) – The weights of the pretrained network to load into the model.
layers (Union[str, List[str]]) – The layers of the pretrained network from which to read out features.
img_size (int) – The size of the input image.
pooling (Optional[str]) – The type of pooling to apply (e.g., ‘max_2_2’ or ‘avg_2_2’).
probe_layer (Optional[str]) – The type of probe layer (‘nn’ for linear layer).

Returns:

An instance of the Classifier2AFC class.

Return type:

Classifier2AFC

osculari.models.readout.paradigm_ooo_merge_concatenate(input_nodes: int, **kwargs: Any) → OddOneOutNet[source]

Create an Odd-One-Out Classifier with merging paradigm ‘concatenate’.

Parameters:

input_nodes (int) – The number of input nodes.
**kwargs – Additional keyword arguments for OddOneOutNet.

Keyword Arguments:

architecture (str) – The name of the pretrained neural network architecture.
weights (str) – The weights of the pretrained network to load into the model.
layers (Union[str, List[str]]) – The layers of the pretrained network from which to read out features.
img_size (int) – The size of the input image.
pooling (Optional[str]) – The type of pooling to apply (e.g., ‘max_2_2’ or ‘avg_2_2’).
probe_layer (Optional[str]) – The type of probe layer (‘nn’ for linear layer).

Returns:

An instance of the OddOneOutNet class.

Return type:

OddOneOutNet

osculari.models.readout.paradigm_ooo_merge_difference(input_nodes: int, **kwargs: Any) → OddOneOutNet[source]

Create an Odd-One-Out Classifier with merging paradigm ‘difference’.

Parameters:

input_nodes (int) – The number of input nodes.
**kwargs – Additional keyword arguments for OddOneOutNet.

Keyword Arguments:

architecture (str) – The name of the pretrained neural network architecture.
weights (str) – The weights of the pretrained network to load into the model.
layers (Union[str, List[str]]) – The layers of the pretrained network from which to read out features.
img_size (int) – The size of the input image.
pooling (Optional[str]) – The type of pooling to apply (e.g., ‘max_2_2’ or ‘avg_2_2’).
probe_layer (Optional[str]) – The type of probe layer (‘nn’ for linear layer).

Returns:

An instance of the OddOneOutNet class.

Return type:

OddOneOutNet