bspysmg.model package#

Module contents#

Package containing the files for training a surrogate model in pytorch.

Submodules#

bspysmg.model.training module#

File containing functions for training a surrogate model in pytorch taking into account the error in nano amperes.

bspysmg.model.training.default_train_step(model: Module, dataloader: DataLoader, criterion: _Loss, optimizer: Optimizer) → Tuple[Module, float][source]#

Performs the training step of a model within a single epoch and returns the current loss and current trained model.

Parameters

  • model (custom model of type torch.nn.Module) – Model to be trained.

  • dataloader (torch.utils.data.DataLoader) – A PyTorch Dataloader containing the training dataset.

  • criterion (<method>) – Loss function that will be used to train the model.

  • optimizer (torch.optim.Optimizer) – Optimization method used to train the model which decreases model’s loss.

Returns

Trained model and training loss for the current epoch.

Return type

tuple

bspysmg.model.training.default_val_step(model: Module, dataloader: DataLoader, criterion: _Loss) → float[source]#

Performs the validation step of a model within a single epoch and returns the validation loss.

Parameters

  • model (custom model of type torch.nn.Module) – Model to be trained.

  • dataloader (torch.utils.data.DataLoader) – A PyTorch Dataloader containing the training dataset.

  • criterion (<method>) – Loss function that will be used to train the model.

Returns

Validation loss for the current epoch.

Return type

float

bspysmg.model.training.generate_surrogate_model(configs: dict, custom_model: ~torch.nn.modules.module.Module = <class 'brainspy.processors.simulation.model.NeuralNetworkModel'>, criterion: ~torch.nn.modules.loss._Loss = MSELoss(), custom_optimizer: ~torch.optim.optimizer.Optimizer = <class 'torch.optim.adam.Adam'>, main_folder: str = 'training_data') → None[source]#

It loads the training and validation datasets from the npz file specified in the field data/dataset_paths of the configs dictionary. These npz files can be created when running the postprocessing of the sampling data. The method will train a neural network with the structure specified in the model_structure field of the configs using the loaded training and validation datasets. It provides, on the specified saving directory, a trained model, the plots of the training performance, and the error of the model.

Note that the method will only save a model (in each epoch) if current validation loss is less than the validation loss from the previous epoch.

Parameters

configs (dict) –

Training configurations for training a model with following keys:

1. results_base_dir: str Directory where the trained model and corresponding performance plots will be stored.

2. seed: int Sets the seed for generating random numbers to a non-deterministic random number.

3. hyperparameters: epochs: int learning_rate: float

4. model_structure: dict The definition of the internal structure of the surrogate model, which is typically five fully-connected layers of 90 nodes each.

4.1 hidden_sizes : list A list containing the number of nodes of each layer of the surrogate model. E.g., [90,90,90,90,90]

4.2 D_in: int Number of input features of the surrogate model structure. It should correspond to the activation electrode number.

4.3 D_out: int Number of output features of the surrogate model structure. It should correspond to the readout electrode number.

5. data: 5.1 dataset_paths: list[str] A list of paths to the Training, Validation and Test datasets, stored as postprocessed_data.npz

5.2 steps : int It allows to skip parts of the data when loading it into memory. The number indicates how many items will be skipped in between. By default, step number is one (no values are skipped). E.g., if steps = 2, and the inputs are [0, 1, 2, 3, 4, 5, 6]. The only inputs taken into account would be: [0, 2, 4, 6].

5.3 batch_size: int How many samples will contain each forward pass.

5.4 worker_no: int How many subprocesses to use for data loading. 0 means that the data will be loaded in the main process. (default: 0)

5.5 pin_memory: boolean If True, the data loader will copy Tensors into CUDA pinned memory before returning them. If your data elements are a custom type, or your collate_fn returns a batch that is a custom type. custom_model : custom model of type torch.nn.Module

Model to be trained.

criterion<method>

Loss function that will be used to train the model.

custom_optimizertorch.optim.Optimizer

Optimization method used to train the model which decreases model’s loss.

save_dirstring [Optional]

Name of the path where the trained model is to be saved.

Returns

saved_dir – Directory where the surrogate model was saved.

Return type

str

bspysmg.model.training.init_seed(configs: dict) → None[source]#

Initializes a random seed for training. A random seed is a starting point for pseudorandom number generator algorithms which is used for reproducibility. Also see - https://pytorch.org/docs/stable/notes/randomness.html

Parameters

configs (dict) –

Training configurations with the following keys:

1. seed: int [Optional] The desired seed for the random number generator. If the dictionary does not contain this key, a deterministic random seed will be applied, and added to the key ‘seed’ in the dictionary.

bspysmg.model.training.postprocess(dataloader: DataLoader, model: Module, criterion: _Loss, amplification: float, results_dir: str, label: str) → float[source]#

Plots error vs output and error histogram for given dataset and saves it to specified directory.

Parameters

  • dataloader (torch.utils.data.DataLoader) – A PyTorch Dataloader containing the training dataset.

  • model (custom model of type torch.nn.Module) – Model to be trained.

  • criterion (<method>) – Loss function that will be used to train the model.

  • amplification (float) – Amplification correction factor used in the device to correct the amplification applied to the output current in order to convert it into voltage before its readout.

  • results_dir (string) – Name of the path and file where the plots are to be saved.

  • label (string) – Name of the dataset. I.e., train, validation or test.

Returns

Mean Squared error evaluated on given dataset.

Return type

float

bspysmg.model.training.to_device(inputs: Tensor) → Tensor[source]#

Copies input tensors from CPU to GPU device for processing. GPU allows multithreading which makes computation faster. The rule of thumb is using 4 worker threads per GPU. See - https://pytorch.org/docs/stable/tensor_attributes.html#torch.torch.device

Parameters

inputs (torch.Tensor) – Input tensor which needs to be loaded into GPU device.

Returns

Input tensor allocated to GPU device.

Return type

torch.Tensor

bspysmg.model.training.train_loop(model: Module, info_dict: dict, dataloaders: List[DataLoader], criterion: _Loss, optimizer: Optimizer, epochs: int, amplification: float, start_epoch: int = 0, save_dir: Optional[str] = None, early_stopping: bool = True) → Tuple[Module, List[float]][source]#

Performs the training of a model and returns the trained model, training loss validation loss. It also saves the model in each epoch if current validation loss is less than the previous validation loss.

Parameters

  • model (custom model of type torch.nn.Module) – Model to be trained.

  • info_dict (dict) –

    The dictionary used for initialising the surrogate model. It has the following keys: 1. model_structure: dict The definition of the internal structure of the surrogate model, which is typically five fully-connected layers of 90 nodes each.

    1.1 hidden_sizes : list A list containing the number of nodes of each layer of the surrogate model. E.g., [90,90,90,90,90]

    1.2 D_in: int Number of input features of the surrogate model structure. It should correspond to the activation electrode number.

    1.3 D_out: int Number of output features of the surrogate model structure. It should correspond to the readout electrode number.

    2. electrode_info: dict It contains all the information required for the surrogate model about the electrodes. 2.1 electrode_no: int Total number of electrodes in the device

    2.2 activation_electrodes: dict 2.2.1 electrode_no: int Number of activation electrodes used for gathering the data

    2.2.2 voltage_ranges: list Voltage ranges used for gathering the data. It contains the ranges per electrode, where the shape is (electrode_no,2). Being 2 the minimum and maximum of the ranges, respectively.

    2.3 output_electrodes: dict

    2.3.1 electrode_no : int Number of output electrodes used for gathering the data

    2.3.2 clipping_value: list[float,float] Value used to apply a clipping to the sampling data within the specified values.

    2.3.3 amplification: float Amplification correction factor used in the device to correct the amplification applied to the output current in order to convert it into voltage before its readout.

    3. training_configs: dict A copy of the configurations used for training the surrogate model.

    4. sampling_configs : dict A copy of the configurations used for gathering the training data.

  • dataloaders (list) – A list containing a single PyTorch Dataloader containing the training dataset.

  • criterion (<method>) – Loss function that will be used to train the model.

  • optimizer (torch.optim.Optimizer) – Optimization method used to train the model which decreases model’s loss.

  • epochs (int) – The number of iterations for which the model is to be trained.

  • amplification (float) – Amplification correction factor used in the device to correct the amplification applied to the output current in order to convert it into voltage before its readout.

  • start_epoch (int [Optional]) – The starting value of the epochs.

  • save_dir (string [Optional]) – Name of the path and file where the trained model is to be saved.

  • early_stopping (bool [Optional]) – If this is set to true, early stopping algorithm is used during the training of the model. Also see - https://medium.com/analytics-vidhya/early-stopping-with-pytorch-to- restrain-your-model-from-overfitting-dce6de4081c5

Returns

  • model – Trained model

  • losses – list of training loss and validation loss.

  • saved_dir – directory where the model was saved