filter_backward
perforatedai.filter_backward(grad_out, values, candidate_nonlinear_outs) Filter backward pass for gradient processing.
This function processes gradients during the backward pass, ensuring correct input dimensions and applying perforated backpropagation if enabled. It validates that input dimensions match expected shapes and initializes dendrite arrays when needed.
Parameters
grad_out (torch.Tensor) – The gradient output tensor from the backward pass.
values (DendriteValueTracker) – A DendriteValueTracker instance containing values associated with the module being processed.
candidate_nonlinear_outs (dict) – A dictionary of candidate modules outputs after non-linear activation.
Returns
None
set_wrapped_params
perforatedai.set_wrapped_params(model) Set parameters as wrapped with dendrites.
Marks all parameters in a model as wrapped, indicating they will have dendrites added.
Parameters
model (torch.nn.Module) – The model whose parameters are to be marked as wrapped.
Returns
None
set_tracked_params
perforatedai.set_tracked_params(model) Set parameters as tracked without dendrites.
Marks all parameters in a model as tracked, indicating they are monitored but will not have dendrites added.
Parameters
model (torch.nn.Module) – The model whose parameters are to be marked as tracked.
Returns
None
init_params
perforatedai.init_params(model) Randomize weights after duplicating the main module for the next set of dendrites.
Initializes all parameters in the model with random values scaled by the candidate weight initialization multiplier.
Parameters
model (torch.nn.Module) – The model whose parameters should be randomly initialized.
Returns
None
PAINeuronModule
class perforatedai.PAINeuronModule(start_module, name) Wrapper to set a module as one that will have dendritic copies.
This wrapper manages the main neuron module and its associated dendrite modules. It handles the addition of dendrites, manages dendrite-to-neuron connections, and coordinates the forward pass through both neuron and dendrite pathways.
Parameters
start_module (nn.Module) – The module to wrap with dendrite functionality.
name (str) – The name identifier for the neuron module.
Attributes
main_module (nn.Module) – The wrapped original module.
name (str) – The module's name identifier.
processor (PAIProcessor or None) – Custom processor for modules requiring special input/output handling.
activation_function_value (float) – Parameter value for activation functions if needed. (default: -1)
type (str) – Module type identifier. (default: "neuron_module")
this_input_dimensions (torch.Tensor) – Input tensor dimension format specification.
this_node_index (torch.Tensor) – Index of the neuron dimension in the input tensor.
dendrite_modules_added (int) – Count of dendrite modules that have been added.
dendrites_to_top (nn.ParameterList) – Parameters for dendrite-to-neuron connections.
dendrite_module (PAIDendriteModule) – The dendrite module container.
out_channels (int) – Number of output channels, set during first backward pass.
Methods
set_this_input_dimensions
set_this_input_dimensions(new_input_dimensions) Set the input dimensions for the neuron and dendrite blocks.
Signals to this NeuronModule that its input dimensions are different than the global default.
Parameters:
new_input_dimensions (list or torch.Tensor) – Dimension format specification. Use 0 for neuron index, -1 for variable dimensions
Returns:
None
set_mode
set_mode(mode) Switch between neuron training and dendrite training.
Parameters:
mode (str) – The mode to set. Either "n" for neuron training or "p" for dendrite training
Returns:
bool – True if mode was set successfully, False if an error occurred
Notes:
If False is returned, the mode was not changed due to an error (typically a module that should be tracked instead of converted). This problem should not be ignored unless you call GPA.pc.set_checked_skipped_modules(True).
create_new_dendrite_module
create_new_dendrite_module() Add an additional dendrite module.
Returns:
None
clear_processors
clear_processors() Clear processors if they save values for DeepCopy and save operations.
Returns:
None
clear_dendrites
clear_dendrites() Clear and reset dendrites before loading from a state dict.
Returns:
None
forward
**forward(*args, kwargs) Forward pass through the neuron layer.
Executes the main module, processes through any configured processor, runs all dendrite modules, combines dendrite outputs with neuron output using learned weights, and sets up gradient hooks.
Parameters:
*args (tuple) – Positional arguments for the forward pass
**kwargs (dict) – Keyword arguments for the forward pass
Returns:
Any – The output of the module after processing through neuron and dendrite modules, in the same format as the original module
Examples
>>> import torch.nn as nn
>>> from perforatedai import PAINeuronModule
>>>
>>> # Wrap a convolutional layer
>>> conv_layer = nn.Conv2d(3, 64, kernel_size=3)
>>> pai_conv = PAINeuronModule(conv_layer, name="conv1")
>>>
>>> # Set custom input dimensions if needed
>>> pai_conv.set_this_input_dimensions([-1, 0, -1, -1])TrackedNeuronModule
class perforatedai.TrackedNeuronModule(start_module, name) Wrapper for modules you don't want to add dendrites to.
Ensures all modules are accounted for in the PAI system without adding dendrite functionality. Used for modules that should be monitored but not modified with dendrites.
Parameters
start_module (nn.Module) – The module to wrap without dendrite functionality.
name (str) – The name identifier for the tracked module.
Attributes
main_module (nn.Module) – The wrapped original module.
name (str) – The module's name identifier.
type (str) – Module type identifier. (default: "tracked_module")
Methods
set_mode
set_mode(mode) Set mode for tracked layer.
Parameters:
mode (str) – The mode to set. Either "n" for neuron training or "p" for dendrite training
Returns:
bool – Always returns True
Notes:
This function does not change any behavior since this is a tracked layer without dendrites.
forward
**forward(*args, kwargs) Forward pass for tracked layer.
Simply passes through to the wrapped module without any dendrite processing.
Parameters:
*args (tuple) – Positional arguments for the forward pass
**kwargs (dict) – Keyword arguments for the forward pass
Returns:
Any – The output of the wrapped module in its original format
Examples
>>> import torch.nn as nn
>>> from perforatedai import TrackedNeuronModule
>>>
>>> # Track a pooling layer without adding dendrites
>>> pool_layer = nn.MaxPool2d(2)
>>> tracked_pool = TrackedNeuronModule(pool_layer, name="pool1")PAIDendriteModule
class perforatedai.PAIDendriteModule(initial_module, activation_function_value=0.3, name="no_name_given", input_dimensions=None) Module containing all dendrite modules added to the neuron module.
Manages the collection of dendrite modules that are added to a neuron over time. Handles dendrite-to-dendrite connections, candidate dendrite creation, and the forward pass through all dendrite pathways.
Parameters
initial_module (nn.Module) – The module to copy for creating dendrites.
activation_function_value (float, optional) – A value associated with the activation function. (default: 0.3)
name (str, optional) – The name identifier for the dendrite module. (default: "no_name_given")
input_dimensions (torch.Tensor, optional) – The dimensions of the input tensor. If None, uses global default. (default: None)
Attributes
layers (nn.ModuleList) – List of accepted dendrite modules.
processors (list) – List of processors for each dendrite module.
candidate_processors (list) – Processors for candidate dendrite modules.
num_dendrites (int) – Number of accepted dendrite modules.
num_cycles (torch.Tensor) – Number of dendrite training cycles performed.
mode (str) – Current training mode ("n" for neuron, "p" for dendrite).
name (str) – The module's name identifier.
parent_module (nn.Module) – Deep copy of the initial module used as template.
this_input_dimensions (torch.Tensor) – Input tensor dimension format.
this_node_index (torch.Tensor) – Index of the neuron dimension.
dendrites_to_candidates (nn.ParameterList) – Parameters for dendrite-to-candidate connections.
dendrites_to_dendrites (nn.ParameterList) – Parameters for dendrite-to-dendrite connections.
dendrite_values (nn.ModuleList) – List of DendriteValueTracker objects.
candidate_module (nn.ModuleList) – Current candidate dendrite modules.
out_channels (int) – Number of output channels.
Methods
set_this_input_dimensions
set_this_input_dimensions(new_input_dimensions) Set input dimensions for dendrite layer.
Parameters:
new_input_dimensions (list or torch.Tensor) – Dimension format specification
Returns:
None
create_new_dendrite_module
create_new_dendrite_module() Add a new set of candidate dendrites.
Creates new candidate modules with randomized weights and initializes dendrite-to-candidate connections.
Returns:
None
clear_processors
clear_processors() Clear all stored processor states.
Returns:
None
set_mode
set_mode(mode) Perform actions when switching between neuron and dendrite training.
When switching to neuron mode ("n"), converts the best candidate dendrite into an accepted dendrite module.
Parameters:
mode (str) – The mode to set ("n" for neuron, "p" for dendrite)
Returns:
None
forward
**forward(*args, kwargs) Forward pass for dendrite layer.
Processes input through all accepted dendrite modules, applies dendrite-to-dendrite connections, and returns dendrite outputs after activation functions.
Parameters:
*args (tuple) – Positional arguments for the forward pass
**kwargs (dict) – Keyword arguments for the forward pass
Returns:
dict – Dictionary of dendrite outputs after activation
dict – Candidate outputs (used for Perforated Backpropagation)
dict – Candidate nonlinear outputs (used for Perforated Backpropagation)
dict – Candidate non-zeroed outputs (used for Perforated Backpropagation)
Notes:
The additional return values beyond the first are only used when Perforated Backpropagation is enabled.
DendriteValueTracker
class perforatedai.DendriteValueTracker(initialized, activation_function_value, name, input_dimensions, out_channels=-1) Tracker object that maintains certain values for each set of dendrites.
Stores configuration and runtime statistics for dendrite modules, including initialization state, dimensions, and various tracking arrays used during training.
Parameters
initialized (int) – Whether the dendrite has been initialized (1) or not (0).
activation_function_value (float) – A value associated with the activation function.
name (str) – The name of the associated neuron module.
input_dimensions (torch.Tensor) – The dimensions of the input tensor.
out_channels (int, optional) – The number of output channels. If -1, will be set later. (default: -1)
Attributes
layer_name (str) – Name of the associated layer.
initialized (torch.Tensor) – Initialization flag (0 or 1).
current_d_init (torch.Tensor) – Current dendrite initialization state.
activation_function_value (float) – Activation function parameter value.
this_input_dimensions (torch.Tensor) – Input dimension format.
this_node_index (torch.Tensor) – Index of the neuron dimension.
out_channels (int) – Number of output channels.
dendrite_outs (dict) – Dictionary of dendrite outputs by device.
Methods
set_this_input_dimensions
set_this_input_dimensions(new_input_dimensions) Set input dimensions for value tracker.
Parameters:
new_input_dimensions (list or torch.Tensor) – New dimension format specification
Returns:
None
set_out_channels
set_out_channels(shape_values) Set output channels based on shape values and saved node index.
Parameters:
shape_values (list or torch.Size) – Shape values from which to extract channel count
Returns:
None
setup_arrays
setup_arrays(out_channels) Setup tracking arrays for the dendrite.
Initializes all tensor and scalar tracking values based on the output channel count.
Parameters:
out_channels (int) – The number of output channels
Returns:
None
reinitialize_for_pai
reinitialize_for_pai() Reinitialize value tracker to add the next set of dendrites.
Resets all tracking values to prepare for a new dendrite training cycle.
Returns:
None
print() Print value tracker information for debugging.
Returns:
None