tracker_perforatedai.py - Docs

Status Constants

NO_MODEL_UPDATE = 0

No model restructuring occurred during validation score processing.

NETWORK_RESTRUCTURED = 1

Network was restructured by adding dendrites or resetting to previous checkpoint.

TRAINING_COMPLETE = 2

Training has completed, either by reaching maximum dendrites or exhausting improvement attempts.

PAINeuronModuleTracker

class perforatedai.PAINeuronModuleTracker(doing_pai, save_name, making_graphs=True, param_vals_setting=-1, values_per_train_epoch=-1, values_per_val_epoch=-1)

Manager class that tracks all neuron layers and dendrite layers.

Controls when new dendrites are added, communicates signals to modules, manages training state, tracks validation scores and learning rates, generates training graphs, and orchestrates the entire PAI training process.

Parameters

doing_pai (bool) – Whether dendrites should be used during training.

save_name (str) – The base name for saving models and graphs.

making_graphs (bool, optional) – Whether to generate graphs and visualizations. (default: True)

param_vals_setting (int, optional) – Parameter values setting for scheduler tracking. (default: -1, uses global config)

values_per_train_epoch (int, optional) – Number of values to look back for graphing during training. Use -1 for all values. (default: -1)

values_per_val_epoch (int, optional) – Number of values to look back for graphing during validation. Use -1 for all values. (default: -1)

Key Methods

initialize()

Setup the tracker with initial settings and convert the model.

add_validation_score()

Add a validation score and determine if dendrite switching should occur. This is the primary method called after each validation epoch.

setup_optimizer()

Initialize the optimizer and scheduler with proper learning rate handling.

Examples

>>> import torch
>>> import torch.nn as nn
>>> import perforatedai as pai
>>> 
>>> # Create model and tracker
>>> model = YourModel()
>>> tracker = pai.PAINeuronModuleTracker(
>>>     doing_pai=True,
>>>     save_name="my_experiment",
>>>     maximizing_score=True
>>> )
>>> 
>>> # Initialize
>>> model = tracker.initialize(model)
>>> 
>>> # Set optimizer and scheduler
>>> tracker.set_optimizer(torch.optim.Adam)
>>> tracker.set_scheduler(torch.optim.lr_scheduler.ReduceLROnPlateau)
>>> 
>>> # Training loop
>>> for epoch in range(num_epochs):
>>>     # Train
>>>     train(model)
>>>     tracker.stop_epoch()
>>>     
>>>     # Validate
>>>     val_acc = validate(model)
>>>     model, restructured, complete = tracker.add_validation_score(val_acc, model)
>>>     
>>>     # Reinitialize optimizer if restructured
>>>     if restructured:
>>>         optimizer, scheduler = tracker.setup_optimizer(
>>>             model, 
>>>             {'lr': 0.001},
>>>             {'patience': 5}
>>>         )
>>>     
>>>     if complete:
>>>         break

Notes

This is the central coordination class for PAI training. Most users will interact primarily with this class through initialize(), add_validation_score(), and setup_optimizer() methods.

The Artificial Dendrite Network Library for PyTorch