Data and Memory Restricted ML

EMNIST-Balanced

• Handwritten letters and numbers

• “Balanced” split merges classes like zero and capitol and lowercase O’s.

Small architecture from the PyTorch MNIST example

Data Restriction

• Original dataset has 112,800 training images

• 10% restricted data has 11,200 training images

• 1% restricted has only 1,200 training images

Accuracy Increases

• Original size network with all data shows error reduction of 4% with Perforated Backpropagation

• Original size shows 18.3% error reduction when working with only 1% of data

Compression

• With full dataset smaller networks do not catch up to the accuracy of the larger networks

• With 1% restricted data accuracy can be maintained with 16% of the original parameters.

• With 10% restricted data accuracy can be maintained with 50% of the original parameters