Test Results

Pilot Study Datasets and Networks:


For our Pilot studies we tested on assorted open source datasets with open source networks.  For each test test we would find the leaderboard displayed on paperswithcode.com and would download the highest scoring PyTorch based repository.  Datasets were chosen when the best open source results could be run on a single GPU.  We would then run the network with recommended settings and record the score.  After that we would add our system to the code and run again recording the new score.  We display our raw improvement over the open source code as well as how far apart competitors on the leaderboard are as a frame of reference, e.g. an improvement of 0.2% means a lot more if the best people differ by 0.01% than if they differ by 3%.  Size improvements were made by reducing the initial network width by a constant factor.