Skim AI: Same accuracy, 90% smaller model

Skim AI builds custom AI and machine learning solutions that help businesses optimize operations and make better decisions from their own data. Since 2019, Skim AI has run information classification and extraction pipelines built on BERT, a proven and efficient language model from Google.

With Perforated, Skim AI achieved:

• A deployment-ready model that matched and beat baseline accuracy with 90% fewer parameters
• 97% inference cost savings and 15x faster inference on cloud deployment
• Working results across BERT variants in one week, using Skim AI’s existing HuggingFace workflow

The Challenge: Efficiency and accuracy pulled in opposite directions

Skim AI’s BERT-based models handle natural language classification and extraction in production, where inference speed and compute spend climb as usage scales. The obvious fix was smaller architecture. But every smaller BERT the team tests gave up accuracy the business could not afford to lose.

That left Skim AI with two options. Shrink the model and lose accuracy, or hold accuracy and keep paying for size and speed. Neither was good enough for the products the team ships.

The Solution: More performance from the model already in place

Rather than trade accuracy for efficiency, Skim AI took a more data-efficient path by getting more out of the models it already had. Perforated backpropagation helps a model extract more value from its existing training data, so a small model can reach the accuracy of a much larger one without collecting new data or redesigning anything. Skim AI applied Perforated across a range of BERT architectures, model sizes, and benchmark datasets.

Because Perforated required only minor code changes in PyTorch and integrated cleanly with HuggingFace, the team tested across model variations quickly without reworking its workflow. That opened up leaner, faster model designs that had previously been off the table. One such design runs up to 15x faster than a standard BERT model but, without Perforated, could not clear the accuracy bar, which limited it to jobs where speed mattered more than precision. With Perforated, it became accurate enough to deploy where both speed and accuracy matter.

The Outcome: Same accuracy, a fraction of the size

Skim AI’s deployment-ready model matched and exceeded the accuracy of larger baselines while running on 90% fewer parameters. On the cloud, that translated to 97% inference cost savings and 15x faster inference.

The same held true in smaller models. In one representative test, a 497k-parameter model matched a standard BERT model nearly nine times larger. The tradeoff that started this project was gone. Skim AI no longer had to choose between accuracy and the cost of running the model.