January period Microsoft Research ForumDipendra Misra, senior researcher at Microsoft New York Research Labs and AI Frontiers, explains how Layer-Selective Rank Reduction (or LASER) can make large language models more accurate.
Using LASER, researchers can “intervene” and replace one weight matrix with an approximately smaller weight matrix. Weights are contextual connections made by the model. The heavier the weight, the more the model relies on it. So, is it effective to replace something with more relevance and context, will it reduce the accuracy of the model? According to their test results, surprisingly, the answer is no.
“We are intervening with LLM using LASER, so one would expect that as we make more approximations, the model loss should go up, which means the model will perform poorly, right, because we are throwing away information from the LLM, It’s trained on a lot of data,” Misra said. “But to our surprise, we found that if the right type of laser intervention is performed, the model losses do not go up, but actually go down.”
Misra said his team has successfully used LASER on three different open source models: RoBERTa, Llama 2 and Eleuther’s GPT-J. Model improvements sometimes yield improvements of 20 to 30 percentage points, he said. For example, after performance laser intervention of GPT-J, the biography-based gender prediction accuracy of J increased from 70.9% to 97.5%.