Yep. It seems when transformers first came out there was already a flurry of efficiency research. That plus open source models will make for interesting times. Hopefully.