Thread

In Apple's Monday presentation, they mentioned a couple of ML related things. From what I remember: - transformer models for speech2text and autocorrect running on the device - they claimed that the new M2 Ultra with 192GB unified memory could be used to train LLMs.

Speech2Text and autocorrect are way due for an update, so looking forward to those (and hoping my iPhone 11 can run those models, too 😅)

Re: the latter, Apple has been working hard, for example, providing the mps backend for pytorch, but most people still stick to Nvidia GPUs for training.

The M1 Max GPU on my MBP benchmarks a bit slower than the GTX 1660S on my PC. That's not that impressive, but the new M2 Ultras have more than double the cores, so that would put them somewhere along the xx70 series? (assuming roughly performance doubles from series to series)

I found this post by @rasbt about benchmarking the M2 Ultra from a year ago: sebastianraschka.com/blog/2022/pyto… Looks about right, on par with a 3060, maybe a bit faster.