Thread

Prompt engineering on a completely new level. I cannot keep me from seeing GPT-3 as an entity that would say anything to bullshit its way through a conversation. In a way, that‘s exactly what it has been trained to do :) twitter.com/goodside/statu…

In order to understand complex deep learning models, understanding what the task and data it was trained on is even more important than understanding the architecture.

The architecture of a deep learning model is the how, but the task and data is the what.

Large language models are trained on tasks like predicting missing words from context, or deciding whether a sentence follows another one. The current models got quite good at this, but ultimately they are producing plausible sequences of sentences.

Put differently, they say what you want to hear (as encoded by heaps of human generated examples).

Many more details on how LaMDA did exactly what I described above. lastweekin.ai/p/lamdas-senti…