Thread

So I‘ve been reading the chatGPT plugin docs and if I get it right, you supply a textual description of the plugin and the endpoint and then chatGPT automagically decides when to call you and how. platform.openai.com/docs/plugins/g…

Now having worked with ML based systems for the past decade or so, I‘m directly wondering how to evaluate this properly and make sure it works as intended.

Because if I learned one thing it‘s that eye balling a solution doesn‘t work. In „classical“ code you can test edge cases and you‘ll know „it works“, but with ML you really need good coverage and out of sample testing.