Things AI can do

[Naomasa298]

I don't think anyone was bringing up this incident (of an AI deleting a database) as an example of "Look at how incompetent AI is, ha ha."

Well, all I can say is, that's how it read to me.

 

[defaux]

Who wrote the IF ... THEN decision tree that made it reach this conclusion?

See, that's what I meant by required paradigm shift - this is not how neural networks work. That's deterministic programming, meaning you put in the same inputs you get the same outputs. Every time. LLM's are non-deterministic, which means there outputs are unpredictable.

It doesn't make "decisions". It runs probability math based on whatever data it's been trained on, to determine that X is the MOST LIKELY next token in the response set. The tokens all together produce the output, but the machine doesn't know what that is at the time.

This also means that an AI agent will never be capable of making a novel decision. It can only determine probabilities for decisions that already exist within it's training data set.

I wouldn't say that AI makes stupid decisions. But there is a degree of randomness in it's output by the very nature of how probability works.

The only way the LLM would "learn from it's mistakes" is if that response was fed back in as training data with negative reinforcement - ie: you train it that that specific decision was wrong.

 

[Naomasa298]

The only way the LLM would "learn from it's mistakes" is if that response was fed back in as training data with negative reinforcement - ie: you train it that that specific decision was wrong.

That is what AI systems like medical imaging do at the moment. In those kind of real world situations, AI isn't fully deterministic, and aren't static.

But even if you do train the AI to confirm before it does something disastrous, I'll bet the human interfacing with it will get rapidly annoyed by being asked "Are you sure? Are you really, REALLY sure?"