I think you're fine so long as you're sticking to the quantitative aspects, as AI is basically a infinite curator of knowledge. I wouldn't rely on it for tone or emotional response, though. I'm sure there's some quantitative probability it can detect, but I couldn't imagine asking an AI whether a plot twist was effective or not. How would it know?
It would know based on how people have spoken about similar plot twists. Your plot twist is that the narrator is actually a ghost? It knows what people have said about that before. It's not quite that simple though. It will also base it on how the rest of the plot has played out before that point. By analysing billions of words, it'll get the word associations roughly right, because words are how we express our reactions.
It may not get it right. A human isn't guaranteed to either - but at least an AI wouldn't evaluate it based on whether it stubbed its toe that morning and their wife left them.
Addendum: What it can't do is actually feel. It's basically telling you what a large proportion of humanity have written that they feel with similar writing. Are those equivalent? Dunno.
Last edited: