> It doesn't even matter if what the AI generates is true or not, if it leads the human to truth.

A weird statement. I think you only care about humans knowing the thing, right? Then it can be true that it doesn't matter if the machine is right or wrong.

However, if the machine is right.. that will have consequences.