> The context window only has to do with the size of input it has access to - its not related to what it's outputting
The sequential token generation process is closely related to the content of the context window at every step.
Maybe I misunderstood your point. I know these things can hallucinate when asked about obscure facts that they weren't sufficiently trained on.
> The context window only has to do with the size of input it has access to - its not related to what it's outputting
The sequential token generation process is closely related to the content of the context window at every step.
Maybe I misunderstood your point. I know these things can hallucinate when asked about obscure facts that they weren't sufficiently trained on.