Aren't LLMs much more limited on the amount of output tokens than input tokens? For example, GPT-4o seems to support only up to 16 K output tokens. I'm not completely sure what the reason is, but I wonder how that interacts with Chain-of-Thought reasoning.
There's no fundamental difference between input and output tokens technically.
The internal model space is exactly the same after evaluating some given set of token, no matter which of them were produced by the prompter or the model.
The 16k output token limit is just an arbitrary limit in the chatgpt interface.
