Made an integration with one LLM recently and had the same thought. Too many inputs and the responses are incorrect. Too few and it starts to hallucinate.