The way the user prompt makes it into the LLM's context window is not by how we pass it! The LLM makes a deliberate decision to read it from the environment.
In our case, the user's prompt is simple and short. But remember, the user's prompt can be arbitrarily long. For example, in one of my test cases, I input the complete transcripts of 300 Lex Fridman podcasts as a string that contained nearly 10M tokens.
The print statement in the REPL environment does not return the full output dump! Instead, it truncates the output to a fixed length and returns it.
> Even if the RLM tries to overload itself with sensory information, we explicitly prevent the RLM from doing so by truncating the terminal output.
The LLM can always explore slices of the prompt deliberately too:
markdown