Skip to main content

Dynamic Context via API - support larger history window for RAG/other use cases

Feature Request

This can be used to add live information to the AI or implement Retrieval-Augmented Generation (RAG)

In order to add the right live info or gather the right RAG injection content, more context about the current conversation would be useful.

For example, the lastUserMessage might be like "look it up" and we don't know what "it" is. So providing a larger history (with both user and system) will be great.

Use case 1: better context for RAG

Use case 2: TM does not support plugins for "local" models like Ollama-served llama3. However, if we had this larger history instead of just the last user message sent, it can be used to run plugins in the backend (such as Google search) and inject it as RAG effectively. This would allow tooling in local systems.

2 comments

Log in to comment and vote

Comments2

  • Rose Pudding

    •

    Feb 19, 2025

    You speaking of local model make me think if we can just like cline, manually call a tool and inject the tool result into prompt, say like fetch a webpage content and inject into chat. it will also reduce the need for instructing models to call them, especially since we have models like gemini that even you say the doc is on what webpage it will still not use reading tools to see it.

    • Teal Ox

      •

      Feb 22, 2025

      Jourin Town that was really hard to understand but yes i think we are talking about similar/same things