[speed] pre-fetch AI response as you type

AI responses are slow. So prefetch responses, similar to link prefetching*, as you are typing. That way, when you type a message, proof read it, and hit send, the response shows up immediately cause it was fetched seconds ago.

Use case: This could be great for local AI's or cheap models.

Considerations

  • don't automatically trigger plugins for pre-fetched responses (as it could perform write-operations), but be ready to trigger them when hitting enter.

  • close previous connection when fetching new prefetch request.

Probably want some level of customizations:

Please authenticate to join the conversation.

Upvoters
Status

Open

Board
πŸ’‘

Feature Request

Tags

UXUI Improvement

Date

Almost 2 years ago

Subscribe to post

Get notified by email when there are changes.