Issue: the app count tokens on prompts that don't actually make it to the LLM due to any error.

You may want to re-visit your token usage logic.

It might be calculating tokens on prompts that don't actually make it to the LLM due to any error. The analytics dashboard is showing one of my users generating nearly 19M tokens yesterday. But when I go to my OpenAI dashboard, my api key only used 1.4M tokens for the same day. I understand it's "Estimated" token use, but this is waaaayy off.