Help! My AI voice agent keeps waiting for user reply before calling end_call() tool

Hey folks, I’m stuck with a LLM agent behavior that I can’t seem to tame. The agent should end the call right after confirming a reschedule, but it keeps waiting for the candidate to say something else before invoking end_call() tool. Example flow:

  1. AI: “I’ll call you tomorrow at 12 PM…”
  2. AI: (should immediately call end_call() here)
  3. Candidate: “ok”
  4. Only now does the agent trigger end_call().

I need the tool call to happen immediately after the closing sentence so the call shuts down even if the user doesn’t respond.

What I’ve tried:

  • Updated the system prompt with stricter wording: “•After confirmation: Thank them for their flexibility, confirm the rescheduled slot and move to step 9”
  • step 9. Close the call by calling the end_call() tool without explicitly stating that you are ending the call.

Has anyone wrestled with this before? Is there a better prompt pattern or tooling hook that forces the LLM to emit the function call without another user turn? Any tip on how to make gpt-4o mini obey “after your last sentence, emit only the tool call” instructions would be super helpful.

Leave a Reply