Latency issues with the Gemini Live API

I just tested the gemini live api, building a simple voice agent using it. The response latencies seem very high (5-10 secs). I've used the exact same code as on this page: https://ai.google.dev/gemini-api/docs/live?example=mic-stream
These are some of the configs I am using

MODEL = "gemini-2.5-flash-native-audio-preview-12-2025"
CONFIG = types.LiveConnectConfig(
    response_modalities=[types.Modality.AUDIO],
    system_instruction="You are a helpful and friendly AI assistant.",
)

client = genai.Client(api_key=os.getenv("GOOGLE_API_KEY"), http_options=types.HttpOptions(api_version="v1alpha"))

Can someone pls help pointing out the issue if it works for them? Anyone facing similar issues?