Gemini Enterprise / Agentspace Users: Can the AI Agents Directly Control Local PC/Desktop Software (Beyond IDEs)?

Hey everyone,

We're exploring Gemini Enterprise (or whatever you're calling the Agentspace platform these days) for automation, and I have a nagging question that Google's marketing materials completely dodge.

We know it's awesome at connecting to all the big cloud stuff (Salesforce, SAP, Workspace, etc.), and Code Assist works great inside IDEs like VS Code… but what about actual desktop software on a local machine?

The big question is: Can a Gemini Agent be set up to do basic RPA-style tasks on a local/proprietary desktop application?

I'm talking about things like:

  1. Opening a specific financial or legacy desktop app.
  2. Clicking the "Process Order" button, navigating menus, or typing info into a form inside that local app's GUI.
  3. Reading data directly from a window of a non-web application.

Basically, can Gemini replace or work like a traditional RPA tool (like UiPath/Automation Anywhere), but specifically for our older, local apps?

If you've actually managed to get agents to do this, please share your secrets! 🙏

  • What did you have to use to bridge the gap? (Custom connectors, specific libraries, etc.?)
  • How reliable is it? Is it super janky or production-ready?

Any insight from someone actually using this feature in the wild would be huge. Thanks!

Leave a Reply