HeyRocky

Connect Hermes to Google Vertex AI

The Vertex provider routes Hermes to Gemini models billed through a Google Cloud project. It is distinct from the Google AI Studio path: Vertex uses Google Cloud identity and short-lived OAuth2 access tokens rather than a static Gemini API key.

Use this route when the workload should use GCP billing, IAM, quotas, and deployment controls. Confirm current model and region availability in the Google Cloud project before planning around a preview model.

Authentication choices

Hermes resolves Vertex credentials in this order:

  1. VERTEX_CREDENTIALS_PATH
  2. GOOGLE_APPLICATION_CREDENTIALS
  3. Application Default Credentials (ADC)

A service-account JSON is practical for a controlled server or gateway, while ADC is convenient for local development and GCP runtimes. Whichever identity is used needs the Vertex AI API enabled in the project and an appropriate role such as roles/aiplatform.user.

Hermes mints and refreshes short-lived access tokens. Do not paste a temporary token into a custom provider configuration; it can expire during the session and bypasses the provider’s refresh path.

Setup checklist

  • Select a GCP project with billing active and the Vertex AI API enabled.
  • Choose a service account, workload identity, VM identity, or local ADC flow.
  • Grant the runtime identity only the required Vertex permissions.
  • Keep any service-account key file outside source control and reference it through the supported environment variable.
  • Run hermes model and choose More providers → Google Vertex AI.
  • Enter the project ID when it cannot be derived from credentials.
  • Choose the region required by the selected model; use global only when the model and policy allow it.
  • Select the exact google/... model ID shown by the current picker.
  • Run hermes doctor and complete a harmless chat request.
# Local-development ADC path
gcloud auth application-default login
hermes model
hermes doctor
hermes chat

Project and region configuration

Project ID and region are routing values and belong in the vertex section of config.yaml; the credential path belongs in the secret environment file. VERTEX_PROJECT_ID and VERTEX_REGION override the durable configuration, so inspect the runtime environment when a deployment unexpectedly targets a different project or location.

The official guide documents global for the listed Gemini 3.x preview models and warns that regional endpoints may return 404 for them. Preview names and serving locations can change, so treat the live picker and Google Cloud availability as authoritative.

Common pitfalls

  • Configuring Vertex as if it accepted a permanent API key.
  • Pasting an hour-lived OAuth token into a custom endpoint.
  • Enabling Vertex AI in one project while credentials resolve to another.
  • Omitting the google/ prefix from the model ID.
  • Pinning a regional endpoint for a model served only from global, or using global without checking residency requirements.
  • Granting permissions to a developer account instead of the service identity used by a gateway.
  • Committing a service-account JSON file or printing its contents during troubleshooting.

Verification steps

  1. Run hermes doctor and confirm Vertex credentials resolve without revealing credential material.
  2. Inspect the active project, region, provider, and model ID.
  3. Start a new session and send one short prompt.
  4. Run a harmless read-only tool request to verify the model works in the agent loop, not only as chat.
  5. Restart the long-running service and repeat the request so credential refresh and runtime identity are tested outside the setup shell.

Official documentation

https://hermes-agent.nousresearch.com/docs/guides/google-vertex

Related pages