Build a Hermes Video Generation Provider Plugin

Implement a video backend that routes text-to-video and image-to-video through one capability-aware Hermes tool contract. This guide focuses on the public implementation contract; the official Hermes documentation remains the source of truth as interfaces evolve.

How the interface works

Video providers register behind video_generate and mirror the image-provider pattern. The presence of image_url selects image-to-video; its absence selects text-to-video.

Capabilities should state supported modalities, aspect ratios, resolutions, duration limits, audio, negative prompts, and reference-image limits. The tool description can then reflect the active backend accurately.

Model catalogs should represent user-facing model families. A provider may route each family to different text and image endpoints internally without exposing those details to the agent.

Use standardized response helpers and return either a remote video URL or an absolute cached path. Providers should ignore unknown future keyword arguments for compatibility.

Practical checklist

Define honest capability bounds before implementation.
Clamp duration, aspect ratio, and resolution to supported values.
Keep model-family selection separate from endpoint routing.
Handle long-running provider failures and missing credentials as structured errors.
Verify the actual media container, duration, dimensions, and playback.

Common pitfalls

Claiming support for audio or image input that the endpoint does not honor.
Creating one catalog entry per internal endpoint instead of per user-facing family.
Returning before an asynchronous job has a durable result URL.
Calling a file complete because it exists without opening or probing it.

Verification steps

Run tests for text and image modality routing.
Check missing-auth, rejected-input, timeout, and provider-error paths.
Probe the returned media file and confirm it plays.
Deliver the artifact through the intended gateway or UI and verify the attachment path.

Official reference

https://hermes-agent.nousresearch.com/docs/developer-guide/video-gen-provider-plugin