Build a Human-Governed GitHub PR Review Agent
The official Hermes tutorial describes a polling architecture that needs no public webhook endpoint: a cron job lists open pull requests, retrieves each diff through GitHub CLI, applies consistent review guidance, and delivers a structured summary. A messaging channel is optional; local delivery can be used while proving the workflow. Direct comments, change requests, and approvals are an optional later capability, not a prerequisite for useful automation.[1]
Separate observation, judgment, and action
Design the reviewer in four layers:
- Intake: identify candidate pull requests and fetch metadata.
- Analysis: read the complete diff and produce specific findings.
- Routing: deliver the report locally or to a configured messaging destination.
- GitHub action: optionally post a comment, request changes, or approve.
Keep the first rollout report-only. In this mode, an APPROVE, REQUEST_CHANGES, or COMMENT label is a recommendation for a human reviewer, not an executed repository decision. The documented GitHub-writing path uses gh pr review; when enabled, it acts as the identity authenticated in gh and requires the documented repository permission scope.[1]
Make review policy durable
Test one real pull request manually before scheduling anything. Once the output is useful, encode stable review criteria in a skill so scheduled runs do not depend on an improvised prompt. Keep repository conventions—language, framework, testing expectations, and team rules—in appropriate durable context. Require findings to name the affected location, explain impact, distinguish severity, and suggest a concrete next step. The final verdict should summarize the findings rather than replace them.
The scheduler should have a narrow repository list, a sensible cadence, an explicit delivery destination, and an unambiguous response when nothing needs review. The gateway must be running for cron execution, and GitHub CLI must be installed, authenticated, and available in the gateway environment.[1]
Set approval boundaries before write access
Decide who may authorize each action. A safe default is: the agent may read and report; a human decides whether to post, approve, request changes, merge, deploy, or accept a security exception. If direct posting is later enabled, begin with ordinary comments. Add change requests or approvals only after reviewing false positives, confirming the authenticated identity, and documenting when each verdict is allowed. Keep merge and deployment outside this reviewer unless a separate, explicitly authorized workflow owns them.
Rollout checklist
- Verify GitHub CLI access against the intended repository.
- Run a manual review and inspect the complete diff and findings.
- Store stable review criteria in a dedicated skill.
- Start with local or messaging delivery and no GitHub write action.
- Record the authenticated identity and allowed review actions.
- Enable any direct posting only after a human approves the policy.
Common pitfalls
- Automating before a manual review demonstrates acceptable quality.
- Treating a verdict label as proof that tests passed or code is safe.
- Granting repository-wide write capability when report delivery is sufficient.
- Forgetting that reviews are posted as the current
ghidentity. - Letting generic prompts replace repository-specific review standards.
- Combining review, merge, and deployment into one unbounded automation.
Verification
Use a known pull request with understandable risks. Confirm the agent fetched the full diff, cited specific locations, followed the review skill, and delivered to the intended destination. Verify the cron schedule separately. If GitHub posting is enabled, use a controlled repository first and confirm the visible author, action type, and body before expanding scope.