Skip to main content
When you connect a GitHub repository to AgentMark, every push gets a status check on the commit: AgentMark / Build. If the deploy would fail, the PR shows red. You can make the check required so it blocks merge.

​
What gets checked

The check runs every prompt and component file in your push through the same TemplateDX compiler the runtime uses. If a file would fail to load at request time, the check fails, with that file annotated on the PR. What surfaces as a failure:
  • Template syntax errors. Malformed MDX/JSX in .prompt.mdx or .mdx files (unclosed tags, stray HTML comments, unexpected characters).
  • Frontmatter errors. Missing text_config/object_config/image_config/speech_config, invalid model names, or missing required fields.
  • Schema reference errors. Malformed $refs that fail at parse time.
What the check doesn’t cover:
  • Your handler’s TypeScript or Python build. That runs on the code-deploy step, and surfaces in the AgentMark Dashboard under the deployment’s build logs.
  • Files outside the directory set by agentmarkPath in agentmark.json. AgentMark Cloud validates only the files it imports.
  • $refs and imports that point at files outside the push. The check can’t validate those standalone, so it skips them rather than failing, which means a typo’d schema path passes the check and surfaces later at load time.

​
Conclusion states

The check reports one of three conclusions per push:

​
Making the check required

GitHub’s required status check setting is what blocks merge. To enforce the check:
  1. In your GitHub repo, go to Settings → Branches → Branch protection rules.
  2. Add a rule for the branch you deploy from (usually main).
  3. Enable Require status checks to pass before merging.
  4. In the search box, type AgentMark and select AgentMark / Build.
    The check name only appears in GitHub’s picker after the first push that triggers it. If you don’t see it, push any commit to the repo first, then come back to this screen.
  5. Save.
From now on, any PR targeting that branch can’t merge until the check passes.

​
What a failure looks like

When a prompt fails to compile, the PR’s Files changed tab gets an inline annotation on the offending file. The annotation title categorizes the failure so you can spot the class of problem from the sidebar without opening each file:
  • Frontmatter error. Fix the --- block at the top of the prompt.
  • Template syntax error. MDX/JSX in the body didn’t parse.
  • Schema reference error. A malformed $ref failed at parse time.
  • Prompt compile error. Generic fallback.
Annotations include the parser’s line and column when the underlying templatedx error carries them. Frontmatter errors without a line number fall back to line 1 (the start of the frontmatter block).

​
Hardcoded merge blocking via CI

The Cloud-managed status check above takes zero config beyond installing the App, but it depends on AgentMark Cloud being reachable. For a fully self-hosted alternative that blocks merge from any CI provider, run agentmark build as a CI job in your own repository. The CLI compiles every prompt with the same compiler the runtime uses and exits non-zero on failure, which fails the pipeline and triggers the provider’s built-in “pipelines must succeed” merge guard. This pattern has two advantages over (or alongside) the Cloud-managed check:
  • Doesn’t depend on AgentMark Cloud uptime. The validation runs entirely inside your CI. If the AgentMark webhook handler degrades, your merge guard still works.
  • Faster feedback for big repos. No webhook round-trip; the CI job runs in seconds against the pushed commit directly.
agentmark build is the validation gate (does every prompt compile?). To also gate on eval results (pass rate and score regressions) in the same pipeline, see CI/CD.

​
GitLab CI

.gitlab-ci.yml
Then in Settings → Merge requests → Merge checks, enable Pipelines must succeed. From now on, an MR can’t merge until the agentmark-build job is green.

​
GitHub Actions

.github/workflows/agentmark-build.yml
The name: on the job matters: GitHub’s required-status picker lists Actions checks by job display name, not the workflow name, so without it the check appears as build. Then in Settings → Branches → Branch protection rules, add a rule requiring the AgentMark / Build check name. Same picker, same UX as the App-based check: your CI job’s status takes the place of (or runs alongside) the App’s.
You can run both the Cloud-managed check and the self-hosted CI check in parallel. They use the same name (AgentMark / Build) but appear as separate entries in the required-status picker on GitHub; require both and the merge blocks if either fails.

​
Re-running the check

There’s no “Re-run” button. AgentMark keys the check to the commit SHA, so pushing a new commit (even an empty one) is the way to retry:

​
Limitations

  • PRs from forks don’t get checks. AgentMark posts the check from the push webhook, which forks don’t send to the upstream App. PRs from branches in the same repo do get checks.
  • No retries on transient outages. If the status API itself errors when AgentMark posts the result, AgentMark logs the error and the deploy still runs. The check sits at in progress until the provider’s timeout. You can re-push to recover.
  • One check per push, not per file. The aggregate conclusion comes from every prompt file in the change set.

​
Have questions?

Reach out any time: