TestingOps.ai

Issue trackers, wikis and code · Native connection

GitHub and TestingOps: AI test agents on your repository's issues

TestingOps reads the issues, pull requests and README of a GitHub repository into the project's knowledge base, lets AI test agents act on issues as they move through your columns and reports defects back as GitHub issues.

  • Native connection
  • Available
Request a demo GitHub Docs: managing your personal access tokens
Connection
Native connection
Writes back
Within what you allow
Last checked
7 Nov 2026

In short

GitHub and TestingOps in short

The GitHub connection links one repository on github.com or GitHub Enterprise Server to a TestingOps project. Its issues, pull requests and README become context that agents search when they write and review test cases.

With the Tester Agent, a repository webhook delivers issue and comment events. Labels act as board columns, so giving an issue your test-column label can start the agent's work, and its result comes back as a comment.

You connect with Connect with GitHub or a personal access token. The agent's comments and label changes need the link's "Agents may write to this tracker" switch.

What it does

What it does for your testing

  1. 1

    Issues and pull requests become test context

    Input
    The repository's issues and pull requests with title, body and state, and its README
    TestingOps
    Keeps the 300 most recently updated as requirements in the project's knowledge base; the README becomes a know-how document
    Output
    Context agents search when they write or review test cases
  2. 2

    An issue moves through your columns

    Input
    An issue that gets the label of a column in your board rules
    TestingOps
    Runs the steps the rule gives that column: refine the issue, propose acceptance criteria, write draft cases or run the cases linked to it
    Output
    Markdown comments on the issue and, after approval, sections added to its body
  3. 3

    A defect becomes a GitHub issue

    Input
    A TestingOps defect with steps, expected and actual result
    TestingOps
    Creates an issue in the linked repository with the label bug and a line naming the TestingOps defect
    Output
    The issue link stored on the defect
  4. 4

    A deployment starts a test plan

    Input
    A successful GitHub deployment status for an environment you name
    TestingOps
    Starts the run of the test plan the deploy trigger belongs to, on your execution provider (RunningOps on testingops.ai)
    Output
    A run whose verdict a CI step can wait for before the pipeline continues

Step by step

How it works

  1. Connect GitHubUnder Integrations → Project tools, open GitHub and use Connect with GitHub, or enter a personal access token. For GitHub Enterprise Server, enter the server address as the base URL and use a token.
  2. Link a repositoryIn Knowledge Base, add the repository (owner/repo) as a Code source of a project. Each sync reads the 300 most recently updated issues and pull requests and the README, hourly and on Sync now.
  3. Agents use it as contextAgents search these records with the rest of the project's knowledge, and open single files of the repository through the GitHub API when a task needs the code. Nothing is cloned.
  4. Connect the Tester Agent (optional)Generate a webhook address on the GitHub connection, add it as a repository webhook for Issues and Issue comments, and describe in Board rules what the agent does in each column.
  5. Results go back to GitHubThe agent comments in Markdown and swaps column labels; defects reported from TestingOps become issues labelled bug.

GitHub issues are only open or closed, so a column is a label with the same name as a status in your board rules. GitHub Projects status fields are not read.

Connect

Setup

  1. In TestingOps, open Integrations → Project tools, select GitHub and choose Connect with GitHub, or enter a personal access token. For GitHub Enterprise Server, also enter its address under Base URL (optional).
  2. Select Test connection, then Connect.
  3. In Knowledge Base, pick the project, select + Add source under Code, choose the GitHub connection, pick or type owner/repo and select Link to RAG.
  4. Turn on Agents may write to this tracker on the source if agents should comment on or change issues.
  5. Optional: on the GitHub connection select Generate webhook address and add it in the repository under Settings → Webhooks → Add webhook, with content type application/json and the Issues and Issue comments events; then open Board rules.
  6. Optional: on a test plan select Run on deploy, add a deploy trigger and register its address and key as a GitHub webhook for Deployment statuses.

Documentation: GitHub Docs: managing your personal access tokens · GitHub Docs: creating webhooks

Where you connect it in TestingOps: Integrations → Project tools → Issue trackers, wikis and code → GitHub

Use cases

Use cases

Write test cases from feature issues

Trigger
A feature issue gets the label of your refinement or test-design column
Uses
The issue body and nearby entries of the project's knowledge base
Agent activity
Proposes a user story, Given/When/Then acceptance criteria with at least one negative or edge scenario and open questions, then writes draft cases from the approved criteria
Result
A proposal comment, then draft cases linked to the issue
Condition
Approve mode is the default; a person answers /testingops approve, edit or reject

Talk to the agent in a comment

Trigger
Someone writes /testingops in an issue comment
Uses
The issue and its comments
Agent activity
Answers the question or acts on an approval or edit of its pending proposal
Result
A reply comment in Markdown
Condition
Needs the repository webhook with Issue comments; pull request comments are not read

Run the regression plan after a deployment

Trigger
GitHub reports a successful deployment to the environment you named
Uses
A Run on deploy trigger on a TestingOps test plan
Agent activity
TestingOps starts the plan's run; a CI step can poll the gate and stop the pipeline when the pass rate is below your threshold
Result
A run on RunningOps and a pass or fail verdict
Condition
The webhook only starts the run; holding the pipeline needs the CI step

Report a defect to the repository

Trigger
A tester selects Report to… on a defect
Uses
The defect's severity, priority, environment, steps, expected and actual result
Agent activity
Creates the issue in the linked repository
Result
An issue labelled bug, linked from the defect
Condition
Needs a token or GitHub sign-in on the connection

Capabilities

Capabilities and limits

What is read

  • Issues and pull requests of the linked repository: title, body, state and last update; the 300 most recently updated per sync
  • The README, as a know-how document
  • On demand for agents: an issue with its comments, issue search in the repository, folders, single files up to 200 KB and code search through the GitHub API
  • For the Jira Tester Agent: commits and branches whose message or name carries a Jira key, and the files they changed (last 90 days, up to five linked repositories)
  • Tester Agent events: issues opened, labelled, closed or reopened, and new issue comments; pull requests are ignored

What is produced

  • Requirements and a README document in the project's knowledge base
  • Refinement proposals, acceptance criteria and draft test cases linked to an issue
  • Test plan runs started by deployments

Where it can write

  • Issues labelled bug, when someone selects Report to… on a defect
  • With "Agents may write to this tracker": comments, open or close, new issues, and title, label, assignee or body changes
  • Tester Agent: Markdown comments, approved sections in the issue body, and column labels
  • Not written: code, branches, commit statuses or pull request reviews

How it connects

  • github.com: Connect with GitHub (scopes repo and read:user) or a personal access token
  • GitHub Enterprise Server: base URL and a personal access token
  • Tester Agent: a repository webhook to an address generated on the connection
  • Run on deploy: a repository webhook for Deployment statuses, or a CI step, per test plan trigger

Account and permissions

  • TestingOps reads and writes with the permissions of the token owner or the signed-in GitHub user
  • A public repository can be read without a token; writing and agent tools need one
  • A repository administrator adds the webhooks

Limits

  • 300 issues and pull requests per sync; comments and attachments are not stored
  • GitHub pushes are not received, and pull request events do not start agent rules
  • GitHub Projects status fields are not read; columns are labels
  • Agents read files from one repository per project, the first one linked
  • Code search uses GitHub's index of the default branch and may be unavailable on Enterprise Server
  • A deployment webhook carries no changed files, so the whole plan runs

Example

Example: how a team uses it

Illustrative scenario, not a customer story.

A team keeps feature requests as issues in its web-shop repository. When the issue for a new coupon field gets the label Ready for test, the Tester Agent proposes acceptance criteria, including an expired coupon, and comments them in Markdown.

A team member answers /testingops approve, and the agent writes four draft cases linked to the issue. When GitHub later reports a successful deployment to staging, the team's regression plan runs on RunningOps and the pipeline waits for its verdict.

Frequently asked questions

Does TestingOps analyse all the code in my repository?

No. A linked repository brings its issues, pull requests and README into the knowledge base. Agents open single files through the GitHub API when a task needs them, and the Tester Agent for Jira reads commits and branches that carry an issue key. Nothing is cloned.

Can the agent react to pull requests being opened or merged?

No. The Tester Agent listens to issue and issue comment events. Pull request events do not start agent rules, and GitHub pushes are not received.

How do board columns work on GitHub?

GitHub issues are only open or closed, so columns are labels. Give an issue a label with the same name as a status in your board rules and it counts as moved there; when the agent moves an issue, it removes the other column labels and adds the new one.

Do I need a token for a public repository?

Reading a public repository works without one. Comments, label changes, agent tools and defect filing need a personal access token or GitHub sign-in.

Does it work with GitHub Enterprise Server?

Yes. Enter the server address as the base URL and use a personal access token; Connect with GitHub works for github.com only. A server that is only reachable inside your network can be reached through a private network gateway.

What does TestingOps write to GitHub?

Issues labelled bug when someone reports a defect and, with the link's write switch on, the agent's comments, labels, approved body sections and open or close changes. It does not push code, create branches or review pull requests.

Next step

See the Tester Agent work on a GitHub repository: request a demo and bring an issue that is ready for testing.

Request a demoMore project tools integrations