> ## Documentation Index
> Fetch the complete documentation index at: https://docs.testdino.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Playwright GitHub Status Checks
> Configure GitHub status checks from Playwright results, set quality gate thresholds, and make the TestDino check required to merge.
GitHub CI Checks are automated quality gates that block merges when Playwright test results do not meet your configured rules. Define pass rate thresholds, mandatory tags, and flaky test handling for each environment. TestDino evaluates every run and posts a clear pass or fail signal on the PR.
## Quick Reference
| Setting | Default | Purpose |
| :- | :- | :- |
| [Pass Rate](#pass-rate) | 90% | Minimum percentage of tests that must pass |
| [Flaky Handling](#flaky-handling) | Neutral | How flaky tests affect the check (Strict or Neutral) |
| [Mandatory Tags](#mandatory-tags) | None | Tags that must pass regardless of the overall rate |
| [Environment Overrides](#environment-overrides) | None | Custom rules per branch environment |
## What are GitHub CI Checks?
GitHub CI Checks are automated quality gates that run on your pull requests and commits. TestDino shows a pass or fail signal in GitHub based on the test rules you set. If a required check fails, GitHub blocks the merge.
* When your tests finish, TestDino compares the run against your quality gate settings.
* TestDino posts a green check (passed) or a red check (failed) on the PR or commit.
* The result appears in GitHub, so nobody has to open the dashboard to see it.
TestDino posts these checks through the GitHub App. If you have not connected it yet, start with the [GitHub Integration](/integrations/ci-cd/github).
### Why do CI checks matter?
* Stop unstable or failing code from being merged
* Enforce strict rules for critical branches (like main)
* Use different rules for PROD, STAGE, and DEV
* See failures instantly inside GitHub
* Combine real test signals with GitHub's protection rules
## Quality Gate Settings
These rules determine whether TestDino marks a check as pass or fail.
### Default Settings
By default, the **Pass %** and **Flaky** settings apply to all branches. You can override them later for specific environments such as PROD, STAGE, or DEV.
### Pass Rate
Minimum percentage of tests that must pass for the check to succeed.
* **Range**: 0-100%
* **Default**: 90%
* **Example**: If set to 90%, at least 90% of your tests must pass for the check to be green
### Mandatory Tags
All tests carrying these tags must pass. If even one fails, the whole check fails, no matter how high the overall pass rate is.
* Use the `@` prefix (for example `@critical`, `@payment`, `@auth`)
* Useful for login, payments, security, or any flow that you cannot risk breaking
For example, if you set `@critical` as mandatory and one critical test fails, the check is red even when everything else passes.
Tags come from your Playwright test files. Learn how to add them in [Test Annotations](/guides/playwright-test-annotations).
### Flaky Handling
How flaky tests are treated:
* **Strict**: flaky tests count as failures. Use this for production branches where stability is critical.
* **Neutral**: flaky tests are excluded from the pass rate calculation (default). Use this for development branches to focus on actual failures.
To see how TestDino decides that a test is flaky, read [Flaky Tests](/guides/playwright-flaky-test-detection).
## Environment Overrides
You can create different quality gates for different environments.
### How does it work?
1. Set up branch environments in your project (such as Production, Staging, and Development).
2. Each environment appears as a row in the CI Checks settings table.
3. For each environment, set a custom Pass %, Flaky Handling, and whether Tags apply.
4. If you do not override a setting, the default applies.
Environments come from your branch mappings. Set those up in [Environment Mapping](/guides/environment-mapping).
### Example Configuration
In this example the team lowered its own default to 80%, then tightened the rules for Production.
| Environment | Pass Rate | Flaky Handling | What it means |
| :- | :- | :- | :- |
| Default | 80% | Neutral | Every branch without its own row follows this rule |
| Development | 70% | Neutral | 70% of tests must pass, flaky tests are ignored |
| Staging | 85% | Neutral | 85% must pass, suitable before a deploy |
| Production | 95% | Strict | 95% must pass and no flaky failures are allowed |
## Understanding Check Results
Each GitHub check shows green or red based on your rules.
### Passed: Green Check
Your code meets all quality gate requirements:
* Pass rate meets or exceeds the threshold
* All mandatory tag tests passed
You can merge your PR.
### Failed: Red Check
Your code does not meet quality gate requirements because:
* Pass rate is below the threshold, or
* One or more mandatory tag tests failed
Fix the failing tests before merging.
## Check Details
Click **Details** on the GitHub check to open the full report. It opens on the test results table, where every row links to the matching view in your TestDino dashboard.
The report has up to 3 sections. The last 2 appear only when the environment has mandatory tags configured.
| Section | Shows | Effect on the check |
| :- | :- | :- |
| Test results table | Pass, fail, and flaky counts with links to the dashboard | Drives the pass rate verdict |
| Mandatory tag analysis | Which mandatory tags passed and which failed | A failed tag fails the check |
| Tags not found | Mandatory tags configured here but absent from your tests | None, they are skipped |
A tag under "Tags not found" usually means a typo, or a tag that was renamed in the test files. The check still passes, so fix the configuration or that coverage stays unenforced.
## Making CI Checks Required
This step tells GitHub which checks must pass before a pull request can be merged.
1. Go to **Repository Settings → Rulesets**
2. Create or edit a rule
3. Enable **Require status checks to pass**
4. Click **Add checks**
5. Select **TestDino**
6. Set target branches (for example, main)
7. Save the rule
GitHub now stops merges unless the TestDino CI Check is green.
## Common Scenarios
### High Pass Rate, but the Check Failed
**Situation:** 95% of tests passed, but the check is still red.
**Reason:** A mandatory tag test failed.
**Solution:** Fix the mandatory tag test first. Mandatory tags override the pass rate completely.
### Flaky Tests Causing Failures
**Situation:** The check fails because flaky tests are counted as failures.
**Solution:** Switch the environment to **Neutral** flaky handling, fix the flaky tests, or use **Strict** only on stable branches like Production.
### Different Rule Requirements by Branch
**Situation:** You want strict rules for Production but lighter rules for Development.
**Solution:** Use Environment Overrides. Set Development to a 70% pass rate with Neutral flaky handling, and Production to 95% with Strict.
## Best Practices
| Area | Recommendation |
| :- | :- |
| Starting thresholds | Begin at an 80-90% pass rate with Neutral flaky handling |
| Mandatory tags | Reserve them for critical user flows, security features, and data integrity operations. If everything is mandatory, nothing is |
| Environment rules | Production 95% and Strict, Staging 85-90%, Development 70-80% and Neutral |
| Failed checks | Review the failed tests in the check details, then open the dashboard for the full error context |
| Test stability | Fix flaky tests rather than working around them, because they point at real stability issues |
## Troubleshooting
| Symptom | Likely causes | Fix |
| :- | :- | :- |
| Check not appearing on the PR | GitHub Checks are not enabled; no commit SHA in the test run metadata; repository mismatch between the TestDino project and the GitHub connection | Verify the GitHub connection and enable CI Checks |
| Check always failing | Pass rate set too high; mandatory tags misconfigured; flaky handling too strict | Review your quality gate settings and adjust the thresholds |
| Mandatory tags not working | Tag names do not match (they are case-sensitive); missing `@` prefix; the tests do not carry the tags | Check tag spelling and case, then review "Tags not found" in the check details. TestDino adds the `@` prefix when it is missing, but confirm the tag exists in your tests |
| Environment override not applied | The branch pattern does not match; the environment is not configured in project settings | Verify the branch environment mapping in project settings |
## Related
Install the GitHub App and connect a repository
Map branches to the environments used by overrides
How TestDino classifies a flaky result
Add the tags that mandatory tags rely on
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.
## Quick Reference
| Setting | Default | Purpose |
| :- | :- | :- |
| [Pass Rate](#pass-rate) | 90% | Minimum percentage of tests that must pass |
| [Flaky Handling](#flaky-handling) | Neutral | How flaky tests affect the check (Strict or Neutral) |
| [Mandatory Tags](#mandatory-tags) | None | Tags that must pass regardless of the overall rate |
| [Environment Overrides](#environment-overrides) | None | Custom rules per branch environment |
## What are GitHub CI Checks?
GitHub CI Checks are automated quality gates that run on your pull requests and commits. TestDino shows a pass or fail signal in GitHub based on the test rules you set. If a required check fails, GitHub blocks the merge.
* When your tests finish, TestDino compares the run against your quality gate settings.
* TestDino posts a green check (passed) or a red check (failed) on the PR or commit.
* The result appears in GitHub, so nobody has to open the dashboard to see it.
TestDino posts these checks through the GitHub App. If you have not connected it yet, start with the [GitHub Integration](/integrations/ci-cd/github).
### Why do CI checks matter?
* Stop unstable or failing code from being merged
* Enforce strict rules for critical branches (like main)
* Use different rules for PROD, STAGE, and DEV
* See failures instantly inside GitHub
* Combine real test signals with GitHub's protection rules
## Quality Gate Settings
These rules determine whether TestDino marks a check as pass or fail.
### Default Settings
By default, the **Pass %** and **Flaky** settings apply to all branches. You can override them later for specific environments such as PROD, STAGE, or DEV.
### Pass Rate
Minimum percentage of tests that must pass for the check to succeed.
* **Range**: 0-100%
* **Default**: 90%
* **Example**: If set to 90%, at least 90% of your tests must pass for the check to be green
### Mandatory Tags
All tests carrying these tags must pass. If even one fails, the whole check fails, no matter how high the overall pass rate is.
* Use the `@` prefix (for example `@critical`, `@payment`, `@auth`)
* Useful for login, payments, security, or any flow that you cannot risk breaking
For example, if you set `@critical` as mandatory and one critical test fails, the check is red even when everything else passes.
Tags come from your Playwright test files. Learn how to add them in [Test Annotations](/guides/playwright-test-annotations).
### Flaky Handling
How flaky tests are treated:
* **Strict**: flaky tests count as failures. Use this for production branches where stability is critical.
* **Neutral**: flaky tests are excluded from the pass rate calculation (default). Use this for development branches to focus on actual failures.
To see how TestDino decides that a test is flaky, read [Flaky Tests](/guides/playwright-flaky-test-detection).
## Environment Overrides
You can create different quality gates for different environments.
### How does it work?
1. Set up branch environments in your project (such as Production, Staging, and Development).
2. Each environment appears as a row in the CI Checks settings table.
3. For each environment, set a custom Pass %, Flaky Handling, and whether Tags apply.
4. If you do not override a setting, the default applies.
Environments come from your branch mappings. Set those up in [Environment Mapping](/guides/environment-mapping).
### Example Configuration
In this example the team lowered its own default to 80%, then tightened the rules for Production.
| Environment | Pass Rate | Flaky Handling | What it means |
| :- | :- | :- | :- |
| Default | 80% | Neutral | Every branch without its own row follows this rule |
| Development | 70% | Neutral | 70% of tests must pass, flaky tests are ignored |
| Staging | 85% | Neutral | 85% must pass, suitable before a deploy |
| Production | 95% | Strict | 95% must pass and no flaky failures are allowed |
## Understanding Check Results
Each GitHub check shows green or red based on your rules.
### Passed: Green Check
Your code meets all quality gate requirements:
* Pass rate meets or exceeds the threshold
* All mandatory tag tests passed
You can merge your PR.
### Failed: Red Check
Your code does not meet quality gate requirements because:
* Pass rate is below the threshold, or
* One or more mandatory tag tests failed
Fix the failing tests before merging.
## Check Details
Click **Details** on the GitHub check to open the full report. It opens on the test results table, where every row links to the matching view in your TestDino dashboard.
The report has up to 3 sections. The last 2 appear only when the environment has mandatory tags configured.
| Section | Shows | Effect on the check |
| :- | :- | :- |
| Test results table | Pass, fail, and flaky counts with links to the dashboard | Drives the pass rate verdict |
| Mandatory tag analysis | Which mandatory tags passed and which failed | A failed tag fails the check |
| Tags not found | Mandatory tags configured here but absent from your tests | None, they are skipped |
A tag under "Tags not found" usually means a typo, or a tag that was renamed in the test files. The check still passes, so fix the configuration or that coverage stays unenforced.
## Making CI Checks Required
This step tells GitHub which checks must pass before a pull request can be merged.
1. Go to **Repository Settings → Rulesets**
2. Create or edit a rule
3. Enable **Require status checks to pass**
4. Click **Add checks**
5. Select **TestDino**
6. Set target branches (for example, main)
7. Save the rule
GitHub now stops merges unless the TestDino CI Check is green.
## Common Scenarios
### High Pass Rate, but the Check Failed
**Situation:** 95% of tests passed, but the check is still red.
**Reason:** A mandatory tag test failed.
**Solution:** Fix the mandatory tag test first. Mandatory tags override the pass rate completely.
### Flaky Tests Causing Failures
**Situation:** The check fails because flaky tests are counted as failures.
**Solution:** Switch the environment to **Neutral** flaky handling, fix the flaky tests, or use **Strict** only on stable branches like Production.
### Different Rule Requirements by Branch
**Situation:** You want strict rules for Production but lighter rules for Development.
**Solution:** Use Environment Overrides. Set Development to a 70% pass rate with Neutral flaky handling, and Production to 95% with Strict.
## Best Practices
| Area | Recommendation |
| :- | :- |
| Starting thresholds | Begin at an 80-90% pass rate with Neutral flaky handling |
| Mandatory tags | Reserve them for critical user flows, security features, and data integrity operations. If everything is mandatory, nothing is |
| Environment rules | Production 95% and Strict, Staging 85-90%, Development 70-80% and Neutral |
| Failed checks | Review the failed tests in the check details, then open the dashboard for the full error context |
| Test stability | Fix flaky tests rather than working around them, because they point at real stability issues |
## Troubleshooting
| Symptom | Likely causes | Fix |
| :- | :- | :- |
| Check not appearing on the PR | GitHub Checks are not enabled; no commit SHA in the test run metadata; repository mismatch between the TestDino project and the GitHub connection | Verify the GitHub connection and enable CI Checks |
| Check always failing | Pass rate set too high; mandatory tags misconfigured; flaky handling too strict | Review your quality gate settings and adjust the thresholds |
| Mandatory tags not working | Tag names do not match (they are case-sensitive); missing `@` prefix; the tests do not carry the tags | Check tag spelling and case, then review "Tags not found" in the check details. TestDino adds the `@` prefix when it is missing, but confirm the tag exists in your tests |
| Environment override not applied | The branch pattern does not match; the environment is not configured in project settings | Verify the branch environment mapping in project settings |
## Related