Reviews and quality

Quality Intelligence — test automation from your source code

Generate a complete, runnable test automation project from your application's repository, even with no tests today; run it against your test environment, read why tests fail, and get locator repairs as draft pull requests.

Quality Intelligence (Quality › Quality Intelligence) is switched on per organisation. It needs a project whose repository holds the application, and one of your runners (7.5 or later; one with a browser for web tests).

Before you start

A test environment. The address of a test or staging copy of the application. Yavaren never runs tests against an environment marked production.
A test account (optional, recommended). Create the user name and password as secrets of your organisation (for example acme/qa-user and acme/qa-password) and give Yavaren their NAMES. The values stay in your secret store; only your runner reads them, and only while the tests run.
A disposable database (optional, for database tests). A secret holding the address of a PostgreSQL database that holds no real data. The tests create their own schema in it and remove it afterwards.

The twelve steps

Choose the repository. Generate test automation from my application, then the project and repository.
Analyse. Read-only: nothing in your repository changes. It takes a minute or two on your runner.
Review what was found. Pages, forms, API endpoints, tables and screens, each with the file and line it comes from. Found in the source is not the same as seen running: a run confirms it.
Choose what to automate. Web, API, database, mobile — each layer can be left out.
Review the framework. If you already have tests, Yavaren extends that framework. Otherwise it recommends one for your team's language (for example Playwright in TypeScript) and says why, what it needs and roughly what a run costs. You can choose another.
Configure the environment (the Environment tab): address, whether tests may create data.
Approve the strategy. Only a person can, for the analysis they reviewed. Nothing is generated before.
Generate the framework. A complete project under automation/: configuration, fixtures, page objects, test data helpers, a CI job and a README.
Generate the tests. In the same step: one test per designed case, each citing its evidence.
Run. Yavaren installs, checks and runs the project before proposing it.
Review results and coverage. A test counts only when it ran and passed.
Draft pull request (or a zip to download). Review and merge it like any pull request.

Reading the tests

Every test starts with a comment: what it checks and where that comes from, for example # evidence: app/templates/login.html:10. A test never asserts something your code does not show. When Yavaren cannot tell — what a person should see after signing in, which roles may delete — the test is pending: it skips and says why, and the question is listed in the README.

When tests fail

The Runs tab says why each failed test failed, with how sure Yavaren is:

Application defect — the application answered differently from what its code declares. Check the application.
Test-script defect — an element the test looks for is still there under a new name or test id. Propose a repair (locators only) changes only pages/locators.*, never a test or an assertion, re-runs the affected tests and opens a second draft pull request when they pass.
Environment, test account, test data, timing, outside service — what to check first is named.

Running the generated project yourself

automation/run.sh all        # or ui | api | database | mobile

with BASE_URL (and API_BASE_URL), QA_USERNAME / QA_PASSWORD for a test account, QA_ALLOW_WRITES=1 where tests may create data, DATABASE_URL + QA_DATABASE_DISPOSABLE=1 for the database suite, QA_DEVICE=true where an emulator runs. The CI job stays off until you set the repository variable QA_AUTOMATION_ENABLED to true and QA_BASE_URL to your test environment.

What it does not do

It does not run tests against production, make real payments, send messages to real people, merge anything, change your application's code, or claim a mobile test ran without an emulator or simulator. Mobile flows are generated and checked; running them needs a runner with an emulator or simulator, which your administrator declares on that runner.