# 04 · Testing and Verification Every test suite in Rusukh, how to run it, and — more usefully — how to read the result when it goes red. --- ## 1. The suites at a glance | Suite | What it proves | Runtime | |---|---|---| | **Unit** | Money arithmetic, state machines, gates, commission maths, SLA calculations, refund calculators | seconds | | **Feature** | Every screen × every role, positive and negative, plus the full money chain | ~10 min | | **Architecture** | The rules that keep the codebase honest — module boundaries, no float money, no raw status queries, no hand-built Actions | ~30 s | | **Concurrency** | Real multi-process races — two people doing the same thing at the same instant | ~3.5 min | | **Playwright E2E** | The application in a real browser at three screen sizes | ~19 min | | **Evidence capture** (§8) | Not verification — it photographs the application for the client handbook, and refuses to photograph an error or a redirect | ~9 min | Two of these carry checks worth naming, because both were written after a green suite was found to be proving less than it appeared to: - **Reachability.** `NavigationReachabilityTest` walks all thirteen staff roles and asserts that every navigation entry a role is *offered* actually opens for them — a link gated on the wrong permission renders happily and answers 403 when clicked. `PortalReachabilityTest` does the mirror image for the customer portal: it starts at `/account` and follows links, so a screen nothing links to fails the build instead of hiding in it. - **Landing paths.** Every browser navigation now asserts *which* page it arrived at, not merely that something rendered (§3). --- ## 2. Running them ### Everything except concurrency ```bash php artisan test --parallel --exclude-group=concurrency ``` ### The concurrency group — must run alone ```bash php artisan test --group=concurrency ``` > **Why separately:** these tests spawn **real operating-system processes** and release them > against a wall-clock barrier to create genuine race conditions. They manage their own > inter-process concurrency, which is fundamentally incompatible with Pest's `--parallel` worker > splitting. Running them inside a parallel run produces meaningless failures. ### A single file, or a single test ```bash php artisan test tests/Feature/Production/CuttingWorkflowTest.php php artisan test --filter="refuses a cutting job without a bundle photograph" ``` ### The browser suite ```bash npx playwright test --workers=1 ``` **Use `--workers=1` for a full local run.** `php artisan serve` serves one request at a time, and every image and stylesheet queues behind every page — a warm `/login` takes about 750ms on the development machine. Parallel browsers do not make that faster, they make it queue. This is not excessive caution. On 2026-09-05 the same commit produced 256/2, then 256/2 with a *different* two, then 260/1, then 88 failures at once at `--workers=2`; each result had to be investigated to find out whether the application was at fault, and only one of them was. At `--workers=1` it ran **261/0 in 18.9 minutes**. Two workers save roughly two minutes of wall clock and cost far more than that in false leads. CI, running against a real multi-process server (php-fpm behind nginx), does not need this. Narrow it down when chasing something specific: ```bash npx playwright test --project=desktop-1440 # one viewport npx playwright test e2e/customer-portal/payment.spec.ts # one file npx playwright test --workers=1 --headed # watch it happen npx playwright show-report # after a run ``` The three viewport projects are **desktop-1440**, **tablet-768** and **mobile-375** — every responsive test runs at all three. ### Style and static analysis ```bash ./vendor/bin/pint --test # code style, check only ./vendor/bin/pint # code style, fix ./vendor/bin/phpstan analyse # static analysis ``` --- ## 3. Reading a failure honestly This project has been burned repeatedly by failures that were not what they appeared to be. Before you believe a red result, rule these out — in this order. ### Is something else using the test database? By far the most common false alarm. If two things run `RefreshDatabase` against `rusukh_testing` at once, you get a flood of `relation "..." does not exist` errors that look catastrophic and mean nothing. ```bash # Linux/macOS ps aux | grep -E "artisan test|pest" # Windows PowerShell Get-CimInstance Win32_Process -Filter "name='php.exe'" | Select-Object ProcessId, CommandLine ``` If you need a guaranteed-clean database while other work is in flight, **create a scratch one rather than resetting the shared one**: ```bash psql -c "CREATE DATABASE rusukh_testing_scratch TEMPLATE template0;" sed 's/rusukh_testing/rusukh_testing_scratch/' phpunit.xml > phpunit.scratch.xml php artisan test --configuration=phpunit.scratch.xml tests/Feature/Orders # Clean up rm phpunit.scratch.xml && psql -c "DROP DATABASE rusukh_testing_scratch;" ``` The extensions no longer need creating by hand — the first migration installs them, and `RefreshDatabase` runs migrations on the scratch database before the first test. `phpunit.*.xml` is gitignored, so a scratch config left behind will not end up in a commit; delete it anyway. ### Is the dev server saturated? **The signature: several _unrelated_, previously-green specs failing together on timeouts** — and especially `auth.setup.ts` among them, since a failed sign-in cascades into everything that depends on a stored session. That is one overloaded single-process server, not a code change that broke nine screens at once. ```bash # Windows PowerShell — find the server, then restart it Get-CimInstance Win32_Process -Filter "name='php.exe'" | Select-Object ProcessId, CommandLine ``` A copy of `php artisan serve` left running for a day answered `/login` in **2.5 seconds** under two workers and produced 88 failures; a fresh one on the same commit produced none. Re-running without restarting simply reproduces it. ### Is it a Playwright contention artefact? A browser test that fails in a full run and passes alone was contention: ```bash npx playwright test --workers=1 e2e/customer-portal/payment.spec.ts ``` **If it fails alone too, it is real — and take that seriously rather than re-running.** This is exactly how the worst test defect in the project was found: the payment spec had been submitting real payments and reporting "every seeded order has already been paid off", because it checked for its success message without waiting, immediately after an assertion that matched a *stale* error left on screen by the previous attempt. Its own failure screenshot listed the receipt it had just created. Two earlier explanations for that failure — a tight timeout, then two viewport projects racing each other — were both real problems, both worth fixing, and neither was the cause. ### Only then, believe it Once you have ruled out both, treat the failure as genuine and fix the code — not the test. ### And the opposite question: is a *passing* test looking at the page it thinks it is? The failures above are the noisy problem. This is the quiet one, and it has cost this project more. `page.goto()` follows redirects and reports the **final** response. If a middleware bounces the request somewhere else, the status is still 200, and every assertion that follows runs against a page the test never meant to visit. When the destination is also a valid screen, nothing looks wrong. That is not hypothetical. `EnsureTwoFactor` guards every `/admin` request for `super_admin`, `management` and `finance`, and the seeded users in those roles were not enrolled — so all three were redirected to `/admin/identity/security/two-factor` and stayed there. The enrolment screen answers 200, renders `#main-content`, throws no console error and does not overflow, which is every assertion the 61-screen sweep made. **25 of those 61 screens were being verified against the enrolment page instead of themselves**, at three viewports each, for as long as that suite had existed. The fix is one line per navigation, and it is worth applying anywhere a test navigates: ```ts expect(new URL(page.url()).pathname).toBe(screen.path); ``` `smoke/all-admin-screens.spec.ts` now asserts this, and so does the evidence capture's `visit()` helper (§8), which refuses to photograph a screen that redirected. Paths that are genuinely `Route::redirect` aliases carry a `redirectsTo` field in `e2e/admin-screens.json` and are checked against where they are meant to land. **The general lesson:** a test that only asserts "something rendered and nothing exploded" is satisfied by almost any page. Assert *which* page. --- ## 4. The architecture suite, and why you must not "fix" it by editing it These tests encode decisions rather than behaviour. When one goes red, the code is wrong. | Test | The rule | |---|---| | `ModuleDependencyTest` | A module may only reference another's `Domain\{Contracts,Enums,Events,ValueObjects,Models}`. Reaching into another module's `Application` layer is forbidden — publish a contract instead. | | `LayerBoundariesTest` | Domain may not use Infrastructure; Domain may not touch facades. | | `NoRawFlagQueriesTest` | Never `where('status', …)` or `where('is_…', …)`. Use the generated scopes: `withStatus()`, `withoutStatus()`, `current()`. | | `NoFloatMoneyTest` | Money is **integer paisa** in the `Money` value object; percentages are **basis points**. No float may go near a currency value. | | `StrictTypesTest` | Every PHP file in `app/` starts with `declare(strict_types=1)`. | | `NoEnvOutsideConfigTest` | `env()` is called only inside `config/`. | | `NoDebugStatementsTest` | No `dd()`, `dump()`, `ray()` or `var_dump()` ships. | | `ModuleBootTest` | Every module's service provider actually boots. | | `BladeSafetyTest` | No unescaped output where it would be an XSS vector. | `NoRawFlagQueriesTest` in particular has caught real regressions repeatedly — including ones introduced by fixes for other bugs. The rule is not self-enforcing just because a test exists for it; read the failure, then fix the query. If a rule genuinely must be broken, there is a path: a **justified, path-keyed allow-list entry** under `tests/Architecture/allowlists`, which is itself test-guarded so the exception cannot rot silently. Editing the test to make it pass is not the path. --- ## 5. What the concurrency suite covers Twelve documented race conditions from `docs/02-ARCHITECTURE.md` §7.4, each proven with real OS processes rather than a single-connection simulation. A representative few: | Race | The scenario | |---|---| | R1 | Two reps allocate the last fabric lot — exactly N must succeed, not N+1 | | R3 | PayFast sends the same payment notification five times with different hashes | | R4 | A customer pays online while a cashier marks the same order COD-received | | R6 | A journal entry lands inside a "locked" accounting period after the snapshot | | R7 | Two QC officers approve the same garment simultaneously | | R10 | Stock is issued for an order that was cancelled a moment earlier | | R11 | A commission payout is approved four times at once | Every one of these fixes was proven by reverting it and watching the new test fail first — a test that has never failed has not been shown to test anything. --- ## 6. Before you call a release ready ```bash # 1. Backend, in two parts php artisan test --parallel --exclude-group=concurrency php artisan test --group=concurrency # 2. Style and types ./vendor/bin/pint --test ./vendor/bin/phpstan analyse # 3. Browser, all three viewports. # `npm run build` is not optional here: Tailwind only emits the classes it # finds in the templates at build time, so a template change plus a stale # stylesheet gives you unstyled markup and spurious overflow failures. npm run build npx playwright test --project=setup --workers=1 # refresh the stored logins npx playwright test --workers=1 # 4. Regenerate the client handbook if any screen changed (§8) npx playwright test -c e2e/evidence/evidence.config.ts DB_DATABASE=rusukh_demo php artisan handbook:export-reference node e2e/evidence/build-handbook.mjs ``` All green, with **nothing skipped, deleted or weakened to get there**. A suite that reached green by removing a test proves less than the red one it replaced. > **Run the numbers you intend to quote on the PHP you intend to deploy.** `composer.json` pins > `config.platform.php` to **8.2.28** and the deployment guide specifies PHP 8.2; `require.php` is > `^8.1`, so 8.3 is inside the supported range and the suite passes on both. On the development > machine bare `php` is 8.3.30, so a headline figure taken without thinking is an 8.3 figure. --- ## 7. Proving a clean install actually works A passing test suite does not prove the application can be *installed*. Those are different claims, and on 2026-09-05 the second one was false while the first was true: the schema needed two Postgres extensions that no migration created, so every test passed on two databases that had been provisioned by hand a year earlier while `php artisan migrate` against a new database failed outright. Run this before any handover or any first deployment: ```bash # An empty database that shares nothing with your working one. psql -c "CREATE DATABASE rusukh_demo TEMPLATE template0;" # Point one command at it. The env var wins over .env — verify that first, # because a migrate:fresh that ignored it would destroy your real database. DB_DATABASE=rusukh_demo php artisan tinker \ --execute="echo DB::connection()->getDatabaseName();" # must print rusukh_demo DB_DATABASE=rusukh_demo php artisan migrate:fresh --seed --force ``` It must reach the end with no error, from nothing but `CREATE DATABASE`. Then check that the demo data is *honest* rather than merely present — the stage gates are the test worth running, because they are the one place seeded data can look complete while proving nothing: ```sql SELECT (SELECT count(*) FROM stage_gates) AS gates, (SELECT count(*) FROM stage_gate_evidence) AS evidence; ``` **Expected: 151 gates and 569 evidence rows.** A gate row without evidence behind it is decorative — the gate register displays the stored `passed` status, so the board looks correct, but the evaluator recomputes from evidence the moment anyone tries to advance a garment, and refuses. Verified on 2026-09-05: all 151 seeded gates re-evaluate as genuinely passing on a fresh install. ### Which database to demonstrate from **Use `rusukh_demo`.** It was built by the procedure above and verified: the gates hold up, the approval inbox carries real decided requests, and nothing in it came from a test run. ```bash # In .env, for a demo session only: DB_DATABASE=rusukh_demo ``` `rusukh` — the everyday development database — is deliberately **not** the one to demo from. It was seeded before the stage-gate work landed, so it holds 286 gate rows with **zero** evidence behind them: the board looks right and the first attempt to advance a garment is refused. It also holds around 40 orders created by browser-test runs rather than by the seeder, which is why it is not simply regenerated. Reseeding it is a decision for whoever owns the data, not a step in a guide; a `pg_dump` is in `storage/backups/` either way. --- ## 8. Regenerating the illustrated handbook [`07-ILLUSTRATED-HANDBOOK.html`](07-ILLUSTRATED-HANDBOOK.html) and its PDF are **generated**, not written. Every screenshot in them is taken by a Playwright run that signs in as a real seeded role and visits a real URL, so a UI change is answered by re-running three commands rather than by re-cropping images. ```bash # 1. Take the pictures — signs in as all 14 accounts, walks 128 captures. npx playwright test -c e2e/evidence/evidence.config.ts # 2. Read the role, permission and approval tables out of the database. DB_DATABASE=rusukh_demo php artisan handbook:export-reference # 3. Write the HTML and print the PDF from it. node e2e/evidence/build-handbook.mjs ``` To re-take a single figure after a fix, skip the setup project — it clears the whole previous run: ```bash npx playwright test -c e2e/evidence/evidence.config.ts \ --project=handbook --no-deps --grep "Executive Dashboard" ``` ### Why it runs on its own port and its own database The capture starts its own `php artisan serve` on **8001** with `DB_DATABASE=rusukh_demo`, and proves which database it is on — by searching for an `RSK-DEMO-` order — before taking a single picture. The everyday `rusukh` database is the wrong thing to photograph for a client: it carries 286 gate rows with **zero** evidence behind them, so its gate board renders green and refuses the first garment anyone tries to advance (§7). A handbook illustrated from it would be showing the reader a board that does not mean what it appears to mean. `reuseExistingServer` is deliberately `false` there. Reusing whatever happened to be listening on 8001 would silently photograph an unknown database, which is the one failure the whole arrangement exists to prevent. ### What the capture refuses to do - **Photograph an error.** Any response at or above HTTP 400 fails the capture. The single exception is the deliberate 403 in the handbook's permissions section, where the refusal *is* the evidence. - **Photograph a redirect.** If the landing path is not the requested one, it fails rather than quietly documenting a different screen (§3). - **Ship a missing image.** The generator drops any figure whose file is not on disk and says how many it dropped, so a renamed or deleted capture cannot leave a convincing stale photograph behind. ### Analytics figures The dashboards read *"awaiting data"* on a freshly seeded database, because the KPI engine writes snapshots on a nightly schedule (`RUNBOOK.md` §11) and a database created this afternoon has not had one. That is correct behaviour, not a fault — the application shows an em dash rather than a fabricated zero. To photograph populated dashboards, dispatch the jobs first: ```bash DB_DATABASE=rusukh_demo php artisan tinker --execute=" dispatch_sync(new \App\Modules\Analytics\Application\Jobs\CaptureKpiSnapshotsJob); dispatch_sync(new \App\Modules\Analytics\Application\Jobs\CaptureManagementWidgetSnapshotsJob); dispatch_sync(new \App\Modules\Analytics\Application\Jobs\RefreshAnalyticsMaterializedViewsJob);" ``` These are **queued jobs, not artisan commands** — `analytics:capture-kpi-snapshots` is the schedule's label for one of them, and typing it as a command gets you *"There are no commands defined in the analytics namespace."* On a real deployment the scheduler entry and a running queue worker are what fill these dashboards; without a worker they stay empty forever and nothing reports an error. --- ## 9. Last verified All numbers below were taken on **PHP 8.2.28** — the version `composer.json` pins and this documentation specifies. The suite also passes on 8.3.30 (`require.php` is `^8.1`), but quote the version you deploy. | Suite | Result | Date | |---|---|---| | Pest — everything except concurrency | **2716 passed, 0 failed** · 13,372 assertions · 458s | 2026-09-05 | | Pest — concurrency group, run alone | **28 passed, 0 failed** · 156 assertions · 210s | 2026-09-05 | | Architecture | **267 passed, 0 failed** — included in the first line | 2026-09-05 | | Clean install (§7) | `migrate:fresh --seed` from an empty database, no error | 2026-09-05 | | Seeded gate integrity (§7) | 151 gates / 569 evidence rows · **151 of 151 re-evaluate as passing** | 2026-09-05 | | Playwright — 4 projects at 375 / 768 / 1440 | **262 passed, 0 failed** · `--workers=1` · 20.6m | 2026-09-05 | | Evidence capture (§8) | **129 passed, 0 failed** · 118 handbook figures | 2026-09-05 | | `./vendor/bin/pint --test` | passed | 2026-09-05 | Current known-outstanding items are tracked in [`../RESUME.md`](../RESUME.md) §4 and in [`../PROGRESS.md`](../PROGRESS.md)'s BACKLOG sections. Nothing there is hidden: if a check has not been done, it says so.