test: add turn command state differential harness
This commit is contained in:
@@ -0,0 +1,156 @@
|
||||
# Turn command state differential testing
|
||||
|
||||
## Scope
|
||||
|
||||
This harness compares observable state changes produced by:
|
||||
|
||||
- general reserved commands;
|
||||
- nation reserved commands;
|
||||
- live `che_출병`, including persisted battle aftermath.
|
||||
|
||||
It complements the battle simulator differential suite. The simulator suite
|
||||
compares phase order, RNG and battle numbers. This harness compares the command
|
||||
boundary, logs, queues, diplomacy and final database state.
|
||||
|
||||
## Execution flow
|
||||
|
||||
```text
|
||||
canonical case
|
||||
├─ ref ng_compare command runner
|
||||
│ ├─ clone root/HWE MariaDB
|
||||
│ ├─ execute legacy GeneralCommand or NationCommand
|
||||
│ └─ before/after snapshot + command RNG trace
|
||||
└─ core2026 execution boundary
|
||||
├─ snapshot selected PostgreSQL rows
|
||||
├─ execute reserved turn or daemon command
|
||||
└─ before/after snapshot + command RNG trace
|
||||
|
||||
ref delta ─┐
|
||||
├─ exact semantic path comparison
|
||||
core delta ┘
|
||||
```
|
||||
|
||||
The comparison uses entity IDs rather than physical row order. Numeric leaves
|
||||
are compared as `after - before`, so a fixture may use different harmless
|
||||
starting balances while still requiring the same effect. Insertions, deletions
|
||||
and non-numeric changes retain explicit before/after values.
|
||||
|
||||
Logs and messages are sequence-sensitive and are therefore paired by observed
|
||||
order rather than database primary key. Their physical IDs can be ignored
|
||||
without losing ordering checks. Legacy `*ID` command argument keys are
|
||||
normalized to the core `*Id` spelling before command identity comparison.
|
||||
|
||||
## Components
|
||||
|
||||
- Ref `ng_compare`
|
||||
- `hwe/compare/turn_state_snapshot.php`: read-only canonical projection.
|
||||
- `hwe/compare/turn_command_trace.php`: guarded CLI action runner.
|
||||
- Workspace reference stack
|
||||
- `scripts/run-turn-differential-case.sh`: clones both MariaDB databases,
|
||||
injects temporary DB configuration, runs one case and deletes only
|
||||
validated `sammo_td_*` databases.
|
||||
- Core integration tools
|
||||
- `canonical.ts`: shared snapshot and trace contracts.
|
||||
- `databaseSnapshot.ts`: PostgreSQL projection.
|
||||
- `trace.ts`: before/execute/after capture boundary.
|
||||
- `compare.ts`: exact snapshot and delta comparison.
|
||||
- `turnTraceFiles.integration.test.ts`: compares saved ref/core traces.
|
||||
|
||||
The ref runner refuses mutation unless `TURN_DIFFERENTIAL_ENABLED=1` is present.
|
||||
The wrapper injects it only into the disposable tool container. Direct
|
||||
execution against the reference database is not a supported workflow.
|
||||
|
||||
Two committed cases exercise the guarded runner end to end:
|
||||
|
||||
- `nation-declaration.json`: chief nation command, directed diplomacy
|
||||
`state/term` changes and national messages.
|
||||
- `live-sortie-conquest.json`: real `che_출병 -> processWar`, city capture,
|
||||
defeated-general neutralization and last-city nation collapse.
|
||||
|
||||
Each case may include a structured `setup` object for `world`, `nations`,
|
||||
`cities`, `generals` and `diplomacy`. The runner accepts only explicitly mapped
|
||||
fields; it does not accept SQL. Setup and command mutation happen only inside
|
||||
the cloned `sammo_td_*` databases.
|
||||
|
||||
## Case request
|
||||
|
||||
```json
|
||||
{
|
||||
"kind": "general",
|
||||
"actorGeneralId": 101,
|
||||
"action": "che_출병",
|
||||
"args": { "destCityID": 12 },
|
||||
"observe": {
|
||||
"generalIds": [101, 201],
|
||||
"cityIds": [11, 12],
|
||||
"nationIds": [1, 2],
|
||||
"logAfterId": 0,
|
||||
"messageAfterId": 0
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Legacy argument spelling is retained at the ref boundary (`destCityID`,
|
||||
`destNationID`). The core execution request uses the core spelling
|
||||
(`destCityId`, `destNationId`), while the trace's semantic entity IDs remain
|
||||
the same.
|
||||
|
||||
Run a ref case:
|
||||
|
||||
```sh
|
||||
cd docker_compose_files/reference
|
||||
./scripts/run-turn-differential-case.sh \
|
||||
fixtures/turn-differential/nation-declaration.json \
|
||||
> /tmp/ref-trace.json
|
||||
```
|
||||
|
||||
Capture the core side by wrapping the real reserved-turn or daemon execution
|
||||
with `captureCoreDatabaseTurnTrace()`. Save the returned JSON, then compare:
|
||||
|
||||
```sh
|
||||
TURN_REFERENCE_TRACE=/tmp/ref-trace.json \
|
||||
TURN_CORE_TRACE=/tmp/core-trace.json \
|
||||
pnpm --filter @sammo-ts/integration-tests test:integration \
|
||||
turnTraceFiles.integration.test.ts
|
||||
```
|
||||
|
||||
## Canonical coverage
|
||||
|
||||
Snapshots currently include:
|
||||
|
||||
- world year/month, tick term, last turn time and unification state;
|
||||
- selected general numeric state, location, nation, troop, equipment metadata,
|
||||
last turn and lifecycle counters;
|
||||
- selected city ownership and all domestic/defence values;
|
||||
- selected nation resources, type, capital, technology, cached power/counts;
|
||||
- directed diplomacy state, term and death flag;
|
||||
- general and nation reserved queues;
|
||||
- action/history/battle logs;
|
||||
- legacy messages and log/message watermarks.
|
||||
|
||||
Core2026 has no physical legacy `message` table, so its message projection is
|
||||
empty. Message-equivalent effects must currently be compared through core log
|
||||
effects or a command-specific projection. The saved-trace comparison explicitly
|
||||
ignores the `messages` subtree for this reason; this is a documented schema
|
||||
boundary, not a claim that message delivery is equal.
|
||||
|
||||
For live `che_출병`, use both suites:
|
||||
|
||||
1. `battleDifferential.test.ts` for internal war RNG/event/numeric parity.
|
||||
2. Turn command traces for route choice RNG, costs, general/city/nation changes,
|
||||
queues, logs, conquest and nation deletion.
|
||||
|
||||
## Test trust boundary
|
||||
|
||||
Comparator unit tests prove path identity, order independence, numeric delta
|
||||
handling and deletion detection. Adapter integration tests prove that both
|
||||
actual databases can produce the canonical form. They do not by themselves
|
||||
claim that all 55 general and 38 nation commands match.
|
||||
|
||||
Compatibility is established per case only when:
|
||||
|
||||
1. both engines executed the requested action rather than a fallback;
|
||||
2. RNG operations and results match;
|
||||
3. the semantic delta comparison is empty;
|
||||
4. live sortie also passes the battle trace comparison;
|
||||
5. any ignored path is documented in the case evidence.
|
||||
@@ -65,6 +65,7 @@ environment-dependent check.
|
||||
| ----------------------------------------------- | ----------- | ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `test/crewTypeExecution.test.ts` | kept | compatibility | Crew action router ordering and legacy NPC arm-type weighting. |
|
||||
| `test/databaseCommandQueue.integration.test.ts` | kept | integration | PostgreSQL single claim across consumers, persisted result, and expired-versus-active lease recovery. Explicitly skipped without a DB URL. |
|
||||
| `test/turnDaemonLease.integration.test.ts` | kept | integration | PostgreSQL profile lease exclusivity, expiry takeover with epoch increment, stale-owner fencing rollback, and clean release handoff. Explicitly skipped without a DB URL. |
|
||||
| `test/inputEventAtomicity.test.ts` | kept | contract | Dirty-state retention, mutation/commit/respond ordering, and pause/no-ack behavior on handler or commit failure. |
|
||||
| `test/nationCollapseOnConquest.test.ts` | kept | smoke | Last-city conquest removes the nation and neutralizes its general. It is a focused engine scenario, not full battle parity. |
|
||||
| `test/nationTurnCompatibility.test.ts` | kept | compatibility | Legacy-backed 12-turn research accumulation and completion, diplomacy-message emission, and exact monthly strategy/diplomacy limit decay through the engine harness. |
|
||||
@@ -117,12 +118,17 @@ environment-dependent check.
|
||||
|
||||
### `tools/integration-tests`
|
||||
|
||||
| Test source | Disposition | Layer | What it establishes |
|
||||
| --------------------------------- | ----------- | ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `test/auctionFlow.test.ts` | kept | integration | Real API/PostgreSQL/Redis/daemon resource and unique-auction bidding, extension, finalization, payout, and ownership constraints. |
|
||||
| `test/battleDifferential.test.ts` | kept | compatibility | PHP reference metadata and attacker/defender comparisons for all 145 scenario items, plus event order, RNG results, phase values, multi-defender, siege, and no-defender branches. The two exhaustive item cases currently fail on 13 attacker and 2 defender items; this is active compatibility evidence, not a green regression. |
|
||||
| `test/initialization.test.ts` | kept | integration | Real gateway/game APIs, database/Redis reset, scenario seed, users, joins, turns, and founding state. |
|
||||
| `test/orchestrator.e2e.test.ts` | kept | integration | Real PM2 game API/daemon startup, command serving, SSE completion, and persisted world update. |
|
||||
| Test source | Disposition | Layer | What it establishes |
|
||||
| --------------------------------------------------- | ----------- | --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `test/auctionFlow.test.ts` | kept | integration | Real API/PostgreSQL/Redis/daemon resource and unique-auction bidding, extension, finalization, payout, and ownership constraints. |
|
||||
| `test/battleDifferential.test.ts` | kept | compatibility | PHP reference metadata and attacker/defender comparisons for all 145 scenario items, plus event order, RNG results, phase values, multi-defender, siege, and no-defender branches. The two exhaustive item cases currently fail on 13 attacker and 2 defender items; this is active compatibility evidence, not a green regression. |
|
||||
| `test/initialization.test.ts` | kept | integration | Real gateway/game APIs, database/Redis reset, scenario seed, users, joins, turns, and founding state. |
|
||||
| `test/orchestrator.e2e.test.ts` | kept | integration | Real PM2 game API/daemon startup, command serving, SSE completion, and persisted world update. |
|
||||
| `test/turnSnapshotComparator.test.ts` | kept | contract | Canonical entity ordering, exact changed paths, numeric before/after delta equivalence, and live-conquest nation-deletion mismatch detection. |
|
||||
| `test/turnSnapshotCoreDatabase.integration.test.ts` | kept | integration | Real PostgreSQL projection plus before/execute/after capture around a transaction boundary. Explicitly skipped without a DB URL. |
|
||||
| `test/turnSnapshotReference.integration.test.ts` | kept | integration | Read-only canonical projection from the isolated PHP/MariaDB reference service. Explicitly enabled with `TURN_DIFFERENTIAL_REFERENCE=1`. |
|
||||
| `test/turnCommandReference.integration.test.ts` | kept | compatibility harness | Disposable cloned-MariaDB execution of real legacy nation declaration and live sortie through conquest and nation collapse. Explicitly enabled with `TURN_DIFFERENTIAL_REFERENCE=1`. |
|
||||
| `test/turnTraceFiles.integration.test.ts` | kept | compatibility harness | Saved ref/core command identity, RNG sequence and semantic delta comparison. It is skipped until both independently executed trace files are supplied. |
|
||||
|
||||
## Shared test support
|
||||
|
||||
|
||||
Reference in New Issue
Block a user