core2026/docs/architecture/turn-daemon-lifecycle.md

# Turn Daemon Lifecycle

This document defines the lifecycle of the turn daemon for the rewrite when a
single daemon instance is responsible for turn resolution and all triggers come
from the API server.

## Assumptions

- One daemon process per server+scenario profile.
- API server is the only ingress for user/admin requests.
- The daemon owns world-state mutations; the API server mutates reserved turns
  and messages inside the common API input-event transaction.

## Current Implementation Status

- The daemon loop handles scheduled runs plus PostgreSQL inbox control commands
  (run/pause/resume/shutdown/getStatus).
- API mutation requests (troopJoin, vacation, etc.) are drained and handled by the daemon loop between turns.
- `getStatus` and command results are stored durably on their input-event row.
- API server currently writes reserved turns and messages directly to the DB.

## Responsibilities

- Maintain authoritative in-memory state.
- Execute turn resolution when scheduled.
- Flush state changes and checkpoints to the DBMS.
- Expose status for ops and admin tooling.

## Trigger Sources

- Scheduled tick: next turn time is reached.
- API poke: user/admin request asks the daemon to run now.
- Admin control: pause/resume or manual catch-up.

## State Model

```
Idle -> Running -> Flushing -> Idle
   \-> Paused (admin)
   \-> Stopping (shutdown)
```

- `running` flag prevents re-entrant execution when multiple triggers arrive.
- `pendingReason` tracks why a run was requested (schedule, manual, poke).

## Main Loop Sketch

The daemon interleaves API request handling between scheduled turn executions.
API requests are drained until the next turn time is reached; once the turn
starts, incoming requests are queued and processed after the run.
The current implementation follows this interleaving model.

```ts
while (!stopping) {
    const signal = await waitForNextSignal(nextTurnTime, wakeSignal);
    if (signal.type === 'pause') {
        paused = true;
    }
    if (signal.type === 'resume') {
        paused = false;
    }
    if (paused || running) {
        continue;
    }

    await drainApiRequestsUntil(nextTurnTime);

    if (now() < nextTurnTime && signal.type !== 'run') {
        continue;
    }

    running = true;
    try {
        await runUntil(now(), budget);
        await flushChanges();
        publishTurnEvents();
    } finally {
        running = false;
    }
}
```

- `drainApiRequestsUntil()` processes queued user/admin requests and applies
  them to in-memory state until the next scheduled turn time is reached.
- During `running`, new API requests are enqueued and handled after the run.

## Run Budget and Checkpoints

Replace PHP `max_execution_time` with explicit limits to allow partial progress:

- `budgetMs`: max wall-clock time per run.
- `maxGenerals`: max generals processed per run.
- `catchUpCap`: max turns (or months) processed in one run.

When a limit is hit, persist a checkpoint so the next run resumes safely.
Minimum checkpoint data:

- `game_env.turntime` (last fully processed general turn time)
- optional cursor (last processed general id) if you batch within a turntime
- last known year/month if monthly transitions are mid-flight

## API Server Interaction

- API server inserts mutations and pokes into the PostgreSQL inbox.
- Read-only queries can read from DBMS or a read model.
- During `running`, incoming requests are queued and processed after the run.

## Status and Admin Controls

Expose a status endpoint to replace `proc.php` output:

- `running`, `paused`, `lastRunAt`, `lastDurationMs`
- `lastTurnTime`, `nextTurnTime`
- `pendingReason`, `queueDepth`

Admin controls should toggle `paused`, trigger manual run, and request catch-up.

### Suggested Status Endpoint (Draft)

`GET /admin/turn-daemon/status`

```json
{
    "profile": "che:default",
    "state": "idle",
    "running": false,
    "paused": false,
    "lastRunAt": "2026-01-01T12:00:00Z",
    "lastDurationMs": 842,
    "lastTurnTime": "2026-01-01 12:00:00",
    "nextTurnTime": "2026-01-01 12:10:00",
    "pendingReason": "schedule",
    "queueDepth": 3,
    "checkpoint": {
        "turnTime": "2026-01-01 12:00:00",
        "generalId": 1201,
        "year": 12,
        "month": 3
    }
}
```

### Suggested Admin Endpoints (Draft)

`POST /admin/turn-daemon/run`

```json
{
    "reason": "manual",
    "targetTime": "2026-01-01 12:10:00",
    "budgetMs": 2500,
    "catchUpCap": 3
}
```

`POST /admin/turn-daemon/pause`

```json
{
    "reason": "maintenance"
}
```

`POST /admin/turn-daemon/resume`

```json
{
    "reason": "maintenance complete"
}
```

### Daemon Control Contract (Draft)

API server to daemon control payloads (PostgreSQL inbox or in-process test
channel):

```ts
export type RunReason = 'schedule' | 'manual' | 'poke';

export type DaemonCommand =
    | { type: 'run'; reason: RunReason; targetTime?: string; budget?: TurnRunBudget }
    | { type: 'pause'; reason?: string }
    | { type: 'resume'; reason?: string }
    | { type: 'getStatus'; requestId: string };

export type DaemonEvent =
    | { type: 'status'; requestId?: string; status: TurnDaemonStatus }
    | { type: 'runStarted'; at: string; reason: RunReason }
    | { type: 'runCompleted'; at: string; result: TurnRunResult }
    | { type: 'runFailed'; at: string; error: string };
```

## Recovery and Shutdown

- On startup, load `game_env.turntime` and catch up to `now`.
- Claim pending input events with `FOR UPDATE SKIP LOCKED`. A worker ID and
  expiry timestamp prevent another live consumer from claiming the same event;
  only expired processing leases return to `PENDING`.
- A handler exception or transaction failure pauses the daemon without
  acknowledging the event. Restart/recovery reloads world state before retry.
- On shutdown, stop accepting new triggers, finish the current run, flush, then
  exit cleanly.

## Input Event Commit Boundary

For engine-owned mutations the lifecycle opens one database-owned unit of work:

```text
claim input_event
  -> execute command against in-memory world
  -> specialized domain DB writes (auction/tournament)
  -> persist world/reserved turns/logs
  -> store command result and mark input_event SUCCEEDED
  -> commit
  -> acknowledge in-memory dirty sets
  -> respond
```

If any step before commit fails, PostgreSQL rolls back and dirty sets are not
cleared. Realtime publication happens after commit and cannot roll gameplay
state back.

## Testing with a Controlled Clock

- Turn daemon tests should use a controllable `Clock` implementation (예: `ManualClock`) to
  advance time deterministically without relying on wall-clock time.
- 기준 시간은 DB에 저장된 `game_env.turntime`을 우선으로 삼고, 테스트에서도 동일한 기준을
  사용해 스케줄 계산과 체크포인트 동작을 검증한다.

## TypeScript Sketch (Draft)

```ts
export type TurnDaemonState = 'idle' | 'running' | 'flushing' | 'paused' | 'stopping';

export interface TurnRunBudget {
    budgetMs: number;
    maxGenerals: number;
    catchUpCap: number;
}

export interface TurnCheckpoint {
    turnTime: string;
    generalId?: number;
    year: number;
    month: number;
}

export interface TurnRunResult {
    lastTurnTime: string;
    processedGenerals: number;
    processedTurns: number;
    durationMs: number;
    partial: boolean;
    checkpoint?: TurnCheckpoint;
}

export interface TurnDaemonStatus {
    state: TurnDaemonState;
    running: boolean;
    paused: boolean;
    lastRunAt?: string;
    lastDurationMs?: number;
    lastTurnTime?: string;
    nextTurnTime?: string;
    pendingReason?: RunReason;
    queueDepth: number;
    checkpoint?: TurnCheckpoint;
}

export interface TurnRunner {
    getStatus(): Promise<TurnDaemonStatus>;
    requestRun(reason: RunReason, targetTime?: string, budget?: TurnRunBudget): Promise<void>;
    pause(reason?: string): Promise<void>;
    resume(reason?: string): Promise<void>;
}
```