Useful signals.
Costly mistakes.

07 / A FINDING NEEDS A DENOMINATOR

One alarming replay cannot estimate a detector.

A detector flags a case for review. A label records what the test author knows about that case. Here positive means a synthetic client repeatedly sent invalid actions. A declared legal automation client is negative; its automation metadata is not passed to the classifier. These labels do not establish human intent. A true positive is a labelled-positive case that is flagged; a false positive is a labelled-negative case that is flagged. Missed positives are false negatives. Correctly unflagged negatives are true negatives.

A zero denominator gives no estimate: report null, not an invented zero. Count each case once. Repeated observations from one case are not additional independent people.

Use the exploration split to compare threshold 3 at windows 8 and 20, then compare a low threshold with a stricter one. Record which setting you expect to behave differently on evaluation and why. Freeze your settings before revealing evaluation results. The corpus includes an honest-shaped case and a valid automation case with identical observed signals. A classifier using only those signals must give them the same prediction; changing a threshold cannot create missing information.

07 / CONSTRUCT AND TEST

Separate prediction from scoring.

npm run core:prepare -- evaluation

In the printed workspace, implement evaluation/policy.mjs from its guide. classify receives signals and settings. evaluate records each prediction before using the label to update its counts. Returning the label as the prediction makes the measurement meaningless.

npm run core:check -- --assistance "none"

From the reference project, use the prepared evaluation workspace to freeze your completed self-contained policy and settings before running it: The report identifies the corpus, split, settings and source bytes. Exploration, frozen plans and one-use evaluation runs are saved under the prepared session. Later plans require --revision "reason". The next check links their paths and hashes in a new submission; earlier attempts remain unchanged. Explain the limits of the chosen threshold.

npm run evaluation:explore -- /path/to/evaluation-workspace 3 8 500
npm run evaluation:explore -- /path/to/evaluation-workspace 3 20 500
npm run evaluation:plan -- /path/to/evaluation-workspace 3 20 500
npm run evaluation:run -- /path/to/evaluation-workspace <printed-plan-id>

A useful transfer question: the next deployment has far fewer positive cases than this corpus. Which of these measurements can you directly carry over, and which need new population evidence? Do not substitute the synthetic labels for observed player intent.

09 / DATA HAS A PURPOSE AND A COST

Keep enough to answer a named question.

Retention is the rule for how long data stays available. The denial-event export in this lesson keeps only tick, actor identifier, reason and squared distance. That is enough for this detector's window calculation; it is not enough to replay the whole game. The richer checkpoint/window format from module 6 has a different purpose.

Actor identifiers still link behavior inside a run. Removing a capability token or a device field does not make the remaining events anonymous. An age limit and a record-count limit answer different questions: how old can an event be, and how much can the store hold?

These elapsed times were measured on a named local runtime, with a stated workload, warmup and sample count. Whole-operation timing includes validation and cloning. Policy-plus-encoding timing isolates less work; neither estimates production capacity. With fewer than 20 samples, the tail shown is the maximum, not p95. Raw samples remain inspectable. They are not a promise for another machine. Serialized byte counts are reproducible for these records; process-memory measurements also include runtime allocation and garbage collection.

09 / CONSTRUCT, MEASURE, DELETE

Make the limits executable.

npm run core:prepare -- retention

Implement retention/policy.mjs in the printed workspace. Filter by the stated age boundary, keep the newest permitted count and project to exactly the allowed fields. Preserve input order and leave the input unchanged.

npm run core:check -- --assistance "none"

From the reference project, use your completed self-contained policy to compare two settings: Export a denial-event file, inspect its fields, then delete that export using the supplied command and verify that the path is gone. This deletes that file only; copied files, earlier reports and original records are separate artifacts.

npm run retention:measure -- /path/to/policy.mjs artifacts/measure-01.json 4095:512:64 4095:128:16 --warmup 5 --samples 20
npm run retention:export -- /path/to/policy.mjs artifacts/denials-01.json artifacts/receipt-01.json 256 128
npm run retention:delete -- artifacts/receipt-01.json artifacts/deleted-01.json

Every output must be a fresh path; use a new numbered path for each revision. The export source is a named synthetic workload held in memory. These commands run your JavaScript locally; they are not a security sandbox.

Keep settings, measured counts, byte lengths and runtime metadata with your explanation. State the investigation this export supports and one question that its reduced fields or shorter history can no longer answer. Do not put real player identifiers or external credentials in this teaching corpus.