dc/sas/mocks/test/parity/README.md

SAS parity harness

Records what a deployed SAS backend answers, then replays it against the JS mock estate and reports the differences. The mock services are a second implementation; this is the only check that compares behaviour rather than source.

The privacy rule

A golden is a projection, never a copy. project.js keeps a fixed set of members per scenario, drops the whole envelope (SYSHOSTNAME, SYSSITE, SYSUSERID, AUTOEXEC, MF_GETUSER, job and process ids, timings, _PROGRAM), replaces the identifiers the comparison needs (DC###### and DC_JSLIB to <LIBREF>, absolute paths, URLs, emails, staging ids, ISO timestamps, secrets by key name), and then refuses to publish if:

• any identifier class survives (FQDN, IPv4, absolute path, libref, email, eight-digit id), or
• any literal the envelope just discarded still appears anywhere in the projection.

That second check is what catches a hostname echoed inside an error message or a home path inside a DCPATH value, which no regex would find. record.js runs both before writing; parity-golden.test.js runs the same scan over every committed golden, so a hand-edited file cannot slip through.

Record only from the organisation's synthetic estate - one seeded by makedata, plus createdemodata when a scenario needs the fixtures. A customer site would put their rows into a file that looks anonymous and is not: row values are data, not identifiers, and no scan can tell a customer's rows from a demo's. The recorder records estate: synthetic in _meta as a claim a reviewer can check.

Use

# record every scenario from the server target (needs sas/.env.server)
node sas/mocks/test/parity/record.js --target server
# a private estate with its own certificate
node sas/mocks/test/parity/record.js --target server --ca /path/to/estate.pem
# throwaway host only - warns, and _meta records tls: unverified
node sas/mocks/test/parity/record.js --target server --insecure

# record one
node sas/mocks/test/parity/record.js --target server --scenario default

# replay against a mock estate and diff (no server, no credentials)
node sas/mocks/test/parity/check.js
node sas/mocks/test/parity/check.js --advisory      # report, exit 0

TLS is verified by default: the response becomes the reference the mock is measured against, and the tool runs with estate credentials. A target that is not https:// is refused outright - the request carries a bearer token and would send it in the clear. --insecure is opt-in, says so on stderr, and is recorded in _meta.tls, so a golden recorded over unverified transport can never be mistaken for one that was not.

Raw responses land in .captures/ (gitignored) so a refusal can be diagnosed without hitting the estate again. Only the projection is committed, under golden/<service>/<scenario>.json.

Scenarios

scenarios.json names, per scenario: the service, the URL parameters, the members to keep, and why the scenario exists. Everything not named in members is dropped by the projection, so a member added upstream is excluded until someone names it - the default is to publish less.

Refreshing the goldens

The build pipeline cannot reach a SAS estate, so re-recording is an ad-hoc job run by an agent or a maintainer holding the estate credentials - not a scheduled CI job. That splits the harness in two:

• The gate is offline and stays in CI. parity-golden.test.js scans the committed goldens, and parity-check.test.js replays every scenario through parity/replay.js against a mock estate built by the test harness - no server, no credentials. Both run in node --test sas/mocks/test/*.test.js, so a mock change that alters a payload fails the pull request. A scenario with no golden fails rather than skipping.
• Differences are declared, not ignored. A scenario's accepted list in scenarios.json names the JSON path of each difference that is expected (the estate's own admin group, its timezone, SAS's space padding of a blank character value) with the reason. Anything else fails, and so does an accepted entry that has stopped differing - the list shrinks to the real residue instead of hiding a fix.
• The refresh needs the estate. Where sas/.env.server exists: record.js --target server, then check.js, then the suite; commit the changed projections. A non-empty diff names the member that moved, so the refresh is a reviewed change rather than a silent overwrite - and when only SAS moved (a version bump, an upstream service change), the diff in the pull request is the whole report.

Fixture divergences that are deliberate

createdemodata owns two differences from a SAS estate, both driven by existing specs rather than by the product:

• MPE_X_TEST carries no validation rules on the mock, although mpe_makedata.sas seeds SAS's demo rules for it (a HARDREGEX on SOME_CHAR among them). The excel fixtures upload arbitrary values into that table and expect them to reach the approval screen; editor.cy.ts already says "MPE_X_TEST stays clean to match the excel fixtures". The validation demos live on TESTDATA.MPE_X_NEW.
• MPE_X_TEST's SOME_CHAR/SOME_DATE/SOME_DATETIME/SOME_TIME carry LABELs that differ from their names, which viewer-labels.cy.ts asserts, while a SAS estate has no labels on that table.

Both are asserted in test/createdemodata.test.js, so a change to either side is a failing test rather than a broken Cypress run. A future scenario covering editors/getdata on MPE_X_TEST would show them as accepted differences; today the startup scenario does not touch that table.

Recording is always against the synthetic estate, and that is enforced rather than asserted: record.js refuses any target that is not declared "synthetic": true in sasjsconfig.json. An agent refreshing goldens against a customer site would put their rows into a file that looks anonymous and is not, and no scan can tell the two apart afterwards.

Status

Phase 0: one scenario (public/startupservice), the projection, the publish gate, and the replay gate in CI. The corpus, the fault-path scenarios and the refresh runbook are next; see the parity section of the data-controller skill.