The captures wrote values like OMSOBJ:SASLibrary\A59LNVZG.B500000K inside a
template literal, where \A is an escape, so the separator never reached the
client: it served OMSOBJ:SASLibraryA59LNVZG.B500000K. The same happened to
D:\MyDemo\Sankey and BUILTIN\Administrators. Converting the service to webout()
faithfully preserved the corrupted value - and the metadetails ASSOCURI, the one
field fixed by hand at conversion time, was left as the only member that kept its
separator, which is the inconsistency the review named.
Seventeen values corrected across five services:
lineage/fetchtablelineage info[0].LIBURI 1
lineage/getmetacols metacols[].COLURI 11
lineage/getmetatables PATH_SCHEMA, TABLEURI 2
public/refreshlibinfo libinfo[0].OWNERS 1
usernav/usermembersbyrole sasgroups[].URIMEM,
sasmembers[].URIMEM 2
Verified by deploying both trees on one server and comparing the parsed
responses: those seventeen fields are the only differences, and every other
member of all thirteen converted services is unchanged (metadetails' two
ASSOCURI, already carrying its separator, still differ from the capture for the
same reason).
Anything that splits a URI or a Windows path on the separator, or displays
OWNERS or PATH_SCHEMA, now sees the value the SAS metadata server returns.
3.14.0 ships @sasjs/lint 5.0.0, so the studio reports OS commands, and it is the
release the estate runs. It still declares @sasjs/utils ^4.7.0, so the envelope
formats from utils 4.8 (SYSCC as text, DATE7, TIME5, E8601DT26.6) are not yet
guaranteed by the floor - a fresh install resolves 4.8.0, a cached tree need not.
The comment says so, and the pin moves again when the floor lands.
Verified on a local 3.14.0 install (clean, resolving utils 4.8.0): the mock
estate's startupservice returns SYSDATE 04NOV26, SYSTIME 12:44, SYSCC "0" (text),
END_DTTM 2026-11-04T12:44:14.954000 (no Z). With that estate seeded, and against
a fresh app location, licensing.cy.ts passes 10/10 and metanav.cy.ts 1/1 on
Chromium.
The other nine services that answer with a captured response also assigned it to
_webout, envelope and all. They now emit their members through the runtime's
webout():
lineage/fetchtablelineage info, finalfinal, flatdata, idlookup
lineage/getmetacols metacols 11 rows
lineage/getmetatables metatables 1 row
usernav/usermembers users 2 rows
usernav/usermembersbygroup sasmembers 2 rows
usernav/usergroupsbymember groups 1 row
usernav/usermembersbyrole sasgroups 1 row, sasmembers 1 row
usernav/userroles roles 46 rows
public/refreshlibinfo libinfo 1 row
Each payload was taken from the response the old service serves, then both
trees were deployed on one server (/Public/app/dc and /Public/app/dcnew) and
compared as parsed JSON with the envelope keys excluded: all nine are equal row
for row, including the escaped values (the OMS URIs keep their backslash, and
refreshlibinfo keeps the quoted paths).
The envelope is the runtime's now, so SYSCC is numeric 0 rather than the string
"0", SYSDATE/SYSTIME are today's, SYSSITE is reported (these captures carried a
lower-cased syssite, which the client never read), and the SAS-only
_METAUSER/_METAPERSON are gone.
lineage/fetchlineage is deliberately left alone: its captured body is not valid
JSON today (bare quotes in the graphviz labels, raw newlines inside the STRING
values, and a lost backslash), so there is no payload to preserve and the
intended text needs a decision rather than a rewrite. Tracked separately.
The four metanav services assigned a captured 2022 response to _webout, envelope
and all, instead of using the runtime's webout() like every other service. The
members are unchanged:
metarepos outrepos 2 rows
metaobjects objects 1 row
metatypes types 71 rows
metadetails attributes 12 rows
associations 2 rows
Both trees were deployed on one server, at /Public/app/dc and /Public/app/dcnew,
and the responses compared as parsed JSON with the envelope keys excluded:
outrepos, objects and types are equal row for row.
metadetails changes in one place. The capture wrote the OMS URI as
OMSOBJ:Property\A59..., but \A is an escape in a JavaScript template literal,
so the backslash was consumed and the client received OMSOBJ:PropertyA59... -
a URI that cannot resolve. The member now carries the backslash the SAS
metadata server returns.
The envelope comes from the runtime now, which also drops three artefacts of
the recording: SYSCC "4" on metarepos (an error code from the captured day,
which the client reads as a SAS error), SYSDATE/SYSTIME frozen at 26SEP22, and
the SAS-only _METAUSER/_METAPERSON.
The mock services call webout(), as the server's runtime does. The browser mock
runtime that @sasjs/cli ships is what provides it there, and the 6.5.0 this repo
was pinned to does not: its runtime-core.js has no reference to the webout
prelude, and the build writes no webout-prelude.json for it to load. Every
service that calls webout() therefore throws "ReferenceError: webout is not
defined" in a published mock site while the build itself reports success.
6.9.0 is the first release that ships the prelude; this pins 6.10.0, the current
release. The transitive @sasjs/adapter and @sasjs/utils entries move with it.
The guard that keeps a runtime without webout()'s options argument from serving a
thinner response tested webout.length >= 4. Function.length stops counting at the
first default or rest parameter, so a runtime that does carry the option but
declares it `options = {}` or gathers the tail with `...args` reports three, and
every service that calls the guard refuses to start on a working runtime.
It now asserts the outcome instead: the member the option produces is looked for
in the response body the program is writing. `$sasdata`, `$viewdata`, `$vals` are
the members the client reads for column formats and labels, and their presence is
the thing that actually has to hold.
Where it runs matters too, and the first attempt got that wrong: called after
webout('CLOSE') the throw leaves a complete body behind, and the server serves it
- the caller sees a normal response with SYSCC 0 and no metadata, which is exactly
the silent failure this exists to prevent. It is called before the envelope is
closed, so an unhonoured option fails the run.
@sasjs/utils 4.7 makes webout() emit SYSSITE from a configured site id,
defaulting to 123, where %webout takes &syssite. Until 3.13.0 the prelude
wrote an empty string, and dc's client routes an expired licence key and a
wrong-organisation key only when startup_site_id is truthy - so on the mock
estate both pages were unreachable and licensing.cy.ts failed three cases.
Three values reached `run:` blocks through `${{ }}`, so the runner pasted them
into the script before the shell ever saw it - which makes the value part of the
code rather than an argument to it.
- `BASE="${{ github.base_ref }}"` (build.yaml, lighthouse.yaml) - a ref name in
the script.
- `echo ${{ secrets.SHEET_PWD }} | gpg --passphrase-fd 0` (all four workflows) -
a passphrase in the script. A quote in it ends the quoting, and a command
substitution in it runs.
- two `replace-in-files --replacement=...${{ secrets.CYPRESS_* }}...` lines
(build.yaml, release.yaml) - the same, inside a replacement string.
Each now arrives as a step `env:` value and is expanded as `"$VAR"`, so the shell
only ever sees data. Checked the replacement lines side by side with the old form:
identical for an ordinary value, and the new form keeps a value containing a space
intact where the old one word-splits it, and survives a value containing a single
quote where the old one is a shell syntax error.
The passphrase pipe also moves from `echo` to `printf '%s\n'`, so the byte the
decryption sees does not depend on how the shell's `echo` treats backslashes.
Review finding: when every body is refused by carriesClause, or the response
carries no FILTER_HASH, capture.hash stays empty - and storedClauseFor('')
then spends its 60 retries before reporting "no clause for <var> under",
which names neither the refused body nor the missing hash.
The hash is now asserted non-empty before the read, with the case's own
message, so a broken delivery fails where it happened.
The call count stays where it was - after the data load - because moving it
ahead of the read would check it before the app has issued the second POST
that the count exists to catch.
The duplicate check waited a fixed 250ms after reading the stored row, which is
wrong in both directions: on a slow runner the window can elapse before the first
response callback has incremented the counter, and a second POST landing after it
passes with two deliveries.
The app's sequence for an apply is validatefilter, then the data load for the
filtered route, so the load is the completion signal: the count is asserted after
it, and nothing depends on how fast the machine is. A delivery issued later than
that is a second apply, not a duplicate of this one.
Also corrects the ordering paragraph in cypress-specs.sh: it still said specs are
appended, in a file that now places va-ddc-host ahead of metanav. va-ddc-host does
not depend on the estate being unlicensed either - it passed 3/3 immediately after
metanav planted a key in the full CI run - so the bullet says what is true and
metanav stays the only spec with a fixed place.
Review finding: the matcher only accepted the variable name at the start of the
body or after a comma or newline, so a delivery wrapped any other way was not
counted - and the case then failed at the call-count assertion, reading as
"the app never called validatefilter". Reproduced against the previous
implementation: a urlencoded form value, a form value that starts with the
name, a JSON object keyed on the column and a JSON string value holding the
row all matched nothing.
The body is now expanded into the candidate strings it can carry - the raw
text, the percent-decoded text, each form field's value, and every string leaf
of a JSON body - and the name has to be a whole CSV field of one of them.
Field equality rather than delimiters means a quoted literal, a longer name,
an SQL fragment or another variable's name is still refused, and there is no
regular expression over user text to get wrong.
Also restores the ordering note in cypress-specs.sh: va-ddc-host asserts
nothing about licensing, so it runs before metanav, the one spec that plants a
licence key.
The pin moved to 3.12.3 with the webout() change but the comment above it still
named 3.12.2 as the current release - true when it was written, wrong under the
pin it now sits on. The comment also only carried the /ddc-host reason, which
does not explain this bump.
3.12.3 is the first server release declaring @sasjs/utils ^4.6.0, and that is
what gives webout() the options argument getdata, viewdata and getcolvals pass as
{ showmeta }. Say so where the constant is, so the next person to move it knows
the metadata members depend on it.
The services in this branch pass { showmeta } to webout(), which is
@sasjs/utils 4.6 and up. sasjs/server 3.12.3 is the first release whose
package.json asks for ^4.6.0 (a clean install resolves 4.6.1), so the pin
moves from 3.12.2 to 3.12.3 in every workflow that starts the mock estate.
Verified against the published package rather than the release notes:
`npm view @sasjs/server@3.12.3 dependencies.@sasjs/utils` answers ^4.6.0, and
`npm install @sasjs/server@3.12.3` in an empty directory installs
@sasjs/utils 4.6.1, whose webout prelude carries showmeta.
Review of the release diff: `npm install -g @sasjs/cli` and
`replace-in-files-cli` were unpinned, so CI ran whatever release was current
and a bad one would reach the pipeline unreviewed - next to a SASjs server
that is pinned to an exact version.
The CLI now comes from the version sas/package.json pins, which is the shape
docs.yaml and release-assets.yaml already use, so there is one source for it.
replace-in-files-cli, pm2 and wait-on are pinned to the versions in use
(4.0.0, 7.1.2, 9.5.1).
Both node expressions are checked from the directories they run in:
`require('./sas/package.json')` at the root and `require('../../package.json')`
in sas/mocks/sasjs both answer 6.5.0.
The three services that pass { showmeta } - getdata, viewdata, getcolvals - rely
on webout()'s fourth argument, which arrived in @sasjs/utils 4.6.0. @sasjs/server
3.12 declares ^4.4.0, so an install can resolve 4.5.x, and there the argument is
dropped silently: the response still looks complete but $sasdata, $viewdata and
$vals are absent, so the client's format and label paths stop being exercised
while the mock estate reports success.
requireWeboutOptions() checks that the generated webout() takes an options
argument (arity, because the option names live outside the callable function) and
throws naming the member, the cause and both fixes. A 4.6 runtime is unchanged.
The matcher tested the variable as a substring of the whole request body, so a
RAW_VALUE, an SQL fragment or another variable whose name contains it was counted
as another delivery of this clause and failed the count. It now requires the name
between its CSV delimiters, which is the filterquery VARIABLE_NM field; a quoted
literal cannot match, because its delimiters are quotes.
cy.wait('@validatefilter') used the 5s default, where the polling loop it replaced
tolerated 15s, and the message only arrives once the framed page has rendered.
It now waits for longerCommandTimeout, as the rest of this spec does.
The capture assertion also carries the tail of the host page's log in its message:
the failure reads "expected -1 to be above -1" otherwise, and the log is what says
whether the app never exposed its buffer or the frame never loaded.
Two review findings:
- The load handler re-posted pendingPayload on every load event of the frame,
so an in-app navigation or a self-reload posted a filter that belonged to
the mount which had already delivered it, and logged it as that mount's.
The handler now takes the payload (clearing pendingPayload and
pendingTiming) before posting, so a later load has nothing to send.
- The README illustrated report.json with one parameter while the committed
file carries three, and the prose below reads as though one parameter were
the shape. The example block is gone: the file is the example, and the keys
are documented as a table with the parameter rules kept underneath.
The last eleven services still built the response envelope through
dcMockUtils' webOutOpen/webOutObj/webOutClose. They now use the webout() the
SASjs Server emits into every JS program, so dcMockUtils no longer emulates
%webout at all - the helper existed only for these callers and is deleted
with them.
That needed the options object added to webout() in @sasjs/utils 4.6
(showmeta, maxobs), which is what a clean install of @sasjs/server@3.12.2
resolves to (^4.4.0 -> 4.6.1, checked). A server whose install still holds
4.4/4.5 refuses the option by name rather than ignoring it, so the failure is
loud rather than a response that is quietly missing its column metadata.
getdata, viewdata and getcolvals send their metadata as
`webout('OBJ', name, rows, { showmeta })`, and every other call keeps its
member name and rows unchanged.
One call was positional all along: startupservice's xlmaps is an array of
arrays, which the old helper serialised as-is and webout() refuses for OBJ.
It is now webout('ARR', 'xlmaps', xlmaps) - caught by the comparison run
below, not by the rewrite.
Two gaps in the guard that decides which validatefilter response belongs to the
case.
A body Cypress parses into an object stringified to "[object Object]", so no
call matched and the case failed on the count with nothing naming the body shape.
The body is now serialised the way the response already was - a string is used as
it is, anything else is JSON.stringify'd.
The count was asserted immediately after the first response, which is before a
second POST from the same apply could reach its own callback, so a duplicate read
as one call. The count is now asserted after the stored row has been read back,
with a settle that covers a second POST still in flight when the read returned.
Two review findings on host.js:
- postToFrame() used '*' as targetOrigin, so the filter clause went to
whatever document occupied the frame. The frame is the app on report.server,
so that origin is the only one that should receive it; the origin is now
resolved once from report.server (new URL(...).origin) and a report.server
that is not a URL logs a visible error instead of widening delivery. The
page also says so when it is not on the frame's origin, because the capture
probe cannot read across origins and that would otherwise look like a
harness failure.
- startEarlyReplay() accepted any __vaLastMessage once it had posted, and on
a remount to the same URL the old document is still on screen carrying the
buffer the previous mount captured. A capture now only counts once the
document has been replaced (mountDocument), both in the interval and in the
load-event check. Keying to the document rather than waiting for a load
event keeps the first post where the early window is - the interval starts
exactly as before.
Every push to a pull request queued another full run - lint, licence check, the
mocked estate and the twenty-six minute suite - and nothing cancelled the run for
the head it replaced. With two runners shared by every pull request in the repo,
those stale runs are picked oldest-first and delay the head that can actually be
merged: runs 3002, 3003 and 3005 were queued for heads that had already been
superseded while the current one waited behind them.
The group is the pull request, so a newer push cancels the run for the older
head. Pushes that are not pull requests are not affected - the release workflow
answers to its own group.
Three review findings on the harness:
- The usage line advertised `--pass`, which puts the secret in argv - readable
from `ps` by every account on the host, and kept in the shell history. The
flag is gone: SASJS_PASS stays the documented route, and `--pass-stdin`
reads one line from stdin for scripts that cannot use the environment.
- deploy() reported success on the deploy route's 2xx alone. That means the
tree was accepted, not that the app is served: a truncated `type: 'file'`
member still serves a 200 page, and a stale asset from a previous deploy is
a 200 by definition - both of which the README already lists as the sharp
edges of that route. The page and all three assets are now fetched back and
compared with the bytes that were sent, failing with the size and hash of
both copies; a 401 is reported as deployed-but-gated, since on a server-mode
estate that is the gate working. `--verify-only` runs just that half.
- Two harnesses stand in for a VA report (this page and
client/cypress/e2e/va-ddc-host.cy.ts) and nothing said which does what. The
README now names both roles and points at the contract of record - the
message shape and __vaLastMessage in va-messaging.service.ts, and the
pre-bootstrap capture rule in va-early.js - including why the probe differs
between them (Cypress proxies the framed document).
The intercept kept one hash from any validatefilter response in the case, so a
second call - another table, a revalidate on load - would have replaced it and
the rows read back would belong to a different clause. The response is now only
taken when the request body carries this case's variable, which is what tells one
clause from another, and a case that sees more than one call for that variable
fails on the count rather than silently keeping the last.
A hash that cannot be read is captured and rethrown in the command chain: thrown
inside the route handler, Cypress reports it as an uncaught error and the case
fails later as a bare "no clause under" timeout instead of naming the response.
The spec located its evidence as "the last stored row for this variable", which
cannot tell this run's row from one an earlier case or run left behind - so it
deleted the DC_JSLIB filter tables before each case instead, and that bought a
scheduling contract: it had to run after every other spec on the shared mock
estate, and it left the estate falsified for anything that ran after it.
The mock validatefilter already returns the FILTER_HASH it computed for the
clause, so the rows are now selected by the hash this case's own response names.
Nothing is cleared, and no spec has to run before or after it. A store that
already holds this clause holds the same rows; a clause that changed hashes
differently, so a regression cannot be satisfied by a stale row.
va-ddc-host moves back in front of metanav, which is the one spec with a real
ordering constraint (its licence key must not change another spec).
The SASjs Server emits a webout() block into every JS program it runs
(@sasjs/utils >= 4.4, server >= 3.12.0), mirroring %ms_webout in
@sasjs/core: OPEN, ARR, OBJ, CLOSE. Ten of the mock services are two-year
functional implementations of their SAS counterparts, and they were
hand-building that same envelope through dcMockUtils' webOutOpen/Obj/Close.
Move those ten onto webout(), so the response envelope is built by the
runtime that owns it - real SYSUSERID, SYSPROCESSNAME, host name, a numeric
SYSCC and a content-type header - instead of constants in a mock. The
services that send showmeta metadata stay on the dcMockUtils helpers,
which webout() has no argument for yet.
The old 3.6.2 server pin in release, lighthouse and sites predates the
block, so those three move to 3.12.2 with it.
The interval tested its deadline only after the contentWindow guard, so a frame
that never became postable (detached, mid-navigation, or a load that never
fires) returned on every tick and the timer ran for the life of the page.
The capture check and the deadline now both run before the guard, so the replay
always stops. Capture keeps precedence, so a message caught on the last tick is
still reported as a capture, and it only counts once something has been posted
in this replay - a buffer left by the previous load of the same frame cannot be
read as one that belongs to this mount.
attempts keeps counting posts rather than ticks, because that is what the lines
quote ("after N post(s)"); a tick skipped for want of a frame sent nothing. The
zero case is now stated separately, since "0 post(s)" otherwise reads as a
broken probe rather than a frame that never appeared.
allan
dc