The runtime package went with the import, but its types package stayed - in
dependencies, so a type-only package was installed and shipped. Nothing
references the types now that the call site uses the platform TextDecoder.
Review found two more loadChildren in the eager form, in
home-routing.module.ts: excel-maps (XLMapModule) and multi-load
(MultiDatasetModule). They were inside the home chunk, so opening the app's
default screen /home/tables pulled both.
Both now return dynamic imports, so each gets its own chunk (47 kB apiece) and
the tables screen no longer pays for them. The initial total is unchanged -
home was already lazy - which is why the earlier measurement did not show
them.
app.routing.ts already declared loadChildren, but as the eager form - a
function returning the module class - so Angular bundled every screen into
the initial chunk and loaded it before the first paint. Only exceljs, which
uses a dynamic import, was ever split.
Each route now returns a dynamic import, so webpack emits a chunk per screen:
before after
main 8.96 MB 3.50 MB
initial total 10.29 MB 4.83 MB
initial transfer 2.29 MB 849.30 kB
The modules declare no providers, so a lazy module cannot introduce a second
instance of a service the shell already holds - the failure this change would
otherwise risk.
polyfills.ts imported the grid so that every route parsed it before the app
bootstrapped - the login screen included. Every module that uses Handsontable
imports it itself (editor, viewer, viewboxes, the shared utils), so the
polyfill import was redundant as well as misplaced.
It does not move the grid out of the initial bundle on its own: the viewer is
an eager route, so Handsontable still ships in main until the routes are split
(504). What it fixes is the pretence that the grid is a polyfill.
text-encoding was imported for one call in the licence service:
new encoding.TextDecoder().decode(value). TextDecoder has been native in
every browser this app targets for years, so the polyfill bought nothing and
cost 701 kB in the initial bundle (encoding-indexes.js 607 kB + encoding.js
94 kB).
Measured on the production build: main 8.96 MB -> 8.41 MB raw, initial total
10.29 MB -> 9.74 MB. The dependency goes with it.
Two comments outran their evidence.
The getcolvals docstring said no fixture reached a decorating format, while the
same commit adds a dollar8.2 one that does - the padding and decimals are
pinned, the symbol is not. COMMA and SIZEKMG are the formats with no fixture,
and it says that now.
The _PROGRAM exemption was justified by the path being uniform across
deployments. It is not: it carries the target's appLoc. The reason that holds is
that the path is the route of the request that was just made and is already in
that request's URL, so publishing it adds nothing. Both the projection comment
and the README say that instead.
Three findings, all of them about the same field.
`sasNumericFormatted` only parsed `BEST<w>.<d>`, so a DOLLAR8.2 column fell back
to a fixed width of 12 while `$vals` declared $8./8 - the value contradicted the
metadata a caller sizes the column from. Any `NAME<w>.<d>` now yields the width
and decimals, and the declared width is the fallback; a decorating format is
padded and rounded but still arrives without its symbol, which the comment says
rather than an implementation it does not have.
The `formattedWidth` ternary had a temporal arm that nothing could reach - every
temporal column is numeric, so the numeric arm answered it - and the docstring
justified a 200 fallback no call site asked for. The width is now computed once,
before the values are read, from the numeric arm or the character length, and the
fallback is the caller's.
The formatting itself had no test with a value in it: the MPE_AUDIT fixture is
empty, so only metadata was asserted. A mock table with rows and a formatted
numeric now pins the padding and the decimals, for best12.2 and for dollar8.2.
The comment and the README explained the exemption by what it buys - a MAC
compared rather than masked - which was not the reason. A service route is not
sensitive and is the same on every deployment, so it is dropped with the rest of
the envelope but not denied; that the abort's MAC therefore arrives as it is is
a consequence, and the diff comparing it is incidental.
The manifest lists a report under its route while that report's own finalUrl
can have lost the fragment, so matching the two on the URL printed the same
run twice - once under #/home/tables and once under an empty label. Dedupe
by file instead, and label a navigation that ended on the app root as '/'.
Four findings from the review of this branch, all of them comparisons against a
live estate response.
`public/getcolvals` hardcoded best12 for every numeric column. The estate
formats `formatted` with put() at the column's own format - mf_getVarFormat
with force=1 - so a BEST32. column reports $32./32 and pads the value to 32,
and the metadata's unformatted copy keeps the column's label rather than the
variable name. A widthless format now gets its own default width: the estate
answers $16./16 for a DATETIME. column, where the fallback was 200.
The same service also searched every libref, so a library the estate does not
have still answered for a table that exists elsewhere. The named libref is
resolved first and refused as "Invalid libds" - the message %mpe_validatecol
gives, before the member is looked for; a missing member in a known library
stays "Target table not found".
`public/getcols` sorted the column list with localeCompare, the ordering this
branch already fixed in viewtables. SAS compares the character value, where
'_' sorts after every letter, so A_B comes after AB; the list now sorts that
way, with a fixture whose names differ only in underscore placement.
An empty getcolvals result answered empty strings where the estate sends one
row of SAS missings - a blank for the character and null for the number.
Recorded from the estate: a formatted number (MPE_AUDIT.OLDVAL_NUM), a
widthless temporal (MPE_DATASTATUS_CATS.CREATED) and a wrong library that names
a real table. The header-file test now polls for the line instead of waiting a
fixed 20ms.
The `<SERVICE>` placeholder this branch added to publish the getddl abort also
projected the mock's MAC, so a mock reporting a different program would have
compared equal to the estate. The service path is the app's own route, identical
on every deployment, so it is not an identifier to protect: `_PROGRAM` is now
dropped from the response but exempt from the deny list, and the MAC is compared
as it arrives. The golden is re-recorded with the literal path.
The FQDN class grows a lookbehind so a path ending in `.sas` is not read as a
hostname - the abort's MAC ends that way.
The README says which envelope member is dropped but not denied, and its runbook
step 1 now names the raw capture that holds the control library: the committed
startup golden masks `DCLIB` to `<LIBREF>`, so the step could not be followed
from it.
The audit's numbers were only visible in the job log and the artifact. One
comment per pull request, rewritten in place on every push (the body carries
a marker the script looks for, so a re-run replaces the table instead of
adding a second one), with a row per route and the lowest accessibility
score against the gate.
.gitea/scripts/lighthouse-report.mjs reads the reports, and posts through the
API with the workflow's token; a comment that cannot be posted logs and
leaves the job's verdict alone, because the audit's own step is the gate.
The job needs pull-requests: write (and issues: write, which is where Gitea
keeps pull request comments) for that and nothing else.
The collect comment still argued numberOfRuns:1 from "accessibility 1 - a
fixed score that does not vary with load", which is no longer the threshold
and was only ever true of an empty estate. It now states the trade against
the scores the audit actually measures (0.87-0.96, issue 497).
The assert comment records why the floor is global rather than per route:
lhci strips the fragment, so the seventeen routes are one URL to it and the
assertion aggregates their median - assertMatrix with matchingUrlPattern
cannot express per-page allowances here.
Also drops include-hidden-files from the artifact upload: that is a v4.4
input and v3 warns about it as unexpected on every run.
`public/getddl` for a dataset that is not there and `public/getcolvals` for a
table that is not there - the two aborts the mock used to answer with a body.
Both carry one accepted difference, `SYSCC`, for the reason the existing abort
scenarios already record: the estate's wrapper adds it to an aborted response,
and a JS program's envelope is finalised at CLOSE.
The README gains the refresh runbook the status section kept promising: deploy
first, how to find the estate's control library, what each gate line means,
and how to read the two refusals.
It also names what cannot be recorded yet and why. The "not authorised" abort
needs a non-admin login, because %mpe_accesscheck lets the admin group through
and the synthetic estate's user is in it. The getddl body needs index metadata
the mock's table files do not carry. And getcols' libref tolerance is a
behaviour change that needs a survey of its callers first.
Two requests that the estate refuses were answered as if they had worked.
`public/getddl` streamed its fixed sample body for any request at all - a
dataset it has never heard of, an unassigned library - and the message it did
throw for a missing `ds` carried two dots where SAS writes one. It now resolves
the member through the mock's own table files and aborts with SAS's message,
"Dataset <libref>.<ds> was not found", reported against the program because
that is the `mac` the service passes to `%mp_abort`.
`public/getcolvals` answered an empty dropdown for a table that is not there.
The estate aborts from `%mp_derivedvars` before any values are selected, so the
mock does too, with that macro's own MAC.
The projection needed one rule to publish either golden: `%mp_abort(mac=
&_program)` puts the service's own path in the payload, and `_PROGRAM` is an
envelope literal the deny list refuses. A service path is now `<SERVICE>`,
which is the app's own route rather than anything about the estate.
Both rules are asserted in test/fault-paths.test.js.
Recorded from the synthetic estate: the library browse (public/viewtables),
a dropdown's values for a character column and for a numeric one
(public/getcolvals), and an Excel map (editors/getxlmaps). The first four
scenarios recorded in the earlier pass now gate the download that already
matched - public/getrawdata had no differences left but still carried
"gated": false, so a regression there would have been reported and not failed.
public/getgroups is deliberately not recorded. Its answer is the SASjs
Server's own group list - all-users, public, and every group created on that
install - so a golden would be a copy of one estate's setup, and the
difference is in the rows rather than in a path that `accepted` can name. The
mock derives its groups from MPE_GROUPS, and the startup scenario already
accepts the admin group's name as estate state. The README now says so, and
names the two paths still to align: the security abort a table the caller may
not see produces, and getddl's hardcoded body.
The parity recorder found two payload differences that no test covered.
`public/viewtables` listed the mock's own file name for the format catalog -
MPE_X_CATALOG, a dataset the estate does not have - where SAS lists a catalog
as <name>-FC. A member with no variables is the catalog, so it is named that
way, and the list is now sorted in SAS byte order: '_' is 0x5F and sorts after
'L', so MPE_XLMAP_RULES precedes MPE_X_CATALOG-FC, which localeCompare got
backwards.
`public/getcolvals` reported the metadata of a column it had not looked at:
$32767. and a length of 32767 for the formatted value whatever the column, an
empty SASFORMAT for a column whose format is blank (SAS answers a space), and
a bare number where put(x,best.) right-justifies into best12. The metadata now
describes the two variables the service creates - $<len>. at the column's own
length, best12 for a number, the format's width for a temporal - and the value
is formatted to match.
Each rule is asserted in test/read-paths-payload.test.js so a regression names
the rule rather than a JSON path.
Seeding the estate made the audit measure the app with data in it for the
first time, and accessibility is 0.87-0.96 across the seventeen URLs rather
than 1.00. Those are real defects, tracked in 497; the gate cannot stay at 1
while they exist, and leaving it there would block every pull request for a
reason unrelated to what they change.
0.85 sits just below the observed minimum so a busy runner does not turn the
threshold into a coin flip, and goes back to 1 as 497 clears.
The server refuses to run with CORS=enable and no WHITELIST, which is what
made the job fail at startup. CORS is not needed here and was only ever
needed because the audit used to run against ng serve on :4200: the app and
the services are both served by this server on :5000, so every call is
same-origin. The Cypress job keeps its CORS setting, because there the
browser really is on another origin.
The audit's URLs are mostly data views - tables, view/data, the review
lists, excel-maps, multi-load - and the job deployed the services without
the database behind them. makedata builds what a real deployment builds,
createdemodata adds the Cypress fixtures (TESTDATA, the search and wide
viewers, the multi-load library), and both run here before lhci, exactly as
the Cypress job runs them. makedata's payload asks for OUTFMT=JSON, so a
failed seed fails the job instead of auditing an empty estate.
Two things carried over from lighthouse.yaml verbatim did not hold:
- The job ran cd sas/mocks, which has no sasjsconfig.json. The CLI then
searched for the project and npm ci installed whatever package.json the
prefix walk found (sas/, 159 packages), neither of which is what the step
is for. It now cds into sas/mocks/sasjs, as the Cypress job does, and
installs nothing: the pinned global CLI carries @sasjs/core.
- The built copy was patched to appLoc /proj/sasjs/genesis-mocks, but the
only target this job deploys - server-ci - publishes services at
/Public/app/dc, which is also what the production build already carries.
The audited app was therefore calling a path with no services behind it.
The patch step goes: the build's own appLoc is the deployed one, so the
two cannot drift, and the unused replace-in-files install goes with it.
Its docstring still described the trimmed comparison the previous commit
removed, which is exactly the kind of comment a later reader follows back into
the bug. It now states that cells are kept as they arrive, padding included,
that a leading or trailing blank is part of a SAS character value, and that only
the framing - the quotes and the line endings the state machine consumes - is
removed.
Rebased onto release/7.23 after #492 and #494 merged: the scenario list carried
two entries with the id mpe_tables (public/viewdata from #492, public/getcols
from this PR), which the merge made a conflict; both are kept, keyed by service
and id together, which is how the golden path and the checker already treat
them. 114 pass, 0 fail, eight scenarios ok.
Three findings from review.
- the abort envelope: no service depended on the CLOSE-time assignment. Members
are on the session's webout file as they are added, and the server serves that
file whether or not the program finished (api/src/controllers/session.ts
sessionOutput returns webout.txt, catching to ''), so a caller receives what
was flushed; every abort writes sasjsAbort first, so an abort is never an
empty body. The comment now says that, says that dropping CLOSE is estate
parity (a SAS soft abort stops before the envelope is finalised), and keeps
the note that the estate's wrapper is what adds SYSCC to its own aborts.
- parseCsvTable trimmed every cell, quoted content included, so the download
comparison could not see a leading or trailing blank - the very class of
difference the viewer work just established. Cells are kept as they arrive,
a quote opens a field only at its start, and a test pins all three cases.
Removing the trim changed no scenario: all seven still ok.
- provenance: parity/README.md now has a section saying where each _meta field
comes from. sas is the envelope's SYSVLONG, which only a SAS runtime emits, so
a golden carrying it cannot have come from the mock; a scenario that answers
with something other than a SAS JSON envelope (the getrawdata CSV) records
unknown, and that is why that one golden says so while its accepted
differences still come from a live response. estate is the target's declared
flag, which record.js refuses to do without; dc is the repository version at
record time and is named as the one field not read from the response.
Four findings from review.
- mpAbort's comment still said the runtime puts SYSCC on an aborted envelope,
which is what I believed before checking: a JS program stops before the
envelope is finalised, so an abort carries neither SYSCC nor SYSENCODING. The
comment now says that, and says that the SAS estate's wrapper is what adds
SYSCC to an abort - the difference the abort scenarios declare.
- runService removes the previous run's session directory when it creates the
next one: a directory is only read back through estate.weboutPath, which has
moved by then, and a test file that runs many services was leaving one behind
per call.
- brCheckPath: both callers are macro-bound (a DIRLIST parent and a WRITECHECK
target that is written into a filename statement), so rejecting % and & at
every call is right; the comment says so now instead of leaving a reader to
check.
- getdata's abort message is the recorded text - 'dataset X.Y does not
exist!!' with MAC mpestp_getdata.sas, from the estate's own response - so the
mock keeps it rather than inventing a diagnosis. The control-library mismatch
that could put a misleading name in that sentence is the hardcoded DC_JSLIB,
and #494 removes it.
Four findings from review.
- stagedata: the abort needs a registry that loaded and does not hold the row; a
drive where MPE_TABLES itself is absent is a different failure and the message
would be a lie.
- record.js checks for {{LIBREF}} before the first substitution rather than only
before the POST, so a scenario that names its table in the query string fails
with the same instruction instead of recording against an empty libref.
- makedata_actions: the macro triggers are asserted one at a time ('%', '&', and
a control character), so removing one from brCheckPath fails the test rather
than being masked by the other; the path-shape case is its own test.
- the getcols drop is parity, not a regression: the estate's getcols returns
NAME, LENGTH, VARNUM, LABEL, FMTNAME, FORMAT, TYPE, DDTYPE and nothing else,
and the picker's column-level security comes from getdata's dqrules/READONLY.
A fallback that needs CLS_RULE is a change to getcols.sas, not to its mock.
The abort shape change (no SYSCC/SYSENCODING on a soft abort) is estate-wide and
deliberate - it is what an aborted SAS response looks like.
Four findings from review.
- stagedata aborted on the target's data file existing, which is the state of a
first load into a newly registered table - staging there has to work. The gate
is the registration now (finalReg, already computed above), and the comment
says why: a table registered nowhere is the case a user hits when it is dropped
between loading the grid and saving.
- getcols rebuilt a character column's FORMAT as '$<length>.' whatever the column
declared, so a $CHAR10. or $UPCASE. column was described with a format it does
not have. mpGetcols' own FORMAT wins when there is one, the length-derived
default applies only when there is none, and an undeclared length no longer
produces '$0.'.
- getrawdata pushed the CSV header row for every requested type, so the EXCEL,
MARKDOWN and PDF bodies - which stream the same lines under their own content
type - began with a CSV header. Only type=CSV gets one now.
- the harness put SYSCC on the envelope at OPEN, claiming the runtime does that
for a program that never closes. It does not: sasjs/server's JS runtime
finalises the envelope at CLOSE (createJSProgram.ts -> weboutFinaliseStatement),
which is why a JS abort carries neither SYSCC nor SYSENCODING. The stub is back
to adding both at CLOSE, and the two abort scenarios declare the difference
they now show: the SAS estate's wrapper adds SYSCC to an abort, a JS program
stops before the envelope is finalised. The message and macro name the client
renders are unchanged.
check.js - seven scenarios, five at zero accepted differences.
Suite: 99 pass, 0 fail.
The runtime creates a session directory per execution - <root>/sessions/<id>
with webout.txt and stpsrv_header.txt beside it - so a program that only assigns
_webout cannot be served the previous request's body. The harness used one
directory for the whole estate, which is how a stagedata abort once answered for
a download.
runService now creates <root>/sessions/s<n> per call and points weboutPath and
_SASJS_WEBOUT_HEADERS at it, as the runtime does, and estate.weboutPath follows
the most recent run so a caller can still inspect a response afterwards. The
hooks test writes beside weboutPath and reads it back from the session directory
of that run rather than from a hardcoded s1.
Two more scenarios, and both needed recorder support that did not exist.
editors/stagedata on a table that is registered nowhere (0 accepted
differences). An edit submission sends *two* table inputs - the control table
and the edited rows as jsdata - so the recorder now posts several parts and the
harness declares several inputs, as the runtime does. The mock answered a
completed body where the service aborts with "Target table not found"; it aborts
that way now, and mpAbort no longer closes the body, because a SAS soft abort
does not reach the state that carries SYSENCODING.
public/getrawdata (0 accepted differences). A download answers CSV rather than
JSON, so scenarios can now declare format: csv and the body is parsed to
{header, rows} on both sides before the diff - SAS quotes and pads each cell to
the variable's width, so a text diff would fail on spacing that says nothing.
The mock was missing the header row, was exporting the registration's validity
columns (getrawdata.sas drops them: data staged(drop=&txfrom &txto)), and broke
any value containing a newline across two records because it did not quote it.
The harness now clears the response file before each run. Without that, a
service that only assigns _webout was handed the *previous* service's body -
which was hiding a stale-result bug in a DIRLIST test: the mock accepted
/tmp/%sysfunc(system(id)) as a parent, and the test passed because it was
reading the abort from the run before it. brCheckPath refuses '&' and '%' now,
which is the guard the test was written for.
check.js - seven scenarios, five of them at zero accepted differences:
ok public/startupservice/default (8 accepted difference(s))
ok editors/getdata/mpe_tables (1 accepted difference(s))
ok public/viewlibs/default (1 accepted difference(s))
ok public/getcols/mpe_tables (0 accepted difference(s))
ok editors/getdata/missing_table (0 accepted difference(s))
ok editors/stagedata/target_missing (0 accepted difference(s))
ok public/getrawdata/selectbox (0 accepted difference(s))
read-paths.test.js covers the two-input abort and the download's header/validity
columns/quoted newline. Suite: 99 pass, 0 fail.
Two more scenarios, both read-only, and both silent in the mock until now -
neither of them broke a Cypress spec, they just tested less than they claimed.
public/getcols (0 accepted differences). The filter panel builds its operator
set and value quoting from this list, and the mock answered the *getdata* column
shape: CLS_RULE, MEMLABEL, DESC, LONGDESC and COLTYPE, with FORMAT blank for
every column. The estate answers NAME, LENGTH, VARNUM, LABEL, FMTNAME, FORMAT,
TYPE, DDTYPE - VARNUM being the column's position in the table rather than in
the sorted list - so the mock now emits exactly that.
editors/getdata on a table that is registered nowhere (0 accepted differences).
getdata.sas aborts, and the client renders the message; the mock answered an
empty grid with an empty dsmeta instead, which is a different answer rather than
a quieter one. It now aborts with the same message and macro name, and - like
mp_abort - stops before CLOSE.
That last point needed the harness to be more faithful: the runtime puts SYSCC on
the envelope whether or not the program closed it, while SYSENCODING only appears
on a completed body. The stub adds SYSCC at OPEN now and SYSENCODING at CLOSE, so
an aborted response matches the estate's shape instead of being accepted as a
difference.
record.js also cuts the response at SASJS_LOGS_SEPARATOR before parsing: an
aborted call comes back with the log appended, the same way a _debug response
does, and the recorder used to fail on the trailing text.
check.js:
ok public/startupservice/default (8 accepted difference(s))
ok editors/getdata/mpe_tables (1 accepted difference(s))
ok public/viewlibs/default (1 accepted difference(s))
ok public/getcols/mpe_tables (0 accepted difference(s))
ok editors/getdata/missing_table (0 accepted difference(s))
read-paths.test.js pins both fixes; the suite is 97 pass, 0 fail.
The audit needs the production build, and an artifact belongs to the run that
produced it (needs: cannot cross workflows), so as a separate workflow it
compiled the frontend a second time - 92-104s measured - and then started an
ng serve that nothing audited: every URL in client/lighthouserc.js is on
:5000, which the SASjs server feeds from client/dist.
As a job in build.yaml it takes the change decision from the build job
(the same inputs gate both), downloads client-dist, and patches appLoc in the
built index.html instead of compiling a patched copy of the source. The
second compile, the dev server and the source generator in this job are gone;
what remains is node_modules for lhci, the pinned CLI, the SASjs server and
the mock estate deploy.
The check name moves from 'Lighthouse Checks / lighthouse' to
'Build / lighthouse'; there are no required contexts to update. The
changed-path matcher already covers everything this job reads, so its only
change is a comment and the fixture that used to assert lighthouse.yaml was
not an input.
Three findings from review.
- the trailing-blank strip now matches only blanks. SAS pads with blanks, so a
value that ends in a tab is not padding and is not silently altered while the
same character mid-string survives.
- the `typeof val === 'number'` arm was unreachable: both branches above return
text (viewCell and formatSasValue), so the ternary is gone and a comment says
why a string is all that arrives.
- provenance: the docstrings cited "the recorded response" as the authority, and
the golden's `_meta.estate` says synthetic, which invites exactly the doubt you
raised. They now name the file and the field that answers it - `_meta.sas` is
9.4M9, read from the live envelope's SYSVLONG, which only a SAS runtime emits
and which the mock cannot produce - and point at the Provenance section added
to parity/README.md in #493.
Three findings from review.
- getrawdata no longer parses settings.js itself. The reader moved into
services/dcSettings.js, which dcMockUtils.loadDcSettings calls and which a
service that cannot eval the whole utility file (its own formatSasValue and
SAS_EPOCH_MS would collide) can load on its own. One file, one parser: the
greedy object match now parses JSON where it can and falls back to a
quote-agnostic pattern, and the ENOENT-only catch is in that one place.
- a behavioural test moves the control library and its data and asserts a
getrawdata export still comes from it. This was the least-covered path in the
change - static tests cannot follow a reader that does not load the utilities.
- viewlibs' comment still said a `|| 'DC_JSLIB'` fallback mattered; it lives in
loadDcSettings (which defaults dcLibref to DC_JSLIB), so the comment says that
and the dead `||` stays gone.
Three findings from review.
- viewdata trimmed both ends of a character cell. A leading blank is part of a
SAS character value and the viewer renders it; only the trailing blanks SAS
pads to are noise, so the cell is trimmed at the end and an all-blank cell is
still SAS missing. viewdata-cells.test.js fails on the previous version.
- viewDate floored a stored day count while the ISO branch rounded, so the same
instant could land on different days depending on how the row carried it. Both
round now.
- the two label members stay different. The finding was that cols[].LABEL uses
the data dictionary business label while $viewdata.vars[.].label uses the
physical one; the recorded estate response does exactly that - cols DSN is
"Dataset Name to be edited", $viewdata.vars.DSN.label is "DSN" - so unifying
them would break parity. The comment says so, and a test asserts the split
against the recorded answer so the next reader does not tidy it away.
Five findings from review.
- getrawdata's settings reader was a second parser that only matched a
double-quoted key and value, so a hand-edited settings.js would leave this one
service on DC_JSLIB while the estate used the configured library. The object is
parsed as JSON first (what makedata writes) and a quote-agnostic pattern is the
fallback, so both a written and an edited file are read the same way.
- the bare catch treated every read failure as "makedata has not run": an
EACCES or an encoding failure now surfaces instead of silently defaulting.
- `dcInit(...).dcLibref || 'DC_JSLIB'` was unreachable - loadDcSettings already
defaults dcLibref to DC_JSLIB - so the `||` is gone from all fourteen services;
it would have masked a changed settings shape as a silent default.
- the "Load shared DC mock utilities" comment had ended up below the eval it
describes in getdynamiccolvals, getcols and getcolvals; it describes the eval,
so it sits above it, and the note about which tables live in the control
library no longer names DC_JSLIB.
- merge order with #493 (getcols, getrawdata, stagedata): whichever lands second
is rebased; the getrawdata hunks are separate from that PR's CSV changes.
The settings guard now also accepts a `let` declaration and a service that keeps
the literal as a default while reading settings.js (getrawdata does).
Three findings from review, and tests that hold them.
- a DATE below the year 1000 printed three digits ('999'), so DATE9.'s
four-digit year is now zero-padded
- a stored ISO text reached a TIME column through the datetime branch, giving
seconds since 1960 rather than seconds since midnight; a time's day is
discarded, and a text without a zone is read as UTC rather than in the host's
zone (the control box sits east of UTC, which shifted every value by an hour)
- the datetime fraction is clamped at 99 as the time branch already was, so a
fraction that rounds up cannot print three digits
viewdata-temporal.test.js builds a table with DATE, TIME and DATETIME columns and
covers each renderer plus a stored ISO string; the year and the ISO cases fail on
the previous version of the service.
Three findings from review.
- validatefilter used fetchTable/colVal above the eval that defines them, so it
threw on every call - the conversion moved the eval down past the parse. The
eval and the dcInit read are back above the parse, with a comment saying why
they have to be. Reproduced and checked: the service answers now.
- a source-level test walks sas/mocks/sasjs/services/**/*.js and asserts the
invariant the copy-paste is supposed to keep: a service that loads the
utilities reads the settings after them, calls no utility before them, and
does not name the control library; makedata is exempt (it creates it) and a
hook is exempt (its body is eval'd in the calling service's scope). This is the
guard that would have caught the validatefilter slip, and it costs no estate.
- getrawdata reads dc_libref out of services/settings.js rather than naming
DC_JSLIB, so the documented CSV row-diff path works on a deployment with a
different DCPATH. It cannot eval dcMockUtils - its own formatSasValue and
SAS_EPOCH_MS would collide - so it reads the one value it needs; createdemodata
stays as it is, because it creates the library.
Two findings from review.
- viewTime was reached with the unrounded value (viewCell passed the number
straight through for TIME), so a stored 3661.9999 produced a fraction of 100
and printed "01:01:01.100". The value is rounded to two decimals on entry, as
the DATETIME branch already did, and the fraction is clamped as well so a
rounding edge cannot reach three digits.
- a temporal column with no format fell back on DATETIME19.3 whatever its
ddtype, so DATE and TIME columns were labelled as datetimes. The default
follows the ddtype now: DATE9., TIME8., DATETIME19.3.
viewdata's scenario is still at zero accepted differences, and the suite is 96
pass, 0 fail.
makedata writes services/settings.js - the JS twin of the settings program the
backend generates in %mpe_init - with dc_libref, dc_admin_group, dc_staging_area
and dcpath. dcInit() reads it. getdata and viewlibs used it; the other thirteen
services that load dcMockUtils still carried const dcLibref = 'DC_JSLIB' and
built their paths from it, so a deployment with a different DCPATH kept working
in two services and broke in the rest.
Each of those services now loads dcMockUtils first and takes its libref from
dcInit(driveRoot, appLoc).dcLibref, with 'DC_JSLIB' only as the default for a
drive makedata has not seeded. Services that computed paths from the constant
(stagedata's staging directory, getapprovals' data directory, gethistory's) have
the eval above that line, so the derived paths come from the settings too.
Two services do not load dcMockUtils at all - createdemodata and getrawdata
carry their own table and format helpers - so they keep naming the constant, and
say why in a comment: adding the eval just for the settings would collide with
their own helper names.
mock-settings.test.js gains a case that would fail without this: it renames the
control library, moves the data, rewrites the settings file, and asserts a row
written into MPE_SUBMIT comes back from approvers/getapprovals. With the
hardcoded service it is empty, which is the failure mode this removes.
The viewer was the third read path the harness could not see. Recorded from the
seeded estate it showed 364 differences; all of them were the mock's, and none
remain.
- cols: sorted by column name, and a column LABEL comes from the
MPE_DATADICTIONARY row for the lib.ds.column when there is one - viewdata.sas
augments mp_getcols the same way getdata does. The format name is upper case,
as SAS reports it.
- rows: a temporal value is rendered the way the estate renders it - two-digit
year and a two-decimal fraction ("01JAN60:00:00:00.00") - rather than by the
format label on the column, which reads DATETIME19.3. A character value is
trimmed and an all-blank one is a space, matching the missing-value rule the
data rows and the synthetic no-data row already share.
- $viewdata: the column's own metadata (num, 8 bytes, DATETIME19.3). The mock
reused the entry built for the edit grid, where the temporal columns have been
converted to character, so the viewer was told $200 char.
- sasparams.PK_FIELDS: the validity column ahead of the business key, which is
what viewdata answers ("TX_TO LIBREF DSN").
sas/mocks/test/viewdata.test.js pins the datetime text and the empty-table row.
getdata padded every blank character cell to a space, including one the row
holds as an explicit ''. That broke nine editor specs: the grid classifies a row
by comparing what it holds against the payload it was given (classifyRow), so a
payload whose empty strings had become spaces made every row read as modified on
load - the row header showed +/- and DC.ROW_STATUS answered 'M' where 'U' is
correct.
The rule is now the one SAS expresses: a cell the row does not carry is missing
and reaches the client as one space, and a value the row does carry is passed
through. MPE_TABLES rows omit their blank columns, so the recorded golden still
matches; the TESTDATA formula fixture stores '', and keeps it.
getdata-blanks.test.js pins both sides.
allan
dc