1b26cc57e35401190eb6a5d9c42aa4f872d3bf89
1898
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 dc
|
1b26cc57e3 |
docs(mocks): say what parseCsvTable does now that it keeps cells verbatim
Its docstring still described the trimmed comparison the previous commit removed, which is exactly the kind of comment a later reader follows back into the bug. It now states that cells are kept as they arrive, padding included, that a leading or trailing blank is part of a SAS character value, and that only the framing - the quotes and the line endings the state machine consumes - is removed. Rebased onto release/7.23 after #492 and #494 merged: the scenario list carried two entries with the id mpe_tables (public/viewdata from #492, public/getcols from this PR), which the merge made a conflict; both are kept, keyed by service and id together, which is how the golden path and the checker already treat them. 114 pass, 0 fail, eight scenarios ok. |
||
|
dc
|
ab2d9131df |
docs(mocks): say how a golden was obtained, and stop trimming CSV cells
Three findings from review. - the abort envelope: no service depended on the CLOSE-time assignment. Members are on the session's webout file as they are added, and the server serves that file whether or not the program finished (api/src/controllers/session.ts sessionOutput returns webout.txt, catching to ''), so a caller receives what was flushed; every abort writes sasjsAbort first, so an abort is never an empty body. The comment now says that, says that dropping CLOSE is estate parity (a SAS soft abort stops before the envelope is finalised), and keeps the note that the estate's wrapper is what adds SYSCC to its own aborts. - parseCsvTable trimmed every cell, quoted content included, so the download comparison could not see a leading or trailing blank - the very class of difference the viewer work just established. Cells are kept as they arrive, a quote opens a field only at its start, and a test pins all three cases. Removing the trim changed no scenario: all seven still ok. - provenance: parity/README.md now has a section saying where each _meta field comes from. sas is the envelope's SYSVLONG, which only a SAS runtime emits, so a golden carrying it cannot have come from the mock; a scenario that answers with something other than a SAS JSON envelope (the getrawdata CSV) records unknown, and that is why that one golden says so while its accepted differences still come from a live response. estate is the target's declared flag, which record.js refuses to do without; dc is the repository version at record time and is named as the one field not read from the response. |
||
|
dc
|
e8fb891883 |
docs(mocks): make the abort comments agree, and stop leaving session directories behind
Four findings from review. - mpAbort's comment still said the runtime puts SYSCC on an aborted envelope, which is what I believed before checking: a JS program stops before the envelope is finalised, so an abort carries neither SYSCC nor SYSENCODING. The comment now says that, and says that the SAS estate's wrapper is what adds SYSCC to an abort - the difference the abort scenarios declare. - runService removes the previous run's session directory when it creates the next one: a directory is only read back through estate.weboutPath, which has moved by then, and a test file that runs many services was leaving one behind per call. - brCheckPath: both callers are macro-bound (a DIRLIST parent and a WRITECHECK target that is written into a filename statement), so rejecting % and & at every call is right; the comment says so now instead of leaving a reader to check. - getdata's abort message is the recorded text - 'dataset X.Y does not exist!!' with MAC mpestp_getdata.sas, from the estate's own response - so the mock keeps it rather than inventing a diagnosis. The control-library mismatch that could put a misleading name in that sentence is the hardcoded DC_JSLIB, and #494 removes it. |
||
|
dc
|
d21152d4e2 |
fix(mocks): gate the abort on the registry, guard the placeholder, cover the path check
Four findings from review.
- stagedata: the abort needs a registry that loaded and does not hold the row; a
drive where MPE_TABLES itself is absent is a different failure and the message
would be a lie.
- record.js checks for {{LIBREF}} before the first substitution rather than only
before the POST, so a scenario that names its table in the query string fails
with the same instruction instead of recording against an empty libref.
- makedata_actions: the macro triggers are asserted one at a time ('%', '&', and
a control character), so removing one from brCheckPath fails the test rather
than being masked by the other; the path-shape case is its own test.
- the getcols drop is parity, not a regression: the estate's getcols returns
NAME, LENGTH, VARNUM, LABEL, FMTNAME, FORMAT, TYPE, DDTYPE and nothing else,
and the picker's column-level security comes from getdata's dqrules/READONLY.
A fallback that needs CLS_RULE is a change to getcols.sas, not to its mock.
The abort shape change (no SYSCC/SYSENCODING on a soft abort) is estate-wide and
deliberate - it is what an aborted SAS response looks like.
|
||
|
dc
|
d84331659d |
fix(mocks): gate the stagedata abort on registration, and stop claiming SYSCC on an abort
Four findings from review. - stagedata aborted on the target's data file existing, which is the state of a first load into a newly registered table - staging there has to work. The gate is the registration now (finalReg, already computed above), and the comment says why: a table registered nowhere is the case a user hits when it is dropped between loading the grid and saving. - getcols rebuilt a character column's FORMAT as '$<length>.' whatever the column declared, so a $CHAR10. or $UPCASE. column was described with a format it does not have. mpGetcols' own FORMAT wins when there is one, the length-derived default applies only when there is none, and an undeclared length no longer produces '$0.'. - getrawdata pushed the CSV header row for every requested type, so the EXCEL, MARKDOWN and PDF bodies - which stream the same lines under their own content type - began with a CSV header. Only type=CSV gets one now. - the harness put SYSCC on the envelope at OPEN, claiming the runtime does that for a program that never closes. It does not: sasjs/server's JS runtime finalises the envelope at CLOSE (createJSProgram.ts -> weboutFinaliseStatement), which is why a JS abort carries neither SYSCC nor SYSENCODING. The stub is back to adding both at CLOSE, and the two abort scenarios declare the difference they now show: the SAS estate's wrapper adds SYSCC to an abort, a JS program stops before the envelope is finalised. The message and macro name the client renders are unchanged. check.js - seven scenarios, five at zero accepted differences. Suite: 99 pass, 0 fail. |
||
|
dc
|
469e98b2e6 |
test(mocks): give every harness run its own session directory
The runtime creates a session directory per execution - <root>/sessions/<id> with webout.txt and stpsrv_header.txt beside it - so a program that only assigns _webout cannot be served the previous request's body. The harness used one directory for the whole estate, which is how a stagedata abort once answered for a download. runService now creates <root>/sessions/s<n> per call and points weboutPath and _SASJS_WEBOUT_HEADERS at it, as the runtime does, and estate.weboutPath follows the most recent run so a caller can still inspect a response afterwards. The hooks test writes beside weboutPath and reads it back from the session directory of that run rather than from a hardcoded s1. |
||
|
dc
|
bfadbf24fd |
feat(mocks): record the edit submission and the download, and stop serving stale bodies
Two more scenarios, and both needed recorder support that did not exist.
editors/stagedata on a table that is registered nowhere (0 accepted
differences). An edit submission sends *two* table inputs - the control table
and the edited rows as jsdata - so the recorder now posts several parts and the
harness declares several inputs, as the runtime does. The mock answered a
completed body where the service aborts with "Target table not found"; it aborts
that way now, and mpAbort no longer closes the body, because a SAS soft abort
does not reach the state that carries SYSENCODING.
public/getrawdata (0 accepted differences). A download answers CSV rather than
JSON, so scenarios can now declare format: csv and the body is parsed to
{header, rows} on both sides before the diff - SAS quotes and pads each cell to
the variable's width, so a text diff would fail on spacing that says nothing.
The mock was missing the header row, was exporting the registration's validity
columns (getrawdata.sas drops them: data staged(drop=&txfrom &txto)), and broke
any value containing a newline across two records because it did not quote it.
The harness now clears the response file before each run. Without that, a
service that only assigns _webout was handed the *previous* service's body -
which was hiding a stale-result bug in a DIRLIST test: the mock accepted
/tmp/%sysfunc(system(id)) as a parent, and the test passed because it was
reading the abort from the run before it. brCheckPath refuses '&' and '%' now,
which is the guard the test was written for.
check.js - seven scenarios, five of them at zero accepted differences:
ok public/startupservice/default (8 accepted difference(s))
ok editors/getdata/mpe_tables (1 accepted difference(s))
ok public/viewlibs/default (1 accepted difference(s))
ok public/getcols/mpe_tables (0 accepted difference(s))
ok editors/getdata/missing_table (0 accepted difference(s))
ok editors/stagedata/target_missing (0 accepted difference(s))
ok public/getrawdata/selectbox (0 accepted difference(s))
read-paths.test.js covers the two-input abort and the download's header/validity
columns/quoted newline. Suite: 99 pass, 0 fail.
|
||
|
dc
|
2903d1421c |
feat(mocks): record the column list and the abort path
Two more scenarios, both read-only, and both silent in the mock until now - neither of them broke a Cypress spec, they just tested less than they claimed. public/getcols (0 accepted differences). The filter panel builds its operator set and value quoting from this list, and the mock answered the *getdata* column shape: CLS_RULE, MEMLABEL, DESC, LONGDESC and COLTYPE, with FORMAT blank for every column. The estate answers NAME, LENGTH, VARNUM, LABEL, FMTNAME, FORMAT, TYPE, DDTYPE - VARNUM being the column's position in the table rather than in the sorted list - so the mock now emits exactly that. editors/getdata on a table that is registered nowhere (0 accepted differences). getdata.sas aborts, and the client renders the message; the mock answered an empty grid with an empty dsmeta instead, which is a different answer rather than a quieter one. It now aborts with the same message and macro name, and - like mp_abort - stops before CLOSE. That last point needed the harness to be more faithful: the runtime puts SYSCC on the envelope whether or not the program closed it, while SYSENCODING only appears on a completed body. The stub adds SYSCC at OPEN now and SYSENCODING at CLOSE, so an aborted response matches the estate's shape instead of being accepted as a difference. record.js also cuts the response at SASJS_LOGS_SEPARATOR before parsing: an aborted call comes back with the log appended, the same way a _debug response does, and the recorder used to fail on the trailing text. check.js: ok public/startupservice/default (8 accepted difference(s)) ok editors/getdata/mpe_tables (1 accepted difference(s)) ok public/viewlibs/default (1 accepted difference(s)) ok public/getcols/mpe_tables (0 accepted difference(s)) ok editors/getdata/missing_table (0 accepted difference(s)) read-paths.test.js pins both fixes; the suite is 97 pass, 0 fail. |
||
 allan
|
15eb7c1121 |
Merge pull request 'feat(mocks): record public/viewdata and bring the viewer payload to parity' (#492) from feat/mock-viewdata-parity into release/7.23
Reviewed-on: #492 |
||
|
allan
|
0948f1df91 |
Merge pull request 'refactor(mocks): read the control library from the settings file in every service' (#494) from feat/mock-settings-rollout into release/7.23
Reviewed-on: #494 |
||
|
dc
|
2144ee96b0 |
fix(mocks): strip blanks, not whitespace, and cite the golden's provenance
Three findings from review. - the trailing-blank strip now matches only blanks. SAS pads with blanks, so a value that ends in a tab is not padding and is not silently altered while the same character mid-string survives. - the `typeof val === 'number'` arm was unreachable: both branches above return text (viewCell and formatSasValue), so the ternary is gone and a comment says why a string is all that arrives. - provenance: the docstrings cited "the recorded response" as the authority, and the golden's `_meta.estate` says synthetic, which invites exactly the doubt you raised. They now name the file and the field that answers it - `_meta.sas` is 9.4M9, read from the live envelope's SYSVLONG, which only a SAS runtime emits and which the mock cannot produce - and point at the Provenance section added to parity/README.md in #493. |
||
|
dc
|
bcfd81207c |
refactor(mocks): one settings parser for the whole estate
Three findings from review. - getrawdata no longer parses settings.js itself. The reader moved into services/dcSettings.js, which dcMockUtils.loadDcSettings calls and which a service that cannot eval the whole utility file (its own formatSasValue and SAS_EPOCH_MS would collide) can load on its own. One file, one parser: the greedy object match now parses JSON where it can and falls back to a quote-agnostic pattern, and the ENOENT-only catch is in that one place. - a behavioural test moves the control library and its data and asserts a getrawdata export still comes from it. This was the least-covered path in the change - static tests cannot follow a reader that does not load the utilities. - viewlibs' comment still said a `|| 'DC_JSLIB'` fallback mattered; it lives in loadDcSettings (which defaults dcLibref to DC_JSLIB), so the comment says that and the dead `||` stays gone. |
||
|
dc
|
4b71cf5b03 |
fix(mocks): keep a leading blank, and round a date the way the text branch does
Three findings from review. - viewdata trimmed both ends of a character cell. A leading blank is part of a SAS character value and the viewer renders it; only the trailing blanks SAS pads to are noise, so the cell is trimmed at the end and an all-blank cell is still SAS missing. viewdata-cells.test.js fails on the previous version. - viewDate floored a stored day count while the ISO branch rounded, so the same instant could land on different days depending on how the row carried it. Both round now. - the two label members stay different. The finding was that cols[].LABEL uses the data dictionary business label while $viewdata.vars[.].label uses the physical one; the recorded estate response does exactly that - cols DSN is "Dataset Name to be edited", $viewdata.vars.DSN.label is "DSN" - so unifying them would break parity. The comment says so, and a test asserts the split against the recorded answer so the next reader does not tidy it away. |
||
|
dc
|
8fabd582f1 |
fix(mocks): read the settings object, not its spelling
Five findings from review. - getrawdata's settings reader was a second parser that only matched a double-quoted key and value, so a hand-edited settings.js would leave this one service on DC_JSLIB while the estate used the configured library. The object is parsed as JSON first (what makedata writes) and a quote-agnostic pattern is the fallback, so both a written and an edited file are read the same way. - the bare catch treated every read failure as "makedata has not run": an EACCES or an encoding failure now surfaces instead of silently defaulting. - `dcInit(...).dcLibref || 'DC_JSLIB'` was unreachable - loadDcSettings already defaults dcLibref to DC_JSLIB - so the `||` is gone from all fourteen services; it would have masked a changed settings shape as a silent default. - the "Load shared DC mock utilities" comment had ended up below the eval it describes in getdynamiccolvals, getcols and getcolvals; it describes the eval, so it sits above it, and the note about which tables live in the control library no longer names DC_JSLIB. - merge order with #493 (getcols, getrawdata, stagedata): whichever lands second is rebased; the getrawdata hunks are separate from that PR's CSV changes. The settings guard now also accepts a `let` declaration and a service that keeps the literal as a default while reading settings.js (getrawdata does). |
||
|
allan
|
3487db1dd2 |
Merge pull request 'fix(mocks): pad only a character cell the row does not carry' (#495) from fix/mock-blank-cell-padding into release/7.23
Reviewed-on: #495 |
||
|
dc
|
40550ca56b |
fix(mocks): pad a date's year, read a stored time as seconds since midnight
Three findings from review, and tests that hold them.
- a DATE below the year 1000 printed three digits ('999'), so DATE9.'s
four-digit year is now zero-padded
- a stored ISO text reached a TIME column through the datetime branch, giving
seconds since 1960 rather than seconds since midnight; a time's day is
discarded, and a text without a zone is read as UTC rather than in the host's
zone (the control box sits east of UTC, which shifted every value by an hour)
- the datetime fraction is clamped at 99 as the time branch already was, so a
fraction that rounds up cannot print three digits
viewdata-temporal.test.js builds a table with DATE, TIME and DATETIME columns and
covers each renderer plus a stored ISO string; the year and the ISO cases fail on
the previous version of the service.
|
||
|
dc
|
38be2382cd |
fix(mocks): load the utilities before using them, and read the settings in getrawdata
Three findings from review. - validatefilter used fetchTable/colVal above the eval that defines them, so it threw on every call - the conversion moved the eval down past the parse. The eval and the dcInit read are back above the parse, with a comment saying why they have to be. Reproduced and checked: the service answers now. - a source-level test walks sas/mocks/sasjs/services/**/*.js and asserts the invariant the copy-paste is supposed to keep: a service that loads the utilities reads the settings after them, calls no utility before them, and does not name the control library; makedata is exempt (it creates it) and a hook is exempt (its body is eval'd in the calling service's scope). This is the guard that would have caught the validatefilter slip, and it costs no estate. - getrawdata reads dc_libref out of services/settings.js rather than naming DC_JSLIB, so the documented CSV row-diff path works on a deployment with a different DCPATH. It cannot eval dcMockUtils - its own formatSasValue and SAS_EPOCH_MS would collide - so it reads the one value it needs; createdemodata stays as it is, because it creates the library. |
||
|
dc
|
4cfc5b1e3d |
fix(mocks): round a time before printing its fraction, and default a temporal format by type
Two findings from review. - viewTime was reached with the unrounded value (viewCell passed the number straight through for TIME), so a stored 3661.9999 produced a fraction of 100 and printed "01:01:01.100". The value is rounded to two decimals on entry, as the DATETIME branch already did, and the fraction is clamped as well so a rounding edge cannot reach three digits. - a temporal column with no format fell back on DATETIME19.3 whatever its ddtype, so DATE and TIME columns were labelled as datetimes. The default follows the ddtype now: DATE9., TIME8., DATETIME19.3. viewdata's scenario is still at zero accepted differences, and the suite is 96 pass, 0 fail. |
||
|
dc
|
f831c0eecc |
refactor(mocks): every service that loads dcMockUtils reads the control library from the settings file
makedata writes services/settings.js - the JS twin of the settings program the backend generates in %mpe_init - with dc_libref, dc_admin_group, dc_staging_area and dcpath. dcInit() reads it. getdata and viewlibs used it; the other thirteen services that load dcMockUtils still carried const dcLibref = 'DC_JSLIB' and built their paths from it, so a deployment with a different DCPATH kept working in two services and broke in the rest. Each of those services now loads dcMockUtils first and takes its libref from dcInit(driveRoot, appLoc).dcLibref, with 'DC_JSLIB' only as the default for a drive makedata has not seeded. Services that computed paths from the constant (stagedata's staging directory, getapprovals' data directory, gethistory's) have the eval above that line, so the derived paths come from the settings too. Two services do not load dcMockUtils at all - createdemodata and getrawdata carry their own table and format helpers - so they keep naming the constant, and say why in a comment: adding the eval just for the settings would collide with their own helper names. mock-settings.test.js gains a case that would fail without this: it renames the control library, moves the data, rewrites the settings file, and asserts a row written into MPE_SUBMIT comes back from approvers/getapprovals. With the hardcoded service it is empty, which is the failure mode this removes. |
||
|
dc
|
b6b6a3fefd |
feat(mocks): record public/viewdata and bring the viewer's payload to parity
The viewer was the third read path the harness could not see. Recorded from the
seeded estate it showed 364 differences; all of them were the mock's, and none
remain.
- cols: sorted by column name, and a column LABEL comes from the
MPE_DATADICTIONARY row for the lib.ds.column when there is one - viewdata.sas
augments mp_getcols the same way getdata does. The format name is upper case,
as SAS reports it.
- rows: a temporal value is rendered the way the estate renders it - two-digit
year and a two-decimal fraction ("01JAN60:00:00:00.00") - rather than by the
format label on the column, which reads DATETIME19.3. A character value is
trimmed and an all-blank one is a space, matching the missing-value rule the
data rows and the synthetic no-data row already share.
- $viewdata: the column's own metadata (num, 8 bytes, DATETIME19.3). The mock
reused the entry built for the edit grid, where the temporal columns have been
converted to character, so the viewer was told $200 char.
- sasparams.PK_FIELDS: the validity column ahead of the business key, which is
what viewdata answers ("TX_TO LIBREF DSN").
sas/mocks/test/viewdata.test.js pins the datetime text and the empty-table row.
|
||
|
dc
|
4c5b38810b |
fix(mocks): pad only a character cell the row does not carry
getdata padded every blank character cell to a space, including one the row holds as an explicit ''. That broke nine editor specs: the grid classifies a row by comparing what it holds against the payload it was given (classifyRow), so a payload whose empty strings had become spaces made every row read as modified on load - the row header showed +/- and DC.ROW_STATUS answered 'M' where 'U' is correct. The rule is now the one SAS expresses: a cell the row does not carry is missing and reaches the client as one space, and a value the row does carry is passed through. MPE_TABLES rows omit their blank columns, so the recorded golden still matches; the TESTDATA formula fixture stores '', and keeps it. getdata-blanks.test.js pins both sides. |
||
|
allan
|
c2bf223e35 |
Merge pull request 'feat(licensing): make the app the entry point, licensing via the banner' (#490) from fix/licensing-routes into release/7.23
Reviewed-on: #490 |
||
|
dc
|
3096fb0a13 |
test(licensing): pin the banner state precedence
Every existing case set one state and left the rest empty, so the order the component relies on was untested: a licence problem and an over-capacity state were never rendered for a user who is also unregistered. Two cases now do that and assert the problem/limit message with the registration text absent. Making the problem state lose to the unregistered state turns the first one red. |
||
|
dc
|
0fa6fee85b |
refactor(licensing): one banner component with a state per licence problem
The banner is the only route into the licensing pages now, so it needs a state for every way the licence can block the user, and each state needs to point at the page that can fix it: - a licence problem with no free tier (the startupservice-without-globvars case, which previously fell through to 'not registered') reports the problem and opens its encoded page; - free tier keeps its update-key and problem links; - over capacity, and an unregistered user who cannot register, point at licensing/limit, which had no entry point once the forced navigation went; - an unregistered user points at licensing/register when they can register; - the expiry banner keeps its update link. Moving the banner out of app.component means its states can be tested: ten TestBed cases assert the message and the rendered link for each, including that nothing renders before the startup data lands or when the licence is in order. currentUserCanRegister finally has a reader - it decides between the register and limit links - and the Cypress limit case follows the banner instead of expecting a redirect. |
||
|
dc
|
b159b13a76 |
fix(licensing): only show the registration banner when startup data loaded
A startupservice failure sets isAppActivated false but never emits startupDataLoaded, so the banner would have claimed the user is not registered when the app had not started at all. The banner now waits for the startup data, which is the same flag that removes the loading page. |
||
|
dc
|
8048fcfed3 |
refactor(licensing): report a runtime site-id mismatch without navigating
The incorrect-site-id subscription deactivated the app and then navigated to the licensing key page. With the banner as the entry point the app stays where it is: licenceProblem already carries the URL the banner's More details link opens, and deactivateApp's redirect argument had no other caller. |
||
|
dc
|
7e2047af68 |
feat(licensing): make the app the entry point and the banner the way to licensing
Startup no longer moves the user: an unactivated app stays on the page it loaded instead of being sent to licensing/key, and an unregistered user is no longer sent to licensing/register. The banner in app.component reports the licence state and links to the licensing page - Register while the current user is not registered, Update key for free tier, and More details for a licence problem - so the licence page is somewhere the user goes rather than somewhere they are put. With nothing forcing the user onto licensing/register there is nothing to lock them onto it either, so LicensingGuard (canDeactivate) and its route hook go. A licence problem found while the user is already on a licensing page still opens the page that explains it (navigateWithinLicensing), which keeps the apply-key error flow - and its specs - unchanged. The licensing spec follows the new route through the banner for the expired, invalid and wrong-organisation cases, and updateLicenseKeyQuick waits for the reboot that applying keys triggers before the caller continues. Startup navigation is pinned by app.service.spec: an unactivated app on any page, licensing included, is left where it is. |
||
|
allan
|
d4fd34b167 |
Merge pull request 'feat(mocks): record getdata and viewlibs, and show what the mock gets wrong' (#491) from feat/parity-getdata-viewlibs into release/7.23
Reviewed-on: #491 |
||
|
dc
|
f32027fe59 |
fix(mocks): read the control rows as they are stored, and say what is gated
Nine findings from review, all taken. Row reads (getdata): - approversForTable reads every column through field(): the security and group rows are seeded with UPPERCASE keys and an edited row comes back lower-cased, so a direct property read compared undefined with 'UNDEFINED' and found no grant - silently falling back to the admin group for every table. - tx_to is handled by rowInForce(): epoch seconds, which is what Date.now()/1000 assumes, with an ISO string accepted; a blank, absent or unparseable end date is open-ended rather than expired, so a rule or a grant that omits it is no longer dropped as NaN. The dqrules filter reads rule_active/base_lib/base_ds through field() for the same reason. Rows and the text that describes them (getdata): - the current-rows clause in FILTER_TEXT is now applied to the selection: the grid no longer shows expired rows while telling the user they were excluded. - the primary-key sort resolves each key to the column's own casing, so a lowercase DDL sorts on real values instead of comparing undefined. Settings (getdata, viewlibs): - dcLibref falls back to 'DC_JSLIB' like dcAdminGroup already did: a deploy that has not run makedata has no settings.js, and viewlibs must still list the control library. Wording that had gone stale: - the parity README's Status paragraph now states the invariant (every scenario is gated; gated:false is advisory) instead of counts and a list of differences that were fixed, and the scenarios.json notes say what each scenario pins now. - viewdata's comment states the contract rather than the behaviour it replaced, and the synthetic no-data row writes the same space for a blank character cell as the data rows do. Tests: getdata-rules.test.js covers an uppercase-keyed APPROVE grant and an open-ended rule (both fail on the previous code); viewdata.test.js covers the numeric missing and the empty-table row. |
||
|
dc
|
b301399b5c |
fix(mocks): viewdata sends a blank cell as a space, not the word undefined
String(undefined) put the literal text "undefined" into every cell of a column the row does not carry - the ordinary case in MPE_TABLES, where nine columns are blank on every row - so the viewer rendered the word instead of an empty cell. A value the row does not carry is SAS's missing, so a character column gets a space and a numeric one '.', which is what the empty-table branch beside it already did. sas/mocks/test/viewdata.test.js fails without the change and passes with it. |
||
|
dc
|
3bb9fb52e3 |
fix(mocks): make getdata and viewlibs answer what the estate answers
The two scenarios recorded in the previous commit are now gated, with one declared difference each instead of 388 and 4. The diffs were the mock's, and they are fixed rather than accepted: - cols: dropped the DESC key SAS never emits, and took LABEL/LONGDESC from the MPE_DATADICTIONARY rows for this lib.ds.column, as getdata.sas augments mp_getcols. COLTYPE is blank for DATE/DATETIME/TIME, which is what the estate answers. - sasdata: sorted by the business key; character values trimmed the way the JSON writer writes them, with an all-blank value as a single space. - dqrules: the active MPE_VALIDATIONS rows unioned with the schema's NOTNULL (limited to the columns left in the selection, which is why a NOT NULL on tx_from never reached the grid) and the derived columns' READONLY, then de-duplicated and sorted by column and rule type with X left at the missing value - getdata.sas's union/sort/by-group logic. - sasparams: the current-rows clause mpe_filtermaster appends, and blank character values as the single space SAS sends. - maxvarlengths: the value as stored (trimmed), and the key columns in upper case, which is how the selection spells them. - approvers: computed from MPE_SECURITY's APPROVE rows with the admin group as the fallback, instead of a hardcoded sasdemo. - viewlibs: the shape %mp_getlibs answers - engine in LIBRARYID, blank ENGINE, libref as LIBRARYNAME. The control library and the admin group now come from services/settings.js through dcInit(), which is the mock's %mpe_init(): makedata already writes dc_libref, dc_admin_group and dcpath there, so a deployment with a different DCPATH no longer needs a service change. test/mock-settings.test.js proves it by renaming the library and moving the data. The remaining services still carry const dcLibref = 'DC_JSLIB' and compute paths from it at file scope, so each is a separate change. The sas9 mock keeps its own getdata/viewlibs variant, so it is untouched: the goldens are recorded from the SASjs Server estate. |
||
|
dc
|
3c6cd88159 |
refactor(licensing): drop the inert appLocked navigation gate
appLocked is constructed as false and nothing sets it: the two intended writers in licence.service.ts were commented out, and the only reader was canDeactivate. The branch could not fire in the app, so the guard's tested contract was not the one that ships. Remove the branch, its spec case and the now-unused subject. |
||
|
dc
|
b20261881e |
fix(licensing): one subscription for the licensing action and licence state
The route-params handler and the isAppActivated subscription both called the same redirect check, so a load ran it twice. They now feed one combineLatest subscription that assigns the action and decides the redirects, which still covers a licence state arriving after the page loads and a switch between licensing actions without a reload. The same subscription sends an unknown :action home. Only the values in LicenseActions have a page, so a URL like licensing/whatever previously rendered the component with no action and left an empty page. Licensing spec: 12/12 locally in 49s, including the new unknown-action case. |
||
|
dc
|
47869ef6b7 |
test(licensing): queue the update-page visit before the caller's commands
gotoLicensingUpdate() deferred visitPage('licensing/update') into a
cy.location().then() callback, so callers that queued commands after the bare
call had those commands enqueued ahead of the visit and ran them against
whichever page the app was on. It now returns the chainable and the callers
chain on it, so nothing is queued before the navigation.
waitForAppSettled() also gains .card-header: the key, update and limit pages
render the licensing card even when there is no error to show, and an install
that is not activated yet has no nav tree, no demo banner and no username
field, so the previous set could time out on a page the spec was already on.
Licensing spec: 11/11 locally in 43s.
|
||
|
dc
|
590fa7288e |
fix(licensing): send an activated install home from the registration page
Dropping canActivate left licensing/register reachable by URL on an install that is already registered. Nothing there can succeed: accepting the EULA posts public/registeruser, which aborts for an already-registered user, so a bookmark, a reload or a back navigation into the page ended in an error. The component now sends an activated install home whenever the register action is reached, watching isAppActivated so the decision also covers a licence state that lands after the page loads. That is the part of the old canActivate behaviour the register page still needs, without closing the other licensing pages. The licensing spec's terms helper now waits for the EULA card to disappear: accepting it reloads the app, so the card is the marker that the reload happened, and a refused registration fails there rather than in the selector wait that follows. |
||
|
dc
|
8b6b9baa1d |
test(licensing): hold only a known-unregistered app on the registration page
The guard now holds the registration page only while isAppActivated is false. A null state means startup data has not landed, so no registration has begun and there is nothing to protect; a failed startupservice does not leave it null either - app.service.ts sets false on the malformed, missing-property and request-failure paths - but a route loaded before that point should not depend on it. The spec case for the unknown state flips to expect leaving; reverting the condition makes it red. |
||
|
dc
|
ce153f0a29 |
fix(licensing): limit the registration lockout to unregistered apps
An activated install that reaches licensing/register (stale bookmark, hand-typed URL) had nothing to register: accepting the EULA posted public/registeruser, which aborts for an already-registered user, and every navigation out was then refused, trapping the tab until a hard reload. The lockout now applies only while isAppActivated is not true, which is the state it was written for. The force query parameter was read only by the deleted canActivate, so it is dropped from the five licensing URLs and from the AppService spec rather than left as a dead parameter pinned by a test. Adds unit coverage for the guard's canDeactivate branches. |
||
|
dc
|
bdc50b8838 |
test(licensing): wait for app state instead of sleeping
52 fixed waits (350s declared) replaced by explicit conditions: the app settling out of .app-loading, and the licensing page's own markers. With the licensing routes rendered normally these waits observe real state, so the spec runs in the same wall-clock (1:32 against 1:33) without arbitrary timeouts to tune - three consecutive local runs at 10/10. |
||
|
dc
|
cb3f806d6b |
fix(licensing): render licensing pages like any other route
An activated app that loaded a licensing URL (bookmark, browser reload, or cy.visit in a spec) was redirected to /home as soon as the startupservice response landed, and LicensingGuard redirected there too while isAppActivated was still null. Both redirects meant the page only existed for the window between the router rendering it and startup completing - so the same link worked or failed depending on whether a key was installed. Drop both: the guard keeps only canDeactivate, which still holds a user on the registration page and blocks leaving a locked app. Unactivated apps are still sent to licensing/key from any other page. |
||
|
dc
|
49a9bbf022 |
feat(mocks): record getdata and viewlibs, and show what the mock gets wrong
Two more scenarios, and the recorder could not take either of them as it stood. - editors/getdata reads a table input, not URL parameters, so record.js can now post one: it writes the informational header the server expects (the first line is the SAS input statement, not a list of column names), posts it as multipart/form-data, and takes --libref so the estate's control library name stays out of scenarios.json. replay.js hands the mock its own rows through mockInput, since the two sides do not share a libref. - the test harness's webout() stub now honours the showmeta option and writes the body as members are added. Without them the mocks' own requireMemberMetadata() guard throws - it reads the body before CLOSE - and a service that stopped sending formats and labels would have passed here. - --scenario accepts service/id, because two screens legitimately call their first service 'default'. The two goldens are recorded but marked gated: false: the gate prints their difference counts and does not fail, and check.js labels them ADVISORY. They are a work list, not an acceptance. editors/getdata/mpe_tables - 388 differences, all of them the mock's: cols carries a DESC key SAS never emits and misses the LABEL/LONGDESC/COLTYPE SAS reads from MPE_DATADICTIONARY; sasdata leaves blank character values out instead of padding them and is not ordered by the business key; dqrules differ in content and order; approvers is hardcoded to one user where SAS computes it and returns none. public/viewlibs - 4 differences: SAS answers LIBRARYID='V9' with a blank ENGINE and LIBRARYNAME equal to the libref, and lists SASHELP; the mock answers LIBRARYID=<libref>, ENGINE='V9', LIBRARYNAME='Data Controller(<libref>)' and has no SASHELP. dsmeta is excluded on purpose: it is the engine's report on the dataset (owner, host, encoding, paths, created dates), estate state rather than behaviour, and the projection refused the recorded response until it was dropped - the site id and the service account name were in it. |
||
|
allan
|
c3985b941b |
Merge pull request 'feat(mocks): check the mock against recorded SAS responses' (#488) from feat/mocks-parity-harness into release/7.23
Reviewed-on: #488 |
||
|
dc
|
909580dbda |
docs(mocks): cite the fixture that actually hits the SOME_SHORTNUM rule
Two documentation findings from review:
- The comment in createdemodata.js and the parity README said the excel
fixtures hold "3, 18, 34, 79", which points at values the dropped rule never
saw. The uploads that the rule applies to go to dc_jslib, and
regular_excel.xlsx has five rows where SOME_SHORTNUM is 5 - that is the value
the 1-5 HARDREGEX rejects. The 3s belong to a multi-load fixture whose sheet
targets DC996664.MPE_X_TEST, a libref that carries no seeded validations
because getdata filters MPE_VALIDATIONS on base_lib === requestedLibref and
createdemodata copies only data files into it. Both places now name
regular_excel.xlsx and the value 5.
- README:9 promised the scan refuses an "eight-digit id" while the added class
is \bDC\d{8}\b. It now says "eight-digit DC id", which is what the scan does
and what the date case in parity-golden.test.js relies on.
Checked the fixture rather than the diff: column H (SOME_SHORTNUM) of
regular_excel.xlsx holds 18, 34, 79, 74, 98 ... and exactly five rows of 5.
|
||
|
dc
|
038c40499b |
refactor(mocks): name the synthetic estate by target name, not a sasjsconfig key
The previous commit added "synthetic": true to the server target to stop the recorder claiming an estate it had not checked. That put a harness-only key into the shared build config, where nothing in sasjs reads it and a maintainer could reasonably read it as a sasjs feature. The recorder now carries the list of target names that point at the estate makedata builds (SYNTHETIC_TARGETS = ['server'], which is sas.4gl.io locally) and refuses anything else, so sasjsconfig.json is untouched. The declaration is the same one - it cannot be derived from a response, because makedata is the installer and a customer's control library holds the same rows - it just lives with the tool that acts on it. --target server passes the guard (and needs a token, as before); --target server-mihajlo is refused with a message naming the allowed target. |
||
|
dc
|
7815dcccb8 |
fix(mocks): drop only the MPE_X_TEST rule the excel fixtures break
The previous commit removed every validation rule SAS seeds for MPE_X_TEST, which fixed the excel specs and broke editor 3: the cell dropdown they click for comes from the HARDSELECT_HOOK on SOME_NUM, and that hook had gone with the rest. Only one of SAS's rules actually conflicts with the fixtures - the HARDREGEX on SOME_SHORTNUM, which rejects 1-5 while duplicate_row_excel.xlsx and friends hold 3, 18, 34, 79. Everything else stays, so the mock's rule set for the table is again exactly what it was before the seed work: SOFTSELECT on SOME_BESTNUM, HARDREGEX and SOFTREGEX on SOME_CHAR, SOFTREGEX on PRIMARY_KEY_FIELD and the SOME_NUM hook. The parity README now says this, rather than claiming the table is rule-free, and createdemodata.test.js asserts both halves - the hook is present and the SOME_SHORTNUM rule is not. |
||
|
dc
|
dd2a777438 |
fix(mocks): restore the MPE_X_TEST fixtures and pin the replay's timezone
The parity gate's Cypress job went red after the seed work landed, and the two
causes are the same change seen from the fixture side:
- makedata now carries SAS's demo validation rules for MPE_X_TEST, including the
HARDREGEX on SOME_CHAR that blocks any text but 'the'/'data'. The excel
fixtures upload arbitrary values into that table and expect the submission to
reach the approval screen, so ten excel specs timed out with no Approve
button. createdemodata drops MPE_X_TEST's rules, which is what editor.cy.ts
already documents ("MPE_X_TEST stays clean to match the excel fixtures").
- the LABELs the viewer-labels spec asserts went with the hand-written schema.
createdemodata restores them, because a SAS estate has none and the control
library should not claim otherwise.
Both are declared as fixture divergences in the parity README and asserted in
createdemodata.test.js, so the next change to either side fails a test instead
of a Cypress run.
Review findings:
- the SYSTIMEZONEOFFSET acceptance depended on the host's zone; both entry
points pin TZ=UTC before the harness loads, so the accepted list is the same
everywhere.
- the sas9 mock still published '+0100'; it now answers seconds east of UTC
like the standard estate.
- the README promised an eight-digit id scan that IDENTIFIER_CLASSES no longer
had. A DC-prefixed eight-digit class is back (a plain digit class would fire
on a yyyymmdd date), with a case proving a date is not refused.
|
||
|
dc
|
94cd4912f9 |
fix(mocks): make the golden's claims evidence, not labels
Six findings from review: - estate: 'synthetic' was a constant. The target now declares "synthetic": true in sasjsconfig.json and record.js refuses any other target, so the claim in the file is one the tool had to check before it wrote anything. - project() dropped the key when scrubbing a value, so a top-level member named like a secret was published unredacted. The key is passed through, and checkPublishable now runs secretViolations on the projection, so the recorder refuses before writing rather than leaving it to the next pull request. - denyList restated a subset of ENVELOPE_NEVER. It is built from that list now, so an error text or a hostname in a discarded field is caught too (values shorter than four characters are skipped - they would match half the file). - replay() projected with golden._meta.members while the test validated scenario.members, so editing the scenario had no effect until a re-record. It projects with the scenario's list and reports a member-list mismatch. - replay() deleted SYSCC/SYSENCODING from the expected side when the harness omitted them, which would hide a service that stopped emitting them. The harness now adds the two members the way the server does, and they are compared. - the eight-digit siteId scan class matched any eight-digit payload value (a date, an amount) and refused clean goldens. Removed; the site id is caught by the deny list, which now covers it. |
||
|
dc
|
a57557c60e |
fix(mocks): wire the replay into CI and close the review gaps
Four findings:
- the replay was never run by CI, so the README's "a mock change fails the
pull request" was not true. parity-check.test.js now replays every scenario
through a shared parity/replay.js (which check.js also uses), fails on a
difference that is not declared and on a declared one that no longer differs,
and fails a scenario that has no golden. The differences that remain are
declared per scenario with a reason: the estate's admin group, its
registration state, its timezone and SAS's space padding of blank characters.
- recording stamped tls: verified for a non-https target while sending the
bearer token in the clear. A non-https target is refused outright now, and
--insecure still records tls: unverified.
- the committed-golden gate never applied the secret-key rule. secretViolations
now walks a golden (and a fresh projection) for secret-named fields whose
value is not <REDACTED>, with a case proving a hand-typed licence key fails.
- arg() returned argv[index + 1] unconditionally, so a trailing --ca reached
readFileSync(undefined). Flags without a value are a usage error now.
The replay also found a real mock defect, fixed here: SYSTIMEZONEOFFSET was
emitted as +HHMM while SAS answers seconds east of UTC ('3600'), and the mock's
own test asserted the wrong form - exactly the drift this harness exists for.
The sasdatasets sort used localeCompare, which puts MPE_X_CATALOG-FC before
MPE_XLMAP_RULES; SAS sorts on the byte value, so the mock now does too.
|
||
|
dc
|
6c831bb230 |
fix(mocks): verify TLS when recording, and project staging ids whole
Two findings from review: - record.js turned certificate verification off unconditionally, on the request whose response becomes the golden. Verification is now the default; --ca trusts a private estate's own certificate and --insecure is opt-in, warns on stderr and is recorded as _meta.tls so a golden taken over unverified transport cannot be mistaken for one that was not. The estate's certificate verifies, so the committed golden is tls: verified. - the libref placeholder ran before the staging-id one, so a staging id came out as <LIBREF>-1_2_3. The staging rule now runs first and matches both minted shapes (DC<date>-<time>_<n>_<n>, and the mock's millisecond form); a project() case pins the ordering. |
||
|
dc
|
b98fde1c74 |
docs(mocks): record that the parity refresh is an ad-hoc agent job
The pipeline cannot reach a SAS estate, so the goldens are refreshed where the credentials are - an agent or a maintainer running record.js - and the CI half stays offline: scan the committed goldens, replay them against a mock estate. Say that in the harness README so the next reader does not go looking for a scheduled refresh job that cannot exist. |
||
|
dc
|
6262843f3f |
feat(mocks): check the mock against recorded SAS responses
The mock services are a second implementation of the SAS backend, and the only thing comparing them was a human reading two files. This adds the harness that compares their output instead. A golden is a projection of a SAS response, never a copy: project.js keeps a fixed member list per scenario, drops the whole envelope (hostname, site id, user, home paths, job ids, _PROGRAM), substitutes the identifiers the diff needs (<LIBREF>, <PATH>, <EMAIL>, <STAGING_ID>, <DATETIME>), and redacts secrets by key name. The recorder then refuses to publish if any identifier class survives, or if any literal the envelope just discarded still appears in the projection - which is what catches a hostname echoed inside an error message. record.js records from a live estate into .captures/ (gitignored) and writes only the projection to golden/. check.js replays a golden against a mock estate built by the test harness and prints field-level differences; it needs no server and no credentials. parity-golden.test.js re-runs the identifier scan over every committed golden, so a hand-edited file cannot slip through either. Phase 0 ships one scenario, public/startupservice. The replay is advisory and already reports twelve differences worth a look (admin group casing, the registration counters, timezone representation, the sasdatasets order, blank versus empty Excel map descriptions); the corpus, the merge gate and the nightly drift job follow. Stacked on the mocks context in the parent branch: it seeds the mock estate with seededEstate(), which that branch adds. |