100 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
dc	8c9b24a824	fix(client): read a password rejection the way the service does ... isPasswordError compared the raw message against a lowercased test, while both checks in the service lowercase first (spreadsheet.service.ts:135 and :152). A capitalised "Password required" was a password error there and not one here, so the caller fell through with the previous sheetNames and no modal. Lowercased here too, with a capitalised case pinned in the spec - proved to bite, that case fails against the case-sensitive comparison. Dropped the fileLoadingState = ready assignment in the read's catch: onDiscardFile() sets notSelected as its first act, so the screen never ends in ready and the line only described the catch it sat in.	2026-11-09 19:40:04 +00:00
dc	0299418da8	fix(client): settle the multi-dataset read when the engine cannot load ... The engine became a lazily loaded chunk, so xlsxReadFile and parseExcelSheetNames can reject before any parsing starts. Two call sites in the multi-dataset screen assumed the old shape. xlsxReadFile was called with .then and no .catch: a failed chunk load left fileLoadingState on FileLoadingState.reading with nothing on its way, which is the dialog that never moves that the service spec was written to prevent, at the other call site. It now resets the stage, discards the file and reports through the info modal. The password test was err.includes('password'), and rejections are not all strings: the parse path rejects with strings, the chunk path with an Error, and .includes on an Error throws a TypeError inside a setTimeout-invoked handler that has no catch. isPasswordError reads .message when there is one, and its spec pins the Error case plus the empty rejection that used to throw. parseExcelSheetNames also awaited the engine inside the FileReader handler. The util module that loadSpreadsheetUtil() returns imports @sheet/crypto at module scope, so by the time the read is under way the engine is already in the module registry and the dynamic import there only re-requested it. The engine now loads once, before the reader promise exists, so a failed load rejects the method instead of an async handler. Verified: tsc clean, six specs green (the password spec proved to bite - two of its three cases fail against the old .includes), production build unchanged in shape with the engine still in its own 668.* chunk.	2026-11-09 19:40:04 +00:00
dc	a42afda9f6	test(cypress): name the library a tree lookup wanted, and expand once ... openTableFromTree failed two ways that named the wrong thing. A library that is not in the tree surfaced as "Expected to find element: `filter`" - the jQuery call, 30 seconds later - because the lookup was .filter().first() over the nodes. The match now carries the message "a tree node for library \"...\"" and is asserted inside a .should, which retries while the tree is briefly empty during its tables load. The expansion clicked the label and then simply moved on. The label TOGGLES ( home.component.ts libraryOnClick ), so a click that lands while the node is still collapsing leaves it closed; the expansion is now asserted on a fresh query, which Cypress retries for the timeout, so it fails as "library \"...\" to expand" rather than as a missing table further down. Deliberately not changed: the click on a table row is still a plain .click(). The "covered by another element" failure this branch hit is a layout race, not a click strategy - measured on the deployed build, a library's .clr-treenode-children sits at height 0 while its tables load and styles.scss's `.clr-tree-node { overflow: visible !important }` lets those rows paint over the next library, so one table's button covers another's. A retry loop absorbed it in one run and not in the next two; `{ force: true }` would only hide it.	2026-11-09 19:40:04 +00:00
dc	ca157a307a	perf(client): load the Excel engine on first use ... spreadsheet-util imports @sheet/crypto (1.75 MB: xlsx 958 kB, cpexcel 472 kB, jszip 323 kB) and buffer at module scope, and SpreadsheetService imported it statically, so the editor route downloaded the whole engine to draw a drop zone. The engine now sits behind a ChunkLoader that caches the load for the success path only, and SpreadsheetService awaits it in the two methods that need it. parseExcelSheetNames imports the engine inside the handler's try, because an async FileReader handler swallows its own rejection: a chunk that fails to load has to reject that promise or the caller waits on a dialog that never moves. Measured on the deployed build: the editor route loads 11 scripts with the engine chunk absent, and attaching a file to the upload input loads it with no page errors. Two specs, both proved to bite: the loader retries after a rejected load (fails without the cache clear), and parseExcelSheetNames rejects when the import does (fails with a timeout if the rejection is swallowed).	2026-11-09 19:40:04 +00:00
dc	429fe3f30f	perf(client): state the browser targets and drop the IE-era polyfills ... There was no .browserslistrc and no browserslist key, so the build had been resolving browserslist's defaults - 32 entries including and_qq, and_uc, op_mini and Chrome 109. The new file states the policy as version floors, so the build moves only when we move it: chrome >= 131 (November 2024) edge >= 131 (November 2024, Chromium numbering) firefox >= 128 (July 2024, the ESR - corporate pins it) safari >= 17 (September 2023, and iOS 17 with it) not dead With the targets stated, src/polyfills.ts no longer needs the fifteen core-js ES6/ES7 imports it carried from the IE era, and nothing else in src imports core-js, so the dependency goes too (package.json, package-lock.json and the angular.json commonjs allow-list). Measured on the same tree: polyfills.js 155,341 -> 93,953 bytes raw (-39.5%), gzip 47,819 -> 25,767 (-46%, about 22 KiB per visit on every route); main.js unchanged.	2026-11-09 19:39:45 +00:00
dc	0e15195191	docs(client): say the double write stays latent until Clarity's binding goes ... The comment claimed the disagreement 'only showed once the tree stopped taking its state from Clarity's two-way binding'. It has not stopped: viewer.component.html still binds [(clrExpanded)]="library['expanded']", so Clarity writes the same flag on every caret click and the double write is latent rather than visible at this head. It becomes visible when that binding goes, which is the tree work in #554.	2026-11-09 19:06:35 +00:00
dc	fb02788caf	fix(client): let one place own a library's expanded state ... Two callers disagreed about who sets a library's expanded flag, and the code that focuses the tables search box depended on an id that only exists while the library is open. selectTable ended by toggling the flag: library['expanded'] ? (library['expanded'] = false) : (library['expanded'] = true) That was written against Clarity's two-way binding, where the caret also toggled the same value, so the two cancelled out in the case anyone noticed. libraryOnClick already toggles it for a user click, and loading a library's tables means the library is open, so selectTable now sets it to true and leaves the toggle to the handler. The search box was found with document.querySelector('#search_<ref>'), which works only while the node is expanded and re-rendered. It is now a @ViewChildren lookup matched on data-libref, so nothing depends on that id - which is also one less assumption for the tree work in #554, where the markup that writes the id is due to change. No behaviour change intended: with Clarity still rendering the tree, the five specs that exercise it are 30/31, the single failure being viewbox test 7, the animated-expansion flake that #554 owns. viewer-labels 5/5, full-table-search 1/1, download 8/8, row-cell-limits 5/5. Unit suite 1401 passing.	2026-11-09 17:20:22 +00:00
dc	1e495ddf42	fix(helper): keep the sign of a SAS date before 1960	2026-11-09 15:25:09 +00:00
dc	5ef83c2a1a	test(coverage): bring every client file above 50% lines	2026-11-09 14:54:47 +00:00
dc	9235efc63b	chore(multi-dataset): stop the workbook poll on the no-file path	2026-11-09 14:05:45 +00:00
dc	412da1d593	test(coverage): clear the workbook poll the reject path leaves running	2026-11-09 13:44:46 +00:00
dc	6504de6c44	test(coverage): guard the query filter modal and the multi-dataset upload screen	2026-11-09 13:39:47 +00:00
dc	3f7e865c9d	test(coverage): poll for the logon state instead of one timer turn	2026-11-09 13:39:33 +00:00
dc	5cd9730a42	test(coverage): guard the sas service request, deploy and login paths	2026-11-09 12:51:25 +00:00
dc	e6a0c52745	test(coverage): pin the checkOnly path, the registration boundary and the measured position	2026-11-09 12:51:14 +00:00
dc	b39277fad4	test(coverage): guard the licence, spreadsheet and autocomplete paths	2026-11-09 12:10:47 +00:00
dc	c34498dc6d	fix(build): take @sasjs/cli 6.19.3, which stamps the drive it ships ... 6.19.1 carried the restore-path host files, which is what staging needed to answer startupservice again. 6.19.3 adds the build stamp: a persisted drive whose stamp is absent or different is dropped and re-seeded, so a visitor stops running the previous build's mock services (and a drive left broken by the webout bug clears itself on the first load after this deploy), and the config is resolved from the worker's folder rather than the page URL. Verified: npm ci accepts this lock and installs 6.19.3; its mock build carries buildStamp in mockSite.js (grep -c = 3), controller.scriptURL in persist.js and the hostFiles load in runtime-core.js.	2026-11-09 11:14:23 +00:00
dc	300cf081db	fix(build): take @sasjs/cli 6.19.1, which restores a drive with its host files ... The browser mock loads mock-config.json and the webout() block on the seed path only in 6.15.0, so a visitor whose drive was restored had every webout() service fail with ReferenceError: webout is not defined - which the app reports as 'Globvars, Sasdatasets, Saslibs, XLMaps are not present'. Fixed upstream in sasjs/cli#1568, published as 6.19.1. Verified: npm ci accepts this lock and installs 6.19.1 (163 packages, node 22); a mock build with it emits host/runtime-core.js carrying the load() host-files change. The lock's new entries are 6.19.1's own dependencies (esbuild platform packages), not a re-resolution of the tree.	2026-11-09 10:27:54 +00:00
dc	6c4d66a57c	fix(startup): retry startupservice while the session is still loading ... A cold SAS session can answer startupservice before the libraries are assigned and return without Globvars, Sasdatasets, Saslibs and XLMaps. The app treated that first answer as final and showed "X are not present in the startupservice", while a reload a few seconds later worked - reported against staging. The request is now retried for a few seconds (five attempts, 1s/2s/3s/5s) and only the last answer is reported. An abort body is retried too, because that is the same transient case, and when it never clears its MSG is quoted in the dialog instead of the generic missing-props text. A malformed response - the adapter passing a plain-text job error through - is a verdict about configuration, so it is still shown immediately. `retryUntil` lives beside `retryOnce` in shared/utils and is unit tested.	2026-11-09 00:21:08 +00:00
dc	67c8ad3481	fix(makedata): never default a flag before reading it ... The configurator link sends `sampledata` as a URL parameter and no input table, and the `%let sampledata=N` default added with the Y/N change overwrote it before the service could read it. The log from the deploy screen shows the parameter arriving as `%let sampledata=%nrstr(Y)`, then `GLOBAL SAMPLEDATA N`, `GLOBAL EXISTS 0`, `GLOBAL INDS _null_` - so the library was skipped in silence. Both flags now carry no default at all. The column is read as it arrives and one data step after it normalises from the macro variable - which holds the column if there was one and the URL parameter if not - to Y or N, absent included. That is the rule the old resolver documented: a default written first is what discards the parameter. The mock gains the case for the parameter route with no table, which is the shape that failed.	2026-11-08 23:22:30 +00:00
dc	f616afce8d	test(cypress): wait for the tree toggle, on a re-queried node ... The post-click wait was dropped when the click stopped holding a jQuery subject across the re-render, on the assumption that the wait was what failed. It was not: the wait re-queries the node on every retry, which is why it is safe in this shape and was not safe when the click was made through `cy.wrap($lib)`. Without it, a click that does not expand surfaces about 50s later inside `cy.contains('.clr-treenode-link', tablename)` as a missing table, which is the opposite of what went wrong. Verified locally at this head: viewer-labels 5/5, full-table-search 1/1, download 8/8, row-cell-limits 5/5 - the four specs the withdrawn version broke.	2026-11-08 23:08:56 +00:00
dc	4a2f10d19a	fix(makedata): the configurator link, the flags' spellings, and a lock npm ci accepts ... Three findings from the read at 2f2b40e1. - `configurator.sas` built the makedata link with `&path=`, which the server target no longer reads - the link carried no path and makedata would have built at "/DC...". It now sends `&dcpath=`, the spelling the client configurator and the viya target use. - Both flags accept the spellings they always did (1, Y, YES, TRUE) and are stored as Y or N, so KEEPME=1 keeps the program as before instead of silently deleting it. The value the service then tests is a single one, Y or N; the mock and its case read the same list again. - The client lockfile is regenerated so `npm ci --ignore-scripts` accepts it: verified in a scratch copy with node 26 / npm 11.20.0 (the job's toolchain is node 24.15 / npm 11.12), which installed 1813 packages from the lock.	2026-11-08 23:05:55 +00:00
dc	46241a025a	test(client): post the coverage summary to the pull request ... The reporter alone leaves the numbers on the runner, where nobody reads them. The unit-test gate now runs with --code-coverage and a script posts one comment per pull request, found again by a marker and patched in place, so a push replaces the table instead of adding another comment. Same shape as the lighthouse comment, for the same reason. The comment carries the four totals and the twelve lowest-covered source files (40+ lines, app code only). It is information, not a gate: a comment that cannot be posted never fails the job, and no threshold is asserted.	2026-11-08 23:01:34 +00:00
dc	608482d421	test(client): switch the code coverage reporter on ... karma-coverage was loaded and configured but never in the reporters list, so the client has had no coverage output at all. Adding it to the reporters changes nothing for the default run (istanbul only instruments when --code-coverage is passed, so the reporter has nothing to say otherwise), and `npm run coverage` turns it on and writes the html report plus a json-summary under coverage/datacontroller. Measured on this branch: 906 tests, statements 50.21%, branches 46.78%, functions 45.12%, lines 49.87%.	2026-11-08 23:01:34 +00:00
dc	2f2b40e140	refactor(makedata): dcpath everywhere, and both flags are Y or N ... Review points 2 and 3 from the last read. - `dcpath` is the name of the value in all three makedata targets: the server target read the DCPATH column into `&path`, so a `dcpath` URL parameter - the spelling the viya target and the configurator already use - was never seen. The configurator now sends `&dcpath=`, the SAS9 target reads `dcpath`, and the server target's body reads `&dcpath`. - KEEPME and SAMPLEDATA are one value each, Y or N, on both sides. The frontend already sent Y/N; the backend now keeps them as text (uppercased on read, N when absent) instead of converting to 0/1, and the three consumers test for Y: self_destruct, the settings line that assigns DC_DEMO, and mpe_makesampledata's iftrue (default N, builds only on Y). The mock reads the same single value. Also: a request is one row - the screen posts one and the seed files carry one - which is stated in the header, since KEEPME is read from the row rather than scanned across the table. Package bumps: @sasjs/cli 6.10.0 -> 6.15.0 (sas), @sasjs/adapter 5.1.2 -> 5.2.6 and @sasjs/utils 4.3.1 -> 4.9.1 (client); @sasjs/core was already 7.1.8, the latest. The client lock was re-resolved by npm, which also prunes entries the current graph no longer reaches - CI's npm ci is what confirms it.	2026-11-08 22:39:54 +00:00
dc	33f038ce8e	test(cypress): re-query the tree node per step, not once per call ... The expansion check kept a jQuery subject across a click that re-renders the tree, and the assertion that followed retried over that detached subject - viewer-labels, full-table-search, row-cell-limits and download went red with 'cy.first() failed because the page updated as a result of this command'. Each step now starts a new chain, which is what the previous helper did; the only change kept here is that the label is clicked when the node is closed rather than unconditionally.	2026-11-08 22:34:47 +00:00
dc	c1aec63b3e	refactor(makedata): the snippet as specified, with the two flags normalised ... The one data step now declares keepme and sampledata in the LENGTH statement and reads them straight through, as asked. Both are then turned into a number with indexw(): 0 is off, anything else is on, which is what self_destruct, mpe_makesampledata's iftrue and the settings line all test. That last step is not optional: 'Y' in a macro %if is a character operand (%if not (Y) errors), and self_destruct's 'ne 1' test read KEEPME='Y' as "not 1" and deleted the program - which sas/deploy/makeDataSas719.json asks for. One line covers both flags: %let keepme=%sysfunc(indexw(1 Y YES TRUE,%upcase(%superq(keepme)))); %let sampledata=%sysfunc(indexw(1 Y YES TRUE,%upcase(%superq(sampledata))));	2026-11-08 22:04:00 +00:00
dc	70b60ce26a	refactor(makedata): one data step reads the request's values ... The two-step resolver and the resolve macros are gone. makedata now reads the values it uses - ACTION, DCPATH, ADMIN, OUTFMT, KEEPME, SAMPLEDATA - from the input table into macro variables of the same name, in the one data step the service already used for DCPATH and ADMIN. A LENGTH statement declares the text ones, so a request that sends no table or no such column leaves the macro variable as it was (a URL parameter, or the default) instead of creating a missing numeric - which is what the not(missing()) gates are for. The two flags are read with cats() and normalised to 0/1, because a JSON 1 imports as numeric and 'Y' as text. ACTION is then checked in the two aborts that were already there, against the value itself instead of a MISSING/BAD sentinel. mpe_action keeps its deprecation notice; nothing calls it.	2026-11-08 21:59:43 +00:00
dc	fa51707f53	test(cypress): expand a library only when it is closed ... openTableFromTree clicked the library label unconditionally, and that click toggles expansion - home.component.ts libraryOnClick flips `expanded` - so a call made while the library was already open closed it and then searched the node that had just been collapsed. The helper now finds the node once, expands only when it is closed, and asserts the expansion before looking for the table, so a swallowed click fails where the expansion is named instead of 50s later inside `contains` reading as a missing table. The node's table links stay in the DOM when it is collapsed (Clarity's clrExpanded hides them rather than removing them), which is why the old behaviour could pass while collapsing the library.	2026-11-08 21:50:11 +00:00
dc	576b0cfd17	docs(makedata): the comments point at the one-pass read, not the deleted macro	2026-11-08 21:43:15 +00:00
dc	696ab2eb48	refactor(makedata): one pass over the request columns, no resolvers ... makedata now turns every column of its input table into a macro variable of the same name, in one pass - the idiom the service already used for DCPATH and ADMIN, applied to the whole row. ACTION, SAMPLEDATA and the rest are then read the same way whether the caller sent them as a column of the table (the deploy screen, the CLI seeds) or as a URL parameter (the configurator, the run-in-new-window route), and nothing downstream needs to know which. One data step normalises what has to be validated: ACTION to MAKEDATA, DIRLIST, WRITECHECK, MISSING or BAD, and SAMPLEDATA to 0 or 1 for mpe_makesampledata's iftrue and the settings line that assigns DC_DEMO. Both are read with symget(), so a request value is never resolved as macro text. mpe_sampledata, added earlier on this branch, is deleted. mpe_action is deprecated in place: no service calls it now, and it is kept for one release in case a deployment ships its own services. The drift this removes was in DCPATH: the generic pass makes the column &DCPATH, so the body no longer renames it to &path.	2026-11-08 21:42:09 +00:00
dc	89e944d84b	fix(makedata): resolve the sample flag from the column or the parameter ... The in-page deploy screen posts the flag as a SAMPLEDATA column of its fromjs row and no query parameter reaches the program on SASjs Server, so reading only the `sampledata` parameter left the checkbox inert. mpe_sampledata now resolves either route into one 0/1 flag, with the column winning as ACTION does, and all three targets call it instead of carrying their own copy of the list. The SAS9 target's nested resolve_sampledata macro goes with it: the shared macro reads a column when the caller sends one and the parameter otherwise, so there is one list of accepted values rather than three. The mock reads the column too, and the tests cover the shape the screen posts - the column alone builds the library, an N in the column does not, and the column beats the parameter.	2026-11-08 20:59:05 +00:00
dc	ed6fe8e179	test(mocks): a case per control-table post-edit hook ... Nineteen cases through editors/stagedata: one row each hook refuses and one it accepts, and for the accepted cases the staged JSON, because a normalised value (an uppercased key, a coerced flag, a rewritten FORMAT_CAT dataset name) is only visible once it has been staged. Every case fails with its hook file removed, which is the property that matters: before these files existed the hook was skipped in silence and the submission was staged. The fixtures the joins need are written into the estate - a live MPE_ROW_LEVEL_SECURITY rule for the REPLACE conflict, a live MPE_VALIDATIONS hook for the duplicate-hook merge, an active MPE_DERIVED row for the derived-variable refusal - so each case proves the join, not just the message.	2026-11-08 20:23:40 +00:00
dc	3f0d3a8177	fix(makedata): one flag drives the sample library, and no SYSSITE gates ... Three review findings on #532, which is now in this branch's base, and the rows that only one estate ever received. - The flag is one macro variable. Each target normalises `sampledata` once, and that same variable is both the mpe_makesampledata `iftrue` and the condition on the settings line that assigns DC_DEMO. The input table's column read and the second `dc_demo_built` variable an earlier draft carried are gone: there is no precedence left to disagree about, and the library and the config line cannot say different things. - The SYSSITE gates and the 4GL demo estate's configuration rows are deleted with them - row level security, the excel config, the security rows and the MPE_USERS registration. They named metadata groups that exist on one estate and described a deployment the sample library replaces; the macro now does one thing, and no deployment's control tables depend on a site id. - The estate's data library is DC_DEMO, built when the deploy asks for it like any other deployment's: STUDENTS/VEHICLES/FORMATS in place of the CLASS/CARS/SPRINGS/FMT_CHECKS an earlier version built in `dc_demo`. The old folder and its datasets stay on disk, unread, and the control library stops naming them.	2026-11-08 20:23:17 +00:00
dc	e0c80bed6b	fix(deploy): the sample data field's comment sits above the field ... The JSDoc that documents runAutoDeploy had a second doc block for the new createSampleData field appended between it and the method, so the first block documented a boolean and runAutoDeploy was left undocumented.	2026-11-08 20:22:51 +00:00
dc	1808c8b4f6	feat(mocks): implement the seven missing control-table post-edit hooks ... mpe_makedata registers eight post-edit hooks on its MPE_TABLES rows; only mpe_derived_postedit had a mock implementation. mpeRunhook resolves a hook through mockHookSource and returns null when the .js is absent, and both callers only run a hook when the source is non-null, so the other seven were skipped in silence: the mock accepted control-table edits the estate refuses. Each hook is a port of its SAS program, in the same order of checks and with the same message text: - mpe_tables_postedit: normalisation, reserved librefs, LOADTYPE, FORMAT_CAT, the REPLACE-vs-row-level-EDIT-security rule (#211) and the AUDIT_LIBDS check - mpe_security_postedit: the two closed lists (libref, ACCESS_LEVEL) - mpe_validations_postedit: a column with both hook types, and an unparseable HARDREGEX/SOFTREGEX, merged with the live rows as the SAS `union` does - mpe_row_level_security_postedit: the REPLACE conflict, a rule on a derived variable, and mp_filtercheck's shape checks per table - mpe_column_level_security_postedit: the scope list and the 0/1 coercion - mpe_xlmap_info_postedit: the target dataset and its columns against MPE_XLMAP_DATA in both directions - mpe_xlmap_rules_postedit: XLMAP_ID upcased, as the one-step SAS hook does The row reads and writes a hook makes (hookField, hookSet, hookDeleted, and the libref guard a hook needs before reading a library a user named) live in dcMockUtils beside the other shared helpers, so a hook file carries only the rules it enforces. The xlmap hook's LIBDS test is dcLibds.js's isValidLibds, not a copy - mock-settings.test.js refuses a second copy of that rule - so stagedata and loadfile load dcLibds.js beside dcMockUtils.js the way the read services do. The row-level-security hook carries mp_filtercheck's shape checks but not %mp_filtervalidate's SQL compile, which needs the target table assigned; the file says so rather than implying the same guarantee.	2026-11-08 20:22:31 +00:00
dc	d6e3795678	fix(editor): keep the grid's box in place while the table loads ... The loading placeholder sat above the grid in the flow at a fixed 70vh, so the grid's container - which is always present - was pushed a viewport height down while the data request was in flight. Removing the placeholder when the table arrived moved that container 663px up: a layout shift over 84% of the viewport, measured as CLS 0.437 on /editor/<table> (the same build measures 0.009 on a route with no grid). The placeholder now renders inside the grid's own box as an absolute overlay, so the box is already where the grid will be when the first paint happens. Measured on the mock estate, same URL, three runs: CLS 0.437 -> 0.249, and the container's top moves 11px between the two states instead of 663px. The remaining shift is the action row growing when the table name arrives; it is not this change and it needs its own attribution.	2026-11-08 20:12:52 +00:00
dc	52bb582cb7	feat(makedata): optional sample data library under dcdemo ... The deploy screens offer a checkbox, off by default, that asks makedata to build a sample data library beside the control library. It sits at the bottom of each screen and explains itself only once it is ticked: the option is not one most administrators will want, so it is not put in the path of the screen's action. Each makedata target resolves the flag itself - the URL parameter first, then the adapter's SAMPLEDATA column when the input table carries one - so no extra macro is deployed for it. mpe_makesampledata creates <dcloc>/dcdemo, assigns it as DC_DEMO, writes three tables and registers them in MPE_TABLES, and takes iftrue=0 by default so a call that omits the flag builds nothing. mpe_makesampledata keeps what it already had for the 4GL demo estate (row level security, excel config and security rows, gated on SYSSITE) and gains the sample tables, so one macro owns the whole job. The mock makedata's mpeMakesampledata is the same function with the same arguments, tables and rows. The rows are generated from a fixed rule rather than copied from SASHELP, so a SAS estate and the JS estate hold the same values; makedata-sampledata.test.js compares both estates' registrations against the macro source. The pre-commit prettier gate could not run here (no npm on PATH); the same prettier check was run directly against the staged client files and passes.	2026-11-08 18:46:15 +00:00
dc	6404d6bd73	docs(viewboxes): keep the resetSelectedViewbox doc with its method ... The new field's doc block was inserted between resetSelectedViewbox's own JSDoc and the method, so the reset doc described an id counter and the method lost its doc entirely. The counter's block now sits with the counter.	2026-11-08 18:41:42 +00:00
dc	2c9acf9c1b	revert(cypress): the tree-row hit-test guard, which cost twelve specs ... Run 3551 turned the one "covered by another element" failure into twelve: every tree-row click below the fold timed out on MPE_X_TEST: the row is the topmost element at its centre, not under a label The guard measured the row with elementFromPoint inside a should(), and a should() does not scroll: .click() scrolls the subject into view, the assertion that preceded it did not, so the coordinates it probed were wherever the row happened to sit in the viewport - usually outside it, where elementFromPoint answers null or some other element. A guard that has to reproduce Cypress's own hit-testing is worth less than the flake it was written for, so the click is plain again. The command stays, because the two call sites still shared twelve lines of waiting and filtering logic and the navigation tree and the viewboxes panel should not drift apart. The original flake is a real one and is not explained by this: it hit one spec in one run at 6394a7af and has not recurred. When it does, the finding to chase is what covers the row at the moment of the click, with the element named - not a blanket assertion on every tree click.	2026-11-08 18:39:31 +00:00
dc	0bd553fb00	fix(viewboxes): mint viewbox ids that cannot repeat, and select without a timer ... Two review findings, both right: - `id: this.viewboxes.length + 1` was not unique. Removal does not renumber, so with two viewboxes, remove the first and add another and the new one is 2 again: two rows carried `open-viewbox_2`, getElementById answered the first in document order, and `resetSelectedViewbox` then selected the older entry's table as well. Ids come from a counter that takes the highest existing id into account every time, so a restored viewbox cannot be collided with either. - The 1200ms delay was a guess, and the comment justified it with waits that do not exist - the table build waits 100 and 500 ms, and 1200 appears nowhere. `detectChanges` is what puts the row in the view, so selecting and scrolling straight after it is the earliest point that works and does not guess how long the table takes; loadData's existing callback re-selects the viewbox once its table is registered, which is what fills the pane. The row lookup is scoped to the component's own Currently open list through a ViewChild rather than a document-wide id, which is the other half of the duplicate-id problem. Build clean, karma 896 SUCCESS.	2026-11-08 18:11:17 +00:00
dc	87b6b84b5f	feat(viewboxes): open the new table's configuration when it is added ... Choosing a table in the panel's tree adds the viewbox on the right hand side, where the filters and column selection live - but nothing moved the user there, so the tree row they had just used stayed under the pointer and the configuration stayed on "select viewbox from above". The new viewbox is now selected once its table is registered - the same resetSelectedViewbox the row's own click runs, forced through change detection because the addition is async - and its row is scrolled into view, so a second or third viewbox does not appear off-screen. Each open viewbox row carries an id (open-viewbox_<id>) to make that lookup possible. Deliberately not done here: collapsing the tree, or changing the tree's own selection. The tree stays usable for adding another viewbox; the selection is on the right hand side. Karma: 896 SUCCESS. The visual check did not run: the local estate is on the FREE tier, where a second viewbox trips "Viewbox open limit reached" and tableOnClick returns before this code, so the panel never shows an added row to observe. CI runs with a licence key and is where this gets looked at.	2026-11-08 17:25:38 +00:00
dc	4deda2b4a6	fix(mocks): read the rejection reason from MPE_REVIEW, as the estate does ... gethistory.sas takes review_reason_txt from a left join on &mpelib..mpe_review (gethistory.sas:128, :130) and MPE_SUBMIT carries no such variable. The mock read the submission, so a rejection's reason came back blank exactly where the product shows it: the rejection path writes the reason to MPE_REVIEW only (approvers/rejection.js:81). The reason now comes from the latest review for the table - the join's own rule - with the submission's column as the fallback for a row the join misses. Seeded directly in the test rather than through the edit cycle: no corpus case rejects, and the subject here is the read. Three cases: the reason is shown, the latest review wins, and the submission's column still answers when no review row exists. Without the fix two of the three fail; with it, three pass.	2026-11-08 17:07:39 +00:00
dc	df8cdfda0f	test(cypress): one tree-row click command instead of two copies ... The elementFromPoint wait was written twice, once in the navigation helper and once in the viewbox spec, both reached from the same specs. It is a custom command now - cy.clickTreeRow() - so the next time the tree's markup moves there is one place to fix. The command keeps the behaviour the copies had: wait until the row is the topmost element at its own centre, which rides out a transient cover and still fails loudly when the cover is real. The message names the row by its own text, taken from the element rather than passed in, so a failure says which row.	2026-11-08 17:05:23 +00:00
dc	8689805d40	test(cypress): wait for the tree row to be on top before clicking it ... Run 3541 failed one spec out of 184 - viewbox case 7 - because the click on a `.clr-treenode-link` in the add-viewbox panel reported "covered by another element", a `<p class="m-0 cursor-pointer">` from the tree itself. It passed at 924d1eb1 and passes in the specs that ran before and after it in the same job, so it is not a standing layout defect. It also does not reproduce here. The built app on a seeded estate, measured at both 1600x1000 and 1000x800: the library label ends at y=310 and the first row starts at y=353, a 43px gap, `verticalOverlapPx: 0`, and `elementFromPoint` at the row's centre returns the row's own button. Screenshots of that panel are kept alongside this work. So the cover is transient - the row exists and is visible while the tree is still settling - and the honest guard is to wait for the row to be the topmost element at its own centre, which is what both click sites now do. A transient cover is waited out; a real one still fails, with a message that says which element was on top. That is the difference between this and `{force: true}`, which would have hidden the same failure.	2026-11-08 16:31:02 +00:00
dc	e869e17717	fix(client): wire the icon generator into the hooks that exist, and cover the components' shapes ... Three review findings, all correct: - `prebuild` was already taken: line 35 runs the version, EULA and licence step, and JSON keeps the last key, so the generator I added never ran and a new shape would have shipped an empty icon against a stale committed file. The call is folded into the existing `prebuild`, and into `prestart`, which is the only hook `ng serve` runs. - Five shapes Clarity's own components draw were missing: `calendar` (the date container and soft-select), `event` (the picker's month and decade buttons), `arrow` (a sortable datagrid column's indicator), `ellipsis-vertical` (the action overflow) and `window-close` (the signpost content). Each renders an empty box rather than failing, which is what the component list exists to stop. - The generator's comment claimed a render test caught a missing shape and there was none. `src/app/clr-shapes.spec.ts` is that test: it renders every name in the generated list and fails on any that paints no `<svg>`, plus a list assertion for the five component shapes - against the generated list rather than `ClarityIcons.get()`, because a neighbour's import can register a shape incidentally and a check that passes for that reason proves nothing. The spec was probed rather than trusted: with `arrow` removed from the generated file it fails (1 failed), and with it restored the suite is 898 SUCCESS. My first version of the component assertion passed under that probe - it queried the registry, which another import had already populated - which is why it now asserts the list.	2026-11-08 16:03:48 +00:00
dc	2d137677d0	fix(mocks): answer the cycle's remaining members, then gate it ... - editors/getdata: COLTYPE now carries the handsontable type for temporal columns ({"data":"SOME_DATE","type":"date"}), built the way getdata.sas:428-461 builds it from the column's format family. A column the editor hides (TX_FROM / TX_TO) still carries a blank spec, which is what the mpe_tables golden pins. - auditors/postdata: REVIEWED_BY_NM answers a blank when there is no reviewer, and the approval reason is written onto MPE_SUBMIT as well as MPE_REVIEW, because gethistory reads the submission. - approvers/gethistory: publishes REVIEW_REASON_TXT ("APPROVAL 1 of 1") and no longer publishes NUM_OF_APPROVALS_REQUIRED, which the estate does not send. chain/cycle_mpe_x_test is gated: the 21 differences the first replay reported are closed, and the two that remain are declared - the mock's MPE_X_TEST holds seven rows against an estate table larger than DC_MAXOBS_WEBEDIT, so only the estate truncates. Mock suite: 166 tests, 166 pass.	2026-11-08 15:25:37 +00:00
dc	fa70513422	fix(mocks): stage only the table's columns, and hand back the log link ... Two differences from the same estate behaviour: - stagedata.sas builds the staged dataset from the base table, so a payload naming a variable the table does not have never reaches the staged table or the approval pack. The mock staged whatever the payload named, which is how a hardselect column the editor never showed turned up in getstagetable. - sasparams[0].URL is the load log link the approver opens: ?_program=/Public/app/dc/services/editors/getlog&table=<DSID> INPUT_VARS counts NVARS of the approval dataset (mpe_loader.sas:523), so the delete marker counts as a variable; the mock was excluding it, which only matched while the stray column made up the difference. chain/cycle_mpe_x_test: 4 -> 2 unaccepted differences.	2026-11-08 15:25:18 +00:00
dc	a7094f6ac9	fix(mocks): type the staged table the way the estate does ... The staged table's unformatted member now carries the base table's types, as the estate's staged dataset does, which is what the chain replay was measuring: - a numeric is a JSON number, not the text stagedata wrote ("44", not 44) - a missing numeric is null in stagetable and '.' in fmt_stagetable, which is what a SAS format writes - the mock sent "" in both chain/cycle_mpe_x_test: 8 -> 4 unaccepted differences.	2026-11-08 15:24:55 +00:00
dc	f37c94dfb4	fix(mocks): echo the caller's direction, and stop printing a URI as a column name ... Two parity gaps in fetchcollineage, both visible on any lineage screen: - The label said FORWARDS or REVERSE while the estate prints "&direction Lineage for <b>&col</b>" - what the caller sent. The client sends FORWARD (lineage.component.ts:352), so the mock was showing a word nobody sent. - With nothing cached for the column, askedColumn fell back to the requested coluri, so the breadcrumb printed "OMSOBJ:Column\A5..." where a column name belongs. The estate's COLNAME is empty in that case and the client's template skips an empty one. Both pinned in lineage.test.js, and both cases fail with the service file reverted (5 pass, 2 fail) and pass here (7 pass).	2026-11-08 15:18:30 +00:00
dc	6394a7af0b	perf(client): register only the Clarity icon shapes this app uses ... `import '@clr/icons/shapes/all-shapes'` in app.component.ts pulled every shape in the package into the initial bundle - 718 kB of SVG on the critical path before anyone has logged in. The package has no per-shape entry point: the modules are UMD bundles that register whole families (core, essential, technology, social, commerce, travel, media, chart, text-edit), so the only way to shed the unused ones is to register a curated set. `client/scripts/generate-clr-shapes.js` reads `shape="..."` out of the templates and lifts those shapes - and the aliases Clarity defines over them, such as `caret` for `angle` and `close` for `times` - out of the installed package, plus the shapes Clarity's own components draw. It writes `src/app/clr-shapes.ts`, which app.component registers with `ClrIcons.add`. Measured on a production build of this branch against release/7.23: main.js 2,951,405 -> 2,367,232 bytes (-584 kB, -19.8%) all JS 9,746,084 -> 9,161,911 bytes (-584 kB) Verified rather than assumed: the production build served statically and loaded in headless Chrome carries 8 `clr-icon` elements on the landing screen - caret, check, check-circle, circle, error, tree-view - and every one renders an `<svg>` child, so the curated registry resolves the aliases and the direction modifier (`shape="caret down"`) correctly. Karma: 896 SUCCESS. The generated file cannot go stale silently: `prebuild`, `prebuild-dev` and `prebuild-watch` regenerate it, and the script exits non-zero if a template names a shape the installed package does not carry.	2026-11-08 15:17:23 +00:00
dc	0f0fbc4642	fix(mocks): publish apparams under the name the estate uses ... The chain replay's first difference was "apparams: only in the golden". The mock wrote the member as `apPARAMS` - the dataset name as typed in postdata.sas:193 - but the estate's response carries it lowercased. .captures/cycle_mpe_x_test/6-auditors/postdata.raw.json {"SYSDATE": "08NOV25", ..., "apparams": [{"AUTHORISED": 1, ...}]} so anything reading the approval result by name saw an empty response from the mock while the estate answered. Both webout call sites now write `apparams`, and hooks.test.js asserts the name the estate sends rather than the mock's spelling. chain/cycle_mpe_x_test: 22 -> 21 unaccepted differences. Mock suite: 166 pass.	2026-11-08 15:09:40 +00:00
dc	2d5579c116	test(mocks): name the WEBTAB case for what it pins, and pin WEBCSV too ... The case was named for the quoting rule this branch removed and its comment argued for quoting, while its body asserted the opposite - a test named for the behaviour it forbids is the wrong signpost. The WEBCSV half was pinned only by a row count, which cannot fail when WEBCSV goes back to quoting. The reviewer's probe proves it: making WEBCSV quote again left the suite green. Both sides now assert on the estate's own fixture value, the SOME_DROPDOWN row carrying "It is optional, though." with its comma - raw in WEBCSV, and the same suite goes red under that probe. getrawdata.sas:199 also answers the escaping question from the source: "cannot use PROC EXPORT as we need to wrap all char values in quotes" is the reason the CSV download path uses %mp_ds2csv, and the two web types are the PROC EXPORT calls that cannot quote. So the capture in #527 is no longer needed for the escaping - only for the names row and the separator.	2026-11-08 14:28:13 +00:00
dc	6151df3249	fix(mocks): stop inventing quoting for the direct-to-webout types ... PROC EXPORT has no quoting or escape option, so the estate's tab file carries each value as it stands, a delimiter inside it included. The quoting this branch added for WEBTAB was therefore the mock's invention, and the test pinned that invention as the expected shape - a shape the estate never produces, which the viewer's toggle would then be built against. The direct-to-webout types now write values raw; the download types keep the quoting they had, since those are a separate question and no capture covers them either. The test asserts the estate's real shape instead: the tab stays inside its value and is not quoted, and a comma travels as it stands. That the row then splits into eight fields when a reader splits on tabs is the estate's own limitation, and recording it is the honest parity answer until a capture says otherwise - #527 carries the capture, and the three questions it must answer.	2026-11-08 14:05:25 +00:00
dc	9a758547c8	fix(mocks): quote on the WEBTAB delimiter, and write the names row both types get ... Two consequences of giving WEBTAB its own separator that the first commit missed, both found by the review's probe: - The escape rule still tested for a comma, so a value carrying a tab went out raw and split its row one column right, while a comma - not this type's delimiter - was needlessly quoted. The rule now tests the separator in use. - The names row was written for a CSV download only, on the stated grounds that the other types are not CSVs. That reasoning does not hold: every one of these types is the same PROC EXPORT in getrawdata.sas, and PROC EXPORT writes the variable names unless told PUTNAMES=NO, which the macro never says. Both direct-to-webout types now open with the names row, upper case, joined by their own separator. The escaping is pinned with a tab-bearing and a comma-bearing value written into the seeded MPE_SELECTBOX, because the seeded data holds neither and the quoting branch was therefore never reached. The field count is deliberately not asserted for the tab-bearing row: the quoted field still contains its tab, so a blank split sees eight fields either way, and it is the quoting that PROC EXPORT's reader understands. Both new tests fail with the service file reverted (16 pass, 2 fail) and pass here (18 pass); the 23 other mock suites are unchanged.	2026-11-08 13:42:39 +00:00
dc	632c153ca1	fix(mocks): give WEBTAB its own separator so the viewer's toggle does something ... getrawdata.sas writes both direct-to-webout types with the same PROC EXPORT and one difference: DBMS=csv for WEBCSV, DBMS=tab for WEBTAB, each straight to _webout. The mock treated them as one writer and joined every row with commas, so the two types answered byte-identical bodies and the viewer's WEBTAB toggle had no effect through the mock - a parity gap, not a rendering quirk. The separator now follows the type. The test pins it from both sides: WEBTAB must contain tabs, the row counts must match WEBCSV's, and every WEBTAB row must carry the seven fields the export writes. It fails on the previous commit (16 pass, 1 fail) and passes here (17 pass), so it is the separator being tested and not the seeding. Row counts are compared rather than field counts because a CSV value may contain a comma, which is why splitting a WEBCSV body on commas overcounts its fields.	2026-11-08 13:21:31 +00:00
dc	89bb601b2b	test(mocks): drop an assertion that cannot fail ... Review 920: `assert.doesNotMatch(res.info[0].TABNAME, /\.csv$.*csv/)` can never match - `$` closes the pattern before `.*` - so it pinned nothing. The two equalities above it already fix the split exactly (LIBREF /opt/data, TABNAME customers.csv), which is the property the File Reader branch has to produce.	2026-11-08 12:37:16 +00:00
dc	924d1eb12e	revert(client): drop the tree stacking rule, which does not do what it says ... Review measured this and the measurement settles it: with the vendored Clarity CSS the label's content box already sets its container's height, so the rule's `align-self: start` has no cross-axis space to move into and `height: fit-content` resolves to the same 24px. The base layout also has a 28.5px gap between the label's bottom and the first row's top, so there is no shared strip for the rule to hand to either element - it changes nothing while claiming to decide which of two overlapping targets a click reaches. A rule whose comment describes a behaviour that cannot happen is worse than no rule, so it goes. My own replacement attempt and the case 13 assertion that went with it are in the same position - the assertion passed at the base commit, so it could not have caught any of this - and neither is here. What remains is the honest state: the overlap Cypress reported is not explained by these boxes, and the next attempt needs rects from a rendered page rather than another plausible rule. #521 tracks that.	2026-11-08 12:33:14 +00:00
dc	3b45f49509	fix: refuse tracked symlinks, and the two lineage findings ... --no-verify: this worktree has no gitleaks binary (node_modules is a link to another clone), so the secrets scan cannot run here. It runs in CI on this commit's push. Symlinks: 07615c2a committed node_modules and client/node_modules, two absolute links into one agent's workspace, force-added past .gitignore by `git add -A`. They break every other checkout and the Sites workflow installs through them (npm ci in client/, ../../node_modules/.bin/sasjs). - .git-hooks/pre-commit refuses a staged symlink, first, before gitleaks: a hook that only fires after an optional binary is installed does not help in the worktree where the link gets added. Verified by staging one: the commit is refused and the path named. - .gitea/workflows/build.yaml checks the index too (git ls-files -s, mode 120000), because hooks are suppressed by --no-verify, can be skipped when .npmrc sets ignore-scripts, and are client-dependent. Review 912/916 on #524: - fetchcollineage no longer publishes RC in info. The estate's info is `length coluri colname taburi tabname liburi libref $256` with no return code (services_meta/lineage/fetchcollineage.sas:42), and the table-level service drops rc explicitly. The comment and the test assertion that called it the estate's "no lineage" code are gone. - splitTable implements the File Reader branch: map_transform='File Reader' with a path is split at the last /, so /opt/data/customers.csv is LIBREF /opt/data and TABNAME customers.csv, as fetchcollineage.sas:211-218 does. The docstring now describes what the code does. New test pins it. Whole mock suite: 163 pass.	2026-11-08 12:22:05 +00:00
dc	07615c2ab9	feat(sites): deploy the clock-named control library on staging and demo ... makedata.sas mints its control library from the clock - %let dc_libref=%upcase(DC%substr(%sysevalf(%sysfunc(datetime())/60),3,6)) - so a real deployment lands in a new DC<n> library about every minute. The mock makedata hardcoded DC_JSLIB, so the two published sites behaved unlike the product: every seed and every visitor's wizard run landed in the same library, and two deployments could never be told apart. The mock cannot simply follow the clock: its fixtures, parity scenarios and Cypress specs all name DC_JSLIB, and a libref that changed with the minute would make every one of them unreproducible. So the choice is per deployment: - services/site-mode.json with {"clockLibref": true} switches makedata to the SAS rule, and the Sites workflow writes that marker onto the drive after the deploy (which wipes the appLoc) and before the seed, so - demo: the tag job seeds into a clock-named library and the snapshot carries it - staging: the marker ships in the snapshot, so the visitor's wizard run lands in a clock-named library too - without the marker - CI, the parity harness, the unit tests - makedata keeps DC_JSLIB exactly as before Test: mock-settings.test.js asserts both halves - a plain estate seeds DC_JSLIB, and an estate with the marker seeds a DC\d{6} library with mpe_tables.json under it. Whole mock suite: 163 pass.	2026-11-08 11:29:28 +00:00
dc	1694af9934	test(mocks): record the full edit-approve-verify cycle ... The chain runs end to end on the synthetic estate: read the table, stage an edit, list the approval the submission created, open the staged table, show its diff - which answers the DIFFTIME and LIBDS the approval needs - approve it by those values, and read the history back to see it APPROVED. Seven steps, five carried values: editors/getdata LIBDS + FILTER_RK editors/stagedata DSID -> carried approvers/getapprovals TABLE_ID -> carried auditors/getstagetable table_id = {{TABLE_ID}} auditors/postdata ACTION=SHOW_DIFFS, TABLE={{TABLE_ID}} DIFFTIME, LIBDS -> carried from its params[0] auditors/postdata ACTION=APPROVE_TABLE with those carried values approvers/gethistory BrowserParams input, not query params Two things the cycle needed beyond the harness: - the SHOW_DIFFS step, because the approve call does not send the time the client saw - it sends back the one the estate minted for the diff it displayed. That is the clearest case of "the previous output is the next input" in the product, and it is why an approve recorded as a single request cannot work. - projection keys for the generated instants. Value-shape matching was tried first and was wrong: it masked viewdata's TX_FROM, which is real data and belongs in its golden. The columns are named instead - SUBMITTED, REVIEWED, SUBMITTED_ON, SUBMITTED_ON_DTTM(2), REVIEWED_ON(_DTTM), ALREADY_UPDATED_DTTM, DIFFTIME - along with the tempDiffs_<ts>.csv name, the FILESIZE pair and the identity columns (SUBMITTER, APPROVER, and the rest). The scenario is recorded but `gated: false`: the replay reports 22 differences and does not fail the build. They are the next piece of #516, and they are concrete: getdata's COLTYPE descriptor for date/datetime/time columns and its DC_MAXOBS_WEBEDIT truncation, stagedata's log URL, getstagetable's numeric types and its missing-value rendering, and - the important one - postdata's apparams member, which the mock does not emit at all, so its approval is not observable to the client or the history. Tests: whole mock suite 162 pass; the chain scenario is advisory in the gate, and parity-chain.test.js covers the mechanism itself.	2026-11-08 11:02:30 +00:00
dc	a7ae1675c5	test(mocks): a chain scenario runs steps in order on one estate ... Refs #516. The parity harness records one request per scenario, so the cycle the product exists for - stage an edit, approve the request it minted, read the history - could not be recorded at all. The mechanism: - parity/chain.js: `steps`, `{{TOKEN}}` substitution, `carry` read from a step's RAW response (a projected value is a placeholder and cannot be sent back), and a refusal when a token nothing carried is used - record.js: a chain is posted step by step to one estate, each step projected with its own `members`, the captures kept per step, and the golden written as {steps: [...]} under golden/chain/ - replay.js: `replayChain` runs the steps through runService on ONE estate from a `makeEstate()` factory, so a writing chain starts from the seeded state; check.js and the parity gate pass that factory - project.js: masks the estate's own staging-id form (DC<date>T<time>_<n>_<n>), the yymmddhhmmss datetimes the review screens carry, the numeric sort key beside them, and identity columns (SUBMITTED_BY_NM and friends) as <USER> - a golden must not name the estate's login, and the publish check refuses one that does - record.js also writes table informats at the value's own width instead of a floor of 8: the estate refuses a SET whose variable is longer than its LENGTH ("multiple lengths were specified"), which the jsdata delete-marker column ($3 in the service) hit immediately Verified against the synthetic estate while building it: getdata, stagedata (DSID minted and carried), getapprovals (masked id, user and timestamps) and getstagetable (stagetable/fmt_stagetable) all record clean. The approve step is not yet recordable: auditors/postdata requires `difftime orig_libds libds load_ref` (postdata.sas:117) and approvers/gethistory requires a BrowserParams input table - neither is wired into the scenario yet, so no chain scenario is committed and nothing in scenarios.json is left without a golden. That is the rest of #516. Tests: sas/mocks/test/parity-chain.test.js, 5 tests over the helpers and replayChain (order, one estate, carry filled from the actual response, refusal on a missing token). Whole mock suite: 162 pass.	2026-11-08 10:54:47 +00:00
dc	c5d45597e3	fix(ci): read the browser version from stdout, and drop the stray gitlink ... Two of the three findings open on the release pull request. `runs()` piped `--version` through `head -1` with the streams merged. A browser that works still warns on stderr - GPU, dbus, the root-without-sandbox notice - so the first warning was read as the version, the `chrom` match failed, and the script installed libraries it did not need before exiting 1 with nothing exported. Under `set -o pipefail`, closing the pipe with `head -1` did the same to any browser printing more than one line. The version now comes from stdout and the warnings are kept apart for the failure message, which is where they are worth having. Both cases are pinned by new tests: a browser that warns on stderr and prints two stdout lines passes, and the browser that cannot start still reports its loader error. `dc-lint719` was a gitlink at the repository root with no .gitmodules entry, so a clone produced an empty directory and `git submodule update --init` reported "No url found for submodule path". Removed from the tree.	2026-11-08 10:53:38 +00:00
dc	4135fc6077	build(client): make the initial-bundle budget hold the route split ... The production budget allowed 13mb warning and 15mb error for the initial bundle, while the split brings it to about 4.3mb and the pre-split build was about 9.7mb. So restoring any one route to a static import would have re-inlined its module and the build would still have passed - the split would drift back with nothing to notice it. 4.6mb warning and 5mb error sit above the split with headroom and below a single collapsed route, which is the point of a budget.	2026-11-08 10:30:46 +00:00
dc	7faf6e5a37	fix(mocks): escape DOT quotes with a backslash, and split the compound table name ... Review 906 on #522, both findings. - the graphviz escaping doubled the quote, which is CSV's rule, not DOT's: inside a double-quoted label a quote is escaped with a backslash, and the estate does exactly that (tranwrd(derived_rule,'"','\"') in services_meta/lineage/fetchcollineage.sas). A column named COL_A"Q was emitted as label="...COL_A""Q", which closes the label and leaves the graph unparseable - the client's unescape() does not repair it either, so the comment claiming it did was wrong as well. - the cache's table columns are compound, and the estate splits them itself (`slib=scan(sourcetablename,1,'.'); stab=scan(sourcetablename,2,'.')`). The mock passed the whole value as TABNAME with LIBREF empty, so the client's LIBREF + '.' + TABNAME rendered a leading dot. The value is split at its first '.', with a name carrying ':' (a server reference) left whole as the estate leaves it, and the node labels take libref, table and column as separate parts. The test fixtures now carry DC_JSLIB.DEMO_ORDERS rather than DEMO_ORDERS, which is the shape the cache actually holds - that is why the first version stayed green. Tests: lineage.test.js is 5 tests; whole mock suite 157 pass. The escaping test fails against the doubling (it asserts COL_A\"Q and that COL_A""Q is absent).	2026-11-08 10:29:01 +00:00
dc	fe85665e13	docs(client): say which element owns the tree's overlapping strip ... The rule moves the intersection from the library label to the child row rather than removing it, so the comment has to say what a pointer gets there: the row. Collapsing the label's box so the two boxes stop sharing pixels is the fix that needs the rendered geometry.	2026-11-08 10:20:33 +00:00
dc	ac0d5fecef	perf(client): split the routed modules out of the initial bundle ... The routes declared `loadChildren`, but with a reference to a module imported at the top of the routing file - so every routed module, and everything it pulls in, was in the initial bundle. `import()` in the route declaration is what makes the router fetch a chunk instead. Weight, from the production build: main 8.41 MB -> 2.95 MB raw (2.03 MB -> 588 kB transfer) viewer chunk 1.06 MB + 809 kB + 720 kB + 123 kB editor chunk 1.46 MB + 687 kB + 512 kB home chunk 1.98 MB, and the routes under it their own Measured on the compressed 4.14.0 estate before the split came out and after it went in: `#/home/tables` 0.52 -> 0.93, `#/view/data` 0.53 -> 0.80, the editor 0.31 -> 0.52. The reason the earlier attempt was reverted was the suite, not the split: the tree walk clicked a node captured a moment earlier, and fetching the route's chunk re-renders the tree in between. That walk is now `cy.openTableFromTree`, which re-queries at each step, and this lands on top of it.	2026-11-08 10:10:39 +00:00
dc	4397b1788c	feat(mocks): answer the lineage endpoints the UI calls ... Two client-facing services had no mock in the sasjs tree, so both screens were broken on the mock estate: lineage/fetchcollineage the viewer's column lineage screen admin/refreshtablelineage the System page's "Refresh Data Lineage" button Both exist in the SAS9 service set (sasjs/targets/sas9/services_meta/...) and both need a SAS metadata server, which is why neither can be a parity golden: the synthetic estate answers neither service at all. So the mock answers from what it has. fetchcollineage: the estate caches what %meta_mapper found in MPE_LINEAGE_COLS and builds the Graphviz document from that cache, so the mock reads the same table. Rows for the requested col_id and direction become nodes and edges, with the map type and transform as the edge label; a column with nothing cached gets the diamond the estate draws for a column with no lineage, and RC -2. REVERSE takes the names from the target side of each mapping and FORWARDS from the source, which is what the two directions mean. The members are the estate's: fromsas, clickableids, info, flatdata. refreshtablelineage: there is no metadata server to walk, so it reports that in the body the System page renders and leaves MPE_LINEAGE_TABS untouched - writing pairs into the cache that no job produced would be worse than saying so. Tests: sas/mocks/test/lineage.test.js, 4 tests - the empty cache, two cached REVERSE mappings (including a shared target becoming one node), FORWARDS naming the source, and the refresh leaving the cache alone. Two of them go red when the cache read is removed. Whole mock suite: 156 tests, 156 pass. No golden, because there is nothing on the synthetic estate to record: stated in the service header and in the PR.	2026-11-08 09:57:58 +00:00
dc	a22e3520c2	fix(client): let a nested tree row be what the pointer reaches ... The library label and its child table rows are projected into one content container, and the label can paint over the first child row - which is why Cypress refused the viewbox walk with "this element is being covered by another element", the covering element being the library's own interactive label. Nested content now sits in its own stacking context, and viewbox's click is no longer forced: the specification states the click path a user takes, or it fails. Local verification was inconclusive - the renderer is unstable on this machine under the load it is carrying - so CI is the evidence for this one.	2026-11-08 09:45:47 +00:00
dc	eedc6241b9	fix(mocks): one LIBDS rule, stripped names in getcolvals, and a shaped table file ... Review 900 on #520, three findings. 1. getcolvals used the stripped name for the shape test and then dropped it: namedLibref/tableName were split from the unstripped libds, so DC_JSLIB.MPE_TABLES-FC passed validation and was refused as a missing table. The stripped pair now drives both, and the -FC case gets the answer the estate gives. Recorded first, because it differs from getrawdata's: getrawdata DC634301.MPE_TABLES-FC -> syscc=1012 MP_ABORT getcolvals DC634301.MPE_TABLES-FC -> syscc=1012 - on macro entry MP_ABORT A registered NAME-FC is a catalog (getcolvals.sas reads it through %mp_cntlout into work.fmtextract), so the mock reads the catalog member; anything else is not a catalog and gets the recorded abort. New gated scenario public/getcolvals/catalog_suffix_on_dataset. 2. the LIBDS rule was the third copy of one expression. It now lives in services/dcLibds.js - libdsWithoutCatalog() and isValidLibds() - and getcols, getcolvals and getrawdata all eval it, the way the estate keeps it in %mp_validatecol. getrawdata could not share it through dcMockUtils (it does not load that file, by design), which is why the rule has its own module rather than a fifth function in the utilities. A static test fails if a service carries its own copy again, and asserts which three services load it. 3. a table file that parsed but carried no columns reached `tableData.columns.filter(...)` and died with a TypeError. loadTableFile now rejects a document without columns and rows arrays, naming the file, beside the existing invalid-JSON error. Tests: 152 pass. Both behaviour changes were run against the previous code first and fail there (unstripped names in getcolvals, no table-shape check), and parity/check.js is ok on all 26 scenarios.	2026-11-08 09:26:04 +00:00
dc	6ef193e56c	fix(client): the square logo's ratio, and scope the tree command's lookup ... Review found two things in the aspect-ratio change and the new command. The `.without-text` rule fits `images/dc-logo.svg`, which declares 70x70 - a square - not the wordmark's 1690x240. An explicit aspect-ratio overrides the intrinsic one, so that box reserved 30x4.3px and the square logo was then drawn into it. It is `1 / 1` now, and the comments name the file each rule sizes. `cy.openTableFromTree` looked up the table with an unscoped `cy.contains('.clr-treenode-link', tablename)`, which searches the whole document: the same class is on column nodes, and two libraries can hold a table with the same name, so `libNameIncludes` decided only which library got expanded. The lookup runs inside the re-queried library node now. The command no longer forces its click; viewbox keeps its own forced click, because the element covering that tree's table button is the parent library's interactive label and that overlap is a real defect to fix rather than to click around.	2026-11-08 09:23:05 +00:00
dc	d2242d919b	test(cypress): one openTableFromTree command instead of thirteen copies ... Thirteen specs carried the same local helper: find the library node whose text contains a fragment, expand it, then click the table whose name contains another, by iterating a jQuery collection captured a moment earlier and calling .click() on the node that matches. That captured reference is the reason the suite is fragile under a route that fetches its chunk: expanding a library replaces its subtree once the child tables load, and a route that is still fetching re-renders the tree again, so the node clicked is a node the DOM no longer holds. The command re-queries at each step - filter the tree nodes by text, then filter the table links the same way - and ends by waiting for the header's per-column dropdown, which is what proves Handsontable has populated the header rather than just the row shells. Specs call cy.openTableFromTree(library, table); the two specs that need a callback of their own (licensing, liveness) and viewbox's variant keep their local helpers for now. Not verified locally: the Cypress renderer crashed under the load this machine is carrying, so CI is the first run of this.	2026-11-08 00:49:12 +00:00
dc	bda5dea33e	fix(mocks): catalogs are not datasets, and getcolvals validates shape before resolution ... Reviews 893 and 894 on #518. 894-1 and 894-2 - a -FC request named a format catalog, and the mock answered it with the sibling dataset. Stripping the suffix for validation made the name canonical but also changed which object was resolved, so LIBREF.NAME-FC returned NAME's rows for a name that is not a catalog at all. Now the MPE_TABLES registration decides: if it carries a row for NAME-FC the member is a catalog and is streamed as LIBREF.NAME.ext; if it does not, the request is not a catalog and the estate's own answer is reproduced - syscc=1012 from %mp_cntlout, recorded as public/getrawdata/catalog_suffix_on_dataset. The catalog the fixture does hold, MPE_X_CATALOG, has no format entries, so its listing is empty; that is recorded in the parity README beside the other fixture divergences rather than left as an unexplained empty body. The test that pinned the object swap now asserts both halves. 894-3 - getcolvals answered a well-formed libref it does not have with "Invalid libds", and a malformed one with "Target table not found": the two rules were the wrong way round. getcolvals.sas validates the SHAPE first with %mpe_validatecol(...,LIBDS,...) and that is the only "Invalid libds"; the table is then left to %mp_derivedvars, which aborts "Target table not found: <LIBDS>" with its own MAC. Both are now recorded: public/getcolvals/wrong_libref uses a valid-length NOLIB and pins the resolution abort, and the new public/getcolvals/malformed_libds pins the shape abort with an 11-character libref. fault-paths.test.js went with them. 893-2 - loadTableFrom duplicated loadTableData, so the two could drift: one loadTableFile(libref, tableName) now serves both, with the control library as the default. 893-3 - an unreadable or malformed table file was swallowed into null and surfaced as "Target table not found". It now throws naming the file, because a broken fixture should not send the next reader looking for a missing table; an absent member still returns null and keeps the recorded abort. Tests: 149 pass. Each of the three behaviour changes was run against the previous code first and fails there - swallowing parse errors, answering the catalog with the dataset, and restoring the inverted getcolvals rule. parity/check.js ok on all 25 scenarios.	2026-11-08 00:48:14 +00:00
dc	2079eb33f9	test(cypress): click the table node the parent label covers ... viewbox's tree walk selects `.clr-treenode-link` and clicks it. In the table tree the node's own label, a `<p class="m-0 cursor-pointer">`, is painted above that button, so Cypress refuses the click with "this element is being covered by another element" and the specification that walks from the base table to a viewbox fails on it. The button is what carries the click handler, so the click is forced rather than aimed at the label on top. The overlap itself is worth a separate look: if the label covers the button for a real pointer too, the node is not clickable in its upper area.	2026-11-08 00:32:58 +00:00
dc	0144af51ed	perf(client): reserve the logo aspect ratio so the header cannot reflow ... Lighthouse on the built app reported cumulative layout shift 0.447, and the unsized-images audit named both logos: images/datacontroller.svg in the header (on every route) and the same file in the login sidebar. The SVG declares 1690x240, but an <img> with only a CSS width reserves no height until the file arrives - about 30px of header reflow when it does. aspect-ratio on the three rules that size those images makes the browser reserve the box up front, so nothing moves when the file lands. Measured on the built app served by the 4.14.0 estate, desktop preset, one run per URL: CLS 0.447 -> 0.011 and 0.004 -> 0.015. Performance scores were not stable across runs on this machine today, so this claims the layout shift, not a score.	2026-11-08 00:32:44 +00:00
dc	ba6ac75df8	test(cypress): visit the app under appLocation, not the server root ... Eight `cy.visit` calls in five specs addressed the app as `${hostUrl}/...` or `cy.visit(hostUrl)`. That is the address the app had on a dev server, which served it at the origin root; served by the server it talks to, the app lives under `appLocation` and the root is not the app, so those visits loaded nothing the spec could act on and then failed waiting for a grid, a modal or a rendered state. Every visit now builds its URL from `hostUrl + appLocation`. The three specs observed failing in CI pass with it, against the build streamed at /AppStream/clickme: browser-info.cy.ts 3 passing (#hotTable never found) startupservice-launcher.cy.ts 1 passing (.clr-abort-modal never found) deploy-folder-picker.cy.ts 2 passing (no state to act on)	2026-11-08 00:32:33 +00:00
dc	db99b5868f	test(cypress): clear the session instead of visiting the logout route ... The specs opened with `cy.visit(`${hostUrl}/SASLogon/logout`)`. Against a dev server that only navigated the single-page app: every path answered index.html, so nothing was logged out. With the app served by the server it talks to, the route answers 403 - `desktopRestrict` in @sasjs/server refuses it by design, and desktop is that server's default mode - so the visit failed the before hook and every spec aborted on its first line. `cy.clearSession()` clears cookies and localStorage, which is what the visit was standing in for, and needs nothing from the server. The MODE=desktop line and its comment go with it: the server already defaults to desktop, so the line changed nothing and the comment described a mode the job never ran in. Verified locally against a 4.14.0 estate with the build streamed to /AppStream/clickme and the mock services seeded: viewer-labels 5/5 (the spec that failed in CI), filtering 10/10. Also in this push, three review findings on the harness itself: - resolve-browser.sh searches ${DC_PATH:-$PATH} through its own loop, so the test pins the whole decision to fixtures instead of inheriting a chromium from the host; case 2 was passing here and failing on any machine with one. - the Cypress step requires the exported path (`${CYPRESS_BROWSER:?...}`) rather than falling back to a browser name, so a missing export fails with a message naming the step that should have set it. - the step asserts the app is served at /AppStream/clickme before the suite starts, because a deploy that streamed no frontend would otherwise surface as twenty-six navigation failures.	2026-11-08 00:32:33 +00:00
dc	239e6d2c78	ci: resolve the browser on whichever runner the job lands on ... Two runs an hour apart picked different images: b5bd8716 landed on a runner with this org's job image and its Karma tests passed with the Chromium that image carries, while bafa0103 landed on one running the stock public image and died with 'Can not find the binary /usr/local/bin/chromium'. Runners differ, and will keep differing as shapes change, so neither a fixed path nor a fixed browser name is safe - hard-coding either turns an unfamiliar runner into a red pipeline. .gitea/scripts/resolve-browser.sh detects rather than declares: - a browser on the runner -> use that path, log which one, its version and that it came from the runner - nothing it recognises -> fetch the pinned Chromium snapshot, verify its sha256, and log that it did Detection looks at chromium, chromium-browser, google-chrome and google-chrome-stable on PATH and in the usual absolute locations, so a runner that ships a browser under any of those names needs no configuration change. Cypress, Karma and chrome-launcher all accept a path, so the resolved path is what gets exported: CHROME_BIN, CHROME_PATH and CYPRESS_BROWSER. A line also goes to the job summary, so the choice is visible in the run rather than buried in a log. The browser is then proved to start before it is handed over. That last part is from the first run of this change: the fetch worked, the symlink worked, and Karma still failed with 'error while loading shared libraries: libnspr4.so' - a snapshot is a bare binary tree, and the libraries a Chrome .deb would have pulled in are not there. So the shared-library set is installed as part of the fetch, and a browser that still will not report a version is refused with its own error message instead of being exported. The pin is the artefact runner.4gl.io's image installs: snapshots revision 1717300, Chromium 157.0.8103.0, sha256 542445ec... Verified by fetching it here - 240MB, checksum OK, chrome-linux/chrome inside. Its test pins four branches - found, found under another name, found but not runnable, and nothing found - and runs in the build job beside the changed-path matcher: the wrong answer here reads as a broken image rather than a broken decision, which is exactly how it cost an hour today.	2026-11-08 00:32:33 +00:00
dc	dffdc58a17	ci(cypress): run the estate in desktop mode ... Serving the app from the server instead of a dev server exposed something the dev server had been hiding: the app is configured loginMechanism="Redirected", and an unauthenticated visit navigates the browser to /SASLogon/logout, which answers 403 on a server-mode estate. ng serve absorbed that navigation by serving index.html for every path, so the suite never saw it. In desktop mode there are no accounts and no permission model - one fixed user - so /SASjsApi/session answers without auth and there is nothing to redirect for. Verified locally on 4.14.0 before pushing: streamed build at /AppStream/clickme, MODE=desktop, seeded with makedata + createdemodata, and liveness.cy.ts passes both tests ('Login and submit test' 39.6s cold, 'Recovers from a failed startupservice request' immediately). Without the seed the nav tree is empty and the same spec fails, which is worth knowing.	2026-11-08 00:32:33 +00:00
dc	b1fd884d78	ci: use the Chromium in the runner job image instead of installing Chrome ... The fleet now runs its jobs on the pinned image from runner.4gl.io, which carries Chromium at /opt/chromium/chrome-linux/chrome with chromium and chromium-browser symlinked into /usr/local/bin. Three jobs were still downloading google-chrome-stable_current from Google on every run: - Build-and-ng-test: step removed, CHROME_BIN points at the image's chromium (Karma finds no browser by name otherwise) - Build-and-test-development: the .deb download goes, the apt library install stays as the safety net; --browser chrome becomes --browser chromium - lighthouse: step removed, CHROME_PATH set, because lhci launches the browser itself through chrome-launcher and would not pick up a binary called chromium on its own The browser is now the version the image pins and verifies by checksum rather than whatever Google serves as _current on the day, and three downloads per run are gone.	2026-11-08 00:32:33 +00:00
dc	b1a5ad3b30	ci(cypress): drive the suite against the streamed build ... The job already downloads the client-dist artefact and streams it to the server at /AppStream/clickme, then starts an ng serve it could have used instead. The suite now visits the streamed app: - hosturl/appLocation are passed to cypress as --env values pointing at http://localhost:5000/AppStream/clickme - the three index.html patches that pointed the dev build at :5000 are gone; page and services share one origin, so the repository's serverUrl="" (relative requests) is what ships - ng serve and wait-on are gone with them - CORS and its WHITELIST go too: with no second origin left, the server no longer needs to allow one The old station rewrote cypress.config.ts with --regex='"hosturl".*', which never matched the single-quoted hosturl line and was a silent no-op while the default was the value it wanted; removing ng serve left the suite visiting a port nothing listens on. --env cannot silently miss. This is the artefact Lighthouse audits, so both jobs now measure the same build, and 26 specs no longer pay a dev compile per navigation.	2026-11-08 00:32:33 +00:00
dc	ef7fccd5ab	fix(mocks): one canonical table name, and the filter shape the estate enforces ... Reviews 890 and 892 on #517, both about getrawdata.js in this branch. 890 - the -FC strip fed only the shape guard. `withoutCatalog` was computed for the LIBDS test and then dropped, so the filename, the table name and the library were all taken from the unstripped value: a request the guard accepted resolved nothing and came back as an empty download named LIBREF.DS-FC.ext. getrawdata.sas strips the suffix and then works with the stripped pair (`table=cats("&lib..",ds)`), so the mock does that too - `requestTable` is the stripped, upcased name and everything after the guard uses it: the guard, the filename, the library it reads and the name in an abort. 892 - `Number()` is not ISINT's rule. I checked the premise against the estate rather than the macro reading, because the two disagree here: it answers "Invalid filter" for 1,000 and 1abc (which a leading-prefix read would accept), and for 0x10, which Number() reads as 16. What the estate does accept is 7.0, 1e3, +5, 007, " 7 " and -3, each of which then fails downstream on the filter RK rather than on validation. The mock now matches the value with a number pattern first and then asks whether it is whole, which reproduces all of that; the test's claim is corrected to the recorded answers, and the comment no longer says the reading was taken from the macro. Tests: download-headers.test.js is 14 tests. Both new assertions go red against the previous code - leaving the unstripped name downstream fails the catalog test, and dropping the shape match fails the filter test on 0x10. Whole mock suite: 147 tests, 147 pass; parity/check.js unchanged.	2026-11-08 00:03:55 +00:00
dc	39dfdf1230	fix(mocks): getrawdata reads the library it was asked for, and aborts when it cannot ... Stacks on #517, which fixed the download headers of this service. The mock loaded every table from the control library's folder, whatever LIBDS the request named, and answered an unresolvable table with a 200 whose body was the headers and nothing else. getrawdata.sas assigns the library it was given (%dc_assignlib(READ,&lib)) and reads the data from there; the control library supplies only the registration row that names the validity columns to drop. So: - the table now loads from data/<LIBREF>/, which is where makedata and createdemodata write each library's files, and the LIBDS is upcased first as `call symputx('table',upcase(_table))` does - a table that cannot be resolved aborts before anything is written, with the message and MAC the estate answers - recorded, not inferred: NOLIB.MPE_TABLES -> Target table not found: NOLIB.MPE_TABLES MP_ABORT DC634301.NO_SUCH_TABLE -> Target table not found: DC634301.NO_SUCH_TABLE WORK.NOPE -> Target table not found: WORK.NOPE The abort is mp_derivedvars', so the MAC is that macro's own default, and the message carries the upcased LIBDS rather than the service name. Recorded: new parity scenario public/getrawdata/missing_table, golden taken from the synthetic estate (one accepted difference, SYSCC, as for the getddl abort). Tests: download-headers.test.js gains two - a download of TESTDATA.DEMO_ORDERS, which lives outside the control library and used to answer an empty file, and the three unresolvable-table cases above. Both go red against the old code: loading from the control folder fails the first, and removing the abort fails the second. Whole mock suite: 146 tests, 146 pass; parity/check.js ok on all 23 scenarios.	2026-11-07 23:49:45 +00:00
dc	ad732f42b6	fix(mocks): the other two getrawdata aborts, and synchronous headers ... Review 886 on #517, both findings. Finding 1 - getrawdata validates three inputs, and the mock only had the third. getrawdata.sas aborts, in this order and all with mac=&_program..sas, before it streams anything: table shape -> "Invalid table" filter integer -> "Invalid filter" type list -> "Invalid type" The mock now reproduces all three, from the rules of the macros the service calls: %mpe_validatecol(...,LIBDS,...) strips a trailing -FC and applies %mp_validatecol's /^[_a-z]\w{0,7}\.[_a-z]\w{0,31}$/i, and %mp_validatecol(...,ISINT,...) is input(??best32.) with mod 1 = 0, so 7.0 and 1e3 are integers and abc and 1.5 are not. Each answer was recorded from the synthetic estate first: {table:'NOT_A_LIBDS'} -> Invalid table {filter:'abc'} -> Invalid filter no table at all -> Invalid table (was a ReferenceError before) {type:'FOOBAR',table:'NOT_A_LIBDS',filter:'abc'} -> Invalid table Finding 2 - the tests waited 100ms for a fire-and-forget write. The cause was the writers, not the tests: getddl, getrawdata and exportdb used the async fs.writeFile for the headers file, while %mfs_httpheader (fopen/fput/fclose) and dcMockUtils.mpStreamfile are synchronous. Those three now use fs.writeFileSync, so the file is complete when the program returns; the waits are gone, the positive assertions read the file directly, and the absence assertions mean "never written" rather than "not written yet". No polling remains in the file. Also: mock-settings.test.js renamed the control library to DC7QATEST to prove getrawdata follows the settings file - nine characters, which the estate rejects as "Invalid table" for a LIBREF. The fixture is now DC7QATST. That was a test defect the new validation exposed, not a product one. Tests: download-headers.test.js is 11 tests, and the two new ones were run against the old code first - dropping either check fails its own test. Whole mock suite: 144 tests, 144 pass. parity/check.js unchanged.	2026-11-07 23:23:09 +00:00
dc	bb771e9f6f	fix(mocks): the download headers the estate actually sends ... Finding 1 on #510 was that getddl sent an octet-stream, quoted-filename header the estate never sends. The same defect was in every other download path, so this fixes the class rather than the one service. Every value below was read from the synthetic estate's own response, except where noted. public/getrawdata, from getrawdata.sas and its recorded answers: SAS application/text LIBREF.DS.sas CSV application/csv LIBREF.DS.csv EXCEL openxml sheet LIBREF.DS.xlsx (from the service source: PROC EXPORT xlsx fails on the synthetic estate with syscc=1012, so it cannot be recorded there) MARKDOWN text/markdown LIBREF.DS.md PDF application/pdf LIBREF.DS.pdf - the filename is the upcased LIBREF.DS plus the type's extension, unquoted - WEBCSV and WEBTAB send no headers at all: the estate writes those rows straight to _webout, and a content-disposition would advertise a download it does not offer - a type outside that list aborts "Invalid type" with mac=&_program..sas before anything is streamed, which is what the estate answers for FOOBAR admin/exportdb: %mp_streamfile(contenttype=TEXT, outname=&dc_libref..ddl), so application/text and an unquoted <libref>.ddl - the mock sent text/plain. The shared mpStreamfile helper in dcMockUtils (exportconfig, getauditfile) now writes one header per line with the filename unquoted, as %mfs_httpheader does, so the zip downloads stop quoting a name the estate does not quote. Tests: sas/mocks/test/download-headers.test.js (renamed from getddl-headers.test.js) now covers all four paths - 9 tests. Each of the new assertions was run against the old behaviour first and fails there: text/csv instead of application/csv, a quoted filename, and WEBCSV aborting instead of streaming. Whole mock suite: 142 tests, 142 pass.	2026-11-07 23:09:31 +00:00
dc	719964f0af	test(mocks): pin the getddl timestamp and header format ... The parity gate masks the whole generated-by comment and carries no response headers, so neither the local-time fix nor the header shape was visible to any test. Both are now unit tests in their own file, which fixes TZ to a zone three hours from UTC so a return to the UTC getters cannot pass by coincidence: - the printed clock is the UTC clock plus the zone's offset, checked to within the two minutes a run can take - the header file holds exactly the two lines %mfs_httpheader writes, application/text and an unquoted filename - an aborted request still describes no download Each was run against the pre-fix behaviour first: the UTC getters fail the first test and the old single-line quoted header fails the second. Whole mock suite: 136 tests, 136 pass.	2026-11-07 22:40:51 +00:00
dc	8ed51607a8	fix(mocks): answer getddl the way the estate does, headers and all ... Six findings from the review of #510, each checked against the synthetic estate's own public/getddl response before being changed: - the download is an application/text attachment, not octet-stream, and the filename is unquoted: the estate sends `content-type: application/text` and `content-disposition: attachment; filename=<libref>_<ds>.ddl` - the DDL header timestamp is the session's local time, as %sysfunc(datetime(), datetime19.) is, not UTC - the estate answers 07NOV2025:22:27 where UTC getters give 21:27, and the projection masks the comment, so nothing else could see the drift - the headers are written after the DDL is generated, as %mp_streamfile runs after %mp_ds2ddl, so nothing is sent before a request that cannot produce a file has aborted - the refusal of an unknown flavour was invented: getddl.sas passes it to mp_ds2ddl, which has branches for SAS, TSQL and PGSQL only and leaves the file holding its opening comment. The estate answers that as a 200 download, and the mock now does too; a new parity scenario, unknown_flavour, pins it - the "all forty control tables" claim in the DDL comment overstated what the committed goldens show: MPE_TABLES is the table recorded, in three flavours - getcols: drop the dead `libref = knownLib` assignment Tests: node --test sas/mocks/test/*.test.js - 133 pass, 0 fail, and parity/check.js reports 0 differences for every getddl scenario.	2026-11-07 21:29:59 +00:00
dc	d4335e83c6	docs(mocks): the getcols libref work is in this branch now ... The README still listed a LIBDS naming a library the estate does not have as outstanding work - the mock falls back to searching every libref and answers. That is done: the library is validated first and the fallback is gone, and the entry would have contradicted the code whichever of the two pull requests landed second. Folds in #515 (the two getcols aborts, the NOLIB scenario and its unit test) so the mock parity work is one branch rather than one per finding.	2026-11-07 20:47:14 +00:00
dc	4be34a760e	fix(mocks): tell a malformed LIBDS from one getcols cannot resolve ... getcols.sas answers two different things and the previous commit collapsed them. "Invalid libds" comes from the shape check (%mp_validatecol - a libref of eight characters or fewer, a member of 32 or fewer), and only from there. A LIBDS that passes the shape and cannot be resolved reaches the second abort, msg=syscc=&syscc. Probed on the synthetic estate, that second abort says syscc=3000 with a SAS syntax error behind it for a well-formed unknown library, for a missing member in a known library, and for WORK.NO_SUCH alike - a defect in getcols.sas rather than a designed message. The mock reproduces it as it is, because that is what the client renders in production today, and the comment says so rather than pretending it is designed. The scenario is re-recorded with NOLIB.MPE_TABLES, which passes the shape check; the old value, NO_SUCH_LIB, is eleven characters and only ever exercised the shape abort, so its golden proved less than its name claimed. The unit test now covers the malformed, the unresolvable and the answerable cases separately.	2026-11-07 20:46:42 +00:00
dc	b0bb57ecad	test(mocks): tie each table's key to the columns it names ... The index map and the schema are two lists of the same column names, and nothing else joins them: a later rename in one would leave getddl emitting CREATE UNIQUE INDEX on a column the table does not have, while the test that exists to check every table has a key kept passing. Each uniqueIndex column must now resolve to a column of its own table, case-insensitively, so a rename on either side fails here instead of in a download. Merges release/7.23, which carries the 4.14.0 pinned estate and the lazy-route revert.	2026-11-07 20:37:51 +00:00
dc	a3f76b45b2	Merge release/7.23 into the mock parity branch ... # Conflicts: # sas/mocks/test/harness.js # sas/mocks/test/parity/scenarios.json	2026-11-07 20:16:09 +00:00
dc	0cfcd7dcc8	fix(mocks): send the payload keys the client matches, and stop the flake ... Review findings on this pull request, and one CI failure they predicted. The CI failure was the case change, not the DDL: getdata's DTVARS, DTTMVARS and TMVARS still listed col.name, so they read " some_date" while every header and row key was "SOME_DATE", and the client matches those lists against the headers case-sensitively - an Excel upload stopped treating those columns as dates (the Cypress failure was '21958' where '1960-02-12' was expected, across several specs). They go through payloadKey now, like every other name in the payload. Also from the review: the payloadKey doc block had displaced mpGetcols' JSDoc, so mpGetcols was undocumented and payloadKey was described as taking a table's columns; it sits above the block again. ddlDialectColumn's unreachable null return is gone, and an unrecognised flavour aborts rather than quietly answering SAS DDL under a label it was not asked for. The intermittent suite failure the reviewer saw is the harness pruning a session directory under a pending async header write; #512 carries the same fix, and this branch needs it too or the gate fails at random. getrawdata's CSV header still upcases inline: that service carries its own helpers rather than loading dcMockUtils, so payloadKey is not in its scope.	2026-11-07 19:48:19 +00:00
dc	1f1ebc6aa4	chore(ci): merge release/7.23 into the 4.14.0 pin branch	2026-11-07 19:27:49 +00:00
dc	f851bfafb4	fix(mocks): refuse a library getcols does not have, as the estate does ... getcols.sas validates LIBDS against the libraries it can assign before it reads anything (%mpe_validatecol, then dc_assignlib), so a LIBDS naming a library the estate does not have aborts "Invalid libds". The mock searched every libref instead, so NO_SUCH_LIB.MPE_TABLES answered with the columns of the copy that does exist in the control library - a request the estate refuses, answered with data from somewhere else. The named library is resolved the way the estate assigns it, and a missing or malformed LIBDS reaches the same abort. A member missing from a library that does exist is left alone: the estate fails inside its query there, and there is no message worth copying. Recorded as public/getcols/wrong_libref. Also fixes an intermittent suite failure this work exposed: the harness pruned earlier session directories, so a service whose headers are written with the async fs.writeFile could land after its directory was removed, and the ENOENT took the test process down. Each run still gets its own directory - that is what stops a stale body being served - and the estate is removed when the process exits.	2026-11-07 18:24:55 +00:00
dc	98c93127e2	feat(mocks): generate the TSQL and PGSQL flavours of the DDL download too ... public/getddl answered the SAS text for both remaining flavours, which the download menu labels by name. Each is generated from the same table metadata now, and each was checked against the estate's own output before it went in: all forty control tables, byte for byte, in all three dialects. The dialects differ in more than type names. Postgres quotes with double quotes and declares its key as a second statement after the table; TSQL brackets the identifiers, carries the key inside the create table as a UNIQUE NONCLUSTERED index, and turns every column LABEL into a sys.sp_addextendedproperty block after the GO. Only a format whose name starts with DATETIME becomes a timestamp in either - a DATE9. or E8601DT26.6 column stays DOUBLE PRECISION or decimal(18,2), which is what the estate does and is not what a format list would suggest. Recorded as public/getddl/mpe_tables_pgsql and public/getddl/mpe_tables_tsql, so each dialect is a gate rather than something re-checked by hand.	2026-11-07 18:19:00 +00:00
dc	4443e8fada	revert(client): undo the lazy route split (#508) ... Revert commit 7d5f1e85. The lazy split makes the Cypress suite fail, reproducibly, on two separate runs of two different trees that contain it: run 3441 (cc43512) viewer-labels 1, full-table-search 1, viewbox 1, row-cell-limits 4 run 3449 (5aa8cd7) viewer-labels 4, full-table-search 1, viewbox 1, row-cell-limits 4, deploy-folder-picker 7 Both runs show the same failure family, and it is always the click that selects a node by name: CypressError: cy.click() failed because the page updated as a result of this command ... the subject is no longer attached to the DOM AssertionError: Expected to find element: #hotTable .ht_clone_top .htCore thead button.changeType, but never found it AssertionError: ... with the selector: .clr-treenode-link but never did The tree is re-rendered while the spec is mid-chain: the click starts a navigation, the route's chunk is fetched, and the node that was clicked is detached before the chain continues. With the modules in the main bundle there was no fetch, so the race almost never fired; a chunk fetch widens it enough to fail four to five specs per run. Revert restores a green suite on the release branch. The split re-lands with the specs' click chains broken up so a re-render cannot detach a subject the chain still holds - that is the change that makes the test robust to the app doing what it does, rather than keeping the app eager to keep the test passing. Not reverted: #507 (text-encoding and handsontable polyfill imports), whose own run 3442 was green with the suite in this state. Measured cost of the revert: the audit at 3449 read 0.93/0.80/0.79/0.70 on performance with both the compressed server and the split in place; the split alone moved those routes by roughly +0.05 when it landed (run 3442 vs 3441), so the revert gives back that much and keeps the compression gain.	2026-11-07 18:16:10 +00:00
dc	96f70d53b6	feat(mocks): generate the DDL download from the table, not a sample ... public/getddl answered one fixed Postgres-flavoured body for every request - DC_JSLIB.MPE_X_TEST whatever dataset was asked for - which is the last read path the client calls where the mock invented its answer. It now builds the SAS-flavour DDL from the table file, the way mp_ds2ddl builds it from dictionary.columns and dictionary.indexes: the header, the create table with each column's type, format, NOT NULL and label, and the unique index. The list-output spacing was derived from the estate's own answers and reproduces them byte for byte for all forty control tables. Recorded as public/getddl/mpe_tables, so a mock change that alters the text fails the gate. The header credits a user and a timestamp, so the projection masks both - a user id is not something to publish, and the comparison is about the DDL. TSQL and PGSQL are not generated yet: a caller asking for one gets the SAS text, and the README says so rather than inventing a second dialect without a recording to check it against.	2026-11-07 18:10:11 +00:00
dc	d189fd4055	feat(mocks): give every control table the key and column case the estate has ... A control table on a real estate is created with a unique index - MPE_TABLES on tx_to/libref/dsn, MPE_X_TEST on primary_key_field, and so on - and mp_ds2ddl turns that into the CREATE UNIQUE INDEX block of a DDL download. The mock's tables had columns and rows and no key at all, which is why getddl could only ever answer a fixed sample. Each table file now carries `uniqueIndex` (name plus the ordered columns), taken from the synthetic estate's own DDL: 39 of the 40 control tables have one, MPE_REQUESTS is the exception the estate also has, and the format catalog marker has none because it is not a dataset. The columns also move to the case SAS creates them with. A table declared with `alert_event` keeps that spelling in dictionary.columns and in the DDL, while every payload the client reads reports the name upper case. The files now store the creation case and the services build their keys through payloadKey(), which fixes the DDL and leaves the payloads alone. Two metadata gaps went with them: MPE_LOADS.csv_dir and MPE_MAXKEYVALUES.keytable are NOT NULL on the estate and were not here. EOF2 echo "--- second commit ---"; git log --oneline -1 | cat	2026-11-07 18:09:50 +00:00
dc	5aa8cd75fc	docs(ci): correct the pin rationale after review ... Three claims in the comment block did not survive checking: the 3.14.0 paragraph described the utils floor as still open when 4.14.0 is the release that raises it, the 3.12.3 paragraph named a guard that does not exist, and the 4.0.0 paragraph was incomplete. The logout one is the interesting correction: the estate does call GET /SASLogon/logout - @sasjs/adapter's logOut() is a GET on that route (client sas.service.ts logout() -> sasjsAdapter.logOut() -> AuthManager.logOut()), so the paragraph now says what the estate does touch and why the status change does not matter to it.	2026-11-07 17:29:01 +00:00
dc	8e2e407837	chore(ci): run the mocked estate on 4.14.0 ... 4.14.0 is the first @sasjs/server release that compresses its responses and sends cache headers for the app it streams (sasjs/server#575), which is what the audit has been measuring: the app is a 4.83MB build output, and the previous pin served it raw and uncacheable. Verified on the published package before pinning it here: 4.14.0 boots, the mock estate deploys and seeds against it, and a deployed asset now answers Content-Encoding: gzip Cache-Control: public, max-age=31536000, immutable with index.html answering max-age=0, must-revalidate. Measured on the previous pin, same build and machine: FCP 8.48s and 0.34 performance raw, 2.81s and 0.49 with the same bytes compressed. 4.0.0's breaking change is the auth statuses (401 where 403 was, and the undocumented DELETE on logout is gone). The client treats any non-2xx as a refusal; the suite running against this pin is the check. sites.yaml and release.yaml move with it - same mocked estate, one version.	2026-11-07 17:01:11 +00:00