Commit Graph
172 Commits
Author SHA1 Message Date
allan 45db0785fb Merge pull request 'fix(multi-dataset): a sheet that failed to extract is not a clean finish' (#437) from fix/multi-dataset-extraction-failures into release/7.22
Sites / deploy (push) Successful in 4m13s
Build / Build-and-ng-test (pull_request) Successful in 5m46s
Lighthouse Checks / lighthouse (pull_request) Successful in 19m29s
Build / Build-and-test-development (pull_request) Successful in 31m10s
Reviewed-on: #437
2026-11-02 15:57:23 +00:00
dc 312964864e fix(multi-dataset): a sheet that failed to extract is not a clean finish
Build / Build-and-ng-test (pull_request) Successful in 5m47s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m54s
Build / Build-and-test-development (pull_request) Successful in 35m45s
The stage that reports what the file is doing is meant to belong to the whole file, and it finished at "Extracting the data finished" whatever happened. A sheet whose parse rejected had already blanked its datasource and dropped itself from the submission in the per-sheet catch, so the user was told the file had extracted while a table was silently missing from the submission.

`allSettled` was already the right primitive, but its results said nothing: the catch swallowed the error, so every promise fulfilled. The catch now names the sheet on the component, the stage ends at a new `extractionFailed` state when anything was left out, and the pane lists what was left out and why.

The two reasons a sheet contributes nothing are kept apart, because they are not the same thing to the reader: a sheet the sheet utility could not read is a "could not extract" problem in red, while a sheet it parsed and found no rows in - which arrives as the rejection "No relevant data found" - is reported as "no rows found" without calling the file corrupt. Both leave the sheet out of the submission, and either one stops the stage reporting a clean finish.

The row derivation sits inside the try with the parse. Outside it, a throw on a row shape the sheet utility did not expect escaped as a rejection that `allSettled` swallows - the dataset stayed in the submission with `parsingTable` true and the stage reported a clean finish over a sheet that had been dropped.

The messages live in the no-table-selected pane, which is where the user is once extraction has finished; the file pane they were first put in unmounts as soon as the datasets are fetched, so they would never have been read there. The stage line keeps one id per state rather than two of the same: `file-loading-stage` in the file pane while the datasets are fetched, `file-extraction-stage` in the overlay while the sheets are extracted, and the Cypress spec asks for both.

The Cypress spec asks for both stage ids, because the element that carries the stage changes state, and asserts that a sheet left out of the submission is named in the pane - under whichever reason the sheet utility reported, since only the utility decides which of the two it is.

The spec builds the component through the test injector rather than with seven positional arguments, so a constructor that gains or reorders a dependency fails there (`NG0201: No provider found for ElementRef`) instead of binding a stub to the wrong collaborator and passing.
2026-11-02 14:49:52 +00:00
dc 32d3d49b59 fix(client): stop synthesising aria-labels; name the filter modal's submit in the test
Build / Build-and-ng-test (pull_request) Successful in 6m31s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m12s
Build / Build-and-test-development (pull_request) Successful in 35m23s
Scoping the ARIA pass to the grid exposed a dependency the document-wide
sweep had been hiding: the sweep used to write aria-label from an
element's text, so cypress/e2e/full-table-search.cy.ts could select the
filter modal's submit button as aria-label="Ok" - a label only the sweep
ever wrote. Once the sweep stopped reaching the modal, that selector
matched nothing and the dev job failed on it.

An element's text already is its accessible name, so the synthesis is
gone from the pass, and the spec now selects the button by its text the
way a user would.
2026-11-02 02:02:31 +00:00
allan 2ee3c2241b Merge pull request 'test(excel): guard that discarding a failed upload clears the red cells' (#427) from fix/122-discard-excel-clears-red-cells into release/7.22
Sites / deploy (push) Successful in 4m40s
Build / Build-and-ng-test (pull_request) Successful in 6m29s
Lighthouse Checks / lighthouse (pull_request) Failing after 7m36s
Build / Build-and-test-development (pull_request) Failing after 12m7s
Reviewed-on: #427
2026-11-01 14:49:51 +00:00
dc cf759db3d1 test(excel): guard that discarding a failed upload clears the red cells
Lighthouse Checks / lighthouse (pull_request) Failing after 11m39s
Build / Build-and-ng-test (pull_request) Successful in 5m41s
Build / Build-and-test-development (pull_request) Successful in 31m23s
An uploaded file's preview is validated in place, so the cells that fail
a rule carry Handsontable's invalid cell meta. Discarding the file
restores the loaded data, but cell meta is keyed by row/column and
survives that restore - without a re-validation the grid keeps painting
cells that are no longer invalid.

cancelEdit re-runs the validation (hot.validateCells()) after restoring
the data source, which is what clears them; this test drives the flow and
pins it: import a file that fails validation, discard it, then assert the
row count is back, no .htInvalid cell remains and the page is read-only
again. The manual-edit path (test 25) keeps the parsed values by design.

Refs #122
2026-11-01 13:29:13 +00:00
dc 45d6c025c9 fix(multi-dataset): show the file size and loading stage while the file loads
Build / Build-and-ng-test (pull_request) Successful in 6m31s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m37s
Build / Build-and-test-development (pull_request) Successful in 35m39s
The multi-load page rendered the loading stage only inside a branch guarded by
`parsedDatasets.length`, and that collection is empty for the whole read and
sheet-scan phase - so two of the four stage labels ("Reading the file",
"Searching for the data in the file") were assigned but never reached the DOM,
and a large file sat on screen with no feedback at all.

The stage is now shown next to the file name and size from the moment a file is
attached, and again in the content area while the sheets are being parsed. The
labels cover every phase the user waits through: reading the file, searching
for the data in the file, file ready to load, fetching the dataset details (one
getdata call per dataset - the longest wait), extracting the data, and
extracting the data finished. A spinner accompanies the stages that have work
running; the idle stages and the finished state show the label alone.

Two corrections fall out of that: the stage only moves to `extracted` once
every sheet has been parsed rather than when the first one finishes, and the
read's continuation only advances the stage if the read is still the phase in
progress (the sheet scan runs in parallel and can finish first).

excel-multi-load.cy.ts gains test 4, which asserts the file size, the ready
stage, the fetching stage (the getdata calls are held open so it is on screen
long enough to assert) and the handover to the "select a dataset" hint.

Closes #121
2026-11-01 11:22:16 +00:00
dc ad6e4c6e78 chore: merge release/7.22 into fix/395-metanav-spec
Build / Build-and-ng-test (pull_request) Successful in 5m32s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m50s
Build / Build-and-test-development (pull_request) Successful in 32m20s
2026-10-31 23:05:09 +00:00
dc 02a8590310 fix(test): run metanav.cy.ts in CI and open a metadata object
Build / Build-and-ng-test (pull_request) Successful in 7m14s
Lighthouse Checks / lighthouse (pull_request) Successful in 19m4s
Build / Build-and-test-development (pull_request) Successful in 35m25s
`cy.get` inside `.within(<tree node>)` only searches that node, and the
metadata accordion renders in the content area - so `.clr-accordion-header-button`
was never found however long it retried. The spec queries the accordion at
document level, selects the first metadata type by its class rather than by
node index, and clicks the header button directly (Clarity puts
`clr-accordion-header-button` on the button itself).

The assertions are on the app's own markers: the single open panel
(`clr-accordion-panel-open`, `aria-expanded`), the association tree inside the
panel, and the attribute grid that `metanav/metadetails` fills.

metanav moves from EXCLUDED to SPECS in `.gitea/scripts/cypress-specs.sh`.

Closes #395.
2026-10-31 22:36:33 +00:00
dc dff4bbdc09 test(download): find the approval row by table, not by the stage URL
Build / Build-and-ng-test (pull_request) Successful in 4m48s
Lighthouse Checks / lighthouse (pull_request) Successful in 18m57s
Build / Build-and-test-development (pull_request) Successful in 28m58s
The case took the id from the stage route and looked for a download button
carrying it.  For an upload the route does not hold a table id at all - it holds
a load reference, the CI failure naming DC20261031-201657.470_601010_4449 - so
the button it looked for could never exist, however long it waited or however
many times the screen was reloaded.  That is why the row "never appeared" while
the approval was in fact there.

The row is now found the way hook-programs finds its own: by the table it names.
The id the download is named after is read from the button, which is the only
place that knows it.

Verified against the mock estate locally, with the licence the spec now plants:
7 passing, exit 0.
2026-10-31 20:24:58 +00:00
dc 1455f9d435 test(download): licence the estate this spec needs
Build / Build-and-ng-test (pull_request) Successful in 4m59s
Lighthouse Checks / lighthouse (pull_request) Successful in 18m36s
Build / Build-and-test-development (pull_request) Failing after 29m25s
The estate ships no licence key, so it runs on the free tier, and the free tier
disables file upload - which is exactly what the audit case needs. Whether it
could run depended on another spec having planted a key earlier in the run: the
key persists in the estate config, so the case passed or failed on spec order.

loginAndUpdateValidKey(true) plants one here. csv-limited, which asserts the
free-tier modal, runs first in cypress-specs.sh and is unaffected; download runs
last, so nothing downstream inherits the key.

Found by running the spec against the mock estate locally: the upload input
never rendered, the modal read "Locked Feature (File Upload) ... site id (123)",
and the run went 7/7 green once the key was planted.
2026-10-31 19:43:32 +00:00
dc fbf0ed676b test(download): renumber the fixture so every attempt is an insert
Build / Build-and-ng-test (pull_request) Successful in 5m15s
Lighthouse Checks / lighthouse (pull_request) Successful in 19m9s
Build / Build-and-test-development (pull_request) Failing after 29m51s
The audit-file case uploads a fixture and then looks for the approval it should
create.  An upload identical to what the table already holds produces no
changes, and a changeset with no changes leaves nothing waiting for approval -
so the case finds no row.  A retry lands in exactly that state: the first
attempt's rows are already in the table, so the second upload is a no-op and the
failure looks like a missing approval rather than a duplicate upload.

The fixture is now copied per attempt with its primary keys renumbered from a
random base, so the upload is always an insert.  The copy is removed in
afterEach - deleteFolder cannot be used there because the source fixture lives
in the same directory.
2026-10-31 18:17:14 +00:00
dc 86c8a1a76f test(download): reload the approve screen until the submitted row appears
Build / Build-and-ng-test (pull_request) Successful in 5m1s
Lighthouse Checks / lighthouse (pull_request) Successful in 18m37s
Build / Build-and-test-development (pull_request) Failing after 29m22s
The upload and submit now work; the failure moved to the approve grid, which is
loaded when the screen opens and does not yet contain the row just submitted.
The screen is reloaded up to five times, two seconds apart, until the row with
this test's id is present.

The mock getddl aborts when ds is missing, as production does (getddl.sas:43-45),
instead of quietly returning a library-level DDL - a client that dropped ds would
otherwise stay green against the mocks.

The comment above the case no longer claims a leftover row cannot matter: the
grid paginates, so a row pushed past the first page would not be found.
2026-10-31 13:19:04 +00:00
dc 5eb8b0bf46 test(download): assert the info modal's title, and wait for the submit to land
Build / Build-and-ng-test (pull_request) Successful in 5m1s
Lighthouse Checks / lighthouse (pull_request) Successful in 17m43s
Build / Build-and-test-development (pull_request) Failing after 32m49s
The upload does parse now - CI showed the info modal appearing - but the helper
treated it as an abort because the info modal and the abort modal are the same
component (.clr-abort-modal).  The title is what separates them: a good upload
raises "Table Found", so that is asserted and anything else fails with its own
text instead of being clicked through.

After Submit, the stage route is already current, so waiting on the URL proved
nothing; the editor leaving preview mode is the signal that the submit was
accepted.  Also trims the comment that narrated the old route.
2026-10-31 12:10:07 +00:00
dc 39b12e9a18 test(download): open the upload modal before attaching the fixture
Build / Build-and-ng-test (pull_request) Successful in 5m14s
Lighthouse Checks / lighthouse (pull_request) Successful in 19m14s
Build / Build-and-test-development (pull_request) Failing after 29m36s
The first case failed in CI with "Expected to find element:
input[type=file]#file-upload", because the editor renders that input inside its
upload modal - the Upload button in the button bar opens it.  Clicking the button
by its label rather than by position, as excel.cy.ts does.

The viewer download cases (2-7) passed, so the rest of the spec is sound.
2026-10-31 11:01:09 +00:00
dc 4c61ef374b test(download): one note for the menu selector
Build / Build-and-ng-test (pull_request) Failing after 1m57s
Build / Build-and-test-development (pull_request) Skipped
Lighthouse Checks / lighthouse (pull_request) Successful in 17m19s
Two comment blocks sat back to back above the same call - the click handler's
and the anchored label's. They describe one selector, so they read as one note.
2026-10-31 00:29:34 +00:00
dc 0ae552ea44 test(download): strip a trailing slash from the id, anchor the menu label
Build / Build-and-ng-test (pull_request) Successful in 4m51s
Lighthouse Checks / lighthouse (pull_request) Successful in 17m5s
Build / Build-and-test-development (pull_request) Failing after 31m27s
Three review points.

The table id was sliced out of the stage URL without trimming a trailing slash,
so a route rendered as /stage/<id>/ produced an id that never matched the button.
Trailing slashes are removed before use.

The options menu entry is matched on the whole label, /^\s*Download\s*$/, because
an entry like "Download web query" would make cy.contains ambiguous and it fails
outright rather than picking one.

The getddl guard covers both hosts: libref is read behind a typeof too, so a call
with either variable out of scope falls back instead of throwing.
2026-10-31 00:04:08 +00:00
dc b47fb2367b test(download): wait for the modal instead of sampling the body once
Build / Build-and-ng-test (pull_request) Successful in 4m50s
Lighthouse Checks / lighthouse (pull_request) Successful in 18m3s
Build / Build-and-test-development (pull_request) Failing after 29m26s
Three review points.

attachExcelFile read the body immediately after attachFile, so a slow upload made
abortModal false and skipped the abort branch - the failure then arrived later as
the timeout the branch exists to avoid. It now waits for either the confirmation
button or the abort modal title, with longerCommandTimeout, and branches on which
one matched; the confirm click uses the same timeout.

getddl guards ds with typeof: the truthiness fallback only covers a ds that is
bound and empty, while a missing ds in scope throws before the fallback runs.

The submitted row test 1 leaves behind is now documented in both places. The CI job
deletes the appLoc and re-runs makedata before the specs, so it does not accumulate
between runs; a hand-run against a reused estate leaves one pending row per run, and
the selector is anchored on the id the test created, so a stale row changes nothing.
2026-10-30 23:24:01 +00:00
dc 1c0d636708 test(download): fail loudly when the upload aborts
Build / Build-and-ng-test (pull_request) Successful in 4m35s
Lighthouse Checks / lighthouse (pull_request) Successful in 17m15s
Build / Build-and-test-development (pull_request) Failing after 29m21s
The abort branch asserted the modal title did not include 'Abort Message', so
any other title let the run continue with the abort modal open and the next step
died on a timeout that never mentioned the upload. It now throws with the title,
which points at the upload as the cause.
2026-10-30 23:00:54 +00:00
dc 468d2e9a80 test(download): check for the modal before using it, and click the menu item
Build / Build-and-ng-test (pull_request) Successful in 4m56s
Lighthouse Checks / lighthouse (pull_request) Successful in 17m48s
Build / Build-and-test-development (pull_request) Failing after 31m24s
Three review points.

attachExcelFile read '.clr-abort-modal .modal-title' unconditionally, so an
upload that raised no abort modal failed on the lookup, and the abort branch
then left the modal open for the next command to click through.  It asks the body
first: no abort modal means confirm the upload; an abort modal means assert its
title is not an Abort Message and stop there.

openDownloadModal clicked the deepest element holding the text.  The click
handler sits on the div[clrDropdownItem], so the item is selected by that
attribute and its text.

The spec list comment claimed download leaves nothing for later specs to trip
over.  It seeds the approval it needs and that approval stays, which the comment
now says.
2026-10-30 22:28:29 +00:00
dc f50a0de9ae test(download): drive the file input directly and name DDL files as getddl does
Build / Build-and-ng-test (pull_request) Successful in 4m35s
Lighthouse Checks / lighthouse (pull_request) Failing after 11m5s
Build / Build-and-test-development (pull_request) Failing after 31m4s
Two reasons the last CI run still had four failures.

test 1 timed out on label[for="file-upload"].  The label is a click target for a
hidden input; attachFile can drive the input itself, so the helper no longer
clicks anything and selects by the input's id, asserting there is exactly one.
That also answers the earlier review point about positional selectors without
reintroducing a click on a control that is not always rendered.

tests 5-7 timed out waiting for the DDL file.  The mock names a DDL download
after the libref alone, while services/public/getddl.sas names it
<libref>_<ds>.ddl - which is what the viewer expects and what the spec looks
for.  The mock now follows the service.
2026-10-30 22:18:03 +00:00
dc 628092d136 test(download): single separator in the fixture path
Build / Build-and-ng-test (pull_request) Successful in 4m49s
Lighthouse Checks / lighthouse (pull_request) Successful in 18m1s
Build / Build-and-test-development (pull_request) Failing after 35m54s
fixturePath carried a trailing slash that the template also supplies, so
attachFile was given /excels//regular_excel.xlsx.  Also drops the stale duplicate
doc comment above attachExcelFile - the helper targets the editor's import label,
which is what the remaining line says.
2026-10-30 21:23:56 +00:00
dc 3e4070ae49 test(download): select the download format by value
Build / Build-and-ng-test (pull_request) Successful in 5m23s
Lighthouse Checks / lighthouse (pull_request) Successful in 17m58s
Build / Build-and-test-development (pull_request) Failing after 35m4s
Second CI run put the spec at 2 passing / 5 failing, and the five failures were
one more stale assumption: cy.select() matches an option's text against the whole
label, and the labels carry a suffix - the option for the Excel writer is
"Excel (.xlsx)", not "Excel". Every format is now chosen by the option's value
(EXCEL, SAS, SAS_DDL, TSQL_DDL, PGSQL_DDL), which the template sets and which does
not change when a label gains or loses a qualifier.

Also the review points on this PR:

- attachExcelFile starts from label[for="file-upload"] instead of
  .buttonBar button:last-child, and submitExcel no longer takes a callback that
  nothing passed.
- test 1 asserts on the download button whose id is the table it submitted: the
  stage URL carries the id, so the assertion names its own row and asserts one
  match, instead of indexing into every match on the screen.
- the comment in cypress-specs.sh no longer claims the spec depends on approvals
  left by earlier specs - it seeds its own - and says why it is appended.
2026-10-30 21:04:41 +00:00
dc 847d329d5c test(download): point the spec at the live route and the dropdown markup
Build / Build-and-ng-test (pull_request) Successful in 4m19s
Lighthouse Checks / lighthouse (pull_request) Successful in 17m52s
Build / Build-and-test-development (pull_request) Failing after 34m0s
Three separate reasons the spec could not pass, all of them stale routes or
selectors rather than an overlay covering the button:

- test 1 opened #/approve/toapprove, which is not a route. The review module is
  namespaced under review/, so the approve screen is #/review/approve. The dead
  URL rendered the 404 page, and the only .btn.btn-success on that page is not a
  download button - which is why the failure looked like a covered element.

- on the right route the approve grid still renders nothing on a fresh estate:
  it draws a download button per table waiting for approval, and there are none
  ("There are no approvals remaining"). Test 1 now submits an excel fixture for
  mpe_x_test first, the way stage.cy.ts does, so it owns the approval it
  downloads instead of depending on what earlier specs happen to leave behind.

- the viewer's options menu is a list of div[clrDropdownItem] entries, not
  buttons, so cy.get('clr-dropdown-menu button') never matched; Download is now
  selected by its text, as filter-panel.cy.ts selects Filter. The dropdown-toggle
  class the helper also required is gone from that element too.

The download button is selected by aria-label, which is unique, rather than a
bare .btn.btn-success that also matches the editor toolbar's small buttons.

Closes #398
2026-10-30 19:54:14 +00:00
dc c1e7314417 test(download): target the approve button and the viewer options dropdown
Build / Build-and-ng-test (pull_request) Successful in 4m39s
Lighthouse Checks / lighthouse (pull_request) Successful in 17m28s
Build / Build-and-test-development (pull_request) Failing after 36m15s
Both selectors in download.cy.ts are stale or ambiguous, which is why the spec
fails as soon as it runs:

- `.btn.btn-success` also matches the editor toolbar's buttons (`btn btn-sm
  btn-success`), and those sit inside a fixed-position container on the approve
  screen. The first match was one of those, so `should('be.visible')` retried
  against an element that is never visible and the error read like the download
  button was covered. The approve row's button is `btn btn-success` with
  aria-label="Download audit file", so select it by that label.

- `.btn.btn-sm.btn-outline.filterSide.dropdown-toggle` cannot match anything:
  the viewer's options dropdown is `class="btn btn-sm btn-outline filterSide"`,
  with no dropdown-toggle class. filter-panel.cy.ts and row-cell-limits.cy.ts
  already select it as `.filterSide`.

The spec goes back into the CI list, appended at the end because it needs
approvals to exist and the earlier specs create them. metanav stays excluded
(#395).

Closes #398
2026-10-30 18:21:14 +00:00
dc c803773d3c fix(cypress): register the deleteFolder task, restore the spec order, hold metanav out
Build / Build-and-ng-test (pull_request) Successful in 6m19s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m9s
Build / Build-and-test-development (pull_request) Failing after 41m32s
Four specs failed on release/7.20 once the list went from 19 to 24. Three of the
four are addressed here; the fourth is a spec that cannot pass and is now
excluded with an issue.

download.cy.ts - it clears the downloads folder in an afterEach through
cy.task('deleteFolder'), and that task was registered only in
cypress/plugins/index.js, which Cypress 10+ never loads unless it is required
from cypress.config.ts. The task is now registered in setupNodeEvents, where
Cypress looks, and the login in its beforeEach only fills the form when the form
is actually on screen - the unconditional version failed with 'the center of this
element is hidden from view' whenever the session was already live.

csv-limited.cy.ts and hook-programs.cy.ts - neither is broken; both depend on
where they sit in the run. csv-limited asserts the free-tier 'Locked Feature'
modal, which an earlier spec removes by planting a valid licence, and
hook-programs asserts an approval row against its target table before other specs
have added rows to the approval tables. The list is now the sequence that was
green for months, with csv-limited first and hook-programs second, and the newer
specs appended rather than inserted. The script says why order matters, because
the next person to alphabetise it will break the same two specs.

metanav.cy.ts - cannot pass, and says so in its own source: '// BLOCKER ...
.clr-accordion-header-button always null although it is present on the page'.
With the duplicate login in its beforeEach removed it now reaches that assertion
and fails there, so it is excluded from the list with #395 as the reason, and the
login fix is kept for whoever picks it up. The coverage check now distinguishes a
deliberate exclusion from a forgotten file, so nothing can go missing silently.

Not verified locally: the specs need an estate seeded by 'sasjs cbd -t server-ci'
plus makedata, which a scratch mock does not reproduce - the app never leaves
app-loading. The CI run is the verification.
2026-10-28 21:29:50 +00:00
dc 689b5b6709 Merge feat/va-param-map into feat/va-parmap-transforms
Build / Build-and-ng-test (pull_request) Successful in 5m14s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m29s
Build / Build-and-test-development (pull_request) Successful in 33m58s
Brings in the diagnostics fixes from the base branch; conflicts were the
paramMap type widening to VaParmapTarget and the new paramsStable argument,
both of which the transform branch already carries.
2026-10-28 12:24:36 +00:00
dc 541a61ba08 fix(embed): make the va_parmap diagnostics say what they mean
Build / Build-and-ng-test (pull_request) Successful in 5m16s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m14s
Build / Build-and-test-development (pull_request) Successful in 33m26s
Review findings, all about a warning that misleads:

- The banner heading said the mapping was ignored, but two of the three
  lists it renders are mappings that are applied (the AND-ed pair) or that
  were never applicable (a parameter VA does not send). Neutral heading.
- unmatchedMappings reported a key as unsent as soon as any parameter had
  been seen. VA sends the parameters a report object's query uses, so a
  name that has not arrived yet looked like a typo. A mapping is now only
  judged once a message introduces no new parameter name, i.e. once the
  object's parameter set has settled.
- sameColumnMappings counted a column the table does not have, so one
  mistake produced the unresolved-mapping line and the AND line. A column
  that is not in the table is now left to unresolvedMappings.
- The Cypress cases asserted exact row counts, which encode the demo
  table's contents. They now read the unfiltered count from the app and
  assert the filtered grid is smaller (or empty), so a fixture change
  cannot fail a behaviour test.
2026-10-28 12:20:37 +00:00
dc 88010c3980 feat(embed): resolve va_parmap values through named date anchors
Build / Build-and-ng-test (pull_request) Failing after 1m55s
Build / Build-and-test-development (pull_request) Skipped
Lighthouse Checks / lighthouse (pull_request) Successful in 20m22s
A VA parameter can now carry a transform on the column side of a mapping:

  va_parmap=SELECTED_DATE-D_DATE:ADD(QUARTER_START,-1)

which filters D_DATE on the report value AND the same point one quarter
earlier, as a single IN clause - so the AND rule, the clause signature,
saveQuery and the reload loop are untouched.

Transforms resolve to one clause with these operations:

- ADD(<ANCHOR>,<offset>)   the value plus one anchor beside it -> IN
- RANGE(<PERIOD>,<offset>) a closed period -> BETWEEN

Anchors: QUARTER_START/END, MONTH_START/END, YEAR_START/END.
Periods: QUARTER, MONTH, YEAR. Offset is a whole number of periods,
defaulting to 0.

The names describe the result rather than mirroring SAS intnx: the
arithmetic runs in the browser, so mirroring intnx would mean
reimplementing its alignment letters and their corners (week alignment,
's' and 'm') with no guarantee of matching SAS. A report author can read
ADD(QUARTER_START,-1) without knowing intnx.

Resolution is anchored on the parameter value, never on today, so a saved
report reproduces the same rows. A transform that cannot run where it is
used - a non-temporal column, a value that is not a date - is skipped and
reported in the existing warning banner rather than silently filtering on
the raw value.
2026-10-28 11:52:53 +00:00
dc 0011f7e933 fix(embed): log a parse-time mapping problem once
Build / Build-and-ng-test (pull_request) Successful in 5m0s
Lighthouse Checks / lighthouse (pull_request) Successful in 19m41s
Build / Build-and-test-development (pull_request) Successful in 31m54s
The parse-time problems were logged at app init and then again by the
editor's warning loop, which spreads them into the banner's text. The loop
now logs only what the instance discovers for itself (unresolved column,
unmatched key, AND-ed pair); the parse problems stay in the banner and keep
their single init-time console line.

E2e case 11 pins it: one unparseable entry plus one valid-but-missing
mapping, so the loop certainly runs and logs its own line, and exactly one
line mentions the parse problem.
2026-10-28 11:13:10 +00:00
dc 4ed1e0684d test(embed): prove va_parmap case-insensitivity end to end
Build / Build-and-ng-test (pull_request) Successful in 5m31s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m42s
Build / Build-and-test-development (pull_request) Successful in 33m51s
2026-10-28 09:57:58 +00:00
dc 2623a3fb4b fix(embed): defer the column check until metadata loads, report AND-ed pairs
Build / Build-and-ng-test (pull_request) Successful in 5m50s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m50s
Build / Build-and-test-development (pull_request) Successful in 33m51s
Two review findings, both paths where a filter could look applied while
producing nothing.

- `unresolvedMappings` judged a mapping against an empty column list while the
  grid was still loading, so every entry was briefly reported as naming a
  column the table does not have. An empty or unset `cols` now means "not known
  yet" and returns nothing, so the banner appears only once metadata exists.
- Two parameters mapped to one column are AND-ed, so different values yield no
  rows at all, and nothing said so. `sameColumnMappings()` now reports the
  contended column and both parameters. The clauses are deliberately left as an
  intersection: turning that AND into an OR is a filter-semantics decision, not
  a diagnostic, so it is tracked separately.

Covered by unit tests for both (unloaded column list, one line per contended
column) and e2e case 10, which drives two parameters onto one column and
asserts the banner plus the empty grid.
2026-10-28 09:54:48 +00:00
dc f83dc359e2 fix(embed): one parameter maps one column, and report typos on both sides
Build / Build-and-ng-test (pull_request) Successful in 5m42s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m20s
Build / Build-and-test-development (pull_request) Successful in 33m57s
Three follow-ups from review, all the same class of defect: an author error
that leaves an unfiltered grid looking like a working filter.

- A duplicate key in `va_parmap` silently kept whichever entry came last, so
  `va_parmap=DEPT-REGION&va_parmap=DEPT-OTHER` filtered OTHER with nothing to
  show for it. One VA parameter now maps one column: the first entry wins and
  a conflicting duplicate is reported. The same mapping written twice is not a
  conflict, and two different parameters mapping to one column stay valid -
  that is how two report objects filter the same column from their own
  parameters.
- A parameter-side typo (mapping `DROPDOWN_TAB1` when the report parameter is
  `DROPDOWN_TAB_1`) produced no clause, no banner and no console line.
  VaMessagingService now records every parameter key it has seen, across
  editor reloads, and the editor reports a mapping key that VA has never sent.
  Judged only once parameters are actually arriving, because VA sends just the
  parameters its query uses - an absent key is not on its own an error, and a
  key that turns up later stops being reported.
- The deferred first load cannot check a mapped column against metadata. On
  SAS `mp_filtercheck` aborts the save, so the load falls back to unfiltered;
  the banner now covers that path too, and an e2e case seeds the pre-bootstrap
  message buffer to prove it (and to prove a valid mapping filters the very
  first getdata).
2026-10-28 09:34:02 +00:00
dc 411f6c3354 fix(embed): warn when a va_parmap entry is ignored
Build / Build-and-ng-test (pull_request) Successful in 5m48s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m9s
Build / Build-and-test-development (pull_request) Successful in 33m7s
A `va_parmap` entry that cannot be used was dropped in silence, in both
places it can fail: at parse time (no `-` separator, a missing side, or a
column that is not a plain SAS name) and at apply time (the mapped column is
not in the table being opened). Either way the parameter is skipped, which
degrades to "no filter" - a typo in the report's URL then looks exactly like
a working filter over the whole table.

Both are now reported in terms the report author can act on:

- a warning banner in the editor (VA mode) naming the parameter, the column
  and the table, and pointing at the `va_parmap` entries in the URL
- console.warn for each distinct problem, so it is findable from the report's
  console too

The parser returns `{ map, problems }` rather than a bare Map, VaFilterService
gains `unresolvedMappings()` for the entries the loaded table cannot satisfy,
and the editor merges both sets for the banner (logging once per distinct set
rather than once per VA message).

The banner is covered by two e2e cases, one asserting its painted box inside
the viewport (VA hides the app header, so it has to stand on its own) and one
covering a parse-time failure before any VA message arrives. The alert markup
keeps the message inside the alert's text flow so it wraps, instead of being
laid out beside a title and pushed off the right edge.
2026-10-28 09:08:30 +00:00
dc 15a1f34d52 feat(embed): map VA parameters to DC columns from the URL
Build / Build-and-ng-test (pull_request) Successful in 5m33s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m20s
Build / Build-and-test-development (pull_request) Successful in 32m46s
Two DC instances over the same table in one VA report could not filter the
same column from different parameters: parameter -> column matching is by
name, and two VA parameters cannot share a label.

The `va_parmap` URL parameter overrides that match per instance, one entry
per mapping (`va_parmap=<VA parameter>-<DC column>`, repeated for more):

  ...?embed=va&va_parmap=REGION_TAB1-REGION

The mapping is additive (unmapped parameters keep the name match), the
column is validated as a plain SAS name at parse time, and an entry only
applies where the named column exists in the table being opened - a miss
skips that parameter rather than falling back to the same-name match the
mapping exists to override. Entries split at the last hyphen, so a label
may itself contain hyphens, and the key matches the parameter label first,
then its internal name (pr###).

Parsed once at app init onto globals (like embed / vaApplyMode) so the
editor reloads a filter apply performs keep the mapping, and both filter
paths (the metadata build and the deferred initial load) honour it.

Also repairs embed-va.cy.ts: a cy.visit differing from the current URL
only in the hash is a same-document navigation, so ?embed=va never took
effect and both existing cases failed; their layout assertions also
described an earlier design (VA now opens read-only with an Edit button
and hides Filter). Cases 3 and 4 drive two mappings of the same column
through a real postMessage and assert the applied filter and row count.
2026-10-28 08:41:15 +00:00
dc c463a4e03c test(editor): assert the derived block marker, not a column name
Build / Build-and-ng-test (pull_request) Successful in 5m22s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m58s
Build / Build-and-test-development (pull_request) Canceled after 17m25s
derived-variables.cy.ts:97 expected dc-derived-first on the REGION header,
because the comment assumed REGION opens the computed block. The grid follows
MPE_DERIVED order, and that order is LINE_VALUE, REGION, REGION_LABEL - so the
marker was on LINE_VALUE and correct. The spec asserted the wrong column, and
the development pipeline has been red on it since.

Dumping the header row on a local mock estate (sasjs/server desktop mode,
RUN_TIMES=js) shows:

  6 | LINE_VALUE | dc-derived-header dc-derived-first
  7 | REGION | dc-derived-header
  8 | REGION_LABEL | dc-derived-header beforeHiddenColumn htLastVisibleHeader

and the same dump against the unmodified release/7.19 client, so this was never
a client defect.

The assertion now states the invariant instead: at least one derived header, the
first of them opens the block, and no other one carries the marker. That holds
whatever order the control table returns the variables in.
2026-10-27 22:21:52 +00:00
dc cfe6647ca8 feat(editor): show the derived column label in the header tooltip
Lighthouse Checks / lighthouse (pull_request) Canceled after 0s
Build / Build-and-ng-test (pull_request) Canceled after 0s
Build / Build-and-test-development (pull_request) Canceled after 0s
The hover text on a derived column said only that the column cannot be edited,
which tells the user nothing about what it holds.  It now carries the column
label - the configured VAR_LABEL, or the generated name=expression that
mp_derivedvars writes when the administrator left the label blank - and falls
back to the plain rule when the label merely repeats the column name.

The cypress case that marks derived columns apart now also checks the tooltip
for both shapes: REGION_LABEL carries a configured label, REGION falls back to
the generated one.
2026-10-27 19:36:51 +00:00
dc 6513d09f05 feat(editor): mark derived variables as computed in the EDIT grid
Derived variables are read-only, but they were rendered exactly like any other
read-only column: dimmed via readonlyCell, with no indication that the column is
computed at read time rather than locked by a rule or by column level security.
The grid could not tell the difference either - getdata holds a derived column
read-only with a READONLY DQ rule, which is also a rule type a table can be
configured with.

getdata now names the derived columns in sasparams.DERIVEDVARS (space separated,
upper case, from MPE_DERIVED's outvars). The EDIT screen holds that list, and
marks those columns with their own classes instead of the read-only dimming:
a tinted header with an 'fx' marker and a tooltip, tinted italic cells, and a
left edge on the first derived column so the computed block reads as a group.
Blue is the info family, leaving the primary key green and the warning ochre to
their existing meanings. Light and dark values are both set.

The SASjs mock getdata.js returns the same field, so the Cypress suite exercises
the styling; the existing derived-variables spec asserted readonlyCell on the
derived header, which is now the fallback for a plain read-only column, so that
assertion moves to dc-derived-header and a physical column stays the control.

Lint: prettier --check clean on the touched client files; sasjs lint reports no
new warnings for getdata.sas.
2026-10-27 14:34:13 +00:00
dc 1b13120839 Merge release/7.19 into fix/derived-readonly-and-mock-config
Build / Build-and-ng-test (pull_request) Successful in 5m40s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m7s
Build / Build-and-test-development (pull_request) Successful in 33m25s
#359 has landed, so the CLI bump this branch was waiting on is now in the base
and the merge brings it in. One conflict, in
client/cypress/support/commands.ts: both sides reworked loginAndUpdateValidKey.

The base version decides from the body captured before login, which is where
the original bug was - on an estate that shows a login form that capture has no
nav tree, so the command early-returned and never applied a key, and the upload
specs then found no file input. This branch re-reads the loaded page and treats
the licence banner as the signal. Kept this side; prettier re-indented the
result. Every other file merged automatically, and the hookErrmsg veto in
stagedata.js and loadfile.js survives the merge.
2026-10-26 08:34:32 +00:00
dc 5b70c000d7 test(derived): assert the read-only marker instead of inferring it
The case rested on an editor not appearing after a fixed wait, which passes for
any reason the editor did not open. The editor marks a read-only column with
readonlyCell on the header and on its cells, and STATE in the same grid is the
control, so the state is asserted directly and the sleep goes.

Mutating the mock to drop the READONLY rule fails the case on exactly that
assertion (expected '<th.darkTH>' to have class 'readonlyCell'); restoring it
passes 5/5 in about 50s.
2026-10-25 23:04:19 +00:00
dc e0ee99c178 test(derived): wait for the editor before asserting its absence
should('not.exist') passes the moment the element is missing, so a timeout on
it does not widen the window: an editor that opens after the assertion has
already passed goes unnoticed. The wait gives it that window, which is what the
case needs to rule the cell's editability out.
2026-10-25 21:39:19 +00:00
dc fa7a468fd3 test(derived): state the negative assertion's wait explicitly
Build / Build-and-ng-test (pull_request) Successful in 5m34s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m14s
Build / Build-and-test-development (pull_request) Successful in 33m49s
The window a 'not to exist' assertion covers is its timeout, so it is set
here rather than inherited from the config default.
2026-10-25 20:50:38 +00:00
dc d14395a0f8 test(derived): enter edit mode, and add the positive control
Build / Build-and-ng-test (pull_request) Successful in 4m52s
Lighthouse Checks / lighthouse (pull_request) Successful in 19m48s
Build / Build-and-test-development (pull_request) Failing after 33m17s
The screen opens read-only until the edit button is pressed, so the case
asserted nothing about the column: no cell opened an editor. It now enters edit
mode first, asserts the editor's absence directly rather than typing at
whatever happens to be focused, and a sibling case proves a physical column in
the same grid does open one - so the silence is about the column, not the
screen.

Mutating the mock to drop the READONLY rule fails the derived case and passes
the physical one; restoring it passes both (5/5, ~48s).
2026-10-25 20:30:09 +00:00
dc be8f445ae1 fix(cypress): apply the licence key when the estate has none
Build / Build-and-ng-test (pull_request) Successful in 5m31s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m36s
Build / Build-and-test-development (pull_request) Successful in 32m51s
loginAndUpdateValidKey decided whether a key was already working by looking
for the nav tree, and looked for it in the body captured before login.  A free
tier estate renders the tree as well, so on an unlicensed estate the command
took the 'already licensed' branch every time and never applied a key.  Every
upload spec then lost its feature set: onShowUploadModal shows the locked
feature dialog instead of the panel, so input#file-upload never renders.

The loaded page is read again after login, the licence banner decides whether
a key is needed, and an estate with no key is left untouched unless the caller
asked for one - the free tier specs are written against that state.

Measured on an unlicensed mock estate: excel.cy.ts 24 failing in 14m 1s before,
24 passing in 5m 44s after; csv.cy.ts 1 failing before, 1 passing after.
2026-10-25 19:31:14 +00:00
dc 5f443b0b16 fix(derived): hold a derived column read-only whatever the table's security
COLS.CLS_RULE is the only thing that marks a column read-only in the grid,
and the client applies it only when CLS_FLAG is set - which column level
security drives.  A table with no CLS rules therefore rendered its derived
columns as ordinary editable cells, although the values are computed at read
time and dropped from the submission.

getdata now unions a READONLY dq rule per derived variable, which the client
honours unconditionally, and the mock emits the same rule.  The SAS test
asserts the rule; the cypress spec double-clicks a derived cell and asserts the
value is unchanged.
2026-10-25 15:17:40 +00:00
dc 75b4acf1b1 test(cypress): upload the licence on a free-tier estate
Sites / deploy (push) Canceled after 1m40s
Build / Build-and-ng-test (pull_request) Successful in 5m43s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m10s
Build / Build-and-test-development (pull_request) Failing after 36m22s
The estate now ships no licence key at all - the placeholder word 'invalid' made
startupservice return an unusable key pair and produced the decrypt-failure page
on every HTTPS site. An unlicensed estate simply runs on the free tier.

loginAndUpdateValidKey therefore no longer infers 'already licensed' from the
nav tree: it detects the free-tier banner (#demo-banner) and uploads a key
through the licensing screen, which posts to admin/registerkey - the API the
licence page's upload button uses.
2026-10-24 20:30:22 +00:00
dc 489819b051 fix(mocks): address the audit-pack review findings
Sites / deploy (push) Successful in 3m52s
Build / Build-and-ng-test (pull_request) Successful in 5m27s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m5s
Build / Build-and-test-development (pull_request) Successful in 33m8s
- zip entries carry flag bit 11 when the name is non-ASCII, so an extractor
  reads `BKP_` plus an accented filename back by name instead of CP437
- loadfile and stagedata name the staged CSV after the libds the request
  arrived with, before a hook can re-point the submission
- stagedata writes that CSV at all (%mp_ds2csv does), so a grid submission's
  audit pack carries the same artefact as an upload's
- loadfile writes no BKP_ file: loadfile.sas copies one only for a second,
  non-CSV fileref, which the mock does not accept
- getauditfile says access is unenforced rather than that there is no
  permission store (MPE_SECURITY is seeded)
- the special-missings clip no longer claims the estate carries a valid key
2026-10-24 20:06:09 +00:00
dc 3445107d9a fix: name the startupservice launcher button "Show log"
Build / Build-and-ng-test (pull_request) Successful in 5m13s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m26s
Build / Build-and-test-development (pull_request) Successful in 31m20s
2026-10-23 03:01:23 +00:00
dc ad2da45389 fix: offer the launcher on the globvars failure modal too
Build / Build-and-ng-test (pull_request) Successful in 5m53s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m10s
Build / Build-and-test-development (pull_request) Successful in 31m25s
The globvars check in licence.service reports a failed startupservice run as
well, so its modal needs the service name for the new-window button - that was
the fourth startup failure path and the only one still a dead end.

Also guard the decodeURIComponent call in the Cypress drive/folder intercept,
so a lone '%' in a URL cannot throw inside the handler and abort the request.
2026-10-23 02:17:49 +00:00
dc de71b4456e chore: merge feat/derived-vars into the startupservice launcher branch
Build / Build-and-ng-test (pull_request) Successful in 5m43s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m45s
Build / Build-and-test-development (pull_request) Successful in 32m14s
2026-10-23 01:46:08 +00:00
dc 40f61331aa feat: run startupservice in a new window from its error modal
Build / Build-and-ng-test (pull_request) Failing after 1m56s
Build / Build-and-test-development (pull_request) Skipped
Lighthouse Checks / lighthouse (pull_request) Successful in 19m25s
When the startupservice fails, the modal is a dead end: the adapter returned
no log, so there is nothing to act on. The modal now offers a button that runs
the same service through the JES / STP web app in a new browser window, where
the SAS log is returned as the page body.

`_debug=131` is always added (the log is the point), and `_executionTasks` is
never included, so on Viya the run is not submitted as a Compute task and
cannot hit the execution-task timeout. The URL is built by `buildServiceUrl`,
which works for SAS 9, Viya and SASjs Server, and is the same shape the deploy
configurator already uses to open makedata.

The startup failure modals now carry the service name, which is what shows the
button: all three failure branches (malformed response, missing properties,
request rejected) pass `public/startupservice`.
2026-10-23 01:15:31 +00:00