Commit Graph
1660 Commits
Author SHA1 Message Date
dc 0f729d7101 fix(report): treat a blank PDF footer parameter as its default
Build / Build-and-ng-test (pull_request) Successful in 5m5s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m47s
Build / Build-and-test-development (pull_request) Successful in 28m54s
A parameter named without a value arrives blank, and a blank reached the
footer's numeric comparisons as nothing to evaluate:

    %if &filter eq 0                     /* blank filter */
    %if %eval(&nobs) gt %eval(&maxobs)   /* blank obs count */

That raised %EVAL arithmetic errors, stopped mpe_pdffootnote and failed the
whole download instead of producing a report.  Both are now normalised to
their defaults first, maxobs included.

The footer's user= is taken from mf_getuser() at the top of the service - it
is deliberately not an input, so a caller cannot assert who ran the report -
and it is passed through %superq because a display name may hold a comma,
which the macro call would otherwise read as a parameter separator.

Test 5 gains a case for the blank call.
2026-11-03 08:47:41 +00:00
dc 67ca129c4d feat(viewer): download a server-side PDF report
Build / Build-and-ng-test (pull_request) Successful in 5m41s
Lighthouse Checks / lighthouse (pull_request) Successful in 19m0s
Build / Build-and-test-development (pull_request) Successful in 28m53s
Adds PDF to getrawdata: the staged, filtered dataset is rendered by
mp_ds2pdf and streamed with contenttype=PDF, so the Download modal gains
a typeset report - landscape, SAS formats and labels applied, headers
repeated on every page, 'Page n of m', and the filter, user and timestamp
in the footer.

The report is capped at 2000 observations, about 75 landscape pages, and
the footer states the truncation when it bites.  The cap is one macro
variable driving both maxobs= and the footer text, so the two cannot
disagree, and the footer itself is composed by mpe_pdffootnote so the
truncation wording can be asserted without reading text out of a PDF.

@sasjs/core goes 7.0.3 -> 7.1.0 for the macro; the bump also brings
mp_validatecol ISPATH, two mp_dirlist guards and an mx_execute test.

On Viya the report renders but the download does not arrive: mp_streamfile
fails ahead of the PDF branch inside mp_binarycopy (RECFM=N with lrecl=1
needs 256), which is upstream of this change and left alone here.
2026-11-02 23:08:06 +00:00
allan f82f21dc3b Merge pull request 'docs(install): drop the root install script this repo no longer has' (#438) from docs/install-section-current into release/7.22
Sites / deploy (push) Successful in 4m46s
Build / Build-and-ng-test (pull_request) Successful in 6m32s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m38s
Build / Build-and-test-development (pull_request) Successful in 31m15s
Reviewed-on: #438
2026-11-02 09:39:17 +00:00
allan 1b06857213 Merge pull request 'fix(editor): a container resize must not clear an active sort' (#433) from fix/editor-resize-keeps-sort into release/7.22
Sites / deploy (push) Successful in 4m7s
Build / Build-and-ng-test (pull_request) Successful in 5m39s
Lighthouse Checks / lighthouse (pull_request) Successful in 19m25s
Build / Build-and-test-development (pull_request) Successful in 31m18s
Reviewed-on: #433
2026-11-02 08:37:30 +00:00
dc e9a2abd2e2 docs(install): drop the root install script this repo no longer has
Build / Build-and-ng-test (pull_request) Successful in 6m31s
Lighthouse Checks / lighthouse (pull_request) Successful in 19m4s
Build / Build-and-test-development (pull_request) Successful in 35m52s
The root `package.json` carries no install or postinstall script, so the
guidance to rely on "the composite install script ... which installs both
subprojects" sends the reader after something that does not exist, and
`client/.npmrc` told them the same thing in a comment.

The install section also contradicted itself: it opened by telling the reader
to write the SheetJS registry token into `client/.npmrc`, and closed by
explaining that `client/.npmrc` is a tracked policy file that must not carry a
token (the pre-commit hook rejects one). The token goes in the user config,
which is what the first block now says.

The install sequence itself is unchanged - root `npm ci`, then `client` and
`sas` explicitly - because that is what the repo actually needs.
2026-11-02 08:33:24 +00:00
dc ba4ddb06d3 test(editor): cover the resize observer wiring, not just the helper
Build / Build-and-ng-test (pull_request) Successful in 6m28s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m1s
Build / Build-and-test-development (pull_request) Successful in 31m39s
The helper-only spec passes even if the observer goes back to calling
updateSettings() - which is the regression the helper exists to prevent.
Move the observer into refreshGridAfterContainerResize.ts so a spec can
drive a real container resize through it, and assert that the resize
reaches refreshDimensions() without any settings call. Verified by
re-injecting the old wiring: the spec fails with 'Expected spy
updateSettings not to have been called', and passes once it is removed.

Also corrects the helper's claim about what refreshDimensions() measures:
it compares rootElement.getBoundingClientRect() with the last measured
size, and both grids take their height from a viewport-derived calc(), so
a resize that matters is a width or viewport change.
2026-11-01 23:27:41 +00:00
dc 5bd7b5d724 fix(editor): a container resize must not clear an active sort
Build / Build-and-ng-test (pull_request) Successful in 6m37s
Build / Build-and-test-development (pull_request) Canceled after 0s
Lighthouse Checks / lighthouse (pull_request) Canceled after 5m53s
The grid's container ResizeObserver (added in c7ba025 to replace the
afterRender window-resize dispatch of #246) refreshed by calling
updateSettingsSortSafe({ height }), and that wrapper's whole reason for
existing is that any updateSettings() while a multi-column sort is active
corrupts formula references into #REF!. It works around that by clearing the
sort, applying the settings and re-applying the sort - so every sidebar
toggle, modal open/close or window resize re-sorted the user's data, and an
open cell editor (anchored to visual coordinates) could end up writing to a
different row than the one being typed into.

refreshGridAfterContainerResize() replaces it with hot.refreshDimensions():
it compares the element's box with the last known size, re-measures the
overlays and redraws only when the size actually moved, touching no settings.
The decision lives in its own module so the spec can pin it - a real
Handsontable instance with an active sort and a formula column asserts that
the resize path calls refreshDimensions() and does not call updateSettings(),
clearSort() or sort(), that the sort config is unchanged, and that no cell
degrades to #REF!.

This is the narrow, independent piece of #246; the wider removal of the
remaining synthetic window resize dispatches (sidebar, viewboxes) is analysed
on the issue and is not part of this change.
2026-11-01 22:25:22 +00:00
allan 663088ad16 Merge pull request 'feat(viewer): export the grid as PDF from the context menu' (#430) from feat/238-pdf-export into release/7.22
Sites / deploy (push) Successful in 4m8s
Build / Build-and-ng-test (pull_request) Successful in 5m54s
Lighthouse Checks / lighthouse (pull_request) Successful in 19m8s
Build / Build-and-test-development (pull_request) Successful in 35m52s
Reviewed-on: #430
2026-11-01 22:19:24 +00:00
allan 0d8fa59e81 Merge pull request 'feat(logs): read the logs modal with the shared log parser' (#431) from feat/sas-logs-modal into release/7.22
Sites / deploy (push) Canceled after 0s
Build / Build-and-ng-test (pull_request) Successful in 5m39s
Lighthouse Checks / lighthouse (pull_request) Failing after 21m22s
Build / Build-and-test-development (pull_request) Successful in 31m37s
Reviewed-on: #431
2026-11-01 22:18:15 +00:00
dc b5bb704a4f fix(viewer): defer the blob URL revoke, reject a selection inside skipped columns
Build / Build-and-ng-test (pull_request) Successful in 6m24s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m53s
Build / Build-and-test-development (pull_request) Successful in 35m51s
Two findings from review.

downloadTablePdf() revoked the blob URL in the same task as the anchor click.
Firefox and Safari read the blob asynchronously, so that can cancel the
download and leave the user with no file; the revoke now runs on a later turn,
after the click has been handed to the download machinery.

getExportRange() clamped the selection's left edge up to skipLeadingCols but
then forced the right edge up to match, so a selection lying entirely inside
the leading housekeeping column(s) was shifted sideways into a column the user
never selected - the PDF contained column 1 when they had selected column 0.
The range is now clamped into the exportable columns and comes back empty
(right < left) when nothing is left, which the PDF path reports as "no table"
and the CSV/Excel path skips rather than exporting the wrong column.

The revoke change is only covered by the spec (it needs a browser to exercise);
the range change is covered by the spec and by the local harness, now 28 checks.
2026-11-01 21:57:55 +00:00
dc aabc578d5e feat(logs): read the logs modal with the shared log parser
Build / Build-and-ng-test (pull_request) Successful in 6m34s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m44s
Build / Build-and-test-development (pull_request) Successful in 32m1s
The modal decided what was wrong with two regular expressions of its own, and the answer disagreed with the log it was showing:

- an ODS HTML line was cut from the first capital `E` in it and lost its last character, so an error read `ERROR: File WORK.X.DATA does not exist.</font>`;
- a message SAS wrapped over several lines was reported once, truncated at the break, and the rest of it was not reported at all;
- the colon-less form SAS writes (`ERROR Libref X is not assigned.`) was missed on any line carrying markup;
- the notable notes - uninitialised variable, invalid data, a format too small - were not shown anywhere;
- clicking an issue searched the rendered text for its message, so a log that repeats a message (two `ERROR 22-322:` lines is ordinary) always jumped to the first of them.

`parseLog` from `@sasjs/utils` replaces all of it, the same function the VS Code extension and the SASjs server Studio use, so one log now reads identically in every host. Each request keeps the issues the parser reported, grouped by severity, and the tree gains a Notes list beside Errors and Warnings. The click scrolls to `issue.line` and flashes from that line to `issue.endLine`, and the lists no longer render at zero.

The modal is SAS Logs now, as are the menu entry, the deploy button and the abort modal's button; the request history is unchanged underneath.

`formatTimestamp` decided on `moment().format()`, which is 'Invalid date' - truthy - for anything it cannot parse, so its fallback could never fire. It asks `isValid()` now.

The warning flash was yellow where the warning text is ochre; both are ochre now, and the notes flash blue.

`@sasjs/utils` goes to 4.3.1, which is where the parser lives.
2026-11-01 21:45:08 +00:00
dc 4528d26927 test(viewer): give the rowless-grid fixture its columns
Build / Build-and-ng-test (pull_request) Successful in 6m28s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m46s
Build / Build-and-test-development (pull_request) Successful in 32m25s
The new empty-grid spec built a grid from `data: []` alone, and Handsontable
reports zero columns when the data is empty and no `columns` setting defines
the schema - so buildGridPdfInput() correctly returned null and the spec failed
in CI while passing against the stub harness. The real grids declare their
columns, so the fixture does now too.
2026-11-01 21:09:23 +00:00
dc 10229d5efe fix(viewer): clamp over-tall PDF rows, always emit an otherwise-empty page
Build / Build-and-ng-test (pull_request) Failing after 3m40s
Build / Build-and-test-development (pull_request) Skipped
Lighthouse Checks / lighthouse (pull_request) Successful in 19m2s
Two findings from the second review.

A row taller than the printable area was drawn straight through the footer and
off the page, losing the tail of the cell unannounced - MAX_COL_CHARS bounds a
column, not a value, so one long text field wraps to more lines than a page
holds. Rows are now clamped to the tallest row that fits the page, with ' ...'
marking the cut, and the height budget accounts for both margins so an
exactly-fitting row stays on the page it started on instead of leaving the
first page with headers only.

exportGridPdf() also returned silently for a grid with no rows, although the
writer renders an empty table as a valid header-only page. The input is now
built by buildGridPdfInput(), which returns null only when there are no
columns, so a rowless grid downloads a page saying "0 rows x N columns".

Tests: the tall-row spec asserts the marker, the single page and that no text
start falls outside the printable band; the empty-grid spec asserts the input
is built rather than skipped.
2026-11-01 21:02:52 +00:00
dc 2c4be5e94b fix(viewer): review findings - CSV skipped column, PDF row cap before reading
Build / Build-and-ng-test (pull_request) Successful in 5m35s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m50s
Build / Build-and-test-development (pull_request) Successful in 31m35s
Two findings from review.

First, a regression in the CSV/Excel path: getExportRange() defaults `left`
to skipLeadingCols, so comparing it against skipLeadingCols made every
no-selection export look like a whole-grid export and the plugin put the
leading housekeeping column back in. The whole-grid test now compares the left
edge against visual column 0, so a skipped column stays out.

Second, the PDF row cap was applied after collectGridTable() had already read
every cell, so a very large grid still froze before the cap took effect.
collectGridTable() now takes a maxRows limit that clamps the row loop before
any cell is read, and exportGridPdf() still reports the true total in the
subtitle.

Both are covered: a spec asserts the CSV export receives range
[0, skipLeadingCols, lastRow, lastCol], and one asserts the cap stops cell
reads rather than collecting and slicing.
2026-11-01 20:22:16 +00:00
dc f84d4ca7af fix(viewer): keep the PDF export working where hiddenColumns is not enabled
Build / Build-and-ng-test (pull_request) Successful in 6m33s
Lighthouse Checks / lighthouse (pull_request) Successful in 18m48s
Build / Build-and-test-development (pull_request) Successful in 35m20s
CI caught this on the real grid specs: getPlugin('hiddenColumns') returns the
plugin instance even when it is switched off, and isHidden() then dereferences
a map that enablePlugin() never created, so any grid without a hiddenColumns
setting threw instead of exporting.

Ask the plugin's enabled flag first and treat a failure as "visible" - an
export is better off with a column the user could not see than with an error
dialog. The spec adds a regression case for exactly the CI failure, and its
afterEach no longer destroys an already-destroyed instance when a test creates
no grid.
2026-11-01 19:24:39 +00:00
dc e58e735610 feat(viewer): export the grid as PDF from the context menu
Build / Build-and-ng-test (pull_request) Failing after 3m11s
Build / Build-and-test-development (pull_request) Skipped
Lighthouse Checks / lighthouse (pull_request) Successful in 18m51s
Adds a PDF entry to the right-click Export submenu, next to CSV and Excel,
so the grid as displayed can leave the browser as a printable document.

Implemented directly against PDF 1.4 rather than adding a PDF library: the
output is a text table, which needs no embedded fonts, no images and no
compression. The base-14 Courier font is used because its glyphs are exactly
600/1000 em wide, so column widths are arithmetic on character counts instead
of a font-metrics table - a 1200 x 12 grid is laid out in ~60 ms.

Layout: A4 landscape, bold header band repeated on every page with "Page n of
m" footers, numeric columns right-aligned, values word-wrapped inside their
column, and long tables paginated. Wide tables are fitted by shrinking the
body font first (7pt down to 5pt, so a 20-column table keeps full headers at
6pt) rather than squeezing columns; only at 5pt are columns scaled, and they
stop at 4 characters - past that trailing columns are dropped and the first
page says how many were kept, so nothing is omitted silently.

Encoding is WinAnsi, the widest set the base-14 fonts cover without embedding;
characters outside cp1252 print as '?'. A typeset, branded report is a
different feature and belongs on the server, where SAS formats and DC metadata
are available.

The export keeps the existing "what you see" contract: current selection (when
it spans more than one cell), otherwise the whole grid, hidden columns
omitted, leading housekeeping columns skipped. The range calculation that CSV
and Excel already used is now a shared getExportRange(), so the three formats
cannot drift apart. Rows are capped at 5000 and the subtitle says so.

Tests: pdf-export.util.spec.ts covers document structure (xref offsets, page
count, footer numbering), string escaping, WinAnsi mapping and fallback,
wrapping, alignment, multi-line values, pagination, font fitting and the
page-width invariant; hot-export.util.spec.ts covers grid extraction against a
real Handsontable instance.
2026-11-01 19:19:28 +00:00
allan 440ea9dc0f Merge pull request 'ci: fail a client job fast on a runner too small for it' (#429) from ci/runner-memory-guard into release/7.22
Sites / deploy (push) Successful in 4m8s
Build / Build-and-ng-test (pull_request) Successful in 6m32s
Lighthouse Checks / lighthouse (pull_request) Successful in 19m0s
Build / Build-and-test-development (pull_request) Successful in 35m35s
Reviewed-on: #429
2026-11-01 18:56:06 +00:00
dc f3d85d49bf ci: make the memory check report its own input errors clearly
Build / Build-and-ng-test (pull_request) Successful in 5m45s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m47s
Build / Build-and-test-development (pull_request) Successful in 31m22s
A MIN_TOTAL_MB typo reached the integer test under set -e and aborted with "integer expression expected"; it now prints usage and exits 2. A kernel without MemAvailable no longer reports 0 MB available in the summary or the warning, so the oversubscription warning stays meaningful.
2026-11-01 18:06:28 +00:00
dc b1892ebb98 ci: gate every job that builds the frontend, and scope the check in its header
Build / Build-and-ng-test (pull_request) Successful in 5m37s
Lighthouse Checks / lighthouse (pull_request) Successful in 19m3s
Build / Build-and-test-development (pull_request) Successful in 35m31s
release.yaml (all three jobs), sites.yaml and release-assets.yaml run the same cold production build that the review jobs do, so they get the same pre-install check.

The header now says what the check rules out (an undersized box) and what it does not (two jobs sharing one box), so the MemAvailable warning is not read as a gate.
2026-11-01 16:24:03 +00:00
dc ab4f4c9b60 ci: fail a client job fast on a runner too small for it
Build / Build-and-ng-test (pull_request) Successful in 6m25s
Lighthouse Checks / lighthouse (pull_request) Successful in 19m12s
Build / Build-and-test-development (pull_request) Successful in 35m37s
An undersized runner does not fail honestly: the kernel OOM killer takes
the build, so the job reports exitcode 137 at ng build, or Karma
disconnects mid-suite after twenty minutes. Both read as a flaky test
rather than as a box that cannot hold the job, and both cost a whole CI
slot.

The check runs first in the three client jobs (Build-and-ng-test,
Build-and-test-development, lighthouse), prints total and available RAM,
vCPU count and any cgroup limit, and exits 1 when the effective size is
under 8 GB - the supported minimum. A cold production build measures
3.8 GB peak on its own, so 8 GB leaves roughly 2x headroom.
2026-11-01 15:47:48 +00:00
allan 2ee3c2241b Merge pull request 'test(excel): guard that discarding a failed upload clears the red cells' (#427) from fix/122-discard-excel-clears-red-cells into release/7.22
Sites / deploy (push) Successful in 4m40s
Build / Build-and-ng-test (pull_request) Successful in 6m29s
Lighthouse Checks / lighthouse (pull_request) Failing after 7m36s
Build / Build-and-test-development (pull_request) Failing after 12m7s
Reviewed-on: #427
2026-11-01 14:49:51 +00:00
allan cfa72ff879 Merge pull request 'fix(multi-dataset): show the file size and loading stage while the file loads' (#426) from fix/121-multi-load-file-stage into release/7.22
Sites / deploy (push) Canceled after 44s
Build / Build-and-ng-test (pull_request) Successful in 6m27s
Lighthouse Checks / lighthouse (pull_request) Failing after 15m13s
Build / Build-and-test-development (pull_request) Successful in 31m28s
Reviewed-on: #426
2026-11-01 14:49:07 +00:00
dc cf759db3d1 test(excel): guard that discarding a failed upload clears the red cells
Lighthouse Checks / lighthouse (pull_request) Failing after 11m39s
Build / Build-and-ng-test (pull_request) Successful in 5m41s
Build / Build-and-test-development (pull_request) Successful in 31m23s
An uploaded file's preview is validated in place, so the cells that fail
a rule carry Handsontable's invalid cell meta. Discarding the file
restores the loaded data, but cell meta is keyed by row/column and
survives that restore - without a re-validation the grid keeps painting
cells that are no longer invalid.

cancelEdit re-runs the validation (hot.validateCells()) after restoring
the data source, which is what clears them; this test drives the flow and
pins it: import a file that fails validation, discard it, then assert the
row count is back, no .htInvalid cell remains and the page is read-only
again. The manual-edit path (test 25) keeps the parsed values by design.

Refs #122
2026-11-01 13:29:13 +00:00
dc 45d6c025c9 fix(multi-dataset): show the file size and loading stage while the file loads
Build / Build-and-ng-test (pull_request) Successful in 6m31s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m37s
Build / Build-and-test-development (pull_request) Successful in 35m39s
The multi-load page rendered the loading stage only inside a branch guarded by
`parsedDatasets.length`, and that collection is empty for the whole read and
sheet-scan phase - so two of the four stage labels ("Reading the file",
"Searching for the data in the file") were assigned but never reached the DOM,
and a large file sat on screen with no feedback at all.

The stage is now shown next to the file name and size from the moment a file is
attached, and again in the content area while the sheets are being parsed. The
labels cover every phase the user waits through: reading the file, searching
for the data in the file, file ready to load, fetching the dataset details (one
getdata call per dataset - the longest wait), extracting the data, and
extracting the data finished. A spinner accompanies the stages that have work
running; the idle stages and the finished state show the label alone.

Two corrections fall out of that: the stage only moves to `extracted` once
every sheet has been parsed rather than when the first one finishes, and the
read's continuation only advances the stage if the read is still the phase in
progress (the sheet scan runs in parallel and can finish first).

excel-multi-load.cy.ts gains test 4, which asserts the file size, the ready
stage, the fetching stage (the getdata calls are held open so it is on screen
long enough to assert) and the handover to the "select a dataset" hint.

Closes #121
2026-11-01 11:22:16 +00:00
allan f3bbcc8a9f Merge pull request 'chore(lint): enable noMacroIfInOpenCode and move the %if blocks into macros' (#425) from fix/lint-no-macro-if-in-open-code into release/7.22
Sites / deploy (push) Successful in 4m8s
Build / Build-and-ng-test (pull_request) Failing after 8m23s
Build / Build-and-test-development (pull_request) Skipped
Lighthouse Checks / lighthouse (pull_request) Successful in 19m8s
Reviewed-on: #425
2026-11-01 10:47:45 +00:00
dc 9f2cd24b20 chore(lint): require %if to sit inside a macro definition
Build / Build-and-ng-test (pull_request) Successful in 6m23s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m28s
Build / Build-and-test-development (pull_request) Successful in 31m26s
Enables noMacroIfInOpenCode and promotes it to "error". Open-code %if is a SAS
9.4M5 feature, so the 19 sites it found in nine files were hiding a version
requirement the codebase does not document (the requirements page says 9.4M3).
Each is rewritten so the construct no longer appears in open code:

- Six blocks become a small macro called unconditionally, which is where the
  conditional then lives: drop_cas_tables (demodata), add_derived_cols
  (getdata), build_sasdata4 (getdata test 4), drop_derived (stagedata),
  build_staged (getrawdata), and one per option in the 20261003 migration.
- Four blocks in the two makedata STPs get the same treatment: check_outfmt,
  check_keepme, emit_response and self_destruct. Their `call symputx` writes
  are now explicit about the global symbol table, because a %let or a plain
  symput inside a wrapper would be local to it.
- Two guards in stagedata and getrawdata are removed rather than wrapped: the
  SQL only selects into a list that is already %let blank, and with no derived
  variables it returns no rows, so the guard was doing nothing. The trailing
  drop keeps a conditional, now in a macro and tested with %trim, since a
  zero-row select can leave a single blank in the variable.
- validatefilter's two %let branches become one data step with call symputx,
  which writes to the global table the later %mp_filterstore reads.

Verified: `npm run lint:sas` clean with the rule as an error; `sasjs compile`
succeeds; on sas.4gl.io the full suite passes 64/64, including the derived
variable tests in getdata test 4, stagedata tests 3 and 4 and validatefilter,
which are the paths these wrappers touch.
2026-11-01 10:41:52 +00:00
allan 0528a229b7 Merge pull request 'chore(lint): enforce 80 characters and make every rule an error' (#424) from fix/lint-max-line-length into release/7.22
Sites / deploy (push) Canceled after 0s
Build / Build-and-ng-test (pull_request) Successful in 6m23s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m54s
Build / Build-and-test-development (pull_request) Failing after 23m9s
Reviewed-on: #424
2026-11-01 10:29:25 +00:00
dc 90fa2e3b74 chore(lint): raise every lint rule to error severity
Build / Build-and-ng-test (pull_request) Successful in 5m29s
Lighthouse Checks / lighthouse (pull_request) Successful in 19m9s
Build / Build-and-test-development (pull_request) Successful in 31m18s
The tree is clean under all of them, so the remaining warn-level rules can be
promoted without a single code change: hasDoxygenHeader, hasMacroNameInMend,
hasMacroParentheses, noNestedMacros, strictMacroDefinition, noSpacesInFileNames,
lowerCaseFileNames, noTabIndentation, noTrailingSpaces, noGremlins and
noNestedCommentsInHeader.

With that, every rule @sasjs/lint 4.9.0 can raise is an error - only
noEncodedPasswords was an error by default - so any new violation fails
`npm run lint:sas` and the CI step that runs it, instead of printing a warning
that leaves the exit code at 0.

Checked: a trailing space appended to a file reports
"Error  Line contains trailing spaces" and a non-zero exit, where it used to be
a Warning.
2026-11-01 10:25:50 +00:00
dc 33efc5ae1e chore(lint): enforce an 80 character line and give the oversize files exact ceilings
Build / Build-and-ng-test (pull_request) Successful in 5m42s
Lighthouse Checks / lighthouse (pull_request) Successful in 18m48s
Build / Build-and-test-development (pull_request) Failing after 13m0s
sas/.sasjslint takes maxLineLength from 180 to 80 - the linter's own default -
and promotes it to "error".

53 of the 258 .sas files already hold a line longer than 80. Rewrapping them
would be a large, reviewable-only-in-theory diff, so each keeps its current
longest line as an exact ceiling via a per-file override, e.g.
@sasjslint {"maxLineLength": 95} in services/validations/tables_all.sas. A
file with no override is held to 80, and an overridden file cannot grow past
the length it has today: the ceiling is the file's longest line, not a round
number.

Both directions were checked: a 101 character line added to an
un-overridden file fails with "Line exceeds maximum length by 21 characters",
and a 114 character line added to macros/mpe_execute.sas (ceiling 101) fails
by 13. `npm run lint:sas` is clean with the setting in force.
2026-11-01 10:21:06 +00:00
allan e1a33c9ae9 Merge pull request 'chore(lint): normalise line endings and make lineEndings an error (ratchet 6/n, #376)' (#423) from fix/lint-line-endings into release/7.22
Sites / deploy (push) Canceled after 0s
Build / Build-and-ng-test (pull_request) Successful in 5m39s
Lighthouse Checks / lighthouse (pull_request) Successful in 19m2s
Build / Build-and-test-development (pull_request) Successful in 35m25s
Reviewed-on: #423
2026-11-01 10:05:50 +00:00
dc 0662f6230b chore(lint): normalise line endings and make lineEndings an error
Build / Build-and-ng-test (pull_request) Successful in 6m25s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m42s
Build / Build-and-test-development (pull_request) Successful in 31m10s
Rung 6 of the #376 ratchet: `"lineEndings": "lf"` is enabled and promoted to
"error". Its 37 findings are cleared and nothing but line endings changed.

- 34 files ended without a newline; each now ends with exactly one LF.
- 3 files were stored with CRLF (services/public/getgroups.sas and the two
  sas9 lineage services, getmetacols and getmetatables), against the repo's
  own `* text=auto eol=lf` in .gitattributes. They are converted to LF.

Checked mechanically that this is whitespace only: for every changed file,
HEAD's content and the new content are identical once CRLF is folded to LF
and the trailing newline is stripped.

VERIFICATION

- `npm run lint:sas`: 0 warnings, 0 errors, exit 0.
- `sasjs compile`: succeeds.
- On sas.4gl.io, against a scratch appLoc seeded with makedata:
  "Tests that pass: 64/64 (100%)".
2026-11-01 09:28:58 +00:00
allan a232b2ee5a Merge pull request 'fix(lint): enable noSingleAsteriskComments and noUnusedLibnames (ratchet 5/n, #376)' (#422) from fix/lint-asterisk-comments into release/7.22
Sites / deploy (push) Canceled after 11s
Build / Build-and-ng-test (pull_request) Successful in 5m37s
Lighthouse Checks / lighthouse (pull_request) Successful in 19m44s
Build / Build-and-test-development (pull_request) Successful in 35m5s
Reviewed-on: #422
2026-11-01 09:28:48 +00:00
dc 9da9f88ed1 docs(agents): record why noMacroIfInOpenCode stays off
Build / Build-and-ng-test (pull_request) Successful in 6m30s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m43s
Build / Build-and-test-development (pull_request) Successful in 35m15s
The rule flags the %if blocks service bodies run in open code, which need
SAS 9.4M5. The documented minimum is 9.4M3, so enabling it would mean
raising the floor or wrapping every service body in a macro - neither is
a lint change. Written down so the next agent does not enable it by
mistake.
2026-11-01 09:10:08 +00:00
dc 9071801ac5 fix(lint): enable noSingleAsteriskComments and noUnusedLibnames
Rung 5 of the #376 ratchet. Both rules are off by default; enabling them
surfaced 10 findings, all now cleared, and both are promoted to "error".

- Nine `* text;` comment statements become block comments: three disabled
  debug lines (a putlog in bitemporal_dataloader, an alternative type
  assignment in mpe_loader, a put in demodata) and six explanatory
  comments in the two sas9 usernav services.
- loadfile.sas assigned `libname approve "&dir"` and never used the
  libref; stagedata.sas and restore.sas assign the same libref because
  they write approve.jsdset, loadfile writes through a fileref instead.
  The statement is removed.

VERIFICATION

- `npm run lint:sas`: 0 warnings, 0 errors, exit 0.
- `sasjs compile`: succeeds.
- On sas.4gl.io, against a scratch appLoc seeded with makedata:
  "Tests that pass: 64/64 (100%)".
2026-11-01 09:10:08 +00:00
allan de5cb029eb Merge pull request 'fix(lint): rename the short and mixed-case helper macros, and clear the last warnings (ratchet 4/n, #376)' (#421) from fix/lint-macro-names into release/7.22
Sites / deploy (push) Canceled after 0s
Build / Build-and-ng-test (pull_request) Successful in 5m36s
Lighthouse Checks / lighthouse (pull_request) Successful in 18m45s
Build / Build-and-test-development (pull_request) Successful in 31m19s
Reviewed-on: #421
2026-11-01 09:06:08 +00:00
dc 7da67eb17b fix(lint): rename the short and mixed-case helper macros, and clear the last warnings
Build / Build-and-ng-test (pull_request) Successful in 6m23s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m45s
Build / Build-and-test-development (pull_request) Successful in 35m22s
Rung 4 of the #376 ratchet. It clears the last 5 findings and promotes the
three rules that produced them to "error": minMacroNameLength,
lowerCaseMacroNames and indentationMultiple. `sasjs lint` now reports no
warnings and no errors across the project.

- viewdata.sas wraps its service body in a helper macro named `x`; it
  becomes viewdata_body. The wrapper is what keeps the %global and the
  %return in one macro scope.
- usergroupsbymember.sas (sas9) builds work.iwant inside a helper macro
  named `x`; it becomes build_iwant.
- refreshtablelineage.sas (sas9) defines `getTables` with uppercase; it
  becomes `gettables`, at the definition, the %mend and its seven calls.
- getstagetable.test.sas: one comment line carried three leading spaces
  where the file uses two.

VERIFICATION

- `npm run lint:sas`: 0 warnings, 0 errors, exit 0, "All matched files
  use @sasjs/lint code style!".
- `sasjs compile`: succeeds.
- Deployed to sas.4gl.io and ran the SAS suite there against a scratch
  appLoc (/Public/app/dc-lint-ratchet) seeded with makedata: "Tests that
  pass: 64/64 (100%)". The viewdata and usergroupsbymember suites are in
  that run, so the two renames are covered by passing tests.
2026-11-01 08:47:05 +00:00
allan a15bf99462 Merge pull request 'fix(lint): make noBannedStrings an error and clear its 13 findings (ratchet 3/n, #376)' (#420) from fix/lint-no-banned-strings into release/7.22
Sites / deploy (push) Successful in 4m10s
Build / Build-and-ng-test (pull_request) Successful in 6m59s
Lighthouse Checks / lighthouse (pull_request) Successful in 19m2s
Build / Build-and-test-development (pull_request) Successful in 31m17s
Reviewed-on: #420
2026-11-01 08:46:48 +00:00
dc 0b85f7bf1e fix(lint): make noBannedStrings an error and clear its 13 findings
Build / Build-and-ng-test (pull_request) Successful in 6m42s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m38s
Build / Build-and-test-development (pull_request) Successful in 35m26s
Rung 3 of the #376 ratchet: `sas/.sasjslint` promotes noBannedStrings to
"error". The rule flags the literal words "error " / "warning " anywhere in
a file, comments included, because they turn up in a grep of the SAS log
and bury the real messages. Its 13 findings are cleared:

- Nine are prose and are reworded, so the sentence both reads normally and
  stops matching: a truncation that aborts a build, a syntax fault, an
  abort condition, "only records the failure in work.mp_abort_errds" (four
  copies of the mp_abort-inside-%include comment), a test description, a
  hook comment.
- Two are the text the user sees. loadfile.sas reports
  "(mf_abort.error file found)" - the marker file carries the word in its
  name - and buildtermsas9.sas reports "Some final error in &_program";
  both are reworded to say the same thing without the literal, since the
  text sits in a macro argument where the %str(WARN)ING split does not
  survive the quoting.
- Three keep the word because the word is the content: the doxygen
  @warning tags in macros/bitemporal_dataloader.sas and the two SAS 9
  admin STPs (makedata, makelib). Each carries
  @sasjslint {"noBannedStrings": false}, as
  targets/server/services_server/admin/makedata.sas already did.

VERIFICATION

- `npm run lint:sas`: 5 warnings, 0 errors, exit 0.
- `sasjs compile`: succeeds.
- Deployed to sas.4gl.io and ran the SAS suite there:
  `sasjs test` reports "Tests that pass: 64/64 (100%)". The run used a
  scratch appLoc (/Public/app/dc-lint-ratchet) seeded with makedata, so
  the shared /Public/app/dc deployment was left alone; the scratch target
  was removed from sasjsconfig.json before this commit.

5 warnings remain: macro name length (2) and case (2),
indentationMultiple (1) - the last rung.
2026-11-01 01:52:06 +00:00
allan 4c09c02998 Merge pull request 'fix(lint): make noUndeclaredMacros an error, and fix what it found (ratchet 2/n, #376)' (#419) from fix/lint-no-undeclared-macros into release/7.22
Sites / deploy (push) Successful in 4m8s
Build / Build-and-ng-test (pull_request) Successful in 5m33s
Lighthouse Checks / lighthouse (pull_request) Successful in 18m45s
Build / Build-and-test-development (pull_request) Successful in 31m6s
Reviewed-on: #419
2026-11-01 01:41:33 +00:00
dc af28a79bd5 fix(lint): make noUndeclaredMacros an error, and fix what it found
Build / Build-and-ng-test (pull_request) Successful in 6m26s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m36s
Build / Build-and-test-development (pull_request) Successful in 35m20s
Rung 2 of the #376 ratchet: `sas/.sasjslint` promotes noUndeclaredMacros to
"error". Its 25 findings on release/7.22 are cleared, and two of them were
real faults rather than missing prose.

WHAT WAS BROKEN

Comparing the compiled program against the macros it calls (compile, then
list every project macro each generated service calls but does not define)
found three services calling macros that were absent from their own
compiled program:

- services/validations/tables_all.sas calls %mf_getuniquename() on its
  third line and never declared it.
- services/validations/mpe_tables.dsn.sas calls %mf_getuniquename() and
  %dc_assignlib() and declared neither.
- services/hooks/mpe_tables_postedit.sas calls %dc_assignlib(), which is
  supplied by the service that includes the hook (see below).

The two validation services are configured as rule_value entries in
MPE_TABLES (mpe_makedata.sas and the v3.12/v3.13 migrations), so the calls
run when a user reviews or filters a library - each would have raised
"Apparent invocation of macro ... not resolved". Both now declare what
they call, and the definitions are present in the compiled program.

WHAT CHANGED

- 14 files declare the macros they call but had been relying on another
  declaration to bring in: mf_getquotedstr, mf_nobs, mf_getuser (twice),
  mf_getapploc, mf_getattrn, mp_abort (six services), mf_getuniquename
  (twice), dc_assignlib, bitemporal_closeouts. These were defined in the
  compiled program already; the declarations make the dependency visible
  and the two validation services correct.
- 6 hook scripts get a file-local @sasjslint {"ignoredMacros": [...]}
  (mp_abort, plus dc_assignlib in mpe_tables_postedit) instead of a macro
  list. Hooks are included at run time by the service that owns them, so
  the macros are already in that session, and these files are copied and
  edited by end users - they should not carry a declaration section to
  maintain. Recorded in AGENTS.md.
- bitemporal_dataloader.test.2.sas loses its test 30 block: it is gated
  with `and 0` ("skip until have a db") and its only setup line calls
  %assign_direct_lib, a macro that exists nowhere in this project or in
  @sasjs/core. Enabling it would fail on the first line.

VERIFICATION

- `npm run lint:sas`: 18 warnings, 0 errors, exit 0.
- `sasjs compile`: succeeds; the called-but-undefined list drops from
  three services to the one hook exception; the only definition-count
  changes are additions.
- 18 warnings remain for the next rungs: noBannedStrings (13), macro name
  length (2) and case (2), indentationMultiple (1).
2026-11-01 01:19:55 +00:00
allan bf979802fd Merge pull request 'fix(lint): make noUnusedMacros an error and clear its 54 findings (ratchet 1/n, #376)' (#418) from fix/lint-no-unused-macros into release/7.22
Sites / deploy (push) Successful in 4m10s
Build / Build-and-ng-test (pull_request) Successful in 6m33s
Lighthouse Checks / lighthouse (pull_request) Successful in 19m7s
Build / Build-and-test-development (pull_request) Successful in 31m16s
Reviewed-on: #418
2026-11-01 01:13:40 +00:00
dc c8c0858295 fix(lint): make noUnusedMacros an error and clear its 54 findings
Build / Build-and-ng-test (pull_request) Successful in 5m45s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m2s
Build / Build-and-test-development (pull_request) Successful in 31m7s
sas/.sasjslint now promotes noUnusedMacros to "error", so the rule fails
the build rather than printing a warning. The 54 findings it reported are
gone:

- 46 header entries nothing calls are dropped. Each removal was checked
  against the compiled artefacts rather than the linter alone: the
  project was compiled before and after, and the two sasjsbuild trees
  compared. 37 macro definitions disappear from the generated programs -
  every one of them either a duplicate of a definition the file still
  reaches through another declared macro, or a macro no generated program
  calls. No macro that a generated program calls lost its definition.
- 8 entries are kept, in five files that carry
  @sasjslint {"noUnusedMacros": false} with the reason written above it:
  the three serviceInit programs and serviceterm.sas declare mpeinit and
  mpeterm without calling them, because that declaration is how a service
  body reaches those macros at all; loadfile.sas declares mp_binarycopy
  for the fallback that is commented out on SAS 9.4 and later.
- Four headers are left with no macros to declare, so their empty
  <h4> SAS Macros </h4> section is removed rather than kept as a stub.

@sasjs/cli is bumped 6.2.0 -> 6.5.0 in sas/package.json. The lint engine
ships inside the CLI and is not a direct dependency, so the pinned CLI is
what fixes the rule set: 6.2.0 resolves @sasjs/lint 4.3.0, 6.5.0 pins
4.9.0 exactly. Both versions were compiled against this tree and the
sasjsbuild output is byte-identical, so the bump changes the lint engine
and nothing else.

`sasjs lint` is also now reachable as `npm run lint:sas`, and a CI step in
build.yaml runs it. Until now nothing ran SAS lint automatically, and a
bare `sasjs` resolves whatever CLI is installed globally - a different
lint version reports a different rule set, which is why the counts in
issue #376 (taken with 6.2.5) do not match the pinned 6.2.0.

43 warnings remain under the rules not yet promoted: noUndeclaredMacros
(25), noBannedStrings (13), macro name length and case (4),
indentationMultiple (1). They are the next rungs of the ratchet.

The pre-commit Prettier gate could not run: it needs client/node_modules,
which this worktree does not install, and the fix it attempted touched
three client files that this change does not modify (reverted). No client
file is in this commit.
2026-11-01 00:36:05 +00:00
allan 6fc3f0c819 Merge pull request 'fix(test): run metanav.cy.ts in CI and open a metadata object' (#417) from fix/395-metanav-spec into release/7.22
Sites / deploy (push) Successful in 4m41s
Build / Build-and-ng-test (pull_request) Successful in 5m33s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m35s
Build / Build-and-test-development (pull_request) Successful in 31m18s
Reviewed-on: #417
2026-10-31 23:59:16 +00:00
dc ad6e4c6e78 chore: merge release/7.22 into fix/395-metanav-spec
Build / Build-and-ng-test (pull_request) Successful in 5m32s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m50s
Build / Build-and-test-development (pull_request) Successful in 32m20s
2026-10-31 23:05:09 +00:00
dc 02a8590310 fix(test): run metanav.cy.ts in CI and open a metadata object
Build / Build-and-ng-test (pull_request) Successful in 7m14s
Lighthouse Checks / lighthouse (pull_request) Successful in 19m4s
Build / Build-and-test-development (pull_request) Successful in 35m25s
`cy.get` inside `.within(<tree node>)` only searches that node, and the
metadata accordion renders in the content area - so `.clr-accordion-header-button`
was never found however long it retried. The spec queries the accordion at
document level, selects the first metadata type by its class rather than by
node index, and clicks the header button directly (Clarity puts
`clr-accordion-header-button` on the button itself).

The assertions are on the app's own markers: the single open panel
(`clr-accordion-panel-open`, `aria-expanded`), the association tree inside the
panel, and the attribute grid that `metanav/metadetails` fills.

metanav moves from EXCLUDED to SPECS in `.gitea/scripts/cypress-specs.sh`.

Closes #395.
2026-10-31 22:36:33 +00:00
allan ea789c4977 Merge pull request 'test(download): target the approve button and the viewer options dropdown' (#413) from test/download-spec-selectors into release/7.22
Sites / deploy (push) Successful in 4m8s
Build / Build-and-ng-test (pull_request) Successful in 5m27s
Lighthouse Checks / lighthouse (pull_request) Successful in 18m49s
Build / Build-and-test-development (pull_request) Successful in 30m58s
Reviewed-on: #413
2026-10-31 21:26:04 +00:00
allan b927276abe Merge pull request 'refactor(mpe_assert_abort): read the child abort message from the status row' (#412) from fix/derived-hook-abort-messages into release/7.22
Sites / deploy (push) Successful in 4m2s
Build / Build-and-ng-test (pull_request) Successful in 5m20s
Lighthouse Checks / lighthouse (pull_request) Successful in 18m47s
Build / Build-and-test-development (pull_request) Failing after 42m46s
Reviewed-on: #412
2026-10-31 20:41:46 +00:00
dc dff4bbdc09 test(download): find the approval row by table, not by the stage URL
Build / Build-and-ng-test (pull_request) Successful in 4m48s
Lighthouse Checks / lighthouse (pull_request) Successful in 18m57s
Build / Build-and-test-development (pull_request) Successful in 28m58s
The case took the id from the stage route and looked for a download button
carrying it.  For an upload the route does not hold a table id at all - it holds
a load reference, the CI failure naming DC20261031-201657.470_601010_4449 - so
the button it looked for could never exist, however long it waited or however
many times the screen was reloaded.  That is why the row "never appeared" while
the approval was in fact there.

The row is now found the way hook-programs finds its own: by the table it names.
The id the download is named after is read from the button, which is the only
place that knows it.

Verified against the mock estate locally, with the licence the spec now plants:
7 passing, exit 0.
2026-10-31 20:24:58 +00:00
dc 1455f9d435 test(download): licence the estate this spec needs
Build / Build-and-ng-test (pull_request) Successful in 4m59s
Lighthouse Checks / lighthouse (pull_request) Successful in 18m36s
Build / Build-and-test-development (pull_request) Failing after 29m25s
The estate ships no licence key, so it runs on the free tier, and the free tier
disables file upload - which is exactly what the audit case needs. Whether it
could run depended on another spec having planted a key earlier in the run: the
key persists in the estate config, so the case passed or failed on spec order.

loginAndUpdateValidKey(true) plants one here. csv-limited, which asserts the
free-tier modal, runs first in cypress-specs.sh and is unaffected; download runs
last, so nothing downstream inherits the key.

Found by running the spec against the mock estate locally: the upload input
never rendered, the modal read "Locked Feature (File Upload) ... site id (123)",
and the run went 7/7 green once the key was planted.
2026-10-31 19:43:32 +00:00
dc 01893036c2 refactor(mpe_execute): build the abort text where the status row is read
Build / Build-and-ng-test (pull_request) Successful in 4m53s
Lighthouse Checks / lighthouse (pull_request) Successful in 19m5s
Build / Build-and-test-development (pull_request) Successful in 28m7s
mpe_abortmessage existed to assemble the message mpe_execute aborts with - the
program, the state and the child message - inside a data step, so that a percent
sign, an ampersand or a quote in the child message was not macro resolved on the
way into a symbol.  mpe_execute already runs a data step over the status row for
state and errmsg, so the text is built there now and the helper, plus the %let
fallback that duplicated it, is gone.

What the helper did is kept: the values come from the status row through
symget/symputx, never through %let, and a message already at the 32767 limit is
cut to the room the prefix leaves.

The comma and percent cases its tests covered now live in core's mx_execute
test, where the message is captured from a real child; what stays here is the
assembly DC owns - the prefix, the state and the child message from the status
row.
2026-10-31 19:05:48 +00:00