fix(client): restrict the table pass to Handsontable's own tables, add the viewer fallback

Review follow-up. The table pass matched every table inside the grid root,
so a table a cell renderer owns picked up role=table and scope=col - which
contradicted the reason the wrapper loop avoids [role="treegrid"]. It now
matches table.htCore only (every HOT table carries that class, one per
view), and header cells are read with :scope > thead > tr > th so a
renderer's nested table is not touched. The separate aria-rowcount=-1 pass
is gone: the wrapper strip already removes that attribute from the grid's
own element, and matching it anywhere else was the same overreach.

The viewer also gets the editor's fallback chain, so a pass that runs
before hotInstance is assigned still finds the grid through
hotTableComponent.

.agents/skills/dc-estate-testing/SKILL.md

@@ -44,7 +44,7 @@ An unmatched `/SASjsApi/...` path answers `200` with the Swagger page, so check
 
 `npm run serverdata` sends `deploy/makeDataServer.json`: one `fromjs` row with `ADMIN`, `DCPATH` and `OUTFMT`. Two things about it are worth knowing before you trust its result.
 
-- **`ADMIN` must name a group the calling account is actually in.** The shipped value is `AllUsers`, which is not a group on the shared estate - the account belongs to `all-users` (check with a call to `services/public/getgroups`, or read the `MF_GETUSER` the response echoes back). With a group the account is not in, the seed still answers `SUCCESS` and every later `getdata`/`stagedata` call aborts with `<user> is not authorised to edit <lib>.<ds> in the <lib>.MPE_SECURITY table`. Override `ADMIN` to `all-users` for the estate rather than editing the shipped JSON.
+- **`ADMIN` must name a group the calling account is actually in.** The shipped value is `all-users`, the SASjs Server group that every logged-in user is a member of, which is what the shared estate needs. A name the account is not in - a Viya-style group such as `AllUsers`, or one estate's own group carried into another - still answers `SUCCESS` from the seed, and then every later `getdata`/`stagedata` call aborts with `<user> is not authorised to edit <lib>.<ds> in the <lib>.MPE_SECURITY table`. Check what the account belongs to with a call to `services/public/getgroups` before changing the value.
 - **Each run creates a new library.** The response carries `STATUS`, `DCPATH` and `LIBREF` (for example `DC621835`), and the LIBREF is clock-derived, so a libref from an earlier run answers `dataset <lib>.<table> does not exist` - take the current one from the response.
 - **`KEEPME` controls whether the seed can be repeated.** Without it `makedata` deletes itself and the configurator after a successful run (deliberately, so it cannot be executed twice over real data), and a second seed fails with `The Program at (<appLoc>/services/admin/makedata) does not exist` until you run `sasjs cbd` again. A harness that reseeds between runs sets `KEEPME` in the input row.
 - **Seed once per deploy, not before every test run.** The suite builds its own `dctest` library from the seeded control tables; reseeding in between is wasted work, and there is no need to run `makedata` and `sasjs test` in the opposite order.

.git-hooks/pre-commit

@@ -20,6 +20,16 @@ if ! "$gitleaks_bin" protect --staged --redact; then
     exit 1
 fi
 
+# Every tracked .npmrc is a policy file - root, sas/ and client/ - so a registry
+# credential pasted into any of them is one `git add -A` from being committed.
+# Each staged one is checked, not just the client.
+for npmrc in $(git diff --cached --name-only --diff-filter=ACMR | grep -E '(^|/)\.npmrc$'); do
+    if git show ":$npmrc" | grep -Eq '(_authToken|_auth[^a-z]|_password|:_auth)'; then
+        echo "ERROR: $npmrc holds a registry credential - keep it in ~/.npmrc instead"
+        exit 1
+    fi
+done
+
 ## Avoid large commits
 # https://www.backblaze.com/blog/how-many-bytes-are-in-a-megabyte-really/
 size_limit=$((2 * 1024 * 1024)) # 2MB

.gitea/scripts/check-runner-memory.sh

@@ -0,0 +1,99 @@
+#!/usr/bin/env bash
+# Fail a client job before it starts when the runner cannot hold it.
+#
+# Every client job here compiles the frontend from cold: `ng build
+# --configuration production` peaks at 3.8 GB resident on a 4 vCPU box
+# (measured with no .angular/cache present, which is what a fresh runner
+# has), and the Cypress and Lighthouse jobs add a dev server and a browser on
+# top of that. On a 4 GB runner the kernel OOM killer takes the build part way
+# through, which surfaces as `exitcode '137'` at `ng build`, or as Karma
+# disconnecting mid-suite ("no message in 30000 ms"). Both read as a flaky
+# test rather than as an undersized box, and both cost a whole CI slot.
+#
+# 8 GB is the minimum supported runner size: it leaves roughly 2x headroom
+# over that peak. This check runs before the installs, so an undersized runner
+# fails in seconds with the numbers in the log.
+#
+# The floor is 7 GB rather than 8192 MB because an 8 GB box does not report 8
+# GB: the fleet's report 7751 MB total, the rest going to firmware and the
+# kernel. A 4 GB box reports under 4000 MB, so 7 GB separates the two sizes
+# with room for a box that reserves a little more than ours do.
+#
+# What this does not catch is oversubscription: two client jobs landing on one
+# box. A box that passes the floor can still OOM a build when another job is
+# resident, and MemAvailable at start is the only sign of it - that is reported
+# as a warning, not a failure, because a busy box is transient where a small
+# box is not. Keep one client job per runner.
+#
+# Usage: check-runner-memory.sh [min_total_mb]
+#   or:  MIN_TOTAL_MB=4096 check-runner-memory.sh
+set -euo pipefail
+
+MIN_TOTAL_MB="${1:-${MIN_TOTAL_MB:-7000}}"
+
+# A typo here (8GB, a stray flag, an empty variable) would otherwise reach the
+# integer test under `set -e` and abort with "integer expression expected".
+case "$MIN_TOTAL_MB" in
+  ''|*[!0-9]*)
+    echo "usage: check-runner-memory.sh [min_total_mb] - got '$MIN_TOTAL_MB'" >&2
+    exit 2
+    ;;
+esac
+
+meminfo_field() {
+  awk -v key="$1:" '$1 == key { print $2 }' /proc/meminfo
+}
+
+total_kb=$(meminfo_field MemTotal || true)
+avail_kb=$(meminfo_field MemAvailable || true)
+cpus=$(nproc 2>/dev/null || echo '?')
+
+if [ -z "$total_kb" ]; then
+  echo "runner memory: /proc/meminfo has no MemTotal - cannot measure, continuing"
+  exit 0
+fi
+
+total_mb=$(( total_kb / 1024 ))
+avail_mb=$(( ${avail_kb:-0} / 1024 ))
+# Older kernels have no MemAvailable; report that rather than 0, which would
+# read as a permanently exhausted runner.
+if [ -n "$avail_kb" ]; then
+  avail_display="${avail_mb} MB available"
+else
+  avail_display="available unknown"
+fi
+
+# A container usually sees the host's RAM, but may carry its own lower ceiling
+# (cgroup v2 memory.max, v1 memory.limit_in_bytes). That ceiling is what the
+# build is actually killed against, so it counts as the runner's size.
+limit_mb=""
+for f in /sys/fs/cgroup/memory.max /sys/fs/cgroup/memory/memory.limit_in_bytes; do
+  [ -r "$f" ] || continue
+  v=$(cat "$f")
+  case "$v" in
+    ''|max|*[!0-9]*) continue ;;
+  esac
+  # An unlimited v1 cgroup reports a sentinel near 2^63 bytes.
+  [ "$v" -lt 100000000000 ] || continue
+  candidate=$(( v / 1024 / 1024 ))
+  if [ -z "$limit_mb" ] || [ "$candidate" -lt "$limit_mb" ]; then
+    limit_mb="$candidate"
+  fi
+done
+
+echo "runner memory: ${total_mb} MB total, ${avail_display}, ${cpus} vCPU${limit_mb:+, ${limit_mb} MB cgroup limit}"
+
+effective_mb=$total_mb
+if [ -n "$limit_mb" ] && [ "$limit_mb" -lt "$effective_mb" ]; then
+  effective_mb=$limit_mb
+fi
+
+if [ "$effective_mb" -lt "$MIN_TOTAL_MB" ]; then
+  echo "::error::runner has ${effective_mb} MB of memory, a client job needs at least ${MIN_TOTAL_MB} MB (8 GB is the supported minimum; a cold production build alone peaks at about 3.8 GB, and the Cypress and Lighthouse jobs add a dev server and a browser to that)."
+  exit 1
+fi
+
+# Not a gate - a second job landing on the same box shows up here first.
+if [ -n "$avail_kb" ] && [ "$avail_mb" -lt 5000 ]; then
+  echo "::warning::only ${avail_mb} MB is available on this runner; another job may be running on the same box"
+fi

.gitea/scripts/cypress-specs.sh

@@ -9,21 +9,27 @@
 #   - hook-programs asserts that an edit raises an approval row against its
 #     target table; it runs early, before other specs have added rows to the
 #     approval tables.
+#   - metanav only reads the metadata tree, and applies a licence key so it can
+#     run on its own; it goes last so that key cannot change another spec.
 #
 # Specs added later are appended rather than inserted, so the sequence that was
-# known green keeps its shape. metanav is deliberately absent - see
-# https://git.datacontroller.io/dc/dc/issues/395.
+# known green keeps its shape.
 #
-# download is held out for a different reason - its download button is covered by
-# a fixed-position element, see
-# https://git.datacontroller.io/dc/dc/issues/398.
+# download was held out while its selectors were stale - see
+# https://git.datacontroller.io/dc/dc/issues/398. Its first case uploads a fixture
+# to mpe_x_test and submits it, so it creates the approval it downloads instead of
+# depending on what earlier specs leave behind. That approval stays: the CI job
+# deletes the appLoc and re-runs makedata before the specs, so nothing accumulates
+# between runs, but a hand-run against a reused estate leaves one pending row per
+# run. It is appended because a spec added later goes on the end rather than
+# reordering the sequence that was known green.
 set -euo pipefail
 
-SPECS="csv-limited hook-programs liveness editor excel-multi-load excel csv filtering filter-panel licensing viewer-labels full-table-search viewbox stage browser-info ddl-export row-cell-limits derived-variables startupservice-launcher coltype-delete-record submitted-details embed-va"
+SPECS="csv-limited hook-programs liveness editor excel-multi-load excel csv filtering filter-panel licensing viewer-labels full-table-search viewbox stage browser-info ddl-export row-cell-limits derived-variables startupservice-launcher coltype-delete-record submitted-details embed-va download metanav"
 
 # Specs that exist but are held out of CI, each with the reason. Listed here so
 # the coverage check below can tell a deliberate exclusion from a forgotten file.
-EXCLUDED="metanav download"
+EXCLUDED=""
 
 case "${1:-}" in
   all)

.gitea/workflows/build.yaml

@@ -3,6 +3,8 @@ run-name: Running Lint Check and Licence checker on Pull Request
 on: [pull_request]
 
 env:
+  # the server the mocked services and app tests run against - bump deliberately
+  SASJS_SERVER_VERSION: '3.6.2'
   NODE_VERSION: '24.15.0'
 
 jobs:
@@ -11,6 +13,11 @@ jobs:
 
     steps:
       - uses: actions/checkout@v4
+      - name: Check runner size
+        # An undersized runner dies inside `ng build` (exitcode 137) or as a
+        # Karma disconnect twenty minutes in. Fail here instead, in seconds,
+        # with the measured numbers in the log - see the script.
+        run: bash .gitea/scripts/check-runner-memory.sh
       - uses: actions/setup-node@v4
         with:
           node-version: ${{ env.NODE_VERSION }}
@@ -22,9 +29,8 @@ jobs:
         # credential must fail the build, not just a commit.
         # Uses the same @nogoo9/gitleaks binary the pre-commit hook pins - the
         # gitleaks-action requires a license key for organization repos.
-        # Runs before client/.npmrc is written below, because that file is
-        # created from the NPMRC secret and a later scan would flag the
-        # registry token this job injects itself.
+        # Runs before the registry config is written, so the scan cannot see
+        # the token this job injects.
         # --no-git scans the working tree, which is what a pull request gate
         # is about.
         run: |
@@ -38,7 +44,11 @@ jobs:
           apt install -y ./google-chrome*.deb
 
       - name: Write .npmrc file
-        run: echo "$NPMRC" >> client/.npmrc
+        run: |
+          # the registry secret stays outside the repo: client/.npmrc is
+          # tracked now, and one git add -A would commit a live token
+          echo "$NPMRC" > "$RUNNER_TEMP/npmrc"
+          echo "NPM_CONFIG_USERCONFIG=$RUNNER_TEMP/npmrc" >> "$GITHUB_ENV"
         shell: bash
         env:
           NPMRC: ${{ secrets.NPMRC}}
@@ -51,7 +61,13 @@ jobs:
             gpg --batch --yes --passphrase-fd 0 \
             --output ./libraries/sheet-crypto.tgz \
             --decrypt ./libraries/sheet-crypto.tgz.gpg
-          npm ci
+          # dependency hooks stay off (client/.npmrc); the client's own
+          # generator runs explicitly, because the app imports the file it makes
+          npm ci --ignore-scripts
+          npm run postinstall
+          # cypress downloads its binary in a dependency install hook, which the
+          # policy skips, so it is installed explicitly here
+          npx cypress install
 
       - name: Check audit
         # Audit should fail and stop the CI on any vulnerability in root and sas, and on low+ in client
@@ -62,9 +78,27 @@ jobs:
           cd ../client
           npm audit --omit=dev
 
+      - name: Check archiver directory walk
+        # the readdir-glob minimatch override in client/package.json is a major
+        # bump; archiver's directory() is the only runtime caller of that path,
+        # so exercise it rather than trusting that the install succeeded
+        run: |
+          cd client
+          npm run check:archiver
+
       - name: Lint check
         run: npm run lint:check
 
+      - name: SAS lint check
+        # sas/.sasjslint promotes each rule to "error" as its findings are
+        # cleared, and only errors change the exit code - so this gate fails on
+        # a regression in a cleared rule and passes while other rules still
+        # carry warnings.
+        run: |
+          cd sas
+          npm ci --ignore-scripts
+          npm run lint:sas
+
       - name: Licence checker
         run: |
           cd client
@@ -102,6 +136,11 @@ jobs:
   # limits how far sharding helps.  The specs it runs come from
   # .gitea/scripts/cypress-specs.sh.
 
+      - name: Remove registry config
+        # written earlier in this job; nothing deletes it on the way out
+        if: always()
+        run: rm -f "$RUNNER_TEMP/npmrc"
+
   Build-and-test-development:
     runs-on: ubuntu-latest
     needs: Build-and-ng-test
@@ -113,14 +152,24 @@ jobs:
 
     steps:
       - uses: actions/checkout@v4
+      - name: Check runner size
+        # Each Cypress job compiles the frontend and runs a browser on top; a
+        # runner too small for that fails as a flaky-looking disconnect - see
+        # the script.
+        run: bash .gitea/scripts/check-runner-memory.sh
       - uses: actions/setup-node@v4
         with:
           node-version: ${{ env.NODE_VERSION }}
 
       - name: Write .npmrc file
         run: |
-          touch client/.npmrc
-          echo '${{ secrets.NPMRC}}' > client/.npmrc
+          # the secret arrives through the environment, not interpolated into
+          # this script: a quote or $( in the value would otherwise break the
+          # step or execute
+          printf '%s\n' "$NPMRC" > "$RUNNER_TEMP/npmrc"
+          echo "NPM_CONFIG_USERCONFIG=$RUNNER_TEMP/npmrc" >> "$GITHUB_ENV"
+        env:
+          NPMRC: ${{ secrets.NPMRC}}
 
       - name: Install system dependencies
         run: |
@@ -143,7 +192,13 @@ jobs:
           cd client
           # Decrypt and Install sheet
           echo ${{ secrets.SHEET_PWD }} | gpg --batch --yes --passphrase-fd 0 ./libraries/sheet-crypto.tgz.gpg
-          npm ci
+          # dependency hooks stay off (client/.npmrc); the client's own
+          # generator runs explicitly, because the app imports the file it makes
+          npm ci --ignore-scripts
+          npm run postinstall
+          # cypress downloads its binary in a dependency install hook, which the
+          # policy skips, so it is installed explicitly here
+          npx cypress install
 
       - name: Build frontend for web streaming
         # The server-ci target streams client/dist as the 'clickme' web app (see
@@ -154,11 +209,21 @@ jobs:
           cd client
           npm run build
 
+      - name: Cache the npm downloads
+        # The server is installed from npm at a pinned version, so the tarball is
+        # immutable and the key never needs invalidating by hand: a hit skips
+        # the 10.7MB download, a miss costs exactly what the zip cost before.
+        # continue-on-error, because a runner without cache support should still
+        # run the pipeline - this step only ever saves time.
+        continue-on-error: true
+        uses: actions/cache@v4
+        with:
+          path: ~/.npm
+          key: npm-sasjs-server-${{ env.SASJS_SERVER_VERSION }}
+
       - name: Setup and start SASjs server
         run: |
           npm i -g pm2
-          curl -L https://github.com/sasjs/server/releases/latest/download/linux.zip > linux.zip
-          unzip linux.zip
           touch .env
           echo RUN_TIMES=js >> .env
           echo NODE_PATH=node >> .env
@@ -170,11 +235,10 @@ jobs:
           # in a throwaway container, so opt in explicitly.
           echo RUN_AS=root >> .env
           cat .env
-          pm2 start api-linux --wait-ready
-          # `pm2 start --wait-ready` can return while the app is still
-          # launching (it does not always wait for the ready signal), and the
-          # next step hits :5000 immediately. Wait for the server to answer,
-          # and dump its log if it never does.
+          pm2 start npx --name sasjs-server -- --yes @sasjs/server@${{ env.SASJS_SERVER_VERSION }}
+          # `pm2 start` returns before the app is listening, and the next step
+          # hits :5000 immediately. Wait for the server to answer, and dump its
+          # log if it never does.
           for i in $(seq 1 60); do
             if curl -sf -o /dev/null http://localhost:5000/; then
               echo "SASjs server answered on attempt ${i}"
@@ -184,7 +248,7 @@ jobs:
           done
           echo "SASjs server did not come up within 120s"
           pm2 list
-          pm2 logs api-linux --lines 100 --nostream
+          pm2 logs sasjs-server --lines 100 --nostream
           exit 1
 
       - name: Deploy mocked services
@@ -252,3 +316,8 @@ jobs:
         with:
           name: cypress-videos.zip
           path: cypress-videos.zip
+
+      - name: Remove registry config
+        # written earlier in this job; nothing deletes it on the way out
+        if: always()
+        run: rm -f "$RUNNER_TEMP/npmrc"

.gitea/workflows/docs.yaml

@@ -28,8 +28,9 @@ jobs:
 
       - name: Write .npmrc file
         run: |
-          printf '%s\n' "$NPMRC" >> client/.npmrc
-          echo "legacy-peer-deps=true" >> client/.npmrc
+          printf '%s\n' "$NPMRC" > "$RUNNER_TEMP/npmrc"
+          echo "legacy-peer-deps=true" >> "$RUNNER_TEMP/npmrc"
+          echo "NPM_CONFIG_USERCONFIG=$RUNNER_TEMP/npmrc" >> "$GITHUB_ENV"
         shell: bash
         env:
           NPMRC: ${{ secrets.NPMRC }}
@@ -40,6 +41,9 @@ jobs:
           # Decrypt and Install sheet
           echo "$SHEET_PWD" | gpg --batch --yes --passphrase-fd 0 ./libraries/sheet-crypto.tgz.gpg
           npm ci
+          # client install scripts are off (client/.npmrc); the generator that
+          # creates src/environments/version.ts runs explicitly
+          npm run postinstall
         shell: bash
         env:
           SHEET_PWD: ${{ secrets.SHEET_PWD }}
@@ -76,6 +80,11 @@ jobs:
             --password "$SURFER_PASSWORD" \
             --message "Typedoc from ${REF_NAME} @ ${COMMIT_SHA:0:8}"
 
+      - name: Remove registry config
+        # written earlier in this job; nothing deletes it on the way out
+        if: always()
+        run: rm -f "$RUNNER_TEMP/npmrc"
+
   sas-docs:
     runs-on: ubuntu-latest
     concurrency:

.gitea/workflows/lighthouse.yaml

@@ -3,6 +3,8 @@ run-name: Running Lighthouse Performance and Accessibility Checks on Pull Reques
 on: [pull_request]
 
 env:
+  # the server the mocked services and app tests run against - bump deliberately
+  SASJS_SERVER_VERSION: '3.6.2'
   NODE_VERSION: '24.15.0'
 
 jobs:
@@ -12,6 +14,12 @@ jobs:
     steps:
       - uses: actions/checkout@v4
 
+      - name: Check runner size
+        # This job runs a production build, then a SASjs server, a dev server
+        # and Chrome. An undersized runner dies inside the build with exitcode
+        # 137; fail here in seconds instead - see the script.
+        run: bash .gitea/scripts/check-runner-memory.sh
+
       - name: Use Node.js ${{ env.NODE_VERSION }}
         uses: actions/setup-node@v4
         with:
@@ -26,6 +34,18 @@ jobs:
       - name: Install global packages
         run: npm i -g pm2 @sasjs/cli wait-on
 
+      - name: Cache the npm downloads
+        # The server is installed from npm at a pinned version, so the tarball is
+        # immutable and the key never needs invalidating by hand: a hit skips
+        # the 10.7MB download, a miss costs exactly what the zip cost before.
+        # continue-on-error, because a runner without cache support should still
+        # run the pipeline - this step only ever saves time.
+        continue-on-error: true
+        uses: actions/cache@v4
+        with:
+          path: ~/.npm
+          key: npm-sasjs-server-${{ env.SASJS_SERVER_VERSION }}
+
       - name: Setup and start SASjs server
         run: |
           touch .env
@@ -39,13 +59,10 @@ jobs:
           # in a throwaway container, so opt in explicitly.
           echo RUN_AS=root >> .env
           cat .env
-          curl -L https://github.com/sasjs/server/releases/latest/download/linux.zip > linux.zip
-          unzip linux.zip
-          pm2 start api-linux --wait-ready
-          # `pm2 start --wait-ready` can return while the app is still
-          # launching (it does not always wait for the ready signal), and the
-          # next step hits :5000 immediately. Wait for the server to answer,
-          # and dump its log if it never does.
+          pm2 start npx --name sasjs-server -- --yes @sasjs/server@${{ env.SASJS_SERVER_VERSION }}
+          # `pm2 start` returns before the app is listening, and the next step
+          # hits :5000 immediately. Wait for the server to answer, and dump its
+          # log if it never does.
           for i in $(seq 1 60); do
             if curl -sf -o /dev/null http://localhost:5000/; then
               echo "SASjs server answered on attempt ${i}"
@@ -55,11 +72,15 @@ jobs:
           done
           echo "SASjs server did not come up within 120s"
           pm2 list
-          pm2 logs api-linux --lines 100 --nostream
+          pm2 logs sasjs-server --lines 100 --nostream
           exit 1
 
       - name: Write .npmrc file
-        run: echo "$NPMRC" > client/.npmrc
+        run: |
+          # the registry secret stays outside the repo: client/.npmrc is
+          # tracked now, and one git add -A would commit a live token
+          echo "$NPMRC" > "$RUNNER_TEMP/npmrc"
+          echo "NPM_CONFIG_USERCONFIG=$RUNNER_TEMP/npmrc" >> "$GITHUB_ENV"
         shell: bash
         env:
           NPMRC: ${{ secrets.NPMRC}}
@@ -73,6 +94,9 @@ jobs:
             --output ./libraries/sheet-crypto.tgz \
             --decrypt ./libraries/sheet-crypto.tgz.gpg
           npm ci
+          # client install scripts are off (client/.npmrc); the generator that
+          # creates src/environments/version.ts runs explicitly
+          npm run postinstall
           npm install -g replace-in-files-cli
 
       - name: Update appLoc in index.html
@@ -109,3 +133,8 @@ jobs:
           name: Lighthouse results
           path: client/lighthouse-reports
           include-hidden-files: true
+
+      - name: Remove registry config
+        # written earlier in this job; nothing deletes it on the way out
+        if: always()
+        run: rm -f "$RUNNER_TEMP/npmrc"

.gitea/workflows/release-assets.yaml

@@ -34,14 +34,19 @@ jobs:
       - uses: actions/checkout@v4
         with:
           fetch-depth: 0
+      - name: Check runner size
+        # Builds the same frontend as the release jobs; an undersized runner
+        # fails inside the build - see the script.
+        run: bash .gitea/scripts/check-runner-memory.sh
       - uses: actions/setup-node@v4
         with:
           node-version: ${{ env.NODE_VERSION }}
 
       - name: Write .npmrc file
         run: |
-          printf '%s\n' "$NPMRC" >> client/.npmrc
-          echo "legacy-peer-deps=true" >> client/.npmrc
+          printf '%s\n' "$NPMRC" > "$RUNNER_TEMP/npmrc"
+          echo "legacy-peer-deps=true" >> "$RUNNER_TEMP/npmrc"
+          echo "NPM_CONFIG_USERCONFIG=$RUNNER_TEMP/npmrc" >> "$GITHUB_ENV"
         shell: bash
         env:
           NPMRC: ${{ secrets.NPMRC }}
@@ -91,6 +96,9 @@ jobs:
           # Decrypt and Install sheet
           echo "$SHEET_PWD" | gpg --batch --yes --passphrase-fd 0 ./libraries/sheet-crypto.tgz.gpg
           npm ci
+          # client install scripts are off (client/.npmrc); the generator that
+          # creates src/environments/version.ts runs explicitly
+          npm run postinstall
           npm i webpack
           npm run build
         shell: bash
@@ -207,3 +215,8 @@ jobs:
           done
           echo "Uploading SHA256SUMS ..."
           curl --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@SHA256SUMS"
+
+      - name: Remove registry config
+        # written earlier in this job; nothing deletes it on the way out
+        if: always()
+        run: rm -f "$RUNNER_TEMP/npmrc"

.gitea/workflows/release.yaml

@@ -6,6 +6,8 @@ on:
       - main
 
 env:
+  # the server the mocked services and app tests run against - bump deliberately
+  SASJS_SERVER_VERSION: '3.6.2'
   NODE_VERSION: '24.5.0'
 
 jobs:
@@ -16,14 +18,24 @@ jobs:
 
     steps:
       - uses: actions/checkout@v4
+      - name: Check runner size
+        # A release job that OOMs costs the tag as well as the slot; fail in
+        # seconds on an undersized runner instead of twenty minutes in - see
+        # the script.
+        run: bash .gitea/scripts/check-runner-memory.sh
       - uses: actions/setup-node@v4
         with:
           node-version: ${{ env.NODE_VERSION }}
 
       - name: Write .npmrc file
         run: |
-          touch client/.npmrc
-          echo '${{ secrets.NPMRC}}' > client/.npmrc
+          # the secret arrives through the environment, not interpolated into
+          # this script: a quote or $( in the value would otherwise break the
+          # step or execute
+          printf '%s\n' "$NPMRC" > "$RUNNER_TEMP/npmrc"
+          echo "NPM_CONFIG_USERCONFIG=$RUNNER_TEMP/npmrc" >> "$GITHUB_ENV"
+        env:
+          NPMRC: ${{ secrets.NPMRC}}
 
       - name: Install Chrome for Angular tests
         run: |
@@ -43,6 +55,9 @@ jobs:
           # Decrypt and Install sheet
           echo ${{ secrets.SHEET_PWD }} | gpg --batch --yes --passphrase-fd 0 ./libraries/sheet-crypto.tgz.gpg
           npm ci
+          # client install scripts are off (client/.npmrc); the generator that
+          # creates src/environments/version.ts runs explicitly
+          npm run postinstall
 
       - name: Check audit
         # Audit should fail and stop the CI on any vulnerability in root and sas, and on low+ in client
@@ -62,8 +77,14 @@ jobs:
         run: |
           cd client
           npm run postinstall
+          npm run check:archiver
           npm run build
 
+      - name: Remove registry config
+        # written earlier in this job; nothing deletes it on the way out
+        if: always()
+        run: rm -f "$RUNNER_TEMP/npmrc"
+
   Build-and-test-development:
     runs-on: ubuntu-latest
     needs: Build-production-and-ng-test
@@ -75,14 +96,23 @@ jobs:
 
     steps:
       - uses: actions/checkout@v4
+      - name: Check runner size
+        # The development build plus a browser: same undersized-runner failure
+        # as the review jobs, on the path that makes the tag - see the script.
+        run: bash .gitea/scripts/check-runner-memory.sh
       - uses: actions/setup-node@v4
         with:
           node-version: ${{ env.NODE_VERSION }}
 
       - name: Write .npmrc file
         run: |
-          touch client/.npmrc
-          echo '${{ secrets.NPMRC}}' > client/.npmrc
+          # the secret arrives through the environment, not interpolated into
+          # this script: a quote or $( in the value would otherwise break the
+          # step or execute
+          printf '%s\n' "$NPMRC" > "$RUNNER_TEMP/npmrc"
+          echo "NPM_CONFIG_USERCONFIG=$RUNNER_TEMP/npmrc" >> "$GITHUB_ENV"
+        env:
+          NPMRC: ${{ secrets.NPMRC}}
 
       - name: Install system dependencies
         run: |
@@ -106,6 +136,9 @@ jobs:
           # Decrypt and Install sheet
           echo ${{ secrets.SHEET_PWD }} | gpg --batch --yes --passphrase-fd 0 ./libraries/sheet-crypto.tgz.gpg
           npm ci
+          # client install scripts are off (client/.npmrc); the generator that
+          # creates src/environments/version.ts runs explicitly
+          npm run postinstall
 
       - name: Build frontend for web streaming
         # The server-ci target streams client/dist as the 'clickme' web app, so
@@ -117,11 +150,21 @@ jobs:
           cd client
           npm run build
 
+      - name: Cache the npm downloads
+        # The server is installed from npm at a pinned version, so the tarball is
+        # immutable and the key never needs invalidating by hand: a hit skips
+        # the 10.7MB download, a miss costs exactly what the zip cost before.
+        # continue-on-error, because a runner without cache support should still
+        # run the pipeline - this step only ever saves time.
+        continue-on-error: true
+        uses: actions/cache@v4
+        with:
+          path: ~/.npm
+          key: npm-sasjs-server-${{ env.SASJS_SERVER_VERSION }}
+
       - name: Setup and start SASjs server
         run: |
           npm i -g pm2
-          curl -L https://github.com/sasjs/server/releases/latest/download/linux.zip > linux.zip
-          unzip linux.zip
           touch .env
           echo RUN_TIMES=js >> .env
           echo NODE_PATH=node >> .env
@@ -133,11 +176,10 @@ jobs:
           # in a throwaway container, so opt in explicitly.
           echo RUN_AS=root >> .env
           cat .env
-          pm2 start api-linux --wait-ready
-          # `pm2 start --wait-ready` can return while the app is still
-          # launching (it does not always wait for the ready signal), and the
-          # next step hits :5000 immediately. Wait for the server to answer,
-          # and dump its log if it never does.
+          pm2 start npx --name sasjs-server -- --yes @sasjs/server@${{ env.SASJS_SERVER_VERSION }}
+          # `pm2 start` returns before the app is listening, and the next step
+          # hits :5000 immediately. Wait for the server to answer, and dump its
+          # log if it never does.
           for i in $(seq 1 60); do
             if curl -sf -o /dev/null http://localhost:5000/; then
               echo "SASjs server answered on attempt ${i}"
@@ -147,7 +189,7 @@ jobs:
           done
           echo "SASjs server did not come up within 120s"
           pm2 list
-          pm2 logs api-linux --lines 100 --nostream
+          pm2 logs sasjs-server --lines 100 --nostream
           exit 1
 
       - name: Deploy mocked services
@@ -174,6 +216,9 @@ jobs:
           replace-in-files --regex='"password".*' --replacement='"password":"'${{ secrets.CYPRESS_PWD_SASJS }}'" ' ./cypress.env.json
           cat ./cypress.env.json
           npm run postinstall
+          # cypress downloads its binary in a dependency install hook, which the
+          # client policy skips
+          npx cypress install
           # Prepare index.html to SASJS local
           replace-in-files --regex='serverUrl=".*?"' --replacement='serverUrl="http://localhost:5000"' ./src/index.html
           replace-in-files --regex='appLoc=".*?"' --replacement='appLoc="/Public/app/dc"' ./src/index.html
@@ -196,6 +241,11 @@ jobs:
           name: cypress-videos.zip
           path: cypress-videos.zip
 
+      - name: Remove registry config
+        # written earlier in this job; nothing deletes it on the way out
+        if: always()
+        run: rm -f "$RUNNER_TEMP/npmrc"
+
   release:
     runs-on: ubuntu-latest
     needs: [Build-production-and-ng-test, Build-and-test-development]
@@ -216,14 +266,19 @@ jobs:
 
     steps:
       - uses: actions/checkout@v4
+      - name: Check runner size
+        # This job builds the frontend twice (webdoc and the release check);
+        # both die the same way on an undersized runner - see the script.
+        run: bash .gitea/scripts/check-runner-memory.sh
       - uses: actions/setup-node@v4
         with:
           node-version: ${{ env.NODE_VERSION }}
 
       - name: Write .npmrc file
         run: |
-          echo "$NPMRC" > client/.npmrc
-          echo "legacy-peer-deps=true" >> client/.npmrc
+          echo "$NPMRC" > "$RUNNER_TEMP/npmrc"
+          echo "legacy-peer-deps=true" >> "$RUNNER_TEMP/npmrc"
+          echo "NPM_CONFIG_USERCONFIG=$RUNNER_TEMP/npmrc" >> "$GITHUB_ENV"
         shell: bash
         env:
           NPMRC: ${{ secrets.NPMRC}}
@@ -241,6 +296,9 @@ jobs:
           # Decrypt and Install sheet
           echo ${{ secrets.SHEET_PWD }} | gpg --batch --yes --passphrase-fd 0 ./libraries/sheet-crypto.tgz.gpg
           npm ci
+          # client install scripts are off (client/.npmrc); the generator that
+          # creates src/environments/version.ts runs explicitly
+          npm run postinstall
           npm i webpack
           npm run build
 
@@ -459,3 +517,8 @@ jobs:
             -H 'Content-Type: application/json' \
             --data "$(jq -n --arg ref "refs/tags/$TAG" '{ref: $ref}')" \
             "$BASE/actions/workflows/sites.yaml/dispatches"
+
+      - name: Remove registry config
+        # written earlier in this job; nothing deletes it on the way out
+        if: always()
+        run: rm -f "$RUNNER_TEMP/npmrc"

.gitea/workflows/sites.yaml

@@ -10,6 +10,8 @@ on:
   workflow_dispatch:
 
 env:
+  # the server the mocked services and app tests run against - bump deliberately
+  SASJS_SERVER_VERSION: '3.6.2'
   NODE_VERSION: '24.15.0'
   APP_LOC: /Public/app/dc
   SURFER_HOST: https://demo.datacontroller.io
@@ -36,14 +38,23 @@ jobs:
           # The tag is what the client's build stamp describes (src/version.ts
           # runs git-describe), so the demo build needs the tags in the clone.
           fetch-depth: 0
+      - name: Check runner size
+        # The demo build is the same cold production build; an undersized
+        # runner fails inside it - see the script.
+        run: bash .gitea/scripts/check-runner-memory.sh
       - uses: actions/setup-node@v4
         with:
           node-version: ${{ env.NODE_VERSION }}
 
       - name: Write .npmrc file
         run: |
-          touch client/.npmrc
-          echo '${{ secrets.NPMRC}}' > client/.npmrc
+          # the secret arrives through the environment, not interpolated into
+          # this script: a quote or $( in the value would otherwise break the
+          # step or execute
+          printf '%s\n' "$NPMRC" > "$RUNNER_TEMP/npmrc"
+          echo "NPM_CONFIG_USERCONFIG=$RUNNER_TEMP/npmrc" >> "$GITHUB_ENV"
+        env:
+          NPMRC: ${{ secrets.NPMRC}}
 
       - name: Install client dependencies
         run: |
@@ -51,6 +62,9 @@ jobs:
           # Decrypt and Install sheet
           echo ${{ secrets.SHEET_PWD }} | gpg --batch --yes --passphrase-fd 0 ./libraries/sheet-crypto.tgz.gpg
           npm ci
+          # client install scripts are off (client/.npmrc); the generator that
+          # creates src/environments/version.ts runs explicitly
+          npm run postinstall
 
       - name: Build frontend
         # The site serves this build. `sasjs build` rewrites its index.html to
@@ -66,19 +80,29 @@ jobs:
           cd sas
           npm ci
 
+      - name: Cache the npm downloads
+        # The server is installed from npm at a pinned version, so the tarball is
+        # immutable and the key never needs invalidating by hand: a hit skips
+        # the 10.7MB download, a miss costs exactly what the zip cost before.
+        # continue-on-error, because a runner without cache support should still
+        # run the pipeline - this step only ever saves time.
+        continue-on-error: true
+        uses: actions/cache@v4
+        with:
+          path: ~/.npm
+          key: npm-sasjs-server-${{ env.SASJS_SERVER_VERSION }}
+
       - name: Start SASjs server
         # Needed only to run makedata: the drive the site ships with is captured
         # from a real server (see the snapshot step below).
         run: |
-          curl -L https://github.com/sasjs/server/releases/latest/download/linux.zip > linux.zip
-          unzip -q linux.zip
           touch .env
           echo RUN_TIMES=js >> .env
           echo NODE_PATH=node >> .env
           echo HTTP_PORT=5000 >> .env
           # The server refuses to run as root unless RUN_AS names an account
           echo RUN_AS=root >> .env
-          nohup ./api-linux > server.log 2>&1 &
+          nohup npx --yes @sasjs/server@${{ env.SASJS_SERVER_VERSION }} > server.log 2>&1 &
           for i in $(seq 1 60); do
             if curl -sf -o /dev/null http://localhost:5000/; then
               echo "SASjs server answered on attempt ${i}"
@@ -181,3 +205,8 @@ jobs:
             --message "$MESSAGE" \
             "${SITE_ARGS[@]}" \
             sas/mocks/sasjsbuild/mock
+
+      - name: Remove registry config
+        # written earlier in this job; nothing deletes it on the way out
+        if: always()
+        run: rm -f "$RUNNER_TEMP/npmrc"

.gitignore

@@ -20,7 +20,6 @@ sasjsbuild
 sasjsresults
 .env*
 .sasjsrc
-client/.npmrc
 *~
 .lighthouseci
 audit.log

AGENTS.md

@@ -31,9 +31,12 @@ Rationale: hard-wrapped prose produces noisy diffs when sentences are edited and
 Never consider a change complete until the relevant linters pass on the files you touched — do not rely on the user's pre-commit hooks to catch it:
 
 - **Client (TypeScript/HTML/etc.)**: run `npm run lint:check` from the `client/` directory (prettier). Fix any failures with `npm run lint:fix`.
-- **SAS**: after creating or modifying any `.sas` files, run `sasjs lint` from the `sas/` directory and ensure the files you touched have no lint warnings (the repo currently has pre-existing warnings in other files, which can be ignored).
+- **SAS**: after creating or modifying any `.sas` files, run `npm run lint:sas` from the `sas/` directory. That script runs the `@sasjs/cli` version pinned in `sas/package.json`, so it matches CI; a global or `npx` `sasjs` may resolve a different `@sasjs/lint` and report a different rule set.
+  Rules are promoted to `error` in `sas/.sasjslint` one at a time as their findings reach zero, and only errors change the exit code - so the whole project must stay clean of the promoted rules, not just the files you touched. Warnings under rules not yet promoted are the remaining backlog; do not add to them in files you touch, and do not clear other files' findings in the same change unless that rule is the point of the change.
   `sas/.sasjslint` carries `ignoredMacros: ["webout", "mpeinit", "mpeterm"]`. `webout` is the linter's own default - `sasjs compile` injects `%webout` into every service - and an explicit list replaces that default, so it has to be restated there. `mpeinit` and `mpeterm` are listed because every service is compiled with a serviceInit program (`sasjs/utils/serviceinitsas9|viya|server.sas`) that declares both macros in its own header, so a service body may call them without declaring them. Add a macro to that list only when it reaches a file the same way - injected by the compiler, through serviceInit, testSetUp or another implicit provider - never to silence a genuinely missing declaration.
   `noBannedStrings` flags the literal words `error ` and `warning ` anywhere in a file, comments included. Reword prose; where the text has to stay (a real `%put`, a doxygen `@warning` tag), keep the log output identical with `%put %str(WARN)ING:` rather than a per-file override, and use `@sasjslint {"noBannedStrings": false}` only when the word itself is the content.
+  Hook scripts (`sasjs/services/hooks/*.sas`) run inside the service that includes them, so the macros they call come from that service's session. They carry a file-local `@sasjslint {"ignoredMacros": [...]}` rather than a macro list of their own, because end users copy and edit these scripts and should not have to maintain a declaration section.
+  `noMacroIfInOpenCode` is on, so `%if` must sit inside a `%macro` definition. Where a conditional has to stay in a service body, wrap the block in a small macro and call it unconditionally (`%macro build_staged(); ... %end; %mend build_staged; %build_staged()`), or resolve the value in a data step with `call symputx`. Two traps: a `%let` inside such a wrapper is local, so a macro variable that must survive the block has to be written with `call symputx(...,'G')`; and open-code `%if` itself needs SAS 9.4M5, which the code no longer relies on.
 
 ## No external assets
 

CHANGELOG.md

@@ -1,3 +1,17 @@
+# [7.21.0](https://git.datacontroller.io/dc/dc/compare/v7.20.0...v7.21.0) (2026-10-29)
+
+
+### Bug Fixes
+
+* **client:** parse word-date values from VA parameters ([405b28d](https://git.datacontroller.io/dc/dc/commit/405b28dcccfe5994521bf63f0270d4877f544294))
+* **client:** parse word-date values from VA parameters ([ce5b1aa](https://git.datacontroller.io/dc/dc/commit/ce5b1aa96175d0fd1f048fea6d127cbc743804f2))
+* **makedata:** name the sasjs server group in the estate seed ([eeffa92](https://git.datacontroller.io/dc/dc/commit/eeffa92f9e973b323171bbefe2ef1cbc27e3aee7))
+
+
+### Features
+
+* **derived:** apply the core field scan in the MPE_DERIVED hook ([a4a0f9c](https://git.datacontroller.io/dc/dc/commit/a4a0f9cb2f7c0b6252405b71df64aae912d1b075))
+
 # [7.20.0](https://git.datacontroller.io/dc/dc/compare/v7.19.0...v7.20.0) (2026-10-28)
 
 

CONTRIBUTING.md

@@ -46,8 +46,31 @@ echo "YOUR_PASSPHRASE" | gpg --batch --yes --passphrase-fd 0 \
   --decrypt ./libraries/sheet-crypto.tgz.gpg
 
 npm i
+npm run postinstall
 ```
 
+The root and `sas/` `.npmrc` files set `ignore-scripts=true` too, so the composite `install`
+script in the root `package.json` (which installs both subprojects) does not run on `npm i` at the
+root. Install each part explicitly:
+
+```bash
+npm ci              # root tooling: gitleaks, prettier, semantic-release
+cd client && npm ci && npm run postinstall
+cd ../sas && npm ci
+```
+
+The client `.npmrc` is now tracked and sets `ignore-scripts=true`, so dependency install hooks do not
+run. Two of them matter here: the client's own `postinstall` - which generates
+`src/environments/version.ts`, imported throughout the app - and cypress's binary download. Run
+`npm run postinstall` after installing (and `npx cypress install` if you are running the Cypress
+suite).
+
+Registry credentials do **not** belong in `client/.npmrc`: it is a tracked policy file, and a token
+written there is one `git add -A` away from being committed. Keep the token in `~/.npmrc`, npm's
+default user config, or point npm at a file outside the repo with
+`NPM_CONFIG_USERCONFIG=/path/to/npmrc`. The pre-commit hook rejects a staged `client/.npmrc` that
+holds `_authToken`, `_auth` or `_password`.
+
 [Handsontable](https://www.npmjs.com/package/handsontable)
 
 The licence key comes from the backend licence: the decrypted licence carries `hot_license_key`,

client/.npmrc

@@ -0,0 +1,5 @@
+ignore-scripts=true
+save-exact=true
+# dependency install hooks are skipped: the client's own postinstall generates
+# src/environments/version.ts and the hot icons, so run `npm run postinstall`
+# after installing (the root install script does this for you)

client/cypress.config.ts

@@ -54,6 +54,43 @@ export default defineConfig({
               err ? reject(err) : resolve(null)
             )
           })
+        },
+
+        /**
+         * Writes a copy of a fixture whose primary keys are renumbered from a
+         * random base, and returns the path to write it to.
+         *
+         * An upload that matches what the table already holds produces no
+         * changes, and a changeset with no changes has no approval waiting - so
+         * re-running a spec that uploads the same file finds nothing on the
+         * approve screen.  That is not a UI fault, and a retry after a partial
+         * first attempt hits it every time.  New keys make the upload a real
+         * insert whichever attempt gets there.
+         */
+        makeUniqueExcel({ source, dest }: { source: string; dest: string }) {
+          // eslint-disable-next-line @typescript-eslint/no-var-requires
+          const ExcelJS = require('exceljs')
+          const workbook = new ExcelJS.Workbook()
+          const base = 1000000 + Math.floor(Math.random() * 8000000)
+
+          return workbook.xlsx.readFile(source).then(() => {
+            const sheet = workbook.worksheets[0]
+            let rowNumber = 0
+
+            sheet.eachRow((row: any, index: number) => {
+              if (index === 1) return // header
+              row.getCell(1).value = base + rowNumber++
+            })
+
+            return workbook.xlsx.writeFile(dest).then(() => dest)
+          })
+        },
+
+        /** removes one file - deleteFolder would take the source fixture too */
+        deleteFile(filePath: string) {
+          return new Promise((resolve) => {
+            ;(fs as any).rm(filePath, { force: true }, () => resolve(null))
+          })
         }
       })
 

client/cypress/e2e/download.cy.ts

@@ -7,12 +7,21 @@ const serverType = Cypress.env('serverType')
 const libraryToOpenIncludes = Cypress.env(`libraryToOpenIncludes_${serverType}`)
 const downloadsFolder = Cypress.config('downloadsFolder')
 
+/* set by test 1 and removed afterwards, since it is written per attempt */
+let generatedExcel = ''
+
 import { deleteDownloadsFolder } from '../util/deleteDownloadFolder'
 
 context('download files test: ', function () {
   this.beforeAll(() => {
     cy.visit(`${hostUrl}/SASLogon/logout`)
-    cy.loginAndUpdateValidKey()
+    /* the estate ships no licence, so it runs on the free tier - and the free
+      tier disables file upload, which is the whole point of the audit case.
+      Asking for a licensed estate here means the spec does not depend on
+      another spec having planted a key first: the key persists in the estate's
+      config, so whether this case could run depended on the order the specs
+      were sharded into. */
+    cy.loginAndUpdateValidKey(true)
   })
 
   this.beforeEach(() => {
@@ -37,22 +46,84 @@ context('download files test: ', function () {
 
   this.afterEach(() => {
     deleteDownloadsFolder()
+
+    if (generatedExcel) {
+      cy.task('deleteFile', 'cypress/fixtures/excels/' + generatedExcel)
+      generatedExcel = ''
+    }
   })
 
   it('1 | downloads audit file', (done) => {
-    visitPage('approve/toapprove')
+    /* The approve grid renders a download button per table that is waiting for
+      approval, and a freshly seeded estate has none, so create one: upload a
+      fixture to the test table and submit it from the stage screen.  The row is
+      then found on the approve screen by the table it names, because the stage
+      route does not carry that table's id.
+      The submitted row is left behind: CI recreates the appLoc and re-runs
+      makedata before the specs.  On an estate reused by hand the rows accumulate
+      and the approve grid paginates, so a row pushed past the first page would
+      not be found - the table name still anchors the assertion, but it cannot
+      rescue a row that is not on the page. */
+    /* the fixture is renumbered before every attempt: an upload identical to
+      what the table already holds produces no changes, and a changeset with no
+      changes leaves nothing waiting for approval.  On a retry that is exactly
+      what happens - the first attempt's rows are already in the table - so the
+      keys come from a random base and the upload is always an insert. */
+    generatedExcel = 'generated_' + Date.now() + '.xlsx'
 
-    cy.get('.app-loading', { timeout: longerCommandTimeout })
-      .should('not.exist')
-      .then(() => {
-        cy.get('.btn.btn-success')
-          .should('be.visible')
-          .then((buttons) => {
-            buttons[0].click()
-            const id = buttons[0].id
+    cy.task('makeUniqueExcel', {
+      source: 'cypress/fixtures/excels/regular_excel.xlsx',
+      dest: 'cypress/fixtures/excels/' + generatedExcel
+    })
 
-            checkForFileDownloaded(id, 'zip', () => done())
-          })
+    openTableFromTree(libraryToOpenIncludes, 'mpe_x_test')
+
+    attachExcelFile(generatedExcel)
+
+    submitExcel()
+
+    /* the stage route carries a load reference, not the table id: an upload
+      lands on /stage/DC20261031-201657.470_601010_4449 rather than on
+      /stage/<table id>, so taking the id from the URL looked for a button that
+      could never exist.  The row is found by the table it names, and the id the
+      download is named after is read from the button itself. */
+    visitPage('review/approve')
+
+    cy.get('.app-loading', { timeout: longerCommandTimeout }).should(
+      'not.exist'
+    )
+
+    cy.get('clr-datagrid clr-dg-row', { timeout: longerCommandTimeout }).should(
+      ($rows) => {
+        const texts = Array.from($rows).map((row: any) =>
+          row.innerText.replace(/\s+/g, ' ')
+        )
+
+        expect(
+          texts.some((text: string) => text.includes('MPE_X_TEST')),
+          `no approval row names MPE_X_TEST - rows: ${texts
+            .slice(0, 5)
+            .join(' | ')
+            .slice(0, 300)}`
+        ).to.be.true
+      }
+    )
+
+    cy.get('clr-datagrid clr-dg-row')
+      .filter(':contains("MPE_X_TEST")')
+      .first()
+      .find('[aria-label="Download audit file"]')
+      .should('have.length', 1)
+      .then(($btn) => {
+        const tableId = $btn.attr('id') as string
+
+        expect(tableId, 'the approve button carries the table id').to.be.a(
+          'string'
+        )
+
+        cy.wrap($btn).click()
+
+        checkForFileDownloaded(tableId, 'zip', () => done())
       })
   })
 
@@ -83,7 +154,7 @@ context('download files test: ', function () {
 
     openDownloadModal(() => {
       cy.get('select')
-        .select('Excel')
+        .select('EXCEL')
         .then(() => {
           cy.get('.btn.btn-sm.btn-success-outline').then((button) => {
             button.trigger('click')
@@ -103,7 +174,7 @@ context('download files test: ', function () {
 
     openDownloadModal(() => {
       cy.get('select')
-        .select('SAS Datalines')
+        .select('SAS')
         .then(() => {
           cy.get('.btn.btn-sm.btn-success-outline').then((button) => {
             button.trigger('click')
@@ -123,7 +194,7 @@ context('download files test: ', function () {
 
     openDownloadModal(() => {
       cy.get('select')
-        .select('SAS DDL')
+        .select('SAS_DDL')
         .then(() => {
           cy.get('.btn.btn-sm.btn-success-outline').then((button) => {
             button.trigger('click')
@@ -143,7 +214,7 @@ context('download files test: ', function () {
 
     openDownloadModal(() => {
       cy.get('select')
-        .select('TSQL DDL')
+        .select('TSQL_DDL')
         .then(() => {
           cy.get('.btn.btn-sm.btn-success-outline').then((button) => {
             button.trigger('click')
@@ -163,7 +234,7 @@ context('download files test: ', function () {
 
     openDownloadModal(() => {
       cy.get('select')
-        .select('PGSQL DDL')
+        .select('PGSQL_DDL')
         .then(() => {
           cy.get('.btn.btn-sm.btn-success-outline').then((button) => {
             button.trigger('click')
@@ -209,18 +280,23 @@ const checkForFileDownloaded = (
 }
 
 const openDownloadModal = (callback?: any) => {
-  cy.get('.btn.btn-sm.btn-outline.filterSide.dropdown-toggle')
+  /* filterSide is the viewer's options dropdown; it no longer carries a
+    dropdown-toggle class - see filter-panel.cy.ts and row-cell-limits.cy.ts.
+    Its entries are div[clrDropdownItem], not buttons, so select Download by its
+    text the way filter-panel.cy.ts selects Filter. */
+  cy.get('.btn.btn-sm.btn-outline.filterSide', {
+    timeout: longerCommandTimeout
+  })
     .click()
     .then(() => {
-      cy.get('clr-dropdown-menu button').then((buttons) => {
-        for (let button of buttons) {
-          if (button.innerText.toLowerCase().includes('download')) {
-            button.click()
+      /* the click handler is on the clrDropdownItem element, not on the span
+        holding the text, and the whole label is matched: more than one entry
+        may mention Download, and an ambiguous cy.contains fails outright */
+      cy.contains('clr-dropdown-menu [clrDropdownItem]', /^\s*Download\s*$/, {
+        timeout: longerCommandTimeout
+      }).click()
 
-            if (callback) callback()
-          }
-        }
-      })
+      if (callback) callback()
     })
 }
 
@@ -259,3 +335,45 @@ const openTableFromTree = (libNameIncludes: string, tablename: string) => {
       })
     })
 }
+
+const fixturePath = 'excels'
+
+/**
+ * Uploads a fixture to the open table.  The editor renders the file input inside
+ * its upload modal, which the Upload button in the button bar opens, so that
+ * button is clicked first - by its label, not by position, so a change to the
+ * bar's button order cannot break this.
+ */
+const attachExcelFile = (excelFilename: string) => {
+  cy.get('.buttonBar button', { timeout: longerCommandTimeout })
+    .contains('Upload')
+    .click()
+
+  cy.get('input[type="file"]#file-upload', { timeout: longerCommandTimeout })
+    .should('have.length', 1)
+    .attachFile(`/${fixturePath}/${excelFilename}`)
+
+  /* a parsed upload raises the info modal with this title.  The modal component
+    is the same one aborts use, so the title is what distinguishes them: any
+    other text here - an abort, a validation error - fails the case with its own
+    message rather than being clicked through. */
+  cy.get('.clr-abort-modal .modal-title', { timeout: longerCommandTimeout })
+    .should('be.visible')
+    .invoke('text')
+    .should('include', 'Table Found')
+
+  cy.get('.clr-abort-modal .modal-footer button').contains('Close').click()
+}
+
+const submitExcel = () => {
+  cy.get('.buttonBar button.preview-submit', {
+    timeout: longerCommandTimeout
+  }).click()
+
+  /* the URL is already a stage route before the submit lands, so waiting on it
+    proves nothing; the editor leaving preview mode is the signal that the
+    submit was accepted */
+  cy.get('.buttonBar button.preview-submit', {
+    timeout: longerCommandTimeout
+  }).should('not.exist')
+}

client/cypress/e2e/excel-multi-load.cy.ts

@@ -193,6 +193,43 @@ context('excel multi load tests: ', function () {
     })
   })
 
+  // The file's own status line: size and stage. The stages are momentary, so
+  // the dataset lookups are held open to make "Fetching the dataset details"
+  // assertable (and visible in the recording).
+  it('4 | Shows the file size and the loading stage while the file is processed', () => {
+    attachExcelFile('multi_load_test_1.xlsx', () => {
+      cy.contains('File size:')
+        .should('be.visible')
+        .invoke('text')
+        .should('match', /File size:\s*[\d.]+ MB/)
+
+      // The sheet scan is quick; from then on the file waits for the user.
+      cy.get('#file-loading-stage', { timeout: longerCommandTimeout })
+        .should('be.visible')
+        .and('contain.text', 'File ready to load')
+
+      // The adapter posts to /SASjsApi/stp/execute/?_program=... - a glob on
+      // 'execute**' does not match the trailing slash, so match on the path.
+      cy.intercept({ url: /SASjsApi\/stp\/execute/ }, (req) => {
+        req.on('response', (res) => {
+          res.setDelay(1500)
+        })
+      })
+
+      cy.get('#continue-btn').trigger('click')
+
+      cy.get('#file-loading-stage', { timeout: longerCommandTimeout })
+        .should('be.visible')
+        .and('contain.text', 'Fetching the dataset details')
+
+      // Once every sheet has been extracted the stage gives way to the hint.
+      cy.get('#file-loading-stage', { timeout: longerCommandTimeout }).should(
+        'not.exist'
+      )
+      cy.contains('Please select a dataset on the left').should('be.visible')
+    })
+  })
+
   this.afterEach(() => {
     colorLog(`TEST END -------------`, '#3498DB')
   })

client/cypress/e2e/excel.cy.ts

@@ -419,6 +419,46 @@ context('excel tests: ', function () {
     })
   })
 
+  it('26 | Discarding an upload that fails validation clears the red cells', () => {
+    openTableFromTree(libraryToOpenIncludes, 'mpe_x_test')
+
+    // The loaded table's row count is what the grid must return to - the
+    // preview replaces the data source with the parsed file.
+    cy.get('h4.editor-title', { timeout: longerCommandTimeout })
+      .invoke('text')
+      .should('match', /\(\d+ rows?/)
+      .then((titleBefore) => {
+        const rowsBefore = titleBefore.match(/\(([\d,]+) rows?/)?.[1]
+
+        attachExcelFile('mixed_content_excel.xlsx', () => {
+          // The preview paints the cells that fail validation red - the state
+          // the user is looking at when they decide to abandon the file.
+          cy.get('#hotTable .htInvalid', {
+            timeout: longerCommandTimeout
+          }).should('have.length.greaterThan', 0)
+
+          cy.get('.buttonBar .btn-upload-preview.btn-outline-danger').click()
+          cy.get('clr-modal .modal-footer .btn-success-outline').click()
+
+          // Back to the loaded table, with no red cells left over. cancelEdit
+          // re-runs the HOT validation after restoring the data; without that
+          // the invalid cell meta survives the restore and the grid stays red.
+          cy.get('h4.editor-title', { timeout: longerCommandTimeout }).should(
+            'contain.text',
+            `(${rowsBefore} rows`
+          )
+
+          cy.get('#hotTable .htInvalid').should('not.exist')
+
+          // And the page is back in its normal read-only state, not the
+          // preview's button bar.
+          cy.get('.btnCtrl .btn-primary', {
+            timeout: longerCommandTimeout
+          }).should('contain.text', 'Edit')
+        })
+      })
+  })
+
   // Large files break Cypress
 
   // it ('? | Uploads Excel with size of 5MB', (done) => {

client/cypress/e2e/full-table-search.cy.ts

@@ -331,7 +331,10 @@ const setFilterWithValue = (
   closeAutocompleteList('#vals_0_0')
   cy.get('#datalist_vals_0_0').should('have.class', 'hidden')
 
-  cy.get('.filter-modal button[aria-label="Ok"]').click()
+  // Live on the markup rather than a label: the filter modal's submit button
+  // has its text ("Ok") and no explicit aria-label, and the ARIA pass no
+  // longer synthesises one.
+  cy.contains('.filter-modal .modal-footer button', /^\s*Ok\s*$/).click()
 
   if (callback) callback()
 }

client/cypress/e2e/metanav.cy.ts

@@ -1,16 +1,13 @@
-const username = Cypress.env('username')
-const password = Cypress.env('password')
+export {}
+
 const hostUrl = Cypress.env('hosturl')
 const appLocation = Cypress.env('appLocation')
 const longerCommandTimeout = Cypress.env('longerCommandTimeout')
-const serverType = Cypress.env('serverType')
-const libraryToOpenIncludes = Cypress.env(`libraryToOpenIncludes_${serverType}`)
-const fixturePath = 'excels_general/'
 
 context('metanav tests: ', function () {
   this.beforeAll(() => {
     cy.visit(`${hostUrl}/SASLogon/logout`)
-    cy.loginAndUpdateValidKey()
+    cy.loginAndUpdateValidKey(true)
   })
 
   this.beforeEach(() => {
@@ -21,14 +18,34 @@ context('metanav tests: ', function () {
     visitPage('view/metadata')
   })
 
-  it('1 | Opens metadata object', (done) => {
-    openFirstMetadataFromTree(() => {
-      // BLOCKER
-      // For unkown reasons, .clr-accordion-header-button always null although it is present on the page.
-      cy.get('.clr-accordion-header-button').then((panelNodes: any) => {
-        panelNodes[0].querySelector('button').click()
-      })
-    })
+  it('1 | Opens metadata object', () => {
+    openFirstMetadataType()
+
+    /**
+     * Query the accordion at document level. The panel renders in the content
+     * area, while `.within(<tree node>)` scopes every following `cy.get` to
+     * that node alone - which finds nothing however long it retries.
+     */
+    cy.get('.clr-accordion-header-button', { timeout: longerCommandTimeout })
+      .should('have.length.greaterThan', 0)
+      .first()
+      // Clarity puts the class on the button itself, so it is the click target.
+      .click()
+
+    // The panel reports its own open state, so the assertion is on the app's
+    // marker rather than on the content happening to be present.
+    cy.get('.clr-accordion-panel-open').should('have.length', 1)
+    cy.get('.clr-accordion-header-button')
+      .first()
+      .should('have.attr', 'aria-expanded', 'true')
+
+    // Opening the panel loads the object's details: the association tree and
+    // the attribute grid both come from `metanav/metadetails`.
+    cy.get('.clr-accordion-inner-content clr-tree-node', {
+      timeout: longerCommandTimeout
+    }).should('have.length.greaterThan', 0)
+    cy.get('clr-dg-row').should('have.length.greaterThan', 0)
+    cy.get('clr-dg-cell').contains('Name')
   })
 
   this.afterEach(() => {
@@ -36,23 +53,16 @@ context('metanav tests: ', function () {
   })
 })
 
-const openFirstMetadataFromTree = (callback?: any) => {
-  cy.get('.app-loading', { timeout: longerCommandTimeout })
-    .should('not.exist')
-    .then(() => {
-      cy.get('.nav-tree clr-tree > clr-tree-node', {
-        timeout: longerCommandTimeout
-      }).then((treeNodes: any) => {
-        let firstMetaNode
-
-        firstMetaNode = treeNodes[1]
-
-        cy.get(firstMetaNode).within(() => {
-          cy.get('.clr-treenode-content').click()
-          callback()
-        })
-      })
-    })
+const openFirstMetadataType = () => {
+  cy.get('.app-loading', { timeout: longerCommandTimeout }).should('not.exist')
+  // The metadata types are the tree nodes after the search box; the search box
+  // is its own node (`.search-node`), so selecting by position would tie the
+  // spec to that detail. The <p> is the element that owns the click handler.
+  cy.get('.nav-tree clr-tree > clr-tree-node:not(.search-node) p', {
+    timeout: longerCommandTimeout
+  })
+    .first()
+    .click()
 }
 
 const visitPage = (url: string) => {

client/package-lock.json

@@ -22,7 +22,7 @@
         "@clr/ui": "file:libraries/clr-ui-17.9.0.tgz",
         "@handsontable/angular-wrapper": "18.1.2",
         "@sasjs/adapter": "5.1.2",
-        "@sasjs/utils": "4.0.0",
+        "@sasjs/utils": "4.3.1",
         "@sheet/crypto": "file:libraries/sheet-crypto.tgz",
         "@types/d3-graphviz": "^2.6.7",
         "@types/text-encoding": "0.0.35",
@@ -82,6 +82,7 @@
         "@types/randombytes": "^2.0.3",
         "@typescript-eslint/eslint-plugin": "8.65.0",
         "@typescript-eslint/parser": "8.65.0",
+        "archiver": "8.0.0",
         "core-js": "^2.5.4",
         "cypress": "^15.19.0",
         "cypress-file-upload": "^5.0.8",
@@ -5208,6 +5209,12 @@
         "node": ">=18.0.0"
       }
     },
+    "node_modules/@isaacs/nonrecursive-glob-match": {
+      "version": "1.0.3",
+      "resolved": "https://registry.npmjs.org/@isaacs/nonrecursive-glob-match/-/nonrecursive-glob-match-1.0.3.tgz",
+      "integrity": "sha512-SI7kyKHq4XXDRozw+r9GlRZkia1ZkFrpLW98r2IC2Bpr8YkFGE3fvjzmxCG0x78A++69HmDNrDvSMYBZmHGpKA==",
+      "license": "BlueOak-1.0.0"
+    },
     "node_modules/@istanbuljs/schema": {
       "version": "0.1.6",
       "resolved": "https://registry.npmjs.org/@istanbuljs/schema/-/schema-0.1.6.tgz",
@@ -8722,9 +8729,9 @@
       }
     },
     "node_modules/@sasjs/utils": {
-      "version": "4.0.0",
-      "resolved": "https://registry.npmjs.org/@sasjs/utils/-/utils-4.0.0.tgz",
-      "integrity": "sha512-lARoCnRM4gSVf8ShoP5elHi7W28K/14MK037dQ0ABgMlP3+r6OuhAZYwK/a70tED47tKxyIoCAKLdP6jrG0ewQ==",
+      "version": "4.3.1",
+      "resolved": "https://registry.npmjs.org/@sasjs/utils/-/utils-4.3.1.tgz",
+      "integrity": "sha512-P98zjjfrhVJ7/ubKTWTn8Oiyb4/a+SIfVQOIa9oMXZrsRsZ5RTLj40sl6jmv1iCxbsqfziOrnNc0Yu6gx5Qp1A==",
       "license": "ISC",
       "dependencies": {
         "@fast-csv/format": "5.0.7",
@@ -9877,9 +9884,9 @@
       }
     },
     "node_modules/@typescript-eslint/types": {
-      "version": "8.72.0",
-      "resolved": "https://registry.npmjs.org/@typescript-eslint/types/-/types-8.72.0.tgz",
-      "integrity": "sha512-Adfoy59kxNEjT5E5fnyrp7GpHYuWs8ONbdohu4ez6IU46qDq8LltG90tLmkucZB2uhYL8mqBjtXqGd8yhfjXqQ==",
+      "version": "8.72.1",
+      "resolved": "https://registry.npmjs.org/@typescript-eslint/types/-/types-8.72.1.tgz",
+      "integrity": "sha512-LmE5IhQet1BWOWEXqBrVtBPsMAOQ6Nzk35d+WLh3mk4EUF33sqlfxjio1MdYeSvKSG5skactKKuixBIKXhL4Rw==",
       "dev": true,
       "license": "MIT",
       "engines": {
@@ -9946,17 +9953,17 @@
       }
     },
     "node_modules/@typescript-eslint/utils": {
-      "version": "8.72.0",
-      "resolved": "https://registry.npmjs.org/@typescript-eslint/utils/-/utils-8.72.0.tgz",
-      "integrity": "sha512-0up3oArJZBYnzOaEXKo8DmI43QvMZyPoFEt9LWzRJcyl/RcJjfKKPebknN1VbTgXBf3M/KggUh5uUMdURLnseg==",
+      "version": "8.72.1",
+      "resolved": "https://registry.npmjs.org/@typescript-eslint/utils/-/utils-8.72.1.tgz",
+      "integrity": "sha512-5v5liypurJBFHCBNqh/RjsymAD+VWcE0djuGY/BUqp5fC3Qn+luWDEksjuTtWjPn6i5zQlUtkZCRG/CYG0yCkA==",
       "dev": true,
       "license": "MIT",
       "peer": true,
       "dependencies": {
         "@eslint-community/eslint-utils": "^4.9.1",
-        "@typescript-eslint/scope-manager": "8.72.0",
-        "@typescript-eslint/types": "8.72.0",
-        "@typescript-eslint/typescript-estree": "8.72.0"
+        "@typescript-eslint/scope-manager": "8.72.1",
+        "@typescript-eslint/types": "8.72.1",
+        "@typescript-eslint/typescript-estree": "8.72.1"
       },
       "engines": {
         "node": "^18.18.0 || ^20.9.0 || >=21.1.0"
@@ -9971,15 +9978,15 @@
       }
     },
     "node_modules/@typescript-eslint/utils/node_modules/@typescript-eslint/project-service": {
-      "version": "8.72.0",
-      "resolved": "https://registry.npmjs.org/@typescript-eslint/project-service/-/project-service-8.72.0.tgz",
-      "integrity": "sha512-bzHngBbmIgzj6TsTjVUCntGx6FFLgE7mPjpO0PzHycOaZHXqS43XnrkFIHRS16LLr/xO6FHSXG3GJFo/F4bSBQ==",
+      "version": "8.72.1",
+      "resolved": "https://registry.npmjs.org/@typescript-eslint/project-service/-/project-service-8.72.1.tgz",
+      "integrity": "sha512-fAuVMkwdO/huI8nyo/WKnNKf6Gag93S56Rh3Ts5mGcavwSdaN8tRg1gCeIGX8PF+nzp1BjGfO4gHhGzW1J6jTg==",
       "dev": true,
       "license": "MIT",
       "peer": true,
       "dependencies": {
-        "@typescript-eslint/tsconfig-utils": "^8.72.0",
-        "@typescript-eslint/types": "^8.72.0",
+        "@typescript-eslint/tsconfig-utils": "^8.72.1",
+        "@typescript-eslint/types": "^8.72.1",
         "debug": "^4.4.3"
       },
       "engines": {
@@ -9994,15 +10001,15 @@
       }
     },
     "node_modules/@typescript-eslint/utils/node_modules/@typescript-eslint/scope-manager": {
-      "version": "8.72.0",
-      "resolved": "https://registry.npmjs.org/@typescript-eslint/scope-manager/-/scope-manager-8.72.0.tgz",
-      "integrity": "sha512-JewPyJOYcXBSAS7PIgufscHOwZqhTGyBRiQJhhuDln66tMchxvvt7nJ90JKKIoOelmdHA3kAQbm9cEUHc4JSEA==",
+      "version": "8.72.1",
+      "resolved": "https://registry.npmjs.org/@typescript-eslint/scope-manager/-/scope-manager-8.72.1.tgz",
+      "integrity": "sha512-lC/t6eMCZZQAAKZEYGL6RDVSuFehpdmMT6h+nJIjIa6oylN/PwDgDVtEIbB7O9xFfZYtZAYLoLlDNWKJhtF+uQ==",
       "dev": true,
       "license": "MIT",
       "peer": true,
       "dependencies": {
-        "@typescript-eslint/types": "8.72.0",
-        "@typescript-eslint/visitor-keys": "8.72.0"
+        "@typescript-eslint/types": "8.72.1",
+        "@typescript-eslint/visitor-keys": "8.72.1"
       },
       "engines": {
         "node": "^18.18.0 || ^20.9.0 || >=21.1.0"
@@ -10013,9 +10020,9 @@
       }
     },
     "node_modules/@typescript-eslint/utils/node_modules/@typescript-eslint/tsconfig-utils": {
-      "version": "8.72.0",
-      "resolved": "https://registry.npmjs.org/@typescript-eslint/tsconfig-utils/-/tsconfig-utils-8.72.0.tgz",
-      "integrity": "sha512-0vbxCf7xLVPSjcOdm6eQbgViH8EDJ8xnl1laLT9yoqSz2YmcSfSg5Vdl0JUvZf6rcuJ0ZivJIuMD4edTg8obaA==",
+      "version": "8.72.1",
+      "resolved": "https://registry.npmjs.org/@typescript-eslint/tsconfig-utils/-/tsconfig-utils-8.72.1.tgz",
+      "integrity": "sha512-e4D5R5vpeAEPBxT2BAzsFQL3sbwyeeeo5+aczgNSRK1V734jE7pTqaGFU+fXapWEQwnZbpFaTn32+Z8DwS6OTg==",
       "dev": true,
       "license": "MIT",
       "peer": true,
@@ -10031,17 +10038,17 @@
       }
     },
     "node_modules/@typescript-eslint/utils/node_modules/@typescript-eslint/typescript-estree": {
-      "version": "8.72.0",
-      "resolved": "https://registry.npmjs.org/@typescript-eslint/typescript-estree/-/typescript-estree-8.72.0.tgz",
-      "integrity": "sha512-gwLfFgWcw15tN2KXDCJvxEOQjPn+P3j/n3eSqkZ5sX4VpW9Cosodr+tWwBZRvJT3TcKx6HnPRGZMCNAiP1Xhug==",
+      "version": "8.72.1",
+      "resolved": "https://registry.npmjs.org/@typescript-eslint/typescript-estree/-/typescript-estree-8.72.1.tgz",
+      "integrity": "sha512-DAlUoUV93hIvrtIQLwCbmZQYVZwOMhiHJrOVUUIWicFZaIYjTRqlY3I/NZevfkLmSqkNHRu2qYDcq9lrn16oOQ==",
       "dev": true,
       "license": "MIT",
       "peer": true,
       "dependencies": {
-        "@typescript-eslint/project-service": "8.72.0",
-        "@typescript-eslint/tsconfig-utils": "8.72.0",
-        "@typescript-eslint/types": "8.72.0",
-        "@typescript-eslint/visitor-keys": "8.72.0",
+        "@typescript-eslint/project-service": "8.72.1",
+        "@typescript-eslint/tsconfig-utils": "8.72.1",
+        "@typescript-eslint/types": "8.72.1",
+        "@typescript-eslint/visitor-keys": "8.72.1",
         "debug": "^4.4.3",
         "minimatch": "^10.2.2",
         "semver": "^7.7.3",
@@ -10060,14 +10067,14 @@
       }
     },
     "node_modules/@typescript-eslint/utils/node_modules/@typescript-eslint/visitor-keys": {
-      "version": "8.72.0",
-      "resolved": "https://registry.npmjs.org/@typescript-eslint/visitor-keys/-/visitor-keys-8.72.0.tgz",
-      "integrity": "sha512-E2mroviasaWecDzTGiGMtuEguuLVOeRggnrIn93CJuxYJzQBHbyymlT08aVQydbBal5538o+JkBKXh2sRuCyIA==",
+      "version": "8.72.1",
+      "resolved": "https://registry.npmjs.org/@typescript-eslint/visitor-keys/-/visitor-keys-8.72.1.tgz",
+      "integrity": "sha512-SB8xqipSzZHUePCCLNeYxqnxq83X61/qzNhjQGbTfjxGZIeOEP7tBgWYX05Tcd6CgE02WdTJnh5b8kMZJ/+bnQ==",
       "dev": true,
       "license": "MIT",
       "peer": true,
       "dependencies": {
-        "@typescript-eslint/types": "8.72.0",
+        "@typescript-eslint/types": "8.72.1",
         "eslint-visitor-keys": "^5.0.0"
       },
       "engines": {
@@ -15246,9 +15253,9 @@
       "license": "ISC"
     },
     "node_modules/follow-redirects": {
-      "version": "1.16.1",
-      "resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.16.1.tgz",
-      "integrity": "sha512-FNvFGzoMLWmE6Yj9spb/zjd7yiNCHiAW9/Tg9CXrQ8wuu32HtlJOwWO11OJafl5FfY3DxTdQ0vj42zU1kvv5jg==",
+      "version": "1.16.2",
+      "resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.16.2.tgz",
+      "integrity": "sha512-/kVP2U8z596Ia0gwFrwyIeo74ta5aVmSeU1FfAUMAmcEP5EP7zydmag3SLiN64NaRKef3lxsq7i/g7ltJQDGow==",
       "funding": [
         {
           "type": "individual",
@@ -19280,6 +19287,7 @@
       "version": "10.2.6",
       "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-10.2.6.tgz",
       "integrity": "sha512-vpLQEs+VLCr1nU0BXS07maYoFwlDAH0gngQuuttxIwutDFEMHq2blX+8vpgxDdK3J1PwjCJiep77OitTZ4Ll1A==",
+      "dev": true,
       "license": "BlueOak-1.0.0",
       "dependencies": {
         "brace-expansion": "^5.0.8"
@@ -19601,9 +19609,9 @@
       }
     },
     "node_modules/mocha/node_modules/argparse": {
-      "version": "3.0.3",
-      "resolved": "https://registry.npmjs.org/argparse/-/argparse-3.0.3.tgz",
-      "integrity": "sha512-Sn98ftV3KfsO8xfLPHG6oT8y+b0yvbrLgZjgPDXsharNY15tTAjqIsIju4S32nl6/iEUNNE4v+yL2Zhm8ooP3w==",
+      "version": "3.0.4",
+      "resolved": "https://registry.npmjs.org/argparse/-/argparse-3.0.4.tgz",
+      "integrity": "sha512-fwWK3Zq22l4XJJjx5mcBHGikIk+hJqO+32dPm0Jur7+rtG5x1scOjSfdT9ZKeWHPpsya/l0HHK4tDImBkTrtvA==",
       "dev": true,
       "funding": [
         {
@@ -22269,6 +22277,22 @@
         "url": "https://github.com/sponsors/yqnn"
       }
     },
+    "node_modules/readdir-glob/node_modules/minimatch": {
+      "version": "11.0.3",
+      "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-11.0.3.tgz",
+      "integrity": "sha512-iny/40PEGkNJQ5/Pv6z3nm6FbRAby32GeZRjCUJk2WnNDbgPSTcdI+DDvLqsgBdD4cD5sMtMheqPPnehCCa8Nw==",
+      "license": "BlueOak-1.0.0",
+      "dependencies": {
+        "@isaacs/nonrecursive-glob-match": "^1.0.3",
+        "brace-expansion": "^5.0.12"
+      },
+      "engines": {
+        "node": "18 || 20 || >=22"
+      },
+      "funding": {
+        "url": "https://github.com/sponsors/isaacs"
+      }
+    },
     "node_modules/readdir-scoped-modules": {
       "version": "1.1.0",
       "resolved": "https://registry.npmjs.org/readdir-scoped-modules/-/readdir-scoped-modules-1.1.0.tgz",
@@ -25171,9 +25195,9 @@
       }
     },
     "node_modules/vis-data": {
-      "version": "8.0.5",
-      "resolved": "https://registry.npmjs.org/vis-data/-/vis-data-8.0.5.tgz",
-      "integrity": "sha512-tRQKcTGaclo60rTY1j4v7BgD9v5jfD3wl+JgR1q6I4AI7go3QH5OjMCLIuZPpL2sf4IRMlwyPlRLnK4m+Phjsw==",
+      "version": "8.0.6",
+      "resolved": "https://registry.npmjs.org/vis-data/-/vis-data-8.0.6.tgz",
+      "integrity": "sha512-6NjsyqWL18ithgkpPqfYGvwx7JlrxgO0MkMJdZus4eovFl6ebUjNcn/UEzgm5NWvCAz4psUgkvcdLyAXFxeeDg==",
       "dev": true,
       "license": "(Apache-2.0 OR MIT)",
       "peer": true,
@@ -25206,9 +25230,9 @@
       }
     },
     "node_modules/vis-util": {
-      "version": "6.0.2",
-      "resolved": "https://registry.npmjs.org/vis-util/-/vis-util-6.0.2.tgz",
-      "integrity": "sha512-sYawqWllCqwmTPLurRgPChmfRppbWI/XoBWdGNM6N2s4XuVR8VO8gtz/v9AvrAAOfZxel/U5UBNOwyhFn0qb7A==",
+      "version": "6.0.3",
+      "resolved": "https://registry.npmjs.org/vis-util/-/vis-util-6.0.3.tgz",
+      "integrity": "sha512-2ar+MN4JYYfhRSh+sds7o0Sg7/i/mNMVXv32DaMg5I4N1AJjTaHvXqNtHcISXl4WyaVkJ6twiCfqStudO2TYFg==",
       "dev": true,
       "license": "(Apache-2.0 OR MIT)",
       "peer": true,

client/package.json

@@ -3,43 +3,44 @@
   "description": "DataController Client",
   "angular-cli": {},
   "scripts": {
-    "start": "node --max_old_space_size=4096 node_modules/@angular/cli/bin/ng serve",
-    "prestart": "npm run generate-eula",
-    "generate-eula": "node ./src/eula.ts",
-    "license-checker": "node licenseChecker.js",
-    "prebuild": "node ./src/version.ts && npm run generate-eula && npm run license-checker",
+    "add-githook": "[ -d ../.git ] && git config core.hooksPath ./.git-hooks || true",
+    "audit:prod": "npm audit --omit=dev",
     "build": "node --max_old_space_size=4096 node_modules/@angular/cli/bin/ng build --configuration production",
-    "postbuild": "rimraf dist/3rdpartylicenses.txt",
     "build-dev": "node --max_old_space_size=4096 node_modules/@angular/cli/bin/ng build --configuration development",
     "build-watch": "node --max_old_space_size=4096 node_modules/@angular/cli/bin/ng build --configuration development --watch",
-    "sync": "./node_modules/.bin/watch --wait=3 \"echo Account: ${npm_config_account} && npm run deploy_${npm_config_server} --account=${npm_config_account} && notify-send 'Data Controller' 'App is synced!' ; echo 'App is synced!'\" dist",
-    "deploy_sas9": "rsync -avhe ssh ./dist/* --delete ${npm_config_account}@sas.4gl.io:/opt/sas/sas9/config/Lev1/Web/WebServer/htdocs/${npm_config_account}/dc/dev",
-    "deploy_viya": "rsync -avhe ssh ./dist/* --delete ${npm_config_account}@sas.4gl.io:/var/www/html/${npm_config_account}/dc/dev",
-    "deploy_sasjs": "rsync -avhe ssh ./dist/* --delete root@${npm_config_account}.4gl.io:/var/www/html/dc/dev",
-    "viyabuild": "cd build; ./viyabuild.sh",
-    "lint": "npm run lint:fix",
-    "lint:fix": "npx prettier --write \"{src,test}/**/*.{ts,tsx,js,jsx,html,css,sass,less,yml,md,graphql}\" \"cypress/e2e/*.cy.ts\"",
-    "lint:fix:silent": "npx prettier --log-level silent --write \"{src,test}/**/*.{ts,tsx,js,jsx,html,css,sass,less,yml,md,graphql}\" \"cypress/e2e/*.cy.ts\"",
-    "lint:check": "npx prettier --check \"{src,test}/**/*.{ts,tsx,js,jsx,html,css,sass,less,yml,md,graphql}\" \"cypress/e2e/*.cy.ts\"",
-    "lint:check:silent": "npx prettier --log-level silent --check \"{src,test}/**/*.{ts,tsx,js,jsx,html,css,sass,less,yml,md,graphql}\" \"cypress/e2e/*.cy.ts\"",
-    "test": "npx ng test",
-    "test:headless": "npx ng test --no-watch --no-progress --browsers ChromeHeadlessCI",
-    "watch": "ng test watch=true",
-    "pree2e": "webdriver-manager update",
-    "e2e": "protractor protractor.config.js",
-    "postinstall": "node ./src/version.ts && npm run add-githook && node ./scripts/strip-clr-base64-fonts.mjs && node ./scripts/gen-hot-icons.mjs",
-    "add-githook": "[ -d ../.git ] && git config core.hooksPath ./.git-hooks || true",
-    "cypress": "cypress open",
-    "cy:run": "cypress run",
-    "audit:prod": "npm audit --omit=dev",
-    "sasdocs": "sasjs doc && ./sasjs/utils/deploydocs.sh",
+    "check:archiver": "node scripts/check-archiver.mjs",
     "compodoc:build": "compodoc -p tsconfig.doc.json --name 'Data Controller Client'",
     "compodoc:build-and-serve": "compodoc -p tsconfig.doc.json -s --name 'Data Controller Client'",
     "compodoc:serve": "compodoc -s --name 'Data Controller Client'",
+    "cy:run": "cypress run",
+    "cypress": "cypress open",
+    "deploy_sas9": "rsync -avhe ssh ./dist/* --delete ${npm_config_account}@sas.4gl.io:/opt/sas/sas9/config/Lev1/Web/WebServer/htdocs/${npm_config_account}/dc/dev",
+    "deploy_sasjs": "rsync -avhe ssh ./dist/* --delete root@${npm_config_account}.4gl.io:/var/www/html/dc/dev",
+    "deploy_viya": "rsync -avhe ssh ./dist/* --delete ${npm_config_account}@sas.4gl.io:/var/www/html/${npm_config_account}/dc/dev",
+    "e2e": "protractor protractor.config.js",
+    "generate-eula": "node ./src/eula.ts",
+    "license-checker": "node licenseChecker.js",
     "lighthouse": "lhci autorun",
+    "lint": "npm run lint:fix",
+    "lint:check": "npx prettier --check \"{src,test}/**/*.{ts,tsx,js,jsx,html,css,sass,less,yml,md,graphql}\" \"cypress/e2e/*.cy.ts\"",
+    "lint:check:silent": "npx prettier --log-level silent --check \"{src,test}/**/*.{ts,tsx,js,jsx,html,css,sass,less,yml,md,graphql}\" \"cypress/e2e/*.cy.ts\"",
+    "lint:fix": "npx prettier --write \"{src,test}/**/*.{ts,tsx,js,jsx,html,css,sass,less,yml,md,graphql}\" \"cypress/e2e/*.cy.ts\"",
+    "lint:fix:silent": "npx prettier --log-level silent --write \"{src,test}/**/*.{ts,tsx,js,jsx,html,css,sass,less,yml,md,graphql}\" \"cypress/e2e/*.cy.ts\"",
     "ng": "ng",
+    "postbuild": "rimraf dist/3rdpartylicenses.txt",
+    "postinstall": "node ./src/version.ts && npm run add-githook && node ./scripts/strip-clr-base64-fonts.mjs && node ./scripts/gen-hot-icons.mjs",
+    "prebuild": "node ./src/version.ts && npm run generate-eula && npm run license-checker",
+    "pree2e": "webdriver-manager update",
+    "prestart": "npm run generate-eula",
+    "sasdocs": "sasjs doc && ./sasjs/utils/deploydocs.sh",
+    "start": "node --max_old_space_size=4096 node_modules/@angular/cli/bin/ng serve",
+    "sync": "./node_modules/.bin/watch --wait=3 \"echo Account: ${npm_config_account} && npm run deploy_${npm_config_server} --account=${npm_config_account} && notify-send 'Data Controller' 'App is synced!' ; echo 'App is synced!'\" dist",
+    "test": "npx ng test",
+    "test:headless": "npx ng test --no-watch --no-progress --browsers ChromeHeadlessCI",
+    "test:tz-east": "TZ=Pacific/Auckland npx ng test --no-watch --no-progress --browsers ChromeHeadlessCI --include='**/va-*.spec.ts'",
     "test:tz-west": "TZ=America/New_York npx ng test --no-watch --no-progress --browsers ChromeHeadlessCI --include='**/va-*.spec.ts'",
-    "test:tz-east": "TZ=Pacific/Auckland npx ng test --no-watch --no-progress --browsers ChromeHeadlessCI --include='**/va-*.spec.ts'"
+    "viyabuild": "cd build; ./viyabuild.sh",
+    "watch": "ng test watch=true"
   },
   "private": true,
   "dependencies": {
@@ -58,7 +59,7 @@
     "@clr/ui": "file:libraries/clr-ui-17.9.0.tgz",
     "@handsontable/angular-wrapper": "18.1.2",
     "@sasjs/adapter": "5.1.2",
-    "@sasjs/utils": "4.0.0",
+    "@sasjs/utils": "4.3.1",
     "@sheet/crypto": "file:libraries/sheet-crypto.tgz",
     "@types/d3-graphviz": "^2.6.7",
     "@types/text-encoding": "0.0.35",
@@ -118,6 +119,7 @@
     "@types/randombytes": "^2.0.3",
     "@typescript-eslint/eslint-plugin": "8.65.0",
     "@typescript-eslint/parser": "8.65.0",
+    "archiver": "8.0.0",
     "core-js": "^2.5.4",
     "cypress": "^15.19.0",
     "cypress-file-upload": "^5.0.8",
@@ -151,11 +153,12 @@
     "lighthouse": "13.4.0",
     "fast-uri": "4.2.3",
     "readdir-glob": {
-      "brace-expansion": "^5.0.9"
+      "brace-expansion": "5.0.12",
+      "minimatch": "11.0.3"
     },
     "exceljs": {
-      "archiver": "^8.0.0",
-      "unzipper": "^0.12.5"
+      "archiver": "8.0.0",
+      "unzipper": "0.12.5"
     }
   }
 }

client/scripts/check-archiver.mjs

@@ -0,0 +1,74 @@
+/* Guards the readdir-glob -> minimatch override in client/package.json.
+ *
+ * The override forces minimatch 11 where readdir-glob asks for ^10, and
+ * readdir-glob reaches minimatch on two paths: the directory walk, and the
+ * pattern/ignore matching behind archiver's glob(). Both are exercised here,
+ * because a matcher that breaks shows up as files silently skipped rather than
+ * as an error.
+ *
+ * Entries are read from archiver's own 'entry' events, so nothing has to be
+ * unzipped and the check needs no reader dependency.
+ */
+import fs from 'node:fs'
+import os from 'node:os'
+import path from 'node:path'
+import { ZipArchive } from 'archiver'
+
+const root = fs.mkdtempSync(path.join(os.tmpdir(), 'dc-arch-'))
+const source = path.join(root, 'source')
+
+const cleanup = () => fs.rmSync(root, { recursive: true, force: true })
+const fail = (message) => {
+  console.error(message)
+  cleanup()
+  process.exit(1)
+}
+
+fs.mkdirSync(source)
+fs.writeFileSync(path.join(source, 'a.txt'), 'hello')
+fs.mkdirSync(path.join(source, 'sub'))
+fs.writeFileSync(path.join(source, 'sub', 'b.txt'), 'world')
+fs.writeFileSync(path.join(source, 'sub', 'c.md'), 'ignored by pattern')
+
+let runs = 0
+
+const archiveWith = (configure) =>
+  new Promise((resolve, reject) => {
+    const out = path.join(root, `test-${++runs}.zip`)
+    const sink = fs.createWriteStream(out)
+    const archive = new ZipArchive({ zlib: { level: 9 } })
+    const names = []
+    archive.on('entry', (entry) => names.push(entry.name))
+    archive.on('error', reject)
+    sink.on('error', reject)
+    sink.on('close', () => resolve(names))
+    archive.pipe(sink)
+    configure(archive)
+    archive.finalize()
+  })
+
+try {
+  /* the directory walk: every file, nested directories included */
+  const walked = await archiveWith((archive) => archive.directory(source, false))
+  const missing = ['a.txt', 'sub/b.txt'].filter((n) => !walked.includes(n))
+  if (missing.length) {
+    fail('directory walk missed: ' + missing.join(', ') + ' (got: ' + walked.join(', ') + ')')
+  }
+
+  /* pattern and ignore matching, which is minimatch used directly */
+  const globbed = await archiveWith((archive) =>
+    archive.glob('**/*.txt', { cwd: source, ignore: ['sub/**'] })
+  )
+  if (!globbed.includes('a.txt')) {
+    fail('glob did not match a.txt (got: ' + globbed.join(', ') + ')')
+  }
+  if (globbed.some((n) => n.startsWith('sub/'))) {
+    fail('glob ignore did not exclude sub/ (got: ' + globbed.join(', ') + ')')
+  }
+
+  console.log('archiver walk ok: ' + walked.join(', '))
+  console.log('archiver glob ok: ' + globbed.join(', '))
+  cleanup()
+} catch (err) {
+  fail('archiver error: ' + err.message)
+}

client/src/app/app.component.html

@@ -235,7 +235,7 @@
   @if (!errTop) {
     <app-alerts></app-alerts>
   }
-  <app-requests-modal [(opened)]="requestsModal"></app-requests-modal>
+  <app-sas-logs-modal [(opened)]="sasLogsModal"></app-sas-logs-modal>
   <app-excel-password-modal></app-excel-password-modal>
 
   <!-- <app-terms *ngIf="showRegistration"></app-terms> -->

client/src/app/app.component.ts

@@ -68,7 +68,7 @@ export class AppComponent {
   public licenceProblem = this.licenceService.licenceProblem
   public appOverCapacity: boolean = false
   public appActive: boolean | null = null
-  public requestsModal: boolean = false
+  public sasLogsModal: boolean = false
   public showRegistration: boolean = true
   public startupDataLoaded: boolean = false
   public startupSteps: StartupStep[] = []
@@ -181,7 +181,7 @@ export class AppComponent {
     }
 
     this.subscribeToShowAbortModal()
-    this.subscribeToRequestsModal()
+    this.subscribeToSasLogsModal()
     this.subscribeToStartupData()
     this.subscribeToAppActive()
     this.subscribeToDemoLimitModal()
@@ -353,9 +353,9 @@ export class AppComponent {
   /**
    * Opens requests modal when requested from event service
    */
-  public subscribeToRequestsModal() {
-    this.eventService.onRequestsModalOpen.subscribe((value: boolean) => {
-      this.requestsModal = true
+  public subscribeToSasLogsModal() {
+    this.eventService.onSasLogsModalOpen.subscribe((value: boolean) => {
+      this.sasLogsModal = true
     })
   }
 

client/src/app/app.module.ts

@@ -19,7 +19,7 @@ import { LicensingGuard } from './routes/licensing.guard'
 import { UsernavRouteComponent } from './routes/usernav-route/usernav-route.component'
 import { AppService } from './services/app.service'
 import { InfoModalComponent } from './shared/abort-modal/info-modal.component'
-import { RequestsModalComponent } from './shared/requests-modal/requests-modal.component'
+import { SasLogsModalComponent } from './shared/sas-logs-modal/sas-logs-modal.component'
 import { DirectivesModule } from './directives/directives.module'
 import { ViyaApiExplorerComponent } from './viya-api-explorer/viya-api-explorer.component'
 import { NgxJsonViewerModule } from 'ngx-json-viewer'
@@ -32,7 +32,7 @@ import { AppSettingsService } from './services/app-settings.service'
     ReviewRouteComponent,
     ReviewRouteComponent,
     UsernavRouteComponent,
-    RequestsModalComponent,
+    SasLogsModalComponent,
     InfoModalComponent,
     ViyaApiExplorerComponent
   ],

client/src/app/deploy/sections/automatic/automatic.component.html

@@ -8,20 +8,6 @@
       @if (autodeployDone) {
         <p class="m-0 align-self-start">Done</p>
         <hr class="w-100" />
-        @if (autoDeployStatus.deployServicePack !== null) {
-          <div class="deploy-status-row">
-            @if (autoDeployStatus.deployServicePack === true) {
-              <clr-icon
-                class="deploy-success"
-                shape="success-standard"
-              ></clr-icon>
-            }
-            @if (!autoDeployStatus.deployServicePack === false) {
-              <clr-icon class="deploy-error" shape="times-circle"></clr-icon>
-            }
-            <p>Deploy SAS Jobs</p>
-          </div>
-        }
         @if (autoDeployStatus.runMakeData !== null) {
           <div class="deploy-status-row">
             @if (autoDeployStatus.runMakeData) {
@@ -33,16 +19,13 @@
             @if (autoDeployStatus.runMakeData === false) {
               <clr-icon class="deploy-error" shape="times-circle"></clr-icon>
             }
-            <p>Create database</p>
+            <p>Create control library</p>
           </div>
         }
         <hr class="w-100" />
         <div class="buttons">
           <button (click)="navigateToHome()" class="btn btn-primary mt-15 mr-0">
-            @if (
-              autoDeployStatus.deployServicePack === false ||
-              autoDeployStatus.runMakeData === false
-            ) {
+            @if (autoDeployStatus.runMakeData === false) {
               <clr-icon class="deploy-error" shape="times-circle"></clr-icon>
             }
             LAUNCH
@@ -65,10 +48,10 @@
             Close
           </button>
           <button
-            (click)="openSasRequestsModal()"
+            (click)="openSasLogsModal()"
             class="btn btn-primary-outline mt-15 mr-0 align-self-end"
           >
-            SAS Requests
+            SAS Logs
           </button>
         </div>
       }
@@ -78,9 +61,9 @@
 
 <h4 class="text-center my-15">Viya Deploy</h4>
 <p class="deploy-intro">
-  Configure and deploy Data Controller to your Viya environment. Deployment
-  creates the backend SAS services and the control library used by the
-  application.
+  Configure Data Controller for your Viya environment. The frontend and services
+  are already in place; this runs the one-off job that creates the control
+  library and its configuration data, then removes itself.
 </p>
 <hr />
 
@@ -248,19 +231,8 @@
   </div>
 }
 
-<!-- Keeping this for a reference in case future VIYA changes and starts allowing separate backend and frontend) -->
-
-<!-- <clr-checkbox-wrapper>
-  <input
-    clrCheckbox
-    [(ngModel)]="recreateDatabase"
-    (click)="recreateDatabaseClicked($event)"
-    type="checkbox"
-    checked
-    />
-  <label>Recreate database</label>
-</clr-checkbox-wrapper> -->
-
+<!-- Runs the one-off makedata job. This is the screen's only action: the
+     frontend and services are already deployed. -->
 <hr />
 
 <button
@@ -269,53 +241,3 @@
 >
   Deploy
 </button>
-
-<!-- Keeping this for a reference in case future VIYA changes and starts allowing separate backend and frontend) -->
-
-<!-- <button
-(click)="executeJson()"
-class="btn-autodeploy btn btn-primary d-inline-block mr-10"
-[disabled]="!jsonFile"
->
-Deploy {{ !jsonFile ? '(json file is not available)' : '' }}
-</button> -->
-
-<!-- <button
-(click)="uploadJsonAuto.click()"
-class="btn-autodeploy btn btn-primary d-inline-block mr-10"
->
-Upload different file to deploy
-</button>
-<input
-  #uploadJsonAuto
-  type="file"
-  hidden
-  (click)="clearUploadInput($event)"
-  (change)="onJsonFileChange($event)"
-  /> -->
-
-<clr-modal [(clrModalOpen)]="recreateDatabaseModal" [clrModalClosable]="false">
-  <h3 class="modal-title">Warning</h3>
-  <div class="modal-body">
-    This action will recreate the database (if it exists). For an initial
-    deployment, this is expected. If this is a re-deployment, you will lose any
-    existing tables in <strong>{{ dcPath }}</strong
-    >.
-  </div>
-  <div class="modal-footer">
-    <button
-      type="button"
-      class="btn btn-sm btn-primary"
-      (click)="recreateDatabaseModal = false; recreateDatabase = false"
-    >
-      Cancel
-    </button>
-    <button
-      type="button"
-      class="btn btn-sm btn-primary"
-      (click)="recreateDatabaseModal = false; recreateDatabase = true"
-    >
-      Ok
-    </button>
-  </div>
-</clr-modal>

client/src/app/deploy/sections/automatic/automatic.component.spec.ts

@@ -0,0 +1,153 @@
+import { SASjsConfig } from '@sasjs/adapter'
+import { AutomaticComponent } from './automatic.component'
+
+/**
+ * The Viya setup screen has one job: run `makedata` with the selections the
+ * form collected - the admin group, the DC path, and the compute context the
+ * job runs in. These tests pin that payload and the new-window URL, because a
+ * screen whose only action silently stopped calling the service would still
+ * compile, still render, and still look like it worked.
+ */
+describe('AutomaticComponent (Viya setup)', () => {
+  const build = () => {
+    const requests: any[][] = []
+
+    const sasJs: any = {
+      request: (...args: any[]) => {
+        requests.push(args)
+        return Promise.resolve({ result: [{}] })
+      }
+    }
+
+    const sasService: any = {
+      getSasjsConfig: () => new SASjsConfig(),
+      getExecutionPath: () => '/SASJobExecution/',
+      shouldLogin: { next: () => undefined },
+      viyaMakedataSuccessfull: () => Promise.resolve(true),
+      updateFileContent: () => Promise.resolve()
+    }
+
+    const component = new AutomaticComponent(
+      {
+        showAbortModal: () => undefined,
+        showInfoModal: () => undefined
+      } as any,
+      { downloadFile: () => undefined } as any,
+      sasService,
+      {} as any,
+      { error: () => undefined } as any,
+      {} as any
+    )
+
+    component.sasJs = sasJs
+    const config = {
+      contextName: 'Compute Reusable',
+      serverUrl: 'https://viya.example.com',
+      appLoc: '/Users/demo/dc'
+    } as any
+    component.sasJsConfig = config
+    // The new-window URL is built from the adapter config the component holds,
+    // while the request path reads the @Input one.
+    component.sasjsConfig = config
+    component.appLoc = '/Users/demo/dc'
+    component.dcPath = '/export/dc'
+    component.selectedAdminGroup = 'viyagroup18'
+
+    return { component, requests, sasService }
+  }
+
+  it('posts makedata with the admin group and the DC path', async () => {
+    const { component, requests } = build()
+    component.deployInNewWindow = false
+
+    await component.createDatabase()
+
+    expect(requests[0][0]).toBe('services/admin/makedata')
+    expect(requests[0][1]).toEqual({
+      fromjs: [{ ADMIN: 'viyagroup18', DCPATH: '/export/dc' }]
+    })
+  })
+
+  it('runs the job in the configured context, on the web method', async () => {
+    const { component, requests } = build()
+    component.deployInNewWindow = false
+
+    await component.createDatabase()
+
+    expect(requests[0][2]).toEqual({
+      useComputeApi: null,
+      contextName: 'Compute Reusable',
+      debug: true
+    })
+  })
+
+  it('prefers the context chosen in the form over the configured one', async () => {
+    const { component, requests } = build()
+    component.deployInNewWindow = false
+    component.computeContexts = [
+      { id: 'ctx-1', name: 'Selected Context' } as any
+    ]
+    component.selectedComputeContext = 'ctx-1'
+
+    await component.createDatabase()
+
+    expect(requests[0][2].contextName).toBe('Selected Context')
+  })
+
+  it('opens makedata in a new window with the selections as URL parameters', () => {
+    const { component } = build()
+    const opened: string[] = []
+    spyOn(window, 'open').and.callFake((url?: any) => {
+      opened.push(String(url))
+      return null
+    })
+    spyOn(component as any, 'pollMakedataCompletion')
+    component.deployInNewWindow = true
+
+    component.createDatabase()
+
+    expect(opened.length).toBe(1)
+    expect(opened[0]).toContain('/Users/demo/dc/services/admin/makedata')
+    expect(opened[0]).toContain('_contextname=Compute%20Reusable')
+    expect(opened[0]).toContain('admin=viyagroup18')
+    expect(opened[0]).toContain('dcpath=%2Fexport%2Fdc')
+    expect(opened[0]).toContain('_debug=131')
+  })
+
+  it('percent-encodes the values that need it in the new-window URL', () => {
+    // A context name or admin group can hold characters that would otherwise
+    // split the query string - an ampersand would start a new parameter and a
+    // space would arrive truncated.
+    const { component } = build()
+    const opened: string[] = []
+    spyOn(window, 'open').and.callFake((url?: any) => {
+      opened.push(String(url))
+      return null
+    })
+
+    component.runMakedataInNewWindow({
+      contextName: 'SAS Job Execution & co',
+      admin: 'group 1+2',
+      dcPath: '/export/dc data'
+    })
+
+    expect(opened[0]).toContain('_contextname=SAS%20Job%20Execution%20%26%20co')
+    expect(opened[0]).toContain('admin=group%201%2B2')
+    expect(opened[0]).toContain('dcpath=%2Fexport%2Fdc%20data')
+    // the parameter names themselves are not encoded
+    expect(opened[0]).toContain('&admin=group%201%2B2')
+  })
+
+  it('starts the job when deploy is pressed', () => {
+    // The screen's only button calls this; a version where it stopped doing so
+    // would leave the setup page unable to do anything at all.
+    const { component } = build()
+    const spy = spyOn(component, 'createDatabase').and.returnValue(
+      Promise.resolve() as any
+    )
+
+    component.runAutoDeploy()
+
+    expect(spy).toHaveBeenCalled()
+  })
+})

client/src/app/deploy/sections/automatic/automatic.component.ts

@@ -51,17 +51,8 @@ export class AutomaticComponent implements OnInit {
 
   public selectedComputeContext: string = ''
   public makeDataResponse: string = ''
-  public jsonFile: any = null
   public autodeploying: boolean = false
   public autodeployDone: boolean = false
-  public recreateDatabaseModal: boolean = false
-  public isSubmittingJson: boolean = false
-  public isJsonSubmitted: boolean = false
-  /**
-   * Default was `false` when deploy was done with frontend and backend separately.
-   * Now we are using only streaming app, so we always want to recreate database (makedata)
-   */
-  public recreateDatabase: boolean = true
   public createDatabaseLoading: boolean = false
   public adminGroupsLoading: boolean = false
   public currentUserInfoLoading: boolean = false
@@ -86,18 +77,14 @@ export class AutomaticComponent implements OnInit {
   public currentUserInfo: ViyaApiCurrentUser | null = null
   public computeContexts: ComputeContextItem[] = []
 
-  /** autoDeployStatus
-   * This object presents the status for two steps that we have for deploy.
-   * `deployServicePack` - Creating services based on `viya.json`
-   * `runMakeData` - Running `makedata` service
-   * If any of them is `null` or `false` that means step failed
-   * and will be shown to user on deploy done modal.
+  /**
+   * Status of the one step this screen runs: `makedata`, which creates the
+   * control library and its configuration data and then deletes itself.
+   * `null` means it has not run yet, `false` that it failed.
    */
   public autoDeployStatus: {
-    deployServicePack: any
     runMakeData: any
   } = {
-    deployServicePack: null,
     runMakeData: null
   }
 
@@ -420,61 +407,15 @@ export class AutomaticComponent implements OnInit {
   }
 
   /**
-   * Executes sas.json file to deploy the backend
-   * Method will first try to run the `auto deploy`
-   * If that fails the rest of the code is ignored.
-   * If request is successfull, method will continue to try
-   * to create database if checkbox is toggled on
+   * Runs the deployment. The control library and its configuration data are
+   * created by `makedata`, which self-deletes once it has run; the frontend and
+   * services arrived with the sasjs deploy, so there is nothing else to deploy
+   * from here.
    */
-  public async executeJson() {
-    this.isSubmittingJson = true
-
-    try {
-      let uploadJsonFile = await this.sasJs.deployServicePack(
-        this.jsonFile,
-        this.dcAdapterSettings?.appLoc,
-        undefined,
-        undefined,
-        true
-      )
-
-      this.autoDeployStatus.deployServicePack = true
-      this.isJsonSubmitted = true
-    } catch (ex: any) {
-      let textEx = ''
-
-      if (typeof ex.message !== 'string') {
-        textEx = JSON.stringify(ex).replace(/\\/gm, '')
-      } else {
-        textEx = ex.message
-      }
-
-      this.autoDeployStatus.deployServicePack = false
-      this.eventService.showInfoModal(
-        'Deploy error',
-        `Exception: \n ${textEx !== '' ? textEx : ex}`
-      )
-      this.autodeploying = false
-      this.autodeployDone = false
-
-      return
-    }
-
-    this.isSubmittingJson = false
-  }
-
-  public async runAutoDeploy(executeJson: boolean = false) {
+  public runAutoDeploy() {
     if (!this.deployInNewWindow) this.autodeploying = true
 
-    if (executeJson) {
-      this.executeJson()
-    }
-
-    if (this.recreateDatabase) {
-      this.createDatabase()
-    } else {
-      if (!this.deployInNewWindow) this.autodeployDone = true
-    }
+    this.createDatabase()
   }
 
   /**
@@ -578,9 +519,9 @@ export class AutomaticComponent implements OnInit {
     let serverUrl = this.sasjsConfig.serverUrl
     let appLoc = this.sasjsConfig.appLoc
     const execPath = this.sasService.getExecutionPath()
-    let contextname = `&_contextname=${params.contextName}`
-    let admin = `&admin=${params.admin}`
-    let dcPath = `&dcpath=${params.dcPath}`
+    let contextname = `&_contextname=${encodeURIComponent(params.contextName)}`
+    let admin = `&admin=${encodeURIComponent(params.admin)}`
+    let dcPath = `&dcpath=${encodeURIComponent(params.dcPath)}`
     // Debug is ALWAYS enabled for makedata — it runs only once during
     // deployment and the full log is needed to diagnose any issues.
     let debug = '&_debug=131'
@@ -708,24 +649,8 @@ export class AutomaticComponent implements OnInit {
     this.deployService.downloadFile(content, filename, extension)
   }
 
-  public async onJsonFileChange(event: any) {
-    let file = event.target.files[0]
-
-    this.jsonFile = await this.deployService.readFile(file)
-  }
-
-  public recreateDatabaseClicked(event: Event) {
-    ;(<HTMLInputElement>event.target).checked === true
-      ? (this.recreateDatabaseModal = true)
-      : ''
-  }
-
-  public clearUploadInput(event: Event) {
-    this.deployService.clearUploadInput(event)
-  }
-
-  public openSasRequestsModal() {
-    this.eventService.openRequestsModal()
+  public openSasLogsModal() {
+    this.eventService.openSasLogsModal()
   }
 
   public navigateToHome() {

client/src/app/deploy/sections/sasjs-configurator/sasjs-configurator.component.spec.ts

@@ -0,0 +1,88 @@
+import { ServerType } from '@sasjs/utils/types/serverType'
+import { SasjsConfiguratorComponent } from './sasjs-configurator.component'
+
+/**
+ * The SAS9 / SASjs Server setup screen cannot deploy anything: it builds the
+ * one-off `makedata` URL and opens it, with the admin group and the directory
+ * the control library will live in. These tests pin the URL shape, because a
+ * wrong parameter name or a lost encode still produces a page that opens and
+ * merely fails inside SAS, where nobody is looking.
+ */
+describe('SasjsConfiguratorComponent (SAS9 / SASjs setup)', () => {
+  const build = (serverType: ServerType) => {
+    const sasService: any = {
+      getExecutionPath: () => '/SASStoredProcess/do',
+      sasjsMakedataChecking: () => Promise.resolve(false),
+      logout: () => undefined
+    }
+
+    const component = new SasjsConfiguratorComponent(
+      sasService,
+      { getServerInfo: () => ({ subscribe: () => undefined }) } as any,
+      { replaceState: () => undefined } as any
+    )
+
+    component.sasJsConfig = {
+      serverType,
+      serverUrl: 'https://sas9.example.com',
+      appLoc: '/User Folders/dc'
+    } as any
+    component.dcDirectory = '/tmp/DataController'
+    component.dcAdminGroup = 'DCADMINS'
+
+    const opened: string[] = []
+    spyOn(window, 'open').and.callFake((url?: any) => {
+      opened.push(String(url))
+      return null
+    })
+
+    return { component, opened, sasService }
+  }
+
+  it('opens makedata on SAS9 with the admin group and the directory', () => {
+    const { component, opened } = build(ServerType.Sas9)
+
+    component.makeData()
+
+    expect(opened.length).toBe(1)
+    expect(opened[0]).toContain(
+      '_program=/User Folders/dc/services/admin/makedata'
+    )
+    expect(opened[0]).toContain('admin=DCADMINS')
+    expect(opened[0]).toContain('path=%2Ftmp%2FDataController')
+    expect(opened[0]).toContain('_debug=131')
+  })
+
+  it('stays on the page origin for SAS9 rather than the configured server', () => {
+    const { component, opened } = build(ServerType.Sas9)
+
+    component.makeData()
+
+    // A SAS9 deployment is reached through the URL the user is already on; the
+    // configured serverUrl would point the browser at a host it may not reach.
+    expect(opened[0].startsWith(window.location.origin)).toBe(true)
+    expect(opened[0]).not.toContain('sas9.example.com')
+  })
+
+  it('keeps the directory readable for SASjs Server', () => {
+    const { component, opened } = build(ServerType.Sasjs)
+
+    component.makeData()
+
+    // The SASjs branch does not encode the path, so a plain directory stays as
+    // written; encoding it there would make makedata look for a folder with %2F
+    // in its name.
+    expect(opened[0]).toContain('path=/tmp/DataController')
+  })
+
+  it('checks the SASjs Server deployment before reloading the app', () => {
+    const { component, sasService } = build(ServerType.Sasjs)
+    const spy = spyOn(sasService, 'sasjsMakedataChecking').and.returnValue(
+      Promise.resolve(false)
+    )
+
+    component.makeData()
+
+    expect(spy).toHaveBeenCalled()
+  })
+})

client/src/app/editor/editor.component.ts

@@ -15,6 +15,7 @@ import { ActivatedRoute, Router } from '@angular/router'
 import Handsontable, { CellRange } from 'handsontable'
 import { Subject, Subscription } from 'rxjs'
 import { sanitiseForSas } from '../shared/utils/sanitise'
+import { fixGridAria } from '../shared/utils/grid-aria'
 import { SasStoreService } from '../services/sas-store.service'
 
 type AOA = any[][]
@@ -714,8 +715,6 @@ export class EditorComponent
 
   public licenceState = this.licenceService.licenceState
 
-  private ariaObserver: MutationObserver | undefined
-  private ariaCheckInterval: any | undefined
   private gridResizeObserver: ResizeObserver | undefined
   private pasteListener: ((event: ClipboardEvent) => void) | undefined
 
@@ -3647,7 +3646,16 @@ export class EditorComponent
     if (clauses.length === 0) return false
 
     try {
-      const res: any = await this.sasStoreService.saveQuery(libds, clauses)
+      // A `va_parmap` entry is a column the report author typed, so this
+      // request can legitimately fail: the table may not have that column, and
+      // validatefilter aborts on an unknown variable (dc/dc#404). Attempting it
+      // is still worth it - when the mapping is right the very first getdata is
+      // already filtered - but the failure must not stop the page. An abort
+      // modal here covers an editor that is about to load fine; the mapping
+      // banner names the bad entry once the real metadata arrives.
+      const res: any = await this.sasStoreService.saveQuery(libds, clauses, {
+        suppressErrorAbortModal: true
+      })
       const id = res?.result?.[0]?.FILTER_RK
       const table = res?.result?.[0]?.FILTER_TABLE
       if (id === undefined || table === undefined) return false
@@ -3750,18 +3758,6 @@ export class EditorComponent
   ngOnDestroy() {
     this._queryParams?.unsubscribe()
 
-    // Clean up the MutationObserver
-    if (this.ariaObserver) {
-      this.ariaObserver.disconnect()
-      this.ariaObserver = undefined
-    }
-
-    // Clean up the interval
-    if (this.ariaCheckInterval) {
-      clearInterval(this.ariaCheckInterval)
-      this.ariaCheckInterval = undefined
-    }
-
     // Stop observing the grid container for resizes
     if (this.gridResizeObserver) {
       this.gridResizeObserver.disconnect()
@@ -3811,150 +3807,15 @@ export class EditorComponent
   }
 
   /**
-   * Fixes ARIA accessibility issues in the Handsontable component
-   * This addresses the accessibility report issues with treegrid and presentation roles
+   * Corrects the ARIA markup Handsontable puts on its own grid wrapper - see
+   * shared/utils/grid-aria.ts for what is corrected and why. Scoped to this
+   * grid so it does not touch any other grid or component on the page.
    */
-  private fixAriaAccessibility() {
-    // Use a more aggressive approach to find and fix all ARIA issues
-    const fixAriaIssues = () => {
-      // Specifically target Handsontable wrapper elements that are causing issues
-      const hotWrappers = document.querySelectorAll(
-        '.ht-wrapper, .wtHolder, [id^="ht_"]'
-      )
-
-      hotWrappers.forEach((wrapper) => {
-        // Remove problematic ARIA attributes from Handsontable wrappers
-        wrapper.removeAttribute('role')
-        wrapper.removeAttribute('aria-rowcount')
-        wrapper.removeAttribute('aria-colcount')
-        wrapper.removeAttribute('aria-multiselectable')
-      })
-
-      // Find all elements with problematic ARIA roles in the entire document
-      const allTreegridElements = document.querySelectorAll('[role="treegrid"]')
-      const allPresentationElements = document.querySelectorAll(
-        '[role="presentation"]'
-      )
-
-      // Fix treegrid role issues - remove them completely as they're causing problems
-      allTreegridElements.forEach((element) => {
-        element.removeAttribute('role')
-        element.removeAttribute('aria-rowcount')
-        element.removeAttribute('aria-colcount')
-        element.removeAttribute('aria-multiselectable')
-      })
-
-      // Fix presentation role issues - remove them if they contain interactive elements
-      allPresentationElements.forEach((element) => {
-        const hasInteractiveChildren =
-          element.querySelectorAll(
-            'button, input, select, textarea, [tabindex], [onclick], [contenteditable]'
-          ).length > 0
-        if (hasInteractiveChildren) {
-          element.removeAttribute('role')
-        }
-      })
-
-      // Also fix any elements with aria-rowcount="-1" which is problematic
-      const negativeRowCountElements = document.querySelectorAll(
-        '[aria-rowcount="-1"]'
-      )
-      negativeRowCountElements.forEach((element) => {
-        element.removeAttribute('aria-rowcount')
-      })
-
-      // Ensure proper table structure
-      const tableElements = document.querySelectorAll('table')
-      tableElements.forEach((table) => {
-        if (!table.getAttribute('role')) {
-          table.setAttribute('role', 'table')
-        }
-
-        // Ensure table headers have proper scope
-        const headerCells = table.querySelectorAll('th')
-        headerCells.forEach((th) => {
-          if (!th.getAttribute('scope')) {
-            th.setAttribute('scope', 'col')
-          }
-        })
-      })
-
-      // Add proper ARIA labels to interactive elements
-      const interactiveElements = document.querySelectorAll(
-        'button, input, select, textarea, [contenteditable]'
-      )
-      interactiveElements.forEach((element) => {
-        if (
-          !element.getAttribute('aria-label') &&
-          !element.getAttribute('aria-labelledby')
-        ) {
-          const textContent = element.textContent?.trim()
-          if (textContent) {
-            element.setAttribute('aria-label', textContent)
-          }
-        }
-      })
-    }
-
-    // Run the fix immediately
-    fixAriaIssues()
-
-    // Run it again after a short delay to catch any dynamically created elements
-    setTimeout(fixAriaIssues, 100)
-    setTimeout(fixAriaIssues, 500)
-    setTimeout(fixAriaIssues, 1000)
-    setTimeout(fixAriaIssues, 2000)
-
-    // Set up a periodic check to ensure accessibility fixes are maintained
-    if (!this.ariaCheckInterval) {
-      this.ariaCheckInterval = setInterval(fixAriaIssues, 3000)
-    }
-
-    // Set up a MutationObserver to continuously monitor for new problematic elements
-    if (!this.ariaObserver) {
-      this.ariaObserver = new MutationObserver((mutations) => {
-        let shouldFix = false
-        mutations.forEach((mutation) => {
-          if (
-            mutation.type === 'attributes' &&
-            (mutation.attributeName === 'role' ||
-              mutation.attributeName === 'aria-rowcount')
-          ) {
-            shouldFix = true
-          }
-          if (mutation.type === 'childList') {
-            mutation.addedNodes.forEach((node) => {
-              if (node.nodeType === Node.ELEMENT_NODE) {
-                const element = node as Element
-                if (
-                  element.hasAttribute('role') ||
-                  element.hasAttribute('aria-rowcount')
-                ) {
-                  shouldFix = true
-                }
-              }
-            })
-          }
-        })
-
-        if (shouldFix) {
-          setTimeout(fixAriaIssues, 50)
-        }
-      })
-
-      // Start observing the entire document for changes
-      this.ariaObserver.observe(document.body, {
-        childList: true,
-        subtree: true,
-        attributes: true,
-        attributeFilter: [
-          'role',
-          'aria-rowcount',
-          'aria-colcount',
-          'aria-multiselectable'
-        ]
-      })
-    }
+  private fixAriaAccessibility(): void {
+    fixGridAria(
+      this.hotInstance?.rootElement ??
+        this.hotTableComponent?.hotInstance?.rootElement
+    )
   }
 
   initSetup(response: EditorsGetDataServiceResponse, attempt = 0) {

client/src/app/multi-dataset/multi-dataset.component.html

@@ -157,6 +157,16 @@
             <p cds-text="secondary regular" class="mb-20">
               File size: <strong>{{ selectedFile.sizeMB }} MB</strong>
             </p>
+            <p
+              cds-text="secondary regular"
+              class="mb-20 d-flex clr-align-items-center"
+              id="file-loading-stage"
+            >
+              Stage: <strong class="ml-5">{{ fileLoadingState }}</strong>
+              @if (isFileBusy) {
+                <clr-spinner class="ml-10" clrSmall></clr-spinner>
+              }
+            </p>
             <p cds-text="secondary regular" class="mb-15">
               Paste or type the list of datasets to upload:
             </p>
@@ -188,18 +198,26 @@
       @if (parsedDatasets.length && !submittedCsvDatasets.length) {
         @if (!activeParsedDataset) {
           <div class="no-table-selected pointer-events-none">
-            @if (fileLoadingState !== FileLoadingState.parsed) {
+            @if (isFileBusy) {
               <clr-icon
                 aria-hidden="true"
                 shape="process-on-vm"
                 size="40"
                 class="is-info icon-dc-fill"
               ></clr-icon>
-              <p class="text-center color-gray mt-10" cds-text="section">
+              <p
+                class="text-center color-gray mt-10"
+                cds-text="section"
+                id="file-loading-stage"
+              >
                 {{ fileLoadingState }}...
               </p>
+              <p class="text-center color-gray" cds-text="secondary regular">
+                {{ selectedFile?.name }} - {{ selectedFile?.sizeMB }} MB
+              </p>
+              <clr-spinner class="mt-10" clrSmall></clr-spinner>
             }
-            @if (fileLoadingState === FileLoadingState.parsed) {
+            @if (!isFileBusy) {
               <clr-icon
                 aria-hidden="true"
                 shape="warning-standard"

client/src/app/multi-dataset/multi-dataset.component.ts

@@ -40,10 +40,13 @@ import XLSX from 'xlsx'
 
 enum FileLoadingState {
   reading = 'Reading the file',
-  parsing = 'Searching for the data in the file',
-  parsed = 'Searching for the data finished',
+  scanning = 'Searching for the data in the file',
+  ready = 'File ready to load',
+  fetching = 'Fetching the dataset details',
+  extracting = 'Extracting the data',
+  extracted = 'Extracting the data finished',
   /**
-   * Defualt value
+   * Default value
    */
   notSelected = 'File not selected'
 }
@@ -65,7 +68,19 @@ export class MultiDatasetComponent implements OnInit, AfterViewInit {
   public workbookInterval: any
   public fileLoadingState: FileLoadingState = FileLoadingState.notSelected
 
-  public FileLoadingState = FileLoadingState
+  /**
+   * True while the file is being worked on. `ready` and `extracted` are idle
+   * states, so the spinner and the stage line are only shown for the phases
+   * that actually have work running.
+   */
+  public get isFileBusy(): boolean {
+    return (
+      this.fileLoadingState === FileLoadingState.reading ||
+      this.fileLoadingState === FileLoadingState.scanning ||
+      this.fileLoadingState === FileLoadingState.fetching ||
+      this.fileLoadingState === FileLoadingState.extracting
+    )
+  }
 
   public hotTableLicenseKey: string | undefined = undefined
   public hotTableMaxRows =
@@ -282,8 +297,13 @@ export class MultiDatasetComponent implements OnInit, AfterViewInit {
 
         // Read the excel file to be ready
         this.spreadsheetService.xlsxReadFile(this.selectedFile!).then((wb) => {
-          this.fileLoadingState = FileLoadingState.parsing
           this.selectedFile!.workbook = wb
+
+          // The sheet scan runs in parallel and can finish first, so only
+          // advance the stage if the read is still the phase in progress.
+          if (this.fileLoadingState === FileLoadingState.reading) {
+            this.fileLoadingState = FileLoadingState.scanning
+          }
         })
       }
 
@@ -340,10 +360,12 @@ export class MultiDatasetComponent implements OnInit, AfterViewInit {
   onDiscardFile() {
     this.selectedFile = null
     this.userInputDatasets = ''
+    this.fileLoadingState = FileLoadingState.notSelected
   }
 
   async onStartParsingFile() {
     this.uploadLoading = true
+    this.fileLoadingState = FileLoadingState.fetching
 
     const datasetFetchingPromises: Promise<
       EditorsGetDataServiceResponse | undefined
@@ -377,6 +399,7 @@ export class MultiDatasetComponent implements OnInit, AfterViewInit {
         datasetObjects = this.buildDatasetsObjects(datasets)
       } catch (err) {
         console.warn('Error building dataset objects', err)
+        this.fileLoadingState = FileLoadingState.ready
         this.eventService.showInfoModal('Upload error', (err as Error).message)
         return
       }
@@ -391,8 +414,12 @@ export class MultiDatasetComponent implements OnInit, AfterViewInit {
       })
 
       this.workbookLoaded().then((workbook) => {
+        this.fileLoadingState = FileLoadingState.extracting
+
+        const parsingPromises: Promise<void>[] = []
+
         for (let parsedDataset of this.parsedDatasets) {
-          this.spreadsheetService
+          const parsePromise = this.spreadsheetService
             .parseExcelFile({
               file: this.selectedFile!,
               workbook: workbook,
@@ -407,8 +434,6 @@ export class MultiDatasetComponent implements OnInit, AfterViewInit {
               xlRules: parsedDataset.datasetInfo.xlRules
             })
             .then((parseResult: ParseResult | undefined) => {
-              this.fileLoadingState = FileLoadingState.parsed
-
               if (parseResult && parseResult.data) {
                 let datasource: any[] = []
 
@@ -438,7 +463,15 @@ export class MultiDatasetComponent implements OnInit, AfterViewInit {
               parsedDataset.includeInSubmission = false
               parsedDataset.parsingTable = false
             })
+
+          parsingPromises.push(parsePromise)
         }
+
+        // The stage belongs to the whole file, not to the sheet that finished
+        // last, so it only moves on once every sheet has been extracted.
+        Promise.allSettled(parsingPromises).then(() => {
+          this.fileLoadingState = FileLoadingState.extracted
+        })
       })
     })
   }
@@ -460,6 +493,7 @@ export class MultiDatasetComponent implements OnInit, AfterViewInit {
     this.selectedFile = null
     this.userInputDatasets = ''
     this.submitReasonMessage = ''
+    this.fileLoadingState = FileLoadingState.notSelected
   }
 
   initHot() {
@@ -674,6 +708,7 @@ export class MultiDatasetComponent implements OnInit, AfterViewInit {
       }
       if (!this.hotInstanceUserDataset) {
         console.warn('hotInstanceUserDataset not ready after waiting')
+        this.fileLoadingState = FileLoadingState.ready
         return
       }
     }
@@ -746,6 +781,10 @@ export class MultiDatasetComponent implements OnInit, AfterViewInit {
       }, 100)
     }
     this.dynamicCellValidations()
+
+    // The sheet scan is done; the file waits for the user to confirm the
+    // dataset list, so no stage is running any more.
+    this.fileLoadingState = FileLoadingState.ready
   }
 
   onParsedDatasetClick(parsedDataset: ParsedDataset) {

client/src/app/services/deploy.service.ts

@@ -26,22 +26,4 @@ export class DeployService {
       document.body.removeChild(downloadLink)
     }
   }
-
-  public readFile(file: File): Promise<any> {
-    return new Promise((resolve, reject) => {
-      let JsonFileReader = new FileReader()
-
-      JsonFileReader.onload = () => {
-        if (JsonFileReader.result) {
-          resolve(JSON.parse(JsonFileReader.result.toString()))
-        }
-      }
-
-      JsonFileReader.readAsText(file)
-    })
-  }
-
-  public clearUploadInput(event: Event) {
-    ;(<HTMLInputElement>event.target).value = ''
-  }
 }

client/src/app/services/event.service.ts

@@ -13,7 +13,7 @@ export class EventService {
   @Output() onStartupDataLoaded: EventEmitter<any> = new EventEmitter()
   @Output()
   onShowAbortModal: EventEmitter<InfoModal> = new EventEmitter<InfoModal>()
-  @Output() onRequestsModalOpen: EventEmitter<boolean> = new EventEmitter()
+  @Output() onSasLogsModalOpen: EventEmitter<boolean> = new EventEmitter()
   @Output() onDemoLimitModalShow: EventEmitter<string> =
     new EventEmitter<string>()
 
@@ -78,8 +78,8 @@ export class EventService {
     this.onShowAbortModal.emit(infoModalObject)
   }
 
-  public openRequestsModal() {
-    this.onRequestsModalOpen.emit(true)
+  public openSasLogsModal() {
+    this.onSasLogsModalOpen.emit(true)
   }
 
   public catchResponseError(sasService: string | null, err: any) {

client/src/app/services/sas-store.service.spec.ts

@@ -116,6 +116,53 @@ describe('SasStoreService', () => {
       expect(seen.length).toBe(0)
     })
   })
+
+  describe('saveQuery', () => {
+    const capture = (adapterResponse: any = { result: [] }) => {
+      const calls: any[][] = []
+      const store = new SasStoreService(
+        {
+          request: (...args: any[]) => {
+            calls.push(args)
+            return Promise.resolve({ adapterResponse })
+          }
+        } as any,
+        {} as any,
+        {} as any
+      )
+      return { store, calls }
+    }
+
+    it('posts the filter to public/validatefilter', async () => {
+      const { store, calls } = capture()
+      await store.saveQuery('DC.MY_TABLE', [
+        { VARIABLE_NM: 'A', OPERATOR_NM: '=', RAW_VALUE: "'x'" } as any
+      ])
+
+      expect(calls[0][0]).toBe('public/validatefilter')
+      expect(calls[0][1].iwant[0].filter_table).toBe('DC.MY_TABLE')
+      expect(calls[0][1].filterquery.length).toBe(1)
+    })
+
+    it('passes wrapper options through, so a rejected filter can stay quiet', async () => {
+      // The VA deferred load attempts a filter built from a va_parmap entry the
+      // table may not have, and must not cover a page that loads fine with an
+      // abort modal (dc/dc#404).
+      const { store, calls } = capture()
+      await store.saveQuery('DC.MY_TABLE', [], {
+        suppressErrorAbortModal: true
+      })
+
+      expect(calls[0][3]).toEqual({ suppressErrorAbortModal: true })
+    })
+
+    it('sends no wrapper options when the caller gives none', async () => {
+      const { store, calls } = capture()
+      await store.saveQuery('DC.MY_TABLE', [])
+
+      expect(calls[0][3]).toBeUndefined()
+    })
+  })
 })
 
 /**

client/src/app/services/sas-store.service.ts

@@ -8,6 +8,7 @@ import { HelperService } from './helper.service'
 import { QueryClause } from '../models/TableData'
 import { globals } from '../_globals'
 import { FilterClause, FilterGroup, FilterQuery } from '../models/FilterQuery'
+import { RequestWrapperOptions } from '../models/request-wrapper/RequestWrapperOptions'
 import {
   $DataFormats,
   EditorsGetDataSASResponse,
@@ -395,13 +396,30 @@ export class SasStoreService {
     ).adapterResponse
   }
 
-  public async saveQuery(libds: string, filterQuery: QueryClause[]) {
+  /**
+   * Persists a filter and returns its FILTER_RK.
+   *
+   * `wrapperOptions` reaches the request wrapper, so a caller that can live with
+   * a rejected filter (the VA deferred load, whose `va_parmap` entry may name a
+   * column the table does not have) can suppress the abort modal that would
+   * otherwise cover a page which is about to load perfectly well.
+   */
+  public async saveQuery(
+    libds: string,
+    filterQuery: QueryClause[],
+    wrapperOptions?: RequestWrapperOptions
+  ) {
     let tables = {
       iwant: [{ filter_table: libds }],
       filterquery: filterQuery
     }
 
-    const res = await this.sasService.request('public/validatefilter', tables)
+    const res = await this.sasService.request(
+      'public/validatefilter',
+      tables,
+      undefined,
+      wrapperOptions
+    )
 
     this.filter.next(res)
 

client/src/app/services/va-filter.service.spec.ts

@@ -131,6 +131,41 @@ describe('VaFilterService', () => {
       expect(build([{ label: 'SOME_DATE', value: '2025/03/31' }])).toHaveSize(1)
     })
 
+    it('parses word dates, in either order and with an optional comma', () => {
+      const expected = "'01JAN2026'd"
+      // The reported shape: a date parameter displayed as WORDDATE.
+      expect(
+        build([{ label: 'SOME_DATE', value: 'January 1, 2026' }])[0].RAW_VALUE
+      ).toBe(expected)
+      expect(
+        build([{ label: 'SOME_DATE', value: 'Jan 1, 2026' }])[0].RAW_VALUE
+      ).toBe(expected)
+      expect(
+        build([{ label: 'SOME_DATE', value: 'Jan. 1, 2026' }])[0].RAW_VALUE
+      ).toBe(expected)
+      expect(
+        build([{ label: 'SOME_DATE', value: '1 January 2026' }])[0].RAW_VALUE
+      ).toBe(expected)
+      expect(
+        build([{ label: 'SOME_DATE', value: '1-Jan-2026' }])[0].RAW_VALUE
+      ).toBe(expected)
+    })
+
+    it('matches a spelled-out month case-insensitively, and keeps DATE9 working', () => {
+      expect(
+        build([{ label: 'SOME_DATE', value: 'DECEMBER 31, 2025' }])[0].RAW_VALUE
+      ).toBe("'31DEC2025'd")
+      expect(
+        build([{ label: 'SOME_DATE', value: '01AUG1962' }])[0].RAW_VALUE
+      ).toBe("'01AUG1962'd")
+    })
+
+    it('still drops a month-name value that has no day', () => {
+      // A month and year alone is not a date - accepted shapes always carry a
+      // day, so this must not silently become the first of the month.
+      expect(build([{ label: 'SOME_DATE', value: 'January 2026' }])).toEqual([])
+    })
+
     it('resolves ambiguous slash dates day-first (never US), disambiguating by value', () => {
       const iso = build([{ label: 'SOME_DATE', value: '2025-03-31' }])[0]
         .RAW_VALUE

client/src/app/services/va-filter.service.ts

@@ -707,14 +707,42 @@ export class VaFilterService {
     return this.parseMonthNameDate(value) ?? this.parseNumericDate(value, order)
   }
 
-  /** DATE9 "ddMMM[yy]yy" (e.g. 01AUG1962, 01AUG62) -> local Date, or null. */
+  /**
+   * Month-name date shapes -> local Date, or null. Three families:
+   *  - DATE9 "ddMMM[yy]yy" (e.g. 01AUG1962, 01AUG62);
+   *  - word date with the month first, comma optional: "January 1, 2026",
+   *    "Jan 1 2026", "Jan. 1, 2026";
+   *  - word date with the day first: "1 January 2026", "1-Jan-2026".
+   *
+   * A date parameter's display format decides which of these the report author
+   * sees, and VA sends the formatted text, so a report using WORDDATE (or a
+   * long month name in any order) has to parse here rather than being dropped.
+   * The month name may be abbreviated or spelled out; only its first three
+   * letters are matched.
+   */
   private parseMonthNameDate(value: string): Date | null {
-    const m = /^(\d{1,2})([A-Za-z]{3})(\d{2}|\d{4})$/.exec(value)
-    if (!m) return null
-    const [, day, monthAbbr, year] = m
-    const monthIndex = VaFilterService.SAS_MONTHS.indexOf(
-      monthAbbr.toUpperCase()
-    )
+    const monthByName = (name: string): number =>
+      VaFilterService.SAS_MONTHS.indexOf(
+        name.replace(/\./g, '').slice(0, 3).toUpperCase()
+      )
+
+    const date9 = /^(\d{1,2})([A-Za-z]{3})(\d{2}|\d{4})$/.exec(value)
+    if (date9) {
+      const [, day, monthAbbr, year] = date9
+      const monthIndex = monthByName(monthAbbr)
+      if (monthIndex < 0) return null
+      return this.buildDate(this.fullYear(year), monthIndex + 1, Number(day))
+    }
+
+    const monthFirst =
+      /^([A-Za-z]{3,9})\.?[\s-]+(\d{1,2}),?[\s-]+(\d{2}|\d{4})$/.exec(value)
+    const dayFirst =
+      /^(\d{1,2})[\s-]+([A-Za-z]{3,9})\.?,?[\s-]+(\d{2}|\d{4})$/.exec(value)
+    const name = monthFirst ? monthFirst[1] : dayFirst ? dayFirst[2] : null
+    if (!name) return null
+    const day = monthFirst ? monthFirst[2] : dayFirst![1]
+    const year = monthFirst ? monthFirst[3] : dayFirst![3]
+    const monthIndex = monthByName(name)
     if (monthIndex < 0) return null
     return this.buildDate(this.fullYear(year), monthIndex + 1, Number(day))
   }

client/src/app/shared/abort-modal/info-modal.component.html

@@ -52,9 +52,9 @@
       <button
         type="button"
         class="btn btn-primary"
-        (click)="openRequestsModal()"
+        (click)="openSasLogsModal()"
       >
-        Open requests modal
+        Open logs modal
       </button>
     }
 

client/src/app/shared/abort-modal/info-modal.component.ts

@@ -115,8 +115,8 @@ export class InfoModalComponent implements OnInit {
     )
   }
 
-  openRequestsModal() {
-    this.eventService.openRequestsModal()
+  openSasLogsModal() {
+    this.eventService.openSasLogsModal()
   }
 
   reload() {

client/src/app/shared/hot-export/hot-export.util.spec.ts

@@ -0,0 +1,212 @@
+import Handsontable from 'handsontable'
+import {
+  buildGridPdfInput,
+  collectGridTable,
+  exportGrid,
+  isRangeEmpty,
+  getExportRange,
+  buildExportMenuItem
+} from './hot-export.util'
+
+/**
+ * The grid-reading side of the PDF export, against a real Handsontable
+ * instance so `getColHeader`, `getSelectedRangeLast` and the hiddenColumns
+ * plugin behave as they do in the viewer and editor.
+ */
+describe('hot-export grid extraction', () => {
+  let container: HTMLDivElement
+  let hot: Handsontable
+
+  const settings = (overrides: Record<string, any> = {}) => ({
+    data: [
+      ['DELETE?', 'A1', '10.5'],
+      ['', 'A2', '20.25'],
+      ['', 'A3', null]
+    ],
+    colHeaders: ['Delete?', 'ACCOUNT', 'BALANCE'],
+    rowHeaders: false,
+    licenseKey: 'non-commercial-and-evaluation',
+    ...overrides
+  })
+
+  beforeEach(() => {
+    container = document.createElement('div')
+    document.body.appendChild(container)
+  })
+
+  afterEach(() => {
+    if (hot && !hot.isDestroyed) hot.destroy()
+    hot = undefined as any
+    container.remove()
+  })
+
+  it('reads headers and every row when nothing is selected', () => {
+    hot = new Handsontable(container, settings())
+    hot.render()
+
+    const table = collectGridTable(hot)
+
+    expect(table.headers).toEqual(['Delete?', 'ACCOUNT', 'BALANCE'])
+    expect(table.rows.length).toBe(3)
+    expect(table.rows[0]).toEqual(['DELETE?', 'A1', '10.5'])
+    expect(table.rows[2][2]).toBeNull()
+  })
+
+  it('drops the leading housekeeping column when asked', () => {
+    hot = new Handsontable(container, settings())
+    hot.render()
+
+    const table = collectGridTable(hot, 1)
+
+    expect(table.headers).toEqual(['ACCOUNT', 'BALANCE'])
+    expect(table.rows[0]).toEqual(['A1', '10.5'])
+  })
+
+  it('excludes columns hidden in the grid', () => {
+    hot = new Handsontable(
+      container,
+      settings({ hiddenColumns: { columns: [2], indicators: false } })
+    )
+    hot.render()
+
+    const table = collectGridTable(hot)
+
+    expect(table.headers).toEqual(['Delete?', 'ACCOUNT'])
+    expect(table.rows.map((row) => row.length)).toEqual([2, 2, 2])
+  })
+
+  it('survives a hiddenColumns plugin that cannot report (regression)', () => {
+    // getPlugin() hands back the instance even when the plugin is off, and its
+    // isHidden() then dereferences a map that was never created - seen in CI on
+    // grids with no hiddenColumns setting at all.
+    hot = new Handsontable(container, settings())
+    hot.render()
+    spyOn(hot, 'getPlugin').and.returnValue({
+      enabled: true,
+      isHidden: () => {
+        throw new TypeError(
+          "Cannot read properties of null (reading 'getValueAtIndex')"
+        )
+      }
+    } as any)
+
+    const table = collectGridTable(hot)
+
+    expect(table.headers).toEqual(['Delete?', 'ACCOUNT', 'BALANCE'])
+    expect(table.rows.length).toBe(3)
+  })
+
+  it('limits the export to a multi-cell selection', () => {
+    hot = new Handsontable(container, settings())
+    hot.render()
+    hot.selectCells([[1, 1, 2, 2]])
+
+    const range = getExportRange(hot)
+    const table = collectGridTable(hot)
+
+    expect(range).toEqual({ top: 1, left: 1, bottom: 2, right: 2 })
+    expect(table.headers).toEqual(['ACCOUNT', 'BALANCE'])
+    expect(table.rows).toEqual([
+      ['A2', '20.25'],
+      ['A3', null]
+    ])
+  })
+
+  it('exports nothing when the selection is inside the skipped columns', () => {
+    hot = new Handsontable(container, settings())
+    hot.render()
+    // Rows 0-2 of column 0 only: the Delete? column, which the caller skips.
+    hot.selectCells([[0, 0, 2, 0]])
+
+    const range = getExportRange(hot, 1)
+
+    expect(isRangeEmpty(range)).toBeTrue()
+    expect(collectGridTable(hot, 1).headers).toEqual([])
+    expect(buildGridPdfInput(hot, 'WORK.TEST', 1)).toBeNull()
+  })
+
+  it('treats a single-cell cursor as "export everything"', () => {
+    hot = new Handsontable(container, settings())
+    hot.render()
+    hot.selectCell(1, 1)
+
+    expect(collectGridTable(hot).rows.length).toBe(3)
+  })
+
+  it('never starts inside the skipped housekeeping columns', () => {
+    hot = new Handsontable(container, settings())
+    hot.render()
+    // A selection that begins in the Delete? column still exports from column 1.
+    hot.selectCells([[0, 0, 2, 2]])
+
+    const range = getExportRange(hot, 1)
+
+    expect(range.left).toBe(1)
+    expect(collectGridTable(hot, 1).headers).toEqual(['ACCOUNT', 'BALANCE'])
+  })
+
+  it('keeps the skipped column out of CSV/Excel exports (regression)', async () => {
+    // The exportFile plugin is not enabled in these grids, so stand in for it
+    // and read the options the export passes through.
+    hot = new Handsontable(container, settings())
+    hot.render()
+    const downloadFile = jasmine.createSpy('downloadFile')
+    const getPlugin = hot.getPlugin.bind(hot)
+    spyOn(hot, 'getPlugin').and.callFake((name: string) =>
+      name === 'exportFile' ? ({ downloadFile } as any) : getPlugin(name as any)
+    )
+
+    await exportGrid(hot, 'csv', 'WORK.TEST', 1)
+
+    expect(downloadFile).toHaveBeenCalledTimes(1)
+    const opts = downloadFile.calls.mostRecent().args[1]
+    // Column 0 is the Delete? column; the export must not start at it, and the
+    // plugin must not be left to export the whole grid.
+    expect(opts['range']).toEqual([0, 1, 2, 2])
+  })
+
+  it('stops reading rows at the cap instead of collecting them all', () => {
+    hot = new Handsontable(container, settings())
+    hot.render()
+    const read = spyOn(hot, 'getDataAtCell').and.callThrough()
+
+    const table = collectGridTable(hot, 0, 2)
+
+    expect(table.rows.length).toBe(2)
+    expect(read).toHaveBeenCalledTimes(2 * 3) // 2 rows x 3 columns
+  })
+
+  it('still produces a PDF for a grid with no rows', () => {
+    // The real grids declare their columns, so an empty grid still has a
+    // schema - which is the case this guards. Without `columns`, HOT reports
+    // zero columns for `data: []` and there would be no table to describe.
+    hot = new Handsontable(
+      container,
+      settings({
+        data: [],
+        columns: [{ data: 0 }, { data: 1 }, { data: 2 }]
+      })
+    )
+    hot.render()
+
+    const input = buildGridPdfInput(hot, 'WORK.EMPTY')
+
+    expect(input).not.toBeNull()
+    expect(input!.headers).toEqual(['Delete?', 'ACCOUNT', 'BALANCE'])
+    expect(input!.rows).toEqual([])
+    expect(input!.subtitle).toBe('0 rows x 3 columns')
+  })
+
+  it('offers PDF alongside CSV and Excel in the context menu', () => {
+    const item: any = buildExportMenuItem(() => 'WORK.TEST')
+    const keys = item.submenu.items.map((sub: any) => sub.key)
+    const names = item.submenu.items.map((sub: any) => sub.name)
+
+    expect(keys).toEqual([
+      'export_file:csv',
+      'export_file:xlsx',
+      'export_file:pdf'
+    ])
+    expect(names).toContain('PDF (.pdf)')
+  })
+})

client/src/app/shared/hot-export/hot-export.util.ts

@@ -1,7 +1,193 @@
 import Handsontable from 'handsontable'
 import { MenuItemConfig } from 'handsontable/plugins/contextMenu'
+import { downloadTablePdf, TablePdfInput } from './pdf-export.util'
 
-export type HotExportFormat = 'csv' | 'xlsx'
+export type HotExportFormat = 'csv' | 'xlsx' | 'pdf'
+
+/**
+ * Row cap for the PDF export. Unlike CSV/Excel, a PDF is laid out page by page
+ * in the browser, so an unbounded grid would freeze the tab and produce a
+ * document nobody reads; the export says so in its subtitle when it truncates.
+ */
+const MAX_PDF_ROWS = 5000
+
+/**
+ * Visual (0-based, inclusive) block of the grid to export: honours the current
+ * selection when there is one, otherwise the whole grid. `left` never reaches
+ * into the leading housekeeping columns the caller asked to skip, and a
+ * selection that lies entirely inside those columns yields an empty range
+ * (`right < left`) rather than being shifted sideways into a column the user
+ * did not select.
+ */
+export interface GridRange {
+  top: number
+  left: number
+  bottom: number
+  right: number
+}
+
+/** True when the range covers no column, so there is nothing to export. */
+export function isRangeEmpty(range: GridRange): boolean {
+  return range.right < range.left
+}
+
+/**
+ * Works out which block of cells an export should cover.
+ *
+ * HOT's own export item only honours a selection spanning more than one cell:
+ * a right-click places a single-cell cursor and the corner click is select-all
+ * (negative coordinates), and both of those mean "export the whole table".
+ */
+export function getExportRange(
+  hot: Handsontable,
+  skipLeadingCols = 0
+): GridRange {
+  const sel = hot.getSelectedRangeLast()
+  const lastRow = hot.countRows() - 1
+  const lastCol = hot.countCols() - 1
+
+  let top = 0
+  let left = skipLeadingCols
+  let bottom = lastRow
+  let right = lastCol
+
+  // sel.from/to.row/col are typed nullable (CellCoords allows an unset state),
+  // but a range returned by getSelectedRangeLast() always has real coordinates.
+  const isCornerSelectAll = !!sel && sel.from.row! < 0 && sel.from.col! < 0
+  if (sel && !sel.isSingleCell() && !isCornerSelectAll) {
+    top = Math.max(0, Math.min(sel.from.row!, sel.to.row!))
+    bottom = Math.max(sel.from.row!, sel.to.row!)
+    const selectedLeft = Math.min(sel.from.col!, sel.to.col!)
+    const selectedRight = Math.max(sel.from.col!, sel.to.col!)
+    left = Math.max(skipLeadingCols, selectedLeft)
+    right = selectedRight
+    // Selecting only the skipped leading column(s) leaves nothing to export;
+    // `right` below `left` is the empty range, never a sideways shift.
+    if (right < left) right = left - 1
+  }
+
+  return { top, left, bottom, right }
+}
+
+/**
+ * Columns hidden in the grid (DQ rules in the viewer, filters elsewhere).
+ *
+ * `getPlugin()` returns the plugin instance even when it is not switched on, and
+ * `isHidden()` reads a map that only exists once the plugin has been enabled -
+ * so ask the enabled flag first, and treat any remaining failure as "visible"
+ * rather than losing the whole export.
+ */
+function isHiddenColumn(hot: Handsontable, visualColumn: number): boolean {
+  const plugin = hot.getPlugin('hiddenColumns') as any
+  if (
+    !plugin ||
+    plugin.enabled !== true ||
+    typeof plugin.isHidden !== 'function'
+  ) {
+    return false
+  }
+  try {
+    return plugin.isHidden(visualColumn) === true
+  } catch {
+    return false
+  }
+}
+
+export interface GridTable {
+  headers: string[]
+  rows: (string | number | null)[][]
+}
+
+/**
+ * Reads the exportable slice of a grid into plain headers and rows, the same
+ * way the CSV export sees it: raw cell values (not rendered text), hidden
+ * columns dropped, leading housekeeping columns excluded.
+ *
+ * @param maxRows Stop after this many rows. The limit is applied to the range
+ *   before any cell is read, so a capped export never pays for the rows it
+ *   discards.
+ */
+export function collectGridTable(
+  hot: Handsontable,
+  skipLeadingCols = 0,
+  maxRows = Infinity
+): GridTable {
+  const range = getExportRange(hot, skipLeadingCols)
+  const colHeaders = hot.getColHeader()
+  const headerArray = Array.isArray(colHeaders) ? (colHeaders as string[]) : []
+
+  const columns: number[] = []
+  for (let col = Math.max(0, range.left); col <= range.right; col++) {
+    if (!isHiddenColumn(hot, col)) columns.push(col)
+  }
+
+  const headers = columns.map((col) => {
+    const header = headerArray[col]
+    return header === null || header === undefined ? '' : String(header)
+  })
+
+  const firstRow = Math.max(0, range.top)
+  const lastRow = Math.min(range.bottom, firstRow + maxRows - 1)
+  const rows: (string | number | null)[][] = []
+  for (let row = firstRow; row <= lastRow; row++) {
+    rows.push(
+      columns.map(
+        (col) => hot.getDataAtCell(row, col) as string | number | null
+      )
+    )
+  }
+
+  return { headers, rows }
+}
+
+/**
+ * Client-side PDF export of the grid's current view. Same "what you see"
+ * contract as the CSV/Excel export: the current selection if there is one,
+ * otherwise the whole grid, hidden columns excluded.
+ */
+export function exportGridPdf(
+  hot: Handsontable,
+  filename: string,
+  skipLeadingCols = 0
+): void {
+  const input = buildGridPdfInput(hot, filename, skipLeadingCols)
+  // Without columns there is no table to describe; a grid with no rows still
+  // makes a page, with its headers and a "0 rows" subtitle, rather than the
+  // click doing nothing at all.
+  if (!input) return
+  downloadTablePdf(input, filename || 'export')
+}
+
+/**
+ * Builds the PDF input for a grid, or `null` when the grid has no exportable
+ * columns. Split out from {@link exportGridPdf} so the "no rows" case can be
+ * asserted without touching the browser's download plumbing.
+ */
+export function buildGridPdfInput(
+  hot: Handsontable,
+  filename: string,
+  skipLeadingCols = 0
+): TablePdfInput | null {
+  const range = getExportRange(hot, skipLeadingCols)
+  if (isRangeEmpty(range)) return null
+  const availableRows = Math.max(0, range.bottom - range.top + 1)
+  // The cap is applied inside collectGridTable so the discarded rows are never
+  // read; the subtitle still needs the real total.
+  const { headers, rows } = collectGridTable(hot, skipLeadingCols, MAX_PDF_ROWS)
+  if (!headers.length) return null
+
+  const truncated = availableRows > rows.length
+
+  return {
+    title: filename || 'export',
+    subtitle:
+      `${rows.length} row${rows.length === 1 ? '' : 's'} x ` +
+      `${headers.length} column${headers.length === 1 ? '' : 's'}` +
+      (truncated ? ` (first ${rows.length} of ${availableRows} rows)` : ''),
+    headers,
+    rows
+  }
+}
 
 /**
  * Cell types whose values must be exported as raw stored strings rather than
@@ -25,7 +211,7 @@ const RAW_EXPORT_TYPES = new Set([
  * from any server-side (SAS) download.
  *
  * @param hot Live Handsontable instance.
- * @param format `'csv'` or `'xlsx'`.
+ * @param format `'csv'`, `'xlsx'` or `'pdf'`.
  * @param filename File name (without extension).
  * @param skipLeadingCols Number of leading UI/housekeeping columns to exclude
  *   (e.g. the editor's `Delete?` column at index 0).
@@ -36,6 +222,11 @@ export async function exportGrid(
   filename: string,
   skipLeadingCols = 0
 ): Promise<void> {
+  if (format === 'pdf') {
+    exportGridPdf(hot, filename, skipLeadingCols)
+    return
+  }
+
   const plugin = hot.getPlugin('exportFile')
 
   const opts: Record<string, any> = {
@@ -45,26 +236,20 @@ export async function exportGrid(
 
   // Selection-aware, contiguous range (visual indexes). Hidden columns are
   // excluded automatically (exportHiddenColumns defaults to false).
-  const sel = hot.getSelectedRangeLast()
-  const lastRow = hot.countRows() - 1
-  const lastCol = hot.countCols() - 1
-
-  // Mirror HOT's own export item: only honor a selection that spans more than
-  // one cell. A right-click places a single-cell cursor, and the corner click
-  // is select-all (negative coords) — both mean "export the whole table".
-  // sel.from/to.row/col are typed nullable (CellCoords allows an unset state),
-  // but a range returned by getSelectedRangeLast() always has real coordinates.
-  const isCornerSelectAll = !!sel && sel.from.row! < 0 && sel.from.col! < 0
-  if (sel && !sel.isSingleCell() && !isCornerSelectAll) {
-    const top = Math.max(0, Math.min(sel.from.row!, sel.to.row!))
-    const left = Math.max(0, Math.min(sel.from.col!, sel.to.col!))
-    const bottom = Math.max(sel.from.row!, sel.to.row!)
-    const right = Math.max(sel.from.col!, sel.to.col!)
-    opts['range'] = [top, Math.max(left, skipLeadingCols), bottom, right]
-  }
-
-  if (!opts['range'] && skipLeadingCols > 0) {
-    opts['range'] = [0, skipLeadingCols, lastRow, lastCol]
+  const range = getExportRange(hot, skipLeadingCols)
+  if (isRangeEmpty(range)) return
+  // "Whole grid" means the export starts at visual column 0 - not at
+  // `skipLeadingCols`, or a caller that skips a leading housekeeping column
+  // would look like a whole-grid export and the plugin would put that column
+  // back in. Leaving `range` unset lets the plugin export the whole grid
+  // exactly as its own menu item does; it is explicit whenever it is narrower.
+  const isWholeGrid =
+    range.top === 0 &&
+    range.left === 0 &&
+    range.bottom === hot.countRows() - 1 &&
+    range.right === hot.countCols() - 1
+  if (!isWholeGrid) {
+    opts['range'] = [range.top, range.left, range.bottom, range.right]
   }
 
   if (format === 'csv') {
@@ -126,7 +311,7 @@ export async function exportGrid(
 }
 
 /**
- * Builds a reusable `export_file` context-menu item (CSV / Excel submenu) that
+ * Builds a reusable `export_file` context-menu item (CSV / Excel / PDF submenu) that
  * exports the given grid via {@link exportGrid}. Add it to any HOT context-menu
  * `items` map under the `export_file` key.
  *
@@ -166,6 +351,11 @@ export function buildExportMenuItem(
           key: 'export_file:xlsx',
           name: 'Excel (.xlsx)',
           callback: makeCb('xlsx')
+        },
+        {
+          key: 'export_file:pdf',
+          name: 'PDF (.pdf)',
+          callback: makeCb('pdf')
         }
       ] as any
     }

client/src/app/shared/hot-export/pdf-export.util.spec.ts

@@ -0,0 +1,259 @@
+import {
+  createTablePdf,
+  downloadTablePdf,
+  TablePdfInput
+} from './pdf-export.util'
+
+/** PDF bytes are Latin-1 by construction, so decode them the same way. */
+function decode(bytes: Uint8Array): string {
+  let text = ''
+  for (const byte of bytes) text += String.fromCharCode(byte)
+  return text
+}
+
+function build(overrides: Partial<TablePdfInput> = {}): string {
+  return decode(
+    createTablePdf({
+      title: 'WORK.CUSTOMERS',
+      subtitle: '2 rows x 2 columns',
+      headers: ['ACCOUNT', 'BALANCE'],
+      rows: [
+        ['A1', '1.25'],
+        ['A2', '2.50']
+      ],
+      generatedAt: new Date('2026-11-01T19:00:00Z'),
+      ...overrides
+    })
+  )
+}
+
+/** The `(literal) Tj` operands of a content stream, unescaped enough to measure. */
+function literals(pdf: string): string[] {
+  return Array.from(pdf.matchAll(/\((.*?)\) Tj/g)).map((match) => match[1])
+}
+
+describe('downloadTablePdf', () => {
+  it('releases the blob URL after the download has started, not during it', () => {
+    // Firefox and Safari read the blob asynchronously, so revoking in the same
+    // task as the click cancels the download.
+    jasmine.clock().install()
+    const click = spyOn(HTMLAnchorElement.prototype, 'click')
+    spyOn(URL, 'createObjectURL').and.returnValue('blob:test-url')
+    const revoke = spyOn(URL, 'revokeObjectURL')
+
+    downloadTablePdf({ title: 'T', headers: ['A'], rows: [['1']] }, 'export')
+
+    expect(click).toHaveBeenCalledTimes(1)
+    expect(revoke).not.toHaveBeenCalled()
+    jasmine.clock().tick(1000)
+    expect(revoke).toHaveBeenCalledWith('blob:test-url')
+
+    jasmine.clock().uninstall()
+  })
+})
+
+describe('createTablePdf', () => {
+  it('produces a structurally valid PDF with a working xref', () => {
+    const pdf = build()
+
+    expect(pdf.startsWith('%PDF-1.4')).toBeTrue()
+    expect(pdf.trimEnd().endsWith('%%EOF')).toBeTrue()
+
+    const startxref = Number(pdf.match(/startxref\n(\d+)/)![1])
+    expect(pdf.slice(startxref, startxref + 4)).toBe('xref')
+
+    // Every object listed in the xref must actually start at the offset given.
+    const xref = pdf.slice(startxref).split('\n')
+    const entryCount = Number(xref[1].split(' ')[1])
+    for (let id = 1; id < entryCount; id++) {
+      const offset = Number(xref[2 + id].slice(0, 10))
+      expect(pdf.slice(offset, offset + `${id} 0 obj`.length))
+        .withContext(`object ${id}`)
+        .toBe(`${id} 0 obj`)
+    }
+  })
+
+  it('declares one /Type /Page per page and a matching /Count', () => {
+    const pdf = build()
+    const pageObjects = (pdf.match(/\/Type \/Page[^s]/g) || []).length
+
+    expect(pdf).toContain('/Count 1')
+    expect(pageObjects).toBe(1)
+  })
+
+  it('prints the title, subtitle and a generated stamp', () => {
+    const pdf = build()
+
+    expect(literals(pdf)).toContain('WORK.CUSTOMERS')
+    expect(literals(pdf)).toContain('2 rows x 2 columns')
+    expect(literals(pdf)).toContain('Generated 2026-11-01 19:00')
+  })
+
+  it('prints every cell value', () => {
+    const pdf = build()
+    // BALANCE is numeric, so its values are padded to the column's right edge.
+    const values = literals(pdf).map((literal) => literal.trim())
+
+    expect(values).toEqual(
+      jasmine.arrayContaining(['A1', 'A2', '1.25', '2.50'])
+    )
+  })
+
+  it('escapes the PDF string delimiters', () => {
+    const pdf = build({
+      headers: ['NOTE'],
+      rows: [['a (b) c \\ d']]
+    })
+
+    // The literal must carry the backslash escapes, not raw delimiters.
+    expect(pdf).toContain('(a \\(b\\) c \\\\ d) Tj')
+  })
+
+  it('maps text to WinAnsi where it exists and falls back to ? elsewhere', () => {
+    const pdf = build({
+      headers: ['NAME', 'COST', 'OTHER'],
+      rows: [['M\u00fcller', '\u20ac5', '\u4f8b\u3048']]
+    })
+    const values = literals(pdf).join('|')
+
+    expect(values).toContain('M\\374ller') // u-umlaut: octal for cp1252 0xFC
+    expect(values).toContain('\\2005') // euro sign: cp1252 0x80
+    expect(values).toContain('??') // outside cp1252
+  })
+
+  it('right-aligns numeric columns and left-aligns the rest', () => {
+    const pdf = build({
+      headers: ['NAME', 'AMOUNT'],
+      rows: [
+        ['x', '7.5'],
+        ['long-name', '1234.25']
+      ]
+    })
+    const ops = literals(pdf)
+
+    // AMOUNT is 7 characters wide (its widest value), so 7.5 is padded to the
+    // right edge; NAME is left-aligned, so x is unpadded while the widest value
+    // fills the column.
+    expect(ops).toContain('    7.5')
+    expect(ops).toContain('1234.25')
+    expect(ops).toContain('x')
+    expect(ops).toContain('long-name')
+  })
+
+  it('wraps a value too long for its column instead of overflowing it', () => {
+    const word = 'X'.repeat(140)
+    const pdf = build({ headers: ['ONECOL'], rows: [[word]] })
+    const lines = literals(pdf).filter((line) => /^X+$/.test(line))
+
+    expect(lines.length).toBeGreaterThan(1)
+    expect(lines.join('')).toBe(word)
+    for (const line of lines) expect(line.length).toBeLessThanOrEqual(48)
+  })
+
+  it('keeps a multi-line value on separate lines', () => {
+    const pdf = build({ headers: ['ADDR'], rows: [['LINE1', 'LINE2']] })
+    const pdfMulti = build({ headers: ['ADDR'], rows: [['LINE1\nLINE2']] })
+
+    expect(literals(pdfMulti)).toEqual(
+      jasmine.arrayContaining(['LINE1', 'LINE2'])
+    )
+    expect(literals(pdfMulti)).not.toEqual(literals(pdf))
+  })
+
+  it('paginates long tables and repeats the column headers on every page', () => {
+    const rows = Array.from({ length: 400 }, (_, index) => [
+      `row${index}`,
+      index
+    ])
+    const pdf = build({ headers: ['NAME', 'ID'], rows })
+
+    const pageCount = Number(pdf.match(/\/Count (\d+)/)![1])
+    expect(pageCount).toBeGreaterThan(1)
+
+    const headerAppearances = literals(pdf).filter(
+      (op) => op.trim() === 'NAME' || op.trim() === 'ID'
+    )
+    expect(headerAppearances.length).toBe(pageCount * 2)
+
+    // Footers are numbered from 1 to the total page count.
+    expect(literals(pdf)).toContain(`Page 1 of ${pageCount}`)
+    expect(literals(pdf)).toContain(`Page ${pageCount} of ${pageCount}`)
+  })
+
+  it('shrinks the body font rather than truncating headers on a 20-column table', () => {
+    const columnCount = 20
+    const headers = Array.from(
+      { length: columnCount },
+      (_, index) => `COLUMN_${String(index + 1).padStart(2, '0')}`
+    )
+    const rows = [
+      Array.from({ length: columnCount }, (_, index) => `value ${index}`)
+    ]
+
+    const pdf = build({ headers, rows })
+    const sizes = Array.from(pdf.matchAll(/\/F1 ([\d.]+) Tf/g)).map((m) =>
+      Number(m[1])
+    )
+
+    // Every header survives intact, and the page is won back with font size.
+    expect(literals(pdf)).toContain('COLUMN_01')
+    expect(literals(pdf)).toContain('COLUMN_20')
+    expect(Math.min(...sizes)).toBeLessThan(7)
+  })
+
+  it('never places text outside the printable area', () => {
+    const headers = Array.from({ length: 60 }, (_, index) => `COL_${index}`)
+    const rows = [Array.from({ length: 60 }, (_, index) => `v${index}`)]
+    const pdf = build({ headers, rows })
+
+    const starts = Array.from(pdf.matchAll(/([\d.]+) [\d.-]+ Td/g)).map((m) =>
+      Number(m[1])
+    )
+    // 842 - 24 margin, plus a point of rounding slack.
+    expect(Math.max(...starts)).toBeLessThanOrEqual(819)
+  })
+
+  it('says so when the columns cannot fit even at the smallest font', () => {
+    const columnCount = 300
+    const headers = Array.from(
+      { length: columnCount },
+      (_, index) => `HEADER_${index}`
+    )
+    const rows = [
+      Array.from({ length: columnCount }, (_, index) => `v${index}`)
+    ]
+    const pdf = build({ headers, rows })
+    const note = literals(pdf).find((literal) =>
+      /^Showing the first \d+ of 300 columns$/.test(literal)
+    )
+
+    expect(note).toBeTruthy()
+    expect(literals(pdf)).not.toContain('HEADER_299')
+  })
+
+  it('clamps a row taller than the page and marks the omission', () => {
+    // 20k characters at a 48-character column is ~420 wrapped lines, far more
+    // than a page holds; the tail must be marked, not drawn off the page.
+    const value = Array.from({ length: 4000 }, (_, i) => `w${i}`).join(' ')
+    const pdf = build({ headers: ['LONG'], rows: [[value]] })
+    const lines = literals(pdf)
+
+    expect(lines.some((line) => line.trimEnd().endsWith('...'))).toBeTrue()
+    // The clamped row must fit the page it starts on, not spill onto its own.
+    expect(pdf).toContain('/Count 1')
+
+    // Nothing is drawn below the bottom margin (24pt) or above the page.
+    const tops = Array.from(pdf.matchAll(/[\d.]+ ([\d.-]+) Td/g)).map((m) =>
+      Number(m[1])
+    )
+    expect(Math.min(...tops)).toBeGreaterThanOrEqual(24)
+    expect(Math.max(...tops)).toBeLessThanOrEqual(595 - 24)
+  })
+
+  it('renders an empty table as a single valid page', () => {
+    const pdf = build({ headers: ['ONLY'], rows: [] })
+
+    expect(pdf).toContain('/Count 1')
+    expect(literals(pdf)).toContain('ONLY')
+  })
+})

client/src/app/shared/hot-export/pdf-export.util.ts

@@ -0,0 +1,533 @@
+/**
+ * Zero-dependency PDF writer for tabular data (the VIEW/EDIT grid export).
+ *
+ * Implemented directly against the PDF 1.4 spec rather than pulling in a PDF
+ * library: the output is a plain text table, which needs no font embedding,
+ * no images and no compression. The base-14 Courier font is used because its
+ * glyphs are exactly 600/1000 em wide, so column widths are arithmetic on
+ * character counts instead of a font-metrics table, and the file stays a few
+ * kilobytes per page.
+ *
+ * Scope and limits (deliberate):
+ *  - Text is encoded as WinAnsi (cp1252), the encoding the base-14 fonts cover
+ *    without embedding. Western European characters round-trip; characters
+ *    outside cp1252 are written as `?`.
+ *  - Fonts are not embedded and there is no styling beyond a bold header band
+ *    and row rules, so this is a "what you see" data dump, not a branded
+ *    report. A typeset report belongs on the server (ODS), where SAS formats
+ *    and DC metadata are available.
+ */
+
+/** Glyph advance of base-14 Courier, as a fraction of the font size. */
+const COURIER_WIDTH = 0.6
+const HEADING_SIZE = 10
+const MARGIN = 24
+/** A4 landscape, in points. */
+const PAGE_WIDTH = 842
+const PAGE_HEIGHT = 595
+const MIN_COL_CHARS = 5
+const MAX_COL_CHARS = 48
+/** Blank characters kept between columns so values never touch. */
+const GUTTER_CHARS = 2
+const FOOTER_HEIGHT = 16
+
+/**
+ * Body font sizes tried in turn when the natural column widths do not fit the
+ * page. The table keeps its column content (and its headers) intact by going
+ * smaller first; 5pt is the floor, below which the text stops being readable.
+ */
+const BODY_FONT_SIZES = [7, 6.5, 6, 5.5, 5]
+/**
+ * Narrowest column once scaling is unavoidable at the smallest font size. Below
+ * this a header is a fragment and a value is choppped across lines, so the
+ * export drops trailing columns instead (and says so on the page) rather than
+ * squeezing them into unreadable slivers.
+ */
+const MIN_SCALED_COL_CHARS = 4
+/** Gutter used once scaling is unavoidable, to buy back space per column. */
+const MIN_GUTTER_CHARS = 1
+/** Appended to a cell whose value is longer than one page can show. */
+const TRUNCATION_MARKER = ' ...'
+/** How long the blob URL is kept alive after a click, for slow downloaders. */
+const REVOKE_DELAY_MS = 1000
+
+export interface TablePdfInput {
+  /** Heading shown top-left on page 1, and in the "(continued)" marker after. */
+  title: string
+  /** Optional line under the heading, e.g. row/column counts. */
+  subtitle?: string
+  headers: string[]
+  rows: ReadonlyArray<ReadonlyArray<string | number | null | undefined>>
+  /** Override for the "Generated ..." stamp; defaults to now. */
+  generatedAt?: Date
+}
+
+/** cp1252 code points that differ from Latin-1, which PDF needs as its own byte. */
+const WINANSI_EXTRA: Record<number, number> = {
+  0x20ac: 0x80,
+  0x201a: 0x82,
+  0x0192: 0x83,
+  0x201e: 0x84,
+  0x2026: 0x85,
+  0x2020: 0x86,
+  0x2021: 0x87,
+  0x02c6: 0x88,
+  0x2030: 0x89,
+  0x0160: 0x8a,
+  0x2039: 0x8b,
+  0x0152: 0x8c,
+  0x017d: 0x8e,
+  0x2018: 0x91,
+  0x2019: 0x92,
+  0x201c: 0x93,
+  0x201d: 0x94,
+  0x2022: 0x95,
+  0x2013: 0x96,
+  0x2014: 0x97,
+  0x02dc: 0x98,
+  0x2122: 0x99,
+  0x0161: 0x9a,
+  0x203a: 0x9b,
+  0x0153: 0x9c,
+  0x017e: 0x9e,
+  0x0178: 0x9f
+}
+
+/**
+ * Cell values as printable text: nulls become empty, CR/LF is kept as a line
+ * break (so multi-line values stay multi-line), tabs become spaces and other
+ * control characters are dropped since they have no glyph.
+ */
+function sanitise(value: string | number | null | undefined): string {
+  if (value === null || value === undefined) return ''
+  return String(value)
+    .replace(/\r\n?/g, '\n')
+    .replace(/\t/g, '    ')
+    .replace(/[\u0000-\u0009\u000b-\u001f]/g, ' ')
+}
+
+/**
+ * Escape text for a PDF literal string: `(`, `)` and `\` are the delimiters,
+ * and non-printable / non-ASCII bytes are written as three-digit octal of
+ * their WinAnsi code. Characters outside cp1252 become `?`.
+ */
+function escapeText(text: string): string {
+  let out = ''
+  for (const char of text) {
+    const code = char.codePointAt(0) as number
+    let byte: number
+    if (code >= 0x20 && code <= 0x7e) byte = code
+    else if (WINANSI_EXTRA[code] !== undefined) byte = WINANSI_EXTRA[code]
+    else if (code >= 0xa0 && code <= 0xff) byte = code
+    else byte = 0x3f // '?'
+
+    if (byte === 0x28 || byte === 0x29 || byte === 0x5c) {
+      out += '\\' + String.fromCharCode(byte)
+    } else if (byte < 0x20 || byte > 0x7e) {
+      out += '\\' + byte.toString(8).padStart(3, '0')
+    } else {
+      out += String.fromCharCode(byte)
+    }
+  }
+  return out
+}
+
+function textOp(
+  x: number,
+  y: number,
+  size: number,
+  text: string,
+  font = 'F1'
+): string {
+  return `BT /${font} ${size} Tf ${x.toFixed(2)} ${y.toFixed(2)} Td (${escapeText(text)}) Tj ET\n`
+}
+
+/** Longest line of a value; multi-line values are measured line by line. */
+function longestLine(text: string): number {
+  return Math.max(...text.split('\n').map((part) => part.length), 1)
+}
+
+/**
+ * Word wrap to `width` characters. Words longer than the column are hard-broken
+ * rather than allowed to overflow into the neighbouring column.
+ */
+function wrap(text: string, width: number): string[] {
+  const lines: string[] = []
+  for (const paragraph of text.split('\n')) {
+    let line = ''
+    for (const word of paragraph.split(' ')) {
+      if (line === '') {
+        line = word
+      } else if (line.length + 1 + word.length <= width) {
+        line += ' ' + word
+      } else {
+        lines.push(line)
+        line = word
+      }
+      while (line.length > width) {
+        lines.push(line.slice(0, width))
+        line = line.slice(width)
+      }
+    }
+    lines.push(line)
+  }
+  return lines
+}
+
+/** A column is right-aligned when every non-missing value reads as a number. */
+function isNumericColumn(rows: TablePdfInput['rows'], index: number): boolean {
+  let seen = 0
+  for (const row of rows) {
+    const value = row[index]
+    if (value === null || value === undefined || value === '') continue
+    if (typeof value === 'number') {
+      seen++
+      continue
+    }
+    if (!/^-?\d+(\.\d+)?$/.test(String(value))) return false
+    seen++
+  }
+  return seen > 0
+}
+
+/** Characters that fit across the page at a given font size. */
+function usableChars(fontSize: number): number {
+  return Math.floor((PAGE_WIDTH - MARGIN * 2) / (fontSize * COURIER_WIDTH))
+}
+
+/** Characters a table occupies, gutters included. */
+function tableChars(widths: number[], gutter: number): number {
+  return (
+    widths.reduce((total, width) => total + width, 0) + gutter * widths.length
+  )
+}
+
+interface Layout {
+  fontSize: number
+  lineHeight: number
+  charWidth: number
+  /** Column content widths in characters. */
+  widths: number[]
+  gutter: number
+  /** Columns actually exported; below `headers.length` only for absurd widths. */
+  keptColumns: number
+}
+
+/**
+ * Fits the table across the page.
+ *
+ * The content widths are natural first: as wide as the widest value, bounded.
+ * When those do not fit, the body font shrinks through {@link BODY_FONT_SIZES}
+ * rather than the columns being squeezed, which keeps full headers and values
+ * readable on wide tables. Only at the smallest size are the columns scaled
+ * down (and the gutter narrowed), and columns stop shrinking at
+ * {@link MIN_SCALED_COL_CHARS}: past that, trailing columns are dropped and the
+ * page says how many were kept, so the omission is never silent.
+ */
+function fitLayout(input: TablePdfInput): Layout {
+  const naturalWidths = input.headers.map((header, index) => {
+    let widest = longestLine(sanitise(header))
+    for (const row of input.rows) {
+      widest = Math.max(widest, longestLine(sanitise(row[index])))
+    }
+    return Math.min(MAX_COL_CHARS, Math.max(MIN_COL_CHARS, widest))
+  })
+
+  const naturalTotal = tableChars(naturalWidths, GUTTER_CHARS)
+  for (const fontSize of BODY_FONT_SIZES) {
+    if (naturalTotal <= usableChars(fontSize)) {
+      return {
+        fontSize,
+        lineHeight: fontSize * 1.35,
+        charWidth: fontSize * COURIER_WIDTH,
+        widths: naturalWidths,
+        gutter: GUTTER_CHARS,
+        keptColumns: naturalWidths.length
+      }
+    }
+  }
+
+  // Smallest font, so the only remaining lever is the column widths themselves.
+  const fontSize = BODY_FONT_SIZES[BODY_FONT_SIZES.length - 1]
+  const available = usableChars(fontSize)
+  const gutter = MIN_GUTTER_CHARS
+  const scaleToFit = (count: number): number[] => {
+    const natural = naturalWidths.slice(0, count)
+    const scale =
+      (available - gutter * count) / natural.reduce((a, b) => a + b, 0)
+    return natural.map((width) =>
+      Math.max(MIN_SCALED_COL_CHARS, Math.floor(width * scale))
+    )
+  }
+
+  let keptColumns = naturalWidths.length
+  let widths = scaleToFit(keptColumns)
+  while (keptColumns > 1 && tableChars(widths, gutter) > available) {
+    keptColumns--
+    widths = scaleToFit(keptColumns)
+  }
+
+  return {
+    fontSize,
+    lineHeight: fontSize * 1.35,
+    charWidth: fontSize * COURIER_WIDTH,
+    widths,
+    gutter,
+    keptColumns
+  }
+}
+
+interface PagePlan {
+  /** Content stream operators for the page. */
+  content: string
+}
+
+function paginate(input: TablePdfInput): PagePlan[] {
+  const layout = fitLayout(input)
+  const { fontSize, lineHeight, charWidth, widths, gutter, keptColumns } =
+    layout
+  // Columns beyond the page are dropped rather than drawn off it; the first
+  // page says so, so the document never silently omits data.
+  const headers = input.headers.slice(0, keptColumns)
+  const rows = input.rows.map((row) => row.slice(0, keptColumns))
+  const droppedColumns = input.headers.length - keptColumns
+  const numeric = headers.map((_, index) => isNumericColumn(rows, index))
+  const tableWidth = tableChars(widths, gutter) * charWidth
+
+  const columnsX: number[] = []
+  let cursor = MARGIN
+  for (const width of widths) {
+    columnsX.push(cursor)
+    cursor += (width + gutter) * charWidth
+  }
+
+  const stamp = (input.generatedAt ?? new Date())
+    .toISOString()
+    .replace('T', ' ')
+    .slice(0, 16)
+
+  const headerLines = headers.map((header, index) =>
+    wrap(sanitise(header), widths[index])
+  )
+  const headerLineCount = Math.max(
+    ...headerLines.map((lines) => lines.length),
+    1
+  )
+
+  const pages: PagePlan[] = []
+  let ops = ''
+  let y = 0
+
+  const drawHeaderRow = () => {
+    const bandHeight = headerLineCount * lineHeight
+    ops +=
+      `q 0.93 g ${MARGIN.toFixed(2)} ${(y - bandHeight + lineHeight).toFixed(2)} ` +
+      `${tableWidth.toFixed(2)} ${bandHeight.toFixed(2)} re f Q\n`
+    for (let line = 0; line < headerLineCount; line++) {
+      for (let column = 0; column < headers.length; column++) {
+        const text = headerLines[column][line]
+        if (!text) continue
+        const label = numeric[column] ? text.padStart(widths[column]) : text
+        ops += textOp(columnsX[column], y, fontSize, label, 'F2')
+      }
+      y -= lineHeight
+    }
+    y -= 3
+  }
+
+  const startPage = (first: boolean) => {
+    ops = ''
+    y = PAGE_HEIGHT - MARGIN
+    if (first) {
+      ops += textOp(MARGIN, y, HEADING_SIZE, input.title, 'F2')
+      if (input.subtitle)
+        ops += textOp(MARGIN, y - 13, fontSize, input.subtitle)
+      ops += textOp(
+        MARGIN + tableWidth - 26 * charWidth,
+        y - 13,
+        fontSize,
+        `Generated ${stamp}`
+      )
+      if (droppedColumns > 0) {
+        ops += textOp(
+          MARGIN,
+          y - 13 - lineHeight,
+          fontSize,
+          `Showing the first ${keptColumns} of ${input.headers.length} columns`
+        )
+        y -= fontSize + 4
+      }
+      y -= HEADING_SIZE + 20
+    } else {
+      ops += textOp(MARGIN, y, fontSize, `${input.title} (continued)`, 'F2')
+      y -= lineHeight + 4
+    }
+    drawHeaderRow()
+  }
+
+  const bottom = MARGIN + FOOTER_HEIGHT
+  // A single cell can hold thousands of characters, and MAX_COL_CHARS only
+  // bounds the column: at this font one value can wrap to more lines than a
+  // page has. Such a row is clamped to the tallest row that fits, with the
+  // marker on its last line, rather than being drawn through the footer and
+  // losing its tail without a trace.
+  const pageOneOverhead =
+    HEADING_SIZE +
+    20 +
+    (droppedColumns > 0 ? fontSize + 4 : 0) +
+    headerLineCount * lineHeight +
+    3
+  // Both margins, the footer and the page-1 header block all come off the
+  // height a single row may occupy, or an exactly-fitting row would be pushed
+  // onto a page of its own and leave the first page with only its headers.
+  const maxRowLines = Math.max(
+    1,
+    Math.floor(
+      (PAGE_HEIGHT - MARGIN * 2 - FOOTER_HEIGHT - pageOneOverhead) / lineHeight
+    )
+  )
+
+  startPage(true)
+
+  for (const row of rows) {
+    let cells = widths.map((width, index) => wrap(sanitise(row[index]), width))
+    if (cells.some((lines) => lines.length > maxRowLines)) {
+      cells = cells.map((lines) => {
+        if (lines.length <= maxRowLines) return lines
+        const kept = lines.slice(0, maxRowLines)
+        const last = kept[kept.length - 1]
+        kept[kept.length - 1] =
+          last.length > TRUNCATION_MARKER.length
+            ? last.slice(0, last.length - TRUNCATION_MARKER.length) +
+              TRUNCATION_MARKER
+            : TRUNCATION_MARKER.trim()
+        return kept
+      })
+    }
+    const lineCount = Math.max(...cells.map((lines) => lines.length), 1)
+    if (y - lineCount * lineHeight < bottom) {
+      pages.push({ content: ops })
+      startPage(false)
+    }
+    for (let line = 0; line < lineCount; line++) {
+      for (let column = 0; column < cells.length; column++) {
+        const text = cells[column][line]
+        if (!text) continue
+        const padded = numeric[column] ? text.padStart(widths[column]) : text
+        ops += textOp(columnsX[column], y, fontSize, padded)
+      }
+      y -= lineHeight
+    }
+    y -= 1.5
+    ops +=
+      `0.88 G 0.4 w ${MARGIN.toFixed(2)} ${y.toFixed(2)} m ` +
+      `${(MARGIN + tableWidth).toFixed(2)} ${y.toFixed(2)} l S\n`
+  }
+  pages.push({ content: ops })
+
+  // Footers last, once the total page count is known.
+  return pages.map((page, index) => {
+    const footer = `Page ${index + 1} of ${pages.length}`
+    const x = MARGIN + tableWidth - footer.length * charWidth
+    return { content: page.content + textOp(x, MARGIN, fontSize, footer) }
+  })
+}
+
+/**
+ * Builds the PDF bytes for a table. Returns a `Uint8Array` rather than a Blob so
+ * the output can be asserted in unit tests (Blob is the only browser-specific
+ * part, and it lives in {@link downloadTablePdf}).
+ */
+export function createTablePdf(input: TablePdfInput): Uint8Array<ArrayBuffer> {
+  const pages = paginate(input)
+  const chunks: Uint8Array[] = []
+  let length = 0
+
+  const write = (text: string) => {
+    const bytes = new Uint8Array(text.length)
+    for (let index = 0; index < text.length; index++) {
+      bytes[index] = text.charCodeAt(index) & 0xff
+    }
+    chunks.push(bytes)
+    length += bytes.length
+  }
+
+  write('%PDF-1.4\n%\u00e2\u00e3\u00cf\u00d3\n')
+
+  // Object numbering: 1 catalog, 2 page tree, 3 and 4 the two fonts, then a
+  // page object + content stream pair per page.
+  const offsets: number[] = []
+  const markObject = (id: number) => {
+    offsets[id] = length
+  }
+
+  markObject(1)
+  write('1 0 obj\n<< /Type /Catalog /Pages 2 0 R >>\nendobj\n')
+  markObject(2)
+  const kids = pages.map((_, index) => `${5 + index * 2} 0 R`).join(' ')
+  write(
+    `2 0 obj\n<< /Type /Pages /Count ${pages.length} /Kids [${kids}] >>\nendobj\n`
+  )
+  markObject(3)
+  write(
+    '3 0 obj\n<< /Type /Font /Subtype /Type1 /BaseFont /Courier /Encoding /WinAnsiEncoding >>\nendobj\n'
+  )
+  markObject(4)
+  write(
+    '4 0 obj\n<< /Type /Font /Subtype /Type1 /BaseFont /Courier-Bold /Encoding /WinAnsiEncoding >>\nendobj\n'
+  )
+
+  pages.forEach((page, index) => {
+    const pageId = 5 + index * 2
+    const contentId = pageId + 1
+    markObject(pageId)
+    write(
+      `${pageId} 0 obj\n<< /Type /Page /Parent 2 0 R /MediaBox [0 0 ${PAGE_WIDTH} ${PAGE_HEIGHT}] ` +
+        `/Resources << /Font << /F1 3 0 R /F2 4 0 R >> >> /Contents ${contentId} 0 R >>\nendobj\n`
+    )
+    markObject(contentId)
+    write(`${contentId} 0 obj\n<< /Length ${page.content.length} >>\nstream\n`)
+    write(page.content)
+    write('endstream\nendobj\n')
+  })
+
+  const lastObjectId = 4 + pages.length * 2
+  const xrefOffset = length
+  let xref = `xref\n0 ${lastObjectId + 1}\n0000000000 65535 f \n`
+  for (let id = 1; id <= lastObjectId; id++) {
+    xref += String(offsets[id]).padStart(10, '0') + ' 00000 n \n'
+  }
+  write(xref)
+  write(
+    `trailer\n<< /Size ${lastObjectId + 1} /Root 1 0 R >>\nstartxref\n${xrefOffset}\n%%EOF\n`
+  )
+
+  const bytes = new Uint8Array(length)
+  let cursor = 0
+  for (const chunk of chunks) {
+    bytes.set(chunk, cursor)
+    cursor += chunk.length
+  }
+  return bytes
+}
+
+/**
+ * Builds the PDF and hands it to the browser as a download, mirroring what the
+ * CSV/Excel export does for the same grid. No new tab and no print dialog, so
+ * it behaves the same when DC is embedded in an iframe.
+ */
+export function downloadTablePdf(input: TablePdfInput, filename: string): void {
+  const blob = new Blob([createTablePdf(input)], { type: 'application/pdf' })
+  const url = URL.createObjectURL(blob)
+  const link = document.createElement('a')
+  link.href = url
+  link.download = `${filename || 'export'}.pdf`
+  document.body.appendChild(link)
+  link.click()
+  document.body.removeChild(link)
+  // The click only starts the download; Firefox and Safari read the blob
+  // asynchronously, so revoking in the same task can cancel it and the user
+  // gets nothing. Release the URL on a later turn instead - the blob is a few
+  // hundred kilobytes, so holding it briefly costs nothing.
+  setTimeout(() => URL.revokeObjectURL(url), REVOKE_DELAY_MS)
+}

client/src/app/shared/requests-modal/requests-modal.component.ts

@@ -1,278 +0,0 @@
-import {
-  Component,
-  EventEmitter,
-  Input,
-  OnInit,
-  Output,
-  ViewEncapsulation
-} from '@angular/core'
-import { SASjsRequest } from '@sasjs/adapter'
-import moment from 'moment'
-import { HelperService } from 'src/app/services/helper.service'
-import { LoggerService } from '../../services/logger.service'
-import { SasService } from '../../services/sas.service'
-import { ServerType } from '@sasjs/utils/types/serverType'
-interface SASjsRequestExtended extends SASjsRequest {
-  parsedTimestamp?: string
-  logErrors?: string[]
-  logWarnings?: string[]
-  selectedTable?: string
-}
-
-@Component({
-  selector: 'app-requests-modal',
-  templateUrl: './requests-modal.component.html',
-  styleUrls: ['./requests-modal.component.scss'],
-  encapsulation: ViewEncapsulation.None,
-  standalone: false
-})
-export class RequestsModalComponent implements OnInit {
-  private _opened: boolean = false
-  get opened(): boolean {
-    return this._opened
-  }
-  @Input()
-  set opened(value: boolean) {
-    this._opened = value
-    if (value) this.modalOpened()
-
-    this.loggerService.log(this.sasjsRequests)
-  }
-
-  @Output() openedChange = new EventEmitter()
-
-  public sasLogActive: boolean = true
-  public sasSourceCodeActive: boolean = false
-  public sasGeneratedCodeActive: boolean = false
-  public tablesActive: boolean = false
-
-  public sasjsConfig = this.sasService.getSasjsConfig()
-  public serverType: string
-  public sasjsRequests: SASjsRequestExtended[] = []
-  public workTables: any
-
-  constructor(
-    private sasService: SasService,
-    private loggerService: LoggerService,
-    private helperService: HelperService
-  ) {
-    this.sasjsConfig = this.sasService.getSasjsConfig()
-    this.serverType = this.sasService.getServerType()
-  }
-
-  ngOnInit(): void {}
-
-  public parseLogTimestamp(timestamp: any) {
-    return `${this.formatTimestamp(timestamp)} ${this.timestampFromNow(
-      timestamp
-    )}`
-  }
-
-  public cutAppLoc(link: string) {
-    return link.replace(this.sasjsConfig.appLoc + '/', '')
-  }
-
-  public formatTimestamp(timestamp: any) {
-    return moment(timestamp).format()
-      ? moment(timestamp).format('dddd, MMMM Do YYYY, h:mm:ss a')
-      : timestamp
-  }
-
-  public timestampFromNow(timestamp: any) {
-    return moment(timestamp).format() ? ` (${moment(timestamp).fromNow()})` : ''
-  }
-
-  public modalOpenChange(state: any) {
-    this.opened = state
-    this.openedChange.emit(this.opened)
-  }
-
-  public modalOpened() {
-    this.sasjsRequests = this.sasService.getSasRequests()
-
-    for (let request of this.sasjsRequests) {
-      this.parseErrorsAndWarnings(request)
-
-      request.serviceLink = this.cutAppLoc(request.serviceLink)
-      request.parsedTimestamp = this.parseLogTimestamp(request.timestamp)
-    }
-  }
-
-  public goToLogLine(
-    linkingLine: string,
-    requestStackId: string,
-    type: string
-  ) {
-    const logWrapper: HTMLElement | null = document.querySelector(
-      `#${requestStackId} .log-wrapper.saslog`
-    )
-
-    if (!logWrapper) return
-
-    if (this.serverType === 'SASVIYA') {
-      // For VIYA (textContent approach)
-      const logContent = logWrapper.textContent || ''
-      const logLines = logContent.split('\n')
-      let lineIndex = -1
-
-      // Find the matching line index
-      for (let i = 0; i < logLines.length; i++) {
-        if (logLines[i].includes(linkingLine)) {
-          lineIndex = i
-          break
-        }
-      }
-
-      if (lineIndex === -1) return
-
-      // Create a temporary div to calculate line height
-      const tempDiv = document.createElement('div')
-      tempDiv.className = 'temp-line-height-calc'
-      tempDiv.textContent = 'X'
-      logWrapper.appendChild(tempDiv)
-      const lineHeight = tempDiv.clientHeight
-      logWrapper.removeChild(tempDiv)
-
-      // Scroll to the appropriate position
-      logWrapper.scrollTop = lineHeight * lineIndex
-
-      // Create highlighting overlay at that position
-      const overlay = document.createElement('div')
-      overlay.className = `line-highlight-overlay ${type === 'error' ? 'error-highlight' : 'warning-highlight'}`
-
-      // We are setting the height using a CSS variable to the HTML element
-      // this does not trigger CSP errors because it's driven by JS
-      overlay.classList.add('temp-height-setter')
-      document.documentElement.style.setProperty(
-        '--line-height',
-        `${lineHeight}px`
-      )
-      overlay.classList.add('line-position-setter')
-      document.documentElement.style.setProperty(
-        '--line-top',
-        `${lineHeight * lineIndex}px`
-      )
-
-      logWrapper.appendChild(overlay)
-
-      setTimeout(() => {
-        if (logWrapper.contains(overlay)) {
-          logWrapper.removeChild(overlay)
-        }
-      }, 3000)
-    } else {
-      // For non-VIYA (innerHTML approach)
-      const allLines: NodeListOf<Element> = document.querySelectorAll(
-        `#${requestStackId} .log-wrapper.saslog font`
-      )
-
-      for (let line of Array.from(allLines)) {
-        if (line.textContent?.includes(linkingLine)) {
-          logWrapper.scrollTop =
-            (line as HTMLElement).offsetTop - logWrapper.offsetTop
-          // Use class instead of inline style
-          line.classList.add('highlighted-line')
-
-          setTimeout(() => {
-            line.classList.remove('highlighted-line')
-          }, 3000)
-          break
-        }
-      }
-    }
-  }
-
-  public async parseErrorsAndWarnings(req: any) {
-    if (!req || !req.logFile || typeof req.logFile !== 'string') return
-    if (req['logErrors'] !== undefined || req['logWarnings'] !== undefined)
-      return
-
-    let errorLines = []
-    let warningLines = []
-
-    // Create a safe version of the log content
-    let logLines = req.logFile.split('\n')
-    let originalLogLines = [...logLines]
-
-    for (let i = 0; i < logLines.length; i++) {
-      if (/<.*>ERROR/gm.test(logLines[i])) {
-        let errorLine = logLines[i].substring(
-          logLines[i].indexOf('E'),
-          logLines[i].length - 1
-        )
-        errorLines.push(errorLine)
-      } else if (/^ERROR/gm.test(logLines[i])) {
-        errorLines.push(logLines[i])
-
-        if (this.serverType !== 'SASVIYA') {
-          logLines[i] = '<font class="error-line">' + logLines[i] + '</font>'
-        }
-      }
-
-      if (/<.*>WARNING/gm.test(logLines[i])) {
-        let warningLine = logLines[i].substring(
-          logLines[i].indexOf('W'),
-          logLines[i].length - 1
-        )
-        warningLines.push(warningLine)
-      } else if (/^WARNING/gm.test(logLines[i])) {
-        warningLines.push(logLines[i])
-
-        if (this.serverType !== 'SASVIYA') {
-          logLines[i] = '<font class="warning-line">' + logLines[i] + '</font>'
-        }
-      }
-    }
-
-    this.loggerService.log(warningLines)
-
-    // For VIYA, store both versions of the log
-    if (this.serverType === 'SASVIYA') {
-      req.originalLogFile = originalLogLines.join('\n')
-      req.logFileLineMap = {}
-
-      // Create a mapping of error/warning lines to their indices
-      errorLines.forEach((errorLine) => {
-        for (let i = 0; i < originalLogLines.length; i++) {
-          if (originalLogLines[i].includes(errorLine)) {
-            if (!req.logFileLineMap.errors) req.logFileLineMap.errors = {}
-            req.logFileLineMap.errors[errorLine] = i
-            break
-          }
-        }
-      })
-
-      warningLines.forEach((warningLine) => {
-        for (let i = 0; i < originalLogLines.length; i++) {
-          if (originalLogLines[i].includes(warningLine)) {
-            if (!req.logFileLineMap.warnings) req.logFileLineMap.warnings = {}
-            req.logFileLineMap.warnings[warningLine] = i
-            break
-          }
-        }
-      })
-    }
-
-    req.logFile = logLines.join('\n')
-    req.logErrors = errorLines
-    req.logWarnings = warningLines
-  }
-
-  downloadLog(logFile: string) {
-    const timestamp = new Date().valueOf()
-    this.helperService.downloadTextFile(`logFile-${timestamp}`, logFile)
-  }
-
-  downloadSourceCode(sourceCode: string) {
-    const timestamp = new Date().valueOf()
-    this.helperService.downloadTextFile(`sourceCode-${timestamp}`, sourceCode)
-  }
-
-  downloadGeneratedCode(generatedCode: string) {
-    const timestamp = new Date().valueOf()
-    this.helperService.downloadTextFile(
-      `generatedCode-${timestamp}`,
-      generatedCode
-    )
-  }
-}

client/src/app/shared/sas-logs-modal/sas-logs-modal.component.html

@@ -2,10 +2,10 @@
   [clrModalOpen]="opened"
   (clrModalOpenChange)="modalOpenChange($event)"
   clrModalSize="xl"
-  class="requests-modal"
+  class="sas-logs-modal"
 >
   <h3 class="modal-title">
-    Request History
+    SAS Logs
     <p class="mt-0 d-inline-block">App Location: {{ sasjsConfig.appLoc }}</p>
   </h3>
   <div class="modal-body h-70vh">
@@ -20,7 +20,7 @@
             <clr-stack-label>
               {{ programLog.serviceLink }}
               <span class="ml-10">
-                @if (programLog.logWarnings?.length! > 0) {
+                @if (programLog.warnings?.length! > 0) {
                   <span>
                     <clr-icon
                       class="mb-5 color-green"
@@ -28,7 +28,7 @@
                     ></clr-icon>
                   </span>
                 }
-                @if (programLog.logErrors?.length! > 0) {
+                @if (programLog.errors?.length! > 0) {
                   <span>
                     <clr-icon
                       class="mb-5 color-red"
@@ -49,51 +49,70 @@
                     <ng-template [(clrIfActive)]="sasLogActive">
                       <clr-tab-content>
                         <clr-tree>
-                          @if (programLog.logErrors) {
+                          @if (programLog.errors?.length) {
                             <clr-tree-node class="err-links">
                               <clr-icon
                                 class="color-red"
                                 shape="exclamation-triangle"
                               ></clr-icon>
-                              Errors ({{ programLog.logErrors.length! }})
-                              @for (
-                                err of programLog.logErrors || [];
-                                track err
-                              ) {
+                              Errors ({{ programLog.errors!.length }})
+                              @for (issue of programLog.errors; track $index) {
                                 <clr-tree-node
-                                  (click)="
-                                    goToLogLine(err, 'request_' + i, 'error')
-                                  "
+                                  (click)="goToLogLine(issue, 'request_' + i)"
                                 >
                                   <button class="clr-treenode-link">
-                                    {{ err }}
+                                    @if (issue.code) {
+                                      <span class="issue-code">{{
+                                        issue.code
+                                      }}</span>
+                                    }
+                                    {{ issue.message }}
                                   </button>
                                 </clr-tree-node>
                               }
                             </clr-tree-node>
                           }
-                          @if (programLog.logWarnings) {
+                          @if (programLog.warnings?.length) {
                             <clr-tree-node class="err-links">
                               <clr-icon
                                 class="color-green"
                                 shape="exclamation-circle"
                               ></clr-icon>
-                              Warnings ({{ programLog.logWarnings.length! }})
+                              Warnings ({{ programLog.warnings!.length }})
                               @for (
-                                warning of programLog.logWarnings || [];
-                                track warning
+                                issue of programLog.warnings;
+                                track $index
                               ) {
                                 <clr-tree-node
-                                  (click)="
-                                    goToLogLine(
-                                      warning,
-                                      'request_' + i,
-                                      'warning'
-                                    )
-                                  "
+                                  (click)="goToLogLine(issue, 'request_' + i)"
                                 >
                                   <button class="clr-treenode-link">
-                                    {{ warning }}
+                                    @if (issue.code) {
+                                      <span class="issue-code">{{
+                                        issue.code
+                                      }}</span>
+                                    }
+                                    {{ issue.message }}
+                                  </button>
+                                </clr-tree-node>
+                              }
+                            </clr-tree-node>
+                          }
+                          @if (programLog.notes?.length) {
+                            <clr-tree-node class="err-links">
+                              <clr-icon shape="info-circle"></clr-icon>
+                              Notes ({{ programLog.notes!.length }})
+                              @for (issue of programLog.notes; track $index) {
+                                <clr-tree-node
+                                  (click)="goToLogLine(issue, 'request_' + i)"
+                                >
+                                  <button class="clr-treenode-link">
+                                    @if (issue.code) {
+                                      <span class="issue-code">{{
+                                        issue.code
+                                      }}</span>
+                                    }
+                                    {{ issue.message }}
                                   </button>
                                 </clr-tree-node>
                               }
@@ -103,7 +122,7 @@
                         @if (serverType !== 'SASVIYA') {
                           <div
                             class="log-wrapper saslog"
-                            [innerHTML]="programLog.logFile"
+                            [innerHTML]="programLog.logHtml"
                           ></div>
                         }
                         @if (serverType === 'SASVIYA') {

client/src/app/shared/sas-logs-modal/sas-logs-modal.component.spec.ts

@@ -0,0 +1,319 @@
+import { LogIssue } from '@sasjs/utils/logs'
+import { SasLogsModalComponent } from './sas-logs-modal.component'
+
+const LOG = [
+  '1    data want;',
+  '2      set sashelp.class;',
+  '3      x = age / 0;',
+  '4    run;',
+  'NOTE: There were 19 observations read from the data set SASHELP.CLASS.',
+  'WARNING: Division by zero detected at line 3 column 11.',
+  'ERROR: File WORK.MISSING_DS.DATA does not exist.',
+  'ERROR 22-322: Syntax error, expecting one of the following: ;, CANCEL,',
+  '              RUN, QUIT.',
+  'NOTE: Invalid data for age in line 42 3-5.'
+].join('\n')
+
+const issue = (overrides: Partial<LogIssue>): LogIssue => {
+  const line = overrides.line ?? 0
+
+  return {
+    line,
+    endLine: line,
+    column: 0,
+    endColumn: 0,
+    severity: 'error',
+    message: 'a message',
+    ...overrides
+  } as LogIssue
+}
+
+describe('SasLogsModalComponent', () => {
+  let component: SasLogsModalComponent
+  let sasService: any
+  let helperService: any
+
+  const buildComponent = (
+    requests: any[] = [],
+    serverType: string = 'SAS9'
+  ): SasLogsModalComponent => {
+    sasService = {
+      getSasjsConfig: () => ({ appLoc: '/Public/app' }),
+      getServerType: () => serverType,
+      getSasRequests: () => requests
+    }
+
+    helperService = {
+      downloadTextFile: jasmine.createSpy('downloadTextFile')
+    }
+
+    return new SasLogsModalComponent(
+      sasService,
+      { log: () => {} } as any,
+      helperService
+    )
+  }
+
+  beforeEach(() => {
+    component = buildComponent()
+  })
+
+  describe('analyseLog', () => {
+    it('groups the errors, the warnings and the notable notes of a log', () => {
+      const request: any = { logFile: LOG }
+      component.analyseLog(request)
+
+      expect(request.errors.map((i: LogIssue) => i.line)).toEqual([6, 7])
+      expect(request.warnings.map((i: LogIssue) => i.line)).toEqual([5])
+      expect(request.notes.map((i: LogIssue) => i.message)).toEqual([
+        'Invalid data for age in line 42 3-5.'
+      ])
+    })
+
+    it('keeps the message and the code the parser reported', () => {
+      const request: any = { logFile: LOG }
+      component.analyseLog(request)
+
+      expect(request.errors[0]).toEqual(
+        issue({
+          line: 6,
+          endLine: 6,
+          column: 0,
+          endColumn: 'ERROR: File WORK.MISSING_DS.DATA does not exist.'.length,
+          message: 'File WORK.MISSING_DS.DATA does not exist.'
+        })
+      )
+      expect(request.errors[1].code).toEqual('22-322')
+      expect(request.errors[1].endLine).toEqual(8)
+      expect(request.errors[1].message).toEqual(
+        'Syntax error, expecting one of the following: ;, CANCEL, RUN, QUIT.'
+      )
+    })
+
+    it('reports nothing for a log with nothing to report', () => {
+      const request: any = {
+        logFile: 'NOTE: There were 19 observations read.\n'
+      }
+      component.analyseLog(request)
+
+      expect(request.errors).toEqual([])
+      expect(request.warnings).toEqual([])
+      expect(request.notes).toEqual([])
+    })
+
+    it('reports nothing when the request carries no log at all', () => {
+      const request: any = { serviceLink: 'services/x' }
+      component.analyseLog(request)
+
+      expect(request.logIssues).toEqual([])
+      expect(request.errors).toEqual([])
+      expect(request.logHtml).toEqual('')
+    })
+  })
+
+  describe('markIssueLines', () => {
+    it('marks every line an issue occupies, the wrapped ones included', () => {
+      const request: any = { logFile: LOG }
+      component.analyseLog(request)
+
+      const lines: string[] = request.logHtml.split('\n')
+
+      expect(lines[5]).toEqual(
+        '<font class="warning-line">WARNING: Division by zero detected at line 3 column 11.</font>'
+      )
+      expect(lines[6]).toContain('<font class="error-line">ERROR: File WORK')
+      expect(lines[7]).toContain('<font class="error-line">ERROR 22-322:')
+      expect(lines[8]).toEqual(
+        '<font class="error-line">              RUN, QUIT.</font>'
+      )
+    })
+
+    it('leaves a notable note unmarked, and the other lines untouched', () => {
+      const request: any = { logFile: LOG }
+      component.analyseLog(request)
+
+      const lines: string[] = request.logHtml.split('\n')
+
+      expect(lines[9]).toEqual('NOTE: Invalid data for age in line 42 3-5.')
+      expect(lines[0]).toEqual('1    data want;')
+      expect(lines).toHaveSize(10)
+    })
+
+    it('returns the log as it is when it holds no issues', () => {
+      const log = 'NOTE: There were 19 observations read.'
+
+      expect(component.markIssueLines(log, [])).toEqual(log)
+    })
+  })
+
+  describe('goToLogLine', () => {
+    let wrapper: HTMLElement
+
+    beforeEach(() => {
+      const stackBlock = document.createElement('div')
+      stackBlock.id = 'request_0'
+      wrapper = document.createElement('div')
+      wrapper.className = 'log-wrapper saslog'
+      wrapper.textContent = LOG
+      stackBlock.appendChild(wrapper)
+      document.body.appendChild(stackBlock)
+
+      // The height of a rendered line needs a layout; a fixed value keeps the
+      // arithmetic under test deterministic.
+      ;(component as any).measureLineHeight = () => 10
+    })
+
+    afterEach(() => wrapper.parentElement?.remove())
+
+    it('positions the highlight from the line the parser reported', () => {
+      component.goToLogLine(issue({ line: 6 }), 'request_0')
+
+      expect(
+        document.documentElement.style.getPropertyValue('--line-top')
+      ).toEqual('60px')
+      expect(
+        document.documentElement.style.getPropertyValue('--line-height')
+      ).toEqual('10px')
+      expect(
+        document.documentElement.style.getPropertyValue('--line-count')
+      ).toEqual('1')
+    })
+
+    it('does not jump to the first of two identical messages', () => {
+      const repeated = issue({ line: 7, message: 'a message' })
+
+      component.goToLogLine(repeated, 'request_0')
+
+      expect(
+        document.documentElement.style.getPropertyValue('--line-top')
+      ).toEqual('70px')
+    })
+
+    it('covers every line a wrapped message runs to', () => {
+      component.goToLogLine(issue({ line: 7, endLine: 8 }), 'request_0')
+
+      expect(
+        document.documentElement.style.getPropertyValue('--line-count')
+      ).toEqual('2')
+    })
+
+    it('flashes the colour of the severity it was given', () => {
+      component.goToLogLine(
+        issue({ line: 5, severity: 'warning' }),
+        'request_0'
+      )
+
+      const overlay = wrapper.querySelector('.line-highlight-overlay')
+
+      expect(overlay).not.toBeNull()
+      expect(overlay!.classList).toContain('warning-highlight')
+    })
+
+    it('does nothing when the request has no log on screen', () => {
+      expect(() =>
+        component.goToLogLine(issue({ line: 1 }), 'request_9')
+      ).not.toThrow()
+    })
+  })
+
+  describe('modalOpened', () => {
+    it('analyses every request and cuts the app location from its link', () => {
+      component = buildComponent([
+        {
+          serviceLink: '/Public/app/services/editors/save',
+          timestamp: new Date(),
+          logFile: LOG
+        }
+      ])
+      component.modalOpened()
+
+      const [request] = component.sasjsRequests as any[]
+
+      expect(request.serviceLink).toEqual('services/editors/save')
+      expect(request.errors).toHaveSize(2)
+      expect(request.warnings).toHaveSize(1)
+      expect(request.notes).toHaveSize(1)
+    })
+  })
+
+  describe('the opened input', () => {
+    it('analyses the requests when the modal opens', () => {
+      component = buildComponent([
+        {
+          serviceLink: '/Public/app/services/x',
+          timestamp: new Date(),
+          logFile: LOG
+        }
+      ])
+
+      component.opened = true
+
+      expect(component.opened).toBeTrue()
+      expect((component.sasjsRequests[0] as any).errors).toHaveSize(2)
+    })
+
+    it('does not re-read the requests when it closes', () => {
+      const getSasRequests = jasmine
+        .createSpy('getSasRequests')
+        .and.returnValue([])
+      component = buildComponent([])
+      sasService.getSasRequests = getSasRequests
+      component.opened = true
+      getSasRequests.calls.reset()
+
+      component.opened = false
+
+      expect(component.opened).toBeFalse()
+      expect(getSasRequests).not.toHaveBeenCalled()
+    })
+
+    it('reports the new state to the parent', () => {
+      const openedChange = jasmine.createSpy('openedChange')
+      component.openedChange.subscribe(openedChange)
+
+      component.modalOpenChange(false)
+
+      expect(openedChange).toHaveBeenCalledWith(false)
+    })
+  })
+
+  describe('timestamps', () => {
+    it('formats a known date in full and relative to now', () => {
+      const date = new Date('2026-03-04T15:06:07Z')
+
+      expect(component.formatTimestamp(date)).toContain('March 4th 2026')
+      expect(component.parseLogTimestamp(date)).toContain('March 4th 2026')
+      expect(component.timestampFromNow(date)).toContain('ago')
+    })
+
+    it('hands back a timestamp it cannot parse', () => {
+      expect(component.formatTimestamp('not a date')).toEqual('not a date')
+      expect(component.timestampFromNow('not a date')).toEqual('')
+    })
+  })
+
+  describe('downloads', () => {
+    it('writes the log it was given under a timestamped name', () => {
+      component.downloadLog('a log')
+
+      expect(helperService.downloadTextFile).toHaveBeenCalledWith(
+        jasmine.stringMatching(/^logFile-\d+$/),
+        'a log'
+      )
+    })
+
+    it('writes the source and the generated code under their own names', () => {
+      component.downloadSourceCode('source')
+      component.downloadGeneratedCode('generated')
+
+      expect(helperService.downloadTextFile).toHaveBeenCalledWith(
+        jasmine.stringMatching(/^sourceCode-\d+$/),
+        'source'
+      )
+      expect(helperService.downloadTextFile).toHaveBeenCalledWith(
+        jasmine.stringMatching(/^generatedCode-\d+$/),
+        'generated'
+      )
+    })
+  })
+})

client/src/app/shared/sas-logs-modal/sas-logs-modal.component.ts

@@ -0,0 +1,227 @@
+import {
+  Component,
+  EventEmitter,
+  Input,
+  OnInit,
+  Output,
+  ViewEncapsulation
+} from '@angular/core'
+import { SASjsRequest } from '@sasjs/adapter'
+import { LogIssue, parseLog } from '@sasjs/utils/logs'
+import moment from 'moment'
+import { HelperService } from 'src/app/services/helper.service'
+import { LoggerService } from '../../services/logger.service'
+import { SasService } from '../../services/sas.service'
+import { ServerType } from '@sasjs/utils/types/serverType'
+
+interface SASjsRequestExtended extends SASjsRequest {
+  parsedTimestamp?: string
+  /** Every error, warning and notable note in `logFile`, in log order. */
+  logIssues?: LogIssue[]
+  errors?: LogIssue[]
+  warnings?: LogIssue[]
+  notes?: LogIssue[]
+  /** The log with its error and warning lines wrapped in markup, for rendering. */
+  logHtml?: string
+  selectedTable?: string
+}
+
+@Component({
+  selector: 'app-sas-logs-modal',
+  templateUrl: './sas-logs-modal.component.html',
+  styleUrls: ['./sas-logs-modal.component.scss'],
+  encapsulation: ViewEncapsulation.None,
+  standalone: false
+})
+export class SasLogsModalComponent implements OnInit {
+  private _opened: boolean = false
+  get opened(): boolean {
+    return this._opened
+  }
+  @Input()
+  set opened(value: boolean) {
+    this._opened = value
+    if (value) this.modalOpened()
+
+    this.loggerService.log(this.sasjsRequests)
+  }
+
+  @Output() openedChange = new EventEmitter()
+
+  public sasLogActive: boolean = true
+  public sasSourceCodeActive: boolean = false
+  public sasGeneratedCodeActive: boolean = false
+  public tablesActive: boolean = false
+
+  public sasjsConfig = this.sasService.getSasjsConfig()
+  public serverType: string
+  public sasjsRequests: SASjsRequestExtended[] = []
+  public workTables: any
+
+  constructor(
+    private sasService: SasService,
+    private loggerService: LoggerService,
+    private helperService: HelperService
+  ) {
+    this.sasjsConfig = this.sasService.getSasjsConfig()
+    this.serverType = this.sasService.getServerType()
+  }
+
+  ngOnInit(): void {}
+
+  public parseLogTimestamp(timestamp: any) {
+    return `${this.formatTimestamp(timestamp)} ${this.timestampFromNow(
+      timestamp
+    )}`
+  }
+
+  public cutAppLoc(link: string) {
+    return link.replace(this.sasjsConfig.appLoc + '/', '')
+  }
+
+  public formatTimestamp(timestamp: any) {
+    const parsed = moment(timestamp)
+
+    return parsed.isValid()
+      ? parsed.format('dddd, MMMM Do YYYY, h:mm:ss a')
+      : timestamp
+  }
+
+  public timestampFromNow(timestamp: any) {
+    const parsed = moment(timestamp)
+
+    return parsed.isValid() ? ` (${parsed.fromNow()})` : ''
+  }
+
+  public modalOpenChange(state: any) {
+    this.opened = state
+    this.openedChange.emit(this.opened)
+  }
+
+  public modalOpened() {
+    this.sasjsRequests = this.sasService.getSasRequests()
+
+    for (let request of this.sasjsRequests) {
+      this.analyseLog(request)
+
+      request.serviceLink = this.cutAppLoc(request.serviceLink)
+      request.parsedTimestamp = this.parseLogTimestamp(request.timestamp)
+    }
+  }
+
+  /**
+   * Reads the request's log once and keeps what the template renders: the
+   * issues SAS wrote, grouped by severity, and - for a log carrying markup -
+   * the log itself with those lines marked.
+   */
+  public analyseLog(request: SASjsRequestExtended) {
+    const log = typeof request.logFile === 'string' ? request.logFile : ''
+    const issues = parseLog(log)
+
+    request.logIssues = issues
+    request.errors = issues.filter((issue) => issue.severity === 'error')
+    request.warnings = issues.filter((issue) => issue.severity === 'warning')
+    request.notes = issues.filter((issue) => issue.severity === 'note')
+    request.logHtml = this.markIssueLines(log, issues)
+  }
+
+  /**
+   * The log with each line an error or a warning occupies wrapped in a class
+   * that colours it, so the lists and the log agree about what is wrong and
+   * where. A wrapped message is marked on every line it runs to, and a notable
+   * note is left unmarked - it is not a defect.
+   */
+  public markIssueLines(log: string, issues: LogIssue[]): string {
+    const severityByLine = new Map<number, string>()
+
+    for (const issue of issues) {
+      if (issue.severity === 'note') continue
+
+      for (let line = issue.line; line <= issue.endLine; line++) {
+        severityByLine.set(line, issue.severity)
+      }
+    }
+
+    if (severityByLine.size === 0) return log
+
+    return log
+      .split('\n')
+      .map((line, index) => {
+        const severity = severityByLine.get(index)
+
+        return severity ? `<font class="${severity}-line">${line}</font>` : line
+      })
+      .join('\n')
+  }
+
+  /**
+   * Scrolls the log to an issue and flashes its line, using the line number the
+   * parser reported rather than searching the rendered text - a log repeats its
+   * messages, so a search lands on the first of them every time.
+   */
+  public goToLogLine(issue: LogIssue, requestStackId: string) {
+    const logWrapper = document.querySelector(
+      `#${requestStackId} .log-wrapper.saslog`
+    ) as HTMLElement | null
+
+    if (!logWrapper) return
+
+    const lineHeight = this.measureLineHeight(logWrapper)
+    const lineCount = issue.endLine - issue.line + 1
+    const top = lineHeight * issue.line
+
+    logWrapper.scrollTop = top
+
+    // The height and position reach the stylesheet as CSS variables, which
+    // keeps the inline style away from the element and therefore away from CSP.
+    document.documentElement.style.setProperty(
+      '--line-height',
+      `${lineHeight}px`
+    )
+    document.documentElement.style.setProperty('--line-count', `${lineCount}`)
+    document.documentElement.style.setProperty('--line-top', `${top}px`)
+
+    const overlay = document.createElement('div')
+    overlay.className = `line-highlight-overlay ${issue.severity}-highlight`
+    overlay.classList.add('temp-height-setter')
+    overlay.classList.add('line-position-setter')
+
+    logWrapper.appendChild(overlay)
+
+    setTimeout(() => {
+      if (logWrapper.contains(overlay)) {
+        logWrapper.removeChild(overlay)
+      }
+    }, 3000)
+  }
+
+  /** The height of one rendered log line, measured rather than assumed. */
+  private measureLineHeight(logWrapper: HTMLElement): number {
+    const tempDiv = document.createElement('div')
+    tempDiv.className = 'temp-line-height-calc'
+    tempDiv.textContent = 'X'
+    logWrapper.appendChild(tempDiv)
+    const lineHeight = tempDiv.clientHeight
+    logWrapper.removeChild(tempDiv)
+
+    return lineHeight
+  }
+
+  downloadLog(logFile: string) {
+    const timestamp = new Date().valueOf()
+    this.helperService.downloadTextFile(`logFile-${timestamp}`, logFile)
+  }
+
+  downloadSourceCode(sourceCode: string) {
+    const timestamp = new Date().valueOf()
+    this.helperService.downloadTextFile(`sourceCode-${timestamp}`, sourceCode)
+  }
+
+  downloadGeneratedCode(generatedCode: string) {
+    const timestamp = new Date().valueOf()
+    this.helperService.downloadTextFile(
+      `generatedCode-${timestamp}`,
+      generatedCode
+    )
+  }
+}

client/src/app/shared/user-nav-dropdown/header-actions.component.html

@@ -71,8 +71,8 @@
           </clr-toggle-wrapper>
         </clr-toggle-container>
       </div>
-      <a (click)="openRequestsModal()" clrDropdownItem>
-        <span>SAS Requests</span>
+      <a (click)="openSasLogsModal()" clrDropdownItem>
+        <span>SAS Logs</span>
       </a>
 
       @if (!isDeployPage) {

client/src/app/shared/user-nav-dropdown/header-actions.component.ts

@@ -59,8 +59,8 @@ export class HeaderActions implements OnInit, OnDestroy {
     this.eventService.toggleDarkMode(value)
   }
 
-  openRequestsModal() {
-    this.eventService.openRequestsModal()
+  openSasLogsModal() {
+    this.eventService.openSasLogsModal()
   }
 
   onDebugModeChange(dropdownItem?: any): void {

client/src/app/shared/utils/grid-aria.spec.ts

@@ -0,0 +1,183 @@
+import Handsontable from 'handsontable'
+import { fixGridAria } from './grid-aria'
+
+const buildGrid = (id: string): HTMLElement => {
+  const container = document.createElement('div')
+  container.id = `container_${id}`
+  container.innerHTML = `
+    <div class="ht-wrapper handsontable" id="ht_${id}" role="treegrid"
+         aria-rowcount="-1" aria-colcount="3" aria-multiselectable="true">
+      <div class="wtHolder" role="presentation">
+        <table class="htCore">
+          <thead><tr><th>Class</th></tr></thead>
+          <tbody><tr><td><button>Edit record</button></td></tr></tbody>
+        </table>
+      </div>
+    </div>`
+  return container
+}
+
+describe('fixGridAria', () => {
+  it('strips the treegrid role and the row/column counts from the grid wrapper', () => {
+    const container = buildGrid('a')
+    const wrapper = container.querySelector('.ht-wrapper') as HTMLElement
+
+    fixGridAria(container)
+
+    expect(wrapper.getAttribute('role')).toBeNull()
+    expect(wrapper.getAttribute('aria-rowcount')).toBeNull()
+    expect(wrapper.getAttribute('aria-colcount')).toBeNull()
+    expect(wrapper.getAttribute('aria-multiselectable')).toBeNull()
+  })
+
+  it('drops a presentation role only when the element holds focusable content', () => {
+    const container = buildGrid('b')
+    const interactive = container.querySelector('.wtHolder') as HTMLElement
+    const empty = document.createElement('div')
+    empty.setAttribute('role', 'presentation')
+    empty.innerHTML = '<span>overlay</span>'
+    container.appendChild(empty)
+
+    fixGridAria(container)
+
+    expect(interactive.getAttribute('role')).toBeNull()
+    expect(empty.getAttribute('role')).toBe('presentation')
+  })
+
+  it('gives the table an explicit table role and its headers a column scope', () => {
+    const container = buildGrid('c')
+
+    fixGridAria(container)
+
+    expect(container.querySelector('table')?.getAttribute('role')).toBe('table')
+    expect(container.querySelector('th')?.getAttribute('scope')).toBe('col')
+  })
+
+  it('leaves accessible names to the markup', () => {
+    const container = buildGrid('d')
+    const labelled = document.createElement('button')
+    labelled.setAttribute('aria-label', 'Existing')
+    labelled.textContent = 'Overwrite me'
+    container.appendChild(labelled)
+
+    fixGridAria(container)
+
+    // The grid's own button has a text name already; nothing should invent an
+    // aria-label for it, and an existing one must survive untouched.
+    expect(
+      container.querySelector('table button')?.getAttribute('aria-label')
+    ).toBeNull()
+    expect(labelled.getAttribute('aria-label')).toBe('Existing')
+  })
+
+  it('leaves markup outside the grid container untouched', () => {
+    const container = buildGrid('e')
+    const other = buildGrid('f')
+    document.body.appendChild(container)
+    document.body.appendChild(other)
+    const otherWrapper = other.querySelector('.ht-wrapper') as HTMLElement
+
+    fixGridAria(container)
+
+    expect(otherWrapper.getAttribute('role')).toBe('treegrid')
+    expect(otherWrapper.getAttribute('aria-rowcount')).toBe('-1')
+
+    container.remove()
+    other.remove()
+  })
+
+  it('strips the root element itself, whatever its id or class', () => {
+    const root = document.createElement('div')
+    root.setAttribute('role', 'treegrid')
+    root.setAttribute('aria-rowcount', '-1')
+    root.setAttribute('aria-colcount', '4')
+    root.setAttribute('aria-multiselectable', 'true')
+
+    fixGridAria(root)
+
+    expect(root.getAttribute('role')).toBeNull()
+    expect(root.getAttribute('aria-rowcount')).toBeNull()
+    expect(root.getAttribute('aria-colcount')).toBeNull()
+    expect(root.getAttribute('aria-multiselectable')).toBeNull()
+  })
+
+  it('leaves a treegrid that a cell renderer put inside the grid alone', () => {
+    const container = buildGrid('h')
+    const nested = document.createElement('div')
+    nested.setAttribute('role', 'treegrid')
+    nested.setAttribute('aria-rowcount', '3')
+    container.querySelector('td')?.appendChild(nested)
+
+    fixGridAria(container)
+
+    expect(nested.getAttribute('role')).toBe('treegrid')
+    expect(nested.getAttribute('aria-rowcount')).toBe('3')
+  })
+
+  it('leaves a table a cell renderer owns alone', () => {
+    const container = buildGrid('i')
+    const rendererTable = document.createElement('table')
+    rendererTable.innerHTML = '<thead><tr><th>Inner</th></tr></thead>'
+    container.querySelector('td')?.appendChild(rendererTable)
+
+    fixGridAria(container)
+
+    expect(rendererTable.getAttribute('role')).toBeNull()
+    expect(rendererTable.querySelector('th')?.getAttribute('scope')).toBeNull()
+  })
+
+  it('is idempotent', () => {
+    const container = buildGrid('g')
+
+    fixGridAria(container)
+    const first = container.innerHTML
+    fixGridAria(container)
+
+    expect(container.innerHTML).toBe(first)
+  })
+
+  it('accepts a missing grid, which is the case before Handsontable has mounted', () => {
+    expect(() => fixGridAria(undefined)).not.toThrow()
+  })
+})
+
+describe('fixGridAria against a live grid', () => {
+  it('re-strips the attributes Handsontable writes again on a data load', () => {
+    const container = document.createElement('div')
+    document.body.appendChild(container)
+
+    const hot = new Handsontable(container, {
+      data: [
+        [1, 2],
+        [3, 4]
+      ],
+      height: 200,
+      width: 300,
+      licenseKey: undefined
+    })
+
+    try {
+      // Handsontable marks its own wrapper on create - that is what the pass
+      // removes, and why the pass runs after every render rather than once.
+      expect(hot.rootElement.getAttribute('role')).toBe('treegrid')
+
+      fixGridAria(hot.rootElement)
+      expect(hot.rootElement.getAttribute('role')).toBeNull()
+      expect(hot.rootElement.getAttribute('aria-rowcount')).toBeNull()
+
+      hot.loadData([
+        [5, 6],
+        [7, 8],
+        [9, 10]
+      ])
+      expect(hot.rootElement.getAttribute('aria-rowcount')).toBe('-1')
+
+      fixGridAria(hot.rootElement)
+      expect(hot.rootElement.getAttribute('role')).toBeNull()
+      expect(hot.rootElement.getAttribute('aria-rowcount')).toBeNull()
+    } finally {
+      hot.destroy()
+      container.remove()
+    }
+  })
+})

client/src/app/shared/utils/grid-aria.ts

@@ -0,0 +1,81 @@
+/** Elements that count as focusable/interactive for the presentation-role check. */
+const INTERACTIVE =
+  'button, input, select, textarea, [tabindex], [onclick], [contenteditable]'
+
+/**
+ * Handsontable 18 marks its grid wrapper `role="treegrid"` with
+ * `aria-rowcount="-1"`, and marks the scroll holder `role="presentation"`.
+ * Neither survives an axe audit:
+ *
+ * - a treegrid must own rows (axe: `aria-required-children`, critical), but the
+ *   wrapper's direct children are overlay divs, so the role is a lie;
+ * - `aria-rowcount="-1"` tells assistive tech the total row count is unknown
+ *   when the table has already rendered it;
+ * - `role="presentation"` on the scroll holder that contains the focusable
+ *   cells hides those cells from assistive tech.
+ *
+ * Measured on Handsontable 18 with axe-core: the default markup fails the first
+ * two rules; disabling the tags (`ariaTags: false`) only trades the critical
+ * failure for an `empty-table-header` one and leaves the scroll region
+ * unfocusable. So the attributes are corrected here instead - the wrapper is
+ * demoted to a plain container and the table gets an explicit `role="table"`
+ * with scoped headers.
+ *
+ * The pass is scoped to one grid's container, not the document: each grid owns
+ * its own markup, and a document-wide sweep would fight every other grid on the
+ * page (and every other component's DOM) on each render. It is idempotent, so
+ * calling it after every render and every data change is safe.
+ *
+ * It deliberately does not synthesise `aria-label` from an element's text. The
+ * old document-wide sweep did, which meant a button's accessible name depended
+ * on when the sweep last ran - and one Cypress spec had come to select the
+ * filter modal's submit button by a label that only the sweep ever wrote. An
+ * element's text is already its name; anything else belongs in the template.
+ */
+const collect = (root: HTMLElement, selector: string): Element[] => {
+  const found = Array.from(root.querySelectorAll(selector))
+  return root.matches(selector) ? [root, ...found] : found
+}
+
+export const fixGridAria = (root: HTMLElement | null | undefined): void => {
+  if (!root) return
+
+  // The grid's own wrapper is the root itself - Handsontable marks it with
+  // role/aria attributes, and its id carries no fixed prefix (it is the
+  // container the host component created). Everything else with those
+  // attributes is one of the overlay wrappers Handsontable renders inside it.
+  // Matching on [role="treegrid"] instead would also catch a treegrid a cell
+  // renderer deliberately put inside the grid, which is not ours to change.
+  const wrappers = [
+    root,
+    ...Array.from(root.querySelectorAll('.ht-wrapper, .wtHolder, [id^="ht_"]'))
+  ]
+
+  for (const element of wrappers) {
+    element.removeAttribute('role')
+    element.removeAttribute('aria-rowcount')
+    element.removeAttribute('aria-colcount')
+    element.removeAttribute('aria-multiselectable')
+  }
+
+  for (const element of collect(root, '[role="presentation"]')) {
+    if (element.querySelector(INTERACTIVE)) element.removeAttribute('role')
+  }
+
+  // Handsontable's own tables all carry `htCore` (one per view/clone). Matching
+  // plain `table` would also catch a table a cell renderer owns - the same
+  // reason the wrapper loop does not match on [role="treegrid"].
+  for (const table of collect(root, 'table.htCore')) {
+    if (!table.getAttribute('role')) table.setAttribute('role', 'table')
+
+    // :scope keeps this to the grid's own header row - a renderer's nested
+    // table lives in a body cell and has its own thead.
+    for (const headerCell of Array.from(
+      table.querySelectorAll(':scope > thead > tr > th')
+    )) {
+      if (!headerCell.getAttribute('scope')) {
+        headerCell.setAttribute('scope', 'col')
+      }
+    }
+  }
+}

client/src/app/viewer/viewer.component.ts

@@ -48,6 +48,7 @@ import { Col } from '../shared/dc-validator/models/col.model'
 import { parseLabelsParam } from '../shared/utils/parse-labels-param'
 import { getDisplayColHeaders } from '../shared/utils/display-col-headers'
 import { buildColInfoHtml } from '../shared/utils/col-info-html'
+import { fixGridAria } from '../shared/utils/grid-aria'
 
 @Component({
   selector: 'app-viewer',
@@ -174,9 +175,6 @@ export class ViewerComponent
   public licenceState = this.licenceService.licenceState
   public Infinity = Infinity
 
-  private ariaObserver: MutationObserver | undefined
-  private ariaCheckInterval: any | undefined
-
   public hotTable: HotTableInterface = {
     data: [],
     colHeaders: [],
@@ -1525,165 +1523,18 @@ export class ViewerComponent
 
     // Safely destroy Handsontable instance
     this.cleanupHotInstance()
-
-    // Clean up ARIA accessibility observers
-    if (this.ariaObserver) {
-      this.ariaObserver.disconnect()
-      this.ariaObserver = undefined
-    }
-
-    // Clear ARIA check intervals
-    if (this.ariaCheckInterval) {
-      clearInterval(this.ariaCheckInterval)
-      this.ariaCheckInterval = undefined
-    }
   }
 
   /**
-   * Fixes ARIA accessibility issues in the Handsontable component
-   * This addresses the accessibility report issues with treegrid and presentation roles
+   * Corrects the ARIA markup Handsontable puts on its own grid wrapper - see
+   * shared/utils/grid-aria.ts for what is corrected and why. Scoped to this
+   * grid so it does not touch any other grid or component on the page.
    */
-  private fixAriaAccessibility() {
-    // Use a more aggressive approach to find and fix all ARIA issues
-    const fixAriaIssues = () => {
-      // Specifically target Handsontable wrapper elements that are causing issues
-      const hotWrappers = document.querySelectorAll(
-        '.ht-wrapper, .wtHolder, [id^="ht_"]'
-      )
-
-      hotWrappers.forEach((wrapper) => {
-        // Remove problematic ARIA attributes from Handsontable wrappers
-        wrapper.removeAttribute('role')
-        wrapper.removeAttribute('aria-rowcount')
-        wrapper.removeAttribute('aria-colcount')
-        wrapper.removeAttribute('aria-multiselectable')
-      })
-
-      // Find all elements with problematic ARIA roles in the entire document
-      const allTreegridElements = document.querySelectorAll('[role="treegrid"]')
-      const allPresentationElements = document.querySelectorAll(
-        '[role="presentation"]'
-      )
-
-      // Fix treegrid role issues - remove them completely as they're causing problems
-      allTreegridElements.forEach((element) => {
-        element.removeAttribute('role')
-        element.removeAttribute('aria-rowcount')
-        element.removeAttribute('aria-colcount')
-        element.removeAttribute('aria-multiselectable')
-      })
-
-      // Fix presentation role issues - remove them if they contain interactive elements
-      allPresentationElements.forEach((element) => {
-        const hasInteractiveChildren =
-          element.querySelectorAll(
-            'button, input, select, textarea, [tabindex], [onclick], [contenteditable]'
-          ).length > 0
-        if (hasInteractiveChildren) {
-          element.removeAttribute('role')
-        }
-      })
-
-      // Also fix any elements with aria-rowcount="-1" which is problematic
-      const negativeRowCountElements = document.querySelectorAll(
-        '[aria-rowcount="-1"]'
-      )
-      negativeRowCountElements.forEach((element) => {
-        element.removeAttribute('aria-rowcount')
-      })
-
-      // Ensure proper table structure
-      const tableElements = document.querySelectorAll('table')
-      tableElements.forEach((table) => {
-        if (!table.getAttribute('role')) {
-          table.setAttribute('role', 'table')
-        }
-
-        // Ensure table headers have proper scope
-        const headerCells = table.querySelectorAll('th')
-        headerCells.forEach((th) => {
-          if (!th.getAttribute('scope')) {
-            th.setAttribute('scope', 'col')
-          }
-        })
-      })
-
-      // Add proper ARIA labels to interactive elements
-      const interactiveElements = document.querySelectorAll(
-        'button, input, select, textarea, [contenteditable]'
-      )
-      interactiveElements.forEach((element) => {
-        if (
-          !element.getAttribute('aria-label') &&
-          !element.getAttribute('aria-labelledby')
-        ) {
-          const textContent = element.textContent?.trim()
-          if (textContent) {
-            element.setAttribute('aria-label', textContent)
-          }
-        }
-      })
-    }
-
-    // Run the fix immediately
-    fixAriaIssues()
-
-    // Run it again after a short delay to catch any dynamically created elements
-    setTimeout(fixAriaIssues, 100)
-    setTimeout(fixAriaIssues, 500)
-    setTimeout(fixAriaIssues, 1000)
-    setTimeout(fixAriaIssues, 2000)
-
-    // Set up a periodic check to ensure accessibility fixes are maintained
-    if (!this.ariaCheckInterval) {
-      this.ariaCheckInterval = setInterval(fixAriaIssues, 3000)
-    }
-
-    // Set up a MutationObserver to continuously monitor for new problematic elements
-    if (!this.ariaObserver) {
-      this.ariaObserver = new MutationObserver((mutations) => {
-        let shouldFix = false
-        mutations.forEach((mutation) => {
-          if (
-            mutation.type === 'attributes' &&
-            (mutation.attributeName === 'role' ||
-              mutation.attributeName === 'aria-rowcount')
-          ) {
-            shouldFix = true
-          }
-          if (mutation.type === 'childList') {
-            mutation.addedNodes.forEach((node) => {
-              if (node.nodeType === Node.ELEMENT_NODE) {
-                const element = node as Element
-                if (
-                  element.hasAttribute('role') ||
-                  element.hasAttribute('aria-rowcount')
-                ) {
-                  shouldFix = true
-                }
-              }
-            })
-          }
-        })
-
-        if (shouldFix) {
-          setTimeout(fixAriaIssues, 50)
-        }
-      })
-
-      // Start observing the entire document for changes
-      this.ariaObserver.observe(document.body, {
-        childList: true,
-        subtree: true,
-        attributes: true,
-        attributeFilter: [
-          'role',
-          'aria-rowcount',
-          'aria-colcount',
-          'aria-multiselectable'
-        ]
-      })
-    }
+  private fixAriaAccessibility(): void {
+    fixGridAria(
+      this.hotInstance?.rootElement ??
+        this.hotTableComponent?.hotInstance?.rootElement
+    )
   }
 
   async ngAfterContentInit() {

client/src/styles.scss

@@ -298,7 +298,7 @@ app-editor {
 
 // APP.COMPONENT
 my-app {
-  app-requests-modal {
+  app-sas-logs-modal {
     z-index: 10000;
   }
 
@@ -1566,11 +1566,18 @@ body[cds-theme='light'] {
 }
 // USER.COMPONENT END
 
-// REQUESTS-MODAL.COMPONENT
-app-requests-modal {
+// SAS-LOGS-MODAL.COMPONENT
+app-sas-logs-modal {
   --line-height: 16px;
+  --line-count: 1;
   --line-top: 0px;
 
+  .issue-code {
+    margin-right: 4px;
+    font-family: monospace;
+    font-weight: 600;
+  }
+
   .log-wrapper.saslog {
     position: relative;
     white-space: pre;
@@ -1591,7 +1598,7 @@ app-requests-modal {
   }
 
   .temp-height-setter {
-    height: var(--line-height);
+    height: calc(var(--line-height) * var(--line-count));
   }
 
   .line-position-setter {
@@ -1609,7 +1616,11 @@ app-requests-modal {
   }
 
   .warning-highlight {
-    background-color: rgba(255, 255, 0, 0.2);
+    background-color: rgba(242, 175, 0, 0.25);
+  }
+
+  .note-highlight {
+    background-color: rgba(0, 120, 210, 0.15);
   }
 
   @keyframes fadeOut {
@@ -1656,18 +1667,18 @@ app-requests-modal {
   }
 }
 
-.requests-modal .modal-header--accessible--accessible .close clr-icon {
+.sas-logs-modal .modal-header--accessible--accessible .close clr-icon {
   display: block !important;
 }
 
-.requests-modal .modal-content {
+.sas-logs-modal .modal-content {
   padding: 20px 10px 5px 10px;
 }
 
 /* The SAS log pane is `white-space: pre`, so it never wraps.  The stock `xl`
   width is a fixed 57.6rem, which forced left/right scrolling to read a log
   line. */
-.requests-modal .modal-dialog {
+.sas-logs-modal .modal-dialog {
   width: 90vw;
 }
 

package.json

@@ -1,6 +1,6 @@
 {
   "name": "dcfrontend",
-  "version": "7.20.0",
+  "version": "7.21.0",
   "description": "Data Controller",
   "devDependencies": {
     "@nogoo9/gitleaks": "8.30.1-post.2",
@@ -15,7 +15,6 @@
     "got": "11.8.6"
   },
   "scripts": {
-    "install": "cd client && npm i && cd ../sas && npm i",
     "build-frontend": "cd client && npm run build",
     "lint": "cd client && npm run lint",
     "lint:fix": "cd client && npm run lint:fix",

sas/.sasjslint

@@ -4,13 +4,47 @@
   "hasDoxygenHeader": true,
   "noSpacesInFileNames": true,
   "lowerCaseFileNames": true,
-  "maxLineLength": 180,
+  "maxLineLength": 80,
   "noTabIndentation": true,
   "indentationMultiple": 2,
   "hasMacroNameInMend": true,
   "noNestedMacros": true,
   "hasMacroParentheses": true,
   "strictMacroDefinition": true,
-  "ignoredMacros": ["webout", "mpeinit", "mpeterm"],
-  "ignoreList":["sasjs/scratch"]
+  "noSingleAsteriskComments": true,
+  "noUnusedLibnames": true,
+  "lineEndings": "lf",
+  "ignoredMacros": [
+    "webout",
+    "mpeinit",
+    "mpeterm"
+  ],
+  "ignoreList": [
+    "sasjs/scratch"
+  ],
+  "severityLevel": {
+    "hasDoxygenHeader": "error",
+    "hasMacroNameInMend": "error",
+    "hasMacroParentheses": "error",
+    "indentationMultiple": "error",
+    "lineEndings": "error",
+    "lowerCaseFileNames": "error",
+    "lowerCaseMacroNames": "error",
+    "maxLineLength": "error",
+    "minMacroNameLength": "error",
+    "noBannedStrings": "error",
+    "noGremlins": "error",
+    "noNestedCommentsInHeader": "error",
+    "noNestedMacros": "error",
+    "noSingleAsteriskComments": "error",
+    "noSpacesInFileNames": "error",
+    "noTabIndentation": "error",
+    "noTrailingSpaces": "error",
+    "noUndeclaredMacros": "error",
+    "noUnusedLibnames": "error",
+    "noUnusedMacros": "error",
+    "strictMacroDefinition": "error",
+    "noMacroIfInOpenCode": "error"
+  },
+  "noMacroIfInOpenCode": true
 }

sas/deploy/makeDataServer.json

@@ -1,7 +1,7 @@
 {
   "fromjs": [
     {
-      "ADMIN": "AllUsers",
+      "ADMIN": "all-users",
       "DCPATH": "/tmp/dcdata",
       "OUTFMT": "JSON"
     }

sas/mocks/sasjs/services/hooks/mpe_derived_postedit.js

@@ -13,10 +13,12 @@
 // The rules mirror services/hooks/mpe_derived_postedit.sas, in its order:
 // normalise the incoming rows, reject a duplicate name and a bad field, then
 // check the dependency graph that the resulting configuration would have (the
-// incoming rows plus the current rows they do not replace).  As on the server,
-// the collision between VAR_NM and a physical column is left to the read-time
-// engine, which runs with the target library assigned - the hook does not have
-// it here either.
+// incoming rows plus the current rows they do not replace).  The field rules
+// are core's mp_derivedvarcheck, which the server hook calls and the read-time
+// engine runs too - this file spells them out a second time, so a change there
+// has to be mirrored here.  As on the server, the collision between VAR_NM and
+// a physical column is left to the read-time engine, which runs with the target
+// library assigned - the hook does not have it here either.
 // ─────────────────────────────────────────────────────────────────────────────
 
 ;(function mpeDerivedPostedit() {
@@ -83,9 +85,45 @@
         seen.add(key)
     }
 
-    // ─── Field level checks, the rules the engine applies at read time ─────────
-    // C0 controls and DEL: what a single-line label cannot carry
+    // ─── Field level checks ────────────────────────────────────────────────────
+    // Two passes, in the order the SAS hook runs them.  The first is the fields
+    // this hook owns - the control table key, which the loader normalises but
+    // does not require, and ACTIVE, the picker pair the EDIT screen offers.
+    for (const row of rows) {
+        if (isDeleted(row)) continue
+        const name = field(row, 'VAR_NM')
+        let reason = ''
+        if (!field(row, 'LIBREF') || !field(row, 'DSN'))
+            reason = 'LIBREF and DSN are required'
+        // ACTIVE is a picker in the EDIT screen (HARDSELECT over the
+        // MPE_SELECTBOX pair) - the same pair the hook enforces
+        else if (!['0', '1'].includes(String(colVal(row, 'ACTIVE')).trim()))
+            reason = 'ACTIVE must be 1 or 0'
+        if (reason) return reject(`MPE_DERIVED: ${name}: ${reason}`)
+    }
+
+    // The second pass is the rules for the derived variable itself, which are
+    // core's (mp_derivedvarcheck): the engine runs the same macro at read time,
+    // so a definition is measured against one rule list wherever it is checked.
+
+    // C0 controls and DEL: what a label cannot carry once a line break has been
+    // flattened to a space, which is what the engine does with it
     const CONTROL_CHARS = /[\u0000-\u001F\u007F]/
+    const flatten = (v) => v.replace(/[\r\n]+/g, ' ')
+    const AUTOMATIC_VARS = ['_N_', '_ERROR_', '_IORC_']
+
+    // the capabilities refused by name - a function name cannot be assembled at
+    // run time, so naming them is a control the expression cannot evade
+    const DENY_FUNCS = new RegExp(
+        '\\b(DOSUBL|JAVAOBJ|JAVAMETA|SYMGET|SYMPUTX?|SYSGET|GETOPTION' +
+            '|PATHNAME|FILEEXIST|FOPEN|FREAD|FGET|FCLOSE' +
+            '|DOPEN|DREAD|DNUM|MOPEN|SLEEP' +
+            '|FILENAME|FDELETE|FCOPY|DCREATE|DLCREATE)\\s*\\('
+    )
+
+    // quoted literals are data, not calls
+    const unquoted = (v) =>
+        v.replace(/'([^']|'')*'/g, ' ').replace(/"([^"]|"")*"/g, ' ')
 
     // true when the value carries a character outside the allowed set
     const hasInvalidChar = (value, chars) =>
@@ -93,8 +131,6 @@
 
     for (const row of rows) {
         if (isDeleted(row)) continue
-        const libref = field(row, 'LIBREF')
-        const dsn = field(row, 'DSN')
         const name = field(row, 'VAR_NM')
         const type = field(row, 'VAR_TYPE')
         const len = Number(colVal(row, 'VAR_LEN'))
@@ -103,19 +139,16 @@
         const logic = strip(colVal(row, 'VAR_LOGIC'))
 
         let reason = ''
-        if (!libref || !dsn) reason = 'LIBREF and DSN are required'
-        else if (!name) reason = 'VAR_NM is required'
+        if (!name) reason = 'VAR_NM is required'
         else if (hasInvalidChar(name, NAME_CHARS))
             reason = 'VAR_NM must be a valid SAS name'
         else if ('0123456789'.includes(name[0]))
             reason = 'VAR_NM must not start with a digit'
+        else if (AUTOMATIC_VARS.includes(name))
+            reason = 'VAR_NM is an automatic variable of the data step'
         else if (name === DELETE_MARK) reason = 'VAR_NM is reserved'
         else if (type !== 'C' && type !== 'N')
             reason = 'VAR_TYPE must be C or N'
-        // ACTIVE is a picker in the EDIT screen (HARDSELECT over the
-        // MPE_SELECTBOX pair) - the same pair mpe_derived_check.sas enforces
-        else if (!['0', '1'].includes(String(colVal(row, 'ACTIVE')).trim()))
-            reason = 'ACTIVE must be 1 or 0'
         else if (!Number.isInteger(len) || len < 1)
             reason = 'VAR_LEN must be a positive integer'
         else if (type === 'C' && len > 32767)
@@ -132,20 +165,24 @@
             reason = 'VAR_FMT must be a format name, without ; & % or /* */'
         else if (hasInvalidChar(fmt, FMT_CHARS))
             reason = 'VAR_FMT must be a format name'
-        // free text, in any script - what is refused is anything that cannot be
-        // a single-line column label, and a value the engine would cut to 256
-        else if (CONTROL_CHARS.test(label))
-            reason =
-                'VAR_LABEL must be plain text - no line breaks or control characters'
+        // free text, in any script - what is refused is a control character,
+        // and a value the engine would cut to 256
+        else if (CONTROL_CHARS.test(flatten(label)))
+            reason = 'VAR_LABEL must be plain text - no control characters'
         else if ([...label].length > 256)
             reason = 'VAR_LABEL exceeds 256 characters'
         else if (!logic) reason = 'VAR_LOGIC is required'
+        else if (logic.length > 4000)
+            reason = 'VAR_LOGIC exceeds 4000 characters'
         else if (logic.includes(';'))
             reason = 'VAR_LOGIC must be a single expression - remove the ;'
         else if (logic.includes('&') || logic.includes('%'))
             reason = 'VAR_LOGIC must not contain macro triggers'
         else if (logic.includes('/*') || logic.includes('*/'))
             reason = 'VAR_LOGIC must not contain comments'
+        else if (DENY_FUNCS.test(unquoted(logic).toUpperCase()))
+            reason =
+                'VAR_LOGIC may not call a function that submits code or reaches the file system, the environment or the macro symbol table'
 
         if (reason) return reject(`MPE_DERIVED: ${name}: ${reason}`)
     }

sas/mocks/sasjs/services/public/getddl.js

@@ -1,7 +1,20 @@
 let mimeType = `application/octet-stream`
 let filename = `dc-download`
 
-if (libref) filename = `${libref}.ddl`
+/* typeof, not a truthiness test: if the service is called without a ds in scope
+   the read itself throws and the fallback never runs */
+if (
+  typeof libref !== 'undefined' &&
+  libref &&
+  typeof ds !== 'undefined' &&
+  ds
+)
+  filename = `${libref}_${ds}.ddl`
+else if (typeof libref !== 'undefined' && libref) {
+  /* production aborts when ds is missing (getddl.sas:43-45), so the mock does
+     too: a client that dropped ds would otherwise stay green here */
+  throw new Error(`Dataset ${libref}..${ds} was not found`)
+}
 
 fs.writeFile(_SASJS_WEBOUT_HEADERS, `Content-type: ${mimeType} \n Content-Disposition: attachment;filename=\"${filename}\"`, function (err) {
   if (err) throw err;

sas/package-lock.json

@@ -6,8 +6,8 @@
     "": {
       "name": "dc-sas",
       "dependencies": {
-        "@sasjs/cli": "6.2.0",
-        "@sasjs/core": "6.1.0"
+        "@sasjs/cli": "6.5.0",
+        "@sasjs/core": "7.0.3"
       }
     },
     "node_modules/@asamuzakjp/css-color": {
@@ -165,14 +165,15 @@
       }
     },
     "node_modules/@sasjs/cli": {
-      "version": "6.2.0",
-      "resolved": "https://registry.npmjs.org/@sasjs/cli/-/cli-6.2.0.tgz",
-      "integrity": "sha512-zovAOCmyPqcm+gooAZgwvgGH8n5cas3sY0lA80+hyFoFtmZ8VjW/bh19LM8OVwheT9dBCRCx4fnHwjy5XGpzvQ==",
+      "version": "6.5.0",
+      "resolved": "https://registry.npmjs.org/@sasjs/cli/-/cli-6.5.0.tgz",
+      "integrity": "sha512-1I7B0iKd+16ByIGPToGfODU3S1bxO/3VCGpmu5alql/HlXdxu75djdpuzCovbsSeWm+M4FrsrBKDZTc9hhBrLA==",
       "license": "ISC",
       "dependencies": {
         "@sasjs/adapter": "^5.1.1",
-        "@sasjs/core": "^6.0.0",
-        "@sasjs/utils": "^4.1.0",
+        "@sasjs/core": "^6.1.1",
+        "@sasjs/lint": "4.9.0",
+        "@sasjs/utils": "^4.1.2",
         "axios": "1.20.0",
         "chalk": "4.1.2",
         "dotenv": "18.0.8",
@@ -194,15 +195,18 @@
       },
       "engines": {
         "node": ">=22"
-      },
-      "peerDependencies": {
-        "@sasjs/lint": "^4.3.0"
       }
     },
+    "node_modules/@sasjs/cli/node_modules/@sasjs/core": {
+      "version": "6.1.4",
+      "resolved": "https://registry.npmjs.org/@sasjs/core/-/core-6.1.4.tgz",
+      "integrity": "sha512-vIF2LIftxPWg4CvO3i5raEfrHFHY/6S+MqVGpjRR71dgjLRJEYNyPJgGqD6BCZwHv5V6/2nDnBi/BIIMiLXfzA==",
+      "license": "MIT"
+    },
     "node_modules/@sasjs/cli/node_modules/@sasjs/utils": {
-      "version": "4.1.0",
-      "resolved": "https://registry.npmjs.org/@sasjs/utils/-/utils-4.1.0.tgz",
-      "integrity": "sha512-aC5pOugg6/2jKQR7ZaYsLtbsFy/iVacsrN3/Kzh6BouJUszuVrr13HMrQUSKcWL+rJzxYof7ymmmCl+vE6YwAw==",
+      "version": "4.1.2",
+      "resolved": "https://registry.npmjs.org/@sasjs/utils/-/utils-4.1.2.tgz",
+      "integrity": "sha512-5QLBVlCqiiDW/GoxiHUCyT+8GJjJA/S/TX0DS/t4O4HC1Im/QpQzxlS86QkjLAQBBHovYPyh+v4pgdh+IYwaEQ==",
       "license": "ISC",
       "dependencies": {
         "@fast-csv/format": "5.0.7",
@@ -235,62 +239,24 @@
       }
     },
     "node_modules/@sasjs/core": {
-      "version": "6.1.0",
-      "resolved": "https://registry.npmjs.org/@sasjs/core/-/core-6.1.0.tgz",
-      "integrity": "sha512-U0Ekq/TXVMquYV+AG6urBQSUgqUvV37yduufy68QQVgDj3PJiLUJDcOXKOGbhWGD+Nv877VuPCviGYqrqbjXvg==",
+      "version": "7.0.3",
+      "resolved": "https://registry.npmjs.org/@sasjs/core/-/core-7.0.3.tgz",
+      "integrity": "sha512-x67TJOXCKvdtlYoHGKNnVWz76QFwRpcp/+zpzVqjll6prwHTUAQt8SBjCGmC3/fhadBtylGGj37rgC8Lz8fSww==",
       "license": "MIT"
     },
     "node_modules/@sasjs/lint": {
-      "version": "4.3.0",
-      "resolved": "https://registry.npmjs.org/@sasjs/lint/-/lint-4.3.0.tgz",
-      "integrity": "sha512-65cjy4IjA4PgSlicOruF0DI9N2HO3LJqtgphsZCg8g7N/StE2FWLJLJbD1N4OHxN6htrUfj5SEawJJiHo5T9Yw==",
+      "version": "4.9.0",
+      "resolved": "https://registry.npmjs.org/@sasjs/lint/-/lint-4.9.0.tgz",
+      "integrity": "sha512-QEBo19bWUdKIaE62Pk8eb35Q9EjHlifyOmbH+voItX6muCqHTX88SOzLD4MYtFVuagm/cnGplIVcKvUZzO+bSw==",
       "license": "MIT",
-      "peer": true,
       "dependencies": {
-        "@sasjs/utils": "4.0.0",
+        "@sasjs/utils": "^4.0.0",
         "ignore": "7.0.11"
       },
       "engines": {
         "node": ">=22"
       }
     },
-    "node_modules/@sasjs/lint/node_modules/@sasjs/utils": {
-      "version": "4.0.0",
-      "resolved": "https://registry.npmjs.org/@sasjs/utils/-/utils-4.0.0.tgz",
-      "integrity": "sha512-lARoCnRM4gSVf8ShoP5elHi7W28K/14MK037dQ0ABgMlP3+r6OuhAZYwK/a70tED47tKxyIoCAKLdP6jrG0ewQ==",
-      "license": "ISC",
-      "peer": true,
-      "dependencies": {
-        "@fast-csv/format": "5.0.7",
-        "@types/fs-extra": "11.0.4",
-        "@types/prompts": "2.4.9",
-        "chalk": "4.1.1",
-        "consola": "3.4.2",
-        "fs-extra": "11.4.1",
-        "jwt-decode": "4.0.0",
-        "prompts": "2.4.2"
-      },
-      "engines": {
-        "node": ">=22"
-      }
-    },
-    "node_modules/@sasjs/lint/node_modules/chalk": {
-      "version": "4.1.1",
-      "resolved": "https://registry.npmjs.org/chalk/-/chalk-4.1.1.tgz",
-      "integrity": "sha512-diHzdDKxcU+bAsUboHLPEDQiw0qEe0qd7SYUn3HgcFlWgbDcfLGswOHYeGrHKzG9z6UYf01d9VFMfZxPM1xZSg==",
-      "license": "MIT",
-      "peer": true,
-      "dependencies": {
-        "ansi-styles": "^4.1.0",
-        "supports-color": "^7.1.0"
-      },
-      "engines": {
-        "node": ">=10"
-      },
-      "funding": {
-        "url": "https://github.com/chalk/chalk?sponsor=1"
-      }
-    },
     "node_modules/@sasjs/utils": {
       "version": "4.0.1",
       "resolved": "https://registry.npmjs.org/@sasjs/utils/-/utils-4.0.1.tgz",
@@ -825,9 +791,9 @@
       "license": "MIT"
     },
     "node_modules/follow-redirects": {
-      "version": "1.16.1",
-      "resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.16.1.tgz",
-      "integrity": "sha512-FNvFGzoMLWmE6Yj9spb/zjd7yiNCHiAW9/Tg9CXrQ8wuu32HtlJOwWO11OJafl5FfY3DxTdQ0vj42zU1kvv5jg==",
+      "version": "1.16.2",
+      "resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.16.2.tgz",
+      "integrity": "sha512-/kVP2U8z596Ia0gwFrwyIeo74ta5aVmSeU1FfAUMAmcEP5EP7zydmag3SLiN64NaRKef3lxsq7i/g7ltJQDGow==",
       "funding": [
         {
           "type": "individual",
@@ -1106,7 +1072,6 @@
       "resolved": "https://registry.npmjs.org/ignore/-/ignore-7.0.11.tgz",
       "integrity": "sha512-YChdK5txDjwGUvgR7oCJcLGkwi3LDh8Zx8tS7ndYciLEg/oOCL26VUBSUEtgS7EjiybcBqFxh5j0rAdDX7/bbg==",
       "license": "MIT",
-      "peer": true,
       "engines": {
         "node": ">= 4"
       }

sas/package.json

@@ -3,6 +3,7 @@
   "description": "dc-sas",
   "scripts": {
     "cpfavicon": "cp sasjs/utils/favicon.ico ../client/dist/favicon.ico",
+    "lint:sas": "sasjs lint",
     "build": "cd ../client && npm run build",
     "deploy:server": "rsync -avhe ssh ../client/dist/* --delete allbow@sas.4gl.io:/opt/sas/sas9/config/Lev1/Web/WebServer/htdocs/allbow/server",
     "sas9": "rsync -avhe ssh ./dist/* --delete ${npm_config_account}@sas.4gl.io:/opt/sas/sas9/config/Lev1/Web/WebServer/htdocs/${npm_config_account}/dc",
@@ -33,8 +34,8 @@
   },
   "private": true,
   "dependencies": {
-    "@sasjs/cli": "6.2.0",
-    "@sasjs/core": "6.1.0"
+    "@sasjs/cli": "6.5.0",
+    "@sasjs/core": "7.0.3"
   },
   "overrides": {
     "nanoid": "3.3.18"

sas/sasjs/db/migrations/20200121_mpe_groups.sas

@@ -23,4 +23,4 @@ insert into &syslast set
     ,group_name="DC Demo Group"
     ,group_desc="Custom Group for Data Controller Purposes"
     ,user_name="mobeen@saspw"
-    ,tx_to='31DEC5999:23:59:59'dt;
+    ,tx_to='31DEC5999:23:59:59'dt;

sas/sasjs/db/migrations/20200208_mpe_config.sas

@@ -18,4 +18,4 @@ update DC.MPE_CONFIG
 
 /*
 proc sql; drop table backup;
-*/
+*/

sas/sasjs/db/migrations/20200224_mpe_requests.sas

@@ -15,4 +15,4 @@ proc sql;
   );
 
 /* ALSO - make sure there is a DIRECT libname assigned in the
-data_controller_settings stp */
+data_controller_settings stp */

sas/sasjs/db/migrations/20200319_dd_validations.sas

@@ -29,4 +29,4 @@ create table DC.mpe_datadictionary
     DD_SENSITIVITY char(64),
     TX_TO float not null format=datetime19.,
     constraint pk
-        primary key(tx_from, dd_type,dd_source));
+        primary key(tx_from, dd_type,dd_source));

sas/sasjs/db/migrations/20200428_datacatalog.sas

@@ -71,4 +71,4 @@ create table dc.mpe_datacatalog_vars(
     modate num format=DATETIME. label='Date Modified',
     nobs num label='Number of Physical (Actual, inc. deleted) Observations',
   constraint pk
-      primary key(libref,dsn,tx_to));
+      primary key(libref,dsn,tx_to));

sas/sasjs/db/migrations/20210211_mpe_review.sas

@@ -13,4 +13,4 @@ quit;
 proc sql;
 alter table dc.mpe_config add var_active num;
 
-update dc.mpe_config set var_active=1;
+update dc.mpe_config set var_active=1;

sas/sasjs/db/migrations/20210802_v3.13_release.sas

@@ -28,4 +28,4 @@ insert into &dclib..MPE_VALIDATIONS set tx_from=%sysfunc(datetime())
   ,base_col="DSN";
   ,rule_value="services/validations/mpe_tables.dsn"
   ,rule_active=1
-  ,rule_type='SOFTSELECT_HOOK';
+  ,rule_type='SOFTSELECT_HOOK';

sas/sasjs/db/migrations/20220615_v5.0_release.sas

@@ -5,9 +5,6 @@
   BREAKING CHANGE - deprecate MPE_APPROVALS and MPE_REVIEW (now MPE_SUBMIT does
   both)
 
-  <h4> SAS Macros </h4>
-  @li mm_assigndirectlib.sas
-
 **/
 
 %let dclib=YOURDCLIB;

sas/sasjs/db/migrations/20230115_v6.5_release.sas

@@ -80,4 +80,4 @@ create table &dclib..mpe_xlmap_info(
       ,loadtype='UPDATE'
       ,buskey='LOAD_REF XLMAP_ID XLMAP_RANGE_ID ROW_NO COL_NO'
       ,notes='Docs: https://docs.datacontroller.io/complex-excel-uploads'
-    ;
+    ;

sas/sasjs/db/migrations/20230115_v6.8.2_release.sas

@@ -21,4 +21,4 @@ insert into &dclib..mpe_config set
     ,var_value="YES"
     ,var_active=1
     ,var_desc='Setting to NO will prevent each request being logged to the'
-      !!' MPE_REQUESTS table  Default=YES.';
+      !!' MPE_REQUESTS table  Default=YES.';

sas/sasjs/db/migrations/20250611_v7.0_release.sas

@@ -68,4 +68,4 @@ create table &dclib..mpe_datastatus_OBJS(
     modified num format=DATETIME. label='Date Modified',
     level num label='Library Concatenation Level',
   constraint pk_mpe_datastatus_OBJS
-      primary key(libref,memname,objname,objtype,tx_to));
+      primary key(libref,memname,objname,objtype,tx_to));

sas/sasjs/db/migrations/20261003_row_and_cell_limits.sas

@@ -23,9 +23,10 @@
   only for options that are not present at all - an existing install carries
   its own rows, so it keeps 500 / 100 until this is run.
 
-  Each block below uses %do/%end on BOTH branches.  A single-statement
-  %then/%else in open code (outside a %macro) is rejected by the Viya compute
-  engine with "Expected %DO not found", so it must not be shortened back.
+  Each block below sits inside its own macro, with %do/%end on both branches.
+  A single-statement branch is rejected by the Viya compute engine with
+  "Expected %DO not found", so it must not be shortened back, and the macro
+  wrapper is what keeps the conditional out of open code.
 
 **/
 
@@ -47,6 +48,7 @@ proc sql noprint;
     where var_name='DC_MAXCELLS_WEBEDIT' and var_active=0 and &now lt tx_to;
 quit;
 
+%macro apply_view_limit();
 %if &off_view=0 %then %do;
   proc sql noprint;
     update &dclib..mpe_config
@@ -67,7 +69,10 @@ quit;
 %else %do;
   %put NOTE: DC_MAXOBS_WEBVIEW is switched off (var_active=0) - not changed;
 %end;
+%mend apply_view_limit;
+%apply_view_limit()
 
+%macro apply_edit_limit();
 %if &off_edit=0 %then %do;
   proc sql noprint;
     update &dclib..mpe_config
@@ -90,7 +95,10 @@ quit;
 %else %do;
   %put NOTE: DC_MAXOBS_WEBEDIT is switched off (var_active=0) - not changed;
 %end;
+%mend apply_edit_limit;
+%apply_edit_limit()
 
+%macro apply_cell_limit();
 %if &off_cells=0 %then %do;
   proc sql noprint;
     update &dclib..mpe_config
@@ -98,7 +106,7 @@ quit;
       where var_name='DC_MAXCELLS_WEBEDIT' and var_active=1 and &now lt tx_to;
 
     /* Why the cell limit exists, kept out of var_desc - MPE_CONFIG.VAR_DESC
-      is varchar(300), and a truncation WARNING aborts the build.  A wide
+      is varchar(300), and a truncation aborts the build.  A wide
       table is expensive long before it is tall - 250 rows of a 1000 column
       table is a quarter of a million cells. */
     insert into &dclib..mpe_config
@@ -117,3 +125,5 @@ quit;
 %else %do;
   %put NOTE: DC_MAXCELLS_WEBEDIT is switched off (var_active=0) - not changed;
 %end;
+%mend apply_cell_limit;
+%apply_cell_limit()

sas/sasjs/db/migrations/20261019_v7.19_release.sas

@@ -10,6 +10,8 @@
   MPE_DERIVED rows added before this script runs will be lost when the table
   is created, so run it before configuring any derived variables.
 
+  @sasjslint {"maxLineLength": 99}
+
 **/
 
 %let dclib=YOURDCLIB;

sas/sasjs/macros/bitemporal_dataloader.sas

@@ -98,6 +98,7 @@ Areas for optimisation
   @li mf_fmtdttm.sas
   @li mf_getattrn.sas
   @li mf_getengine.sas
+  @li mf_getquotedstr.sas
   @li mf_getschema.sas
   @li mf_getuniquefileref.sas
   @li mf_getuniquename.sas
@@ -122,6 +123,8 @@ Areas for optimisation
   @warning multitemporal loads (bitemporal for multiple points in business time)
     are in experimental stage
 
+  @sasjslint {"noBannedStrings": false, "maxLineLength": 87}
+
 **/
 
 %macro bitemporal_dataloader(
@@ -1449,7 +1452,7 @@ run;
     data work.mp_storediffs;
       set work.mp_storediffs;
       if MOVE_TYPE="M" and IS_PK=0 and IS_DIFF=0 then delete;
-      * putlog load_ref= libref= dsn= key_hash= tgtvar_nm=;
+      /* putlog load_ref= libref= dsn= key_hash= tgtvar_nm=; */
     run;
     proc append base=&outds_audit data=work.mp_storediffs;
     run;

sas/sasjs/macros/bitemporal_dataloader.test.2.sas

@@ -16,6 +16,7 @@
 
 
   <h4> SAS Macros </h4>
+  @li bitemporal_closeouts.sas
   @li bitemporal_dataloader.sas
   @li dc_assignlib.sas
   @li mf_getattrn.sas
@@ -1349,44 +1350,6 @@ run;
   proc sql; insert into results set test="TEST&testnum",result="&result";
 %end;
 
-/* test 30 - testing a load to OLEDB (should use SQL server temp table) */
-%if (&test=ALL or &test=30 ) and 0 %then %do; /* skip until have a db */
-  %let testnum=30;
-
-%assign_direct_lib(libref=XXX);
-  data appendtable&testnum ;
-    set XXX.temp_rk_table
-    (where=(tx_from le %sysfunc(datetime()) lt tx_to));
-    some_num+1;
-    output;
-    some_id=put(ranuni(0)*10000000,8.);output;
-    stop;
-  run;
-  proc sql noprint;
-  select count(*) into:testnobs&testnum from web.temp_rk_table;
-
-  %bitemporal_dataloader(dclib=work,PK=some_rk
-      ,append_dsn=appendtable&testnum.
-      ,ETLSOURCE=bitemporal_tester test&testnum override
-      ,tech_from=tx_from
-      ,tech_to=tx_to
-      ,base_lib=web
-      ,base_dsn=temp_rk_table
-      ,LOG=YES
-      ,rk_underlying=some_id
-      ,LOADTARGET=YES
-      ,LOADTYPE=TXTEMPORAL
-  );
-
-
-  %let result=;
-  proc sql noprint;
-  select count(*) into:testnobs2&testnum from web.temp_rk_table;
-  %if (&&testnobs2&testnum =%eval(&&testnobs&testnum+2)) %then %let result=PASS;
-  %else %let result=FAIL;
-  proc sql; insert into results set test="TEST&testnum",result="&result";
-%end;
-
 /* test 31 - bitemporal closeouts with embedded speechmarks in PK */
 %let testnum=31;
 %if &test=ALL or &test=&testnum %then %do;

sas/sasjs/macros/mpe_accesscheck.test.sas

@@ -5,7 +5,6 @@
 
 
   <h4> SAS Macros </h4>
-  @li mf_getuniquename.sas
   @li mf_getuser.sas
   @li mp_assertdsobs.sas
   @li mpe_getgroups.sas

sas/sasjs/macros/mpe_alerts.sas

@@ -10,7 +10,6 @@
 
   <h4> SAS Macros </h4>
   @li mf_getattrn.sas
-  @li mf_existds.sas
   @li mf_getuser.sas
 
   @version 9.4
@@ -227,4 +226,4 @@ run;
 
 filename __out clear;
 
-%mend mpe_alerts ;
+%mend mpe_alerts ;

sas/sasjs/macros/mpe_assert_abort.sas

@@ -1,92 +1,83 @@
 /**
   @file
   @brief Asserts that a service returned a sasjsAbort object
-  @details Reads the child's response as text and records a PASS when it carries
-  a sasjsAbort object, which is what mp_abort leaves in the webout when a service
-  rejects its input.  When msg is supplied the MSG value inside that object must
-  match it, so the assertion proves the service aborted for the expected reason
-  rather than for any reason at all.
+  @details Reads the child's status row and records a PASS when the child was
+  canceled, which is what happens when a service rejects its input and calls
+  mp_abort.  When msg is supplied the ERRMSG value must match it, so the
+  assertion proves the service aborted for the expected reason rather than for
+  any reason at all.
 
-  The response is scanned a line at a time and only the first MSG that follows
-  the sasjsAbort marker is taken: a single 32767 buffer both truncated longer
-  responses and, with a leading `.*`, returned the last MSG in the text rather
-  than the abort's own.
+  The message comes from the status row rather than from the response text: core
+  publishes STATE and ERRMSG per call, ERRMSG holds the whole message including
+  quotes and anything longer than a single record, and reading it needs no
+  parsing of the response.
 
-  An unassigned outref (a child that died before writing a response) records a
-  FAIL rather than letting the data step fail on an unassigned fileref.
+  Two things are handled here rather than in core:
+
+  - mp_abort appends cats(msg,'\n\nLog Extract:\n',logmsg) to the message, so
+    the tail is cut at the two newlines that precede the marker.  A message that
+    merely mentions "Log Extract:" is not preceded by them and survives.
+  - a platform that reports no job state at all (SAS 9 PROC STP) cannot show the
+    abort this way; the row reads 'unknown' and the assertion fails rather than
+    guessing.
 
   Usage:
 
       %mpe_execute(&_program,
         inputfiles=&f1:sascontroltable,
-        outref=web1,
         outstatus=work.ab1,
         failfast=0
       )
       %mpe_assert_abort(
         desc=An invalid libds aborts the service,
-        outref=web1,
+        outstatus=work.ab1,
         msg=Invalid libds
       )
 
   @param [in] desc The description recorded in the test results
-  @param [in] outref The fileref given to %mx_execute, holding the response
-  @param [in] msg= (0) Expected MSG value in the sasjsAbort object.  0 accepts
-    any abort message.
+  @param [in] outstatus The dataset given to %mpe_execute, holding STATE and
+    ERRMSG for the child call
+  @param [in] msg= (0) Expected ERRMSG value.  0 accepts any abort message.
   @param [out] outds= (work.test_results) Results dataset
 
   <h4> SAS Macros </h4>
   @li mp_assert.sas
 
+  @sasjslint {"maxLineLength": 94}
+
 **/
 
-%macro mpe_assert_abort(desc, outref, msg=0, outds=work.test_results);
+%macro mpe_assert_abort(desc, outstatus, msg=0, outds=work.test_results);
 %local ok seen abortseen;
 %let ok=0;
 %let seen=;
 %let abortseen=0;
 
-%if %sysfunc(fileref(&outref))>0 %then %do;
-  %put %str(WARN)ING: &sysmacroname: fileref &outref is not assigned - the child left no response to read;
-%end;
-%else %if %sysfunc(fileexist(%sysfunc(pathname(&outref)))) ne 1 %then %do;
-  /* the fileref exists but nothing was written to it: a child that aborted
-    before it produced a response.  Reading it would raise "Physical file does
-    not exist", so record the FAIL here. */
-  %put %str(WARN)ING: &sysmacroname: &outref has no response written to it - the child produced nothing;
+%if %sysfunc(exist(&outstatus)) ne 1 %then %do;
+  %put %str(WARN)ING: &sysmacroname: &outstatus does not exist - the child left no status row;
 %end;
 %else %do;
-data _null_;
-  length line $32767 got $256;
-  retain prxid 0 inabort 0 found 0 abortseen 0;
-  if _n_=1 then prxid=prxparse('/"MSG" *: *"([^"]*)"/');
-  infile &outref end=eof lrecl=32767;
-  do while(not eof);
-    input;
-    line=_infile_;
-    if index(line,'sasjsAbort')>0 then do;
-      abortseen=1;
-      inabort=1;
-    end;
-    /* first MSG at or after the abort marker - stops at the first found */
-    if inabort=1 and found=0 then do;
-      if prxmatch(prxid,line)>0 then do;
-        got=prxposn(prxid,1,line);
-        found=1;
-      end;
-    end;
-  end;
-  call symputx('abortseen',abortseen,'l');
-  if found=1 then call symputx('seen',got,'l');
-  stop;
-run;
+  data _null_;
+    length got $32767;
+    set &outstatus;
+    if state='canceled' then call symputx('abortseen',1,'l');
+    got=errmsg;
+    /* mp_abort appends cats(msg,'\n\nLog Extract:\n',logmsg).  Those newlines
+      reach here as characters, as an escaped pair, or as a doubled pair - which
+      one depends on the producer - so all three are tried */
+    cut=index(got,'0A0A'x||'Log Extract:');
+    if cut=0 then cut=index(got,'\n\nLog Extract:');
+    if cut=0 then cut=index(got,'\\n\\nLog Extract:');
+    if cut>1 then got=substr(got,1,cut-1);
+    call symputx('seen',got,'l');
+  run;
 %end;
 
 %if &abortseen=1 %then %do;
   %if "%superq(msg)"="0" %then %let ok=1;
-  %else %if "&seen"="%superq(msg)" %then %let ok=1;
+  %else %if "%superq(seen)"="%superq(msg)" %then %let ok=1;
 %end;
-%if "&seen" ne "" %then %put NOTE: &sysmacroname: sasjsAbort MSG=&seen;
+%if "%superq(seen)" ne "" %then %put NOTE: &sysmacroname: sasjsAbort MSG=&seen;
 
 %mp_assert(
   iftrue=(&ok=1),

sas/sasjs/macros/mpe_assert_abort.test.sas

@@ -2,25 +2,18 @@
   @file
   @brief testing the mpe_assert_abort macro
 
-  @details The macro decides PASS/FAIL by scanning the child's response text, so
-  the cases that matter can be built as files instead of services: an abort with
-  the expected message, an abort with a different one, no abort at all, and a
-  fileref the child never wrote to.
+  @details The macro decides PASS/FAIL from the child's status row - STATE says
+  whether the child was canceled, ERRMSG holds the reason - so the cases can be
+  built as datasets instead of services: an abort with the expected message, an
+  abort with a different one, a completed call, a status dataset that does not
+  exist, a message past the old 256 character width, quotes, backslashes, and
+  the log extract that mp_abort appends to the message.
 
   Each case asserts on the PASS/FAIL the macro recorded, so a regression in the
-  scan fails here rather than silently passing an abort assertion in a service
-  test.
-
-  Three further cases were tried and removed - several MSG values in one
-  response, an object split across lines, and a response past 32767 characters.
-  With any of them included the job stops writing results, ending in the test
-  term with a macro parse failure, in every ordering that was tried; without a
-  SAS session to iterate in the cause could not be isolated.  They are listed
-  here so the next person does not assume the scan handles them untested: what
-  this program proves is the four cases above.
+  comparison fails here rather than silently passing an abort assertion in a
+  service test.
 
   <h4> SAS Macros </h4>
-  @li mf_getuniquefileref.sas
   @li mp_assert.sas
   @li mpe_assert_abort.sas
 
@@ -29,98 +22,206 @@
     and may not be re-distributed or re-sold without the express permission of
     4GL Apps Ltd.
 
+  @sasjslint {"maxLineLength": 83}
+
 **/
 
-%let rc_any=work.assert_rc1;
-%let rc_other=work.assert_rc2;
-%let rc_none=work.assert_rc3;
-%let rc_first=work.assert_rc4;
-%let rc_split=work.assert_rc5;
-%let rc_long=work.assert_rc6;
-%let rc_noref=work.assert_rc7;
+/** a status row as %mpe_execute leaves it: the job state and the message */
+%macro mk_status(ds, state, msg);
+/* the values come from the symbol table rather than from quoted text: a
+  message holding double quotes would otherwise end the literal early */
+data &ds;
+  length state $20 uri $256 errmsg $32767 platform $10;
+  state=symget('state');
+  uri='';
+  errmsg=symget('msg');
+  platform='SASJS';
+run;
+%mend mk_status;
 
 /**
-  * Test 1 - an abort whose MSG matches is a PASS
+  * Test 1 - an abort whose message matches is a PASS
   */
-%let f1=%mf_getuniquefileref();
-data _null_;
-  file &f1 termstr=crlf;
-  put '{"sasjsAbort": [{"MSG": "Invalid libds", "MAC": "mp_abort"}]}';
-run;
+%mk_status(work.st1, canceled, Invalid libds)
 
 %mpe_assert_abort(
   desc=an abort with the expected MSG,
-  outref=&f1,
+  outstatus=work.st1,
   msg=Invalid libds,
-  outds=&rc_any
+  outds=work.rc1
 )
 
 /**
   * Test 2 - an abort for a different reason is a FAIL
   */
-%let f2=%mf_getuniquefileref();
-data _null_;
-  file &f2 termstr=crlf;
-  put '{"sasjsAbort": [{"MSG": "Invalid libds", "MAC": "mp_abort"}]}';
-run;
+%mk_status(work.st2, canceled, Invalid libds)
 
 %mpe_assert_abort(
   desc=an abort with a different MSG,
-  outref=&f2,
+  outstatus=work.st2,
   msg=Invalid filter,
-  outds=&rc_other
+  outds=work.rc2
 )
 
 /**
-  * Test 3 - a completed call (no sasjsAbort) is a FAIL
+  * Test 3 - a completed call is a FAIL
   */
-%let f3=%mf_getuniquefileref();
-data _null_;
-  file &f3 termstr=crlf;
-  put '{"sasparams": [{"STATUS": "SUCCESS"}]}';
-  put '{"MSG": "Invalid libds"}';
+%mk_status(work.st3, completed, )
+
+%mpe_assert_abort(
+  desc=a completed call,
+  outstatus=work.st3,
+  msg=0,
+  outds=work.rc3
+)
+
+/**
+  * Test 4 - a status dataset that does not exist fails the assertion, not the step
+  */
+%mpe_assert_abort(
+  desc=a status dataset that was never written,
+  outstatus=work.doesnotexist,
+  msg=0,
+  outds=work.rc4
+)
+
+/**
+  * Test 5 - a message longer than the old capture width is compared in full
+  *
+  * The status row carries the whole message, so a caller comparing a long one
+  * gets an answer rather than a silent 256 character cut.
+  */
+%let longmsg=;
+data work.st5;
+  length state $20 errmsg $32767;
+  state='canceled';
+  errmsg=repeat('a clause that runs on ',30);
+  call symputx('longmsg',errmsg,'l');
 run;
 
 %mpe_assert_abort(
-  desc=a response with no sasjsAbort object,
-  outref=&f3,
-  msg=0,
-  outds=&rc_none
+  desc=an abort with a message past the old capture width,
+  outstatus=work.st5,
+  msg=%superq(longmsg),
+  outds=work.rc5
 )
 
 /**
-  * Test 4 - a fileref with no response written records a FAIL instead of aborting the step
+  * Test 6 - quotes in the message survive
   */
-%let f7=%mf_getuniquefileref();
+%mk_status(work.st6, canceled, %str(alpha "quoted" beta))
 
 %mpe_assert_abort(
-  desc=an unassigned outref,
-  outref=&f7,
-  msg=0,
-  outds=&rc_noref
+  desc=a message holding double quotes,
+  outstatus=work.st6,
+  msg=%str(alpha "quoted" beta),
+  outds=work.rc6
 )
 
 /**
-  * Test 5 - the recorded results
+  * Test 7 - the log extract mp_abort appends is not part of the message
+  */
+data work.st7;
+  length state $20 errmsg $32767;
+  state='canceled';
+  errmsg=cats('Invalid libds','0A0A'x,'Log Extract:','0A0A'x,'NOTE: a log line');
+run;
+
+%mpe_assert_abort(
+  desc=an abort with the appended log extract,
+  outstatus=work.st7,
+  msg=Invalid libds,
+  outds=work.rc7
+)
+
+/**
+  * Test 8 - a message that itself mentions the marker is not cut at it
+  *
+  * Only the two newlines before the marker identify the appended extract; a
+  * message that merely says "Log Extract:" keeps its text.
+  */
+data work.st8;
+  length state $20 errmsg $32767;
+  state='canceled';
+  errmsg='check the hook Log Extract: is mentioned here';
+run;
+
+%mpe_assert_abort(
+  desc=a message containing the extract marker,
+  outstatus=work.st8,
+  msg=%str(check the hook Log Extract: is mentioned here),
+  outds=work.rc8
+)
+
+/**
+  * Test 9 - a canceled call with no message fails a comparison it cannot support
+  *
+  * Core fills errmsg on both platforms now, so a blank one means the reason was
+  * never recorded - an expected message cannot be proved, and the assertion
+  * says so rather than passing on the state alone.
+  */
+%mk_status(work.st9, canceled, )
+
+%mpe_assert_abort(
+  desc=a canceled call whose status row carries no message,
+  outstatus=work.st9,
+  msg=Invalid libds,
+  outds=work.rc9
+)
+
+/**
+  * Test 10 - backslashes in the message compare as they were raised
+  */
+%mk_status(work.st10, canceled, %str(C:\temp\share))
+
+%mpe_assert_abort(
+  desc=a message holding backslashes,
+  outstatus=work.st10,
+  msg=%str(C:\temp\share),
+  outds=work.rc10
+)
+
+/**
+  * Test 11 - the doubled separator the escaping producer writes
+  *
+  * mp_abort doubles backslashes, so the appended extract arrives as a literal
+  * doubled pair rather than as characters.
+  */
+data work.st11;
+  length state $20 errmsg $32767;
+  state='canceled';
+  errmsg=cats('Invalid libds','\\n\\nLog Extract:\\n','NOTE: a log line');
+run;
+
+%mpe_assert_abort(
+  desc=an abort whose extract arrives doubled,
+  outstatus=work.st11,
+  msg=Invalid libds,
+  outds=work.rc11
+)
+
+/**
+  * Test 12 - the recorded results
   */
 
+%macro getres(ds, var);
+data _null_;
+  set &ds;
+  call symputx("&var",test_result,'g');
+run;
+%mend getres;
 
-data _null_;
-  set &rc_any;
-  call symputx('res_any',test_result,'g');
-run;
-data _null_;
-  set &rc_other;
-  call symputx('res_other',test_result,'g');
-run;
-data _null_;
-  set &rc_none;
-  call symputx('res_none',test_result,'g');
-run;
-data _null_;
-  set &rc_noref;
-  call symputx('res_noref',test_result,'g');
-run;
+%getres(work.rc1, res_any)
+%getres(work.rc2, res_other)
+%getres(work.rc3, res_none)
+%getres(work.rc4, res_noref)
+%getres(work.rc5, res_long)
+%getres(work.rc6, res_quotes)
+%getres(work.rc7, res_logmark)
+%getres(work.rc8, res_marker)
+%getres(work.rc9, res_nomsg)
+%getres(work.rc10, res_bslash)
+%getres(work.rc11, res_doubled)
 
 %mp_assert(
   iftrue=("&res_any"="PASS"),
@@ -136,15 +237,54 @@ run;
 
 %mp_assert(
   iftrue=("&res_none"="FAIL"),
-  desc=%str(Checking a response without a sasjsAbort object fails),
+  desc=%str(Checking a completed call fails),
   outds=work.test_results
 )
 
-
-
-
 %mp_assert(
   iftrue=("&res_noref"="FAIL"),
-  desc=%str(Checking an unassigned outref fails the assertion, not the step),
+  desc=%str(Checking a missing status dataset fails the assertion, not the step),
+  outds=work.test_results
+)
+
+%mp_assert(
+  iftrue=("&res_long"="PASS"),
+  desc=%str(Checking an abort message past the old capture width matches in full),
+  outds=work.test_results
+)
+
+%mp_assert(
+  iftrue=("&res_quotes"="PASS"),
+  desc=%str(Checking a message holding double quotes compares),
+  outds=work.test_results
+)
+
+%mp_assert(
+  iftrue=("&res_logmark"="PASS"),
+  desc=%str(Checking the appended log extract is stripped),
+  outds=work.test_results
+)
+
+%mp_assert(
+  iftrue=("&res_marker"="PASS"),
+  desc=%str(Checking a message containing the extract marker is not cut at it),
+  outds=work.test_results
+)
+
+%mp_assert(
+  iftrue=("&res_nomsg"="FAIL"),
+  desc=%str(Checking a canceled call with no message fails the comparison),
+  outds=work.test_results
+)
+
+%mp_assert(
+  iftrue=("&res_bslash"="PASS"),
+  desc=%str(Checking a message holding backslashes compares),
+  outds=work.test_results
+)
+
+%mp_assert(
+  iftrue=("&res_doubled"="PASS"),
+  desc=%str(Checking the doubled extract separator is stripped),
   outds=work.test_results
 )

sas/sasjs/macros/mpe_checkrestore.test.sas

@@ -54,4 +54,4 @@ run;
 %mp_assert(
   iftrue=(&REASON=IS ADMIN),
   desc=Checking reason code returned
-)
+)

sas/sasjs/macros/mpe_columnlevelsecurity.sas

@@ -38,6 +38,7 @@
 
 
   <h4> SAS Macros </h4>
+  @li mf_nobs.sas
 
   @author 4GL Apps Ltd
   @copyright 4GL Apps Ltd.  This code may only be used within Data Controller

sas/sasjs/macros/mpe_derived.sas

@@ -49,9 +49,19 @@
   being maintained.  A cycle is rejected.
 
   Semicolons, macro triggers and comments are rejected, so a row can only ever
-  be an expression.  A derived variable may not reuse the name of a physical
-  column, and may not reference itself: an intra-row calculation would imply a
-  retained value, which depends on the order rows happen to arrive in.
+  be an expression, and an expression that would let the row submit code of its
+  own - DOSUBL, or a function reaching the file system, the environment or the
+  macro symbol table - is refused by name.
+
+  The field level rules themselves are core's: mp_derivedvarcheck scans the
+  definitions and mp_derivedvars calls it, which is the same macro the MPE_DERIVED
+  post edit hook runs at submit time (services/hooks/mpe_derived_postedit).  A
+  row is therefore measured against one rule list, whether it is being staged
+  or read.
+
+  A derived variable may not reuse the name of a physical column, and may not
+  reference itself: an intra-row calculation would imply a retained value,
+  which depends on the order rows happen to arrive in.
 
   <h4> SAS Macros </h4>
   @li mp_derivedvars.sas
@@ -69,6 +79,8 @@
     and may not be re-distributed or re-sold without the express permission of
     4GL Apps Ltd.
 
+  @sasjslint {"maxLineLength": 89}
+
 **/
 
 %macro mpe_derived(libds=,
@@ -87,7 +99,7 @@
 
 /* dc_dttmtfmt is the bitemporal cut-off used in the where= below - callers get
   it from %mpeinit().  Unset, the filter degenerates to "lt tx_to" and the
-  control-table step fails with a syntax error a long way from the cause. */
+  control-table step fails with a syntax fault a long way from the cause. */
 %mp_abort(iftrue= (%length(&libds)=0 or %length(&dclib)=0 or %length(&dc_dttmtfmt)=0)
   ,mac=&sysmacroname
   ,msg=%str(Missing: libds dclib dc_dttmtfmt)

sas/sasjs/macros/mpe_derived.test.sas

@@ -21,6 +21,8 @@
     and may not be re-distributed or re-sold without the express permission of
     4GL Apps Ltd.
 
+  @sasjslint {"maxLineLength": 119}
+
 **/
 
 

sas/sasjs/macros/mpe_derived_check.sas

@@ -1,75 +0,0 @@
-/**
-  @file
-  @brief Field level checks for a staged MPE_DERIVED submission
-  @details Writes one row per staged row that breaks a rule, with the reason in
-  REASON, so a caller can report the first failure and abort.  Rows marked for
-  deletion are skipped - they are being removed, so their fields are not loaded.
-
-  The rules are the ones %mpe_derived applies at read time, so a submission
-  cannot stage a row the engine would then refuse.
-
-  @param inds the staged rows, already normalised by the caller
-  @param outds the dataset to create: one row per failing row, plus REASON
-
-  @author 4GL Apps Ltd
-  @copyright 4GL Apps Ltd.  This code may only be used within Data Controller
-    and may not be re-distributed or re-sold without the express permission of
-    4GL Apps Ltd.
-
-**/
-
-%macro mpe_derived_check(inds=work.staging_ds, outds=work.derived_check);
-data &outds;
-  set &inds;
-  length reason $200;
-  if upcase(_____DELETE__THIS__RECORD_____)='YES' then delete;
-  call missing(reason);
-
-  if missing(libref) or missing(dsn) then reason='LIBREF and DSN are required';
-  else if var_nm='' then reason='VAR_NM is required';
-  else if verify(strip(var_nm),'ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_')>0
-    then reason='VAR_NM must be a valid SAS name';
-  else if verify(substr(var_nm,1,1),'0123456789')=0
-    then reason='VAR_NM must not start with a digit';
-  else if var_nm='_____DELETE__THIS__RECORD_____'
-    then reason='VAR_NM is reserved';
-  else if var_type not in ('C','N')
-    then reason='VAR_TYPE must be C or N';
-  /* ACTIVE is a picker in the EDIT screen (HARDSELECT over the MPE_SELECTBOX
-    pair) - the same pair is enforced here, so a submission that bypasses the
-    grid cannot land a value the screen would never offer. */
-  else if missing(active) or active not in (0,1)
-    then reason='ACTIVE must be 1 or 0';
-  else if missing(var_len) or var_len ne int(var_len) or var_len<1
-    then reason='VAR_LEN must be a positive integer';
-  else if var_type='C' and var_len>32767
-    then reason='VAR_LEN exceeds 32767 for a character variable';
-  else if var_type='N' and var_len>8
-    then reason='VAR_LEN exceeds 8 for a numeric variable';
-  else if index(var_fmt,';')>0 or indexc(var_fmt,'&%')>0
-    or index(var_fmt,'/*')>0 or index(var_fmt,'*/')>0
-    then reason='VAR_FMT must be a format name, without ; & % or /* */';
-  else if verify(strip(var_fmt),'ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_$.')>0
-    then reason='VAR_FMT must be a format name';
-  /* VAR_LABEL is free text: any character is allowed, in any script, and the
-    quotes in it are escaped by %mp_derivedvars when the attrib statement is
-    built.  What is refused is anything that cannot be a single-line column
-    label - a control character or a line break - and a value long enough to
-    be cut down silently, since the engine truncates the label to 256. */
-  else if anycntrl(var_label)>0
-    then reason='VAR_LABEL must be plain text - no line breaks or control characters';
-  else if klength(var_label)>256
-    then reason='VAR_LABEL exceeds 256 characters';
-  else if missing(var_logic)
-    then reason='VAR_LOGIC is required';
-  else if index(var_logic,';')>0
-    then reason='VAR_LOGIC must be a single expression - remove the ;';
-  else if indexc(var_logic,'&%')>0
-    then reason='VAR_LOGIC must not contain macro triggers';
-  else if index(var_logic,'/*')>0 or index(var_logic,'*/')>0
-    then reason='VAR_LOGIC must not contain comments';
-
-  if not missing(reason) then output;
-run;
-
-%mend mpe_derived_check;

sas/sasjs/macros/mpe_derived_check.test.sas

@@ -1,194 +0,0 @@
-/**
-  @file
-  @brief testing the mpe_derived_check macro
-
-  @details Checks each field level rule the MPE_DERIVED post edit hook applies
-  to a staged submission.  The macro is exercised directly so the rules can be
-  probed with values a service cannot carry: VAR_LABEL is varchar(256) in the
-  table and a staging dataset declares the same length, so a longer label is
-  truncated before any check would see it.
-
-  <h4> SAS Macros </h4>
-  @li mf_nobs.sas
-  @li mp_assert.sas
-  @li mpe_derived_check.sas
-
-  @author 4GL Apps Ltd
-  @copyright 4GL Apps Ltd.  This code may only be used within Data Controller
-    and may not be re-distributed or re-sold without the express permission of
-    4GL Apps Ltd.
-
-**/
-
-/**
-  * Test 1 - one row per failing rule, and only failing rows
-  */
-data work.staging_ds;
-  length libref $8 dsn $32 var_nm $32 var_type $1 var_len 8
-    var_fmt $32 var_label $4000 var_deps $4000 var_logic $4000
-    active 8 _____DELETE__THIS__RECORD_____ $3;
-  libref='DCTEST';
-  dsn='DC_DERIV';
-  /* valid unless a row says otherwise - ACTIVE is mandatory in the screen and
-    the check enforces the same pair here */
-  active=1;
-  _____DELETE__THIS__RECORD_____='No';
-
-  /* meets every rule */
-  var_nm='GOOD';
-  var_type='N';
-  var_len=8;
-  var_fmt='';
-  var_label='a plain label';
-  var_deps='';
-  var_logic='1';
-  output;
-
-  /* delete marked rows are being removed, so their fields are not loaded */
-  var_nm='DELME';
-  var_label='first line'||'0A'x||'second line';
-  _____DELETE__THIS__RECORD_____='Yes';
-  output;
-  _____DELETE__THIS__RECORD_____='No';
-
-  /* free text is allowed, including quotes and punctuation */
-  var_nm='FREETXT';
-  var_label='It''s "quoted", plain text: 100 percent';
-  output;
-
-  /* a line break cannot be a single line column label */
-  var_nm='LBLBREAK';
-  var_label='first line'||'0A'x||'second line';
-  output;
-
-  /* longer than the table holds, so it would be cut down silently */
-  var_nm='LBLLONG';
-  var_label=repeat('x',300);
-  output;
-
-  /* a format name, not an expression */
-  var_nm='BADFMT';
-  var_fmt='DATE9;';
-  var_label='fmt';
-  output;
-
-  /* one expression, no statement separator, no macro triggers */
-  var_nm='BADLOGIC';
-  var_fmt='';
-  var_logic='x; y';
-  output;
-
-  /* C or N */
-  var_nm='BADTYPE';
-  var_logic='1';
-  var_type='X';
-  output;
-
-  /* ACTIVE is a picker in the screen: only 1 or 0 */
-  var_nm='BADACTIVE';
-  var_type='C';
-  var_logic='1';
-  active=2;
-  output;
-
-  stop;
-run;
-
-%mpe_derived_check(inds=work.staging_ds, outds=work.derived_check)
-
-/**
-  * Test 2 - the reported rows, rule by rule
-  */
-data work.expected;
-  length var_nm $32 reason $200;
-  var_nm='LBLBREAK';
-  reason='VAR_LABEL must be plain text - no line breaks or control characters';
-  output;
-  var_nm='LBLLONG';
-  reason='VAR_LABEL exceeds 256 characters';
-  output;
-  var_nm='BADFMT';
-  reason='VAR_FMT must be a format name, without ; & % or /* */';
-  output;
-  var_nm='BADLOGIC';
-  reason='VAR_LOGIC must be a single expression - remove the ;';
-  output;
-  var_nm='BADTYPE';
-  reason='VAR_TYPE must be C or N';
-  output;
-  var_nm='BADACTIVE';
-  reason='ACTIVE must be 1 or 0';
-  output;
-run;
-
-proc sql;
-create table work.cmp as
-  select e.var_nm
-    ,case when a.reason=e.reason then 1 else 0 end as same
-  from work.expected e
-  left join work.derived_check a
-    on upcase(strip(a.var_nm))=upcase(e.var_nm);
-quit;
-
-data _null_;
-  set work.cmp;
-  call symputx(cats('rule_',var_nm),same,'l');
-run;
-
-%mp_assert(
-  iftrue=(%mf_nobs(work.derived_check)=6),
-  desc=%str(Checking only the six failing rows are reported),
-  outds=work.test_results
-)
-
-%mp_assert(
-  iftrue=(&rule_LBLBREAK=1),
-  desc=%str(Checking a label with a line break is rejected),
-  outds=work.test_results
-)
-
-%mp_assert(
-  iftrue=(&rule_LBLLONG=1),
-  desc=%str(Checking a label over 256 characters is rejected),
-  outds=work.test_results
-)
-
-%mp_assert(
-  iftrue=(&rule_BADFMT=1),
-  desc=%str(Checking a format carrying a statement separator is rejected),
-  outds=work.test_results
-)
-
-%mp_assert(
-  iftrue=(&rule_BADLOGIC=1),
-  desc=%str(Checking a logic value with a statement separator is rejected),
-  outds=work.test_results
-)
-
-%mp_assert(
-  iftrue=(&rule_BADTYPE=1),
-  desc=%str(Checking a VAR_TYPE other than C or N is rejected),
-  outds=work.test_results
-)
-
-%mp_assert(
-  iftrue=(&rule_BADACTIVE=1),
-  desc=%str(Checking an ACTIVE outside 1 or 0 is rejected),
-  outds=work.test_results
-)
-
-/**
-  * Test 3 - a row that meets every rule, a free text label, and a delete
-  * marked row are all left alone
-  */
-proc sql noprint;
-select count(*) into: notchecked
-  from work.derived_check
-  where upcase(strip(var_nm)) in ('GOOD','FREETXT','DELME');
-quit;
-
-%mp_assert(
-  iftrue=(&notchecked=0),
-  desc=%str(Checking a valid row, a free text label and a delete marked row are not reported),
-  outds=work.test_results
-)

sas/sasjs/macros/mpe_execute.sas

@@ -47,6 +47,8 @@
   @li mp_abort.sas
   @li mx_execute.sas
 
+  @sasjslint {"maxLineLength": 101}
+
 **/
 
 %macro mpe_execute(program
@@ -129,16 +131,40 @@
   * against whatever the previous call left in outlib.  A missing status table
   * and an empty one are both failures: state would stay blank and the caller
   * would carry on inferring, which is the thing this macro exists to stop. */
+%global _mpe_abortmsg;
+%let _mpe_abortmsg=;
 %let state=;
 %let msg=;
 %if %sysfunc(exist(&a_status))=1 %then %do;
   data _null_;
+    length _prog $32767 _state $32767 _prefix $32767 _childmsg $32767
+      _out $32767 _room 8;
     set &a_status;
     call symputx('state',state,'l');
     call symputx('msg',errmsg,'l');
+    /* the text the caller sees is assembled here rather than with %let: the
+      child's message is arbitrary text, and a %, an & or a quote in it would
+      be macro resolved on the way into a symbol.  symget/symputx hand it over
+      untouched, and a message already at the 32767 limit is cut to the room
+      left rather than overflowed by the join. */
+    if upcase(state) ne 'COMPLETED' then do;
+      _prog=symget('a_pgm');
+      _state=symget('state');
+      _childmsg=symget('msg');
+      _prefix=cat(trim(_prog),' did not complete: state=',trim(_state));
+      _room=32767-length(trim(_prefix))-2;
+      if _room<1 then _room=0;
+      if length(_childmsg)>_room then _childmsg=substr(_childmsg,1,_room);
+      if missing(_childmsg) then _out=trim(_prefix);
+      else _out=trim(_prefix)||' '||_childmsg;
+      call symputx('_mpe_abortmsg',_out,'g');
+    end;
   run;
 %end;
-%else %let state=missing(&a_status);
+%else %do;
+  %let state=missing(&a_status);
+  %let _mpe_abortmsg=&a_pgm did not complete: state=&state;
+%end;
 
 %if %length(&failfast)=0 %then %let failfast=1;
 %if &failfast=1 %then %do;
@@ -152,7 +178,7 @@
     %mp_abort(
       iftrue=(1=1)
       ,mac=&sysmacroname
-      ,msg=%str(&program did not complete: state=&state &msg)
+      ,msg=%superq(_mpe_abortmsg)
     )
   %end;
 %end;

sas/sasjs/macros/mpe_execute.test.sas

@@ -123,3 +123,70 @@ run;
   desc=%str(Checking mdebug=1 keeps the child log),
   outds=work.test_results
 )
+
+/**
+  * Test 3 - the abort text names the program, the state and the child message
+  *
+  * The text is assembled from the status row rather than with %let, so this is
+  * the path that has to hold: a real aborting child, failfast=0, and the text
+  * the caller would see in the abort.
+  */
+%let f3=%mf_getuniquefileref();
+data _null_;
+  file &f3 termstr=crlf;
+  put 'FILTER_TABLE:$41.';
+  put '%sysfunc(coalescec(&dclib..MPE_TABLES,))';
+run;
+
+%let outref3=%mf_getuniquefileref();
+
+%mpe_execute(&_program,
+  viyacontext=&defaultcontext,
+  inputfiles=&f3:iwant,
+  outref=&outref3,
+  outstatus=work.exec_status3,
+  failfast=0,
+  viyaresult=WEBOUT_TXT
+)
+
+%let childmsg3=;
+data _null_;
+  set work.exec_status3 end=last;
+  if last then call symputx('childmsg3',errmsg,'l');
+run;
+
+%mp_assert(
+  iftrue=(%index(%superq(_mpe_abortmsg),%str(did not complete: state=))>0),
+  desc=%str(Checking the abort text names the program state),
+  outds=work.test_results
+)
+
+%mp_assert(
+  iftrue=(%index(%superq(_mpe_abortmsg),%superq(childmsg3))>0),
+  desc=%str(Checking the abort text carries the child message),
+  outds=work.test_results
+)
+
+/**
+  * Test 4 - the assembled text cannot overflow
+  *
+  * SAS character variables stop at 32767; a child message already at that limit
+  * is cut to the room the prefix leaves, so the join neither errors nor quietly
+  * loses the prefix.
+  */
+%let len_out=0;
+data _null_;
+  call symputx('len_out',length(symget('_mpe_abortmsg')),'l');
+run;
+
+%mp_assert(
+  iftrue=(&len_out<=32767),
+  desc=%str(Checking the abort text cannot overflow),
+  outds=work.test_results
+)
+
+%mp_assert(
+  iftrue=(%index(%superq(_mpe_abortmsg),%str(did not complete: state=))>0),
+  desc=%str(Checking the prefix survives whatever the child sent),
+  outds=work.test_results
+)

sas/sasjs/macros/mpe_filtermaster.sas

@@ -31,7 +31,6 @@
   @li mf_getuser.sas
   @li mf_getuniquefileref.sas
   @li mf_getuniquename.sas
-  @li mf_nobs.sas
   @li mp_abort.sas
   @li mp_filtergenerate.sas
   @li mpe_getgroups.sas

sas/sasjs/macros/mpe_getlabels.test.sas

@@ -20,8 +20,6 @@
       labels still come from the target.
 
   <h4> SAS Macros </h4>
-  @li mf_getuniquename.sas
-  @li mf_nobs.sas
   @li mp_assert.sas
   @li mp_assertdsobs.sas
   @li mp_assertscope.sas
@@ -32,6 +30,8 @@
     and may not be re-distributed or re-sold without the express permission of
     4GL Apps Ltd.
 
+  @sasjslint {"maxLineLength": 81}
+
 **/
 
 /* teardown - remove any seeded rows */

sas/sasjs/macros/mpe_getversions.sas

@@ -31,4 +31,4 @@
       and a.submit_status_cd='APPROVED' and a.num_of_approvals_remaining=0
     order by a.reviewed_on_dttm desc;
 
-%mend mpe_getversions;
+%mend mpe_getversions;

sas/sasjs/macros/mpe_getversions.test.sas

@@ -9,7 +9,6 @@
   @li mp_assert.sas
   @li mp_assertscope.sas
   @li mpe_getversions.sas
-  @li mpe_targetloader.sas
   @li mx_execute.sas
 
 **/

sas/sasjs/macros/mpe_loader.sas

@@ -32,6 +32,9 @@
   @copyright 4GL Apps Ltd.  This code may only be used within Data Controller
     and may not be re-distributed or re-sold without the express permission of
     4GL Apps Ltd.
+
+  @sasjslint {"maxLineLength": 82}
+
 **/
 
 %macro mpe_loader(
@@ -315,7 +318,7 @@ data final_check;
     type2='$1.';
   end;
   else type2='best32.';
-  * else type2=cats(length,'.');
+  /* else type2=cats(length,'.'); */
 
   input_statement=catx(' ',input_statement,name,':',type2);
 

sas/sasjs/macros/mpe_makedata.sas

@@ -17,6 +17,9 @@
   @author 4GL Apps Ltd
   @copyright 4GL Apps Ltd - this is a licensed product and NOT FOR RESALE
   OR DISTRIBUTION.
+
+  @sasjslint {"maxLineLength": 99}
+
 **/
 
 %macro mpe_makedata(lib=,mpeadmins=,path=);
@@ -2355,7 +2358,9 @@ insert into &lib..MPE_VALIDATIONS set
       ,some_char="&x bottles of beer on the wall"
       ,some_dropdown='Option 1'
       ,some_num=ranuni(0)
-      ,some_date=round(ranuni(0)*1000,1)
+      /* First of the month, cycling through two years: a randomly spread date
+      gave every row its own day, so a date filter never returned many rows. */
+      ,some_date=intnx('month','01JAN1960'd,mod(&x,24),'S')
       ,some_datetime=round(ranuni(0)*50000,1)
       ,some_time=round(ranuni(0)*100,1)
       ,some_shortnum=6+round(ranuni(0)*94,1)

sas/sasjs/macros/mpe_makedatamodel.sas

@@ -15,6 +15,9 @@
   @copyright 4GL Apps Ltd.  This code may only be used within Data Controller
     and may not be re-distributed or re-sold without the express permission of
     4GL Apps Ltd.
+
+  @sasjslint {"maxLineLength": 91}
+
 **/
 
 %macro mpe_makedatamodel(lib=);

sas/sasjs/macros/mpe_makedatamodel.test.sas

@@ -22,5 +22,5 @@
 
 %mp_assert(
   iftrue=(&syscc = 0),
-  desc=Checking error condition
-)
+  desc=Checking the abort condition
+)

sas/sasjs/macros/mpe_refreshcatalogs.sas

@@ -18,6 +18,8 @@
   @version 9.3
   @author 4GL Apps Ltd
 
+  @sasjslint {"maxLineLength": 86}
+
 **/
 
 %macro mpe_refreshcatalogs(lib,cat=#all