chore(release): 7.21.0 [skip ci]

# [7.21.0](https://git.datacontroller.io/dc/dc/compare/v7.20.0...v7.21.0) (2026-10-29)

### Bug Fixes

* **client:** parse word-date values from VA parameters ([405b28d](405b28dccc))
* **client:** parse word-date values from VA parameters ([ce5b1aa](ce5b1aa961))
* **makedata:** name the sasjs server group in the estate seed ([eeffa92](eeffa92f9e))

### Features

* **derived:** apply the core field scan in the MPE_DERIVED hook ([a4a0f9c](a4a0f9cb2f))

.agents/docs/testing.md

@@ -1,10 +1,11 @@
 # Testing
 
-Backend (SAS) tests are run with the sasjs CLI from the `sas/` directory.
+Backend (SAS) tests are run with the sasjs CLI from the `sas/` directory. For a deploy-and-run on the shared state see the `dc-estate-testing` skill (`.agents/skills/dc-estate-testing/SKILL.md`), which covers the credentials, the `makedata` seed the suite needs and the estate's group name.
 
 ## Commands
 
 - Full deploy + test cycle: `npm run 4gl` (compiles, deploys to the 4gl target, runs makedata) then `sasjs test -t 4gl`.
+- The shared estate under its `server` target: `npm run server` (compile, deploy, seed) then `sasjs test -t server`.
 - Run tests only (no rebuild/redeploy): `sasjs test -t 4gl`.
 - Run a subset of tests: `sasjs test -t 4gl SOMESTRING` — the positional argument filters to matching tests (e.g. `sasjs test -t 4gl stagedata.test.3`).
 

.agents/skills/dc-estate-testing/SKILL.md

@@ -0,0 +1,79 @@
+---
+name: dc-estate-testing
+description: >
+  Use when deploying Data Controller to a shared SASjs Server estate (sas.4gl.io) or running the SAS
+  test suite against it. Covers the target and credential setup, the makedata seed the suite needs on
+  a fresh appLoc and the group it must name, how to read the results, and how to deploy the frontend
+  so the streamed app is the build you just made.
+---
+
+# Deploying and testing on the shared SASjs Server estate
+
+The whole cycle is two npm scripts plus `sasjs test`. Everything below explains what those scripts do and the steps that go wrong.
+
+    cd sas && npm ci                             # macro dependencies
+    cd ../client && npm ci && npm run build      # frontend only - see "Deploying the frontend"
+    cd ../sas && npm run server                  # cpfavicon + sasjs cbd -t server + makedata
+    npx sasjs test -t server
+
+`npm run server` is `sasjs cbd -t server && npm run serverdata`, and the seed in the second half is not optional. `sasjs cbd` on its own deploys the services and the test programs but leaves the appLoc with no Data Controller configuration, and `testsetup` reads `<appLoc>/services/public/settings.sas` through the drive API. `makedata` is what creates that file, so without the seed `testsetup` gets a 404, aborts, and 59 of the 64 test programs fail to complete on setup alone - which reads like a broken deployment rather than a missing seed.
+
+## The targets
+
+`server` is the shared estate: `serverType: SASJS` on https://sas.4gl.io, `appLoc` `/Public/app/dc`, a real SAS 9 binary behind SASjs Server (`GET /SASjsApi/info` reports `mode: server`, `runTimes: ["sas"]`, `localLoginEnabled: true`). `4gl` is the same estate with a slightly different config, hence the older `npm run 4gl` / `4glmakedata` pair.
+
+A run against `server` exercises the SASjs Server branch of every macro and the SAS 9 backend, and nothing else: no Viya branch, no SAS 9 STP branch. Say that when reporting, rather than implying the matrix is covered.
+
+To deploy your own branch without replacing the app everyone else uses, add a target that copies `server` and changes `appLoc` and `streamServiceName` (for example `/Public/app/<you>/dc`), keep it out of the commit, and pass it with `-t`. `sasjs cbd` overwrites the whole appLoc it is pointed at.
+
+## Credentials
+
+The CLI logs in with a registered client, not a username and password: `.env.<target>` (gitignored, mode 600) needs `CLIENT`, `ACCESS_TOKEN` and `REFRESH_TOKEN`. With `CLIENT` alone it prints a `/SASjsLogon?client_id=...` URL and waits for an authorization code, which cannot be answered headless.
+
+Mint the pair from a local login, with the password read from a file rather than argv:
+
+1. `GET /` with a cookie jar - it sets `XSRF-TOKEN`, which every POST must echo in an `X-XSRF-TOKEN` header.
+2. `POST /SASLogon/login` with `{"username","password"}`.
+3. `GET /SASjsApi/client` on the same session - the registered client ids.
+4. `POST /SASLogon/authorize` with `{"clientId"}` -> `{"code"}`.
+5. `POST /SASjsApi/auth/token` with `{"clientId","code"}` -> `accessToken` / `refreshToken`.
+
+An unmatched `/SASjsApi/...` path answers `200` with the Swagger page, so check the content type; HTML means the route does not exist. Delete the token file when the run is over.
+
+## The makedata seed
+
+`npm run serverdata` sends `deploy/makeDataServer.json`: one `fromjs` row with `ADMIN`, `DCPATH` and `OUTFMT`. Two things about it are worth knowing before you trust its result.
+
+- **`ADMIN` must name a group the calling account is actually in.** The shipped value is `all-users`, the SASjs Server group that every logged-in user is a member of, which is what the shared estate needs. A name the account is not in - a Viya-style group such as `AllUsers`, or one estate's own group carried into another - still answers `SUCCESS` from the seed, and then every later `getdata`/`stagedata` call aborts with `<user> is not authorised to edit <lib>.<ds> in the <lib>.MPE_SECURITY table`. Check what the account belongs to with a call to `services/public/getgroups` before changing the value.
+- **Each run creates a new library.** The response carries `STATUS`, `DCPATH` and `LIBREF` (for example `DC621835`), and the LIBREF is clock-derived, so a libref from an earlier run answers `dataset <lib>.<table> does not exist` - take the current one from the response.
+- **`KEEPME` controls whether the seed can be repeated.** Without it `makedata` deletes itself and the configurator after a successful run (deliberately, so it cannot be executed twice over real data), and a second seed fails with `The Program at (<appLoc>/services/admin/makedata) does not exist` until you run `sasjs cbd` again. A harness that reseeds between runs sets `KEEPME` in the input row.
+- **Seed once per deploy, not before every test run.** The suite builds its own `dctest` library from the seeded control tables; reseeding in between is wasted work, and there is no need to run `makedata` and `sasjs test` in the opposite order.
+
+## Reading the results
+
+Results land in `sas/sasjsresults/`: `testResults.json` (the counts worth quoting: `tests_with_results`, `tests_that_pass`, `failed_to_complete`, `completed_with_failures`), `testResults.csv` (one row per assertion, lower-case headers `test_target`, `test_loc`, `sasjs_test_id`, `test_suite_result`, `test_description`, `test_url`), and one SAS log per program under `logs/`. The CLI's closing lines are not the source of truth when they disagree with the JSON.
+
+A current green run is 64 of 64 test programs with 241 assertions passing and nothing failing to complete.
+
+`ERROR` lines in the per-program logs are expected and are not a failed run: the specs that assert a refusal get `ERROR: Execution canceled by an %ABORT CANCEL statement`, and the ones that feed bad payloads get the rejection message they are asserting on (`Invalid libds:...`, `Invalid libref:...`). Judge the run by the JSON counts, then by whether the errors you find are the ones the tests asked for.
+
+## Deploying the frontend
+
+The streamed web app is a separate artefact and `sasjs cbd` only streams it if `client/dist` exists:
+
+1. `client/libraries/sheet-crypto.tgz.gpg` is an encrypted dependency (`file:libraries/sheet-crypto.tgz` in `client/package.json`) that is not on the public registry. Decrypt it with the passphrase the maintainer holds: `printf '%s' "$SHEET_PWD" | gpg --batch --yes --passphrase-fd 0 --output ./libraries/sheet-crypto.tgz --decrypt ./libraries/sheet-crypto.tgz.gpg`.
+2. `client/.npmrc` is gitignored and needs `legacy-peer-deps=true` (the committed lock is written with it) plus the `@sheet` registry lines.
+3. `cd client && npm ci && npm run build`.
+4. The build carries the repo default appLoc, so point it at the deployment before deploying: rewrite `apploc="/Public/app/dc"` in `dist/index.html` to the target appLoc.
+5. `cd ../sas && npx sasjs cbd -t <target>` - with `client/dist` present the log says `Copying web app files` and `Web app is available at <url>`.
+
+Without step 1 the deploy prints `ERROR  An error has occurred when compiling web app services ... webSourcePath: .../client/dist doesn't exist` and then deploys the services anyway, so the deploy looks successful while the app streams nothing. The AppStream URL is session-protected: an anonymous `curl` gets a `401`, so verify it with a logged-in cookie jar and fetch the `main.<hash>.js` the page names.
+
+Never commit `client/.npmrc` or the decrypted tarball; both are ignored, and `git status` staying clean is the check.
+
+## Reading a failed run
+
+- `ERROR An error occurred while executing the request. Job location: <appLoc>/tests/...` means the response was not JSON, so the program broke before it could write results. Open `sasjsresults/logs/<suite>.log` and read the first `ERROR`; a single compile break cascades into hundreds of `ERROR 180-322: Statement is not valid` lines and the tail is noise.
+- `WARNING: Apparent symbolic reference TESTLOC not resolved` in `testsetup.log`, next to a `404 Not Found` for `services/public/settings.sas`, is the missing seed described above, not a macro problem.
+- `sasjs test` runs the DEPLOYED program, so after editing a service, hook or test, run `sasjs cbd` again before testing or you will debug code that is no longer in the tree.
+- A deployment can succeed and still be missing its web app, its data or its configuration - check the thing you changed rather than the exit code.

.gitea/scripts/cypress-specs.sh

@@ -0,0 +1,49 @@
+#!/usr/bin/env bash
+# The Cypress specs CI runs, and the order they run in.
+#
+# ORDER MATTERS. These specs share one mock estate that is seeded once per job,
+# so several of them depend on the state they find:
+#
+#   - csv-limited asserts the free-tier "Locked Feature" modal, which an earlier
+#     spec can remove by planting a valid licence. It has always run first.
+#   - hook-programs asserts that an edit raises an approval row against its
+#     target table; it runs early, before other specs have added rows to the
+#     approval tables.
+#
+# Specs added later are appended rather than inserted, so the sequence that was
+# known green keeps its shape. metanav is deliberately absent - see
+# https://git.datacontroller.io/dc/dc/issues/395.
+#
+# download is held out for a different reason - its download button is covered by
+# a fixed-position element, see
+# https://git.datacontroller.io/dc/dc/issues/398.
+set -euo pipefail
+
+SPECS="csv-limited hook-programs liveness editor excel-multi-load excel csv filtering filter-panel licensing viewer-labels full-table-search viewbox stage browser-info ddl-export row-cell-limits derived-variables startupservice-launcher coltype-delete-record submitted-details embed-va"
+
+# Specs that exist but are held out of CI, each with the reason. Listed here so
+# the coverage check below can tell a deliberate exclusion from a forgotten file.
+EXCLUDED="metanav download"
+
+case "${1:-}" in
+  all)
+    # comma separated, the form `cypress run --spec` takes
+    for name in $SPECS; do printf 'cypress/e2e/%s.cy.ts,' "$name"; done | sed 's/,$//'
+    ;;
+  coverage)
+    on_disk=$(cd "$2/client" && ls cypress/e2e/*.cy.ts | xargs -n1 basename | sed 's/\.cy\.ts$//' | sort)
+    listed=$(printf '%s\n' $SPECS $EXCLUDED | sort)
+    missing=$(comm -23 <(echo "$on_disk") <(echo "$listed"))
+    if [ -n "$missing" ]; then
+      echo "Specs that are neither run nor explicitly excluded:"
+      echo "$missing"
+      echo "Add them to SPECS (or EXCLUDED, with a reason) in .gitea/scripts/cypress-specs.sh"
+      exit 1
+    fi
+    echo "All $(echo "$on_disk" | wc -l) specs are accounted for: $(echo $SPECS | wc -w) run, $(echo $EXCLUDED | wc -w) excluded."
+    ;;
+  *)
+    echo "usage: $0 {all|coverage <repo-root>}" >&2
+    exit 2
+    ;;
+esac

.gitea/workflows/build.yaml

@@ -75,11 +75,33 @@ jobs:
           cd client
           npm run test:headless
 
+      - name: Angular Tests in other timezones
+        # The VA filter clauses are built from local calendar dates, so a
+        # UTC-only run cannot see a date that slips a day west of UTC or east of
+        # it. Only the VA specs carry that property, so both scripts run that
+        # subset rather than the whole suite a second and third time.
+        run: |
+          cd client
+          npm run test:tz-west
+          npm run test:tz-east
+
+      - name: Every Cypress spec is in the CI list
+        # The spec list used to be spelled out inline in --spec, and five files
+        # quietly fell out of it - among them the VA embed suite.  Fails when a
+        # spec on disk is not listed.
+        run: bash .gitea/scripts/cypress-specs.sh coverage .
+
       - name: Production Build
         run: |
           cd client
           npm run build
 
+
+  # Cypress shard 1 of 4.  The setup repeats in every shard because each
+  # one gets its own runner and its own SASjs server; that fixed cost is what
+  # limits how far sharding helps.  The specs it runs come from
+  # .gitea/scripts/cypress-specs.sh.
+
   Build-and-test-development:
     runs-on: ubuntu-latest
     needs: Build-and-ng-test
@@ -142,6 +164,7 @@ jobs:
           echo NODE_PATH=node >> .env
           echo CORS=enable >> .env
           echo WHITELIST=http://localhost:4200 >> .env
+          echo HTTP_PORT=5000 >> .env
           # The server refuses to run as root unless RUN_AS names an account
           # (a security change in the recent releases). This job runs as root
           # in a throwaway container, so opt in explicitly.
@@ -181,9 +204,20 @@ jobs:
           sasjs request services/admin/makedata -t server-ci -d deploy/makedata.json -o ./makedata_out.json || true
 
       - name: Prepare and run frontend and cypress
-        timeout-minutes: 35
+        # One job, one SASjs server, one ng serve, one Chrome: four of each on one
+        # runner starved the app under test and every failure in that run was a
+        # timeout, not an assertion.
+        timeout-minutes: 50
         run: |
           cd ./client
+          # The spec list lives in one place, and an empty list is a broken
+          # workflow rather than a green run with no tests.
+          SPECS="$(bash ../.gitea/scripts/cypress-specs.sh all)"
+          if [ -z "$SPECS" ]; then
+            echo "the spec list resolved to nothing"
+            exit 1
+          fi
+          echo "specs: $SPECS"
           mv ./cypress.env.example.json ./cypress.env.json
           replace-in-files --regex='"username".*' --replacement='"username":"'${{ secrets.CYPRESS_USERNAME_SASJS }}'",' ./cypress.env.json
           replace-in-files --regex='"password".*' --replacement='"password":"'${{ secrets.CYPRESS_PWD_SASJS }}'" ' ./cypress.env.json
@@ -196,18 +230,24 @@ jobs:
           replace-in-files --regex='"hosturl".*' --replacement='hosturl:"http://localhost:4200",' ./cypress.config.ts
           cat ./cypress.config.ts
           # Start frontend and run cypress
-          # timeout 1800: SIGTERM after 30 min so Cypress can flush video/screenshots
+          # timeout 2700: SIGTERM after 45 min so Cypress can flush video/screenshots
           # before the outer timeout-minutes hard-kills the step (avoids silent multi-hour hangs)
-          npx ng serve --host 0.0.0.0 --port 4200 & npx wait-on http://localhost:4200 && timeout 1800 npx cypress run --browser chrome --spec "cypress/e2e/csv-limited.cy.ts,cypress/e2e/hook-programs.cy.ts,cypress/e2e/liveness.cy.ts,cypress/e2e/editor.cy.ts,cypress/e2e/excel-multi-load.cy.ts,cypress/e2e/excel.cy.ts,cypress/e2e/csv.cy.ts,cypress/e2e/filtering.cy.ts,cypress/e2e/filter-panel.cy.ts,cypress/e2e/licensing.cy.ts,cypress/e2e/viewer-labels.cy.ts,cypress/e2e/full-table-search.cy.ts,cypress/e2e/viewbox.cy.ts,cypress/e2e/stage.cy.ts,cypress/e2e/browser-info.cy.ts,cypress/e2e/ddl-export.cy.ts,cypress/e2e/row-cell-limits.cy.ts,cypress/e2e/derived-variables.cy.ts,cypress/e2e/startupservice-launcher.cy.ts"
+          npx ng serve --host 0.0.0.0 --port 4200 & npx wait-on http://localhost:4200 && timeout 2700 npx cypress run --browser chrome --spec "$SPECS"
 
       - name: Zip Cypress videos
-        if: always()
+        # Only on a failure: a green run has nothing to watch, and zipping and
+        # uploading every run's videos is the part of the video cost that lands
+        # on the clock.
+        if: failure()
         run: |
           mkdir -p ./client/cypress/videos
           zip -r cypress-videos ./client/cypress/videos
 
       - name: Add cypress videos artifacts
-        if: always()
+        if: failure()
+        # v3, not v4: this runner reports "GHESNotSupportedError:
+        # @actions/artifact v2.0.0+, upload-artifact@v4+ ... not supported on
+        # GHES", so the current major cannot upload here at all.
         uses: actions/upload-artifact@v3
         with:
           name: cypress-videos.zip

.gitea/workflows/docs.yaml

@@ -0,0 +1,121 @@
+name: Publish code documentation
+run-name: Publishing typedoc and SAS docs
+
+# The code documentation sites are published here rather than inline in the
+# release job, so that they can be republished on their own - after a secret
+# rotation, or for a release whose artefact upload failed (the release job
+# cannot simply be re-run: semantic-release refuses once the tag exists).
+on:
+  workflow_dispatch:
+
+env:
+  NODE_VERSION: '24.5.0'
+
+jobs:
+  typedoc:
+    runs-on: ubuntu-latest
+    # One deploy per site at a time: `surfer deploy` swaps the site atomically,
+    # so two runs racing is not corruption, but it is a wasted build and the
+    # last swap wins.
+    concurrency:
+      group: surfer-webdoc
+      cancel-in-progress: false
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: ${{ env.NODE_VERSION }}
+
+      - name: Write .npmrc file
+        run: |
+          printf '%s\n' "$NPMRC" >> client/.npmrc
+          echo "legacy-peer-deps=true" >> client/.npmrc
+        shell: bash
+        env:
+          NPMRC: ${{ secrets.NPMRC }}
+
+      - name: Install dependencies
+        run: |
+          cd client
+          # Decrypt and Install sheet
+          echo "$SHEET_PWD" | gpg --batch --yes --passphrase-fd 0 ./libraries/sheet-crypto.tgz.gpg
+          npm ci
+        shell: bash
+        env:
+          SHEET_PWD: ${{ secrets.SHEET_PWD }}
+
+      - name: Build typedoc
+        run: |
+          cd client
+          npm run compodoc:build
+
+      # Deploy tooling is pinned to an exact version: an unpinned `latest` would
+      # let a breaking or compromised upstream release publish straight to the
+      # live site.  `deploy` (v7) replaces the site atomically, and it takes the
+      # Cloudron username plus an app password - the v6 `put --token` flow this
+      # replaces is gone.
+      #
+      # webdoc is a NAMED site on the app whose default site is the marketing
+      # site, so the name and its alias domain are both passed.  Surfer picks
+      # the target only from `--site-name` (it travels as the Surfer-Deployment
+      # header); without it `deploy` replaces the app's DEFAULT site, which is
+      # what serves datacontroller.io - so an unnamed deploy published the
+      # Typedoc over the top of the marketing site instead of to webdoc.
+      - name: Deploy typedoc
+        env:
+          SURFER_PASSWORD: ${{ secrets.WEBDOC_DATACONTROLLER_IO }}
+          REF_NAME: ${{ gitea.ref_name }}
+          COMMIT_SHA: ${{ gitea.sha }}
+        run: |
+          set -euo pipefail
+          npx --yes @cloudron/surfer@7.2.0 deploy client/documentation \
+            --server webdoc.datacontroller.io \
+            --site-name webdoc \
+            --domain webdoc.datacontroller.io \
+            --username allan \
+            --password "$SURFER_PASSWORD" \
+            --message "Typedoc from ${REF_NAME} @ ${COMMIT_SHA:0:8}"
+
+  sas-docs:
+    runs-on: ubuntu-latest
+    concurrency:
+      group: surfer-code-datacontroller
+      cancel-in-progress: false
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: ${{ env.NODE_VERSION }}
+
+      - name: Install dependencies
+        run: |
+          # `sasjs doc` shells out to doxygen
+          apt-get update
+          apt-get install -y doxygen
+          cd sas
+          npm ci
+          CLI_VERSION=$(node -p "require('./package.json').dependencies['@sasjs/cli']")
+          npm i -g "@sasjs/cli@${CLI_VERSION}"
+
+      - name: Build SAS docs
+        run: |
+          cd sas
+          sasjs doc
+
+      # No `--site-name` here on purpose: code.datacontroller.io is its own app
+      # and the SAS docs are its DEFAULT site, so naming one would create a
+      # separate site and leave the served copy stale.  Do not copy the named
+      # form from the typedoc job above - that app's default site is the
+      # marketing site, which is why it must not be the target.
+      - name: Deploy SAS docs
+        env:
+          SURFER_PASSWORD: ${{ secrets.CODE_DATACONTROLLER_IO }}
+          REF_NAME: ${{ gitea.ref_name }}
+          COMMIT_SHA: ${{ gitea.sha }}
+        run: |
+          set -euo pipefail
+          npx --yes @cloudron/surfer@7.2.0 deploy sas/sasjsbuild/sasdocs \
+            --server code.datacontroller.io \
+            --username allan \
+            --password "$SURFER_PASSWORD" \
+            --message "SAS docs from ${REF_NAME} @ ${COMMIT_SHA:0:8}"

.gitea/workflows/lighthouse.yaml

@@ -33,6 +33,7 @@ jobs:
           echo NODE_PATH=node >> .env
           echo CORS=enable >> .env
           echo WHITELIST=http://localhost:4200 >> .env
+          echo HTTP_PORT=5000 >> .env
           # The server refuses to run as root unless RUN_AS names an account
           # (a security change in the recent releases). This job runs as root
           # in a throwaway container, so opt in explicitly.

.gitea/workflows/release-assets.yaml

@@ -0,0 +1,209 @@
+name: Release assets
+run-name: Building and uploading the release assets
+
+# Builds the downloadable artefacts for an existing release and uploads them.
+#
+# The release job cuts the tag before it builds, so a failure anywhere between
+# the semantic-release step and the asset upload leaves a published release with
+# no files - and that job cannot simply be re-run, because semantic-release
+# refuses once the tag exists. This workflow rebuilds the artefacts from the tag
+# and attaches them, so the release can be completed without cutting a new one.
+on:
+  workflow_dispatch:
+    inputs:
+      tag:
+        description: 'Release tag to build and upload (blank = the latest release)'
+        required: false
+        default: ''
+
+env:
+  NODE_VERSION: '24.5.0'
+
+permissions:
+  contents: write
+
+jobs:
+  assets:
+    runs-on: ubuntu-latest
+    # Two runs uploading the same asset names would race the delete-then-upload
+    # below.
+    concurrency:
+      group: release-assets
+      cancel-in-progress: false
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+      - uses: actions/setup-node@v4
+        with:
+          node-version: ${{ env.NODE_VERSION }}
+
+      - name: Write .npmrc file
+        run: |
+          printf '%s\n' "$NPMRC" >> client/.npmrc
+          echo "legacy-peer-deps=true" >> client/.npmrc
+        shell: bash
+        env:
+          NPMRC: ${{ secrets.NPMRC }}
+
+      - name: Install system packages
+        # jq parses the release JSON below, so it is installed before the tag
+        # resolve.
+        run: |
+          apt-get update
+          apt-get install -y zip jq
+
+      - name: Resolve the release tag
+        env:
+          GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
+          REQUESTED_TAG: ${{ inputs.tag }}
+        run: |
+          set -euo pipefail
+          AUTH_HEADER="Authorization: token $GITEA_TOKEN"
+          BASE="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"
+
+          if [ -n "${REQUESTED_TAG}" ]; then
+            TAG="${REQUESTED_TAG}"
+            # Fail here rather than after a ten-minute build if the tag is not
+            # cut yet or has no release.
+            curl --fail-with-body -sS -H "$AUTH_HEADER" "$BASE/releases/tags/${TAG}" > /dev/null
+          else
+            TAG=$(curl --fail-with-body -sS -H "$AUTH_HEADER" "$BASE/releases/latest" | jq -r '.tag_name')
+          fi
+          echo "Building assets for ${TAG}"
+          echo "TAG=${TAG}" >> "$GITHUB_ENV"
+
+      - name: Check out the tag
+        run: |
+          set -euo pipefail
+          git fetch --force --tags origin "+refs/tags/${TAG}:refs/tags/${TAG}"
+          git checkout --force "refs/tags/${TAG}"
+
+      - name: Install the SASjs CLI the tag pins
+        run: |
+          cd sas
+          CLI_VERSION=$(node -p "require('./package.json').dependencies['@sasjs/cli']")
+          npm i -g "@sasjs/cli@${CLI_VERSION}"
+
+      - name: Frontend build
+        run: |
+          cd client
+          # Decrypt and Install sheet
+          echo "$SHEET_PWD" | gpg --batch --yes --passphrase-fd 0 ./libraries/sheet-crypto.tgz.gpg
+          npm ci
+          npm i webpack
+          npm run build
+        shell: bash
+        env:
+          SHEET_PWD: ${{ secrets.SHEET_PWD }}
+
+      - name: Build SAS9 EBI Release
+        run: |
+          cd sas
+          npm i
+          sasjs c -t sas9
+          rm -rf sasjsbuild/tests
+          sasjs b -t sas9
+          cp sasjsbuild/mysas9deploy.sas ./demostream_sas9.sas
+          # remove streamed component and rebuild SAS 9 services
+          rm -rf sasjsbuild/services/web9
+          rm sasjsbuild/services/clickme.sas
+          sasjs b -t sas9
+          cp sasjsbuild/mysas9deploy.sas ./sas9.sas
+
+      - name: Build SASjs Server Release
+        run: |
+          cd sas
+          cp sasjs/utils/favicon.ico ../client/dist/favicon.ico
+          sasjs c -t server
+          rm -rf sasjsbuild/tests
+          server_apploc="/Public/app/dc"
+          sed -i "s|apploc=\"[^\"]*\"|apploc=\"${server_apploc}\"|g" sasjsbuild/services/web/index.html
+          sasjs b -t server
+          cp sasjsbuild/server.json.zip ./sasjs_server.json.zip
+
+      - name: Build Viya Release
+        run: |
+          cd sas
+          sasjs c -t viya
+          rm -rf sasjsbuild/tests
+          sed -i -e 's/servertype="SASJS"/servertype="SASVIYA"/g' sasjsbuild/services/DC.html
+          sasjs b -t viya
+          cp sasjsbuild/viya.sas ./viya.sas
+          # compile Viya Full deploy (without web)
+          rm -rf sasjsbuild/services/web
+          rm sasjsbuild/services/DC.html
+          sasjs b -t viya
+          cp sasjsbuild/viya.sas ./viya_noweb.sas
+          cp sasjsbuild/viya.json ./viya_noweb.json
+
+      - name: Zip Frontend (including viya.json for full viya deploy)
+        run: |
+          cd sas
+          cp sasjsbuild/viya.json ../client/dist/viya.json
+          # Zip from *inside* dist so the archive holds the frontend files at its
+          # root rather than under a client/dist/ prefix (see #147).
+          cd ../client/dist
+          zip -r ../../frontend.zip .
+
+      - name: Upload assets to the release
+        env:
+          GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
+        run: |
+          set -euo pipefail
+          AUTH_HEADER="Authorization: token $GITEA_TOKEN"
+          BASE="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"
+
+          RELEASE_JSON=$(curl --fail-with-body -sS -H "$AUTH_HEADER" "$BASE/releases/tags/${TAG}")
+          RELEASE_ID=$(echo "$RELEASE_JSON" | jq -r '.id')
+          RELEASE_BODY=$(echo "$RELEASE_JSON" | jq -r '.body')
+
+          ASSETS=(frontend.zip
+                  sas/demostream_sas9.sas
+                  sas/viya.sas
+                  sas/sasjs_server.json.zip
+                  sas/sas9.sas
+                  sas/viya_noweb.sas
+                  sas/viya_noweb.json)
+
+          : > SHA256SUMS
+          for f in "${ASSETS[@]}"; do
+            sum=$(sha256sum "$f" | cut -d' ' -f1)
+            printf '%s  %s\n' "$sum" "${f##*/}" >> SHA256SUMS
+          done
+          cat SHA256SUMS
+
+          # Replacing the assets means removing the ones this workflow uploads,
+          # and a second run must not append the verification section twice. An
+          # asset the release carries from another source is left alone.
+          OWNED=$(printf '%s\n' "${ASSETS[@]##*/}" SHA256SUMS)
+          echo "$RELEASE_JSON" | jq -r '.assets[]? | "\(.id) \(.name)"' |
+            while read -r asset_id asset_name; do
+              if printf '%s\n' "$OWNED" | grep -qxF -- "$asset_name"; then
+                echo "Removing previous ${asset_name}"
+                curl --fail-with-body -sS -X DELETE -H "$AUTH_HEADER" "$BASE/releases/${RELEASE_ID}/assets/${asset_id}"
+              else
+                echo "Leaving ${asset_name} in place: this workflow did not upload it"
+              fi
+            done
+
+          if echo "$RELEASE_BODY" | grep -q "## Verifying the download"; then
+            echo "Release body already carries the verification section"
+          else
+            VERIFY_SECTION=$(./.gitea/scripts/verify-section.sh)
+            NEW_BODY=$(jq -nr --arg body "$RELEASE_BODY" --arg verify "$VERIFY_SECTION" \
+              '$body + "\n" + $verify + "\n\nFor installation instructions, please visit https://docs.datacontroller.io/"')
+            curl --fail-with-body -sS -X PATCH \
+              -H "$AUTH_HEADER" \
+              -H 'Content-Type: application/json' \
+              --data "$(jq -n --arg body "$NEW_BODY" '{draft:false, body:$body}')" \
+              "$BASE/releases/$RELEASE_ID"
+          fi
+
+          URL="$BASE/releases/${RELEASE_ID}/assets"
+          for f in "${ASSETS[@]}"; do
+            echo "Uploading $f ..."
+            curl --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@$f"
+          done
+          echo "Uploading SHA256SUMS ..."
+          curl --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@SHA256SUMS"

.gitea/workflows/release.yaml

@@ -107,6 +107,16 @@ jobs:
           echo ${{ secrets.SHEET_PWD }} | gpg --batch --yes --passphrase-fd 0 ./libraries/sheet-crypto.tgz.gpg
           npm ci
 
+      - name: Build frontend for web streaming
+        # The server-ci target streams client/dist as the 'clickme' web app, so
+        # the production build has to exist before the mocked services are
+        # deployed. Without it `sasjs cbd` reports
+        # "webSourcePath: .../client/dist ... doesn't exist", streams no
+        # frontend and the deploy step fails.
+        run: |
+          cd client
+          npm run build
+
       - name: Setup and start SASjs server
         run: |
           npm i -g pm2
@@ -117,6 +127,7 @@ jobs:
           echo NODE_PATH=node >> .env
           echo CORS=enable >> .env
           echo WHITELIST=http://localhost:4200 >> .env
+          echo HTTP_PORT=5000 >> .env
           # The server refuses to run as root unless RUN_AS names an account
           # (a security change in the recent releases). This job runs as root
           # in a throwaway container, so opt in explicitly.
@@ -193,11 +204,15 @@ jobs:
     # and its permissions are derived from this block.
     #   contents: write -> push CHANGELOG.md / package.json commit + tag, and
     #                      create the Release via the Gitea API.
+    #   actions: write  -> dispatch the Sites workflow for the new tag (that
+    #                      route is scoped to the actions unit, which contents
+    #                      does not grant).
     # The TSDOC_TOKEN and CODE_DATACONTROLLER_IO secrets continue to be used
     # because they target *different* servers (webdoc.datacontroller.io and
     # code.datacontroller.io), not the Gitea API.
     permissions:
       contents: write
+      actions: write
 
     steps:
       - uses: actions/checkout@v4
@@ -266,6 +281,11 @@ jobs:
 
           semantic-release
 
+          # The tag this job cuts, for the steps that publish against it: a
+          # `releases/latest` lookup would pick whichever release landed last
+          # if two overlap.
+          echo "RELEASE_TAG=v$(node -p "require('./package.json').version")" >> "$GITHUB_ENV"
+
       - name: Frontend Build
         description: Must be created AFTER the release as the version (git tag) is used in the interface
         run: |
@@ -326,19 +346,6 @@ jobs:
           cd ../client/dist
           zip -r ../../frontend.zip .
 
-      - name: Release Typedoc
-        run: |
-          cd client
-          npm -g install cloudron-surfer
-          npm run compodoc:build
-          surfer put --token ${{ secrets.TSDOC_TOKEN }} --server webdoc.datacontroller.io documentation/* /
-
-      - name: Release code.datacontroller.io
-        run: |
-          cd sas
-          sasjs doc
-          surfer put --token ${{ secrets.CODE_DATACONTROLLER_IO }} --server code.datacontroller.io sasjsbuild/sasdocs/* /
-
       - name: Upload assets to release
         env:
           # Use the same ephemeral per-job Gitea Actions token that the
@@ -353,7 +360,7 @@ jobs:
           AUTH_HEADER="Authorization: token ${GITEA_TOKEN}"
           BASE="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"
 
-          RELEASE_JSON=$(curl -k --fail-with-body -sS -H "$AUTH_HEADER" "$BASE/releases/latest")
+          RELEASE_JSON=$(curl --fail-with-body -sS -H "$AUTH_HEADER" "$BASE/releases/tags/${RELEASE_TAG}")
           RELEASE_ID=$(echo "$RELEASE_JSON" | jq -r '.id')
           RELEASE_BODY=$(echo "$RELEASE_JSON" | jq -r '.body')
 
@@ -387,7 +394,7 @@ jobs:
           VERIFY_SECTION=$(./.gitea/scripts/verify-section.sh)
           NEW_BODY=$(jq -nr --arg body "$RELEASE_BODY" --arg verify "$VERIFY_SECTION" \
             '$body + "\n" + $verify + "\n\nFor installation instructions, please visit https://docs.datacontroller.io/"')
-          curl -k --fail-with-body -sS -X PATCH \
+          curl --fail-with-body -sS -X PATCH \
             -H "$AUTH_HEADER" \
             -H 'Content-Type: application/json' \
             --data "$(jq -n --arg body "$NEW_BODY" '{draft:false, body:$body}')" \
@@ -397,7 +404,58 @@ jobs:
           URL="$BASE/releases/$RELEASE_ID/assets"
           for f in "${ASSETS[@]}"; do
             echo "Uploading $f ..."
-            curl -k --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@$f"
+            curl --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@$f"
           done
           echo "Uploading SHA256SUMS ..."
-          curl -k --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@SHA256SUMS"
+          curl --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@SHA256SUMS"
+
+      - name: Publish the code documentation
+        # Typedoc (webdoc) and the SAS docs (code) are published by docs.yaml,
+        # not inline here. Two reasons: the v6 `surfer put --token` flow is gone
+        # in Surfer 7, which takes a Cloudron username plus an app password; and
+        # a dispatch means a stale or failed docs deploy can be republished
+        # without re-running this job, which semantic-release refuses once the
+        # tag exists.
+        #
+        # This runs AFTER the release assets are uploaded: the documentation is
+        # a side channel, and a docs problem must not cost a cut release its
+        # artefacts.
+        env:
+          GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
+        run: |
+          set -euo pipefail
+          AUTH_HEADER="Authorization: token $GITEA_TOKEN"
+          BASE="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"
+
+          TAG="${RELEASE_TAG}"
+          echo "Dispatching code documentation for ${TAG}"
+
+          curl --fail-with-body -sS -X POST \
+            -H "$AUTH_HEADER" \
+            -H 'Content-Type: application/json' \
+            --data "$(jq -n --arg ref "refs/tags/$TAG" '{ref: $ref}')" \
+            "$BASE/actions/workflows/docs.yaml/dispatches"
+
+      - name: Publish the demo site for this release
+        # The tag is created by this job's semantic-release step, so it carries
+        # the Actions user as its author - and the forge suppresses ref events
+        # from that user, which means `on: push: tags` in sites.yaml never fires.
+        # Dispatch it instead: the API call is authenticated, and the run it
+        # starts checks out the tag the same way.
+        env:
+          GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
+        run: |
+          set -euo pipefail
+          AUTH_HEADER="Authorization: token ${GITEA_TOKEN}"
+          BASE="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"
+
+          TAG="${RELEASE_TAG}"
+          echo "Dispatching Sites for ${TAG}"
+
+          # the dispatch endpoint resolves a bare name as a branch; a tag needs
+          # the refs/tags/ prefix
+          curl --fail-with-body -sS -X POST \
+            -H "$AUTH_HEADER" \
+            -H 'Content-Type: application/json' \
+            --data "$(jq -n --arg ref "refs/tags/$TAG" '{ref: $ref}')" \
+            "$BASE/actions/workflows/sites.yaml/dispatches"

.gitea/workflows/sites.yaml

@@ -75,6 +75,7 @@ jobs:
           touch .env
           echo RUN_TIMES=js >> .env
           echo NODE_PATH=node >> .env
+          echo HTTP_PORT=5000 >> .env
           # The server refuses to run as root unless RUN_AS names an account
           echo RUN_AS=root >> .env
           nohup ./api-linux > server.log 2>&1 &
@@ -113,6 +114,39 @@ jobs:
 
       - name: Build the browser-hosted site
         run: |
+          # The published sites ship a working licence so they are usable beyond
+          # the free tier. The pair is committed in sas/mocks/mock-licence.json
+          # rather than minted here, so every deploy of both sites carries the
+          # same key; the CI estate stays unlicensed because that file is outside
+          # the deployed sync folder, so the specs still exercise the free tier.
+          # The pair must round trip through the same WebCrypto the app uses,
+          # because a committed pair can drift (one half regenerated, a value
+          # truncated on copy) and nothing else would notice before publishing.
+          # Identical values in the two fields is the HTTP-format marker the app
+          # refuses in a secure context, so refuse to publish one.
+          node -e "
+            const d = require('./sas/mocks/mock-licence.json')
+            const b = (s) => Buffer.from(String(s || ''), 'base64')
+            if (!d.licenceKey || !d.activationKey || d.licenceKey === d.activationKey) {
+              console.error('sas/mocks/mock-licence.json must hold an HTTPS-format key pair')
+              process.exit(1)
+            }
+            crypto.subtle
+              .importKey('pkcs8', b(d.activationKey), { name: 'RSA-OAEP', hash: 'SHA-256' }, false, ['decrypt'])
+              .then((key) => crypto.subtle.decrypt({ name: 'RSA-OAEP', hash: 'SHA-256' }, key, b(d.licenceKey)))
+              .then((plain) => {
+                const data = JSON.parse(new TextDecoder().decode(plain))
+                console.log('licence pair round trips: site', data.site_id, 'until', data.valid_until)
+              })
+              .catch((e) => {
+                console.error('sas/mocks/mock-licence.json does not decrypt:', e.message || e)
+                process.exit(1)
+              })
+          "
+          LICENCE_DIR="sasjs_root/drive/files$APP_LOC/mock-storage"
+          mkdir -p "$LICENCE_DIR"
+          cp sas/mocks/mock-licence.json "$LICENCE_DIR/licence.json"
+
           # DC's data files exist only after makedata has run, so the drive is
           # captured first and shipped as the site's starting drive. services/web9
           # is the streamed frontend, which is already the site itself.

.gitleaksignore

@@ -0,0 +1,5 @@
+# sas/mocks/mock-licence.json holds a licence key pair for the mock estate: the
+# activation key is RSA key material the browser demo decrypts with, not a
+# credential for any service.
+sas/mocks/mock-licence.json:generic-api-key:2
+sas/mocks/mock-licence.json:generic-api-key:3

AGENTS.md

@@ -32,6 +32,8 @@ Never consider a change complete until the relevant linters pass on the files yo
 
 - **Client (TypeScript/HTML/etc.)**: run `npm run lint:check` from the `client/` directory (prettier). Fix any failures with `npm run lint:fix`.
 - **SAS**: after creating or modifying any `.sas` files, run `sasjs lint` from the `sas/` directory and ensure the files you touched have no lint warnings (the repo currently has pre-existing warnings in other files, which can be ignored).
+  `sas/.sasjslint` carries `ignoredMacros: ["webout", "mpeinit", "mpeterm"]`. `webout` is the linter's own default - `sasjs compile` injects `%webout` into every service - and an explicit list replaces that default, so it has to be restated there. `mpeinit` and `mpeterm` are listed because every service is compiled with a serviceInit program (`sasjs/utils/serviceinitsas9|viya|server.sas`) that declares both macros in its own header, so a service body may call them without declaring them. Add a macro to that list only when it reaches a file the same way - injected by the compiler, through serviceInit, testSetUp or another implicit provider - never to silence a genuinely missing declaration.
+  `noBannedStrings` flags the literal words `error ` and `warning ` anywhere in a file, comments included. Reword prose; where the text has to stay (a real `%put`, a doxygen `@warning` tag), keep the log output identical with `%put %str(WARN)ING:` rather than a per-file override, and use `@sasjslint {"noBannedStrings": false}` only when the word itself is the content.
 
 ## No external assets
 
@@ -41,7 +43,7 @@ Data Controller must run entirely locally (offline / on-prem, no internet access
 
 Agent-related content lives in `.agents/`: technical/agent-facing documentation goes in `.agents/docs/` (not `docs/`), and skills in `.agents/skills/`. When writing explanatory or technical docs about the codebase, put them in `.agents/docs/`.
 
-Skills installed via `npx skills add` are tracked in `skills-lock.json` and live under `.agents/skills/`. Repo-local custom skills (e.g. `dc-sas` for Data Controller SAS development, `handsontable`, `hyperformula`) are kept here too. Avoid editing npx-managed skill files directly — use `npx skills update` instead.
+Skills installed via `npx skills add` are tracked in `skills-lock.json` and live under `.agents/skills/`. Repo-local custom skills (e.g. `dc-sas` for Data Controller SAS development, `dc-estate-testing` for estate deploys and test runs, `handsontable`, `hyperformula`) are kept here too. Avoid editing npx-managed skill files directly — use `npx skills update` instead.
 
 ## Code comments and test names
 

CHANGELOG.md

@@ -1,3 +1,167 @@
+# [7.21.0](https://git.datacontroller.io/dc/dc/compare/v7.20.0...v7.21.0) (2026-10-29)
+
+
+### Bug Fixes
+
+* **client:** parse word-date values from VA parameters ([405b28d](https://git.datacontroller.io/dc/dc/commit/405b28dcccfe5994521bf63f0270d4877f544294))
+* **client:** parse word-date values from VA parameters ([ce5b1aa](https://git.datacontroller.io/dc/dc/commit/ce5b1aa96175d0fd1f048fea6d127cbc743804f2))
+* **makedata:** name the sasjs server group in the estate seed ([eeffa92](https://git.datacontroller.io/dc/dc/commit/eeffa92f9e973b323171bbefe2ef1cbc27e3aee7))
+
+
+### Features
+
+* **derived:** apply the core field scan in the MPE_DERIVED hook ([a4a0f9c](https://git.datacontroller.io/dc/dc/commit/a4a0f9cb2f7c0b6252405b71df64aae912d1b075))
+
+# [7.20.0](https://git.datacontroller.io/dc/dc/compare/v7.19.0...v7.20.0) (2026-10-28)
+
+
+### Bug Fixes
+
+* **ci:** install doxygen for the SAS docs job ([f58925b](https://git.datacontroller.io/dc/dc/commit/f58925b29956f1f753f0a367fdcd0afe7f3f51a0))
+* **ci:** keep foreign release assets, install jq first, pin the CLI ([27686d3](https://git.datacontroller.io/dc/dc/commit/27686d3aad5e34ace59ca1b735a17cca1a4025e1))
+* **ci:** publish against the tag this job cuts, not releases/latest ([8888734](https://git.datacontroller.io/dc/dc/commit/888873421c3aeaf312d5ec45e3c805317c66976a))
+* **ci:** publish docs after the assets, with the webdoc app password ([dd482c4](https://git.datacontroller.io/dc/dc/commit/dd482c4069202a0c6b9983129c19efc09bd160a7))
+* **ci:** publish the code documentation with surfer 7 and app passwords ([25ffa71](https://git.datacontroller.io/dc/dc/commit/25ffa71adddba5e334f4a8c166f284d1be498fcd))
+* **ci:** publish the typedoc to the webdoc site, not the app default ([c5808a1](https://git.datacontroller.io/dc/dc/commit/c5808a16eba0d29562fcc76f13ac4a33ab8da0b7))
+* **ci:** rebuild and upload release assets on demand ([6486c80](https://git.datacontroller.io/dc/dc/commit/6486c80ed4f368bbf116c0ecdecc0e5e5f02075f))
+* **ci:** use the lockfile and the pinned CLI in the docs job ([af1813f](https://git.datacontroller.io/dc/dc/commit/af1813f829861df41c89b512c824e8d3ebac7a4b))
+* **ci:** verify the forge certificate on the release API calls ([cf7b21a](https://git.datacontroller.io/dc/dc/commit/cf7b21a8b61ec3bf7ebe21968ccf13ade264bc64))
+* **cypress:** hold download out too, and retry once in CI ([c97e886](https://git.datacontroller.io/dc/dc/commit/c97e8866db9ee404b2118d24baf53fc7058debef)), closes [#395](https://git.datacontroller.io/dc/dc/issues/395) [#398](https://git.datacontroller.io/dc/dc/issues/398)
+* **cypress:** register the deleteFolder task, restore the spec order, hold metanav out ([c803773](https://git.datacontroller.io/dc/dc/commit/c803773d3c36476d8263f01fc3b8344aa3503d44)), closes [#395](https://git.datacontroller.io/dc/dc/issues/395)
+* **deploy:** do not offer the deploy wizard inside a report ([80e8911](https://git.datacontroller.io/dc/dc/commit/80e8911af8857fa2c417f028cd59b64a87233b9e))
+* **deploy:** keep the app query when opening the standalone URL ([e51953c](https://git.datacontroller.io/dc/dc/commit/e51953ca8a6734e1649bad68cf4ad2057d677334))
+* **deploy:** one definition of the standalone URL, and test the guards ([459ff2d](https://git.datacontroller.io/dc/dc/commit/459ff2da371375e996839d69deb40f681c356f1d)), closes [#TCS](https://git.datacontroller.io/dc/dc/issues/TCS)
+* **deploy:** strip the embed parameters by name, and scope the timezone runs to the VA specs ([f7e7538](https://git.datacontroller.io/dc/dc/commit/f7e753863b61d8545bd0777d80267661b3c58670))
+* **embed:** build quarter ends from the calendar, not minus 24 hours ([7e3045a](https://git.datacontroller.io/dc/dc/commit/7e3045a21b67053b76a81bfe836e31571ccbffbf))
+* **embed:** defer the column check until metadata loads, report AND-ed pairs ([2623a3f](https://git.datacontroller.io/dc/dc/commit/2623a3fb4ba3bf19c00636b1717b53d5a2d54f3a))
+* **embed:** log a parse-time mapping problem once ([0011f7e](https://git.datacontroller.io/dc/dc/commit/0011f7e933d85be8e33debe3a6ecb9000edaabe3))
+* **embed:** make the va_parmap diagnostics say what they mean ([541a61b](https://git.datacontroller.io/dc/dc/commit/541a61ba08bc7256a911c50de7078859cc46c84e))
+* **embed:** match a mapping on the parameter label only, and warn only when the values differ ([e9216b0](https://git.datacontroller.io/dc/dc/commit/e9216b08956d5599662259a2b05cc1524c458df5))
+* **embed:** never build a clause from a value that did not convert, and say so in the banner ([bd3c220](https://git.datacontroller.io/dc/dc/commit/bd3c220af2678a2a3f8394fe3084ee03c23d0b15))
+* **embed:** one parameter maps one column, and report typos on both sides ([f83dc35](https://git.datacontroller.io/dc/dc/commit/f83dc359e258f5d1cb9fc49f0eaab9748995d823))
+* **embed:** record the parameter label only, so a name-keyed va_parmap is reported ([aa91daf](https://git.datacontroller.io/dc/dc/commit/aa91daf9f78c00cea486d0f885f8e634366bd1a0))
+* **embed:** resolve transform values as local dates and give ranges whole days ([10157c1](https://git.datacontroller.io/dc/dc/commit/10157c11595685efdc206fe7b16a5cafa098f26d))
+* **embed:** warn when a va_parmap entry is ignored ([411f6c3](https://git.datacontroller.io/dc/dc/commit/411f6c3354b7fbfec11fef3b174ae1f8066c92ba))
+* **makedata:** pair the %else of the keepme block so it parses in open code ([c0ea810](https://git.datacontroller.io/dc/dc/commit/c0ea810766ebfda5ae5358ab14146fbcc07fcc8e))
+
+
+### Features
+
+* **embed:** map VA parameters to DC columns from the URL ([15a1f34](https://git.datacontroller.io/dc/dc/commit/15a1f34d5263503cac83b4fe4e52482777dc2f1f))
+* **embed:** resolve va_parmap values through named date anchors ([88010c3](https://git.datacontroller.io/dc/dc/commit/88010c3980c2ea1c39bbb024ed32d63e495e61f0))
+
+# [7.19.0](https://git.datacontroller.io/dc/dc/compare/v7.18.0...v7.19.0) (2026-10-28)
+
+
+### Bug Fixes
+
+* address the review findings on the harness and refresh changes ([36e4e88](https://git.datacontroller.io/dc/dc/commit/36e4e88eb134d3b48217d6dd185ea3c03f03fba5))
+* **ci:** dispatch the demo deploy with a tag ref and the actions scope ([698dea6](https://git.datacontroller.io/dc/dc/commit/698dea6d6b7a8635313f11acfcc7a4f39a02bc99))
+* **ci:** restore the gitleaks ignore entries for the mock licence pair ([0d58be9](https://git.datacontroller.io/dc/dc/commit/0d58be975d8eb2d7f1ab6a648f760151392c7973))
+* **client:** open the SAS log page with noopener ([072a133](https://git.datacontroller.io/dc/dc/commit/072a1337020970b5b4ff8aa31719cc5ffec7701e))
+* **client:** refuse a truncated table in the multi-table upload ([1fb4818](https://git.datacontroller.io/dc/dc/commit/1fb4818d215f538a0ac51772ab1f261aeb91f7e1))
+* **cypress:** apply the licence key when the estate has none ([be8f445](https://git.datacontroller.io/dc/dc/commit/be8f445ae1051e132fb6eb5a69b45264255326b5)), closes [input#file-upload](https://git.datacontroller.io/input/issues/file-upload)
+* **dc_execute:** use = comparisons, not IN, in the fail-fast guard ([fed43cb](https://git.datacontroller.io/dc/dc/commit/fed43cb07ef7f2afc685612a124dbc16d7618e21))
+* **dc_getlibs:** exclude the DC libref from the dictionary scan ([f5aade5](https://git.datacontroller.io/dc/dc/commit/f5aade59760cc376b0f96eb4c892c8bca82dde14))
+* **dc_refreshcatalog:** terminate the generated statements ([1e22476](https://git.datacontroller.io/dc/dc/commit/1e224763ba80500a112812a983801968e3751959))
+* **derived:** a fileref with nothing written is a FAIL, not an error ([9b3cf50](https://git.datacontroller.io/dc/dc/commit/9b3cf50b9c3889786605f0f5d9032d84c50f7107))
+* **derived:** address five review findings ([ccd468b](https://git.datacontroller.io/dc/dc/commit/ccd468b3358c1418ee6d8e198b088a0cf23b8c24))
+* **derived:** carry the placement sequence in mpe_derived ([be50cb5](https://git.datacontroller.io/dc/dc/commit/be50cb5d082e788c6ae106bc1d02e73a8c7b9646))
+* **derived:** close four gaps in the MPE_DERIVED hook ([9914790](https://git.datacontroller.io/dc/dc/commit/9914790777f92128c74ecce67e6c67ee48824b45))
+* **derived:** close the RLS and search gaps on derived columns ([9ef813d](https://git.datacontroller.io/dc/dc/commit/9ef813d439b48e7fefd35c426e62b99e5943002a))
+* **derived:** close the two gaps in the derived/RLS invariant ([422f766](https://git.datacontroller.io/dc/dc/commit/422f766cd43db4ce740e1890fd5df30790eaa7e3))
+* **derived:** compare the outref as text, prove the file is empty ([7830b7b](https://git.datacontroller.io/dc/dc/commit/7830b7bcfa9806dde840ce8dd3e6a1887d0b38be))
+* **derived:** do not drop a shadowed column from staging or downloads ([2d5de9a](https://git.datacontroller.io/dc/dc/commit/2d5de9a324a07a36edc0a4f74c8fad09fde07cbf))
+* **derived:** do not duplicate a shadowed column in the grid payload ([6a1128a](https://git.datacontroller.io/dc/dc/commit/6a1128a46352f77c95dd02df9309f0984a1db2d1))
+* **derived:** eight review findings ([561d7fc](https://git.datacontroller.io/dc/dc/commit/561d7fc3ed2a460835619962cf4dcb29d3dc87c7))
+* **derived:** enforce the ACTIVE picker pair in the extracted check ([1863076](https://git.datacontroller.io/dc/dc/commit/1863076bc956b4ff94bf7b85b3dde0d6e3b54d7a)), closes [#366](https://git.datacontroller.io/dc/dc/issues/366)
+* **derived:** enforce the documented limits, and handle derived filters in VIEW ([d3dec68](https://git.datacontroller.io/dc/dc/commit/d3dec682aafeb143fe983d6546337cb6490f5620))
+* **derived:** filtering a format catalog no longer aborts the service ([469229a](https://git.datacontroller.io/dc/dc/commit/469229aecbaad304a835a5f229df111c37ff7c0d))
+* **derived:** guard the bitemporal cut-off, coalesce the selectbox seed ([7fb981d](https://git.datacontroller.io/dc/dc/commit/7fb981daa91bce29c2d62c246185d01dcad08d6b))
+* **derived:** guard the empty physical column list in the dep check ([5f5d868](https://git.datacontroller.io/dc/dc/commit/5f5d8681b9d78280a5200309917415e135c3ce5b))
+* **derived:** hold a derived column read-only whatever the table's security ([5f443b0](https://git.datacontroller.io/dc/dc/commit/5f443b0b16a2abd3c11d96caa7a043911c29ef01))
+* **derived:** ignore a re-added name when collecting inactive rows ([6b5bf3e](https://git.datacontroller.io/dc/dc/commit/6b5bf3e6be89b22e5a633c34b9aecbc8c7d48ee2))
+* **derived:** ignore inactive rows and VIEW rules in the submit guards ([85c89ca](https://git.datacontroller.io/dc/dc/commit/85c89ca2a4e02112ba496ac36a222f33675544da))
+* **derived:** keep targetds for plain datasets, test the empty file ([f8f6568](https://git.datacontroller.io/dc/dc/commit/f8f65681e1489f4ce94d9127d5e4c0c82798e960))
+* **derived:** make the model enforce the required columns, and leave VAR_FMT optional ([7efbf31](https://git.datacontroller.io/dc/dc/commit/7efbf31ea5ac234bd8f924fdf89a95d101b3aff1))
+* **derived:** only the delete marker is a reserved name ([af31c36](https://git.datacontroller.io/dc/dc/commit/af31c36b670ec63a2a4ba056bf6c4b28f6096e08))
+* **derived:** open the statements file with a long record length ([ee31782](https://git.datacontroller.io/dc/dc/commit/ee31782fbff909b540162b1bdfca6f307208331b))
+* **derived:** pair the filterds %else with %do/%end ([88a38db](https://git.datacontroller.io/dc/dc/commit/88a38db877dba9b98ccf50f0cc7242674654586b))
+* **derived:** raise the VAR_ORDER ceiling to 10000 ([27ddcfb](https://git.datacontroller.io/dc/dc/commit/27ddcfb5759d6eb8f9880adc7c271a408e7741ad))
+* **derived:** refuse dependencies on inactive rows, and keep the RLS scope guard coherent ([7799a30](https://git.datacontroller.io/dc/dc/commit/7799a300df82339683eea21e4a89ce7a6471c575))
+* **derived:** reject a dependency on a row this submission deletes ([8806504](https://git.datacontroller.io/dc/dc/commit/88065049674c2cc0141eef22704b759f5b708b0f))
+* **derived:** repair the validation rows for MPE_DERIVED ([9a047ad](https://git.datacontroller.io/dc/dc/commit/9a047ad42ec86df7d1eaf5058c0edde56016e88f))
+* **derived:** review feedback - cover the empty response, assert the last row ([e0da9f1](https://git.datacontroller.io/dc/dc/commit/e0da9f1098db63185538398ca478696cc849da4b))
+* **derived:** stop the hooks aborting on their own macro variables ([688550a](https://git.datacontroller.io/dc/dc/commit/688550a28d9794ac6b878926a0e1e2ef8f6e9335))
+* **derived:** truncate the derived label on an even quote boundary ([1b5c4b3](https://git.datacontroller.io/dc/dc/commit/1b5c4b3ecd962a28e08fc5671b57dbf7ae0b8d9f))
+* **derived:** verify on stripped values, and label with the variable name ([4d9ad38](https://git.datacontroller.io/dc/dc/commit/4d9ad38df1fa89b0d5a12214d703d9c1fe5295f2))
+* **editor:** a blank truncation note is not a warning ([f182179](https://git.datacontroller.io/dc/dc/commit/f18217940365f27ae388fae087e6126931e1222c))
+* **editor:** keep decimal values when an uploaded file is edited manually ([04ce186](https://git.datacontroller.io/dc/dc/commit/04ce1861e7821063b1be42c22f92f62aed509602))
+* **getcolvals:** report an invalid value, not missing inputs ([4bcee70](https://git.datacontroller.io/dc/dc/commit/4bcee706ffaf1929bdd14bb2fb15111174ca3916))
+* **getdata.test:** pin the derived filter to a libref, and spell the table as it is ([45dce7b](https://git.datacontroller.io/dc/dc/commit/45dce7b11b7a4399c1d0dc9c3e7a283b1cf59df1))
+* **getdata:** clear the globals it declares before use ([359b570](https://git.datacontroller.io/dc/dc/commit/359b570b85aa2f66507e81ea2977243be4590f83))
+* **getddl.test:** ask for a table, not a whole library ([059cdf1](https://git.datacontroller.io/dc/dc/commit/059cdf176a7ec556b45ea1b21d880c048eba14ca))
+* **getrawdata:** declare mp_ds2md so the Markdown download can run ([661ee19](https://git.datacontroller.io/dc/dc/commit/661ee19cfa5f996bf01f2b11817039f38dbb9985))
+* **lint:** clear the sasjs lint warnings on the files 7.19 touched ([355c46a](https://git.datacontroller.io/dc/dc/commit/355c46a7b64994d0146a88657dc5635593249138))
+* **makedata:** read KEEPME from every input row, and keep it out of the Viya seed ([d626cb8](https://git.datacontroller.io/dc/dc/commit/d626cb8f2daf94fde7ff21d924b9a2ca4968d983))
+* **mocks:** address the audit-pack review findings ([489819b](https://git.datacontroller.io/dc/dc/commit/489819b05111323ae1e34edda22a5446cdbc2d87))
+* **mocks:** carry the zip flags into the central directory ([c4021a4](https://git.datacontroller.io/dc/dc/commit/c4021a42ffaed2af80c53aaa1455249190b339b8))
+* **mocks:** clear the licence for published sites, not for the estate ([62f55db](https://git.datacontroller.io/dc/dc/commit/62f55db170185900063f7a431cd8f375ebabf930))
+* **mocks:** copy the physical column list before deriving ([f411b7d](https://git.datacontroller.io/dc/dc/commit/f411b7d269381e3f7cab2d85b6d8f7d7d50ea4d1))
+* **mocks:** derive before filtering in the viewer, and drop the columns ([60ec470](https://git.datacontroller.io/dc/dc/commit/60ec470404e5c279b58ac6a01768dfc83413b197))
+* **mocks:** derive the retained key wherever rows are staged ([74060c0](https://git.datacontroller.io/dc/dc/commit/74060c0585fb8b217cb181ea747ad318f5238868))
+* **mocks:** drop derived variables before staging ([73d25b1](https://git.datacontroller.io/dc/dc/commit/73d25b1220d574ef2547fbd9e6e3cf1ceda375ee))
+* **mocks:** key rk_underlying tables on their business key ([347dbb5](https://git.datacontroller.io/dc/dc/commit/347dbb53b01c47bbd562de14d5ccbae7d96407c6))
+* **mocks:** register MPE_DERIVED and seed its field rules ([ee3c878](https://git.datacontroller.io/dc/dc/commit/ee3c8787efef9cd716bdd2bf80f6cb6751461db9))
+* **mocks:** scope the derived hook's graph to the submission ([6cad23f](https://git.datacontroller.io/dc/dc/commit/6cad23fce9ac0cf64c247c40f3b7eac875fd4cfb))
+* **mocks:** search the physical columns only in the viewer ([293022d](https://git.datacontroller.io/dc/dc/commit/293022d46b27e096fd514cdda49f688612df2f5b))
+* **mocks:** stop the mock advertising invalid licence keys ([fb36379](https://git.datacontroller.io/dc/dc/commit/fb36379520028baaf4bc3f08131bfc88deab3ac3))
+* **mpe_assert_abort:** drop the unused line variable ([9c294c0](https://git.datacontroller.io/dc/dc/commit/9c294c0ab5ca8fe30a235553a298c1c68a61442f))
+* **mpe_assert_abort:** take the abort's own MSG, tolerate a missing response ([0a52d20](https://git.datacontroller.io/dc/dc/commit/0a52d20c5fc27b4adde5fc8ed6f39a58c48f62ff))
+* **mpe_execute:** fail fast when the child left no status ([a036386](https://git.datacontroller.io/dc/dc/commit/a03638667f1968c5909f496d954eeaa13f316763))
+* **mpe_execute:** no apostrophe in the SAS 9 status note ([2204085](https://git.datacontroller.io/dc/dc/commit/220408538b26016bda45e48d619cdbc3e1cbcc59))
+* **mpe_execute:** read an unknown state as unmeasured, not failed ([f2b1428](https://git.datacontroller.io/dc/dc/commit/f2b1428ec3099b2b6e3c6da6627530ef81fab201))
+* name the startupservice launcher button "Show log" ([3445107](https://git.datacontroller.io/dc/dc/commit/3445107d9a91642b67e114bcfcd81c3ce47f3f0f))
+* offer the launcher on the globvars failure modal too ([ad2da45](https://git.datacontroller.io/dc/dc/commit/ad2da453896a51f2cc7e49cebb6c84f7b47d7395))
+* **refresh:** name each catalogue load uniquely in the audit trail ([46bacdd](https://git.datacontroller.io/dc/dc/commit/46bacdd6d9ea814bcd02a1ff8aa3f447baf02f64))
+* **refresh:** one audit folder per refresh call ([56b6620](https://git.datacontroller.io/dc/dc/commit/56b6620ef402b288ce78a2a77662b651e310753e))
+* **sites:** round-trip the committed licence before publishing ([4d462d3](https://git.datacontroller.io/dc/dc/commit/4d462d310f7df23e5d5229cb0702721b7ad41bbd))
+* **test:** getdata test ([da30266](https://git.datacontroller.io/dc/dc/commit/da30266278c4fe27a72810cd5ed647cf3bd823af))
+* **tests:** enable closeouts logging ([fdcd8fa](https://git.datacontroller.io/dc/dc/commit/fdcd8fa4b3e7a7a119c662974d70dc9eea80bdae))
+* **tests:** read the work location from _SASJS_WORK ([033ffd3](https://git.datacontroller.io/dc/dc/commit/033ffd302502c7530fc44a314364f5cdb3be6ad2))
+* **ui:** one error modal at a time ([81ccf75](https://git.datacontroller.io/dc/dc/commit/81ccf758e9dab0c7632ab50bf469e88b8c4c4420))
+* **validatefilter:** build targetds without macro triggers in comments ([42355f1](https://git.datacontroller.io/dc/dc/commit/42355f17bc07bf4d44ba858984d83b9ed72cc0ba))
+* **validatefilter:** drop the dead nobs assignment, keep targetds precedence ([337ea8e](https://git.datacontroller.io/dc/dc/commit/337ea8e5e021c6f2451fa3ad8396148744118fc9))
+
+
+### Features
+
+* **dc_getlibs:** dump the library list when debug is on ([582523d](https://git.datacontroller.io/dc/dc/commit/582523dc87a253b71824359224070ccb4746a386))
+* **derived:** allow a variable to shadow a physical column ([12beeff](https://git.datacontroller.io/dc/dc/commit/12beeff69518376a042fbbc58332acc40d89d132))
+* **derived:** declare VAR_DEPENDENCIES instead of VAR_ORDER ([813bce6](https://git.datacontroller.io/dc/dc/commit/813bce685691ad252043eeab31eb5460de81b756))
+* **derived:** optional VAR_LABEL on MPE_DERIVED ([7c9ce9a](https://git.datacontroller.io/dc/dc/commit/7c9ce9a712137e9825a5f2ff52796d765423147b))
+* **derived:** pickers, and an ACTIVE default, for MPE_DERIVED ([87e794f](https://git.datacontroller.io/dc/dc/commit/87e794f769fe420f214f43b141f0b41cf88eefee))
+* **derived:** report the dependency cycle path ([cee1a95](https://git.datacontroller.io/dc/dc/commit/cee1a954ee97e17f297b7cf43ef0e5cfc6e393a6))
+* **derived:** take any text in VAR_LABEL, but not a line break ([78e88a4](https://git.datacontroller.io/dc/dc/commit/78e88a41128b047c0276ccf647f128229152604c))
+* **derived:** validate MPE_DERIVED at submit time ([0b010f2](https://git.datacontroller.io/dc/dc/commit/0b010f2c65ddada70d62ea83c706943c6c45756f))
+* **derived:** validation rules and a VAR_TYPE dropdown for MPE_DERIVED ([aa6d50e](https://git.datacontroller.io/dc/dc/commit/aa6d50e1ea5fc2fe6be8aa9fa859b92d0a1061c5))
+* **derived:** VARIABLE_ORDER, so a derived variable can build on another ([33d043e](https://git.datacontroller.io/dc/dc/commit/33d043ea92f007dca70c95d5363329d4447eeba9))
+* **editor:** derived variables in the EDIT screen ([b0fd354](https://git.datacontroller.io/dc/dc/commit/b0fd354ecdbd7bf29196ed983ff3cce691affeed))
+* **editor:** mark derived variables as computed in the EDIT grid ([6513d09](https://git.datacontroller.io/dc/dc/commit/6513d09f059de15c5304f99b3f8d898d46ea1be7))
+* **editor:** serve a truncated selection with a warning instead of aborting ([34691dc](https://git.datacontroller.io/dc/dc/commit/34691dceecb63f952b23ad8acc5bc93753c6cfee))
+* **editor:** show the derived column label in the header tooltip ([cfe6647](https://git.datacontroller.io/dc/dc/commit/cfe6647ca870c45a3d8abc2753ce7f070e3fdf4a))
+* **mocks:** add getauditfile and the audit-folder artefacts ([752d944](https://git.datacontroller.io/dc/dc/commit/752d944c2091ef6d2624a15ab2e2e7be35608798))
+* **mocks:** implement derived variables in the JS services ([b4fae88](https://git.datacontroller.io/dc/dc/commit/b4fae88ae13658607e4209449577cd42849d3a99))
+* **mocks:** validate a submitted MPE_DERIVED row like the hook does ([5d183d1](https://git.datacontroller.io/dc/dc/commit/5d183d135085ea023f32e5b1d4992d997394a8fb))
+* run startupservice in a new window from its error modal ([40f6133](https://git.datacontroller.io/dc/dc/commit/40f61331aad6bc8a52c551b924f0d1f1e6426cee))
+* **sites:** mint a licence for the published sites ([7d9df71](https://git.datacontroller.io/dc/dc/commit/7d9df71fdb0ab265e745ed3b708038983e855a87))
+
+
+### Reverts
+
+* **derived:** no shadowing, and no self reference ([5af8e6f](https://git.datacontroller.io/dc/dc/commit/5af8e6fc359221586af74a685159dfd75a40ec45))
+* **stagedata.test.4:** back to the MPE_LOADS evidence for now ([31aa859](https://git.datacontroller.io/dc/dc/commit/31aa8591538409fc1a3ef97d7474120094ce202d))
+
 # [7.18.0](https://git.datacontroller.io/dc/dc/compare/v7.17.0...v7.18.0) (2026-10-19)
 
 

CONTEXT.md

@@ -8,7 +8,7 @@ This file is the shared glossary and orientation for the repo. When your output
 
 - `client/` - the Angular frontend (TypeScript). Uses [Handsontable](.agents/skills/handsontable/SKILL.md) for the editable grid and [HyperFormula](.agents/skills/hyperformula/SKILL.md) for Excel-formula support. Lint with `npm run lint:check` from `client/`.
 - `sas/` - the SAS backend. Services, macros, hooks and tests live under `sas/sasjs/**`; `sas/sasjsbuild/` is generated output (never hand-edit). Lint `.sas` files with `sasjs lint` from `sas/`.
-- `.agents/docs/` - technical deep-dives (see below). `.agents/skills/` - task skills (handsontable, hyperformula, dc-sas for Data Controller SAS development, plus npx-installed sasjs/ sas skills).
+- `.agents/docs/` - technical deep-dives (see below). `.agents/skills/` - task skills (handsontable, hyperformula, dc-sas for Data Controller SAS development, dc-estate-testing for deploying to and testing on the shared estate, plus npx-installed sasjs/ sas skills).
 
 ## Roles
 

CONTRIBUTING.md

@@ -86,6 +86,7 @@ RUN_TIMES=js
 NODE_PATH=node
 CORS=enable
 WHITELIST=http://localhost:4200
+HTTP_PORT=5000
 ```
 
 `RUN_TIMES=js` means the server executes `.js` files as Stored Programs (no SAS binary needed). `CORS=enable` plus `WHITELIST` allows the Angular dev server on port 4200 to call the mock services on port 5000. Start the server:
@@ -94,7 +95,7 @@ WHITELIST=http://localhost:4200
 ./api-linux    # or api-macos / api-win.exe depending on your platform
 ```
 
-The server runs in desktop mode (no auth) on `http://localhost:5000` by default.
+`HTTP_PORT` is not optional in SASjs Server 3.x: `PORT` and `PROTOCOL` are deprecated, and a server started without `HTTP_PORT` or `HTTPS_PORT` exits with "no listener is enabled". With it set, the server runs in desktop mode (no auth) on `http://localhost:5000`.
 
 ### 2. Deploy the mock services and streaming frontend
 
@@ -135,7 +136,7 @@ After changing mock services, re-run `sasjs cbd -t server-ci` from `sas/mocks/sa
 
 ### Demo and staging sites
 
-A push to `release/**` publishes [staging.datacontroller.io](https://staging.datacontroller.io); a release tag (`v*`, created by the release job after its tests pass) publishes [demo.datacontroller.io](https://demo.datacontroller.io) - workflow file `.gitea/workflows/sites.yaml`, using the `DEMO_DATACONTROLLER_IO` Surfer app password (user `allan`). Both are sites on one multi-site Surfer instance; the demo is its default site. The job builds the mocks as a browser-hosted site (SASjs CLI >= 6.1, `streamConfig.mockWeb`), so the services run in a service worker and the drive lives in the visitor's IndexedDB - there is no server. It runs `makedata` on a throwaway SASjs Server first and ships that drive as `sasjsbuild/mock-snapshot.json`; without it the site opens with no data. The mock ships no licence key, so both sites run on the free tier.
+A push to `release/**` publishes [staging.datacontroller.io](https://staging.datacontroller.io); a release tag (`v*`, created by the release job after its tests pass) publishes [demo.datacontroller.io](https://demo.datacontroller.io) - workflow file `.gitea/workflows/sites.yaml`, using the `DEMO_DATACONTROLLER_IO` Surfer app password (user `allan`). Both are sites on one multi-site Surfer instance; the demo is its default site. The job builds the mocks as a browser-hosted site (SASjs CLI >= 6.1, `streamConfig.mockWeb`), so the services run in a service worker and the drive lives in the visitor's IndexedDB - there is no server. It runs `makedata` on a throwaway SASjs Server first and ships that drive as `sasjsbuild/mock-snapshot.json`; without it the site opens with no data. The estate carries no licence key, so CI runs on the free tier until a key is registered through the licensing screen (`admin/registerkey`) - the Cypress setup does that for the specs that need one. The Sites job writes a licence into the published drive, so demo and staging are fully usable. The pair lives in `sas/mocks/mock-licence.json`: the two values are one RSA key and are regenerated together, and the `.gitleaksignore` entries for it are line-scoped, so inserting a line above the pair re-fires the scanner rather than revealing a new secret.
 
 ## GUI Elements
 

client/cypress.config.ts

@@ -1,3 +1,4 @@
+import fs from 'fs'
 import { defineConfig } from 'cypress'
 
 export default defineConfig({
@@ -34,7 +35,28 @@ export default defineConfig({
 
   e2e: {
     video: true,
+    // One retry in a CI run, none interactively.  `editor.cy.ts` test 28
+    // intermittently fails on "app-loading ... continuously found" - a slow load,
+    // not an assertion - and a single retry costs a few seconds where a red
+    // pipeline costs forty minutes.  Failures that are real still fail twice, and
+    // the retried attempt's output is in the run log.
+    retries: { runMode: 1, openMode: 0 },
     setupNodeEvents(on, config) {
+      // `download.cy.ts` clears the downloads folder before each test through
+      // `cy.task('deleteFolder')`.  That task was registered in
+      // cypress/plugins/index.js, which Cypress 10+ never loads unless it is
+      // required from here - so the task had no handler and the spec failed on
+      // its first hook.
+      on('task', {
+        deleteFolder(folderName: string) {
+          return new Promise((resolve, reject) => {
+            fs.rm(folderName, { recursive: true, force: true }, (err) =>
+              err ? reject(err) : resolve(null)
+            )
+          })
+        }
+      })
+
       // Pin the browser locale so locale-formatted cells (intl-date/time/datetime)
       // render deterministically regardless of the runner's system locale.
       on('before:browser:launch', (browser, launchOptions) => {

client/cypress/clips/special-missings-clip.cy.ts

@@ -40,8 +40,8 @@ context('special missings clip (DEMO_01)', function () {
   this.beforeEach(() => {
     cy.visit(hostUrl + appLocation)
 
-    // The mock estate carries a valid licence key (mock-storage/licence.json),
+    // Assumes the estate is already activated (the licensing spec registers a
-    // so the app activates directly with no free-tier banner in shot.
+    // valid key); otherwise a licensing page would sit in shot.
     cy.get('.nav-tree', { timeout: longerCommandTimeout }).should('exist')
   })
 

client/cypress/e2e/derived-variables.cy.ts

@@ -75,6 +75,76 @@ context('derived variables: ', function () {
 
     cy.screenshot('derived-3-filter-value', { capture: 'viewport' })
   })
+
+  it('4 | a derived column is marked read-only, and opens no editor', () => {
+    openEditor('TESTDATA.DERIVED_DEMO')
+
+    // a derived column is read-only AND computed, so the grid marks it apart
+    // from an ordinary read-only column - its own header/cell class, no
+    // readonlyCell dimming (editor.component.ts, afterGetColHeader /
+    // afterGetCellMeta). STATE is the control: same grid, a physical column.
+    headerCell('REGION').should('have.class', 'dc-derived-header')
+    headerCell('REGION').should('not.have.class', 'readonlyCell')
+    headerCell('STATE')
+      .should('not.have.class', 'dc-derived-header')
+      .and('not.have.class', 'readonlyCell')
+    getCellByHeaderAndRow(0, 'REGION').should('have.class', 'dc-derived-cell')
+    getCellByHeaderAndRow(0, 'STATE').should(
+      'not.have.class',
+      'dc-derived-cell'
+    )
+    // The computed block opens at whichever derived column comes first in the
+    // grid - MPE_DERIVED order, so LINE_VALUE here, not REGION. Assert the
+    // invariant rather than a name: exactly one derived header carries the
+    // marker, and it is the first one.
+    cy.get('.ht_clone_top .htCore thead tr th').then(($ths) => {
+      const derived = [...$ths].filter((th) =>
+        th.classList.contains('dc-derived-header')
+      )
+      expect(derived.length, 'derived headers').to.be.greaterThan(0)
+      expect(
+        derived[0].classList.contains('dc-derived-first'),
+        'the first derived header opens the block'
+      ).to.be.true
+      derived
+        .slice(1)
+        .forEach(
+          (th) =>
+            expect(
+              th.classList.contains('dc-derived-first'),
+              'only the first derived header carries the marker'
+            ).to.be.false
+        )
+    })
+
+    // the hover text carries the label, which is what says what the column is:
+    // the configured VAR_LABEL where there is one, otherwise the generated
+    // name=expression
+    headerCell('REGION_LABEL')
+      .should('have.attr', 'title')
+      .and('contain', 'Region and state')
+    headerCell('REGION')
+      .should('have.attr', 'title')
+      .and('contain', 'REGION=ifc')
+
+    enterEditMode()
+
+    getCellByHeaderAndRow(0, 'REGION').dblclick({ force: true })
+    cy.get('.handsontableInput').should('not.exist')
+    getCellByHeaderAndRow(0, 'REGION').should('have.text', 'West')
+  })
+
+  /**
+   * The positive control for the case above: same screen, same edit mode, a
+   * physical column - so its silence is about the column, not the screen.
+   */
+  it('5 | a physical column in the same grid does accept an edit', () => {
+    openEditor('TESTDATA.DERIVED_DEMO')
+    enterEditMode()
+
+    getCellByHeaderAndRow(0, 'STATE').dblclick({ force: true })
+    cy.get('.handsontableInput').should('exist')
+  })
 })
 
 /**
@@ -103,6 +173,23 @@ const openEditor = (libMem: string) => {
   cy.wait(2500)
 }
 
+/** The EDIT screen opens read-only; this is the button that starts an edit. */
+const enterEditMode = () => {
+  cy.get('.btnCtrl button.btn-primary', { timeout: longerCommandTimeout })
+    .click()
+    .then(() => cy.wait(2500))
+}
+
+/** The grid header cell for a column, by its rendered text. */
+const headerCell = (headerText: string) =>
+  cy.get('.ht_clone_top .htCore thead tr th').then(($ths) => {
+    const th = [...$ths].find(
+      (candidate) => (candidate as HTMLElement).innerText.trim() === headerText
+    )
+    if (!th) throw new Error(`header not found: ${headerText}`)
+    return cy.wrap(th)
+  })
+
 const openFilterPopup = () => {
   cy.contains('button.btnView', 'Filter', {
     timeout: longerCommandTimeout

client/cypress/e2e/download.cy.ts

@@ -17,9 +17,20 @@ context('download files test: ', function () {
 
   this.beforeEach(() => {
     cy.visit(hostUrl + appLocation)
-    cy.get('input.username').type(username)
+
-    cy.get('input.password').type(password)
+    // beforeAll already logged in. Typing into the login form unconditionally
-    cy.get('.login-group button').click()
+    // failed with "the center of this element is hidden from view" whenever the
+    // session was live and the form was not on screen, so only fill it when it
+    // is actually shown.
+    cy.get('body').then(($body) => {
+      const usernameInput = $body.find('input.username')[0]
+
+      if (usernameInput && !Cypress.dom.isHidden(usernameInput)) {
+        cy.get('input.username').type(username)
+        cy.get('input.password').type(password)
+        cy.get('.login-group button').click()
+      }
+    })
 
     visitPage('home')
   })

client/cypress/e2e/embed-va.cy.ts

@@ -6,44 +6,90 @@ const libraryToOpenIncludes = Cypress.env(`libraryToOpenIncludes_${serverType}`)
 
 // SAS Visual Analytics data-driven content embed mode (`?embed=va`):
 //  - chrome is hidden (same as embed=true),
-//  - the editor opens in edit mode immediately (not read-only),
+//  - the editor opens READ-ONLY (the user clicks Edit, as in normal use),
-//  - the only action button is a single Submit below the grid (stubbed in v1),
+//  - the Filter button is hidden - VA filters replace the filter builder,
-//  - the top button bar and Add Record are hidden.
+//  - the Add Row / Add Record buttons are hidden.
 //
 // The detailed PK+label data-merge logic is covered by the VaMessagingService
-// unit spec; here we only assert the deterministic UI layout and that pushing a
+// unit spec; here we assert the deterministic UI layout, that pushing a VA
-// VA postMessage does not break the page.
+// postMessage does not break the page, and (case 3) that a `va_parmap` URL
+// mapping filters the column it names end to end.
+/**
+ * Assert on the row count the editor shows. The assertion runs inside a Cy
+ * `should`, so it retries until the grid has loaded: reading the text once would
+ * capture the count from before a filter is applied.
+ */
+const parseRows = (text: string): number => {
+  const match = /\(([\d,]+) rows?/.exec(text)
+  return match ? Number(match[1].replace(/,/g, '')) : NaN
+}
+
+const expectRows = (check: (rows: number) => void) =>
+  cy.get('h4', { timeout: longerCommandTimeout }).should(($h4) => {
+    check(parseRows($h4.text()))
+  })
+
+/** Assert the grid shrank, without pinning a fixture-specific number. */
+const expectFewerRowsThan = (baseline: number) =>
+  expectRows((rows) => {
+    expect(rows, 'filtered row count').to.be.lessThan(baseline)
+  })
+
+/** Read the unfiltered row count from the app itself, retrying until it loads. */
+const readRowCount = (): Cypress.Chainable<number> =>
+  expectRows((rows) => {
+    expect(rows, 'row count').to.be.greaterThan(0)
+  })
+    .get('h4')
+    .invoke('text')
+    .then(parseRows)
+
 context('embed=va (VA data-driven content) tests: ', function () {
+  // How many rows the table has unfiltered. Captured once, from the app itself.
+  let unfilteredRows = 0
+
   this.beforeAll(() => {
     cy.visit(`${hostUrl}/SASLogon/logout`)
     cy.loginAndUpdateValidKey()
+    // Baseline for the relative row-count assertions: open the table with no VA
+    // message and record what the editor itself reports.
+    cy.visit(`${hostUrl}${appLocation}/#/editor/DC_JSLIB.MPE_X_TEST`, {
+      timeout: 60000
+    })
+    readRowCount().then((count) => {
+      unfilteredRows = count
+    })
   })
 
   this.beforeEach(() => {
     cy.visit(hostUrl + appLocation)
+    // A cy.visit that differs from the current URL only in the hash is a
+    // same-document navigation, so the app is not re-bootstrapped. Reload so
+    // each test starts from a fresh document (the previous test may have left
+    // ?embed=va in the URL, which the app reads once at init).
+    cy.reload()
     visitPage('home')
   })
 
-  it('1 | opens the editor in VA mode: chrome hidden, editable, single bottom Submit', () => {
+  it('1 | opens the editor in VA mode: chrome hidden, read-only, no Filter button', () => {
     openEditorInVaMode('mpe_x_test', () => {
       // Chrome hidden (same as embed=true)
       cy.get('header.app-header').should('not.exist')
 
-      // Grid present and in edit mode (read-only Filter/Edit/Upload bar absent)
+      // Grid present, read-only: the user enters edit mode with Edit, as usual
       cy.get('#hotTable', { timeout: longerCommandTimeout }).should('exist')
-      cy.get('.btnCtrl button').contains('Edit').should('not.exist')
+      cy.get('.btnCtrl button').contains('Edit').should('exist')
+
+      // The filter builder is replaced by VA filters
       cy.get('.btnCtrl button').contains('Filter').should('not.exist')
 
-      // Top edit-mode buttons hidden in VA mode
+      // Not in edit mode, so no edit-mode controls
       cy.get('.btnCtrl button.btn-outline-danger').should('not.exist') // Cancel
       cy.get('.btnCtrl').contains('Add Row').should('not.exist')
+      cy.get('.btnCtrl').contains('Submit').should('not.exist')
 
-      // Bottom Add Record hidden
+      // Bottom Add Record hidden in VA mode
       cy.contains('button', 'Add Record').should('not.exist')
-
-      // Exactly one Submit button, below the grid, disabled in v1
-      cy.get('button').filter(':contains("Submit")').should('have.length', 1)
-      cy.get('button').contains('Submit').should('be.disabled')
     })
   })
 
@@ -69,14 +115,382 @@ context('embed=va (VA data-driven content) tests: ', function () {
       // Editor still loads in VA mode after the message (chrome hidden, grid up)
       cy.get('#hotTable', { timeout: longerCommandTimeout }).should('exist')
       cy.get('header.app-header').should('not.exist')
-      cy.get('button').contains('Submit').should('exist')
+      cy.get('.btnCtrl button').contains('Edit').should('exist')
+    })
+  })
+  it('3 | va_parmap filters a column from a parameter that is not its name (tab 1)', () => {
+    // Two DC instances over one table (one per report tab), each filtering the
+    // same DC column from its own report parameter. Without the URL mapping the
+    // parameter label is not a column name, so nothing would filter at all. The
+    // mapping here is written in lower case against an upper-case label and
+    // column, which pins the case-insensitivity of both sides end to end.
+    openEditorInVaMode(
+      'mpe_x_test',
+      () => {
+        pushVaMessage({
+          name: 'pr1',
+          label: 'DROPDOWN_TAB1',
+          dataType: 'string',
+          value: 'Option 2'
+        })
+        // The applied filter names the MAPPED column, and the grid shrinks to
+        // the rows matching that value (Option 2 occurs twice in this table).
+        cy.get('.infoBar-text', { timeout: longerCommandTimeout }).should(
+          'contain.text',
+          'SOME_DROPDOWN'
+        )
+        cy.get('.infoBar-text').should('contain.text', "'Option 2'")
+        expectFewerRowsThan(unfilteredRows)
+      },
+      'va_parmap=dropdown_tab1-some_dropdown'
+    )
+  })
+
+  it('4 | the other tab maps its own parameter to the same column', () => {
+    // Same table, same column, a different parameter label and value (Option 1
+    // occurs four times) - the per-instance URL mapping is what keeps the two
+    // tabs independent.
+    openEditorInVaMode(
+      'mpe_x_test',
+      () => {
+        pushVaMessage({
+          name: 'pr1',
+          label: 'DROPDOWN_TAB2',
+          dataType: 'string',
+          value: 'Option 1'
+        })
+        cy.get('.infoBar-text', { timeout: longerCommandTimeout }).should(
+          'contain.text',
+          'SOME_DROPDOWN'
+        )
+        cy.get('.infoBar-text').should('contain.text', "'Option 1'")
+        expectFewerRowsThan(unfilteredRows)
+      },
+      'va_parmap=DROPDOWN_TAB2-SOME_DROPDOWN'
+    )
+  })
+
+  it('5 | warns in the grid when a mapping names a column the table lacks', () => {
+    // Written by hand in the report's URL, so a typo must be visible to the
+    // report author: the parameter is skipped, and the reason is shown rather
+    // than leaving an unfiltered grid that looks like a working filter.
+    openEditorInVaMode(
+      'mpe_x_test',
+      () => {
+        pushVaMessage({
+          name: 'pr1',
+          label: 'DROPDOWN_TAB1',
+          dataType: 'string',
+          value: 'Option 2'
+        })
+        cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
+          'contain.text',
+          'VA parameter mapping problem'
+        )
+        cy.get('.va-parmap-warning').should('contain.text', 'DROPDOWN_TAB1')
+        cy.get('.va-parmap-warning').should('contain.text', 'NOT_A_COLUMN')
+        // VA mode hides the app header, so the banner must paint inside the
+        // viewport on its own - assert the real box, not just its presence.
+        cy.get('.va-parmap-warning').should(($alert) => {
+          const rect = $alert[0].getBoundingClientRect()
+          expect(rect.width, 'banner width').to.be.greaterThan(100)
+          expect(rect.height, 'banner height').to.be.greaterThan(20)
+          expect(rect.top, 'banner top').to.be.at.least(0)
+          expect(rect.bottom, 'banner bottom').to.be.at.most(
+            Cypress.config('viewportHeight')
+          )
+        })
+        cy.screenshot('va-parmap-banner', { capture: 'viewport' })
+        // Nothing was filtered: the row count matches the unfiltered baseline.
+        expectRows((rows) => expect(rows).to.equal(unfilteredRows))
+      },
+      'va_parmap=DROPDOWN_TAB1-NOT_A_COLUMN'
+    )
+  })
+
+  it('6 | warns straight away when a mapping entry cannot be parsed', () => {
+    // No separator, so the entry never becomes a mapping - reported before any
+    // VA message arrives.
+    openEditorInVaMode(
+      'mpe_x_test',
+      () => {
+        cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
+          'contain.text',
+          'it needs a "-" between the VA parameter and the column name'
+        )
+      },
+      'va_parmap=DROPDOWN_TAB1'
+    )
+  })
+
+  it('7 | warns when a mapping names a parameter VA is not sending', () => {
+    // Parameter-side typo: the mapping never applies, so without the banner this
+    // is indistinguishable from a working filter over every row.
+    openEditorInVaMode(
+      'mpe_x_test',
+      () => {
+        pushVaMessage({
+          name: 'pr1',
+          label: 'DROPDOWN_TAB1',
+          dataType: 'string',
+          value: 'Option 2'
+        })
+        // One message only tells us this object's parameter set is still being
+        // discovered; the warning waits for a message that introduces no new name.
+        cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
+          'not.exist'
+        )
+        pushVaMessage({
+          name: 'pr1',
+          label: 'DROPDOWN_TAB1',
+          dataType: 'string',
+          value: 'Option 2'
+        })
+        cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
+          'contain.text',
+          'names VA parameter "dropdown_tab_1", which VA is not sending'
+        )
+        expectRows((rows) => expect(rows).to.equal(unfilteredRows))
+      },
+      'va_parmap=DROPDOWN_TAB_1-SOME_DROPDOWN'
+    )
+  })
+
+  it('8 | applies a mapping from the buffered message on the very first load', () => {
+    // VA posts once on iframe load, before the app bootstraps, so the first
+    // getdata is already filtered from that buffered message - with no column
+    // metadata available, the mapping is what names the column.
+    visitWithBufferedMessage(
+      'va_parmap=DROPDOWN_TAB1-SOME_DROPDOWN',
+      {
+        name: 'pr1',
+        label: 'DROPDOWN_TAB1',
+        dataType: 'string',
+        value: 'Option 2'
+      },
+      () => {
+        cy.get('.infoBar-text', { timeout: longerCommandTimeout }).should(
+          'contain.text',
+          "SOME_DROPDOWN = 'Option 2'"
+        )
+        expectFewerRowsThan(unfilteredRows)
+        cy.url().should('match', /\/editor\/[^/]+\/\d+/)
+      }
+    )
+  })
+
+  it('9 | a bad mapping on the first load still ends at the banner', () => {
+    // The deferred first load cannot check the column against metadata, so its
+    // saveQuery may fail and fall back to an unfiltered load - the banner is
+    // what stops that reading as a working filter.
+    visitWithBufferedMessage(
+      'va_parmap=DROPDOWN_TAB1-NOT_A_COLUMN',
+      {
+        name: 'pr1',
+        label: 'DROPDOWN_TAB1',
+        dataType: 'string',
+        value: 'Option 2'
+      },
+      () => {
+        cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
+          'contain.text',
+          'NOT_A_COLUMN'
+        )
+        cy.get('.va-parmap-warning').should('contain.text', 'DROPDOWN_TAB1')
+        // Deliberately no row-count assertion: this path cannot validate the
+        // column (no metadata yet), so the outcome depends on the platform - on
+        // SAS, mp_filtercheck aborts the save and the load falls back to
+        // unfiltered; the JS mock stores the filter and returns 0 rows. The
+        // banner is the part that must hold either way, because both outcomes
+        // are otherwise indistinguishable from a working filter.
+      }
+    )
+  })
+
+  it('10 | warns when two parameters filter the same column (an empty grid)', () => {
+    // Clauses are AND-ed, so two different values on one column produce no rows
+    // at all - the last silent-empty-grid path. Reported rather than rewritten,
+    // so the author decides whether to use one parameter carrying both values.
+    openEditorInVaMode(
+      'mpe_x_test',
+      () => {
+        pushVaMessage(
+          {
+            name: 'pr1',
+            label: 'DROPDOWN_TAB1',
+            dataType: 'string',
+            value: 'Option 1'
+          },
+          {
+            name: 'pr2',
+            label: 'DROPDOWN_TAB2',
+            dataType: 'string',
+            value: 'Option 2'
+          }
+        )
+        cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
+          'contain.text',
+          'combined with AND'
+        )
+        cy.get('.va-parmap-warning').should('contain.text', 'DROPDOWN_TAB1')
+        cy.get('.va-parmap-warning').should('contain.text', 'DROPDOWN_TAB2')
+        // The consequence: rows must match both clauses, and the two values
+        // differ, so none can match.
+        expectRows((rows) => expect(rows, 'matching rows').to.equal(0))
+      },
+      'va_parmap=DROPDOWN_TAB1-SOME_DROPDOWN&va_parmap=DROPDOWN_TAB2-SOME_DROPDOWN'
+    )
+  })
+
+  it('12 | a transform adds the previous quarter to the filter', () => {
+    // ADD(QUARTER_START,-1) is the reported requirement: one clause carrying the
+    // selected date and the same point one quarter earlier. 2026-10-01 = 24380,
+    // 2026-07-01 = 24288 in SAS days; nothing in the fixture matches either, so
+    // the assertion is on the clause rather than the row count.
+    openEditorInVaMode(
+      'mpe_x_test',
+      () => {
+        pushVaMessage({
+          name: 'pr1',
+          label: 'SELECTED_DATE',
+          dataType: 'string',
+          value: '01OCT2026'
+        })
+        cy.get('.infoBar-text', { timeout: longerCommandTimeout }).should(
+          'contain.text',
+          'SOME_DATE IN ('
+        )
+        cy.get('.infoBar-text').should('contain.text', '24380')
+        cy.get('.infoBar-text').should('contain.text', '24288')
+        cy.get('h4').should('contain.text', '(0 rows')
+      },
+      'va_parmap=SELECTED_DATE-SOME_DATE:ADD(QUARTER_START,-1)'
+    )
+  })
+
+  it('13 | warns when a transform lands on a column it cannot apply to', () => {
+    openEditorInVaMode(
+      'mpe_x_test',
+      () => {
+        pushVaMessage({
+          name: 'pr1',
+          label: 'SELECTED_DATE',
+          dataType: 'string',
+          value: 'x'
+        })
+        cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
+          'contain.text',
+          'is not a date or datetime column'
+        )
+        cy.get('h4').should('contain.text', '(7 rows')
+      },
+      'va_parmap=SELECTED_DATE-SOME_CHAR:ADD(QUARTER_START,-1)'
+    )
+  })
+
+  it('11 | logs a parse-time problem once, however many messages arrive', () => {
+    // The parse problem is logged at app init; the editor's own loop must not log
+    // it again. The second entry is valid-but-missing, so the loop definitely runs
+    // and logs its own line - which is what makes this a real assertion.
+    cy.visit(
+      `${hostUrl}${appLocation}/index.html#/editor/DC_JSLIB.MPE_X_TEST?embed=va&va_parmap=DROPDOWN_TAB1&va_parmap=DROPDOWN_TAB2-NOT_A_COLUMN`,
+      {
+        onBeforeLoad(win: any) {
+          win.__warns = []
+          const original = win.console.warn.bind(win.console)
+          win.console.warn = (...args: any[]) => {
+            win.__warns.push(args.join(' '))
+            original(...args)
+          }
+        }
+      }
+    )
+    cy.get('.app-loading', { timeout: longerCommandTimeout }).should(
+      'not.exist'
+    )
+    pushVaMessage({
+      name: 'pr2',
+      label: 'DROPDOWN_TAB2',
+      dataType: 'string',
+      value: 'Option 2'
+    })
+    cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
+      'contain.text',
+      'NOT_A_COLUMN'
+    )
+    cy.window().then((win: any) => {
+      const warns: string[] = win.__warns || []
+      const parseLines = warns.filter((w) =>
+        w.includes('va_parmap "DROPDOWN_TAB1"')
+      )
+      const runtimeLines = warns.filter((w) => w.includes('NOT_A_COLUMN'))
+      expect(parseLines.length, warns.join(' | ')).to.eq(1)
+      expect(runtimeLines.length, warns.join(' | ')).to.eq(1)
     })
   })
 })
 
+/**
+ * Loads the editor with a VA message already in the pre-bootstrap buffer, the
+ * way VA's own postMessage arrives before Angular starts. A distinct pathname is
+ * used so this is a real document load: a cy.visit that differs only in the hash
+ * is a same-document navigation and would not re-bootstrap the app.
+ */
+const visitWithBufferedMessage = (
+  extraParams: string,
+  parameter: any,
+  callback?: any
+) => {
+  cy.visit(
+    `${hostUrl}${appLocation}/index.html#/editor/DC_JSLIB.MPE_X_TEST?embed=va&${extraParams}`,
+    {
+      onBeforeLoad(win: any) {
+        win.__vaLastMessage = {
+          t: Date.now(),
+          origin: win.location.origin,
+          data: {
+            version: '1',
+            resultName: 'dd1',
+            rowCount: 0,
+            availableRowCount: 0,
+            data: [],
+            columns: [],
+            parameters: [parameter]
+          }
+        }
+      }
+    }
+  )
+  cy.get('.app-loading', { timeout: longerCommandTimeout }).should('not.exist')
+  if (callback) callback()
+}
+
+/** Pushes one VA data-driven content message carrying the given parameters. */
+const pushVaMessage = (...parameters: any[]) => {
+  cy.window().then((win) => {
+    win.postMessage(
+      {
+        version: '1',
+        resultName: 'dd1',
+        rowCount: 0,
+        availableRowCount: 0,
+        data: [],
+        columns: [],
+        parameters
+      },
+      '*'
+    )
+  })
+}
+
 // Opens a table from the tree (which routes to #/editor/LIB.TABLE), then
 // re-visits the same route with ?embed=va so the app parses VA embed mode.
-const openEditorInVaMode = (tablename: string, callback?: any) => {
+// `extraParams` is appended to that query string (e.g. va_parmap entries).
+const openEditorInVaMode = (
+  tablename: string,
+  callback?: any,
+  extraParams?: string
+) => {
   openTableFromTree(libraryToOpenIncludes, tablename)
 
   cy.get('#hotTable', { timeout: longerCommandTimeout })
@@ -84,7 +498,13 @@ const openEditorInVaMode = (tablename: string, callback?: any) => {
     .then(() => {
       cy.url().then((url) => {
         const separator = url.includes('?') ? '&' : '?'
-        cy.visit(`${url}${separator}embed=va`)
+        const extra = extraParams ? `&${extraParams}` : ''
+        cy.visit(`${url}${separator}embed=va${extra}`)
+        // A cy.visit that differs from the current URL only in the hash is a
+        // same-document navigation, so the app is not re-bootstrapped and the
+        // embed parameter (read once at app init) never takes effect. Reload to
+        // bootstrap it, as the VA iframe load does.
+        cy.reload()
         cy.get('.app-loading', { timeout: longerCommandTimeout }).should(
           'not.exist'
         )

client/cypress/e2e/metanav.cy.ts

@@ -14,11 +14,10 @@ context('metanav tests: ', function () {
   })
 
   this.beforeEach(() => {
-    cy.visit(hostUrl + appLocation)
+    // beforeAll has already logged in. Typing into the login form here failed
-    cy.get('input.username').type(username)
+    // once the session existed - the form is not on screen, so Cypress reported
-    cy.get('input.password').type(password)
+    // "the center of this element is hidden from view" and the whole suite was
-    cy.get('.login-group button').click()
+    // skipped behind that hook.
-
     visitPage('view/metadata')
   })
 

client/licenseChecker.js

@@ -10,7 +10,7 @@ const check = (cwd) => {
         onlyAllow:
           'AFLv2.1;Apache 2.0;Apache-2.0;Apache*;Artistic-2.0;BlueOak-1.0.0;0BSD;BSD*;BSD-2-Clause;BSD-3-Clause;CC0-1.0;CC-BY-3.0;CC-BY-4.0;ISC;MIT;MPL-2.0;ODC-By-1.0;Python-2.0;Unlicense;',
         excludePackages:
-          '@cds/city@1.1.0;@handsontable/angular-wrapper@16.0.1;@handsontable/angular-wrapper@17.1.0;@handsontable/angular-wrapper@18.0.0;@handsontable/angular-wrapper@18.1.1;handsontable@^16.0.1;handsontable@16.2.0;handsontable@17.1.0;handsontable@18.0.0;handsontable@18.1.1;hyperformula@2.7.1;hyperformula@3.0.0;hyperformula@3.1.0;hyperformula@3.2.0;hyperformula@3.3.0;jackspeak@3.4.3;path-scurry@1.11.1;package-json-from-dist@1.0.1;buffers@0.1.1'
+          '@cds/city@1.1.0;@handsontable/angular-wrapper@16.0.1;@handsontable/angular-wrapper@17.1.0;@handsontable/angular-wrapper@18.0.0;@handsontable/angular-wrapper@18.1.1;@handsontable/angular-wrapper@18.1.2;handsontable@^16.0.1;handsontable@16.2.0;handsontable@17.1.0;handsontable@18.0.0;handsontable@18.1.1;handsontable@18.1.2;hyperformula@2.7.1;hyperformula@3.0.0;hyperformula@3.1.0;hyperformula@3.2.0;hyperformula@3.3.0;jackspeak@3.4.3;path-scurry@1.11.1;package-json-from-dist@1.0.1;buffers@0.1.1'
       },
       (error, json) => {
         if (error) {

client/package-lock.json

@@ -20,7 +20,7 @@
         "@clr/angular": "file:libraries/clr-angular-17.9.0.tgz",
         "@clr/icons": "^13.0.2",
         "@clr/ui": "file:libraries/clr-ui-17.9.0.tgz",
-        "@handsontable/angular-wrapper": "18.1.1",
+        "@handsontable/angular-wrapper": "18.1.2",
         "@sasjs/adapter": "5.1.2",
         "@sasjs/utils": "4.0.0",
         "@sheet/crypto": "file:libraries/sheet-crypto.tgz",
@@ -35,7 +35,7 @@
         "d3-graphviz": "^5.0.2",
         "exceljs": "^4.4.0",
         "fs-extra": "^7.0.1",
-        "handsontable": "18.1.1",
+        "handsontable": "18.1.2",
         "https-browserify": "1.0.0",
         "hyperformula": "^2.5.0",
         "iconv-lite": "^0.5.0",
@@ -969,7 +969,6 @@
       "integrity": "sha512-pQiOg+se1AU/ncMlnJ9V6xYnMQ84qI1BGWuJpbU6A99VTXJg90scg0+T7DWmKssR1YjP5qmmBtrZfKsHEcLW/A==",
       "dev": true,
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "@angular-eslint/bundled-angular-compiler": "19.8.1",
         "eslint-scope": "^8.0.2"
@@ -1000,7 +999,6 @@
       "integrity": "sha512-HdDfbLwpxzx49wAWz4oOUQKSkZcFNRpWEGsDEgufZ3ExwxRk1/2QPJuvZ2CbhKeEcLPrLKUorQ8XlGrNsJmNiQ==",
       "deprecated": "@angular/animations is deprecated. Use `animate.enter` and `animate.leave` instead. For more information see: https://v22.angular.dev/guide/animations.",
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "tslib": "^2.3.0"
       },
@@ -1146,7 +1144,6 @@
       "resolved": "https://registry.npmjs.org/@angular/cdk/-/cdk-20.2.14.tgz",
       "integrity": "sha512-7bZxc01URbiPiIBWThQ69XwOxVduqEKN4PhpbF2AAyfMc/W8Hcr4VoIJOwL0O1Nkq5beS8pCAqoOeIgFyXd/kg==",
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "parse5": "^8.0.0",
         "tslib": "^2.3.0"
@@ -1443,7 +1440,6 @@
       "integrity": "sha512-nqhDw2ZcAUrKNPwhjinJny903bRhI0rQhiDz1LksjeRxqa36i3l75+4iXbOy0rlDpLJGxqtgoPavQjmmyS5UJw==",
       "dev": true,
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "@inquirer/checkbox": "^4.2.1",
         "@inquirer/confirm": "^5.1.14",
@@ -1586,7 +1582,6 @@
       "resolved": "https://registry.npmjs.org/@angular/common/-/common-20.3.34.tgz",
       "integrity": "sha512-OhB/l0cNldOvQeICqbWqRUzK5fF6zNYy44Q4cIGJZql8HXL/invGiqWLE9f+kU8z4jRisofHIGTBtyJQYYOudg==",
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "tslib": "^2.3.0"
       },
@@ -1603,7 +1598,6 @@
       "resolved": "https://registry.npmjs.org/@angular/compiler/-/compiler-20.3.34.tgz",
       "integrity": "sha512-8agPW384c/JER1ersPTmJJfcCOnlYcBbdj0MYRxNdiYe+7gtnspF3ORuJmxqCEpMS17AljSmk2skwUcYnHB6xA==",
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "tslib": "^2.3.0"
       },
@@ -1617,7 +1611,6 @@
       "integrity": "sha512-PKHzhSOyDUFh64Ull3fUwgFlxnsQNEAuRyz4KqQ8FsTGZ4pOpUaZntO+2Xn1nX/gT2whSyGBJhD3H37zA8ZGEQ==",
       "dev": true,
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "@babel/core": "7.29.7",
         "@jridgewell/sourcemap-codec": "^1.4.14",
@@ -1650,7 +1643,6 @@
       "resolved": "https://registry.npmjs.org/@angular/core/-/core-20.3.34.tgz",
       "integrity": "sha512-xOvSmGoao3SVPz//R7zTrq7+WdrIGi+WNpAdxJEvJJXuSpLGS3kSUqukjY4kOMochg2v0NCEPIdCyH+yPs8JSw==",
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "tslib": "^2.3.0"
       },
@@ -1694,7 +1686,6 @@
       "resolved": "https://registry.npmjs.org/@angular/platform-browser/-/platform-browser-20.3.34.tgz",
       "integrity": "sha512-0bd2Bj05WPoBTFM0A13IwbzxKVQPstx/N+7X0ZWKCifliyydIc7DBHu+r37ybJpl8ZsCVBe7DoR9Gon4lZ+zlw==",
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "tslib": "^2.3.0"
       },
@@ -1790,7 +1781,6 @@
       "integrity": "sha512-RgHBCvtjbOK2gXSNBNIkNoEc9qoVEtau3hj8gEqKQuL3HZAibKarWFEI3Lfm6EYKkLalOh8eSrj9b+ch9H/VBA==",
       "dev": true,
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "@babel/code-frame": "^7.29.7",
         "@babel/generator": "^7.29.7",
@@ -2317,6 +2307,7 @@
       "integrity": "sha512-oBNVCvnO5tND+xSopWvV8WNGfpTfgP4Zr/YXXSj8zfmcPktp5Ku/aZlsIowgSD4fjmgHn6sGmB9APVsU5zOdhA==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "dependencies": {
         "@babel/helper-plugin-utils": "^7.29.7",
         "@babel/helper-skip-transparent-expression-wrappers": "^7.29.7"
@@ -3362,6 +3353,7 @@
       "integrity": "sha512-GYzX36n1nsciIb0uyH0GHwxwtNwPQIcpxSeiVLDtG/B7jB5xXgchnmL1f/jCX5o+pwnaDBtO60ONSJhEBJfxYA==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "dependencies": {
         "@babel/compat-data": "^7.29.7",
         "@babel/helper-compilation-targets": "^7.29.7",
@@ -3448,6 +3440,7 @@
       "integrity": "sha512-d98gXZkgswvkyohMBABkhm3GeXhYj8psWfwQ2C7gtfrKGTykQa/iOIi+JJhwMjPlZ6Vm2XN+DCf3Es1EoG4ZLA==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "dependencies": {
         "@babel/helper-plugin-utils": "^7.29.7",
         "@babel/helper-remap-async-to-generator": "^7.29.7",
@@ -3466,6 +3459,7 @@
       "integrity": "sha512-pcUb2SS+RMo9TWVBwKGI5ShtoG7R+zBsFmCKDa6fe8c+hPr3XJlZgoE5j6i8W7gDjhyvy+85vmYexanvXh3d1w==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "dependencies": {
         "@babel/helper-module-imports": "^7.29.7",
         "@babel/helper-plugin-utils": "^7.29.7",
@@ -3484,6 +3478,7 @@
       "integrity": "sha512-coWpDLJ410R781Npmn/SIBZEsAetR4xVi0SxLMXPaMO4lSf1MwnkGYMtkFxew0Dn8B3/CpbpYxN0JCgg8mn67g==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "dependencies": {
         "@babel/helper-define-polyfill-provider": "^0.6.8",
         "core-js-compat": "^3.48.0"
@@ -3498,6 +3493,7 @@
       "integrity": "sha512-BR7VvDCVHO+q2xBEWskxS6DJE1qRnb7DxzUrogb71CWoSficBxYsiAGd+Kl0mmq/MprG9yArRkyrQxTO6XjMzA==",
       "dev": true,
       "license": "ISC",
+      "peer": true,
       "bin": {
         "semver": "bin/semver.js"
       }
@@ -3604,7 +3600,6 @@
       "resolved": "https://registry.npmjs.org/@cds/core/-/core-6.17.0.tgz",
       "integrity": "sha512-afG+KwNE4BDc6CAylgGbJrJUeGYFbOIfagylaqqbS7fgIX2h3ZqFcUOv/2hujSZ9GZiuCSa+A/BkJi7K83QEgg==",
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "lit": "^2.1.3",
         "ramda": "^0.29.0",
@@ -3786,7 +3781,6 @@
       "integrity": "sha512-TQMmc3w+5AxjpL8iIiwebF73dRDF4fBIieAqGn9RGCWaEVwQ6Fb2cGe31Yns0RRIzii5goJ1Y7xbMwo1TxMplw==",
       "dev": true,
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "readdirp": "^5.0.0"
       },
@@ -4904,9 +4898,9 @@
       }
     },
     "node_modules/@handsontable/angular-wrapper": {
-      "version": "18.1.1",
+      "version": "18.1.2",
-      "resolved": "https://registry.npmjs.org/@handsontable/angular-wrapper/-/angular-wrapper-18.1.1.tgz",
+      "resolved": "https://registry.npmjs.org/@handsontable/angular-wrapper/-/angular-wrapper-18.1.2.tgz",
-      "integrity": "sha512-paiINuWOevybN6RFUrtSOgRv5SaFlMd5dRbBOzeLyJJzqhLW7Jg+QSy6+51FAL6FKHZPfWHYoP0F2YgNPEXYbg==",
+      "integrity": "sha512-LoUc3jIO7Y5rkSzHW+njvQWA6x8RipkWmBkR7vmnEqgxcytejgd7RIy6/b3O0AqYuv2XUT+vXrNacn9wEekWhQ==",
       "license": "SEE LICENSE IN LICENSE.txt",
       "optionalDependencies": {
         "tslib": "^2.3.0"
@@ -7176,7 +7170,6 @@
       "integrity": "sha512-gLyJlPHPZYdAk1JENA9LeHejZe1Ti77/pTeFm/nMXmQH/HFZlcS/O2XJB+L8fkbrNSqhdtlvjBVjxwUYanNH5Q==",
       "dev": true,
       "license": "Apache-2.0",
-      "peer": true,
       "engines": {
         "node": ">=8.0.0"
       }
@@ -7200,7 +7193,6 @@
       "integrity": "sha512-s5vvxXPVdjqS3kTLKMeBMvop9hbWkwzBpu+mUO2M7sZtlkyDJGwFe33wRKnbaYDo8ExRVBIIdwIGrqpxHuKttA==",
       "dev": true,
       "license": "Apache-2.0",
-      "peer": true,
       "engines": {
         "node": ">=14"
       },
@@ -7214,7 +7206,6 @@
       "integrity": "sha512-OOCM2C/QIURhJMuKaekP3TRBxBKxG/TWWA0TL2J6nXUtDnuCtccy49LUJF8xPFXMX+0LMcxFpCo8M9cGY1W6rQ==",
       "dev": true,
       "license": "Apache-2.0",
-      "peer": true,
       "dependencies": {
         "@opentelemetry/semantic-conventions": "1.28.0"
       },
@@ -7241,7 +7232,6 @@
       "integrity": "sha512-BdBGhQBh8IjZ2oIIX6F2/Q3LKm/FDDKi6ccYKcBTeilh6SNdNKveDOLk73BkSJjQLJk6qe4Yh+hHw1UPhCDdrg==",
       "dev": true,
       "license": "Apache-2.0",
-      "peer": true,
       "dependencies": {
         "@opentelemetry/api-logs": "0.57.2",
         "@types/shimmer": "^1.2.0",
@@ -7672,7 +7662,6 @@
       "integrity": "sha512-5UxZqiAgLYGFjS4s9qm5mBVo433u+dSPUFWVWXmLAD4wB65oMCoXaJP1KJa9DIYYMeHu3z4BZcStG3LC593cWA==",
       "dev": true,
       "license": "Apache-2.0",
-      "peer": true,
       "dependencies": {
         "@opentelemetry/core": "1.30.1",
         "@opentelemetry/semantic-conventions": "1.28.0"
@@ -7700,7 +7689,6 @@
       "integrity": "sha512-jVPgBbH1gCy2Lb7X0AVQ8XAfgg0pJ4nvl8/IiQA6nxOsPvS+0zMJaFSs2ltXe0J6C8dqjcnpyqINDJmU30+uOg==",
       "dev": true,
       "license": "Apache-2.0",
-      "peer": true,
       "dependencies": {
         "@opentelemetry/core": "1.30.1",
         "@opentelemetry/resources": "1.30.1",
@@ -7729,7 +7717,6 @@
       "integrity": "sha512-eSYWTm620tTk45EKSedaUL8MFYI8hW164hIXsgIHyxu3VobUB3fFCu5t0hQby6OoWRPsG1KkKUG2M5UadiLiVg==",
       "dev": true,
       "license": "Apache-2.0",
-      "peer": true,
       "engines": {
         "node": ">=14"
       }
@@ -9377,7 +9364,8 @@
       "resolved": "https://registry.npmjs.org/@types/hammerjs/-/hammerjs-2.0.46.tgz",
       "integrity": "sha512-ynRvcq6wvqexJ9brDMS4BnBLzmr0e14d6ZJTEShTBWKymQiHwlAyGu0ZPEFI2Fh1U53F7tN9ufClWM5KvqkKOw==",
       "dev": true,
-      "license": "MIT"
+      "license": "MIT",
+      "peer": true
     },
     "node_modules/@types/http-errors": {
       "version": "2.0.5",
@@ -9721,7 +9709,6 @@
       "integrity": "sha512-CZ4nMxWwgu1HEEFNkeaCptra9QCtkmKdgf3sWh1rl1trIhmxLilgTV4cwcbQ4wemnT4sWQN8CaKOmdYx+g2gMA==",
       "dev": true,
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "@typescript-eslint/scope-manager": "8.65.0",
         "@typescript-eslint/types": "8.65.0",
@@ -9895,7 +9882,6 @@
       "integrity": "sha512-Adfoy59kxNEjT5E5fnyrp7GpHYuWs8ONbdohu4ez6IU46qDq8LltG90tLmkucZB2uhYL8mqBjtXqGd8yhfjXqQ==",
       "dev": true,
       "license": "MIT",
-      "peer": true,
       "engines": {
         "node": "^18.18.0 || ^20.9.0 || >=21.1.0"
       },
@@ -9990,6 +9976,7 @@
       "integrity": "sha512-bzHngBbmIgzj6TsTjVUCntGx6FFLgE7mPjpO0PzHycOaZHXqS43XnrkFIHRS16LLr/xO6FHSXG3GJFo/F4bSBQ==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "dependencies": {
         "@typescript-eslint/tsconfig-utils": "^8.72.0",
         "@typescript-eslint/types": "^8.72.0",
@@ -10012,6 +9999,7 @@
       "integrity": "sha512-JewPyJOYcXBSAS7PIgufscHOwZqhTGyBRiQJhhuDln66tMchxvvt7nJ90JKKIoOelmdHA3kAQbm9cEUHc4JSEA==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "dependencies": {
         "@typescript-eslint/types": "8.72.0",
         "@typescript-eslint/visitor-keys": "8.72.0"
@@ -10030,6 +10018,7 @@
       "integrity": "sha512-0vbxCf7xLVPSjcOdm6eQbgViH8EDJ8xnl1laLT9yoqSz2YmcSfSg5Vdl0JUvZf6rcuJ0ZivJIuMD4edTg8obaA==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "engines": {
         "node": "^18.18.0 || ^20.9.0 || >=21.1.0"
       },
@@ -10047,6 +10036,7 @@
       "integrity": "sha512-gwLfFgWcw15tN2KXDCJvxEOQjPn+P3j/n3eSqkZ5sX4VpW9Cosodr+tWwBZRvJT3TcKx6HnPRGZMCNAiP1Xhug==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "dependencies": {
         "@typescript-eslint/project-service": "8.72.0",
         "@typescript-eslint/tsconfig-utils": "8.72.0",
@@ -10075,6 +10065,7 @@
       "integrity": "sha512-E2mroviasaWecDzTGiGMtuEguuLVOeRggnrIn93CJuxYJzQBHbyymlT08aVQydbBal5538o+JkBKXh2sRuCyIA==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "dependencies": {
         "@typescript-eslint/types": "8.72.0",
         "eslint-visitor-keys": "^5.0.0"
@@ -10093,6 +10084,7 @@
       "integrity": "sha512-tD40eHxA35h0PEIZNeIjkHoDR4YjjJp34biM0mDvplBe//mB+IHCqHDGV7pxF+7MklTvighcCPPZC7ynWyjdTA==",
       "dev": true,
       "license": "Apache-2.0",
+      "peer": true,
       "engines": {
         "node": "^20.19.0 || ^22.13.0 || >=24"
       },
@@ -10106,6 +10098,7 @@
       "integrity": "sha512-Y7/KDsb8LjooZpwaqGyulO6DQlksgCncchHGk+sZIY4SBvUocMBEFH5Ur1fI4dV+Jvl0w6cjvucaIi40puRioA==",
       "dev": true,
       "license": "ISC",
+      "peer": true,
       "bin": {
         "semver": "bin/semver.js"
       },
@@ -10398,7 +10391,6 @@
       "resolved": "https://registry.npmjs.org/acorn/-/acorn-8.18.0.tgz",
       "integrity": "sha512-lGq+9yr1/GuAWaVYIHRjvvySG5/4VfKIvC8EWxStPdcDh/Ka7FG3twP6v4d5BkravUilhIAsG4Qj83t02LWUPQ==",
       "license": "MIT",
-      "peer": true,
       "bin": {
         "acorn": "bin/acorn"
       },
@@ -10469,7 +10461,6 @@
       "resolved": "https://registry.npmjs.org/ajv/-/ajv-8.18.0.tgz",
       "integrity": "sha512-PlXPeEWMXMZ7sPYOHqmDyCJzcfNrUr3fGNKtezX14ykXOEIvyK81d+qydx89KY5O71FKMPaQ2vBfBFI5NHR63A==",
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "fast-deep-equal": "^3.1.3",
         "fast-uri": "^3.0.1",
@@ -10933,7 +10924,6 @@
       "resolved": "https://registry.npmjs.org/axios/-/axios-1.20.0.tgz",
       "integrity": "sha512-r8aOh8j9cGKpgQAqpzrUHnSIc6a59Y3Xf/cv8sy1DrHCkZHzQGEuoq1tARk6qSyDdtQGSDgpb9kFlruzPvrgwg==",
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "follow-redirects": "^1.16.0",
         "form-data": "^4.0.6",
@@ -11069,7 +11059,6 @@
       "resolved": "https://registry.npmjs.org/bare-events/-/bare-events-2.9.2.tgz",
       "integrity": "sha512-AIPKioV7/Y/8KfZ3AAhjPJxLLbY49S64Ym5DakZlUg75qQiTgUq9hEJoEwa4eUezPUlXRy/i5NpsKvo9jgKmoA==",
       "license": "Apache-2.0",
-      "peer": true,
       "peerDependencies": {
         "bare-abort-controller": "*"
       },
@@ -11441,7 +11430,8 @@
       "resolved": "https://registry.npmjs.org/browser-stdout/-/browser-stdout-1.3.1.tgz",
       "integrity": "sha512-qhAVI1+Av2X7qelOfAIYwXONood6XlZE/fXaBSmW/T5SzLAmCgzi+eiWE7fUvbHaeNBQH13UftjpXxsfLkMpgw==",
       "dev": true,
-      "license": "ISC"
+      "license": "ISC",
+      "peer": true
     },
     "node_modules/browserify-aes": {
       "version": "1.2.0",
@@ -11499,7 +11489,6 @@
         }
       ],
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "baseline-browser-mapping": "^2.11.12",
         "caniuse-lite": "^1.0.30001809",
@@ -11873,7 +11862,6 @@
       "integrity": "sha512-Qgzu8kfBvo+cA4962jnP1KkS6Dop5NS6g7R5LFYJr4b8Ub94PPQXUksCw9PvXoeXPRRddRNC5C1JQUR2SMGtnA==",
       "dev": true,
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "readdirp": "^4.0.1"
       },
@@ -12305,7 +12293,8 @@
       "resolved": "https://registry.npmjs.org/common-path-prefix/-/common-path-prefix-3.0.0.tgz",
       "integrity": "sha512-QE33hToZseCH3jS0qN96O/bSh3kaw/h+Tq7ngyY9eWDUnTlTNUyqfqvCXioLe5Na5jFsL78ra/wuBU4iuEgd4w==",
       "dev": true,
-      "license": "ISC"
+      "license": "ISC",
+      "peer": true
     },
     "node_modules/common-tags": {
       "version": "1.8.2",
@@ -12986,7 +12975,6 @@
       "dev": true,
       "hasInstallScript": true,
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "@cypress/request": "^4.0.0",
         "@cypress/xvfb": "^1.2.4",
@@ -13388,7 +13376,6 @@
       "resolved": "https://registry.npmjs.org/d3-selection/-/d3-selection-3.0.0.tgz",
       "integrity": "sha512-fmTRWbNMmsmWq6xJV8D19U/gw/bwrHfNXxrIN+HfZgnzqTHp9jOmKMhsTUjXOJnZOdZY9Q28y4yebKzqDKlxlQ==",
       "license": "ISC",
-      "peer": true,
       "engines": {
         "node": ">=12"
       }
@@ -13664,8 +13651,7 @@
       "resolved": "https://registry.npmjs.org/devtools-protocol/-/devtools-protocol-0.0.1625959.tgz",
       "integrity": "sha512-wRBSU330hwOLLcb3N4NIe3eFs6MgT6ku3AiZONjnTSJ7f3dVchJfn6nE0Lfos9jK1na15bgp7xLhaCx40Y47NQ==",
       "dev": true,
-      "license": "BSD-3-Clause",
+      "license": "BSD-3-Clause"
-      "peer": true
     },
     "node_modules/dezalgo": {
       "version": "1.0.4",
@@ -13691,6 +13677,7 @@
       "integrity": "sha512-svtcdpS8CgJyqAjEQIXdb3OjhFVVYjzGAPO8WGCmRbrml64SPw/jJD4GoE98aR7r25A0XcgrK3F02yw9R/vhQw==",
       "dev": true,
       "license": "BSD-3-Clause",
+      "peer": true,
       "engines": {
         "node": ">=0.3.1"
       }
@@ -14311,7 +14298,6 @@
       "deprecated": "This version is no longer supported. Please see https://eslint.org/version-support for other options.",
       "dev": true,
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "@eslint-community/eslint-utils": "^4.2.0",
         "@eslint-community/regexpp": "^4.6.1",
@@ -14757,7 +14743,6 @@
       "integrity": "sha512-Bdcs4+3qlpVlx2NRn6fgX2Ue2/gGRaPeawebgclM0ERSCqDpA+owF1fdPwjJUTAJWMTuAaxjDf+hzb0/4eKvvw==",
       "dev": true,
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "accepts": "~1.3.8",
         "array-flatten": "1.1.1",
@@ -15185,6 +15170,7 @@
       "integrity": "sha512-9ZonPT4ZAK4a+1pUPVPZJapbi7O5qbbJPdYw/NOQWZZbVLdDTYM3A4R9z/DpAM08IDaFGsvPgiGZ82WEwUDWjg==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "dependencies": {
         "common-path-prefix": "^3.0.0",
         "pkg-dir": "^7.0.0"
@@ -15219,6 +15205,7 @@
       "integrity": "sha512-afd4O7zpqHeRyg4PfDQsXmlDe2PfdHtJt6Akt8jOWaApLOZk5JXs6VMR29lz03pRe9mpykrRCYIYxaJYcfpncQ==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "engines": {
         "node": ">=18"
       },
@@ -15744,11 +15731,10 @@
       }
     },
     "node_modules/handsontable": {
-      "version": "18.1.1",
+      "version": "18.1.2",
-      "resolved": "https://registry.npmjs.org/handsontable/-/handsontable-18.1.1.tgz",
+      "resolved": "https://registry.npmjs.org/handsontable/-/handsontable-18.1.2.tgz",
-      "integrity": "sha512-8vz/jp96m0aJQPRepOBoImsC/bLe9mE/O4nOB6mll9DCjN5nqXHX9LV4zu/XDpD89TKJwtA+TNhVUkFQxW+q7w==",
+      "integrity": "sha512-iL4Cia0Fsl+md1oCqUS8PJqfft3583d5ptc5M7ewfp6JTUsNsrIcpF3GPYvZy57e7P5PACX8WboGlkwgHn0h9Q==",
-      "license": "SEE LICENSE IN LICENSE.txt",
+      "license": "SEE LICENSE IN LICENSE.txt"
-      "peer": true
     },
     "node_modules/has-flag": {
       "version": "4.0.0",
@@ -15926,7 +15912,6 @@
       "integrity": "sha512-6E2QDAc9Ick9Sq77ZrGS/dk2WUYni91aufTw6LJKpV7w8kW5/GxVUc650FOADOmlwg3K+f7Pun6XlV+pYmW6gw==",
       "dev": true,
       "license": "MIT",
-      "peer": true,
       "engines": {
         "node": ">=16.9.0"
       }
@@ -17124,8 +17109,7 @@
       "resolved": "https://registry.npmjs.org/jasmine-core/-/jasmine-core-5.1.2.tgz",
       "integrity": "sha512-2oIUMGn00FdUiqz6epiiJr7xcFyNYj3rDcfmnzfkBnHyBQ3cBQUs4mmyGsOb7TTLb9kxk7dBcmEmqhDKkBoDyA==",
       "dev": true,
-      "license": "MIT",
+      "license": "MIT"
-      "peer": true
     },
     "node_modules/jest-worker": {
       "version": "27.5.1",
@@ -17162,7 +17146,6 @@
       "integrity": "sha512-/imKNG4EbWNrVjoNC/1H5/9GFy+tqjGBHCaSsN+P2RnPqjsLmv6UD3Ej+Kj8nBWaRAwyk7kK5ZUc+OEatnTR3A==",
       "dev": true,
       "license": "MIT",
-      "peer": true,
       "bin": {
         "jiti": "bin/jiti.js"
       }
@@ -17449,7 +17432,6 @@
       "integrity": "sha512-LrtUxbdvt1gOpo3gxG+VAJlJAEMhbWlM4YrFQgql98FwF7+K8K12LYO4hnDdUkNjeztYrOXEMqgTajSWgmtI/w==",
       "dev": true,
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "@colors/colors": "1.5.0",
         "body-parser": "^1.19.0",
@@ -18005,7 +17987,6 @@
       "integrity": "sha512-umRhrCH7fCi8Uj2RcwKjJdvUORTjeWqkdKx0LbcZvjIwsAVsnIAGcxHaqowPeBFBjQuWOeC/bve0AlpFzF/+SQ==",
       "dev": true,
       "license": "Apache-2.0",
-      "peer": true,
       "dependencies": {
         "copy-anything": "^3.0.5",
         "parse-node-version": "^1.0.1"
@@ -18467,7 +18448,6 @@
       "integrity": "sha512-SL0JY3DaxylDuo/MecFeiC+7pedM0zia33zl0vcjgwcq1q1FWWF1To9EIauPbl8GbMCU0R2e0uJ8bZunhYKD2g==",
       "dev": true,
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "cli-truncate": "^4.0.0",
         "colorette": "^2.0.20",
@@ -19594,6 +19574,7 @@
       "integrity": "sha512-beammieDdiWdkjANWeWPpkxUROYGi5FN88fPOkLiticz7HTg3Y+Pho3EYE6SUdcuWnZ3h2WD8EXCA55JF4o6wg==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "dependencies": {
         "browser-stdout": "^1.3.1",
         "chokidar": "^5.0.0",
@@ -19634,7 +19615,8 @@
           "url": "https://github.com/sponsors/nodeca"
         }
       ],
-      "license": "PSF-2.0"
+      "license": "PSF-2.0",
+      "peer": true
     },
     "node_modules/mocha/node_modules/chokidar": {
       "version": "5.0.0",
@@ -19642,6 +19624,7 @@
       "integrity": "sha512-TQMmc3w+5AxjpL8iIiwebF73dRDF4fBIieAqGn9RGCWaEVwQ6Fb2cGe31Yns0RRIzii5goJ1Y7xbMwo1TxMplw==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "dependencies": {
         "readdirp": "^5.0.0"
       },
@@ -19658,6 +19641,7 @@
       "integrity": "sha512-Wjlyrolmm8uDpm/ogGyXZXb1Z+Ca2B8NbJwqBVg0axK9GbBeoS7yGV6vjXnYdGm6X53iehEuxxbyiKp8QmN4Vw==",
       "dev": true,
       "license": "BlueOak-1.0.0",
+      "peer": true,
       "dependencies": {
         "minimatch": "^10.2.2",
         "minipass": "^7.1.3",
@@ -19676,6 +19660,7 @@
       "integrity": "sha512-lJJV/5dYS+RcL8uQdBDW9c9uWFLLBNRyFhnAKXw5tVqLlKZ4RMGZKv+YQ/IA3OhD+RpbJa1LLFM1FQPGyIXvOA==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "engines": {
         "node": ">=12"
       },
@@ -19699,6 +19684,7 @@
         }
       ],
       "license": "MIT",
+      "peer": true,
       "dependencies": {
         "argparse": "^3.0.3"
       },
@@ -19712,6 +19698,7 @@
       "integrity": "sha512-Kko+Y5XQ6fM+Ce3dq3m9YGxnacYZYl9cA1wZjaF3Vbry2L3i1qVg8+CAgNPsXRArPMUMCaOR7oa9Nqntc43JKA==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "engines": {
         "node": ">= 20.19.0"
       },
@@ -19726,6 +19713,7 @@
       "integrity": "sha512-1tB5mhVo7U+ETBKNf92xT4hrQa3pm0MZ0PQvuDnWgAAGHDsfp4lPSpiS6psrSiet87wyGPh9ft6wmhOMQ0hDiw==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "engines": {
         "node": ">=14.16"
       },
@@ -19739,6 +19727,7 @@
       "integrity": "sha512-MpUEN2OodtUzxvKQl72cUF7RQ5EiHsGvSsVG0ia9c5RbWGL2CI4C7EpPS8UTBIplnlzZiNuV56w+FuNxy3ty2Q==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "dependencies": {
         "has-flag": "^4.0.0"
       },
@@ -21446,6 +21435,7 @@
       "integrity": "sha512-Ie9z/WINcxxLp27BKOCHGde4ITq9UklYKDzVo1nhk5sqGEXU3FpkwP5GM2voTGJkGd9B3Otl+Q4uwSOeSUtOBA==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "dependencies": {
         "find-up": "^6.3.0"
       },
@@ -21462,6 +21452,7 @@
       "integrity": "sha512-v2ZsoEuVHYy8ZIlYqwPe/39Cy+cFDzp4dXPaxNvkEuouymu+2Jbz0PxpKarJHYJTmv2HWT3O382qY8l4jMWthw==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "dependencies": {
         "locate-path": "^7.1.0",
         "path-exists": "^5.0.0"
@@ -21479,6 +21470,7 @@
       "integrity": "sha512-gvVijfZvn7R+2qyPX8mAuKcFGDf6Nc61GdvGafQsHL0sBIxfKzA+usWn4GFC/bk+QdwPUD4kWFJLhElipq+0VA==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "dependencies": {
         "p-locate": "^6.0.0"
       },
@@ -21495,6 +21487,7 @@
       "integrity": "sha512-5b0R4txpzjPWVw/cXXUResoD4hb6U/x9BH08L7nw+GN1sezDzPdxeRvpc9c433fZhBan/wusjbCsqwqm4EIBIQ==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "dependencies": {
         "yocto-queue": "^1.0.0"
       },
@@ -21511,6 +21504,7 @@
       "integrity": "sha512-wPrq66Llhl7/4AGC6I+cqxT07LhXvWL08LNXz1fENOw0Ap4sRZZ/gZpTTJ5jpurzzzfS2W/Ge9BY3LgLjCShcw==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "dependencies": {
         "p-limit": "^4.0.0"
       },
@@ -21527,6 +21521,7 @@
       "integrity": "sha512-RjhtfwJOxzcFmNOi6ltcbcu4Iu+FL3zEj83dk4kAS+fVpTxXLO1b38RvJgT/0QwvV/L3aY9TAnyv0EOqW4GoMQ==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "engines": {
         "node": "^12.20.0 || ^14.13.1 || >=16.0.0"
       }
@@ -21537,6 +21532,7 @@
       "integrity": "sha512-4LCcse/U2MHZ63HAJVE+v71o7yOdIe4cZ70Wpf8D/IyjDKYQLV5GD46B+hSTjJsvV5PztjvHoU580EftxjDZFQ==",
       "dev": true,
       "license": "MIT",
+      "peer": true,
       "engines": {
         "node": ">=12.20"
       },
@@ -21609,7 +21605,6 @@
         }
       ],
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "nanoid": "^3.3.16",
         "picocolors": "^1.1.1",
@@ -22845,7 +22840,6 @@
       "resolved": "https://registry.npmjs.org/rxjs/-/rxjs-7.8.2.tgz",
       "integrity": "sha512-dhKf903U/PQZY6boNNtAGdWbG85WAbjT/1xYoZIC7FAY0yWapOBQVsVrDl58W86//e1VpMNBtRV4MaXfdMySFA==",
       "license": "Apache-2.0",
-      "peer": true,
       "dependencies": {
         "tslib": "^2.1.0"
       }
@@ -22900,7 +22894,6 @@
       "integrity": "sha512-9GUyuksjw70uNpb1MTYWsH9MQHOHY6kwfnkafC24+7aOMZn9+rVMBxRbLvw756mrBFbIsFg6Xw9IkR2Fnn3k+Q==",
       "dev": true,
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "chokidar": "^4.0.0",
         "immutable": "^5.0.2",
@@ -24209,7 +24202,6 @@
       "integrity": "sha512-+6erLbBm0+LROX2sPXlUYx/ux5PyE9K/a92Wrt6oA+WDAoFTdpHE5tCYCI5PNzq2y8df4rA+QgHLJuR4jNymsg==",
       "dev": true,
       "license": "BSD-2-Clause",
-      "peer": true,
       "dependencies": {
         "@jridgewell/source-map": "^0.3.3",
         "acorn": "^8.14.0",
@@ -24438,7 +24430,6 @@
       "resolved": "https://registry.npmjs.org/tough-cookie/-/tough-cookie-5.1.2.tgz",
       "integrity": "sha512-FVDYdxtnj0G6Qm/DhNPSb8Ju59ULcup3tuJxkFb5K8Bv2pUXILbf0xZWU8PX8Ov19OXljbUyveOFwRMwkXzO+A==",
       "license": "BSD-3-Clause",
-      "peer": true,
       "dependencies": {
         "tldts": "^6.1.32"
       },
@@ -24701,8 +24692,7 @@
       "version": "2.8.1",
       "resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz",
       "integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==",
-      "license": "0BSD",
+      "license": "0BSD"
-      "peer": true
     },
     "node_modules/tsyringe": {
       "version": "4.10.0",
@@ -24869,7 +24859,6 @@
       "integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==",
       "dev": true,
       "license": "Apache-2.0",
-      "peer": true,
       "bin": {
         "tsc": "bin/tsc",
         "tsserver": "bin/tsserver"
@@ -24925,19 +24914,10 @@
       "integrity": "sha512-EvRlUt/QqAEvXdQeRMYUfwOQsfzpDztykrG2owGEE/rBOBoYv9uF39ZzsNDh4lTFHhTr2XOZjKkYgueD12ZmGA==",
       "devOptional": true,
       "license": "MIT",
-      "peer": true,
       "engines": {
         "node": ">=18.17"
       }
     },
-    "node_modules/undici-types": {
-      "version": "8.9.0",
-      "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-8.9.0.tgz",
-      "integrity": "sha512-KTDyRTYX8sWmKXAikPHHSyc63CRPETMctyjKFupcC6OBLXT3xsN0e9aF7m+mIXutFWpUXuedtowG7iLOzp0kQg==",
-      "dev": true,
-      "license": "MIT",
-      "optional": true
-    },
     "node_modules/unicode-canonical-property-names-ecmascript": {
       "version": "2.0.1",
       "resolved": "https://registry.npmjs.org/unicode-canonical-property-names-ecmascript/-/unicode-canonical-property-names-ecmascript-2.0.1.tgz",
@@ -25127,7 +25107,6 @@
         "https://github.com/sponsors/ctavan"
       ],
       "license": "MIT",
-      "peer": true,
       "bin": {
         "uuid": "dist/esm/bin/uuid"
       }
@@ -25251,7 +25230,6 @@
       "integrity": "sha512-AHEzTORMbi7/7aXp7w7KZ5+DCUdSeLsJz0grY8xHpc9DHdYb0vQu6DqJfm2tOw3gDExNgPgnJTXxM6syYFHaCA==",
       "dev": true,
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "esbuild": "^0.27.0 || ^0.28.0",
         "fdir": "^6.5.0",
@@ -25442,7 +25420,6 @@
       "resolved": "https://registry.npmjs.org/webpack/-/webpack-5.112.0.tgz",
       "integrity": "sha512-NIBpsRuCll036haDDt/CfcTDfsWNR9Z782nMRMI9D377BsGcYzZDTNU5TyzhCJzPIcihAxMBaMAhq2WSaDrH8Q==",
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "@types/estree": "^1.0.8",
         "@types/json-schema": "^7.0.15",
@@ -25514,7 +25491,6 @@
       "integrity": "sha512-HNLRmamRvVavZQ+avceZifmv8hmdUjg43t6MI4SqJDwFdW7RPQwH5vzGhDRZSX59SgfbeHhLnq3g+uooWo7pVw==",
       "dev": true,
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "@types/bonjour": "^3.5.13",
         "@types/connect-history-api-fallback": "^1.5.4",
@@ -25906,7 +25882,8 @@
       "resolved": "https://registry.npmjs.org/workerpool/-/workerpool-10.0.3.tgz",
       "integrity": "sha512-6z2Iis68Wqth93/G/wJP9u+R3O+d2XTlgWChGCwuT1qLbBsOYueGRZuJ++v3mtDP5KjYdy+WzvWC+VWETSVXJA==",
       "dev": true,
-      "license": "Apache-2.0"
+      "license": "Apache-2.0",
+      "peer": true
     },
     "node_modules/wrap-ansi": {
       "version": "9.0.2",
@@ -26197,7 +26174,6 @@
       "integrity": "sha512-jIH9yLR9wqr0wOS0TpBvo/g/2UgZH5qePVbjgRliiF0BYvOZyaBknKsF+x9Iht0O6sqgnB93rCICdOZFecJuDw==",
       "dev": true,
       "license": "MIT",
-      "peer": true,
       "dependencies": {
         "pend": "~1.2.0"
       },
@@ -26314,7 +26290,6 @@
       "integrity": "sha512-AvvthqfqrAhNH9dnfmrfKzX5upOdjUVJYFqNSlkmGf64gRaTzlPwz99IHYnVs28qYAybvAlBV+H7pn0saFY4Ig==",
       "dev": true,
       "license": "MIT",
-      "peer": true,
       "funding": {
         "url": "https://github.com/sponsors/colinhacks"
       }
@@ -26333,8 +26308,7 @@
       "version": "0.15.1",
       "resolved": "https://registry.npmjs.org/zone.js/-/zone.js-0.15.1.tgz",
       "integrity": "sha512-XE96n56IQpJM7NAoXswY3XRLcWFW83xe0BiAOeMD7K5k5xecOeul3Qcpx6GqEeeHNkW5DWL5zOyTbEfB4eti8w==",
-      "license": "MIT",
+      "license": "MIT"
-      "peer": true
     }
   }
 }

client/package.json

@@ -37,7 +37,9 @@
     "compodoc:build-and-serve": "compodoc -p tsconfig.doc.json -s --name 'Data Controller Client'",
     "compodoc:serve": "compodoc -s --name 'Data Controller Client'",
     "lighthouse": "lhci autorun",
-    "ng": "ng"
+    "ng": "ng",
+    "test:tz-west": "TZ=America/New_York npx ng test --no-watch --no-progress --browsers ChromeHeadlessCI --include='**/va-*.spec.ts'",
+    "test:tz-east": "TZ=Pacific/Auckland npx ng test --no-watch --no-progress --browsers ChromeHeadlessCI --include='**/va-*.spec.ts'"
   },
   "private": true,
   "dependencies": {
@@ -54,7 +56,7 @@
     "@clr/angular": "file:libraries/clr-angular-17.9.0.tgz",
     "@clr/icons": "^13.0.2",
     "@clr/ui": "file:libraries/clr-ui-17.9.0.tgz",
-    "@handsontable/angular-wrapper": "18.1.1",
+    "@handsontable/angular-wrapper": "18.1.2",
     "@sasjs/adapter": "5.1.2",
     "@sasjs/utils": "4.0.0",
     "@sheet/crypto": "file:libraries/sheet-crypto.tgz",
@@ -69,7 +71,7 @@
     "d3-graphviz": "^5.0.2",
     "exceljs": "^4.4.0",
     "fs-extra": "^7.0.1",
-    "handsontable": "18.1.1",
+    "handsontable": "18.1.2",
     "https-browserify": "1.0.0",
     "hyperformula": "^2.5.0",
     "iconv-lite": "^0.5.0",

client/src/app/_globals.ts

@@ -1,4 +1,5 @@
 import { QueryClause } from './models/TableData'
+import { VaParmapTarget } from './shared/utils/parse-va-parmap'
 
 /**
  * Filtering cache info, to be reused when filtering modal is re-open
@@ -61,6 +62,21 @@ export const globals: {
    * Defaults to 'live'; toggled by the Auto-apply checkbox in the editor.
    */
   vaApplyMode: 'live' | 'confirm'
+  /**
+   * Explicit VA parameter -> DC column mappings, parsed from the `va_parmap`
+   * URL parameter (see parseVaParmap). Empty unless configured; entries override
+   * VA's default same-name (parameter label -> column) resolution, so two DC
+   * instances over one table can filter the same column from differently-named
+   * VA parameters.
+   */
+  vaParamMap: Map<string, VaParmapTarget>
+  /**
+   * `va_parmap` entries that could not be used at all (not a plain SAS column
+   * name, or missing a side of the `-`). Surfaced to the report author, because
+   * these are hand-written in the report's URL and a typo otherwise looks like
+   * a working, unfiltered view.
+   */
+  vaParmapProblems: string[]
   rootParam: string
   dcLib: string
   xlmaps: XLMapListItem[]
@@ -77,6 +93,8 @@ export const globals: {
 } = {
   embed: false,
   vaApplyMode: 'live',
+  vaParamMap: new Map(),
+  vaParmapProblems: [],
   rootParam: <string>'',
   dcLib: '',
   xlmaps: [],

client/src/app/app.component.ts

@@ -12,6 +12,7 @@ import '@clr/icons'
 import '@clr/icons/shapes/all-shapes'
 import { globals } from './_globals'
 import { parseEmbedParam } from './shared/utils/parse-embed-param'
+import { parseVaParmap } from './shared/utils/parse-va-parmap'
 import moment from 'moment'
 import { EventService } from './services/event.service'
 import { AppService } from './services/app.service'
@@ -163,6 +164,21 @@ export class AppComponent {
       globals.embed = embed
       this.embed = embed
     }
+    // VA parameter -> DC column mappings for this instance, held on globals
+    // (like `embed` and `vaApplyMode`) so in-app navigation that rebuilds the
+    // editor keeps them - e.g. returning from the stage / approve pages.
+    if (globals.embed === 'va') {
+      const parmap = parseVaParmap(hashQuery ?? '')
+      globals.vaParamMap = parmap.map
+      globals.vaParmapProblems = parmap.problems
+      // Visible in the browser console as well as the editor banner: the entry
+      // is hand-written in the report's URL, so a typo has to be findable when
+      // someone is looking at the report rather than at DC's UI.
+      for (const problem of parmap.problems) {
+        // eslint-disable-next-line no-console
+        console.warn('Data Controller: ' + problem)
+      }
+    }
 
     this.subscribeToShowAbortModal()
     this.subscribeToRequestsModal()
@@ -314,9 +330,12 @@ export class AppComponent {
    */
   public subscribeToShowAbortModal() {
     this.eventService.onShowAbortModal.subscribe((sasjsAbort: InfoModal) => {
-      let abortId = this.sasjsAborts.length + 1
+      /* One message at a time.  Pushing to the array stacked a second modal on
-      sasjsAbort.id = abortId
+         top of the one already open - two dismissals for one failure, and the
-      this.sasjsAborts.push(sasjsAbort)
+         lower one was often a duplicate of the upper.  Replacing keeps the
+         newest message, which is the one the user has to act on. */
+      sasjsAbort.id = Date.now()
+      this.sasjsAborts = [sasjsAbort]
       this.cdr.detectChanges() //Changes were not triggered while hot is focused
     })
   }

client/src/app/deploy/deploy.component.html

@@ -1,7 +1,36 @@
 <main class="content-area position-relative">
+  <!-- An embedded instance (a report object) has no report author who can deploy:
+       showing the wizard there only produces a screen whose buttons cannot work,
+       because the deployment needs an authenticated, non-embedded session. Name
+       the state instead, and keep the wizard for standalone use. -->
+  @if (isEmbedded) {
+    <div class="clr-row">
+      <div id="not-configured" class="card">
+        <div class="card-header">Data Controller is not configured</div>
+        <div class="card-block">
+          <div class="card-text">
+            <p class="mt-0">
+              This Data Controller instance has not been deployed yet, so it
+              cannot be shown inside a report.
+            </p>
+            <p>
+              Deployment has to be done once, in its own browser tab, by someone
+              who can write to the application location:
+              <a [href]="deployUrl" target="_blank">{{ deployUrl }}</a>
+            </p>
+            <p>
+              When it is configured, reload the report and this object will show
+              the editor.
+            </p>
+          </div>
+        </div>
+      </div>
+    </div>
+  }
+
   <div class="clr-row">
     <!-- T&C section -->
-    @if (step === 0) {
+    @if (!isEmbedded && step === 0) {
       <div id="TCS" class="card">
         <div class="card-header">Terms and Conditions</div>
         <div class="card-block">
@@ -35,7 +64,7 @@
   </div>
   <!-- T&C section end -->
 
-  @if (step > 0) {
+  @if (!isEmbedded && step > 0) {
     @switch (true) {
       @case (sasJsConfig.serverType === ServerType.SasViya) {
         <div class="autodeploy-section card">

client/src/app/deploy/deploy.component.spec.ts

@@ -0,0 +1,229 @@
+import { NO_ERRORS_SCHEMA } from '@angular/core'
+import { ComponentFixture, TestBed } from '@angular/core/testing'
+import { ServerType } from '@sasjs/utils/types/serverType'
+import { DeployComponent } from './deploy.component'
+import { standaloneUrl } from './standalone-url'
+import { globals } from '../_globals'
+import { AppStoreService } from '../services/app-store.service'
+import { SasService } from '../services/sas.service'
+
+/**
+ * An embedded instance cannot deploy itself: the deployment needs an
+ * authenticated, standalone session, and the person looking at a report object
+ * is not the person who installs the app. The component therefore withholds the
+ * wizard and names the state instead.
+ *
+ * These cases render the template, so they fail if the `!isEmbedded` guards in
+ * deploy.component.html are dropped - asserting the getter alone would not.
+ */
+describe('DeployComponent embed guard', () => {
+  let fixture: ComponentFixture<DeployComponent>
+  let component: DeployComponent
+
+  const originalEmbed = globals.embed
+
+  const configure = () =>
+    TestBed.configureTestingModule({
+      declarations: [DeployComponent],
+      providers: [
+        {
+          provide: AppStoreService,
+          useValue: {
+            getDcAdapterSettings: () => ({
+              appLoc: '/Public/app/dc',
+              dcPath: '/tmp/dc'
+            })
+          }
+        },
+        {
+          provide: SasService,
+          useValue: {
+            getSasjsInstance: () => ({}),
+            getSasjsConfig: () => ({ serverType: ServerType.Sasjs })
+          }
+        }
+      ],
+      // The wizard renders child components this spec does not exercise.
+      schemas: [NO_ERRORS_SCHEMA]
+    }).compileComponents()
+
+  const render = () => {
+    fixture = TestBed.createComponent(DeployComponent)
+    component = fixture.componentInstance
+    fixture.detectChanges()
+  }
+
+  const tcsSection = () =>
+    fixture.nativeElement.querySelector('#TCS') as HTMLElement | null
+  const notConfigured = () =>
+    fixture.nativeElement.querySelector('#not-configured') as HTMLElement | null
+  const deploySection = () =>
+    fixture.nativeElement.querySelector(
+      '.autodeploy-section'
+    ) as HTMLElement | null
+
+  afterEach(() => {
+    globals.embed = originalEmbed
+    TestBed.resetTestingModule()
+  })
+
+  it('offers the wizard when running in its own tab', () => {
+    globals.embed = false
+    configure()
+    render()
+
+    expect(tcsSection()).toBeTruthy()
+    expect(notConfigured()).toBeNull()
+    expect(component.isEmbedded).toBeFalse()
+  })
+
+  it('replaces the wizard with the notice for a VA embed', () => {
+    globals.embed = 'va'
+    configure()
+    render()
+
+    expect(tcsSection()).toBeNull()
+    expect(notConfigured()).toBeTruthy()
+  })
+
+  it('replaces the wizard with the notice for a plain embed too', () => {
+    globals.embed = true
+    configure()
+    render()
+
+    expect(tcsSection()).toBeNull()
+    expect(notConfigured()).toBeTruthy()
+  })
+
+  it('names the standalone URL in the embedded notice', () => {
+    globals.embed = 'va'
+    configure()
+    render()
+
+    const link = notConfigured()!.querySelector('a') as HTMLAnchorElement
+
+    expect(link.getAttribute('href')).toBe(component.deployUrl)
+    expect(link.textContent).toBe(component.deployUrl)
+  })
+
+  it('withholds the deploy section after the terms are accepted, when embedded', () => {
+    globals.embed = 'va'
+    configure()
+    render()
+
+    component.step = 1
+    fixture.detectChanges()
+
+    expect(deploySection()).toBeNull()
+  })
+
+  it('shows the deploy section after the terms are accepted, standalone', () => {
+    globals.embed = false
+    configure()
+    render()
+
+    component.step = 1
+    fixture.detectChanges()
+
+    expect(deploySection()).toBeTruthy()
+  })
+})
+
+describe('DeployComponent onNavigateToHome', () => {
+  const originalEmbed = globals.embed
+  const originalHref = window.location.href
+
+  afterEach(() => {
+    globals.embed = originalEmbed
+    history.replaceState(null, '', originalHref)
+    TestBed.resetTestingModule()
+  })
+
+  it('opens the standalone URL, merging an embed tab into the app URL', () => {
+    // Karma serves the runner with its own query, so the app URL is set here
+    // rather than read from window.location: the assertion has to see the
+    // app's query survive the embed parameters being dropped. The shape is
+    // the real SAS 9 one - the app's `_program` query in the search part,
+    // the embed switches in the hash query.
+    history.replaceState(
+      null,
+      '',
+      '/deploy-tc?_program=/Public/app/dc&_debug=131#/editor?embed=va&va_parmap=A-B'
+    )
+    const open = spyOn(window, 'open')
+    TestBed.configureTestingModule({
+      declarations: [DeployComponent],
+      providers: [
+        {
+          provide: AppStoreService,
+          useValue: { getDcAdapterSettings: () => ({ appLoc: '', dcPath: '' }) }
+        },
+        {
+          provide: SasService,
+          useValue: {
+            getSasjsInstance: () => ({}),
+            getSasjsConfig: () => ({ serverType: ServerType.Sasjs })
+          }
+        }
+      ],
+      schemas: [NO_ERRORS_SCHEMA]
+    })
+    const fixture = TestBed.createComponent(DeployComponent)
+
+    fixture.componentInstance.onNavigateToHome()
+
+    const opened = open.calls.mostRecent().args[0] as string
+    expect(opened).toBe(
+      window.location.origin + '/deploy-tc?_program=/Public/app/dc&_debug=131'
+    )
+    expect(opened).not.toContain('#')
+    expect(opened).not.toContain('embed=')
+  })
+})
+
+describe('standaloneUrl', () => {
+  const origin = window.location.origin
+
+  it('keeps the path of a plain URL', () => {
+    expect(standaloneUrl(`${origin}/SASJobExecution/dc`)).toBe(
+      `${origin}/SASJobExecution/dc`
+    )
+  })
+
+  it('keeps a query the app needs, and drops the hash route', () => {
+    expect(standaloneUrl(`${origin}/path?x=1#/deploy`)).toBe(
+      `${origin}/path?x=1`
+    )
+  })
+
+  it('keeps `_program=`, which is how SAS 9 and Viya address the app', () => {
+    // The whole app URL is the query here: /SASJobExecution/?_program=... is
+    // the standalone app, so dropping the query would open a blank executor.
+    const sas9 =
+      `${origin}/SASJobExecution/?_program=/Public/app/dc/services/public/startupservice` +
+      '&_debug=131'
+    expect(standaloneUrl(sas9)).toBe(sas9)
+  })
+
+  it('drops a bare embed query', () => {
+    expect(standaloneUrl(`${origin}/app?embed=va`)).toBe(`${origin}/app`)
+  })
+
+  it('drops the va_parmap entries along with the embed switch', () => {
+    expect(
+      standaloneUrl(
+        `${origin}/app?embed=va&va_parmap=A-B&va_parmap=C-D#/editor/x`
+      )
+    ).toBe(`${origin}/app`)
+  })
+
+  it('keeps a query parameter that merely starts with va_', () => {
+    expect(standaloneUrl(`${origin}/app?embed=va&va_rowlimit=100`)).toBe(
+      `${origin}/app?va_rowlimit=100`
+    )
+  })
+
+  it('resolves a relative URL against the current origin', () => {
+    expect(standaloneUrl('/app/#/deploy')).toBe(`${origin}/app/`)
+  })
+})

client/src/app/deploy/deploy.component.ts

@@ -4,6 +4,8 @@ import { SASjsConfig } from '@sasjs/adapter'
 import { ServerType } from '@sasjs/utils/types/serverType'
 import { AppStoreService } from '../services/app-store.service'
 import { DcAdapterSettings } from '../models/DcAdapterSettings'
+import { globals } from '../_globals'
+import { standaloneUrl } from './standalone-url'
 
 @Component({
   selector: 'app-deploy',
@@ -44,6 +46,20 @@ export class DeployComponent implements OnInit {
     this.setDeployDefaults()
   }
 
+  /**
+   * True when DC is running inside a report rather than in its own tab. The
+   * deployment needs an authenticated, standalone session, so the wizard is not
+   * offered here - a report object cannot deploy the app that it is displaying.
+   */
+  get isEmbedded(): boolean {
+    return !!globals.embed
+  }
+
+  /** The standalone URL an administrator deploys from, for the embedded notice. */
+  get deployUrl(): string {
+    return standaloneUrl(window.location.href)
+  }
+
   /**
    * Setting default values used for deploy request
    */
@@ -60,6 +76,8 @@ export class DeployComponent implements OnInit {
   }
 
   public onNavigateToHome() {
-    window.open(location.href.split('#')[0], '_blank')
+    // Same helper as deployUrl: splitting on '#' left the query string in
+    // place, so a tab opened from `?embed=va#/deploy` stayed embedded.
+    window.open(standaloneUrl(window.location.href), '_blank')
   }
 }

client/src/app/deploy/standalone-url.ts

@@ -0,0 +1,29 @@
+/**
+ * The URL a Data Controller instance runs at when it is not embedded: the same
+ * URL, without the parts that make it an embed - the `embed` switch, the
+ * `va_parmap` entries, and the hash route.
+ *
+ * The rest of the query is kept. On SAS 9 and Viya the app is addressed
+ * entirely through it (`/SASJobExecution/?_program=/.../startupservice`), so
+ * dropping the query would open a blank executor rather than the app, and any
+ * other parameter the deployment relies on would go the same way.
+ *
+ * Only these two parameters are embed parameters. A later parameter that
+ * happens to start with `va_` belongs to the app, not the embed, so it stays.
+ *
+ * Parsing with URL also handles the relative hrefs the adapter can hand back,
+ * which plain string splicing cannot.
+ */
+const EMBED_PARAMS = ['embed', 'va_parmap']
+
+export const standaloneUrl = (href: string): string => {
+  const url = new URL(href, window.location.origin)
+  url.hash = ''
+  for (const key of Array.from(url.searchParams.keys())) {
+    if (EMBED_PARAMS.includes(key.toLowerCase())) {
+      url.searchParams.delete(key)
+    }
+  }
+
+  return url.toString()
+}

client/src/app/editor/editor.component.html

@@ -459,6 +459,43 @@
           <!--<hr>-->
           <!--{{ validationDone }}-->
 
+          <!-- VA embed: something about a `va_parmap` entry in this report
+              object's URL needs the report author's attention - it could not be
+              used, it was applied but AND-ed against another parameter, or it
+              names a parameter VA never sends. Shown as a banner, not just a
+              console warning, because
+              the entries are written by the report author and a typo otherwise
+              looks like a working filter over the whole table. -->
+          @if (isVaEmbed && vaMappingWarnings.length) {
+            <clr-alert
+              class="va-parmap-warning"
+              [clrAlertType]="'warning'"
+              [clrAlertClosable]="false"
+            >
+              <clr-alert-item>
+                <div class="alert-text">
+                  <strong>VA parameter mapping problem</strong>
+                  @for (warning of vaMappingWarnings; track warning) {
+                    <div>{{ warning }}</div>
+                  }
+                  <!-- The reader is the report author, who can only fix this in
+                       the report: name the parameter they edit, and say what a
+                       correct entry looks like, so the message is actionable
+                       without the docs. -->
+                  <div>
+                    Fix: edit the <code>va_parmap</code> entries in the URL of
+                    this Data Controller object in the report. An entry is
+                    <code>parameter-column</code>, optionally
+                    <code>:ADD(anchor,offset)</code> or
+                    <code>:RANGE(period,offset)</code> - for example
+                    <code>va_parmap=SEL_DATE-D_DATE:ADD(QUARTER_START,-1)</code
+                    >.
+                  </div>
+                </div>
+              </clr-alert-item>
+            </clr-alert>
+          }
+
           <div class="hot-wrapper clr-flex-1">
             <hot-table
               #hotInstance

client/src/app/editor/editor.component.scss

@@ -27,3 +27,15 @@
   font-style: italic;
   opacity: 0.7;
 }
+
+/* VA embed: `va_parmap` diagnostic banner. Clarity's alert lays `.alert-title`
+   and `.alert-text` out on one row, which pushes a long message off the right
+   edge; this keeps the whole message in the flow and wraps it. */
+.va-parmap-warning {
+  width: 100%;
+
+  .alert-text {
+    white-space: normal;
+    overflow-wrap: anywhere;
+  }
+}

client/src/app/editor/editor.component.ts

@@ -73,6 +73,7 @@ import {
 } from '../shared/dc-validator/utils/autoEscapedCellTracker'
 import { HyperFormula } from 'hyperformula'
 import { parseLabelsParam } from '../shared/utils/parse-labels-param'
+import { VaParmapTarget } from '../shared/utils/parse-va-parmap'
 import { getDisplayColHeaders } from '../shared/utils/display-col-headers'
 import { buildColInfoHtml } from '../shared/utils/col-info-html'
 import { globals } from '../_globals'
@@ -485,6 +486,46 @@ export class EditorComponent
   get isVaEmbed() {
     return globals.embed === 'va'
   }
+  /**
+   * Explicit VA parameter -> DC column mappings (`va_parmap`) for this instance.
+   * Parsed once at app init (see parseVaParmap) and held on globals so the
+   * editor reloads a filter apply performs keep using the same mapping.
+   */
+  get vaParamMap(): Map<string, VaParmapTarget> {
+    return globals.vaParamMap
+  }
+  /** `va_parmap` entries that name a column this table does not have. */
+  public vaUnresolvedMappings: string[] = []
+  /** `va_parmap` keys that no VA message has named (a parameter-side typo). */
+  public vaUnmatchedMappings: string[] = []
+  /** Parameters in this message that filter the same column (an AND-ed pair). */
+  public vaSameColumnMappings: string[] = []
+  /** Transform mappings this message cannot apply (non-temporal column or value). */
+  public vaTransformProblems: string[] = []
+  /** The warning text last written to the console, so repeated VA messages log once. */
+  private vaWarningsReported = ''
+  /**
+   * Everything wrong with this instance's `va_parmap` entries, in the report
+   * author's terms: entries that could not be parsed at all, plus entries that
+   * name a column the open table does not have. Shown as a banner in the editor
+   * - a report developer is usually not going to read the browser console.
+   */
+  get vaMappingWarnings(): string[] {
+    return [...globals.vaParmapProblems, ...this.vaRuntimeWarnings]
+  }
+  /**
+   * The warnings this instance discovers for itself, i.e. everything except the
+   * parse-time problems - those are logged once at app init (`app.component`),
+   * so logging them again here would double every line in the console.
+   */
+  get vaRuntimeWarnings(): string[] {
+    return [
+      ...this.vaUnresolvedMappings,
+      ...this.vaUnmatchedMappings,
+      ...this.vaSameColumnMappings,
+      ...this.vaTransformProblems
+    ]
+  }
   /**
    * VA filter mode. Live = auto-apply each VA change; confirm = stage the change
    * and wait for the user to click Apply. Backed by globals (toggled by the
@@ -575,6 +616,8 @@ export class EditorComponent
   public hasBaseDropZoneOver = false
   public hasAnotherDropZoneOver = false
   public headerPks: string[] = []
+  /** Derived (computed) columns from MPE_DERIVED - read-only, and marked as such */
+  public derivedVars: string[] = []
   public columnLevelSecurityFlag = false
   public dateTimeHeaders: string[] = []
   public timeHeaders: string[] = []
@@ -1089,6 +1132,20 @@ export class EditorComponent
     return colRules?.readOnly
   }
 
+  /**
+   * A derived (computed) column - configured in MPE_DERIVED, created at read
+   * time. Read-only, and styled apart from an ordinary read-only column so the
+   * user can see that there is nothing to edit rather than that they may not.
+   */
+  isDerivedCol(col: string | number) {
+    const name =
+      typeof col === 'number'
+        ? (this.hotInstance?.colToProp(col) as string) || this.columnHeader[col]
+        : col
+
+    return !!name && this.derivedVars.includes(name.toUpperCase())
+  }
+
   isColHeader(col: string) {
     return this.headerArray.indexOf(col.toUpperCase()) > -1
   }
@@ -2757,17 +2814,24 @@ export class EditorComponent
             this.disableSubmit = true
             this.submit = false
 
-            const errorText =
+            /* sasService has already shown the SAS message when it rejected
-              typeof err.adapterRespnse === 'string'
+               (the abort, or the adapter's error), so reporting it again here
-                ? err.adapterRespnse
+               stacked a second modal on top of the first - and read
-                : JSON.stringify(err.adapterRespnse)
+               'undefined' because it looked for a property that does not
+               exist.  Only speak up when nothing was shown. */
+            if (!err?.errorShown) {
+              const errorText =
+                typeof err?.adapterResponse?.error === 'string'
+                  ? err.adapterResponse.error
+                  : JSON.stringify(err?.adapterResponse ?? err)
 
-            this.eventService.showAbortModal(
+              this.eventService.showAbortModal(
-              'editors/stagedata',
+                'editors/stagedata',
-              `Submit request failed\n${errorText}`,
+                `Submit request failed\n${errorText}`,
-              null,
+                null,
-              'Submit error'
+                'Submit error'
-            )
+              )
+            }
 
             return false
           })
@@ -3579,7 +3643,7 @@ export class EditorComponent
     const msg = this.vaMessaging.latestMessage()
     if (!msg) return false
 
-    const clauses = this.vaFilter.buildInitialClauses(msg)
+    const clauses = this.vaFilter.buildInitialClauses(msg, this.vaParamMap)
     if (clauses.length === 0) return false
 
     try {
@@ -3941,8 +4005,11 @@ export class EditorComponent
 
     // getdata serves a selection over DC_MAXOBS_WEBEDIT / DC_MAXCELLS_WEBEDIT
     // truncated to the first rows within the limit, and carries the reason
-    // here.  Empty means the whole selection is in the grid.
+    // here.  Empty means the whole selection is in the grid - and a blank
-    this.truncationNote = response.data.sasparams[0].TRUNC_MSG || ''
+    // character arrives as a single blank (mp_jsonout cannot emit a zero-length
+    // string, since SAS has none), so trim it: whitespace is "no note", not a
+    // warning with an empty tooltip.
+    this.truncationNote = (response.data.sasparams[0].TRUNC_MSG || '').trim()
 
     const hot = this.hotInstance
 
@@ -3964,6 +4031,12 @@ export class EditorComponent
 
     this.headerColumns = response.data.sasparams[0].COLHEADERS.split(',')
     this.headerPks = response.data.sasparams[0].PK.split(' ')
+    // Derived (computed) columns, named by getdata from MPE_DERIVED - read-only
+    // through a READONLY DQ rule, and marked in the grid so they read as
+    // computed rather than merely non-editable.
+    this.derivedVars = (response.data.sasparams[0].DERIVEDVARS || '')
+      .split(' ')
+      .filter((col: string) => !!col)
     this.columnLevelSecurityFlag = !!response.data.sasparams[0].CLS_FLAG
 
     if (this.columnLevelSecurityFlag)
@@ -4338,9 +4411,37 @@ export class EditorComponent
           // header columns styling - primary keys
           const isPKCol = column && this.isColPk(column)
           const isReadonlyCol = column && this.isReadonlyCol(column)
+          const isDerivedCol = column && this.isDerivedCol(column)
 
           if (isPKCol) th.classList.add('primaryKeyHeaderStyle')
-          if (isReadonlyCol && !isPKCol) th.classList.add('readonlyCell')
+          if (isReadonlyCol && !isPKCol && !isDerivedCol) {
+            th.classList.add('readonlyCell')
+          }
+
+          // a derived column is computed, not merely locked - give it its own
+          // hue and an 'fx' marker rather than the read-only dimming
+          if (isDerivedCol) {
+            th.classList.add('dc-derived-header')
+            // the label says what the column is - for a derived variable that
+            // is the configured VAR_LABEL, or the generated name=expression
+            // when the administrator left it blank.  A label that merely
+            // repeats the name adds nothing, so it falls back to the rule.
+            const derivedLabel = (this.cols || [])
+              .filter((c: any) => c.NAME === column)
+              .map((c: any) => String(c.LABEL || '').trim())
+              .find(
+                (l: string) =>
+                  l.length > 0 &&
+                  l.toUpperCase() !== String(column).toUpperCase()
+              )
+            th.title = derivedLabel
+              ? `${derivedLabel} - derived variable, computed at read time and cannot be edited`
+              : 'Derived variable - computed at read time and cannot be edited'
+            const firstDerivedCol = this.columnHeader.findIndex(
+              (name: string) => this.isDerivedCol(name)
+            )
+            if (col === firstDerivedCol) th.classList.add('dc-derived-first')
+          }
 
           // Remove header arrow from Delete column
           if (col === 0) {
@@ -4356,6 +4457,7 @@ export class EditorComponent
           cellProperties: Handsontable.CellProperties
         ) => {
           const isReadonlyCol = col && this.isReadonlyCol(col)
+          const isDerivedCol = col && this.isDerivedCol(col)
 
           // Check if this cell should be marked as invalid due to duplicate primary key values
           // Only applies to primary key columns (col 1 through readOnlyFields)
@@ -4379,7 +4481,26 @@ export class EditorComponent
           }
 
           // Add readonlyCell class for readonly columns to maintain original styling
-          if (isReadonlyCol && !classes.includes('readonlyCell')) {
+          if (isDerivedCol) {
+            // a derived cell carries its own tint; the read-only dimming would
+            // wash it out, and "greyed out" is a different message from
+            // "computed here"
+            classes = classes.filter((c: string) => c !== 'readonlyCell')
+            if (!classes.includes('dc-derived-cell')) {
+              classes.push('dc-derived-cell')
+            }
+            // derived columns are appended after the physical ones - mark the
+            // start of the block so it reads as one group
+            const firstDerivedCol = this.columnHeader.findIndex(
+              (name: string) => this.isDerivedCol(name)
+            )
+            if (
+              col === firstDerivedCol &&
+              !classes.includes('dc-derived-first')
+            ) {
+              classes.push('dc-derived-first')
+            }
+          } else if (isReadonlyCol && !classes.includes('readonlyCell')) {
             classes.push('readonlyCell')
           }
 
@@ -4968,7 +5089,53 @@ export class EditorComponent
 
     this.updateVaFilterText(msg)
 
-    const clauses = this.vaFilter.buildClauses(msg, labelToVar, this.cols)
+    // A `va_parmap` entry naming a column this table does not have is skipped by
+    // buildClauses. Report it (banner + console, once per distinct set) so the
+    // report author sees a misconfiguration instead of an unfiltered grid.
+    this.vaUnresolvedMappings = this.vaFilter.unresolvedMappings(
+      msg,
+      this.vaParamMap,
+      this.cols,
+      this.libds
+    )
+    // A key naming no parameter VA has ever sent is a typo on the parameter
+    // side: the mapping never applies. Only judged once parameters are actually
+    // arriving, since VA sends just the parameters its query uses.
+    this.vaUnmatchedMappings = this.vaFilter.unmatchedMappings(
+      this.vaParamMap,
+      this.vaMessaging.seenParamKeys,
+      this.vaMessaging.lastMessageParamsStable
+    )
+    // Two parameters filtering one column are AND-ed, so differing values leave
+    // an empty grid. Reported, not rewritten: turning that AND into an OR is a
+    // filter-semantics decision, taken in buildClauses and not in a diagnostic.
+    this.vaSameColumnMappings = this.vaFilter.sameColumnMappings(
+      msg,
+      this.vaParamMap,
+      this.cols
+    )
+    // A transform that cannot run here (char column, value that is not a date) is
+    // skipped by the clause builder, so the banner has to say why.
+    this.vaTransformProblems = this.vaFilter.unsupportedTransforms(
+      msg,
+      this.vaParamMap,
+      this.cols
+    )
+    const warnings = this.vaRuntimeWarnings.join(' | ')
+    if (warnings !== this.vaWarningsReported) {
+      this.vaWarningsReported = warnings
+      for (const warning of this.vaRuntimeWarnings) {
+        // eslint-disable-next-line no-console
+        console.warn('Data Controller: ' + warning)
+      }
+    }
+
+    const clauses = this.vaFilter.buildClauses(
+      msg,
+      labelToVar,
+      this.cols,
+      this.vaParamMap
+    )
     const signature = this.vaFilter.signature(clauses)
 
     if (signature === this.vaMessaging.filterSignature) {

client/src/app/models/TableData.ts

@@ -51,6 +51,11 @@ export interface SASParam {
   TRUNCATED?: number
   /** Reason shown in the tooltip when TRUNCATED is 1, else empty */
   TRUNC_MSG?: string
+  /**
+   * Space separated list of the derived (computed) columns from MPE_DERIVED,
+   * upper case. Empty when the table has none.
+   */
+  DERIVEDVARS?: string
 }
 
 export interface ExcelRule {

client/src/app/multi-dataset/multi-dataset.component.ts

@@ -868,6 +868,21 @@ export class MultiDatasetComponent implements OnInit, AfterViewInit {
       return this.sasStoreService
         .callService(myParams, 'SASControlTable', 'editors/getdata', libds)
         .then((res: EditorsGetDataServiceResponse) => {
+          // getdata serves a selection over DC_MAXOBS_WEBEDIT / DC_MAXCELLS_WEBEDIT
+          // truncated, rather than aborting the way it used to. This flow compares
+          // the whole table against the upload, so a partial read would compute
+          // the changes against the wrong starting state - refuse the table
+          // instead of submitting differences nobody asked for.
+          const sasParams = res?.data?.sasparams?.[0]
+
+          if (sasParams && Number(sasParams.TRUNCATED) === 1) {
+            this.eventService.showInfoModal(
+              'Upload error',
+              `${libds} holds more than the EDIT limit allows (${sasParams.TRUNC_MSG}), so it cannot be compared with the upload. Raise DC_MAXOBS_WEBEDIT / DC_MAXCELLS_WEBEDIT, or submit the table without the multi-table upload.`
+            )
+            return undefined
+          }
+
           return res
         })
         .catch((err: any) => {

client/src/app/services/sas.service.spec.ts

@@ -186,7 +186,8 @@ describe('SasService openServiceInNewWindow', () => {
 
     expect(openSpy).toHaveBeenCalledWith(
       'https://sas.example.com/SASJobExecution/?_program=/Public/app/dc/services/public/startupservice&_contextname=ctx&_debug=131',
-      '_blank'
+      '_blank',
+      'noopener'
     )
   })
 
@@ -199,7 +200,8 @@ describe('SasService openServiceInNewWindow', () => {
 
     expect(openSpy).toHaveBeenCalledWith(
       'https://sas.example.com/SASJobExecution/?_program=/Public/app/dc/services/public/startupservice&_debug=131',
-      '_blank'
+      '_blank',
+      'noopener'
     )
   })
 })

client/src/app/services/sas.service.ts

@@ -286,7 +286,10 @@ export class SasService {
                 adapterResponse: {
                   error: abortMsg
                 },
-                log: sasRequest?.logFile
+                log: sasRequest?.logFile,
+                /* the modal above already told the user: a caller that also
+                   reports the failure would stack a second one on top */
+                errorShown: !wrapperOptions?.suppressSuccessAbortModal
               })
             }
 
@@ -329,7 +332,8 @@ export class SasService {
                 adapterResponse: {
                   error: errorMessage
                 },
-                log: sasRequest?.logFile
+                log: sasRequest?.logFile,
+                errorShown: !wrapperOptions?.suppressErrorAbortModal
               })
             }
 
@@ -867,7 +871,10 @@ export class SasService {
       params
     })
 
-    window.open(url, '_blank')
+    // noopener: the SAS log page is on a different origin (Viya's serverUrl is
+    // not the app's), and without it the opened window keeps a handle on
+    // window.opener and can navigate this tab.
+    window.open(url, '_blank', 'noopener')
   }
 
   public getServerType(): string {

client/src/app/services/va-filter.service.spec.ts

@@ -1,4 +1,5 @@
 import { VaFilterService } from './va-filter.service'
+import { VaParmapTarget } from '../shared/utils/parse-va-parmap'
 import { VaMessage } from './va-messaging.service'
 import { HelperService } from './helper.service'
 
@@ -14,6 +15,7 @@ function msg(parameters: any[]): VaMessage {
 
 describe('VaFilterService', () => {
   let service: VaFilterService
+  let helperService: HelperService
 
   // DC column metadata used by the metadata path. cellValidation[0] is the
   // delete column and must be skipped by buildLabelToVarMap.
@@ -42,7 +44,8 @@ describe('VaFilterService', () => {
   ]
 
   beforeEach(() => {
-    service = new VaFilterService(new HelperService({} as any))
+    helperService = new HelperService({} as any)
+    service = new VaFilterService(helperService)
   })
 
   const labelToVar = () =>
@@ -128,6 +131,41 @@ describe('VaFilterService', () => {
       expect(build([{ label: 'SOME_DATE', value: '2025/03/31' }])).toHaveSize(1)
     })
 
+    it('parses word dates, in either order and with an optional comma', () => {
+      const expected = "'01JAN2026'd"
+      // The reported shape: a date parameter displayed as WORDDATE.
+      expect(
+        build([{ label: 'SOME_DATE', value: 'January 1, 2026' }])[0].RAW_VALUE
+      ).toBe(expected)
+      expect(
+        build([{ label: 'SOME_DATE', value: 'Jan 1, 2026' }])[0].RAW_VALUE
+      ).toBe(expected)
+      expect(
+        build([{ label: 'SOME_DATE', value: 'Jan. 1, 2026' }])[0].RAW_VALUE
+      ).toBe(expected)
+      expect(
+        build([{ label: 'SOME_DATE', value: '1 January 2026' }])[0].RAW_VALUE
+      ).toBe(expected)
+      expect(
+        build([{ label: 'SOME_DATE', value: '1-Jan-2026' }])[0].RAW_VALUE
+      ).toBe(expected)
+    })
+
+    it('matches a spelled-out month case-insensitively, and keeps DATE9 working', () => {
+      expect(
+        build([{ label: 'SOME_DATE', value: 'DECEMBER 31, 2025' }])[0].RAW_VALUE
+      ).toBe("'31DEC2025'd")
+      expect(
+        build([{ label: 'SOME_DATE', value: '01AUG1962' }])[0].RAW_VALUE
+      ).toBe("'01AUG1962'd")
+    })
+
+    it('still drops a month-name value that has no day', () => {
+      // A month and year alone is not a date - accepted shapes always carry a
+      // day, so this must not silently become the first of the month.
+      expect(build([{ label: 'SOME_DATE', value: 'January 2026' }])).toEqual([])
+    })
+
     it('resolves ambiguous slash dates day-first (never US), disambiguating by value', () => {
       const iso = build([{ label: 'SOME_DATE', value: '2025-03-31' }])[0]
         .RAW_VALUE
@@ -199,7 +237,694 @@ describe('VaFilterService', () => {
     })
   })
 
+  describe('buildClauses with an explicit va_parmap', () => {
+    const build = (parameters: any[], paramMap: Map<string, VaParmapTarget>) =>
+      service.buildClauses(msg(parameters), labelToVar(), cols, paramMap)
+
+    it('overrides the same-name match, including the value formatting', () => {
+      // The VA label IS a DC column here (SOME_CHAR), but the mapping points at
+      // a different one - the numeric column - so the value is formatted numeric.
+      const map = new Map([['some_char', { column: 'SOME_NUM' }]])
+      expect(build([{ label: 'SOME_CHAR', value: '10' }], map)).toEqual([
+        {
+          GROUP_LOGIC: 'AND',
+          SUBGROUP_LOGIC: 'AND',
+          SUBGROUP_ID: 0,
+          VARIABLE_NM: 'SOME_NUM',
+          OPERATOR_NM: '=',
+          RAW_VALUE: '10'
+        }
+      ])
+    })
+
+    it('lets two differently-named params filter the same column', () => {
+      // The multi-tab case: one DC instance per report tab, each with its own
+      // VA parameter, both mapping to the same DC column.
+      const map = new Map([
+        ['region_tab1', { column: 'SOME_CHAR' }],
+        ['region_tab2', { column: 'SOME_CHAR' }]
+      ])
+      const clauses = build(
+        [
+          { label: 'REGION_TAB1', value: 'north' },
+          { label: 'REGION_TAB2', value: 'south' }
+        ],
+        map
+      )
+      expect(clauses.map((c) => [c.VARIABLE_NM, c.RAW_VALUE])).toEqual([
+        ['SOME_CHAR', "'north'"],
+        ['SOME_CHAR', "'south'"]
+      ])
+    })
+
+    it('matches the parameter label only, not its internal name', () => {
+      // The internal name is the report's own bookkeeping. Matching it as well
+      // meant one parameter answered to two keys, so an entry written against the
+      // name and one against the label silently fought over it.
+      const map = new Map([['pr273', { column: 'SOME_NUM' }]])
+      expect(
+        build([{ name: 'pr273', label: 'Tab_1_Value', value: '7' }], map)
+      ).toEqual([])
+      // The label still matches, case-insensitively.
+      expect(
+        build(
+          [{ name: 'pr273', label: 'Tab_1_Value', value: 'x' }],
+          new Map([['tab_1_value', { column: 'SOME_CHAR' }]])
+        )[0].VARIABLE_NM
+      ).toBe('SOME_CHAR')
+    })
+
+    it('uses the column case from the loaded metadata', () => {
+      const map = new Map([['region_tab1', { column: 'some_char' }]])
+      expect(
+        build([{ label: 'REGION_TAB1', value: 'x' }], map)[0].VARIABLE_NM
+      ).toBe('SOME_CHAR')
+    })
+
+    it('leaves unmapped params on the same-name path', () => {
+      const map = new Map([['region_tab1', { column: 'SOME_NUM' }]])
+      const clauses = build(
+        [
+          { label: 'REGION_TAB1', value: '10' },
+          { label: 'SOME_CHAR', value: 'x' }
+        ],
+        map
+      )
+      expect(clauses.map((c) => [c.VARIABLE_NM, c.RAW_VALUE])).toEqual([
+        ['SOME_NUM', '10'],
+        ['SOME_CHAR', "'x'"]
+      ])
+    })
+
+    it('skips a mapping whose column is not in this table (no fallback)', () => {
+      // Falling back to the same-name match would silently apply the filter the
+      // mapping exists to override, so the param is dropped instead.
+      const map = new Map([['some_char', { column: 'NOT_IN_THIS_TABLE' }]])
+      expect(build([{ label: 'SOME_CHAR', value: 'x' }], map)).toEqual([])
+    })
+
+    it('ANDs two parameters that map to the same column - it does not union them', () => {
+      // What a report gets when one column's two values arrive in two
+      // parameters: every clause is AND-ed, so the result is the INTERSECTION.
+      // Two different dates therefore match no rows at all rather than both
+      // dates' rows - a single parameter carrying both values is the shape that
+      // ORs, via `IN (...)`.
+      const map = new Map([
+        ['param_a', { column: 'SOME_DATE' }],
+        ['param_b', { column: 'SOME_DATE' }]
+      ])
+      const one = build([{ label: 'PARAM_A', value: '01JUL1960' }], map)[0]
+      const other = build([{ label: 'PARAM_B', value: '01JAN1961' }], map)[0]
+      const clauses = build(
+        [
+          { label: 'PARAM_A', value: '01JUL1960' },
+          { label: 'PARAM_B', value: '01JAN1961' }
+        ],
+        map
+      )
+      expect(one.RAW_VALUE).not.toBe(other.RAW_VALUE)
+      expect(clauses).toHaveSize(2)
+      expect(clauses.map((c) => c.RAW_VALUE)).toEqual([
+        one.RAW_VALUE,
+        other.RAW_VALUE
+      ])
+      for (const clause of clauses) {
+        expect(clause.VARIABLE_NM).toBe('SOME_DATE')
+        expect(clause.OPERATOR_NM).toBe('=')
+        expect(clause.GROUP_LOGIC).toBe('AND')
+        expect(clause.SUBGROUP_LOGIC).toBe('AND')
+        expect(clause.SUBGROUP_ID).toBe(0)
+      }
+    })
+
+    it('applies a transform to the clause it builds', () => {
+      // ADD(QUARTER_START,-1) is the reported requirement: the selected row plus
+      // the row for the previous quarter, as one IN clause so the AND rule is
+      // untouched. 2026-10-01 = 24380, 2026-07-01 = 24288 in SAS days.
+      const map = new Map([
+        [
+          'selected',
+          {
+            column: 'SOME_DATE',
+            transform: {
+              op: 'ADD' as const,
+              anchor: 'QUARTER_START' as const,
+              offset: -1
+            }
+          }
+        ]
+      ])
+      const [clause] = build([{ label: 'SELECTED', value: '01OCT2026' }], map)
+      expect(clause.VARIABLE_NM).toBe('SOME_DATE')
+      expect(clause.OPERATOR_NM).toBe('IN')
+      expect(clause.RAW_VALUE).toBe('(24380,24288)')
+    })
+
+    it('applies a RANGE as BETWEEN', () => {
+      // RANGE(QUARTER,-1) for a mid-quarter value: 2026-04-01..2026-06-30
+      const map = new Map([
+        [
+          'selected',
+          {
+            column: 'SOME_DATE',
+            transform: {
+              op: 'RANGE' as const,
+              period: 'QUARTER' as const,
+              offset: -1
+            }
+          }
+        ]
+      ])
+      const [clause] = build([{ label: 'SELECTED', value: '1962-08-15' }], map)
+      expect(clause.OPERATOR_NM).toBe('BETWEEN')
+      expect(clause.RAW_VALUE).toContain(' AND ')
+      expect(clause.RAW_VALUE.split(' AND ')).toHaveSize(2)
+    })
+
+    it('keeps equality when the value already sits on the anchor', () => {
+      const map = new Map([
+        [
+          'selected',
+          {
+            column: 'SOME_DATE',
+            transform: {
+              op: 'ADD' as const,
+              anchor: 'QUARTER_START' as const,
+              offset: 0
+            }
+          }
+        ]
+      ])
+      const [clause] = build([{ label: 'SELECTED', value: '01OCT2026' }], map)
+      expect(clause.OPERATOR_NM).toBe('=')
+      expect(clause.RAW_VALUE).toBe('24380')
+    })
+
+    it('drops a transform on a column it cannot apply to', () => {
+      // A character column has no dates to shift; the parameter is skipped rather
+      // than filtered on the raw value the author did not see.
+      const map = new Map([
+        [
+          'selected',
+          {
+            column: 'SOME_CHAR',
+            transform: {
+              op: 'ADD' as const,
+              anchor: 'QUARTER_START' as const,
+              offset: -1
+            }
+          }
+        ]
+      ])
+      expect(build([{ label: 'SELECTED', value: 'x' }], map)).toEqual([])
+    })
+
+    it('adds nothing when the value is unparseable as a date', () => {
+      const map = new Map([
+        [
+          'selected',
+          {
+            column: 'SOME_DATE',
+            transform: {
+              op: 'ADD' as const,
+              anchor: 'QUARTER_START' as const,
+              offset: -1
+            }
+          }
+        ]
+      ])
+      expect(build([{ label: 'SELECTED', value: 'not a date' }], map)).toEqual(
+        []
+      )
+    })
+
+    it('round-trips a bare SAS day number unchanged', () => {
+      // VA may send the SAS internal value rather than a formatted string. The
+      // Date a transform works from is local, so the number must come back out of
+      // the clause identical - a UTC-built Date shifts a day west of UTC.
+      const map = new Map([
+        [
+          'selected',
+          {
+            column: 'SOME_DATE',
+            transform: {
+              op: 'ADD' as const,
+              anchor: 'QUARTER_START' as const,
+              offset: -1
+            }
+          }
+        ]
+      ])
+      const [clause] = build([{ label: 'SELECTED', value: 24380 }], map)
+      expect(clause.RAW_VALUE).toBe('(24380,24288)')
+    })
+
+    it('reports a range mapping fed more than one value', () => {
+      const map = new Map([
+        [
+          'selected',
+          {
+            column: 'SOME_DATE',
+            transform: {
+              op: 'RANGE' as const,
+              period: 'QUARTER' as const,
+              offset: -1
+            }
+          }
+        ]
+      ])
+      const problems = service.unsupportedTransforms(
+        msg([{ label: 'SELECTED', value: ['01APR2026', '01JUL2026'] }]),
+        map,
+        [{ NAME: 'SOME_DATE', DDTYPE: 'DATE', FMTNAME: 'DATE9' }]
+      )
+      expect(problems).toHaveSize(1)
+      expect(problems[0]).toContain('is a range')
+      expect(problems[0]).toContain('2 values')
+    })
+
+    it('refuses a range fed more than one value, rather than picking one', () => {
+      // Two values, one of which does not parse: building a BETWEEN from the
+      // survivor would silently filter one period when VA selected two.
+      const map = new Map([
+        [
+          'selected',
+          {
+            column: 'SOME_DATE',
+            transform: {
+              op: 'RANGE' as const,
+              period: 'QUARTER' as const,
+              offset: -1
+            }
+          }
+        ]
+      ])
+      expect(
+        build([{ label: 'SELECTED', value: ['01APR2026', 'not a date'] }], map)
+      ).toEqual([])
+    })
+
+    it('never lets a non-finite date reach the clause', () => {
+      // convertJsDateToSasDate returning NaN would otherwise put a literal NaN in
+      // the WHERE clause.
+      const map = new Map([
+        [
+          'selected',
+          {
+            column: 'SOME_DATE',
+            transform: {
+              op: 'ADD' as const,
+              anchor: 'QUARTER_START' as const,
+              offset: -1
+            }
+          }
+        ]
+      ])
+      spyOn(helperService, 'convertJsDateToSasDate').and.returnValue(NaN as any)
+      expect(build([{ label: 'SELECTED', value: '01OCT2026' }], map)).toEqual(
+        []
+      )
+    })
+
+    it('reports a partly unparseable value set instead of covering the rest', () => {
+      // One good value and one that is not a date: the clause still filters on
+      // the good one, so the author has to be told the other was dropped.
+      const map = new Map([
+        [
+          'selected',
+          {
+            column: 'SOME_DATE',
+            transform: {
+              op: 'ADD' as const,
+              anchor: 'QUARTER_START' as const,
+              offset: -1
+            }
+          }
+        ]
+      ])
+      const problems = service.unsupportedTransforms(
+        msg([{ label: 'SELECTED', value: ['01OCT2026', 'not a date'] }]),
+        map,
+        cols
+      )
+      expect(problems).toHaveSize(1)
+      expect(problems[0]).toContain('1 of its 2 values was not a date')
+    })
+
+    it('reports a range holding a value that does not parse', () => {
+      // Counted on what VA sent, not on what parsed: otherwise the second value
+      // vanishes and the range looks like a clean single-value case.
+      const map = new Map([
+        [
+          'selected',
+          {
+            column: 'SOME_DATE',
+            transform: {
+              op: 'RANGE' as const,
+              period: 'QUARTER' as const,
+              offset: -1
+            }
+          }
+        ]
+      ])
+      const problems = service.unsupportedTransforms(
+        msg([{ label: 'SELECTED', value: ['01APR2026', 'not a date'] }]),
+        map,
+        cols
+      )
+      expect(problems).toHaveSize(1)
+      expect(problems[0]).toContain('is a range')
+      expect(problems[0]).toContain('2 values')
+    })
+
+    it('applies a transform on the deferred initial load too', () => {
+      const map = new Map([
+        [
+          'selected',
+          {
+            column: 'SOME_DATE',
+            transform: {
+              op: 'ADD' as const,
+              anchor: 'QUARTER_START' as const,
+              offset: -1
+            }
+          }
+        ]
+      ])
+      const [clause] = service.buildInitialClauses(
+        msg([{ name: 'pr1', label: 'SELECTED', value: '01OCT2026' }]),
+        map
+      )
+      expect(clause.VARIABLE_NM).toBe('SOME_DATE')
+      expect(clause.RAW_VALUE).toBe('(24380,24288)')
+    })
+
+    it('behaves exactly as before with an empty map', () => {
+      expect(
+        service.buildClauses(
+          msg([{ label: 'SOME_CHAR', value: 'x' }]),
+          labelToVar(),
+          cols
+        )
+      ).toEqual(build([{ label: 'SOME_CHAR', value: 'x' }], new Map()))
+    })
+  })
+
+  describe('unresolvedMappings (what the report author is warned about)', () => {
+    const report = (
+      parameters: any[],
+      map: Map<string, VaParmapTarget>,
+      libds?: string
+    ) => service.unresolvedMappings(msg(parameters), map, cols, libds)
+
+    it('reports a mapping whose column is not in the loaded table', () => {
+      expect(
+        report(
+          [{ label: 'REGION_TAB1', value: 'x' }],
+          new Map([['region_tab1', { column: 'NOT_A_COLUMN' }]]),
+          'DC_JSLIB.MPE_X_TEST'
+        )
+      ).toEqual([
+        'VA parameter "REGION_TAB1" is mapped to NOT_A_COLUMN, which is not a column of DC_JSLIB.MPE_X_TEST - the filter was ignored.'
+      ])
+    })
+
+    it('says "this table" when the table name is not supplied', () => {
+      expect(
+        report(
+          [{ label: 'REGION_TAB1', value: 'x' }],
+          new Map([['region_tab1', { column: 'NOPE' }]])
+        )[0]
+      ).toContain('is not a column of this table')
+    })
+
+    it('reports nothing when the mapping resolves, or the param is unmapped', () => {
+      expect(
+        report(
+          [{ label: 'REGION_TAB1', value: 'x' }],
+          new Map([['region_tab1', { column: 'some_char' }]])
+        )
+      ).toEqual([])
+      expect(report([{ label: 'SOME_CHAR', value: 'x' }], new Map())).toEqual(
+        []
+      )
+      // A mapping whose parameter this message does not carry is not an error.
+      expect(
+        report([], new Map([['region_tab1', { column: 'NOPE' }]]))
+      ).toEqual([])
+    })
+
+    it('reports each mapping once, not once per message parameter', () => {
+      const problems = report(
+        [
+          { label: 'REGION_TAB1', value: '1' },
+          { name: 'pr99', label: 'REGION_TAB1', value: '2' }
+        ],
+        new Map([['region_tab1', { column: 'NOPE' }]])
+      )
+      expect(problems).toHaveSize(1)
+    })
+
+    it('treats an unloaded column list as "not known yet", not as a bad mapping', () => {
+      // While the grid is still loading there is no metadata to judge against, so
+      // reporting would flash the banner on every load.
+      expect(
+        service.unresolvedMappings(
+          msg([{ label: 'REGION_TAB1', value: 'x' }]),
+          new Map([['region_tab1', { column: 'NOPE' }]]),
+          [],
+          'DC_JSLIB.MPE_X_TEST'
+        )
+      ).toEqual([])
+      expect(
+        service.unresolvedMappings(
+          msg([{ label: 'REGION_TAB1', value: 'x' }]),
+          new Map([['region_tab1', { column: 'NOPE' }]]),
+          undefined as any,
+          'DC_JSLIB.MPE_X_TEST'
+        )
+      ).toEqual([])
+    })
+  })
+
+  describe('sameColumnMappings (two parameters, one column)', () => {
+    it('reports an AND-ed pair, naming both parameters and the column', () => {
+      const map = new Map([
+        ['dropdown_tab1', { column: 'SOME_DROPDOWN' }],
+        ['dropdown_tab2', { column: 'SOME_DROPDOWN' }]
+      ])
+      const problems = service.sameColumnMappings(
+        msg([
+          { label: 'DROPDOWN_TAB1', value: 'Option 1' },
+          { label: 'DROPDOWN_TAB2', value: 'Option 2' }
+        ]),
+        map
+      )
+      expect(problems).toHaveSize(1)
+      expect(problems[0]).toContain('"DROPDOWN_TAB1" and "DROPDOWN_TAB2"')
+      expect(problems[0]).toContain('all filter SOME_DROPDOWN')
+      expect(problems[0]).toContain('combined with AND')
+    })
+
+    it('says nothing for one parameter per column', () => {
+      const map = new Map([
+        ['dropdown_tab1', { column: 'SOME_DROPDOWN' }],
+        ['num_tab1', { column: 'SOME_NUM' }]
+      ])
+      expect(
+        service.sameColumnMappings(
+          msg([
+            { label: 'DROPDOWN_TAB1', value: 'Option 1' },
+            { label: 'NUM_TAB1', value: '1' }
+          ]),
+          map
+        )
+      ).toEqual([])
+      expect(
+        service.sameColumnMappings(
+          msg([{ label: 'DROPDOWN_TAB1', value: 'Option 1' }]),
+          map
+        )
+      ).toEqual([])
+      expect(service.sameColumnMappings(msg([]), map)).toEqual([])
+    })
+
+    it('says nothing when the AND-ed parameters carry the same value', () => {
+      // col = 'x' AND col = 'x' filters exactly as one clause does, so a banner
+      // claiming "different values will show no rows" would be wrong.
+      const map = new Map([
+        ['dropdown_tab1', { column: 'SOME_CHAR' }],
+        ['dropdown_tab2', { column: 'SOME_CHAR' }]
+      ])
+      expect(
+        service.sameColumnMappings(
+          msg([
+            { label: 'DROPDOWN_TAB1', value: 'x' },
+            { label: 'DROPDOWN_TAB2', value: 'x' }
+          ]),
+          map,
+          cols
+        )
+      ).toEqual([])
+      // One differing value and the intersection can be empty: reported.
+      const problems = service.sameColumnMappings(
+        msg([
+          { label: 'DROPDOWN_TAB1', value: 'x' },
+          { label: 'DROPDOWN_TAB2', value: 'y' }
+        ]),
+        map,
+        cols
+      )
+      expect(problems).toHaveSize(1)
+      expect(problems[0]).toContain('"DROPDOWN_TAB1" and "DROPDOWN_TAB2"')
+    })
+
+    it('leaves a column the table does not have to unresolvedMappings', () => {
+      // One cause, one line: the column is missing, so the AND story is moot.
+      const map = new Map([
+        ['a', { column: 'NOPE' }],
+        ['b', { column: 'NOPE' }]
+      ])
+      const cols = [
+        { LIBREF: 'DC_JSLIB', TABLE: 'MPE_X_TEST', VARIABLE: 'SOME_CHAR' }
+      ]
+      expect(
+        service.sameColumnMappings(
+          msg([
+            { label: 'A', value: '1' },
+            { label: 'B', value: '2' }
+          ]),
+          map,
+          cols
+        )
+      ).toEqual([])
+      // Without metadata the pair is still reported: the AND semantics hold
+      // whatever the table turns out to be.
+      expect(
+        service.sameColumnMappings(
+          msg([
+            { label: 'A', value: '1' },
+            { label: 'B', value: '2' }
+          ]),
+          map
+        )
+      ).toHaveSize(1)
+    })
+
+    it('reports one line per contended column', () => {
+      const map = new Map([
+        ['a', { column: 'SOME_DROPDOWN' }],
+        ['b', { column: 'SOME_DROPDOWN' }],
+        ['c', { column: 'SOME_NUM' }],
+        ['d', { column: 'SOME_NUM' }]
+      ])
+      const problems = service.sameColumnMappings(
+        msg([
+          { label: 'A', value: '1' },
+          { label: 'B', value: '2' },
+          { label: 'C', value: '3' },
+          { label: 'D', value: '4' }
+        ]),
+        map
+      )
+      expect(problems).toHaveSize(2)
+    })
+  })
+
+  describe('unmatchedMappings (parameter-side typo)', () => {
+    const seen = (...keys: string[]) => new Set(keys)
+
+    it('reports a key that no message has named', () => {
+      expect(
+        service.unmatchedMappings(
+          new Map([['dropdown_tab1', { column: 'SOME_CHAR' }]]),
+          seen('dropdown_tab_1', 'pr1'),
+          true
+        )
+      ).toEqual([
+        'This object\'s va_parmap names VA parameter "dropdown_tab1", which VA is not sending - check the spelling.'
+      ])
+    })
+
+    it('says nothing until a parameter has actually been seen', () => {
+      // Without evidence that this object receives parameters at all, an absent
+      // key proves nothing - VA sends only the parameters its query uses.
+      expect(
+        service.unmatchedMappings(
+          new Map([['dropdown_tab1', { column: 'SOME_CHAR' }]]),
+          seen()
+        )
+      ).toEqual([])
+      expect(
+        service.unmatchedMappings(
+          new Map([['dropdown_tab1', { column: 'SOME_CHAR' }]]),
+          undefined
+        )
+      ).toEqual([])
+    })
+
+    it('says nothing when every key has been seen', () => {
+      expect(
+        service.unmatchedMappings(
+          new Map([['dropdown_tab1', { column: 'SOME_CHAR' }]]),
+          seen('dropdown_tab1'),
+          true
+        )
+      ).toEqual([])
+      expect(
+        service.unmatchedMappings(new Map(), seen('anything'), true)
+      ).toEqual([])
+    })
+
+    it('says nothing while the parameter set is still growing', () => {
+      // A message that introduces a new parameter name means this object's set
+      // has not settled, so an absent key may arrive in the next message.
+      expect(
+        service.unmatchedMappings(
+          new Map([['dropdown_tab1', { column: 'SOME_CHAR' }]]),
+          seen('pr1', 'pr2'),
+          false
+        )
+      ).toEqual([])
+    })
+  })
+
   describe('buildInitialClauses (no metadata, infer kind from value)', () => {
+    it('uses an explicit mapping instead of the label', () => {
+      // The deferred first load has no column metadata, so a mapped column is
+      // used as given and the SAS kind still comes from the value shape.
+      const map = new Map([
+        ['region_tab1', { column: 'REGION' }],
+        ['amt_tab1', { column: 'AMOUNT' }]
+      ])
+      const clauses = service.buildInitialClauses(
+        msg([
+          { name: 'pr1', label: 'REGION_TAB1', dataType: 'string', value: 'n' },
+          { name: 'pr2', label: 'AMT_TAB1', dataType: 'number', value: '10' }
+        ]),
+        map
+      )
+      expect(clauses.map((c) => [c.VARIABLE_NM, c.RAW_VALUE])).toEqual([
+        ['REGION', "'n'"],
+        ['AMOUNT', '10']
+      ])
+    })
+
+    it('falls back to the label for params the map does not name', () => {
+      const map = new Map([['region_tab2', { column: 'REGION' }]])
+      const clauses = service.buildInitialClauses(
+        msg([
+          { name: 'pr1', label: 'REGION_TAB1', value: 'n' },
+          { name: 'pr2', label: 'REGION_TAB2', value: 's' }
+        ]),
+        map
+      )
+      expect(clauses.map((c) => [c.VARIABLE_NM, c.RAW_VALUE])).toEqual([
+        ['REGION_TAB1', "'n'"],
+        ['REGION', "'s'"]
+      ])
+    })
+
     it('uses the VA label as the column and infers kinds by value shape', () => {
       const clauses = service.buildInitialClauses(
         msg([

client/src/app/services/va-filter.service.ts

@@ -2,6 +2,12 @@ import { Injectable } from '@angular/core'
 import { HelperService } from './helper.service'
 import { VaMessage } from './va-messaging.service'
 import { QueryClause } from '../models/TableData'
+import { VaParmapTarget } from '../shared/utils/parse-va-parmap'
+import {
+  applyVaTransform,
+  isRangeTransform,
+  VaTransform
+} from '../shared/utils/va-parmap-transform'
 
 /** SAS data-type kind used to format a VA filter value into a DC RAW_VALUE. */
 type VaColumnKind = 'char' | 'numeric' | 'time' | 'date' | 'datetime'
@@ -54,11 +60,19 @@ export class VaFilterService {
    * Translates VA `parameters` into DC filter clauses. Each parameter is matched
    * to a DC column by label; a single value becomes `col = value`, multiple
    * values become `col IN (...)`. Values are formatted for the column's SAS kind.
+   *
+   * `paramMap` (from the URL `va_parmap` parameter, see parseVaParmap) overrides
+   * that same-name resolution for the parameters it names, so two DC instances
+   * over one table can filter the same column from differently-named VA
+   * parameters. A mapped column must exist in `cols` - if it does not the
+   * parameter is skipped rather than falling back to the same-name match, which
+   * is exactly the behaviour the mapping exists to override.
    */
   buildClauses(
     msg: VaMessage,
     labelToVar: Map<string, string>,
-    cols: any[]
+    cols: any[],
+    paramMap?: Map<string, VaParmapTarget>
   ): QueryClause[] {
     const clauses: QueryClause[] = []
     for (const param of msg.parameters || []) {
@@ -67,60 +81,194 @@ export class VaFilterService {
         .toString()
         .trim()
         .toLowerCase()
-      // A VA param whose label isn't a DC column (beyond case, which is handled)
+      // An explicit URL mapping wins over the label->column lookup. The mapped
-      // simply doesn't map — skip it.
+      // column has to exist in THIS table, else the entry does not apply here
-      const varName = labelToVar.get(key)
+      // (and the same-name fallback would be the behaviour it exists to
+      // override).
+      const mapped = this.mappedColumn(param, paramMap)
+      const mappedCol = mapped ? this.findCol(mapped, cols) : undefined
+      if (mapped && !mappedCol) continue
+      // The table's own case, so the clause matches what the metadata build
+      // produces for a same-name match.
+      const varName: string = mappedCol
+        ? mappedCol.NAME
+        : (labelToVar.get(key) ?? '')
       if (!varName) continue
 
-      const col = this.findCol(varName, cols)
+      const col = mappedCol ?? this.findCol(varName, cols)
       const kind = this.columnKind(col)
       const order = this.dateOrderFromFmt(col?.FMTNAME || col?.FORMAT)
-      const rawValues = Array.isArray(param.value) ? param.value : [param.value]
+      const built = this.buildClauseValueSet(
-      // Format/validate every value for the column's SAS type. Numeric columns
+        param,
-      // must receive a finite number (untrusted postMessage input is
+        kind,
-      // interpolated into a SAS WHERE clause, so a non-numeric value here would
+        order,
-      // be an injection vector); temporal -> SAS internal number; char ->
+        this.mappedTarget(param, paramMap)?.transform
-      // escaped. Invalid values are dropped.
+      )
-      const formatted = rawValues
+      if (!built) continue
-        .filter((v) => v !== undefined && v !== null && v !== '')
+      clauses.push({
-        .map((v) => this.formatValueByKind(v, kind, order))
+        GROUP_LOGIC: 'AND',
-        .filter((v): v is string => v !== null)
+        SUBGROUP_LOGIC: 'AND',
-      if (formatted.length === 0) continue
+        SUBGROUP_ID: 0,
-      clauses.push(this.clause(varName, formatted))
+        VARIABLE_NM: varName,
+        OPERATOR_NM: built.operator,
+        RAW_VALUE: built.rawValue
+      })
     }
     return clauses
   }
 
+  /**
+   * The operator and raw value for one parameter, after any transform.
+   *
+   * Values are formatted for the column's SAS kind: numeric columns must receive
+   * a finite number (untrusted postMessage input is interpolated into a WHERE
+   * clause, so anything else would be an injection vector); temporal values
+   * become SAS internal numbers; char values are escaped. Invalid values are
+   * dropped, and a transform that cannot be applied to the column's kind yields
+   * nothing at all - the caller reports that separately rather than filtering on
+   * a value the author did not choose.
+   */
+  private buildClauseValueSet(
+    param: any,
+    kind: VaColumnKind,
+    order: 'YMD' | 'DMY' | 'MDY' | undefined,
+    transform?: VaTransform
+  ): { operator: '=' | 'IN' | 'BETWEEN'; rawValue: string } | null {
+    const rawValues = (
+      Array.isArray(param.value) ? param.value : [param.value]
+    ).filter((v: any) => v !== undefined && v !== null && v !== '')
+    if (rawValues.length === 0) return null
+
+    if (!transform) {
+      const formatted = rawValues
+        .map((v: any) => this.formatValueByKind(v, kind, order))
+        .filter((v: string | null): v is string => v !== null)
+      if (formatted.length === 0) return null
+      return formatted.length === 1
+        ? { operator: '=', rawValue: formatted[0] }
+        : { operator: 'IN', rawValue: `(${formatted.join(',')})` }
+    }
+
+    // Transforms resolve to dates, so only temporal columns can take one.
+    if (kind !== 'date' && kind !== 'datetime') return null
+    const unit = kind === 'datetime' ? 'seconds' : 'days'
+    const values = new Set<string>()
+    for (const raw of rawValues) {
+      const resolved = this.valueAsDate(raw, kind, order)
+      if (!resolved) continue
+      for (const date of applyVaTransform(
+        resolved,
+        transform,
+        kind === 'datetime'
+      )) {
+        // A non-finite result (an Invalid Date reaching the helper) must never be
+        // interpolated into the clause: `NaN` is not a SAS value. Dropped here,
+        // and reported by unsupportedTransforms so the parameter is not silently
+        // filtered on nothing.
+        const sasDate = this.helperService.convertJsDateToSasDate(date, unit)
+        if (!Number.isFinite(Number(sasDate))) continue
+        values.add(`${sasDate}`)
+      }
+    }
+    const formatted = Array.from(values)
+    if (formatted.length === 0) return null
+    if (isRangeTransform(transform)) {
+      // One BETWEEN carries one period, so more than one value has no clause -
+      // dropping it here (rather than building a BETWEEN from whichever value
+      // parsed) is what unsupportedTransforms' count warning then describes.
+      if (rawValues.length > 1) return null
+      if (formatted.length !== 2) return null
+      return { operator: 'BETWEEN', rawValue: formatted.join(' AND ') }
+    }
+    return formatted.length === 1
+      ? { operator: '=', rawValue: formatted[0] }
+      : { operator: 'IN', rawValue: `(${formatted.join(',')})` }
+  }
+
+  /**
+   * A VA value as a local Date, for a transform to work from. Formatted strings
+   * are parsed by shape; a bare number is already the SAS internal value, so it
+   * is converted back from days (date) or seconds (datetime) since 1960-01-01.
+   *
+   * Built with local calendar construction, matching `parseDate` and
+   * `HelperService.convertJsDateToSasDate`: a UTC instant read back through local
+   * accessors lands on the previous day west of UTC, and the round trip to a SAS
+   * day number would be one out.
+   */
+  private valueAsDate(
+    value: any,
+    kind: VaColumnKind,
+    order?: 'YMD' | 'DMY' | 'MDY'
+  ): Date | null {
+    const s = String(value).trim()
+    if (s === '') return null
+    if (this.isBareNumber(s)) {
+      const internal = Number(s)
+      if (kind !== 'datetime') return new Date(1960, 0, 1 + internal)
+      // Seconds since midnight 1960-01-01, split into whole days and a remainder
+      // so the constructor normalises the clock time locally.
+      const days = Math.floor(internal / 86400)
+      const seconds = internal - days * 86400
+      return new Date(1960, 0, 1 + days, 0, 0, seconds)
+    }
+    return kind === 'datetime'
+      ? this.parseDatetime(s, order)
+      : this.parseDate(s, order)
+  }
+
   /**
    * Pre-getdata variant used on the deferred initial load, when DC column
    * metadata isn't available. Uses the VA `label` as the column name and infers
    * the SAS kind from the value shape, so the clause matches what the metadata
    * build later produces (no reconcile reload).
+   *
+   * `paramMap` overrides the label-as-column rule for the parameters it names,
+   * so the very first (already filtered) load also honours an explicit mapping.
    */
-  buildInitialClauses(msg: VaMessage): QueryClause[] {
+  buildInitialClauses(
+    msg: VaMessage,
+    paramMap?: Map<string, VaParmapTarget>
+  ): QueryClause[] {
     const clauses: QueryClause[] = []
     for (const param of msg.parameters || []) {
       if (!param || param.value === undefined || param.value === null) continue
-      // The DC column name is the VA `label`; `name` is VA's internal id (pr###).
+      const target = this.mappedTarget(param, paramMap)
-      const varName = (param.label ?? param.name ?? '').toString().trim()
+      // Without a mapping the DC column name is the VA `label`; `name` is VA's
+      // internal id (pr###). A `va_parmap` entry wins over both.
+      const varName = (target?.column ?? param.label ?? param.name ?? '')
+        .toString()
+        .trim()
       if (!varName) continue
 
-      const rawValues = Array.isArray(param.value) ? param.value : [param.value]
+      const rawValues = (
+        Array.isArray(param.value) ? param.value : [param.value]
+      ) as any[]
       const nonEmpty = rawValues.filter(
         (v) => v !== undefined && v !== null && v !== ''
       )
       if (nonEmpty.length === 0) continue
 
-      const formatted = nonEmpty
+      // A transform only ever produces dates, so it fixes the kind: inferring it
-        .map((v) =>
+      // from the value would see a formatted date either way, but with the
-          this.formatValueByKind(
+      // transform the column is known to be temporal.
-            v,
+      const kind = target?.transform
-            this.inferKindFromValue(String(v).trim(), param.dataType)
+        ? ('date' as VaColumnKind)
-          )
+        : this.inferKindFromValue(String(nonEmpty[0]).trim(), param.dataType)
-        )
+      const built = this.buildClauseValueSet(
-        .filter((v): v is string => v !== null)
+        param,
-      if (formatted.length === 0) continue
+        kind,
-      clauses.push(this.clause(varName, formatted))
+        undefined,
+        target?.transform
+      )
+      if (!built) continue
+      clauses.push({
+        GROUP_LOGIC: 'AND',
+        SUBGROUP_LOGIC: 'AND',
+        SUBGROUP_ID: 0,
+        VARIABLE_NM: varName,
+        OPERATOR_NM: built.operator,
+        RAW_VALUE: built.rawValue
+      })
     }
     return clauses
   }
@@ -141,16 +289,254 @@ export class VaFilterService {
     )
   }
 
-  private clause(varName: string, formatted: string[]): QueryClause {
+  /**
-    return {
+   * Transform mappings in this message that cannot be applied, one plain-English
-      GROUP_LOGIC: 'AND',
+   * line each: the column is not temporal, or the value is not a date the browser
-      SUBGROUP_LOGIC: 'AND',
+   * can parse. `buildClauseValueSet` skips those parameters, so this is what stops
-      SUBGROUP_ID: 0,
+   * the misconfiguration reading as a filter that simply matched nothing.
-      VARIABLE_NM: varName,
+   *
-      OPERATOR_NM: formatted.length === 1 ? '=' : 'IN',
+   * A missing column is left to `unresolvedMappings`, which already reports it.
-      RAW_VALUE:
+   */
-        formatted.length === 1 ? formatted[0] : `(${formatted.join(',')})`
+  unsupportedTransforms(
+    msg: VaMessage,
+    paramMap: Map<string, VaParmapTarget> | undefined,
+    cols: any[]
+  ): string[] {
+    if (!paramMap || paramMap.size === 0) return []
+    if (!cols || cols.length === 0) return []
+    const problems = new Set<string>()
+    for (const param of msg.parameters || []) {
+      const target = this.mappedTarget(param, paramMap)
+      if (!target?.transform) continue
+      const transform = target.transform
+      const col = this.findCol(target.column, cols)
+      if (!col) continue
+      const kind = this.columnKind(col)
+      const paramName = (param.label ?? param.name ?? '').toString().trim()
+      const spec =
+        target.transform.op === 'ADD'
+          ? `ADD(${target.transform.anchor},${target.transform.offset})`
+          : `RANGE(${target.transform.period},${target.transform.offset})`
+      if (kind !== 'date' && kind !== 'datetime') {
+        problems.add(
+          `VA parameter "${paramName}" maps to ${target.column}:${spec}, but ${target.column} is not a date or datetime column - the filter was ignored.`
+        )
+        continue
+      }
+      const order = this.dateOrderFromFmt(col?.FMTNAME || col?.FORMAT)
+      const sent = (
+        Array.isArray(param.value) ? param.value : [param.value]
+      ).filter((v: any) => v !== undefined && v !== null && v !== '')
+      const usable = sent.filter((v: any) => {
+        const date = this.valueAsDate(v, kind, order)
+        if (!date) return false
+        // Same test the clause builder applies, so this count matches what it uses.
+        return applyVaTransform(date, transform, kind === 'datetime').every(
+          (resolved) => {
+            const sasDate = this.helperService.convertJsDateToSasDate(
+              resolved,
+              kind === 'datetime' ? 'seconds' : 'days'
+            )
+            return Number.isFinite(Number(sasDate))
+          }
+        )
+      })
+      if (usable.length === 0) {
+        problems.add(
+          `VA parameter "${paramName}" maps to ${target.column}:${spec}, but its value was not a date - the filter was ignored.`
+        )
+        continue
+      }
+      // One BETWEEN can only express one period. A multi-select would have to
+      // become an OR of ranges, which the clause model does not carry, so say so
+      // rather than building a BETWEEN from whichever value happened to parse.
+      // Counted on what VA sent: a second value that does not parse must not make
+      // the single-value case look clean.
+      if (isRangeTransform(target.transform) && sent.length > 1) {
+        problems.add(
+          `VA parameter "${paramName}" maps to ${target.column}:${spec}, which is a range, but the parameter has ${sent.length} values - a range needs one value, so the filter was ignored.`
+        )
+        continue
+      }
+      if (usable.length < sent.length) {
+        problems.add(
+          `VA parameter "${paramName}" maps to ${target.column}:${spec}, but ${sent.length - usable.length} of its ${sent.length} values was not a date - the filter covers only the rest.`
+        )
+      }
     }
+    return Array.from(problems)
+  }
+
+  /**
+   * `va_parmap` entries that this message actually uses but that name a column
+   * the loaded table does not have, one plain-English line each. `buildClauses`
+   * skips those parameters (deliberately: falling back to the same-name match
+   * would silently apply the filter the mapping exists to override), so this is
+   * what makes the misconfiguration visible to the report author - otherwise the
+   * grid simply shows every row, which looks like a working filter.
+   */
+  unresolvedMappings(
+    msg: VaMessage,
+    paramMap: Map<string, VaParmapTarget> | undefined,
+    cols: any[],
+    libds?: string
+  ): string[] {
+    const problems = new Set<string>()
+    if (!paramMap || paramMap.size === 0) return []
+    // No metadata yet is not the same as "the table lacks this column": an empty
+    // `cols` means the grid is still loading, so there is nothing to judge yet.
+    // Reporting here would flash a misconfiguration banner during every load.
+    if (!cols || cols.length === 0) return []
+    for (const param of msg.parameters || []) {
+      const mapped = this.mappedColumn(param, paramMap)
+      if (!mapped || this.findCol(mapped, cols)) continue
+      const paramName = (param.label ?? param.name ?? '').toString().trim()
+      problems.add(
+        `VA parameter "${paramName}" is mapped to ${mapped}, which is not a column of ${
+          libds || 'this table'
+        } - the filter was ignored.`
+      )
+    }
+    return Array.from(problems)
+  }
+
+  /**
+   * `va_parmap` keys that no VA message has named, one plain-English line each.
+   * A key that matches nothing is a parameter-side typo: the mapping never
+   * applies and the grid stays unfiltered, which looks the same as a working
+   * filter.
+   *
+   * `seenKeys` must be every key observed across messages so far. `paramsStable`
+   * says the latest message introduced no new parameter name, i.e. this object's
+   * parameter set has settled - until then a missing name may simply not have
+   * arrived, because VA sends only the parameters its query uses.
+   */
+  unmatchedMappings(
+    paramMap: Map<string, VaParmapTarget> | undefined,
+    seenKeys: Set<string> | undefined,
+    paramsStable = false
+  ): string[] {
+    if (!paramMap || paramMap.size === 0) return []
+    if (!seenKeys || seenKeys.size === 0) return []
+    if (!paramsStable) return []
+    const problems: string[] = []
+    for (const key of Array.from(paramMap.keys())) {
+      if (!seenKeys.has(key)) {
+        problems.push(
+          `This object's va_parmap names VA parameter "${key}", which VA is not sending - check the spelling.`
+        )
+      }
+    }
+    return problems
+  }
+
+  /**
+   * Parameters in this message that filter the same column with *different*
+   * values, one plain-English line per column. Clauses are AND-ed, so two
+   * parameters on one column are an intersection: with different values the grid
+   * comes back empty, which looks like a working filter. Two parameters carrying
+   * the same value intersect cleanly and filter correctly, so they are not
+   * reported - a banner there would call a working configuration broken.
+   *
+   * Reported rather than rewritten - folding them into one `IN (...)` would
+   * silently turn an AND into an OR, which is a semantics decision, not a
+   * diagnostic.
+   */
+  sameColumnMappings(
+    msg: VaMessage,
+    paramMap: Map<string, VaParmapTarget> | undefined,
+    cols?: any[]
+  ): string[] {
+    if (!paramMap || paramMap.size === 0) return []
+    // column -> the parameters on it, each with the clause value it produces
+    const byColumn = new Map<string, { name: string; value: string }[]>()
+    for (const param of msg.parameters || []) {
+      const mapped = this.mappedColumn(param, paramMap)
+      if (!mapped) continue
+      // A column the table lacks is unresolvedMappings' finding; counting it here
+      // would report one cause as two symptoms.
+      const col =
+        cols && cols.length > 0 ? this.findCol(mapped, cols) : undefined
+      if (cols && cols.length > 0 && !col) continue
+      // A parameter that produces no clause cannot conflict with anything.
+      const value = this.valueSignature(
+        param,
+        col,
+        this.mappedTarget(param, paramMap)?.transform
+      )
+      if (value === null) continue
+      const column = mapped.toUpperCase()
+      const params = byColumn.get(column) || []
+      const paramName = (param.label ?? '').toString().trim()
+      if (paramName && !params.some((p) => p.name === paramName)) {
+        params.push({ name: paramName, value })
+      }
+      byColumn.set(column, params)
+    }
+    const problems: string[] = []
+    for (const [column, params] of Array.from(byColumn.entries())) {
+      if (params.length < 2) continue
+      // Same value on every parameter: the AND is a no-op, not an empty grid.
+      const distinct = new Set(params.map((p) => p.value))
+      if (distinct.size === 1) continue
+      problems.push(
+        `VA parameters ${params
+          .map((p) => `"${p.name}"`)
+          .join(
+            ' and '
+          )} all filter ${column}. Parameters are combined with AND, so rows must match all of them - different values will show no rows. Use one parameter carrying both values to match either.`
+      )
+    }
+    return problems
+  }
+
+  /**
+   * The DC column an explicit URL mapping (`va_parmap`) assigns to this
+   * parameter, or undefined when the parameter is not mapped. The key is matched
+   * case-insensitively against the parameter's `label`.
+   */
+  private mappedTarget(
+    param: any,
+    paramMap?: Map<string, VaParmapTarget>
+  ): VaParmapTarget | undefined {
+    if (!paramMap || paramMap.size === 0) return undefined
+    // The label only. The internal name (pr273) is an implementation detail of
+    // the report, and matching both made one parameter answer to two keys: a
+    // mapping written against the name could never be distinguished from a typo
+    // once the label was also mapped, and the second entry silently lost.
+    const key = (param.label ?? '').toString().trim().toLowerCase()
+    if (!key) return undefined
+    return paramMap.get(key)
+  }
+
+  /**
+   * The clause value a parameter would filter its column with, as a comparable
+   * string, or null when it produces no clause at all. Used to tell an AND of two
+   * identical values (harmless) from an AND of two different ones (empty grid).
+   *
+   * Delegates to the clause builder so a transform's derived dates take part in
+   * the comparison: two parameters adding the same quarter are the same clause.
+   */
+  private valueSignature(
+    param: any,
+    col: any,
+    transform?: VaTransform
+  ): string | null {
+    const built = this.buildClauseValueSet(
+      param,
+      this.columnKind(col),
+      this.dateOrderFromFmt(col?.FMTNAME || col?.FORMAT),
+      transform
+    )
+    return built ? built.rawValue : null
+  }
+
+  /** The mapped column name, for callers that do not need the transform. */
+  private mappedColumn(
+    param: any,
+    paramMap?: Map<string, VaParmapTarget>
+  ): string | undefined {
+    return this.mappedTarget(param, paramMap)?.column
   }
 
   /** The loaded column spec whose NAME matches varName (case-insensitive). */
@@ -321,14 +707,42 @@ export class VaFilterService {
     return this.parseMonthNameDate(value) ?? this.parseNumericDate(value, order)
   }
 
-  /** DATE9 "ddMMM[yy]yy" (e.g. 01AUG1962, 01AUG62) -> local Date, or null. */
+  /**
+   * Month-name date shapes -> local Date, or null. Three families:
+   *  - DATE9 "ddMMM[yy]yy" (e.g. 01AUG1962, 01AUG62);
+   *  - word date with the month first, comma optional: "January 1, 2026",
+   *    "Jan 1 2026", "Jan. 1, 2026";
+   *  - word date with the day first: "1 January 2026", "1-Jan-2026".
+   *
+   * A date parameter's display format decides which of these the report author
+   * sees, and VA sends the formatted text, so a report using WORDDATE (or a
+   * long month name in any order) has to parse here rather than being dropped.
+   * The month name may be abbreviated or spelled out; only its first three
+   * letters are matched.
+   */
   private parseMonthNameDate(value: string): Date | null {
-    const m = /^(\d{1,2})([A-Za-z]{3})(\d{2}|\d{4})$/.exec(value)
+    const monthByName = (name: string): number =>
-    if (!m) return null
+      VaFilterService.SAS_MONTHS.indexOf(
-    const [, day, monthAbbr, year] = m
+        name.replace(/\./g, '').slice(0, 3).toUpperCase()
-    const monthIndex = VaFilterService.SAS_MONTHS.indexOf(
+      )
-      monthAbbr.toUpperCase()
+
-    )
+    const date9 = /^(\d{1,2})([A-Za-z]{3})(\d{2}|\d{4})$/.exec(value)
+    if (date9) {
+      const [, day, monthAbbr, year] = date9
+      const monthIndex = monthByName(monthAbbr)
+      if (monthIndex < 0) return null
+      return this.buildDate(this.fullYear(year), monthIndex + 1, Number(day))
+    }
+
+    const monthFirst =
+      /^([A-Za-z]{3,9})\.?[\s-]+(\d{1,2}),?[\s-]+(\d{2}|\d{4})$/.exec(value)
+    const dayFirst =
+      /^(\d{1,2})[\s-]+([A-Za-z]{3,9})\.?,?[\s-]+(\d{2}|\d{4})$/.exec(value)
+    const name = monthFirst ? monthFirst[1] : dayFirst ? dayFirst[2] : null
+    if (!name) return null
+    const day = monthFirst ? monthFirst[2] : dayFirst![1]
+    const year = monthFirst ? monthFirst[3] : dayFirst![3]
+    const monthIndex = monthByName(name)
     if (monthIndex < 0) return null
     return this.buildDate(this.fullYear(year), monthIndex + 1, Number(day))
   }

client/src/app/services/va-messaging.service.spec.ts

@@ -1,4 +1,6 @@
 import { VaMessagingService, VaMessage } from './va-messaging.service'
+import { VaFilterService } from './va-filter.service'
+import { HelperService } from './helper.service'
 
 // Sample message taken from the official SAS DDC docs (brush column included).
 const SAMPLE = {
@@ -51,6 +53,57 @@ describe('VaMessagingService', () => {
     })
   })
 
+  describe('parameter keys (what the va_parmap typo diagnostic is judged on)', () => {
+    const PARAMS = [
+      { name: 'pr273', label: 'Tab_1_Date', value: '2025-03-31' },
+      { name: 'pr274', label: 'Tab 2 Region', value: 'north' }
+    ]
+    const withParams = () => ({ ...SAMPLE, parameters: PARAMS })
+
+    it('records each parameter label once, across messages', () => {
+      service.parseMessage({ data: withParams() } as MessageEvent)
+      service.parseMessage({ data: withParams() } as MessageEvent)
+
+      expect(Array.from(service.seenParamKeys)).toEqual([
+        'tab_1_date',
+        'tab 2 region'
+      ])
+      // The second message introduced no new label, so the set has settled.
+      expect(service.lastMessageParamsStable).toBeTrue()
+    })
+
+    it('records the label only, so a mapping written against the internal name is reported', () => {
+      // `pr273` is VA's own bookkeeping. A va_parmap entry written against it
+      // never matches a parameter, and recording the name here would make the
+      // typo diagnostic believe VA is sending it - leaving the author with an
+      // unfiltered grid and no warning at all.
+      service.parseMessage({ data: withParams() } as MessageEvent)
+      service.parseMessage({ data: withParams() } as MessageEvent)
+
+      const filter = new VaFilterService(new HelperService({} as any))
+      const seen = service.seenParamKeys
+      const stable = service.lastMessageParamsStable
+
+      expect(
+        filter.unmatchedMappings(
+          new Map([['pr273', { column: 'D_DATE' }]]),
+          seen,
+          stable
+        )
+      ).toEqual([
+        'This object\'s va_parmap names VA parameter "pr273", which VA is not sending - check the spelling.'
+      ])
+      // The label is a real key, so the same map written that way is silent.
+      expect(
+        filter.unmatchedMappings(
+          new Map([['tab_1_date', { column: 'D_DATE' }]]),
+          seen,
+          stable
+        )
+      ).toEqual([])
+    })
+  })
+
   describe('dataColumns', () => {
     it('excludes brush columns and keeps original indices', () => {
       const msg = service.parseMessage({ data: SAMPLE } as MessageEvent)!

client/src/app/services/va-messaging.service.ts

@@ -82,6 +82,26 @@ export class VaMessagingService {
   /** DC variable names that should be visible (the rest are hidden). */
   visibleColumns: string[] | null = null
 
+  /**
+   * Lower-cased label of every parameter any VA message has carried so far, kept
+   * across editor reloads. Lets the editor tell a `va_parmap` key that names no
+   * parameter in the report (a typo, so the filter would silently never apply)
+   * from one that is merely absent from the current message - VA only sends the
+   * parameters its query uses.
+   *
+   * Labels only: that is the key a mapping is matched on, and recording the
+   * internal name as well would make a name-keyed entry - which never applies -
+   * look like a parameter VA is sending.
+   */
+  seenParamKeys = new Set<string>()
+  /**
+   * True when every parameter name in the latest message had already been seen.
+   * VA sends only the parameters a report object's query uses, and sends them per
+   * object, so a name that is absent from the first messages may simply not have
+   * arrived yet. A stable parameter set is what makes absence meaningful.
+   */
+  lastMessageParamsStable = false
+
   /** True once the pre-bootstrap early buffer has been drained (drain-once). */
   private earlyDrained = false
 
@@ -187,6 +207,22 @@ export class VaMessagingService {
     if (typeof data.resultName !== 'string') return null
     if (!Array.isArray(data.columns) || !Array.isArray(data.data)) return null
 
+    const parameters = Array.isArray(data.parameters) ? data.parameters : []
+    const keys: string[] = []
+    for (const param of parameters) {
+      // The label only, which is the key a `va_parmap` entry is matched on. The
+      // internal name (`pr273`) is the report's own bookkeeping: recording it
+      // here would let the typo diagnostic below believe a name-keyed entry is
+      // being sent when no code path can ever apply it.
+      const key = (param?.label ?? '').toString().trim().toLowerCase()
+      if (key && !keys.includes(key)) keys.push(key)
+    }
+    // Judged before the keys are recorded: a message that introduces a new name
+    // means the set is still growing, so a missing name proves nothing yet.
+    this.lastMessageParamsStable =
+      keys.length > 0 && keys.every((key) => this.seenParamKeys.has(key))
+    for (const key of keys) this.seenParamKeys.add(key)
+
     return {
       version: data.version,
       resultName: data.resultName,
@@ -194,7 +230,7 @@ export class VaMessagingService {
       availableRowCount: data.availableRowCount,
       data: data.data,
       columns: data.columns,
-      parameters: Array.isArray(data.parameters) ? data.parameters : []
+      parameters
     }
   }
 

client/src/app/shared/utils/parse-va-parmap.spec.ts

@@ -0,0 +1,215 @@
+import { parseVaParmap } from './parse-va-parmap'
+
+describe('parseVaParmap', () => {
+  it('returns an empty map and no problems when the parameter is absent', () => {
+    for (const query of ['', 'embed=va', 'embed=va&labels=true']) {
+      const parsed = parseVaParmap(query)
+      expect(parsed.map.size).toBe(0)
+      expect(parsed.problems).toEqual([])
+    }
+  })
+
+  it('maps a VA parameter label to a DC column', () => {
+    const { map, problems } = parseVaParmap(
+      'embed=va&va_parmap=REGION_TAB1-REGION'
+    )
+    expect(map.get('region_tab1')?.column).toBe('REGION')
+    expect(map.size).toBe(1)
+    expect(problems).toEqual([])
+  })
+
+  it('collects repeated entries, so two parameters can map to one column', () => {
+    // The multi-tab case: each DC instance filters the same DC column from a
+    // differently-named VA parameter.
+    const { map } = parseVaParmap(
+      'embed=va&va_parmap=REGION_TAB1-REGION&va_parmap=REGION_TAB2-REGION'
+    )
+    expect(map.get('region_tab1')?.column).toBe('REGION')
+    expect(map.get('region_tab2')?.column).toBe('REGION')
+    expect(map.size).toBe(2)
+  })
+
+  it('is case-insensitive on the key and trims both sides', () => {
+    const { map } = parseVaParmap('va_parmap=%20Region_Tab1%20-%20region%20')
+    expect(map.get('region_tab1')?.column).toBe('region')
+  })
+
+  it('decodes a key containing a space (VA labels are free text)', () => {
+    expect(
+      parseVaParmap('va_parmap=My+Region-DEPT').map.get('my region')?.column
+    ).toBe('DEPT')
+  })
+
+  it('splits at the LAST hyphen, so a label may contain hyphens', () => {
+    expect(
+      parseVaParmap('va_parmap=Sales-2024-DEPT').map.get('sales-2024')?.column
+    ).toBe('DEPT')
+  })
+
+  describe('entries that cannot be used', () => {
+    it('reports an entry with no separator', () => {
+      const { map, problems } = parseVaParmap('va_parmap=DEPT')
+      expect(map.size).toBe(0)
+      expect(problems).toEqual([
+        'va_parmap "DEPT" was ignored: it needs a "-" between the VA parameter and the column name.'
+      ])
+    })
+
+    it('reports an entry with no parameter name', () => {
+      const { map, problems } = parseVaParmap('va_parmap=-DEPT')
+      expect(map.size).toBe(0)
+      expect(problems).toEqual([
+        'va_parmap "-DEPT" was ignored: the VA parameter name is missing.'
+      ])
+    })
+
+    it('reports an entry with no column name', () => {
+      const { map, problems } = parseVaParmap('va_parmap=REGION_TAB1-')
+      expect(map.size).toBe(0)
+      expect(problems).toEqual([
+        'va_parmap "REGION_TAB1-" was ignored: "" is not a valid SAS column name.'
+      ])
+    })
+
+    it('reports a column that is not a plain SAS name (injection guard)', () => {
+      // The column is interpolated into a SAS WHERE clause, so anything beyond a
+      // SAS name is refused at parse time rather than carried into a clause.
+      for (const entry of [
+        'A-x;drop',
+        'A-REGION%20OR%201=1',
+        "A-RE'GION",
+        'A-1ABC'
+      ]) {
+        const { map, problems } = parseVaParmap(`va_parmap=${entry}`)
+        expect(map.size).toBe(0)
+        expect(problems.length).toBe(1)
+        expect(problems[0]).toContain('is not a valid SAS column name')
+      }
+      expect(parseVaParmap('va_parmap=A-A_B9').map.get('a')?.column).toBe(
+        'A_B9'
+      )
+    })
+
+    it('keeps the usable entries and reports only the bad one', () => {
+      const { map, problems } = parseVaParmap(
+        'va_parmap=GOOD-REGION&va_parmap=BROKEN'
+      )
+      expect(map.get('good')?.column).toBe('REGION')
+      expect(problems.length).toBe(1)
+      expect(problems[0]).toContain('va_parmap "BROKEN"')
+    })
+  })
+
+  describe('transforms', () => {
+    it('parses an ADD anchor with an offset', () => {
+      const { map, problems } = parseVaParmap(
+        'va_parmap=SELECTED-D_DATE:ADD(QUARTER_START,-1)'
+      )
+      expect(problems).toEqual([])
+      expect(map.get('selected')?.column).toBe('D_DATE')
+      expect(map.get('selected')?.transform).toEqual({
+        op: 'ADD',
+        anchor: 'QUARTER_START',
+        offset: -1
+      })
+    })
+
+    it('defaults the offset to zero and accepts lower case', () => {
+      const { map, problems } = parseVaParmap(
+        'va_parmap=sel-d_date:add(quarter_start)'
+      )
+      expect(problems).toEqual([])
+      expect(map.get('sel')?.transform).toEqual({
+        op: 'ADD',
+        anchor: 'QUARTER_START',
+        offset: 0
+      })
+    })
+
+    it('parses a RANGE period', () => {
+      const { map, problems } = parseVaParmap(
+        'va_parmap=sel-D_DATE:RANGE(quarter,-1)'
+      )
+      expect(problems).toEqual([])
+      expect(map.get('sel')?.transform).toEqual({
+        op: 'RANGE',
+        period: 'QUARTER',
+        offset: -1
+      })
+    })
+
+    it('leaves a mapping without a transform as a plain column', () => {
+      const { map } = parseVaParmap('va_parmap=sel-D_DATE')
+      expect(map.get('sel')?.column).toBe('D_DATE')
+      expect(map.get('sel')?.transform).toBeUndefined()
+    })
+
+    it('reports an unknown operation, anchor, period or offset', () => {
+      const cases: [string, string][] = [
+        ['sel-D_DATE:SHIFT(QUARTER_START,-1)', 'is not a known transform'],
+        ['sel-D_DATE:ADD(FORTNIGHT_START,-1)', 'is not a known anchor'],
+        ['sel-D_DATE:RANGE(DECADE,-1)', 'is not a known period'],
+        ['sel-D_DATE:ADD(QUARTER_START,3.5)', 'is not a whole number'],
+        ['sel-D_DATE:ADD', 'is not a transform']
+      ]
+      for (const [entry, expected] of cases) {
+        const { map, problems } = parseVaParmap(`va_parmap=${entry}`)
+        expect(`${entry}: ${map.size}`).toBe(`${entry}: 0`)
+        expect(`${entry}: ${problems.length}`).toBe(`${entry}: 1`)
+        expect(problems[0]).toContain(expected)
+      }
+    })
+
+    it('keeps the parameter label free to contain a colon', () => {
+      // The key side is untouched by the transform grammar: only the column side
+      // is split, so a label with a colon still maps.
+      const { map, problems } = parseVaParmap(
+        'va_parmap=Sales:2026-D_DATE:ADD(QUARTER_START,-1)'
+      )
+      expect(problems).toEqual([])
+      expect(map.get('sales:2026')?.column).toBe('D_DATE')
+      expect(map.get('sales:2026')?.transform).toEqual({
+        op: 'ADD',
+        anchor: 'QUARTER_START',
+        offset: -1
+      })
+    })
+  })
+
+  describe('one parameter, one column', () => {
+    it('keeps the first mapping and reports a conflicting duplicate', () => {
+      const { map, problems } = parseVaParmap(
+        'va_parmap=DEPT-REGION&va_parmap=DEPT-OTHER'
+      )
+      expect(map.get('dept')?.column).toBe('REGION')
+      expect(problems).toEqual([
+        'va_parmap "DEPT-OTHER" was ignored: VA parameter "dept" already maps to REGION, and a parameter can filter only one column.'
+      ])
+    })
+
+    it('treats a duplicate key as a duplicate across case', () => {
+      const { map, problems } = parseVaParmap(
+        'va_parmap=Dept-REGION&va_parmap=DEPT-SOMETHING_ELSE'
+      )
+      expect(map.get('dept')?.column).toBe('REGION')
+      expect(problems.length).toBe(1)
+    })
+
+    it('accepts the same mapping written twice (no ambiguity)', () => {
+      const { map, problems } = parseVaParmap(
+        'va_parmap=DEPT-REGION&va_parmap=dept-region'
+      )
+      expect(map.get('dept')?.column).toBe('REGION')
+      expect(problems).toEqual([])
+    })
+
+    it('still allows two parameters to map to one column', () => {
+      // The multi-object case: different parameters, same DC column.
+      const { map, problems } = parseVaParmap(
+        'va_parmap=REGION_TAB1-REGION&va_parmap=REGION_TAB2-REGION'
+      )
+      expect(map.size).toBe(2)
+      expect(problems).toEqual([])
+    })
+  })
+})

client/src/app/shared/utils/parse-va-parmap.ts

@@ -0,0 +1,192 @@
+import {
+  VA_ANCHORS,
+  VA_PERIODS,
+  VaAnchor,
+  VaPeriod,
+  VaTransform
+} from './va-parmap-transform'
+
+/**
+ * What a VA parameter's mapping resolves to: the DC column, and optionally a
+ * transform that adds derived dates to the filter (e.g. the previous quarter's
+ * row alongside the selected one).
+ */
+export interface VaParmapTarget {
+  column: string
+  transform?: VaTransform
+}
+
+/** Result of `parseVaParmap` - the usable mappings, and what was ignored. */
+export interface VaParmap {
+  /** Lower-cased VA parameter key -> DC column (and transform, if any). */
+  map: Map<string, VaParmapTarget>
+  /** One plain-English line per `va_parmap` entry that was ignored. */
+  problems: string[]
+}
+
+/**
+ * Parses the `va_parmap` entries out of a hash query string (the part after `?`
+ * in `window.location.hash`).
+ *
+ * Format - one mapping per occurrence, each `<VA parameter key>-<DC column>`:
+ *
+ *     ?embed=va&va_parmap=REGION_TAB1-REGION&va_parmap=REGION_TAB2-REGION
+ *
+ * Repeating the parameter (rather than packing a comma-separated list into one
+ * value) keeps the grammar unambiguous: VA parameter labels are free text, so
+ * any list separator we picked could also appear inside a label.
+ *
+ * Each entry is split at its LAST `-`. A SAS column name can never contain a
+ * hyphen, so the final one always delimits the column, and a label that does
+ * contain hyphens (e.g. `Sales-2024`) still maps correctly.
+ *
+ * The key is matched case-insensitively, and trailing/leading blanks are
+ * trimmed, against a VA parameter's `label`, which is the same key the
+ * label->column match uses. The internal `name` (e.g. `pr273`) is not a key: it
+ * is the report's own bookkeeping, so an entry written against it never applies
+ * and is reported by `unmatchedMappings`.
+ *
+ * An entry that cannot be used is reported in `problems` rather than dropped in
+ * silence: these entries are hand-written in the report's DDC URL, usually by
+ * the report author, and a typo would otherwise degrade to "no filter" - which
+ * looks exactly like a working filter over the whole table.
+ *
+ * Returns an empty map when the parameter is absent, which leaves the default
+ * same-name resolution untouched (existing embeds are unaffected).
+ *
+ * Modeled on `parseEmbedParam` / `parseLabelsParam`, which read the same hash
+ * query.
+ */
+export function parseVaParmap(hashQuery: string): VaParmap {
+  const map = new Map<string, VaParmapTarget>()
+  const problems: string[] = []
+  if (!hashQuery) return { map, problems }
+
+  const entries = new URLSearchParams(hashQuery).getAll('va_parmap')
+  for (const entry of entries) {
+    // A transform sits at the end, in parentheses: `<param>-<column>:OP(args)`.
+    // It is lifted off FIRST, because its arguments may contain a hyphen (a
+    // negative offset) and the key/column split below uses the hyphen.
+    let body = entry ?? ''
+    let transformSpec = ''
+    if (body.trim().endsWith(')')) {
+      const open = body.lastIndexOf('(')
+      const colon = open > 0 ? body.lastIndexOf(':', open) : -1
+      if (colon > 0) {
+        transformSpec = body.slice(colon + 1).trim()
+        body = body.slice(0, colon)
+      }
+    }
+    // Last hyphen delimits the DC column (see above).
+    const separator = body.lastIndexOf('-')
+    if (separator < 0) {
+      problems.push(
+        `va_parmap "${entry}" was ignored: it needs a "-" between the VA parameter and the column name.`
+      )
+      continue
+    }
+    const key = body.slice(0, separator).trim().toLowerCase()
+    const column = body.slice(separator + 1).trim()
+    if (!key) {
+      problems.push(
+        `va_parmap "${entry}" was ignored: the VA parameter name is missing.`
+      )
+      continue
+    }
+    if (column.includes(':')) {
+      problems.push(
+        `va_parmap "${entry}" was ignored: "${column}" is not a transform - expected e.g. ADD(QUARTER_START,-1).`
+      )
+      continue
+    }
+    // The column name is interpolated into a SAS WHERE clause, so only a plain
+    // SAS name is accepted - anything else is refused rather than mapped.
+    if (!/^[A-Za-z_][A-Za-z0-9_]{0,31}$/.test(column)) {
+      problems.push(
+        `va_parmap "${entry}" was ignored: "${column}" is not a valid SAS column name.`
+      )
+      continue
+    }
+    const transform = transformSpec
+      ? parseTransform(transformSpec, entry, problems)
+      : undefined
+    if (transformSpec && !transform) continue
+    // One VA parameter filters one column. A second entry for the same parameter
+    // is ambiguous, so the FIRST one wins and the conflict is reported - the
+    // alternative (last one wins, silently) would apply a filter the author
+    // never sees the origin of. Two parameters mapping to the same column is
+    // fine and is how two report objects filter one column from their own
+    // parameters.
+    const existing = map.get(key)
+    if (existing) {
+      if (existing.column.toUpperCase() !== column.toUpperCase()) {
+        problems.push(
+          `va_parmap "${entry}" was ignored: VA parameter "${key}" already maps to ${existing.column}, and a parameter can filter only one column.`
+        )
+      }
+      continue
+    }
+    map.set(key, { column, transform })
+  }
+
+  return { map, problems }
+}
+
+/**
+ * Parses `OP(arg[, arg])`, returning undefined (and pushing a problem line) when
+ * the operation or one of its arguments is not in the allow-list.
+ *
+ * ADD takes an anchor and an optional offset; RANGE takes a period and an
+ * optional offset. The offset is whole periods, negative for earlier.
+ */
+function parseTransform(
+  spec: string,
+  entry: string,
+  problems: string[]
+): VaTransform | undefined {
+  const match = /^([A-Za-z]+)\s*\(([^)]*)\)$/.exec(spec)
+  if (!match) {
+    problems.push(
+      `va_parmap "${entry}" was ignored: "${spec}" is not a transform - expected e.g. ADD(QUARTER_START,-1).`
+    )
+    return undefined
+  }
+  const op = match[1].toUpperCase()
+  const args = match[2]
+    .split(',')
+    .map((arg) => arg.trim())
+    .filter((arg) => arg.length > 0)
+  const offsetArg = args[1] === undefined ? '0' : args[1]
+  if (!/^-?\d+$/.test(offsetArg)) {
+    problems.push(
+      `va_parmap "${entry}" was ignored: the offset "${offsetArg}" is not a whole number of periods.`
+    )
+    return undefined
+  }
+  const offset = Number(offsetArg)
+
+  if (op === 'ADD') {
+    const anchor = (args[0] || '').toUpperCase()
+    if (!VA_ANCHORS.includes(anchor as VaAnchor)) {
+      problems.push(
+        `va_parmap "${entry}" was ignored: "${args[0] || ''}" is not a known anchor (${VA_ANCHORS.join(', ')}).`
+      )
+      return undefined
+    }
+    return { op: 'ADD', anchor: anchor as VaAnchor, offset }
+  }
+  if (op === 'RANGE') {
+    const period = (args[0] || '').toUpperCase()
+    if (!VA_PERIODS.includes(period as VaPeriod)) {
+      problems.push(
+        `va_parmap "${entry}" was ignored: "${args[0] || ''}" is not a known period (${VA_PERIODS.join(', ')}).`
+      )
+      return undefined
+    }
+    return { op: 'RANGE', period: period as VaPeriod, offset }
+  }
+  problems.push(
+    `va_parmap "${entry}" was ignored: "${match[1]}" is not a known transform (ADD, RANGE).`
+  )
+  return undefined
+}

client/src/app/shared/utils/va-parmap-transform.spec.ts

@@ -0,0 +1,194 @@
+import {
+  applyVaTransform,
+  isRangeTransform,
+  VA_ANCHORS,
+  VA_PERIODS,
+  VaTransform
+} from './va-parmap-transform'
+
+/** Local y-m-d, so assertions do not depend on the runner's timezone offset. */
+const ymd = (d: Date) =>
+  `${d.getFullYear()}-${String(d.getMonth() + 1).padStart(2, '0')}-${String(
+    d.getDate()
+  ).padStart(2, '0')}`
+
+const date = (y: number, m: number, d: number) => new Date(y, m - 1, d)
+
+describe('va_parmap transforms', () => {
+  it('exposes only allow-listed anchors and periods', () => {
+    expect(VA_ANCHORS).toContain('QUARTER_START')
+    expect(VA_PERIODS).toEqual(['QUARTER', 'MONTH', 'YEAR'])
+  })
+
+  describe('ADD - the value plus one anchor beside it', () => {
+    it('adds the previous quarter start, which is the reported requirement', () => {
+      // intnx('qtr', dt, -1, 'b') for a value on a quarter boundary
+      const dates = applyVaTransform(date(2026, 10, 1), {
+        op: 'ADD',
+        anchor: 'QUARTER_START',
+        offset: -1
+      })
+      expect(dates.map(ymd)).toEqual(['2026-10-01', '2026-07-01'])
+    })
+
+    it('resolves a mid-quarter value to its own value plus the previous quarter start', () => {
+      const dates = applyVaTransform(date(2026, 8, 15), {
+        op: 'ADD',
+        anchor: 'QUARTER_START',
+        offset: -1
+      })
+      expect(dates.map(ymd)).toEqual(['2026-08-15', '2026-04-01'])
+    })
+
+    it('de-duplicates when the value already sits on the anchor', () => {
+      // 2026-01-01 is a quarter start, so "0 quarters back" adds nothing
+      const dates = applyVaTransform(date(2026, 1, 1), {
+        op: 'ADD',
+        anchor: 'QUARTER_START',
+        offset: 0
+      })
+      expect(dates.map(ymd)).toEqual(['2026-01-01'])
+    })
+
+    it('resolves the other anchors at their boundaries', () => {
+      const at = date(2026, 8, 15)
+      expect(
+        ymd(
+          applyVaTransform(at, {
+            op: 'ADD',
+            anchor: 'QUARTER_END',
+            offset: 0
+          })[1]
+        )
+      ).toBe('2026-09-30')
+      expect(
+        ymd(
+          applyVaTransform(at, { op: 'ADD', anchor: 'MONTH_END', offset: 0 })[1]
+        )
+      ).toBe('2026-08-31')
+      expect(
+        ymd(
+          applyVaTransform(at, {
+            op: 'ADD',
+            anchor: 'YEAR_START',
+            offset: -1
+          })[1]
+        )
+      ).toBe('2025-01-01')
+      expect(
+        ymd(
+          applyVaTransform(at, { op: 'ADD', anchor: 'YEAR_END', offset: 0 })[1]
+        )
+      ).toBe('2026-12-31')
+    })
+
+    it('handles February and leap years at month end', () => {
+      expect(
+        ymd(
+          applyVaTransform(date(2026, 3, 1), {
+            op: 'ADD',
+            anchor: 'MONTH_END',
+            offset: -1
+          })[1]
+        )
+      ).toBe('2026-02-28')
+      expect(
+        ymd(
+          applyVaTransform(date(2024, 3, 15), {
+            op: 'ADD',
+            anchor: 'MONTH_END',
+            offset: -1
+          })[1]
+        )
+      ).toBe('2024-02-29')
+    })
+
+    it('crosses a year boundary when the offset takes it out of the year', () => {
+      const dates = applyVaTransform(date(2027, 1, 1), {
+        op: 'ADD',
+        anchor: 'QUARTER_START',
+        offset: -1
+      })
+      expect(dates.map(ymd)).toEqual(['2027-01-01', '2026-10-01'])
+    })
+  })
+
+  describe('RANGE - a closed period', () => {
+    it('covers the previous quarter', () => {
+      const dates = applyVaTransform(date(2026, 8, 15), {
+        op: 'RANGE',
+        period: 'QUARTER',
+        offset: -1
+      })
+      expect(dates.map(ymd)).toEqual(['2026-04-01', '2026-06-30'])
+    })
+
+    it('covers the current month and the previous year', () => {
+      expect(
+        applyVaTransform(date(2026, 8, 15), {
+          op: 'RANGE',
+          period: 'MONTH',
+          offset: 0
+        }).map(ymd)
+      ).toEqual(['2026-08-01', '2026-08-31'])
+      expect(
+        applyVaTransform(date(2026, 8, 15), {
+          op: 'RANGE',
+          period: 'YEAR',
+          offset: -1
+        }).map(ymd)
+      ).toEqual(['2025-01-01', '2025-12-31'])
+    })
+
+    it('reports itself as a range so the caller picks BETWEEN', () => {
+      expect(
+        isRangeTransform({ op: 'RANGE', period: 'QUARTER', offset: 0 })
+      ).toBe(true)
+      expect(
+        isRangeTransform({
+          op: 'ADD',
+          anchor: 'QUARTER_START',
+          offset: 0
+        } as VaTransform)
+      ).toBe(false)
+    })
+  })
+
+  describe('time of day', () => {
+    it('gives a datetime range whole days, not the value clock time', () => {
+      // A BETWEEN that starts at 13:45 on day one loses the earlier part of that
+      // day, and one that ends at 13:45 on the last day loses the rest of it.
+      const value = new Date(2026, 7, 15, 13, 45, 30)
+      const [start, end] = applyVaTransform(
+        value,
+        { op: 'RANGE', period: 'QUARTER', offset: 0 },
+        true
+      )
+      expect([
+        start.getHours(),
+        start.getMinutes(),
+        start.getSeconds()
+      ]).toEqual([0, 0, 0])
+      expect([end.getHours(), end.getMinutes(), end.getSeconds()]).toEqual([
+        23, 59, 59
+      ])
+      expect(ymd(start)).toBe('2026-07-01')
+      expect(ymd(end)).toBe('2026-09-30')
+    })
+
+    it('keeps the time for datetime columns and drops it otherwise', () => {
+      const value = new Date(2026, 9, 1, 13, 45, 30)
+      const add: VaTransform = {
+        op: 'ADD',
+        anchor: 'QUARTER_START',
+        offset: -1
+      }
+      const kept = applyVaTransform(value, add, true)[1]
+      expect([kept.getHours(), kept.getMinutes(), kept.getSeconds()]).toEqual([
+        13, 45, 30
+      ])
+      const dropped = applyVaTransform(value, add, false)[1]
+      expect([dropped.getHours(), dropped.getMinutes()]).toEqual([0, 0])
+    })
+  })
+})

client/src/app/shared/utils/va-parmap-transform.ts

@@ -0,0 +1,151 @@
+/**
+ * Anchors and periods a `va_parmap` transform can resolve a report value to.
+ *
+ * The names describe the result rather than mirroring SAS `intnx`, because the
+ * arithmetic runs in the browser: emulating `intnx`'s alignment letters (`b`,
+ * `e`, `s`, `m`, never mind week alignment and fiscal quarters) would invite
+ * assumptions we could not honour. Each anchor below is one small function, so
+ * adding one is a function plus a test row.
+ */
+export const VA_ANCHORS = [
+  'QUARTER_START',
+  'QUARTER_END',
+  'MONTH_START',
+  'MONTH_END',
+  'YEAR_START',
+  'YEAR_END'
+] as const
+export type VaAnchor = (typeof VA_ANCHORS)[number]
+
+export const VA_PERIODS = ['QUARTER', 'MONTH', 'YEAR'] as const
+export type VaPeriod = (typeof VA_PERIODS)[number]
+
+/**
+ * A parsed transform: the operation plus its validated arguments.
+ *
+ * `ADD` yields the value and one anchor beside it (a list of dates), which is
+ * how "this period's row plus the previous quarter's" is expressed.  `RANGE`
+ * yields a closed period (two dates) for a BETWEEN.
+ */
+export type VaTransform =
+  | { op: 'ADD'; anchor: VaAnchor; offset: number }
+  | { op: 'RANGE'; period: VaPeriod; offset: number }
+
+/** True when the transform produces a closed range rather than a list. */
+export function isRangeTransform(transform: VaTransform): boolean {
+  return transform.op === 'RANGE'
+}
+
+/** Local midnight on the first day of the month `offset` months from `date`. */
+function monthStart(date: Date, offset: number): Date {
+  return new Date(date.getFullYear(), date.getMonth() + offset, 1)
+}
+
+/** Local midnight on the last day of the month `offset` months from `date`. */
+function monthEnd(date: Date, offset: number): Date {
+  return new Date(date.getFullYear(), date.getMonth() + offset + 1, 0)
+}
+
+/** First day of the quarter `offset` quarters from the one containing `date`. */
+function quarterStart(date: Date, offset: number): Date {
+  const firstMonthOfQuarter = 3 * Math.floor(date.getMonth() / 3)
+  return new Date(date.getFullYear(), firstMonthOfQuarter + offset * 3, 1)
+}
+
+/** Last day of the quarter `offset` quarters from the one containing `date`. */
+function quarterEnd(date: Date, offset: number): Date {
+  // Day 0 of a month is the last day of the month before it: calendar
+  // construction, so a daylight-saving transition inside the quarter cannot
+  // shift the result by an hour the way subtracting 24 hours would.
+  const nextStart = quarterStart(date, offset + 1)
+  return new Date(nextStart.getFullYear(), nextStart.getMonth(), 0)
+}
+
+/** First day of the year `offset` years from the one containing `date`. */
+function yearStart(date: Date, offset: number): Date {
+  return new Date(date.getFullYear() + offset, 0, 1)
+}
+
+/** Last day of the year `offset` years from the one containing `date`. */
+function yearEnd(date: Date, offset: number): Date {
+  return new Date(date.getFullYear() + offset, 11, 31)
+}
+
+/** Resolve one anchor, keeping the time of day of `date` for datetime columns. */
+function resolveAnchor(
+  anchor: VaAnchor,
+  date: Date,
+  offset: number,
+  keepTime: boolean
+): Date {
+  let resolved: Date
+  switch (anchor) {
+    case 'QUARTER_START':
+      resolved = quarterStart(date, offset)
+      break
+    case 'QUARTER_END':
+      resolved = quarterEnd(date, offset)
+      break
+    case 'MONTH_START':
+      resolved = monthStart(date, offset)
+      break
+    case 'MONTH_END':
+      resolved = monthEnd(date, offset)
+      break
+    case 'YEAR_START':
+      resolved = yearStart(date, offset)
+      break
+    default:
+      resolved = yearEnd(date, offset)
+      break
+  }
+  if (keepTime) {
+    resolved.setHours(date.getHours(), date.getMinutes(), date.getSeconds(), 0)
+  }
+  return resolved
+}
+
+/**
+ * The dates a transform adds to a mapping, in order.
+ *
+ * `ADD` returns the report's own value plus the anchor beside it, de-duplicated -
+ * so a value that already sits on the anchor yields one date, and the clause
+ * stays an equality rather than an `IN` with a repeat.  `RANGE` returns the
+ * period's first and last day.
+ *
+ * `keepTime` carries the time of day across for datetime columns, since a
+ * DATETIME filter compares to the second; date columns use midnight.
+ */
+export function applyVaTransform(
+  value: Date,
+  transform: VaTransform,
+  keepTime = false
+): Date[] {
+  if (transform.op === 'RANGE') {
+    // A range covers whole days: the bounds are the period's own, never the
+    // value's clock time. Reusing keepTime here would start the range mid-morning
+    // on day one and end it mid-morning on the last day, dropping the rest of
+    // both boundary days from a BETWEEN.
+    const start = resolveAnchor(
+      (transform.period + '_START') as VaAnchor,
+      value,
+      transform.offset,
+      false
+    )
+    const end = resolveAnchor(
+      (transform.period + '_END') as VaAnchor,
+      value,
+      transform.offset,
+      false
+    )
+    if (keepTime) end.setHours(23, 59, 59, 0)
+    return [start, end]
+  }
+  const anchor = resolveAnchor(
+    transform.anchor,
+    value,
+    transform.offset,
+    keepTime
+  )
+  return anchor.getTime() === value.getTime() ? [value] : [value, anchor]
+}

client/src/styles.scss

@@ -5019,6 +5019,57 @@ body[cds-theme='dark'] {
   border: 1px solid var(--cds-alias-status-warning) !important;
 }
 
+/* Derived (computed) columns: read-only, but computed at read time rather than
+  locked by policy.  Tinted and marked with an 'fx' glyph instead of the
+  read-only dimming, so the two do not read as the same thing.  Blue is the
+  info family - the primary key green and the warning ochre keep their meaning. */
+.handsontable th.dc-derived-header {
+  background: #d3e5f2 !important;
+  box-shadow: inset 0 -2px 0 #1a6ea3;
+}
+
+.handsontable th.dc-derived-header div::after {
+  content: 'fx';
+  margin-left: 4px;
+  font-size: 9px;
+  font-style: italic;
+  opacity: 0.7;
+}
+
+.handsontable td.dc-derived-cell {
+  background: #eef5fa !important;
+  font-style: italic;
+}
+
+/* the first derived column starts the computed block */
+.handsontable th.dc-derived-first {
+  border-left: 2px solid #1a6ea3 !important;
+}
+
+.handsontable td.dc-derived-cell.dc-derived-first {
+  box-shadow: inset 2px 0 0 #1a6ea3;
+}
+
+body[cds-theme='dark'] {
+  .handsontable th.dc-derived-header {
+    background: #2b4a5c !important;
+    box-shadow: inset 0 -2px 0 #4b9cc9;
+  }
+
+  .handsontable td.dc-derived-cell {
+    background: #24343d !important;
+    color: #dbe7ee;
+  }
+
+  .handsontable th.dc-derived-first {
+    border-left-color: #4b9cc9 !important;
+  }
+
+  .handsontable td.dc-derived-cell.dc-derived-first {
+    box-shadow: inset 2px 0 0 #4b9cc9;
+  }
+}
+
 .handsontable .numericListbox {
   text-align: right;
 }

package.json

@@ -1,6 +1,6 @@
 {
   "name": "dcfrontend",
-  "version": "7.18.0",
+  "version": "7.21.0",
   "description": "Data Controller",
   "devDependencies": {
     "@nogoo9/gitleaks": "8.30.1-post.2",

sas/.sasjslint

@@ -11,5 +11,6 @@
   "noNestedMacros": true,
   "hasMacroParentheses": true,
   "strictMacroDefinition": true,
+  "ignoredMacros": ["webout", "mpeinit", "mpeterm"],
   "ignoreList":["sasjs/scratch"]
 }

sas/deploy/makeDataSas719.json

@@ -0,0 +1,10 @@
+{
+  "fromjs": [
+    {
+      "ADMIN": "all-users",
+      "DCPATH": "/tmp/dcdata",
+      "OUTFMT": "JSON",
+      "KEEPME": "Y"
+    }
+  ]
+}

sas/deploy/makeDataServer.json

@@ -1,7 +1,7 @@
 {
   "fromjs": [
     {
-      "ADMIN": "AllUsers",
+      "ADMIN": "all-users",
       "DCPATH": "/tmp/dcdata",
       "OUTFMT": "JSON"
     }

sas/mocks/mock-licence.json

@@ -0,0 +1,4 @@
+{
+  "licenceKey": "G4H/DNTdEMkmCIxOrYkRYTQ1P4sWg0gs8Zk9rakxFNYDZcLyEPMJJo82YJQ/S24P2E/YPxaWsDZXgOrWKfeCDS7NrLcwsoFNa8KRF2WTlRgSMiG4sjBRX8zNEIvszj5DAq4LEgOUUAdfctNfpUcygbxT1dhqy6gU3hBu/xRuXGd36rhxQwfahaKkfaCg/rFplkrvqnjoJ6622BxWP/HG1dxZUE7HNJyT4tNaXzlC9U6u/aaJJ66+V+9511BwjM9ss4nyHuHoogVE1yrzLMlL94P3QoM1hwkGHnEHmFzg3zHbQG1f3pUs9YzK0ytsFKU7AK948gJrGbf37Zqa+Mz7EZf6XzqqXx0zQ4y9zAhLV9s4QG8JdhrByhfG5kefrs4L/BpgKq3ElgYL5EMkrLJUrlqWYI2ouxJuIGCQz41VCWXJ2S7pd39ljifc8Zj+t2Mn5EEkRVsrbdxjCkyoZOQ1ALofa1+auUI26Rm5rwEgrfhEocDmphJpO2F8lZgwDxbepSE6v/vEV/JQWBQTyrX10W+qOEu8OhvZrYJo+eIJMaO7q1pjfdrn3XhoH+BeB3Sy1sevGLXQZC+CzOtcvajetGGCDAabtTLc49ouWPUiUOAWPE3uYymsMoRZgpLIMxaUM86hoK0r9T6wSaP0bddK754sXN1jWwIDAHSGz5WO1ko=",
+  "activationKey": "MIIJRAIBADANBgkqhkiG9w0BAQEFAASCCS4wggkqAgEAAoICAQD2w8Ge6tMI82vaQ7x5ZdlwnTJycn0UFZoSgRwObaM0NC5wp9Y8lwkVzN8FUhg2oYSTpvyJpix+VGRj0AlNBwFAVcJG9VDPYh2LE3Us9ghRfkzFUiM2PqorHnzL5DwpailRtmFHoDQPGwwSmKncTo2QHql/K6tWe7g3Nv9uqQNks283hvDQ6PVGU4TaFTSRbjGX4j+JAR651J6wI6nY+9kNxOBwnL/yKTnJGmL8oqZ+oC45quZzgRW4dLIeTODcB4IZe2ruxZ9tvXpSuAESiOv5bBZyMJEKdKwsRdt6vk+bezkZBCzPq+ly892Ncq7x5u4/8dGcGxRxxp5TnIIx3hFoiXZ7svXJ7st6uHuKkuwYq+ebWh6bcvQAwH2wCP40twMjQgmU8nHHo6+pulxu1wVZBxtoyM4m3Lu0ZncTi6mF0duiPvpkNNAZzKOF98aUq3hxYp5PP28qrtWuPsgSdTNWJwnV2R288yAPx4WLUAGB9h889d/m67HUiKadmfG4oyUnChCU/mqeJ/5nQkK1n76ezRPGsTcMZIv2mmPLShA/j38lGMjR/3pzM2Ha6UFjJKpurpeDiyt5pUdmRBpQRoBUx/PHnbXBsO9PBXa0mjdq3UFbgb5iBD1Fz2YClZ/h0d4YlYUMSeharDrLfR15g3fGnzxrkR03zYeHkxMUi5vYHQIDAQABAoICAFxUg5pcH8lOF1FMHfXIY5MM5mbHKsC8knERCPfnEvVB/wxc6D60fow/I0Vg4blzNuFuTjiMykIcfpGQMiK1l8TEMdSODoYCHq6+gY3ootUnLj94i79MtCw9FJHPnKkcVUiDNqPvr/rk8l/06j2J0NMXffEc3FeYJCXYuLtlluTzdI4wueNY/wkoz+EvpQMzWXAMkhxHpIEAqr2aIn+TQDQ0MYxBQlqwDTSPYHFJxIVFlvYx2VdnecKSO2E1/76kOowhTzH3FL+hcWcI/TG7tJuXU1C3llV0sUVBgC1NjwjMkuEm3EN5/ffqgj/eJNJTR1r6tqSzxzMJD42WVmqlPhqEmmRE1yeaio6Ulm5EJLREYEY4/1EotdIO6PCXF1Xeo+10VJLcwi19vzMlz7vFB6vSJzJNYOaiwm4rgmF12+nNt18cHoFvegGPAoCJurUhXU+iV6LlKYB8J/+kpITleZwm4dEaa6EWxgFf4nZ4N+qRVqk4JfC+rN3DQgbPcII+z7axAb8xutq5rvzskGAmKcTEagMG+pnk9DI+0m1C2J6+mC/YruVlpSlPB7Ecu/OF++vJoSCLuDXVuxMTQ3tQXoicOG/S8d1aa3nq4fvLIgZTOZoffUvNvjCyy/Dlv6vaDVCjgqt2kFZVSZv3C6ttyrNVBSjhucASGTq/7q82JKVrAoIBAQD7e7SYqDM8SAjOfpRAHCbIVVSaHw1tul16B6JAF67OIs1uD//Y+maL4GHlpFxnib0tg6z3+sG10NMUhK7elB6SkJ8voT+Ztg/+on659vmaqwTlkGjgks+8DmIw1b0ZbXuIqhnkOeSsll2V9NWItIrbzNwvhtMEQsOtFvmrLlKQvx9Bo3lGmbfdU1/9XpwW4wiA5ONcFCl1xllX9nItB+ActPoAeQgnuejWAISQkITDTOFcGu0AjMjJ66HUlY/WDbf7wY/s2aMbOiVx7B6YzQnK+lZdjjNfoaL8nfN8Z8ZzJFAgmTQ9ztZCJ8fLTfZDsYD6XZnVEIG401+6de0Ea4MHAoIBAQD7Mlr++keNDxYy3k6kwTXDOSKMDfXIDverFy/tiNorcVU5nIpuhjowl4HIQVAhzlXhATp938LM/PvJIeu9HtgsY9BFNL52fr7iGEWknqZQVE+8Um35tzV3vtPH9NVXHIYmcRCAQQJ5a6oVQRJKAM3E1+htM2AeZKMaQSCC6VHGBXJG9nEltdMKTet09pd26Z5y/YIfTqbIekyjRomGdInyWBkpJLJ/uROVb9OGnsYWadDktn7y9+oH+VZoJxW5QwXV0OL7wMzymlrKhhBNMziJRBk4rVztclYzUShpwE8Xfjt5WYkOoieXtmkdpF7TMvRYy6cdflh39l8s7THq4U67AoIBAQDPShR+kIZK9gVsUCHbBllhYBSwRVBIVgmfAJPYyKyp9T3nrwBw1VWZjoajR2tzXLG2xobYkLUDLPIQQ53wkZ7xqIvOMte917gvZ7mVh+IOf1E1+Ga6VcktdZ2yKzwHrZrZ93wIB0BakkbpyDoKyOZ8bsw/0J43MOEh14JMe45LCX62GA1rCGHp6UOCEW4P32uu8vwFdgkGr0wNmXJbc9ZjUzmPEjSKqk8ltp+OmdEVOlW4GempPmJzLeri07UP95bjyZTc4jfzxWYAFWTTBo8/5EE0NHZcKVuzZm7A6LlHR3NUq3PITqmFNk0VY1nOSo3J0UVvvn9u3axWgcWwkv4ZAoIBAQD5GaB/RfZ6fcGHxEi8vRTxedE2jglz0NPuCFpw9ctUVVksfn4QyG8gDL6YtZwfShyDutIkTSdCXIXbmRXLP7Ejo/+/AWT24Q2t7F/ztZg8BL/lguCoR/RvwP8HRVXiwtiNeoXgiZdIjWnnFdZrKIzlbS6CS/Y/bZEI0hjJeVKsSm5ezLi7qqmTiF28aWUeeOG2KotFPJvMJuuk5qgue5boilC5KjkQy5oOEQz0bLMHOvv6MreFfi/IpIv5UPkL9NjuhezxtuoGkucB+xxjUi34gV15hQGoPkAZN5wZ8qj8JSQZXJ67sjYNDskCqiOW0Orau2oHp2lX3SvYw4dC4xdNAoIBAQDSXF8OnWu08smGDs4S57yfqe59N0ULIBDOZdEQcLXRaDeAoOwQj8szUf0pycXUZLvXxv29iF61FJg70iGGCp3lW+9Oe0P+dGsKvzphRGsQPZy4h/qZDoQ9dE667ejI4aDb/C1GaOZ8TT1QfpQcVKbgLjdC19IGQPhFljGM1wMQojU5lp4/iKmA3J3ahOuV4dvEUr6xkqzq+PhH405XdURXsVc1J6qYX2wQXRoB0oDsrCbqidZi14DaeE8UhZRA6m9haUOb/3Y+OkCs2QIW4MNZ+zg8XtHjiUhMxAaJ6zdEFltWPoVFsVfgoecquAHb0uDJJRz/+RfR0AiG0XG0NW+C"
+}

sas/mocks/sasjs/services/admin/makedata.js

@@ -1159,6 +1159,26 @@ const tables = {
       selectbox_value: 'FORMULA',
       selectbox_order: 1,
       ver_to_dttm: 127490111999
+    },
+    {
+      selectbox_rk: 73,
+      ver_from_dttm: 0,
+      select_lib: dcLibref,
+      select_ds: 'MPE_DERIVED',
+      base_column: 'VAR_TYPE',
+      selectbox_value: 'C',
+      selectbox_order: 1,
+      ver_to_dttm: 127490111999
+    },
+    {
+      selectbox_rk: 74,
+      ver_from_dttm: 0,
+      select_lib: dcLibref,
+      select_ds: 'MPE_DERIVED',
+      base_column: 'VAR_TYPE',
+      selectbox_value: 'N',
+      selectbox_order: 2,
+      ver_to_dttm: 127490111999
     }
   ],
   MPE_SUBMIT: [],
@@ -1371,6 +1391,19 @@ const tables = {
         'RLS_SCOPE RLS_GROUP RLS_LIBREF RLS_TABLE RLS_GROUP_LOGIC RLS_SUBGROUP_LOGIC RLS_SUBGROUP_ID RLS_VARIABLE_NM RLS_OPERATOR_NM RLS_RAW_VALUE',
       post_edit_hook: 'services/hooks/mpe_row_level_security_postedit'
     },
+    {
+      tx_from: 0,
+      tx_to: 127490111999,
+      libref: dcLibref,
+      dsn: 'MPE_DERIVED',
+      num_of_approvals_required: 1,
+      loadtype: 'TXTEMPORAL',
+      buskey: 'LIBREF DSN VAR_NM',
+      notes: 'Derived (computed) variables for the EDIT screen',
+      var_txfrom: 'TX_FROM',
+      var_txto: 'TX_TO',
+      post_edit_hook: 'services/hooks/mpe_derived_postedit'
+    },
     {
       tx_from: 0,
       tx_to: 127490111999,
@@ -2002,6 +2035,206 @@ const tables = {
       rule_value: 'services/validations/mpe_x_test.some_num',
       rule_active: 1,
       tx_to: 127490111999
+    },
+    {
+      tx_from: 0,
+      base_lib: dcLibref,
+      base_ds: 'MPE_DERIVED',
+      base_col: 'LIBREF',
+      rule_type: 'NOTNULL',
+      rule_value: ' ',
+      rule_active: 1,
+      tx_to: 127490111999
+    },
+    {
+      tx_from: 0,
+      base_lib: dcLibref,
+      base_ds: 'MPE_DERIVED',
+      base_col: 'LIBREF',
+      rule_type: 'CASE',
+      rule_value: 'UPCASE',
+      rule_active: 1,
+      tx_to: 127490111999
+    },
+    {
+      tx_from: 0,
+      base_lib: dcLibref,
+      base_ds: 'MPE_DERIVED',
+      base_col: 'DSN',
+      rule_type: 'NOTNULL',
+      rule_value: ' ',
+      rule_active: 1,
+      tx_to: 127490111999
+    },
+    {
+      tx_from: 0,
+      base_lib: dcLibref,
+      base_ds: 'MPE_DERIVED',
+      base_col: 'DSN',
+      rule_type: 'CASE',
+      rule_value: 'UPCASE',
+      rule_active: 1,
+      tx_to: 127490111999
+    },
+    {
+      tx_from: 0,
+      base_lib: dcLibref,
+      base_ds: 'MPE_DERIVED',
+      base_col: 'VAR_NM',
+      rule_type: 'NOTNULL',
+      rule_value: ' ',
+      rule_active: 1,
+      tx_to: 127490111999
+    },
+    {
+      tx_from: 0,
+      base_lib: dcLibref,
+      base_ds: 'MPE_DERIVED',
+      base_col: 'VAR_NM',
+      rule_type: 'CASE',
+      rule_value: 'UPCASE',
+      rule_active: 1,
+      tx_to: 127490111999
+    },
+    {
+      tx_from: 0,
+      base_lib: dcLibref,
+      base_ds: 'MPE_DERIVED',
+      base_col: 'VAR_NM',
+      rule_type: 'HARDREGEX',
+      rule_value: '^[A-Za-z_][A-Za-z0-9_]*$',
+      rule_active: 1,
+      tx_to: 127490111999
+    },
+    {
+      tx_from: 0,
+      base_lib: dcLibref,
+      base_ds: 'MPE_DERIVED',
+      base_col: 'VAR_DEPS',
+      rule_type: 'CASE',
+      rule_value: 'UPCASE',
+      rule_active: 1,
+      tx_to: 127490111999
+    },
+    {
+      tx_from: 0,
+      base_lib: dcLibref,
+      base_ds: 'MPE_DERIVED',
+      base_col: 'VAR_DEPS',
+      rule_type: 'HARDREGEX',
+      rule_value: '^[A-Za-z0-9_ ]*$',
+      rule_active: 1,
+      tx_to: 127490111999
+    },
+    {
+      tx_from: 0,
+      base_lib: dcLibref,
+      base_ds: 'MPE_DERIVED',
+      base_col: 'VAR_TYPE',
+      rule_type: 'NOTNULL',
+      rule_value: ' ',
+      rule_active: 1,
+      tx_to: 127490111999
+    },
+    {
+      tx_from: 0,
+      base_lib: dcLibref,
+      base_ds: 'MPE_DERIVED',
+      base_col: 'VAR_TYPE',
+      rule_type: 'CASE',
+      rule_value: 'UPCASE',
+      rule_active: 1,
+      tx_to: 127490111999
+    },
+    {
+      tx_from: 0,
+      base_lib: dcLibref,
+      base_ds: 'MPE_DERIVED',
+      base_col: 'VAR_TYPE',
+      rule_type: 'HARDSELECT',
+      rule_value: ' ',
+      rule_active: 1,
+      tx_to: 127490111999
+    },
+    {
+      tx_from: 0,
+      base_lib: dcLibref,
+      base_ds: 'MPE_DERIVED',
+      base_col: 'VAR_LEN',
+      rule_type: 'NOTNULL',
+      rule_value: ' ',
+      rule_active: 1,
+      tx_to: 127490111999
+    },
+    {
+      tx_from: 0,
+      base_lib: dcLibref,
+      base_ds: 'MPE_DERIVED',
+      base_col: 'VAR_LEN',
+      rule_type: 'MINVAL',
+      rule_value: '1',
+      rule_active: 1,
+      tx_to: 127490111999
+    },
+    {
+      tx_from: 0,
+      base_lib: dcLibref,
+      base_ds: 'MPE_DERIVED',
+      base_col: 'VAR_LEN',
+      rule_type: 'MAXVAL',
+      rule_value: '32767',
+      rule_active: 1,
+      tx_to: 127490111999
+    },
+    {
+      tx_from: 0,
+      base_lib: dcLibref,
+      base_ds: 'MPE_DERIVED',
+      base_col: 'VAR_FMT',
+      rule_type: 'CASE',
+      rule_value: 'UPCASE',
+      rule_active: 1,
+      tx_to: 127490111999
+    },
+    {
+      tx_from: 0,
+      base_lib: dcLibref,
+      base_ds: 'MPE_DERIVED',
+      base_col: 'VAR_FMT',
+      rule_type: 'HARDREGEX',
+      rule_value: '^(\$?[A-Za-z][A-Za-z0-9_]*(\.[0-9]+)?\.?|\$?[0-9]+(\.[0-9]+)?\.?)?$',
+      rule_active: 1,
+      tx_to: 127490111999
+    },
+    {
+      tx_from: 0,
+      base_lib: dcLibref,
+      base_ds: 'MPE_DERIVED',
+      base_col: 'VAR_LOGIC',
+      rule_type: 'NOTNULL',
+      rule_value: ' ',
+      rule_active: 1,
+      tx_to: 127490111999
+    },
+    {
+      tx_from: 0,
+      base_lib: dcLibref,
+      base_ds: 'MPE_DERIVED',
+      base_col: 'VAR_LABEL',
+      rule_type: 'HARDREGEX',
+      rule_value: '^[^\\x00-\\x1F\\x7F]*$',
+      rule_active: 1,
+      tx_to: 127490111999
+    },
+    {
+      tx_from: 0,
+      base_lib: dcLibref,
+      base_ds: 'MPE_DERIVED',
+      base_col: 'VAR_LOGIC',
+      rule_type: 'HARDREGEX',
+      rule_value: '^(?!.*(/\\*|\\*/))[^;&%]*$',
+      rule_active: 1,
+      tx_to: 127490111999
     }
   ],
   MPE_X_TEST: [

sas/mocks/sasjs/services/auditors/getauditfile.js

@@ -47,7 +47,8 @@ if (!tableId || !libds) {
   webOutClose()
 } else {
   // The SAS service also checks EDIT or APPROVE access (mpe_accesscheck). The
-  // mock has no permission store, so access is not emulated.
+  // mock seeds MPE_SECURITY but no mock service enforces it, so access is not
+  // emulated here.
 
   // The approval folder is zipped as it stands. getauditfile.sas lists it with
   // mp_dirlist and adds each file to the package under contents/.

sas/mocks/sasjs/services/dcMockUtils.js

@@ -641,6 +641,27 @@ function zipUtf8(value) {
     return Uint8Array.from(out)
 }
 
+/**
+ * Renders rows as CSV, the counterpart of %mp_ds2csv: the header row, then one
+ * line per row, with values quoted only when they carry a delimiter, quote or
+ * line break.
+ *
+ * @param {string[]} headers column names, in order
+ * @param {Array<Object>} rows objects keyed by column name
+ * @returns {string} CSV text with a trailing newline
+ */
+function toCsv(headers, rows) {
+    const cell = (value) => {
+        const text = value === undefined || value === null ? '' : String(value)
+        return /[",\n\r]/.test(text) ? '"' + text.replace(/"/g, '""') + '"' : text
+    }
+    const lines = [headers.map(cell).join(',')]
+    for (const row of rows) {
+        lines.push(headers.map((header) => cell(row[header])).join(','))
+    }
+    return lines.join('\n') + '\n'
+}
+
 /**
  * Builds a zip archive in memory, entries stored rather than deflated - the JS
  * counterpart of the ODS package getauditfile.sas publishes. The zip format
@@ -661,16 +682,19 @@ function mpZipPackage(entries) {
         const name = zipUtf8(entry.name)
         const data = zipUtf8(entry.content)
         const crc = zipCrc32(data)
+        // Bit 11 marks the name as UTF-8. Without it an extractor decodes a
+        // non-ASCII name as CP437, so the entry cannot be read back by name.
+        const flags = name.some((byte) => byte >= 0x80) ? 0x0800 : 0
 
         // 1980-01-01 00:00, the earliest DOS timestamp: a stable value keeps
         // the archive byte-identical for identical inputs.
         const header = [].concat(
-            u32(0x04034b50), u16(20), u16(0), u16(0), u16(0), u16(0x21),
+            u32(0x04034b50), u16(20), u16(flags), u16(0), u16(0), u16(0x21),
             u32(crc), u32(data.length), u32(data.length), u16(name.length), u16(0)
         )
 
         local.push(Uint8Array.from(header), name, data)
-        central.push({ name, crc, size: data.length, offset })
+        central.push({ name, crc, size: data.length, offset, flags })
         offset += header.length + name.length + data.length
     }
 
@@ -678,7 +702,7 @@ function mpZipPackage(entries) {
     let directorySize = 0
     for (const entry of central) {
         const header = [].concat(
-            u32(0x02014b50), u16(20), u16(20), u16(0), u16(0), u16(0), u16(0x21),
+            u32(0x02014b50), u16(20), u16(20), u16(entry.flags), u16(0), u16(0), u16(0x21),
             u32(entry.crc), u32(entry.size), u32(entry.size),
             u16(entry.name.length), u16(0), u16(0), u16(0), u16(0), u32(0),
             u32(entry.offset)

sas/mocks/sasjs/services/editors/getdata.js

@@ -101,7 +101,7 @@ function pkFirst(columns, keys) {
 // ─── Derived (computed) variables (mpe_derived.sas) ─────────────────────────
 // Applied before the filter, so a filter can reference them.  The derived
 // columns are added to the schema too, which is what puts them in COLS and in
-// the grid (read only, since the mock marks every column READ).
+// the grid; they are held read-only by a READONLY rule below.
 
 const derived = mpeDerived({
     libref: requestedLibref,
@@ -160,6 +160,16 @@ if (tableData && tableData.columns) {
     }
 }
 
+// A derived variable is computed at read time, so the grid must not accept a
+// value for it.  COLS carries READ for the same reason (a derived column has no
+// CLS rule row), but the client only applies COLS.CLS_RULE when CLS_FLAG is
+// set, which column-level security drives - so the rule is declared here too,
+// as getdata.sas does.  dqrules is the request-local array the filter/map above
+// built, so appending to it leaves the loaded MPE_VALIDATIONS rows untouched.
+for (const v of derived) {
+    dqrules.push({ BASE_COL: v.name, RULE_TYPE: 'READONLY', RULE_VALUE: '' })
+}
+
 // ─── DQ dropdown data (from MPE_SELECTBOX + SOFTSELECT from table data) ──────
 // MPE_SELECTBOX lives in the control library.
 
@@ -426,7 +436,11 @@ if (tableData && tableData.columns && tableData.rows) {
             CLS_FLAG: 0,
             ISMAP: 0,
             TRUNCATED: truncated,
-            TRUNC_MSG: truncMsg
+            TRUNC_MSG: truncMsg,
+            // names the derived columns for the grid, as getdata.sas does - the
+            // READONLY rule above is why they cannot be edited, this is how the
+            // EDIT screen knows they are computed rather than locked
+            DERIVEDVARS: derived.map(d => d.name).join(' ')
         }],
         versions: [],
         xl_rules: []
@@ -441,7 +455,7 @@ if (tableData && tableData.columns && tableData.rows) {
         ],
         maxvarlengths: [], query: [], sasdata: [],
         $sasdata: { vars: {} },
-        sasparams: [{ COLHEADERS: '', FILTER_TEXT: ' ', PKCNT: 0, PK: '', DTVARS: '', DTTMVARS: '', TMVARS: '', LOADTYPE: 'UPDATE', RK_FLAG: 0, CLS_FLAG: 0, ISMAP: 0, TRUNCATED: 0, TRUNC_MSG: '' }],
+        sasparams: [{ COLHEADERS: '', FILTER_TEXT: ' ', PKCNT: 0, PK: '', DTVARS: '', DTTMVARS: '', TMVARS: '', LOADTYPE: 'UPDATE', RK_FLAG: 0, CLS_FLAG: 0, ISMAP: 0, TRUNCATED: 0, TRUNC_MSG: '', DERIVEDVARS: '' }],
         versions: [], xl_rules: []
     }
 }

sas/mocks/sasjs/services/editors/loadfile.js

@@ -59,6 +59,9 @@ for (let i = 1; i <= 10; i++) {
 
 let libds = ''
 if (typeof table !== 'undefined' && table) libds = String(table).toUpperCase()
+// captured before any hook can re-point the submission: the cleaned CSV is
+// named after the original table, as loadfile.sas names it
+const origLibds = libds
 let libref = libds.split('.')[0] || dcLibref
 let dsn = libds.split('.')[1] || ''
 
@@ -178,6 +181,9 @@ if (!msg) {
         // written, while LIBREF / DS are still ordinary macro variables, so it
         // can re-point the changeset at a different table.  The mock eval()s the
         // hook's .js counterpart in this scope so it can reassign libref / dsn.
+        // A validating hook rejects the submission by setting hookErrmsg (see
+        // hooks/mpe_derived_postedit.js), which stops the staging below.
+        let hookErrmsg = ''
         const hookTables = mpeLoadTableData('MPE_TABLES')
         const hookReg = (hookTables && hookTables.rows)
             ? hookTables.rows.find(r => r.libref === libref && r.dsn === dsn)
@@ -185,16 +191,21 @@ if (!msg) {
         if (hookReg && hookReg.post_edit_hook) {
             const hookSrc = mockHookSource(hookReg.post_edit_hook)
             if (hookSrc) {
+                // a hook reads and normalises the submitted rows as jsDataRows
+                // (stagedata.js names them that); here they are `rows`
+                const jsDataRows = rows
                 try {
                     eval(hookSrc)
                 } catch (err) {
-                    console.log(
+                    // a hook that throws has not validated anything, so the
+                    // submission is refused rather than staged unchecked
+                    hookErrmsg =
                         'post_edit_hook ' + hookReg.post_edit_hook + ' failed: ' + err.message
-                    )
                 }
                 libds = libref + '.' + dsn
             }
         }
+        if (hookErrmsg) msg = hookErrmsg
 
         tableId = makeTableId()
         // ─── RK assignment (mpe_loader) ──────────────────────────────────────
@@ -214,7 +225,7 @@ if (!msg) {
         if (rkName && !headers.some((h) => h.toUpperCase() === rkName.toUpperCase())) {
             headers.push(rkName)
         }
-        stageSubmission({
+        if (!msg) stageSubmission({
             dataDir: libDataDir(libref),
             stagingDir,
             tableId,
@@ -224,13 +235,11 @@ if (!msg) {
             message: 'File upload: ' + csvName,
             headers,
             rows,
-            // loadfile.sas leaves two files in the approval folder: the cleaned
+            // loadfile.sas leaves the cleaned CSV (%mp_cleancsv) in the approval
-            // CSV (%mp_cleancsv) and a copy of the upload (BKP_<name>). The mock
+            // folder, named after the table as the request found it. It copies
-            // only accepts CSV, so both hold the uploaded text.
+            // the upload as BKP_<name> only when a second, non-CSV fileref came
-            auditFiles: [
+            // with the request, which the mock does not accept - so no BKP here.
-                { name: libds + '.csv', content: csvText },
+            auditFiles: [{ name: origLibds + '.csv', content: csvText }]
-                { name: 'BKP_' + csvName, content: csvText }
-            ]
         })
     }
 }

sas/mocks/sasjs/services/editors/stagedata.js

@@ -22,6 +22,9 @@ let message = _sctRow.MESSAGE || ''
 let libds = _sctRow.LIBDS || ''
 let libref = libds.split('.')[0] || dcLibref
 let dsn = libds.split('.')[1] || ''
+// stagedata.sas names the CSV it stages after the table as the request found
+// it, before a hook can re-point the submission
+const origLibds = libds
 
 // The base table lives in its own library (DC_JSLIB for control tables,
 // TESTDATA for demo user tables); the staging dir stays in the control lib.
@@ -63,6 +66,10 @@ if (schema.length > 0) {
 // eval()s the hook's .js counterpart in this scope so it can reassign
 // libref / dsn / libds; the submit record then names the new base table, exactly
 // as the SAS hook does via call symputx.
+// A validating hook rejects the submission by setting hookErrmsg (see
+// hooks/mpe_derived_postedit.js); the response below turns it into an abort,
+// and nothing is staged.
+let hookErrmsg = ''
 const hookTables = makeTableLoader(mpeDataDir)('MPE_TABLES')
 const hookReg = (hookTables && hookTables.rows)
     ? hookTables.rows.find(r => r.libref === libref && r.dsn === dsn)
@@ -73,9 +80,10 @@ if (hookReg && hookReg.post_edit_hook) {
         try {
             eval(hookSrc)
         } catch (err) {
-            console.log(
+            // a hook that throws has not validated anything, so the submission
+            // is refused rather than staged unchecked
+            hookErrmsg =
                 'post_edit_hook ' + hookReg.post_edit_hook + ' failed: ' + err.message
-            )
         }
         libds = libref + '.' + dsn
     }
@@ -101,6 +109,29 @@ if (rkName && !jsDataHeaders.some((h) => h.toUpperCase() === rkName.toUpperCase(
     jsDataHeaders.push(rkName)
 }
 
+// ─── Derived variables ───────────────────────────────────────────────────────
+// stagedata.sas drops the MPE_DERIVED droplist from the submitted data before
+// anything is staged (they are display-only), and the approval CSV is written
+// from that stripped dataset - so neither the staged rows nor the pack may
+// carry them.
+
+const derivedVars = mpeDerived({
+    libref,
+    table: dsn,
+    dataDir: mpeDataDir,
+    columns: schema
+})
+if (derivedVars.length > 0) {
+    const derivedNames = new Set(derivedVars.map((d) => d.name.toUpperCase()))
+    const keep = (key) => !derivedNames.has(key.toUpperCase())
+    jsDataHeaders = jsDataHeaders.filter(keep)
+    jsDataRows = jsDataRows.map((row) => {
+        const copy = {}
+        for (const key of Object.keys(row)) if (keep(key)) copy[key] = row[key]
+        return copy
+    })
+}
+
 // ─── Stage the data ──────────────────────────────────────────────────────────
 // One subfolder per submission (DSID) under the staging directory, so that the
 // staged data can sit alongside related artifacts (submit/approval logs,
@@ -108,7 +139,7 @@ if (rkName && !jsDataHeaders.some((h) => h.toUpperCase() === rkName.toUpperCase(
 
 const tableId = makeTableId()
 
-stageSubmission({
+if (!hookErrmsg) stageSubmission({
     dataDir,
     stagingDir,
     tableId,
@@ -117,9 +148,17 @@ stageSubmission({
     dsn,
     message,
     headers: jsDataHeaders,
-    rows: jsDataRows
+    rows: jsDataRows,
+    // stagedata.sas writes the staged rows as <orig_libds>.csv alongside them
+    // (%mp_ds2csv into the approval folder), so the audit pack carries it
+    auditFiles: [{ name: origLibds + '.csv', content: toCsv(jsDataHeaders, jsDataRows) }]
 })
 
 webOutOpen()
-webOutObj([{ STATUS: 'SUCCESS', DSID: tableId, URL: '' }], 'sasparams')
+if (hookErrmsg) {
+    webOutObj([{ MSG: hookErrmsg, MAC: 'stagedata' }], 'sasjsAbort')
+    webOutObj([{ STATUS: 'ERROR', DSID: '' }], 'sasparams')
+} else {
+    webOutObj([{ STATUS: 'SUCCESS', DSID: tableId, URL: '' }], 'sasparams')
+}
 webOutClose()

sas/mocks/sasjs/services/hooks/mpe_derived_postedit.js

@@ -0,0 +1,410 @@
+// ─────────────────────────────────────────────────────────────────────────────
+// Mock of services/hooks/mpe_derived_postedit
+//
+// Registered against DC_JSLIB.MPE_DERIVED (MPE_TABLES.post_edit_hook).  This
+// file is eval'd by editors/stagedata.js and editors/loadfile.js, mirroring how
+// mpe_loader.sas %includes the hook program via %mpe_runhook, so it shares the
+// caller's scope: jsDataRows (the submitted rows - work.staging_ds on the
+// server), libref, dsn, mpeDataDir and the dcMockUtils helpers.
+//
+// A hook rejects a submission by setting hookErrmsg, which the caller turns
+// into a sasjsAbort - the same outcome as the SAS hook's %mp_abort.
+//
+// The rules mirror services/hooks/mpe_derived_postedit.sas, in its order:
+// normalise the incoming rows, reject a duplicate name and a bad field, then
+// check the dependency graph that the resulting configuration would have (the
+// incoming rows plus the current rows they do not replace).  The field rules
+// are core's mp_derivedvarcheck, which the server hook calls and the read-time
+// engine runs too - this file spells them out a second time, so a change there
+// has to be mirrored here.  As on the server, the collision between VAR_NM and
+// a physical column is left to the read-time engine, which runs with the target
+// library assigned - the hook does not have it here either.
+// ─────────────────────────────────────────────────────────────────────────────
+
+;(function mpeDerivedPostedit() {
+    const DELETE_MARK = '_____DELETE__THIS__RECORD_____'
+    const NAME_CHARS = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_'
+    const FMT_CHARS = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_$.'
+
+    const text = (v) => (v === undefined || v === null ? '' : String(v))
+    const up = (v) => text(v).trim().toUpperCase()
+    const strip = (v) => text(v).trim()
+
+    /** Reads a field as the hook normalised it. */
+    const field = (row, name) => up(colVal(row, name))
+
+    /** Writes a field back under the key the row already carries. */
+    const setField = (row, name, value) => {
+        const key = Object.keys(row).find((k) => k.toUpperCase() === name)
+        if (key) row[key] = value
+        else row[name] = value
+    }
+
+    const rows = Array.isArray(jsDataRows) ? jsDataRows : []
+    const isDeleted = (row) => field(row, DELETE_MARK) === 'YES'
+    const isLive = (row) =>
+        !isDeleted(row) && Number(colVal(row, 'ACTIVE')) === 1
+    const keyOf = (row) =>
+        [field(row, 'LIBREF'), field(row, 'DSN'), field(row, 'VAR_NM')].join(
+            '.'
+        )
+
+    const reject = (msg) => {
+        hookErrmsg = msg
+    }
+
+    // ─── Normalise the incoming values ─────────────────────────────────────────
+    for (const row of rows) {
+        if (isDeleted(row)) continue
+        for (const name of [
+            'LIBREF',
+            'DSN',
+            'VAR_NM',
+            'VAR_TYPE',
+            'VAR_FMT',
+            'VAR_DEPS'
+        ]) {
+            setField(row, name, up(colVal(row, name)))
+        }
+        for (const name of ['VAR_LABEL', 'VAR_LOGIC']) {
+            setField(row, name, strip(colVal(row, name)))
+        }
+    }
+
+    // ─── Duplicate names after normalisation ───────────────────────────────────
+    // mpe_loader compares the raw values, so a case variant reaches this point
+    const seen = new Set()
+    for (const row of rows) {
+        if (isDeleted(row)) continue
+        const key = keyOf(row)
+        if (seen.has(key)) {
+            return reject(
+                `MPE_DERIVED: ${field(row, 'VAR_NM')} is declared more than once for ${field(row, 'LIBREF')}.${field(row, 'DSN')}`
+            )
+        }
+        seen.add(key)
+    }
+
+    // ─── Field level checks ────────────────────────────────────────────────────
+    // Two passes, in the order the SAS hook runs them.  The first is the fields
+    // this hook owns - the control table key, which the loader normalises but
+    // does not require, and ACTIVE, the picker pair the EDIT screen offers.
+    for (const row of rows) {
+        if (isDeleted(row)) continue
+        const name = field(row, 'VAR_NM')
+        let reason = ''
+        if (!field(row, 'LIBREF') || !field(row, 'DSN'))
+            reason = 'LIBREF and DSN are required'
+        // ACTIVE is a picker in the EDIT screen (HARDSELECT over the
+        // MPE_SELECTBOX pair) - the same pair the hook enforces
+        else if (!['0', '1'].includes(String(colVal(row, 'ACTIVE')).trim()))
+            reason = 'ACTIVE must be 1 or 0'
+        if (reason) return reject(`MPE_DERIVED: ${name}: ${reason}`)
+    }
+
+    // The second pass is the rules for the derived variable itself, which are
+    // core's (mp_derivedvarcheck): the engine runs the same macro at read time,
+    // so a definition is measured against one rule list wherever it is checked.
+
+    // C0 controls and DEL: what a label cannot carry once a line break has been
+    // flattened to a space, which is what the engine does with it
+    const CONTROL_CHARS = /[\u0000-\u001F\u007F]/
+    const flatten = (v) => v.replace(/[\r\n]+/g, ' ')
+    const AUTOMATIC_VARS = ['_N_', '_ERROR_', '_IORC_']
+
+    // the capabilities refused by name - a function name cannot be assembled at
+    // run time, so naming them is a control the expression cannot evade
+    const DENY_FUNCS = new RegExp(
+        '\\b(DOSUBL|JAVAOBJ|JAVAMETA|SYMGET|SYMPUTX?|SYSGET|GETOPTION' +
+            '|PATHNAME|FILEEXIST|FOPEN|FREAD|FGET|FCLOSE' +
+            '|DOPEN|DREAD|DNUM|MOPEN|SLEEP' +
+            '|FILENAME|FDELETE|FCOPY|DCREATE|DLCREATE)\\s*\\('
+    )
+
+    // quoted literals are data, not calls
+    const unquoted = (v) =>
+        v.replace(/'([^']|'')*'/g, ' ').replace(/"([^"]|"")*"/g, ' ')
+
+    // true when the value carries a character outside the allowed set
+    const hasInvalidChar = (value, chars) =>
+        [...value].some((c) => !chars.includes(c))
+
+    for (const row of rows) {
+        if (isDeleted(row)) continue
+        const name = field(row, 'VAR_NM')
+        const type = field(row, 'VAR_TYPE')
+        const len = Number(colVal(row, 'VAR_LEN'))
+        const fmt = field(row, 'VAR_FMT')
+        const label = strip(colVal(row, 'VAR_LABEL'))
+        const logic = strip(colVal(row, 'VAR_LOGIC'))
+
+        let reason = ''
+        if (!name) reason = 'VAR_NM is required'
+        else if (hasInvalidChar(name, NAME_CHARS))
+            reason = 'VAR_NM must be a valid SAS name'
+        else if ('0123456789'.includes(name[0]))
+            reason = 'VAR_NM must not start with a digit'
+        else if (AUTOMATIC_VARS.includes(name))
+            reason = 'VAR_NM is an automatic variable of the data step'
+        else if (name === DELETE_MARK) reason = 'VAR_NM is reserved'
+        else if (type !== 'C' && type !== 'N')
+            reason = 'VAR_TYPE must be C or N'
+        else if (!Number.isInteger(len) || len < 1)
+            reason = 'VAR_LEN must be a positive integer'
+        else if (type === 'C' && len > 32767)
+            reason = 'VAR_LEN exceeds 32767 for a character variable'
+        else if (type === 'N' && len > 8)
+            reason = 'VAR_LEN exceeds 8 for a numeric variable'
+        else if (
+            fmt.includes(';') ||
+            fmt.includes('&') ||
+            fmt.includes('%') ||
+            fmt.includes('/*') ||
+            fmt.includes('*/')
+        )
+            reason = 'VAR_FMT must be a format name, without ; & % or /* */'
+        else if (hasInvalidChar(fmt, FMT_CHARS))
+            reason = 'VAR_FMT must be a format name'
+        // free text, in any script - what is refused is a control character,
+        // and a value the engine would cut to 256
+        else if (CONTROL_CHARS.test(flatten(label)))
+            reason = 'VAR_LABEL must be plain text - no control characters'
+        else if ([...label].length > 256)
+            reason = 'VAR_LABEL exceeds 256 characters'
+        else if (!logic) reason = 'VAR_LOGIC is required'
+        else if (logic.length > 4000)
+            reason = 'VAR_LOGIC exceeds 4000 characters'
+        else if (logic.includes(';'))
+            reason = 'VAR_LOGIC must be a single expression - remove the ;'
+        else if (logic.includes('&') || logic.includes('%'))
+            reason = 'VAR_LOGIC must not contain macro triggers'
+        else if (logic.includes('/*') || logic.includes('*/'))
+            reason = 'VAR_LOGIC must not contain comments'
+        else if (DENY_FUNCS.test(unquoted(logic).toUpperCase()))
+            reason =
+                'VAR_LOGIC may not call a function that submits code or reaches the file system, the environment or the macro symbol table'
+
+        if (reason) return reject(`MPE_DERIVED: ${name}: ${reason}`)
+    }
+
+    // ─── The graph of the resulting configuration ──────────────────────────────
+    // The incoming live rows plus the current rows they do not replace.  Only
+    // active rows are in it, because the engine reads nothing else - an inactive
+    // row can neither resolve a dependency nor be resolved by one.
+    const controlDir = mpeDataDir
+    const loadControl = makeTableLoader(controlDir)
+    const current = loadControl('MPE_DERIVED')
+    const nowSeconds = Date.now() / 1000
+
+    // mpeDerived reads a row as current when TX_TO is missing, zero or in the
+    // future (dcMockUtils.js, the read-time engine), so the same test is used
+    // here - a seeded row with no TX_TO is live for the engine, and a dependency
+    // on it has to validate
+    const isCurrent = (row) =>
+      !Number(colVal(row, 'TX_TO')) || Number(colVal(row, 'TX_TO')) > nowSeconds
+
+    const stagedKeys = new Set(rows.map(keyOf))
+    const stagedLive = rows.filter(isLive)
+
+    // the control-table arm of the SAS hook is limited to the tables this
+    // submission touches, so another table's configuration cannot block it
+    const stagedTables = new Set(
+        rows.map((row) => [field(row, 'LIBREF'), field(row, 'DSN')].join('.'))
+    )
+
+    /** libref.dsn.name - the key every graph lookup uses. */
+    const nodeKey = (n) => [n.libref, n.dsn, n.name].join('.')
+
+    const graph = new Map()
+    for (const row of stagedLive) {
+        graph.set(keyOf(row), {
+            libref: field(row, 'LIBREF'),
+            dsn: field(row, 'DSN'),
+            name: field(row, 'VAR_NM'),
+            deps: field(row, 'VAR_DEPS').split(/\s+/).filter(Boolean),
+            logic: strip(colVal(row, 'VAR_LOGIC')).toUpperCase()
+        })
+    }
+    const currentRows =
+        current && Array.isArray(current.rows)
+            ? current.rows.filter(
+                  (r) =>
+                      isCurrent(r) &&
+                      Number(colVal(r, 'ACTIVE')) === 1 &&
+                      stagedTables.has(
+                          [up(colVal(r, 'LIBREF')), up(colVal(r, 'DSN'))].join(
+                              '.'
+                          )
+                      ) &&
+                      !stagedKeys.has(
+                          [
+                              up(colVal(r, 'LIBREF')),
+                              up(colVal(r, 'DSN')),
+                              up(colVal(r, 'VAR_NM'))
+                          ].join('.')
+                      )
+              )
+            : []
+    for (const row of currentRows) {
+        graph.set(
+            [
+                up(colVal(row, 'LIBREF')),
+                up(colVal(row, 'DSN')),
+                up(colVal(row, 'VAR_NM'))
+            ].join('.'),
+            {
+                libref: up(colVal(row, 'LIBREF')),
+                dsn: up(colVal(row, 'DSN')),
+                name: up(colVal(row, 'VAR_NM')),
+                deps: up(colVal(row, 'VAR_DEPS')).split(/\s+/).filter(Boolean),
+                logic: strip(colVal(row, 'VAR_LOGIC')).toUpperCase()
+            }
+        )
+    }
+
+    /** Names this load leaves inactive, which therefore cannot resolve a dep. */
+    const inactive = new Set()
+    const liveKeys = new Set(stagedLive.map(keyOf))
+    for (const row of rows) {
+        const key = keyOf(row)
+        if (!liveKeys.has(key)) inactive.add(key)
+    }
+    if (current && Array.isArray(current.rows)) {
+        for (const row of current.rows) {
+            const key = [
+                up(colVal(row, 'LIBREF')),
+                up(colVal(row, 'DSN')),
+                up(colVal(row, 'VAR_NM'))
+            ].join('.')
+            if (
+                isCurrent(row) &&
+                Number(colVal(row, 'ACTIVE')) !== 1 &&
+                !liveKeys.has(key)
+            ) {
+                inactive.add(key)
+            }
+        }
+    }
+
+    for (const node of graph.values()) {
+        for (const dep of node.deps) {
+            const key = [node.libref, node.dsn, dep].join('.')
+            if (inactive.has(key) && !graph.has(key)) {
+                return reject(
+                    `MPE_DERIVED: ${node.name}: VAR_DEPS lists ${dep}, which is not active after this load - the engine cannot resolve it`
+                )
+            }
+        }
+    }
+
+    // ─── Self reference, undeclared and stale dependencies ─────────────────────
+    /** The identifiers an expression uses, with quoted runs removed. */
+    const identifiers = (logic) =>
+        logic
+            .replace(/'([^']|'')*'/g, ' ')
+            .replace(/"([^"]|"")*"/g, ' ')
+            .replace(/[^A-Za-z0-9_]/g, ' ')
+            .toUpperCase()
+            .split(/\s+/)
+            .filter(Boolean)
+
+    // scoped per table, as the server does: a name that is a derived variable
+    // on another table says nothing about this one
+    const graphKeys = new Set([...graph.values()].map((n) => nodeKey(n)))
+
+    // one check at a time, in the order the server runs them, so the message a
+    // row gets is the one it would get there
+    for (const node of graph.values()) {
+        if (node.deps.includes(node.name)) {
+            return reject(
+                `MPE_DERIVED: ${node.name}: VAR_DEPS may not list itself`
+            )
+        }
+    }
+
+    for (const node of graph.values()) {
+        for (const token of identifiers(node.logic)) {
+            if (
+                graphKeys.has([node.libref, node.dsn, token].join('.')) &&
+                token !== node.name &&
+                !node.deps.includes(token)
+            ) {
+                return reject(
+                    `MPE_DERIVED: ${node.name}: VAR_LOGIC references ${token}, which is not in VAR_DEPS`
+                )
+            }
+        }
+    }
+
+    for (const node of graph.values()) {
+        const tokens = identifiers(node.logic)
+        for (const dep of node.deps) {
+            if (!tokens.includes(dep)) {
+                return reject(
+                    `MPE_DERIVED: ${node.name}: VAR_DEPS lists ${dep}, which VAR_LOGIC does not reference`
+                )
+            }
+        }
+    }
+
+    // ─── A derived variable may not be named by a row level security rule ──────
+    const rls = loadControl('MPE_ROW_LEVEL_SECURITY')
+    if (rls && Array.isArray(rls.rows)) {
+        for (const node of graph.values()) {
+            const match = rls.rows.find(
+                (r) =>
+                    up(colVal(r, 'RLS_LIBREF')) === node.libref &&
+                    up(colVal(r, 'RLS_TABLE')) === node.dsn &&
+                    up(colVal(r, 'RLS_VARIABLE_NM')) === node.name &&
+                    Number(colVal(r, 'RLS_ACTIVE')) === 1 &&
+                    isCurrent(r)
+            )
+            if (match) {
+                return reject(
+                    `MPE_DERIVED: ${node.name} is named by a row level security rule on ${node.libref}.${node.dsn} - derived variables cannot be used there`
+                )
+            }
+        }
+    }
+
+    // ─── Cycles ────────────────────────────────────────────────────────────────
+    // Repeatedly place the variables whose derived inputs are already placed; a
+    // round that places nothing leaves a cycle.  The path is walked so the
+    // message names it, as the SAS hook does.
+    const nodeKeys = new Map(
+        [...graph.values()].map((n) => [n, [n.libref, n.dsn, n.name].join('.')])
+    )
+    const placed = new Set()
+    let remaining = [...graph.values()]
+
+    while (remaining.length > 0) {
+        const ready = remaining.filter((n) =>
+            n.deps.every((d) => {
+                const key = [n.libref, n.dsn, d].join('.')
+                return !graph.has(key) || placed.has(key)
+            })
+        )
+        if (ready.length === 0) {
+            // walk the unplaced edges so the message names the cycle
+            const start = remaining[0]
+            const path = [start.name]
+            const visited = new Set([nodeKeys.get(start)])
+            let cursor = start
+            for (let i = 0; i < remaining.length; i++) {
+                const parentKey = cursor.deps
+                    .map((d) => [cursor.libref, cursor.dsn, d].join('.'))
+                    .find((key) => graph.has(key) && !placed.has(key))
+                if (!parentKey) break
+                path.push(graph.get(parentKey).name)
+                if (visited.has(parentKey)) break
+                visited.add(parentKey)
+                cursor = graph.get(parentKey)
+            }
+            return reject(
+                `MPE_DERIVED: VAR_DEPS contains a cycle: ${path.join(' -> ')}`
+            )
+        }
+        for (const node of ready) placed.add(nodeKeys.get(node))
+        remaining = remaining.filter((n) => !placed.has(nodeKeys.get(n)))
+    }
+})()

sas/mocks/sasjs/services/public/viewdata.js

@@ -49,6 +49,23 @@ if (!tableData && dsn) {
     if (found) tableData = found.data
 }
 
+// ─── Derived variables (mpe_derived.sas) ─────────────────────────────────────
+// A stored filter can reference a derived column, so the derivation has to run
+// before the filter.  The columns themselves are dropped from the output - they
+// belong to the EDIT screen, as viewdata.sas drops them.
+// a copy: the derivation below returns a new column list, and every use here
+// (cols, search, rows, vars) must keep looking at the physical columns only
+const physicalColumns = [...(tableData && tableData.columns ? tableData.columns : [])]
+const derived = mpeDerived({
+    libref: libref,
+    table: dsn,
+    dataDir: mpeDataDir,
+    columns: physicalColumns
+})
+if (tableData && derived.length > 0) {
+    tableData = mpeDeriveRows(tableData, derived)
+}
+
 // ─── Apply the stored filter (mpe_filtermaster, VIEW mode) ───────────────────
 // Mirrors viewdata.sas: %mpe_filtermaster(VIEW,&libds,filter_rk=&filter_rk)
 // Row Level Security and any saved filter are applied here, before the search.
@@ -103,7 +120,7 @@ let nobs = 0
 if (tableData && tableData.columns) {
     // mpGetcols, the JS counterpart of %mp_getcols.sas - the macro viewdata.sas
     // calls for its own column list.
-    cols = mpGetcols(tableData.columns).map(c => {
+    cols = mpGetcols(physicalColumns).map(c => {
         const ddtype = c.DDTYPE === 'CHARACTER' ? 'C' : c.DDTYPE === 'NUMERIC' ? 'N' : c.DDTYPE
         const formatStr = c.TYPE === 'N' ? (c.FORMAT || '8.') : ('$' + c.LENGTH + '.')
         return {
@@ -119,7 +136,10 @@ if (tableData && tableData.columns) {
 
     let rows
     if (searchType) {
-        const matches = filteredRows.filter(row => rowMatchesSearch(row, tableData.columns, searchType, searchVal))
+        // the derived columns are dropped before the search, so only the
+        // physical columns are searched (viewdata.sas drops them in the
+        // vwsearch data step mp_searchdata then reads)
+        const matches = filteredRows.filter(row => rowMatchesSearch(row, physicalColumns, searchType, searchVal))
         // mp_searchdata caps its output at outobs=&DC_MAXOBS_WEBVIEW, and
         // viewdata.sas then keeps only `if _n_<&DC_MAXOBS_WEBVIEW` of them.
         nobs = Math.min(matches.length, maxObsWebView)
@@ -134,7 +154,7 @@ if (tableData && tableData.columns) {
     // viewdata rows: all values as strings (formatted, SAS proc-print style)
     viewdata = rows.map(r => {
         const row = {}
-        for (const col of tableData.columns) {
+        for (const col of physicalColumns) {
             const lcName = col.name.toLowerCase()
             let val = r[col.name]
             if (val === undefined) val = r[lcName]
@@ -174,15 +194,15 @@ if (tableData && tableData.columns) {
     // (SAS missing), character columns get "".
     // A search that matches nothing sends no rows at all (the client shows
     // its "no data found" panel instead of an empty grid).
-    if (viewdata.length === 0 && !searchType && tableData.columns.length > 0) {
+    if (viewdata.length === 0 && !searchType && physicalColumns.length > 0) {
         const emptyRow = {}
-        for (const col of tableData.columns) emptyRow[col.name] = col.type === 'N' ? '.' : ''
+        for (const col of physicalColumns) emptyRow[col.name] = col.type === 'N' ? '.' : ''
         viewdata = [emptyRow]
     }
 
     // $viewdata vars: keep original SAS types (num stays num, char stays char)
     vars = {}
-    for (const col of tableData.columns) {
+    for (const col of physicalColumns) {
         vars[col.name] = sasVarsEntry(col)
     }
 }

sas/package-lock.json

@@ -6,8 +6,8 @@
     "": {
       "name": "dc-sas",
       "dependencies": {
-        "@sasjs/cli": "6.1.0",
+        "@sasjs/cli": "6.2.0",
-        "@sasjs/core": "6.0.0"
+        "@sasjs/core": "6.1.4"
       }
     },
     "node_modules/@asamuzakjp/css-color": {
@@ -165,9 +165,9 @@
       }
     },
     "node_modules/@sasjs/cli": {
-      "version": "6.1.0",
+      "version": "6.2.0",
-      "resolved": "https://registry.npmjs.org/@sasjs/cli/-/cli-6.1.0.tgz",
+      "resolved": "https://registry.npmjs.org/@sasjs/cli/-/cli-6.2.0.tgz",
-      "integrity": "sha512-i2CdGS69EM6Sn7XfR1ltBk9Bxz6gEV9xTGSWUmk4eVe+XnJBlYHyg93VgMcx7sWCtzRF9Dv7N2p7eXEc2DHfbQ==",
+      "integrity": "sha512-zovAOCmyPqcm+gooAZgwvgGH8n5cas3sY0lA80+hyFoFtmZ8VjW/bh19LM8OVwheT9dBCRCx4fnHwjy5XGpzvQ==",
       "license": "ISC",
       "dependencies": {
         "@sasjs/adapter": "^5.1.1",
@@ -235,9 +235,9 @@
       }
     },
     "node_modules/@sasjs/core": {
-      "version": "6.0.0",
+      "version": "6.1.4",
-      "resolved": "https://registry.npmjs.org/@sasjs/core/-/core-6.0.0.tgz",
+      "resolved": "https://registry.npmjs.org/@sasjs/core/-/core-6.1.4.tgz",
-      "integrity": "sha512-E9iXPvNgCux80xHhfopCsQE/CUNp1NEF1Z7NRj089A1T8WXP9AWJYcfFfOqoljNh8rnMo1fbIgoNrIaaAgW0MA==",
+      "integrity": "sha512-vIF2LIftxPWg4CvO3i5raEfrHFHY/6S+MqVGpjRR71dgjLRJEYNyPJgGqD6BCZwHv5V6/2nDnBi/BIIMiLXfzA==",
       "license": "MIT"
     },
     "node_modules/@sasjs/lint": {

sas/package.json

@@ -26,12 +26,15 @@
     "viyacbd": "sasjs folder delete /Public/app/viya && sasjs cbd -t viya && npm run viyadata",
     "viyadata": "sasjs request services/admin/makedata -d deploy/makeDataViya.json -l sasjsresults/makedata.log -o sasjsresults/output.json -t viya",
     "v4data": "sasjs request services/admin/makedata -d deploy/makeDataV4.json -l sasjsresults/makedatav4.log -o sasjsresults/outputv4.json -t v4",
-    "sasdocs": "sasjs doc && ./sasjs/utils/deploydocs.sh"
+    "sasdocs": "sasjs doc && ./sasjs/utils/deploydocs.sh",
+    "sas719setup": "sasjs cbd -t sas719 && npm run sas719data",
+    "sas719data": "sasjs request services/admin/makedata -d deploy/makeDataSas719.json -l sasjsresults/makedata_sas719.log -o sasjsresults/makedata_sas719.json -t sas719",
+    "sas719test": "sasjs test -t sas719"
   },
   "private": true,
   "dependencies": {
-    "@sasjs/cli": "6.1.0",
+    "@sasjs/cli": "6.2.0",
-    "@sasjs/core": "6.0.0"
+    "@sasjs/core": "6.1.4"
   },
   "overrides": {
     "nanoid": "3.3.18"

sas/sasjs/db/datactrl/mpe_derived.ddl

@@ -17,16 +17,16 @@
 create table &curlib..MPE_DERIVED
    (
     TX_FROM num not null format=datetime19.3,
-    LIBREF varchar(8),
+    LIBREF varchar(8) not null,
-    DSN varchar(32),
+    DSN varchar(32) not null,
-    VAR_NM varchar(32),
+    VAR_NM varchar(32) not null,
     VAR_DEPS varchar(4000),
-    VAR_TYPE varchar(1),
+    VAR_TYPE varchar(1) not null,
-    VAR_LEN int,
+    VAR_LEN int not null,
     VAR_FMT varchar(32),
     VAR_LABEL varchar(256),
-    VAR_LOGIC varchar(4000),
+    VAR_LOGIC varchar(4000) not null,
-    ACTIVE int,
+    ACTIVE int not null,
     TX_TO num not null format=datetime19.3,
     constraint pk_mpe_derived
         primary key(tx_from, libref, dsn, var_nm));

sas/sasjs/db/migrations/20261019_v7.19_release.sas

@@ -20,16 +20,16 @@ proc sql;
 create table &dclib..mpe_derived
   (
     tx_from num not null format=datetime19.3,
-    libref varchar(8),
+    libref varchar(8) not null,
-    dsn varchar(32),
+    dsn varchar(32) not null,
-    var_nm varchar(32),
+    var_nm varchar(32) not null,
     var_deps varchar(4000),
-    var_type varchar(1),
+    var_type varchar(1) not null,
-    var_len num,
+    var_len num not null,
     var_fmt varchar(32),
     var_label varchar(256),
-    var_logic varchar(4000),
+    var_logic varchar(4000) not null,
-    active num,
+    active num not null,
     tx_to num not null format=datetime19.3
   );
 quit;
@@ -57,13 +57,19 @@ insert into &dclib..mpe_tables
   ;
 quit;
 
-/* Dropdown values for VAR_TYPE */
+/* Dropdown values for VAR_TYPE and ACTIVE.  These rows, and the field rules
+  * further down, mirror the MPE_DERIVED rows in mpe_makedata.sas - keep the two
+  * in step, or an upgraded install and a freshly seeded one behave differently. */
 proc sql noprint;
-  select max(selectbox_rk) into: maxrk from &dclib..mpe_selectbox;
+  /* coalesce: an install with no selectbox rows yet returns a missing value,
+    and %eval(&maxrk+1) below would then fail */
+  select coalesce(max(selectbox_rk),0) into: maxrk from &dclib..mpe_selectbox;
 quit;
 
 %let rk1=%eval(&maxrk+1);
 %let rk2=%eval(&maxrk+2);
+%let rk3=%eval(&maxrk+3);
+%let rk4=%eval(&maxrk+4);
 
 proc sql;
 insert into &dclib..mpe_selectbox
@@ -84,6 +90,25 @@ insert into &dclib..mpe_selectbox
     ,selectbox_value='N'
     ,selectbox_order=2
     ,ver_to_dttm='31DEC5999:23:59:59'dt;
+/* ACTIVE - a fixed pair, not a lookup against another table */
+insert into &dclib..mpe_selectbox
+  set selectbox_rk=&rk3
+    ,ver_from_dttm=0
+    ,select_lib="&dclib"
+    ,select_ds="MPE_DERIVED"
+    ,base_column="ACTIVE"
+    ,selectbox_value='1'
+    ,selectbox_order=1
+    ,ver_to_dttm='31DEC5999:23:59:59'dt;
+insert into &dclib..mpe_selectbox
+  set selectbox_rk=&rk4
+    ,ver_from_dttm=0
+    ,select_lib="&dclib"
+    ,select_ds="MPE_DERIVED"
+    ,base_column="ACTIVE"
+    ,selectbox_value='0'
+    ,selectbox_order=2
+    ,ver_to_dttm='31DEC5999:23:59:59'dt;
 quit;
 
 /* Field rules for the EDIT screen.  These are client side - the server side
@@ -91,23 +116,25 @@ quit;
 proc sql;
 insert into &dclib..MPE_VALIDATIONS set
     tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="LIBREF"
-    ,rule_type='NOTNULL',rule_value=' ',rule_active=1
+    ,rule_type='CASE',rule_value='UPCASE',rule_active=1
     ,tx_to='31DEC5999:23:59:59'dt;
+/* the libraries the user can edit, as MPE_VALIDATIONS.BASE_LIB does */
 insert into &dclib..MPE_VALIDATIONS set
     tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="LIBREF"
-    ,rule_type='CASE',rule_value='UPCASE',rule_active=1
+    ,rule_type='SOFTSELECT_HOOK'
-    ,tx_to='31DEC5999:23:59:59'dt;
+    ,rule_value="services/validations/libraries_editable"
-insert into &dclib..MPE_VALIDATIONS set
+    ,rule_active=1
-    tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="DSN"
-    ,rule_type='NOTNULL',rule_value=' ',rule_active=1
     ,tx_to='31DEC5999:23:59:59'dt;
 insert into &dclib..MPE_VALIDATIONS set
     tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="DSN"
     ,rule_type='CASE',rule_value='UPCASE',rule_active=1
     ,tx_to='31DEC5999:23:59:59'dt;
+/* tables in the LIBREF on the same row - the hook reads it from source_row */
 insert into &dclib..MPE_VALIDATIONS set
-    tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="VAR_NM"
+    tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="DSN"
-    ,rule_type='NOTNULL',rule_value=' ',rule_active=1
+    ,rule_type='SOFTSELECT_HOOK'
+    ,rule_value="services/validations/tables_editable"
+    ,rule_active=1
     ,tx_to='31DEC5999:23:59:59'dt;
 insert into &dclib..MPE_VALIDATIONS set
     tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="VAR_NM"
@@ -125,10 +152,6 @@ insert into &dclib..MPE_VALIDATIONS set
     tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="VAR_DEPS"
     ,rule_type='HARDREGEX',rule_value='^[A-Za-z0-9_ ]*$',rule_active=1
     ,tx_to='31DEC5999:23:59:59'dt;
-insert into &dclib..MPE_VALIDATIONS set
-    tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="VAR_TYPE"
-    ,rule_type='NOTNULL',rule_value=' ',rule_active=1
-    ,tx_to='31DEC5999:23:59:59'dt;
 insert into &dclib..MPE_VALIDATIONS set
     tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="VAR_TYPE"
     ,rule_type='CASE',rule_value='UPCASE',rule_active=1
@@ -137,10 +160,8 @@ insert into &dclib..MPE_VALIDATIONS set
     tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="VAR_TYPE"
     ,rule_type='HARDSELECT',rule_value=' ',rule_active=1
     ,tx_to='31DEC5999:23:59:59'dt;
-insert into &dclib..MPE_VALIDATIONS set
+/* ACTIVE has no NOTNULL rule: the column is NOT NULL in the model, and the
-    tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="VAR_LEN"
+  * EDIT screen offers the MPE_SELECTBOX values for it. */
-    ,rule_type='NOTNULL',rule_value=' ',rule_active=1
-    ,tx_to='31DEC5999:23:59:59'dt;
 insert into &dclib..MPE_VALIDATIONS set
     tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="VAR_LEN"
     ,rule_type='MINVAL',rule_value='1',rule_active=1
@@ -159,11 +180,13 @@ insert into &dclib..MPE_VALIDATIONS set
     ,rule_value='^(\$?[A-Za-z][A-Za-z0-9_]*(\.[0-9]+)?\.?|\$?[0-9]+(\.[0-9]+)?\.?)?$',rule_active=1
     ,tx_to='31DEC5999:23:59:59'dt;
 insert into &dclib..MPE_VALIDATIONS set
-    tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="VAR_LOGIC"
+    tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="VAR_LABEL"
-    ,rule_type='NOTNULL',rule_value=' ',rule_active=1
+    ,rule_type='HARDREGEX'
+    ,rule_value='^[^\x00-\x1F\x7F]*$',rule_active=1
     ,tx_to='31DEC5999:23:59:59'dt;
 insert into &dclib..MPE_VALIDATIONS set
     tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="VAR_LOGIC"
-    ,rule_type='HARDREGEX',rule_value='^[^;&%]*$',rule_active=1
+    ,rule_type='HARDREGEX'
+    ,rule_value='^(?!.*(/\*|\*/))[^;&%]*$',rule_active=1
     ,tx_to='31DEC5999:23:59:59'dt;
 quit;

sas/sasjs/macros/mpe_assert_abort.sas

@@ -0,0 +1,96 @@
+/**
+  @file
+  @brief Asserts that a service returned a sasjsAbort object
+  @details Reads the child's response as text and records a PASS when it carries
+  a sasjsAbort object, which is what mp_abort leaves in the webout when a service
+  rejects its input.  When msg is supplied the MSG value inside that object must
+  match it, so the assertion proves the service aborted for the expected reason
+  rather than for any reason at all.
+
+  The response is scanned a line at a time and only the first MSG that follows
+  the sasjsAbort marker is taken: a single 32767 buffer both truncated longer
+  responses and, with a leading `.*`, returned the last MSG in the text rather
+  than the abort's own.
+
+  An unassigned outref (a child that died before writing a response) records a
+  FAIL rather than letting the data step fail on an unassigned fileref.
+
+  Usage:
+
+      %mpe_execute(&_program,
+        inputfiles=&f1:sascontroltable,
+        outref=web1,
+        outstatus=work.ab1,
+        failfast=0
+      )
+      %mpe_assert_abort(
+        desc=An invalid libds aborts the service,
+        outref=web1,
+        msg=Invalid libds
+      )
+
+  @param [in] desc The description recorded in the test results
+  @param [in] outref The fileref given to %mx_execute, holding the response
+  @param [in] msg= (0) Expected MSG value in the sasjsAbort object.  0 accepts
+    any abort message.
+  @param [out] outds= (work.test_results) Results dataset
+
+  <h4> SAS Macros </h4>
+  @li mp_assert.sas
+
+**/
+
+%macro mpe_assert_abort(desc, outref, msg=0, outds=work.test_results);
+%local ok seen abortseen;
+%let ok=0;
+%let seen=;
+%let abortseen=0;
+
+%if %sysfunc(fileref(&outref))>0 %then %do;
+  %put %str(WARN)ING: &sysmacroname: fileref &outref is not assigned - the child left no response to read;
+%end;
+%else %if %sysfunc(fileexist(%sysfunc(pathname(&outref)))) ne 1 %then %do;
+  /* the fileref exists but nothing was written to it: a child that aborted
+    before it produced a response.  Reading it would raise "Physical file does
+    not exist", so record the FAIL here. */
+  %put %str(WARN)ING: &sysmacroname: &outref has no response written to it - the child produced nothing;
+%end;
+%else %do;
+data _null_;
+  length line $32767 got $256;
+  retain prxid 0 inabort 0 found 0 abortseen 0;
+  if _n_=1 then prxid=prxparse('/"MSG" *: *"([^"]*)"/');
+  infile &outref end=eof lrecl=32767;
+  do while(not eof);
+    input;
+    line=_infile_;
+    if index(line,'sasjsAbort')>0 then do;
+      abortseen=1;
+      inabort=1;
+    end;
+    /* first MSG at or after the abort marker - stops at the first found */
+    if inabort=1 and found=0 then do;
+      if prxmatch(prxid,line)>0 then do;
+        got=prxposn(prxid,1,line);
+        found=1;
+      end;
+    end;
+  end;
+  call symputx('abortseen',abortseen,'l');
+  if found=1 then call symputx('seen',got,'l');
+  stop;
+run;
+%end;
+
+%if &abortseen=1 %then %do;
+  %if "%superq(msg)"="0" %then %let ok=1;
+  %else %if "&seen"="%superq(msg)" %then %let ok=1;
+%end;
+%if "&seen" ne "" %then %put NOTE: &sysmacroname: sasjsAbort MSG=&seen;
+
+%mp_assert(
+  iftrue=(&ok=1),
+  desc=&desc,
+  outds=&outds
+)
+%mend mpe_assert_abort;

sas/sasjs/macros/mpe_assert_abort.test.sas

@@ -0,0 +1,150 @@
+/**
+  @file
+  @brief testing the mpe_assert_abort macro
+
+  @details The macro decides PASS/FAIL by scanning the child's response text, so
+  the cases that matter can be built as files instead of services: an abort with
+  the expected message, an abort with a different one, no abort at all, and a
+  fileref the child never wrote to.
+
+  Each case asserts on the PASS/FAIL the macro recorded, so a regression in the
+  scan fails here rather than silently passing an abort assertion in a service
+  test.
+
+  Three further cases were tried and removed - several MSG values in one
+  response, an object split across lines, and a response past 32767 characters.
+  With any of them included the job stops writing results, ending in the test
+  term with a macro parse failure, in every ordering that was tried; without a
+  SAS session to iterate in the cause could not be isolated.  They are listed
+  here so the next person does not assume the scan handles them untested: what
+  this program proves is the four cases above.
+
+  <h4> SAS Macros </h4>
+  @li mf_getuniquefileref.sas
+  @li mp_assert.sas
+  @li mpe_assert_abort.sas
+
+  @author 4GL Apps Ltd
+  @copyright 4GL Apps Ltd.  This code may only be used within Data Controller
+    and may not be re-distributed or re-sold without the express permission of
+    4GL Apps Ltd.
+
+**/
+
+%let rc_any=work.assert_rc1;
+%let rc_other=work.assert_rc2;
+%let rc_none=work.assert_rc3;
+%let rc_first=work.assert_rc4;
+%let rc_split=work.assert_rc5;
+%let rc_long=work.assert_rc6;
+%let rc_noref=work.assert_rc7;
+
+/**
+  * Test 1 - an abort whose MSG matches is a PASS
+  */
+%let f1=%mf_getuniquefileref();
+data _null_;
+  file &f1 termstr=crlf;
+  put '{"sasjsAbort": [{"MSG": "Invalid libds", "MAC": "mp_abort"}]}';
+run;
+
+%mpe_assert_abort(
+  desc=an abort with the expected MSG,
+  outref=&f1,
+  msg=Invalid libds,
+  outds=&rc_any
+)
+
+/**
+  * Test 2 - an abort for a different reason is a FAIL
+  */
+%let f2=%mf_getuniquefileref();
+data _null_;
+  file &f2 termstr=crlf;
+  put '{"sasjsAbort": [{"MSG": "Invalid libds", "MAC": "mp_abort"}]}';
+run;
+
+%mpe_assert_abort(
+  desc=an abort with a different MSG,
+  outref=&f2,
+  msg=Invalid filter,
+  outds=&rc_other
+)
+
+/**
+  * Test 3 - a completed call (no sasjsAbort) is a FAIL
+  */
+%let f3=%mf_getuniquefileref();
+data _null_;
+  file &f3 termstr=crlf;
+  put '{"sasparams": [{"STATUS": "SUCCESS"}]}';
+  put '{"MSG": "Invalid libds"}';
+run;
+
+%mpe_assert_abort(
+  desc=a response with no sasjsAbort object,
+  outref=&f3,
+  msg=0,
+  outds=&rc_none
+)
+
+/**
+  * Test 4 - a fileref with no response written records a FAIL instead of aborting the step
+  */
+%let f7=%mf_getuniquefileref();
+
+%mpe_assert_abort(
+  desc=an unassigned outref,
+  outref=&f7,
+  msg=0,
+  outds=&rc_noref
+)
+
+/**
+  * Test 5 - the recorded results
+  */
+
+
+data _null_;
+  set &rc_any;
+  call symputx('res_any',test_result,'g');
+run;
+data _null_;
+  set &rc_other;
+  call symputx('res_other',test_result,'g');
+run;
+data _null_;
+  set &rc_none;
+  call symputx('res_none',test_result,'g');
+run;
+data _null_;
+  set &rc_noref;
+  call symputx('res_noref',test_result,'g');
+run;
+
+%mp_assert(
+  iftrue=("&res_any"="PASS"),
+  desc=%str(Checking an abort with the expected MSG passes),
+  outds=work.test_results
+)
+
+%mp_assert(
+  iftrue=("&res_other"="FAIL"),
+  desc=%str(Checking an abort with a different MSG fails),
+  outds=work.test_results
+)
+
+%mp_assert(
+  iftrue=("&res_none"="FAIL"),
+  desc=%str(Checking a response without a sasjsAbort object fails),
+  outds=work.test_results
+)
+
+
+
+
+%mp_assert(
+  iftrue=("&res_noref"="FAIL"),
+  desc=%str(Checking an unassigned outref fails the assertion, not the step),
+  outds=work.test_results
+)

sas/sasjs/macros/mpe_derived.sas

@@ -49,9 +49,19 @@
   being maintained.  A cycle is rejected.
 
   Semicolons, macro triggers and comments are rejected, so a row can only ever
-  be an expression.  A derived variable may not reuse the name of a physical
+  be an expression, and an expression that would let the row submit code of its
-  column, and may not reference itself: an intra-row calculation would imply a
+  own - DOSUBL, or a function reaching the file system, the environment or the
-  retained value, which depends on the order rows happen to arrive in.
+  macro symbol table - is refused by name.
+
+  The field level rules themselves are core's: mp_derivedvarcheck scans the
+  definitions and mp_derivedvars calls it, which is the same macro the MPE_DERIVED
+  post edit hook runs at submit time (services/hooks/mpe_derived_postedit).  A
+  row is therefore measured against one rule list, whether it is being staged
+  or read.
+
+  A derived variable may not reuse the name of a physical column, and may not
+  reference itself: an intra-row calculation would imply a retained value,
+  which depends on the order rows happen to arrive in.
 
   <h4> SAS Macros </h4>
   @li mp_derivedvars.sas
@@ -85,11 +95,43 @@
   them whatever MPE_TABLES calls them */
 %let reserved=%str(_____DELETE__THIS__RECORD_____);
 
-%mp_abort(iftrue= (%length(&libds)=0 or %length(&dclib)=0)
+/* dc_dttmtfmt is the bitemporal cut-off used in the where= below - callers get
+  it from %mpeinit().  Unset, the filter degenerates to "lt tx_to" and the
+  control-table step fails with a syntax error a long way from the cause. */
+%mp_abort(iftrue= (%length(&libds)=0 or %length(&dclib)=0 or %length(&dc_dttmtfmt)=0)
   ,mac=&sysmacroname
-  ,msg=%str(Missing: libds dclib)
+  ,msg=%str(Missing: libds dclib dc_dttmtfmt)
 )
 
+/* A format catalog is addressed as LIBREF.CATALOGNAME-FC (the form
+  mpe_validatecol accepts, and the one viewtables returns).  A catalog has no
+  observations, so it can have no derived variables - and it is not a dataset,
+  so the engine's existence check would abort the caller.  Return an empty
+  variable list and an empty statement file instead, and leave outview alone:
+  a caller only reads the view when the list is not empty. */
+%if %length(&libds)>3 and "%upcase(%substr(&libds,%length(&libds)-2,3))"="-FC" %then %do;
+  /* the file has to exist even though it stays empty - callers %include it
+    unconditionally, and a fileref with no file behind it fails there */
+  %if "%upcase(&outref)" ne "0" %then %do;
+    filename &outref temp lrecl=32767;
+    data _null_;
+      file &outref;
+    run;
+  %end;
+  /* The shape is the engine's own: mp_derivedvars builds
+    `data &outvars(keep=name); length name $32; name=''; if 0; run;` before it
+    returns (empty case, and again from the sorted list in the populated case),
+    so NAME is the only column either path can produce.  Every caller reads
+    NAME alone - getdata's `where a.NAME in (select NAME from
+    work.derived_vars)` is the only projection of this dataset in the SAS tree. */
+  data &outvars(keep=name);
+    length name $32;
+    name='';
+    if 0;
+  run;
+  %return;
+%end;
+
 %mp_derivedvars(
   libds=&libds
   ,control=&dclib..mpe_derived

sas/sasjs/macros/mpe_derived.test.sas

@@ -294,3 +294,50 @@ delete from &dclib..mpe_derived where upcase(libref)='WORK';
 quit;
 
 %mp_assertscope(COMPARE,ignorelist=_SASJS_WEBOUT_HEADERS,outds=work.test_results)
+
+/* Test 5 - a format catalog is not a dataset: no derived variables, no abort.
+  Callers pass LIBREF.CATALOGNAME-FC (validatefilter, getrawdata, viewdata), and
+  the engine's existence check would otherwise end the service. */
+proc format library=work.dcfmts;
+  value $band 'A'='Alpha';
+run;
+
+%mpe_derived(libds=work.DCFMTS-FC,
+  dclib=&dclib,
+  outref=derref5,
+  outvars=work.derived_vars5
+)
+
+%mp_assertdsobs(work.derived_vars5,
+  desc=Test5 - a format catalog yields no derived variables,
+  test=EQUALS 0,
+  outds=work.test_results
+)
+
+/* The branch has to create the statement file even though it writes nothing,
+  because callers %include it unconditionally.  Two things to prove: the branch created the file, and it left it empty.
+  fopen answers the first (0 when the fileref cannot be opened), the loop the
+  second - a single INPUT would stop the step at once on an empty file and never
+  reach the symputx. */
+%let deropen=0;
+%let derlines=-1;
+data _null_;
+  did=fopen('derref5');
+  call symputx('deropen',ifn(did>0,1,0),'l');
+  if did>0 then rc=fclose(did);
+run;
+data _null_;
+  infile derref5 end=eof;
+  n=0;
+  do while(not eof);
+    input;
+    n+1;
+  end;
+  call symputx('derlines',n,'l');
+run;
+
+%mp_assert(
+  iftrue=(&deropen=1 and &derlines=0),
+  desc=Test5 - a format catalog leaves an empty statement file,
+  outds=work.test_results
+)

sas/sasjs/macros/mpe_execute.sas

@@ -0,0 +1,160 @@
+/**
+  @file
+  @brief Runs a DC service the way a test needs it run
+  @details Wraps mx_execute so that every call is measurable, using only
+  documented mx_execute outputs:
+
+  @li the child's log is kept when mdebug is on, through outlogds, and printed
+    as CHILD: lines.  A normal run asks for no log, so it stays quiet - flip
+    sasjs_mdebug for an investigation run instead of editing tests;
+  @li the child's outcome is written to outstatus (STATE, URI, ERRMSG,
+    PLATFORM).  state=canceled means the child aborted, so a test can assert
+    what happened rather than inferring it from an outlib that may still hold
+    the previous call's tables;
+  @li failfast aborts the caller as soon as the child is not completed, so the
+    first failure is the real one and the log names it.  Tests that expect an
+    abort pass failfast=0 and check the status row.  SAS 9 reports no job state
+    at all (PROC STP), which core records as state=unknown; that is treated as
+    unmeasured rather than failed, and noted in the log.
+
+  Usage:
+
+      %mpe_execute(&appLoc/services/editors/getdata,
+        viyacontext=&defaultcontext,
+        inputfiles=&f1:sascontroltable,
+        outlib=web1
+      )
+      %put reaching here means the child completed;
+      data _null_;
+        set work.dc_childstatus;
+        putlog "child said: " state= errmsg=;
+      run;
+
+  @param [in] program The _PROGRAM endpoint to run
+  @param [in] inputfiles=(0) fileref:filename pairs, as mx_execute
+  @param [in] inputdatasets=(0) datasets to send as CSV, as mx_execute
+  @param [in] inputparams=(0) name/value dataset, as mx_execute
+  @param [out] outlib=(0) Output libref for the response JSON
+  @param [out] outref=(0) Output fileref for the raw response text
+  @param [in] debug= (131) The _debug value sent to the child.  Numeric, as the
+    server requires; 131 asks for the log.
+  @param [in] mdebug= (&sasjs_mdebug) 1 keeps the child's log and prints it
+  @param [in] viyacontext= (&defaultcontext) Viya compute context
+  @param [out] outstatus=(work.dc_childstatus) One row per child run
+  @param [in] failfast= (1) Abort the caller when the child did not complete
+
+  <h4> SAS Macros </h4>
+  @li mp_abort.sas
+  @li mx_execute.sas
+
+**/
+
+%macro mpe_execute(program
+    ,inputfiles=0
+    ,inputdatasets=0
+    ,inputparams=0
+    ,outlib=0
+    ,outref=0
+    ,debug=
+    ,mdebug=&sasjs_mdebug
+    ,viyacontext=&defaultcontext
+    ,outstatus=work.dc_childstatus
+    ,viyaresult=WEBOUT_JSON
+    ,failfast=1
+  );
+%local logds state msg;
+/* Aliases: passing &program / &inputfiles ... straight into another macro that
+  * declares the same parameter names makes SAS resolve them against the callee
+  * and null them ("recursive reference to the macro variable").  Unique names
+  * on this side keep the values intact. */
+%local a_viyaresult a_pgm a_inds a_indatasets a_inparams a_dbg a_mdbg
+  a_outlib a_outref a_logds a_status a_ctx;
+/* numeric: the server validates _debug as a number, so anything else never asks
+  * for the log and outlogds comes back empty */
+%if %length(&debug)=0 %then %let debug=131;
+%if %length(&mdebug)=0 %then %let mdebug=0;
+%let a_pgm=&program;
+%let a_viyaresult=&viyaresult;
+%let a_inds=&inputfiles;
+%let a_indatasets=&inputdatasets;
+%let a_inparams=&inputparams;
+%let a_dbg=&debug;
+%let a_mdbg=&mdebug;
+%let a_outlib=&outlib;
+%let a_outref=&outref;
+%let a_status=&outstatus;
+%let a_ctx=&viyacontext;
+
+%if &mdebug>0 %then %let logds=work.dc_childlog;
+%else %let logds=_null_;
+%let a_logds=&logds;
+
+/* a status row from an earlier call must not be read as this call's outcome:
+  a child that dies before core writes one would otherwise look completed */
+%if %index(&a_status,.)>0 and %sysfunc(exist(&a_status)) %then %do;
+  proc datasets library=%scan(&a_status,1,.) nolist;
+    delete %scan(&a_status,2,.);
+  quit;
+%end;
+
+%mx_execute(&a_pgm
+  ,inputfiles=&a_inds
+  ,inputdatasets=&a_indatasets
+  ,inputparams=&a_inparams
+  ,debug=&a_dbg
+  ,mdebug=&a_mdbg
+  ,viyacontext=&a_ctx
+  ,outlib=&a_outlib
+  ,outref=&a_outref
+  ,outlogds=&a_logds
+  ,outstatus=&a_status
+  ,viyaresult=&a_viyaresult
+)
+
+/* the child's own log, for an investigation run */
+%if &mdebug>0 %then %do;
+  data _null_;
+    set &a_status;
+    putlog "DC_EXECUTE: " state= errmsg=;
+  run;
+  %if %sysfunc(exist(work.dc_childlog)) %then %do;
+    data _null_;
+      set work.dc_childlog;
+      putlog "CHILD: " line;
+    run;
+  %end;
+%end;
+
+/* fail fast - a child that did not complete must not leave the test running
+  * against whatever the previous call left in outlib.  A missing status table
+  * and an empty one are both failures: state would stay blank and the caller
+  * would carry on inferring, which is the thing this macro exists to stop. */
+%let state=;
+%let msg=;
+%if %sysfunc(exist(&a_status))=1 %then %do;
+  data _null_;
+    set &a_status;
+    call symputx('state',state,'l');
+    call symputx('msg',errmsg,'l');
+  run;
+%end;
+%else %let state=missing(&a_status);
+
+%if %length(&failfast)=0 %then %let failfast=1;
+%if &failfast=1 %then %do;
+  /* SAS 9 (PROC STP) reports no job status at all: core writes state='unknown'
+  * with that explanation on a call that worked, so it cannot be read as a
+  * failure - only as unmeasured.  Everything else that is not completed is a
+  * failure, a blank state included. */
+  %if "%upcase(&state)"="UNKNOWN" %then
+    %put NOTE: &sysmacroname: &program returned no job status (SAS 9) - the outcome was not measured;
+  %else %if "%upcase(&state)" ne "COMPLETED" %then %do;
+    %mp_abort(
+      iftrue=(1=1)
+      ,mac=&sysmacroname
+      ,msg=%str(&program did not complete: state=&state &msg)
+    )
+  %end;
+%end;
+
+%mend mpe_execute;

sas/sasjs/macros/mpe_execute.test.sas

@@ -0,0 +1,125 @@
+/**
+  @file
+  @brief testing the mpe_execute macro
+
+  @details The wrapper exists so that a test can assert what a child call did
+  instead of inferring it from an outlib that still holds the previous call's
+  tables.  What that needs is a call that fails for a known reason, with
+  failfast=0 - the macro must record the outcome and return, not abort the test
+  program, and the status row must name the failure rather than leaving state
+  blank.
+
+  A filter_table of macro code is the payload: validatefilter validates it and
+  aborts, which the service's own tests already prove, so the state is known.
+
+  <h4> SAS Macros </h4>
+  @li mf_getuniquefileref.sas
+  @li mf_nobs.sas
+  @li mp_assert.sas
+  @li mpe_execute.sas
+
+  @author 4GL Apps Ltd
+  @copyright 4GL Apps Ltd.  This code may only be used within Data Controller
+    and may not be re-distributed or re-sold without the express permission of
+    4GL Apps Ltd.
+
+**/
+
+/**
+  * Test 1 - a child that aborts is recorded in outstatus, and the caller
+  * continues (failfast=0)
+  */
+%let _program=&appLoc/services/public/validatefilter;
+
+%let f1=%mf_getuniquefileref();
+data _null_;
+  file &f1 termstr=crlf;
+  put 'FILTER_TABLE:$41.';
+  put '%sysfunc(coalescec(&dclib..MPE_TABLES,))';
+run;
+
+%let outref1=%mf_getuniquefileref();
+
+%mpe_execute(&_program,
+  viyacontext=&defaultcontext,
+  inputfiles=&f1:iwant,
+  outref=&outref1,
+  outstatus=work.exec_status,
+  failfast=0,
+  viyaresult=WEBOUT_TXT
+)
+
+/* The macro reads the LAST row of outstatus - a row is written per HTTP
+  * attempt, so a retried or aborted call leaves more than one - which is what
+  * the assertions below check.  Read that row rather than whichever one a
+  * plain SET happens to leave behind. */
+%let state=;
+%let uri=;
+%let platform=;
+%let errmsg=;
+data _null_;
+  set work.exec_status nobs=nobs;
+  if nobs>0 then do;
+    set work.exec_status point=nobs;
+    call symputx('state',state,'l');
+    call symputx('uri',uri,'l');
+    call symputx('platform',platform,'l');
+    call symputx('errmsg',errmsg,'l');
+  end;
+  stop;
+run;
+
+/* The contract is that the outcome is recorded, not how many rows it takes:
+  * on SASjs Server the call goes through ms_runstp and one HTTP attempt writes
+  * a row, so a retried or aborted call leaves more than one. */
+%mp_assert(
+  iftrue=(%mf_nobs(work.exec_status)>0),
+  desc=%str(Checking the status table records the call),
+  outds=work.test_results
+)
+
+%mp_assert(
+  iftrue=("%upcase(&state)" ne ""),
+  desc=%str(Checking the status row names the child state),
+  outds=work.test_results
+)
+
+%mp_assert(
+  iftrue=("%upcase(&state)" ne "COMPLETED"),
+  desc=%str(Checking an aborted child is not recorded as completed),
+  outds=work.test_results
+)
+
+%mp_assert(
+  iftrue=("%upcase(&platform)" ne ""),
+  desc=%str(Checking the status row names the platform),
+  outds=work.test_results
+)
+
+/**
+  * Test 2 - the child's log is only kept when mdebug asks for it
+  */
+%let f2=%mf_getuniquefileref();
+data _null_;
+  file &f2 termstr=crlf;
+  put 'FILTER_TABLE:$41.';
+  put '%sysfunc(coalescec(&dclib..MPE_TABLES,))';
+run;
+
+%let outref2=%mf_getuniquefileref();
+
+%mpe_execute(&_program,
+  viyacontext=&defaultcontext,
+  inputfiles=&f2:iwant,
+  outref=&outref2,
+  outstatus=work.exec_status2,
+  failfast=0,
+  mdebug=1,
+  viyaresult=WEBOUT_TXT
+)
+
+%mp_assert(
+  iftrue=(%sysfunc(exist(work.dc_childlog))=1),
+  desc=%str(Checking mdebug=1 keeps the child log),
+  outds=work.test_results
+)

sas/sasjs/macros/mpe_makedata.sas

@@ -80,9 +80,9 @@ insert into &lib..mpe_config set
       !!' client side.  The DC_MAXCELLS_WEBEDIT limit is applied at the same'
       !!' time, whichever is reached first.';
 /* Why the cell limit exists, kept out of var_desc - MPE_CONFIG.VAR_DESC is
-  varchar(300), and a truncation WARNING aborts this whole build.  A wide
+  varchar(300), and a truncated value raises a diagnostic that aborts this
-  table is expensive long before it is tall - 250 rows of a 1000 column
+  whole build.  A wide table is expensive long before it is tall - 250 rows of
-  table is a quarter of a million cells. */
+  a 1000 column table is a quarter of a million cells. */
 insert into &lib..mpe_config set
     tx_from=0
     ,tx_to='31DEC9999:23:59:59'dt
@@ -697,6 +697,24 @@ insert into &lib..mpe_security set
       ,selectbox_value='N'
       ,selectbox_order=2
       ,ver_to_dttm='31DEC5999:23:59:59'dt;
+  insert into &lib..mpe_selectbox set
+      selectbox_rk=%mf_increment(rk)
+      ,ver_from_dttm=0
+      ,select_lib="&lib"
+      ,select_ds="MPE_DERIVED"
+      ,base_column="ACTIVE"
+      ,selectbox_value='1'
+      ,selectbox_order=1
+      ,ver_to_dttm='31DEC5999:23:59:59'dt;
+  insert into &lib..mpe_selectbox set
+      selectbox_rk=%mf_increment(rk)
+      ,ver_from_dttm=0
+      ,select_lib="&lib"
+      ,select_ds="MPE_DERIVED"
+      ,base_column="ACTIVE"
+      ,selectbox_value='0'
+      ,selectbox_order=2
+      ,ver_to_dttm='31DEC5999:23:59:59'dt;
   insert into &lib..mpe_selectbox set
       selectbox_rk=%mf_increment(rk)
       ,ver_from_dttm=0
@@ -1987,6 +2005,13 @@ insert into &lib..MPE_VALIDATIONS set
     tx_from=0,base_lib="&lib",base_ds="MPE_DERIVED",base_col="LIBREF"
     ,rule_type='CASE',rule_value='UPCASE',rule_active=1
     ,tx_to='31DEC5999:23:59:59'dt;
+/* the libraries the user can edit, as MPE_VALIDATIONS.BASE_LIB does */
+insert into &lib..MPE_VALIDATIONS set
+    tx_from=0,base_lib="&lib",base_ds="MPE_DERIVED",base_col="LIBREF"
+    ,rule_type='SOFTSELECT_HOOK'
+    ,rule_value="services/validations/libraries_editable"
+    ,rule_active=1
+    ,tx_to='31DEC5999:23:59:59'dt;
 insert into &lib..MPE_VALIDATIONS set
     tx_from=0,base_lib="&lib",base_ds="MPE_DERIVED",base_col="DSN"
     ,rule_type='NOTNULL',rule_value=' ',rule_active=1
@@ -1995,6 +2020,13 @@ insert into &lib..MPE_VALIDATIONS set
     tx_from=0,base_lib="&lib",base_ds="MPE_DERIVED",base_col="DSN"
     ,rule_type='CASE',rule_value='UPCASE',rule_active=1
     ,tx_to='31DEC5999:23:59:59'dt;
+/* tables in the LIBREF on the same row - the hook reads it from source_row */
+insert into &lib..MPE_VALIDATIONS set
+    tx_from=0,base_lib="&lib",base_ds="MPE_DERIVED",base_col="DSN"
+    ,rule_type='SOFTSELECT_HOOK'
+    ,rule_value="services/validations/tables_editable"
+    ,rule_active=1
+    ,tx_to='31DEC5999:23:59:59'dt;
 insert into &lib..MPE_VALIDATIONS set
     tx_from=0,base_lib="&lib",base_ds="MPE_DERIVED",base_col="VAR_NM"
     ,rule_type='NOTNULL',rule_value=' ',rule_active=1
@@ -2027,6 +2059,14 @@ insert into &lib..MPE_VALIDATIONS set
     tx_from=0,base_lib="&lib",base_ds="MPE_DERIVED",base_col="VAR_TYPE"
     ,rule_type='HARDSELECT',rule_value=' ',rule_active=1
     ,tx_to='31DEC5999:23:59:59'dt;
+/* ACTIVE is mandatory, and its NOTNULL rule carries '1' as its value: the EDIT
+  * screen pre-fills a new row from the value of the NOTNULL / READONLY / HIDDEN
+  * rule (getColumnDefault), so a derived variable starts active unless the user
+  * says otherwise.  The values themselves come from MPE_SELECTBOX. */
+insert into &lib..MPE_VALIDATIONS set
+    tx_from=0,base_lib="&lib",base_ds="MPE_DERIVED",base_col="ACTIVE"
+    ,rule_type='NOTNULL',rule_value='1',rule_active=1
+    ,tx_to='31DEC5999:23:59:59'dt;
 insert into &lib..MPE_VALIDATIONS set
     tx_from=0,base_lib="&lib",base_ds="MPE_DERIVED",base_col="VAR_LEN"
     ,rule_type='NOTNULL',rule_value=' ',rule_active=1
@@ -2052,9 +2092,15 @@ insert into &lib..MPE_VALIDATIONS set
     tx_from=0,base_lib="&lib",base_ds="MPE_DERIVED",base_col="VAR_LOGIC"
     ,rule_type='NOTNULL',rule_value=' ',rule_active=1
     ,tx_to='31DEC5999:23:59:59'dt;
+insert into &lib..MPE_VALIDATIONS set
+    tx_from=0,base_lib="&lib",base_ds="MPE_DERIVED",base_col="VAR_LABEL"
+    ,rule_type='HARDREGEX'
+    ,rule_value='^[^\x00-\x1F\x7F]*$',rule_active=1
+    ,tx_to='31DEC5999:23:59:59'dt;
 insert into &lib..MPE_VALIDATIONS set
     tx_from=0,base_lib="&lib",base_ds="MPE_DERIVED",base_col="VAR_LOGIC"
-    ,rule_type='HARDREGEX',rule_value='^[^;&%]*$',rule_active=1
+    ,rule_type='HARDREGEX'
+    ,rule_value='^(?!.*(/\*|\*/))[^;&%]*$',rule_active=1
     ,tx_to='31DEC5999:23:59:59'dt;
 insert into &lib..MPE_VALIDATIONS set
     tx_from=0
@@ -2168,8 +2214,8 @@ insert into &lib..MPE_VALIDATIONS set
     ,rule_active=1
     ,tx_to='31DEC5999:23:59:59'dt;
 /* test soft regex - PRIMARY_KEY_FIELD should be an integer (yellow if it
-contains a decimal point).  All generated keys are integers, so no
+contains a decimal point).  All generated keys are integers, so nothing is
-warning shows until a demo user enters a decimal. */
+flagged until a demo user enters a decimal. */
 insert into &lib..MPE_VALIDATIONS set
     tx_from=0
     ,base_lib="&lib"
@@ -2309,7 +2355,9 @@ insert into &lib..MPE_VALIDATIONS set
       ,some_char="&x bottles of beer on the wall"
       ,some_dropdown='Option 1'
       ,some_num=ranuni(0)
-      ,some_date=round(ranuni(0)*1000,1)
+      /* First of the month, cycling through two years: a randomly spread date
+      gave every row its own day, so a date filter never returned many rows. */
+      ,some_date=intnx('month','01JAN1960'd,mod(&x,24),'S')
       ,some_datetime=round(ranuni(0)*50000,1)
       ,some_time=round(ranuni(0)*100,1)
       ,some_shortnum=6+round(ranuni(0)*94,1)

sas/sasjs/macros/mpe_refreshcatalogs.sas

@@ -2,11 +2,17 @@
   @file
   @brief Refreshes the data catalog
   @details Assumes library is already assigned.
+
+  @param [in] etlsource= (0) Defaults to %mf_getuniquename(prefix=refreshcatalogs); it
+    identifies this load in the MPE_DATALOADS audit row.
+
   Usage:
 
       %mpe_refreshcatalogs(sashelp)
 
   <h4> SAS Macros </h4>
+  @li mf_getuniquename.sas
+  @li mp_abort.sas
   @li bitemporal_dataloader.sas
 
   @version 9.3
@@ -14,9 +20,12 @@
 
 **/
 
-%macro mpe_refreshcatalogs(lib,cat=#all);
+%macro mpe_refreshcatalogs(lib,cat=#all
+  ,etlsource=0
+);
 %let lib=%upcase(&lib);
 %let cat=%upcase(&cat);
+%if "&etlsource"="0" %then %let etlsource=%mf_getuniquename(prefix=refreshcatalogs);
 
 %put running &sysmacroname &lib for &cat;
 proc sql;
@@ -50,7 +59,7 @@ create table datacats as select distinct libref,memname from catdata;
   ,base_dsn=mpe_datacatalog_CATS
   ,append_dsn=datacats
   ,PK=LIBREF MEMNAME
-  ,etlsource=&sysmacroname
+  ,etlsource=&etlsource
   ,loadtype=TXTEMPORAL
   ,tech_from=TX_FROM
   ,tech_to=TX_TO
@@ -76,7 +85,7 @@ quit;
   ,base_dsn=mpe_datacatalog_OBJS
   ,append_dsn=dataobjs
   ,PK=LIBREF MEMNAME OBJNAME OBJTYPE
-  ,etlsource=&sysmacroname
+  ,etlsource=&etlsource
   ,loadtype=TXTEMPORAL
   ,tech_from=TX_FROM
   ,tech_to=TX_TO
@@ -102,7 +111,7 @@ create table statusobjs as
   ,base_dsn=mpe_datastatus_OBJS
   ,append_dsn=statusobjs
   ,PK=LIBREF MEMNAME OBJNAME OBJTYPE
-  ,etlsource=&sysmacroname
+  ,etlsource=&etlsource
   ,loadtype=TXTEMPORAL
   ,tech_from=TX_FROM
   ,tech_to=TX_TO
@@ -127,7 +136,7 @@ create table statuscats as
   ,base_dsn=mpe_datastatus_cats
   ,append_dsn=statuscats
   ,PK=LIBREF MEMNAME
-  ,etlsource=&sysmacroname
+  ,etlsource=&etlsource
   ,loadtype=TXTEMPORAL
   ,tech_from=TX_FROM
   ,tech_to=TX_TO

sas/sasjs/macros/mpe_refreshlibs.sas

@@ -3,7 +3,12 @@
   @brief Refreshes the library data catalog
   @details
 
+  @param [in] etlsource= (0) Defaults to %mf_getuniquename(prefix=refreshlibs), identifies this load
+    in the MPE_DATALOADS audit row.  That table is keyed on etlsource together
+    with processed_dttm, libref and dsn, so the uid keeps each load distinct.
+
   <h4> SAS Macros </h4>
+  @li mf_getuniquename.sas
   @li dc_getlibs.sas
   @li mf_getplatform.sas
   @li bitemporal_dataloader.sas
@@ -16,7 +21,14 @@
 
 **/
 
-%macro mpe_refreshlibs(lib=0);
+%macro mpe_refreshlibs(lib=0
+  ,etlsource=0
+);
+/* a default of 0 rather than a %mf_uid() expression: a default is
+  resolved when the macro is compiled, so it would be one value for the
+  whole session and the audit folder below it would collide on the
+  second call - as it did between the refresh steps of one run */
+%if "&etlsource"="0" %then %let etlsource=%mf_getuniquename(prefix=refreshlibs);
 
 %dc_getlibs(outds=work.mm_getLibs)
 proc sort data=mm_getlibs;
@@ -119,7 +131,7 @@ create table libs5 as
   ,base_dsn=MPE_DATACATALOG_LIBS
   ,append_dsn=libs5
   ,PK=LIBREF
-  ,etlsource=&_program
+  ,etlsource=&etlsource
   ,loadtype=TXTEMPORAL
   ,tech_from=TX_FROM
   ,tech_to=TX_TO

sas/sasjs/macros/mpe_refreshtables.sas

@@ -1,12 +1,18 @@
 /**
   @file mpe_refreshtables.sas
   @brief Refreshes the data catalog
-  @details Assumes library is already assigned.
+  @details
+
+  @param [in] etlsource= (0) Defaults to %mf_getuniquename(prefix=refreshtables), identifies this load
+    in the MPE_DATALOADS audit row.  That table is keyed on etlsource together
+    with processed_dttm, libref and dsn, so the uid keeps each load distinct.
   Usage:
 
       %mpe_refreshtables(sashelp)
 
   <h4> SAS Macros </h4>
+  @li mf_getuniquename.sas
+  @li mp_abort.sas
   @li bitemporal_dataloader.sas
   @li mf_existfeature.sas
   @li mf_getengine.sas
@@ -17,7 +23,14 @@
   @author 4GL Apps Ltd
 **/
 
-%macro mpe_refreshtables(lib,ds=#all);
+%macro mpe_refreshtables(lib,ds=#all
+  ,etlsource=0
+);
+/* a default of 0 rather than a %mf_uid() expression: a default is
+  resolved when the macro is compiled, so it would be one value for the
+  whole session and the audit folder below it would collide on the
+  second call - as it did between the refresh steps of one run */
+%if "&etlsource"="0" %then %let etlsource=%mf_getuniquename(prefix=refreshtables);
 %let lib=%upcase(&lib);
 %let ds=%upcase(&ds);
 %local engine; %let engine=%mf_getengine(&lib);
@@ -180,7 +193,7 @@ create table cols as
   ,base_dsn=mpe_datacatalog_vars
   ,append_dsn=finalcols
   ,PK=LIBREF DSN NAME
-  ,etlsource=&sysmacroname
+  ,etlsource=&etlsource
   ,loadtype=TXTEMPORAL
   ,tech_from=TX_FROM
   ,tech_to=TX_TO
@@ -242,7 +255,7 @@ create table work.finaltabs as
   ,base_dsn=mpe_datacatalog_tabs
   ,append_dsn=finaltabs
   ,PK=LIBREF DSN
-  ,etlsource=&sysmacroname
+  ,etlsource=&etlsource
   ,loadtype=TXTEMPORAL
   ,tech_from=TX_FROM
   ,tech_to=TX_TO
@@ -302,7 +315,7 @@ proc sql;
   ,base_dsn=mpe_datastatus_tabs
   ,append_dsn=statustabs
   ,PK=LIBREF DSN
-  ,etlsource=&sysmacroname
+  ,etlsource=&etlsource
   ,loadtype=TXTEMPORAL
   ,tech_from=TX_FROM
   ,tech_to=TX_TO
@@ -345,7 +358,7 @@ proc sql;
     ,base_dsn=mpe_datastatus_libs
     ,append_dsn=statuslibs
     ,PK=LIBREF
-    ,etlsource=&sysmacroname
+    ,etlsource=&etlsource
     ,loadtype=TXTEMPORAL
     ,tech_from=TX_FROM
     ,tech_to=TX_TO

sas/sasjs/macros/mpe_validatecol.test.sas

@@ -60,7 +60,7 @@ data work.mismatch;
 run;
 
 %mp_assertdsobs(work.mismatch,
-  desc=Every LIBDS value validated as expected (catalog form permitted, nothing smuggled past the dash),
+  desc=%str(Every LIBDS value validated as expected (catalog form permitted, nothing smuggled past the dash)),
   test=EQUALS 0,
   outds=work.test_results
 )

sas/sasjs/services/admin/refreshcatalog.test.1.sas

@@ -8,8 +8,9 @@
   the admin group, so the gate passes here.
 
   <h4> SAS Macros </h4>
+  @li mpe_assert_abort.sas
   @li mp_assert.sas
-  @li mx_execute.sas
+  @li mpe_execute.sas
 
 **/
 
@@ -23,23 +24,19 @@ data work.params1;
   name='libref';value='A.%sysevalf(3+4)B';output;
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputparams=work.params1,
   outref=web1,
+  outstatus=work.ab1,
+  failfast=0,
   viyaresult=WEBOUT_TXT
 )
 
-%let abort1=0;
+%mpe_assert_abort(
-data _null_;
-  set work.results;
-  if state='canceled' then call symputx('abort1',1);
-run;
-
-%mp_assert(
-  iftrue=(&abort1=1),
   desc=Macro content in libref aborts the service,
-  outds=work.test_results
+  outref=web1,
+  msg=Invalid libref
 )
 
 /**
@@ -50,7 +47,7 @@ data work.params2;
   name='libref';value='DCTEST';output;
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputparams=work.params2,
   outlib=web2

sas/sasjs/services/auditors/getdiffs.test.sas

@@ -13,7 +13,9 @@
   against a service that does not validate.
 
   <h4> SAS Macros </h4>
+  @li mpe_assert_abort.sas
   @li mp_assert.sas
+  @li mpe_execute.sas
   @li mx_execute.sas
   @li mf_getuniquefileref.sas
 
@@ -101,23 +103,19 @@ data _null_;
   put _row;
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputfiles=&fb1:BrowserParams,
   outref=web1,
+  outstatus=work.ab1,
+  failfast=0,
   viyaresult=WEBOUT_TXT
 )
 
-%let abort1=0;
+%mpe_assert_abort(
-data _null_;
-  set work.results;
-  if state='canceled' then call symputx('abort1',1);
-run;
-
-%mp_assert(
-  iftrue=(&abort1=1),
   desc=SQL injection payload in libds aborts the service,
-  outds=work.test_results
+  outref=web1,
+  msg=Invalid libds
 )
 
 /**
@@ -137,23 +135,19 @@ data _null_;
   put "&travtable.,&diffscsv.,&dclib..MPE_X_TEST";
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputfiles=&fb2:BrowserParams,
   outref=web2,
+  outstatus=work.ab2,
+  failfast=0,
   viyaresult=WEBOUT_TXT
 )
 
-%let abort2=0;
+%mpe_assert_abort(
-data _null_;
-  set work.results;
-  if state='canceled' then call symputx('abort2',1);
-run;
-
-%mp_assert(
-  iftrue=(&abort2=1),
   desc=Path traversal in table aborts the service,
-  outds=work.test_results
+  outref=web2,
+  msg=Invalid table
 )
 
 /**
@@ -166,23 +160,19 @@ data _null_;
   put "&loadref.,../&loadref./&diffscsv.,&dclib..MPE_X_TEST";
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputfiles=&fb3:BrowserParams,
   outref=web3,
+  outstatus=work.ab3,
+  failfast=0,
   viyaresult=WEBOUT_TXT
 )
 
-%let abort3=0;
+%mpe_assert_abort(
-data _null_;
-  set work.results;
-  if state='canceled' then call symputx('abort3',1);
-run;
-
-%mp_assert(
-  iftrue=(&abort3=1),
   desc=Path traversal in stp_diffs_csv aborts the service,
-  outds=work.test_results
+  outref=web3,
+  msg=Invalid stp_diffs_csv
 )
 
 /**

sas/sasjs/services/editors/getdata.sas

@@ -21,6 +21,10 @@
     DC_MAXOBS_WEBEDIT / DC_MAXCELLS_WEBEDIT limit, else 0
   @li TRUNC_MSG - the reason shown against the table name when TRUNCATED=1,
     else empty
+  @li DERIVEDVARS - space separated list of the derived (computed) columns
+    configured in MPE_DERIVED, upper case, empty when there are none.  They
+    are appended after the physical columns and are read-only; the EDIT grid
+    marks them so a user can see why a column cannot be edited.
 
   <h5> approvers </h5>
   <h5> dqrules </h5>
@@ -181,7 +185,23 @@ run;
 
 
 %global loadtype var_txfrom var_txto var_processed filter_text pk coltype
-  sortpk;
+  sortpk var_busfrom var_busto rk_underlying buskey pre_edit_hook;
+/* clear first: a global survives the call, so in a reused session it would
+  still hold the previous request's values, and any path that skips an
+  assignment would apply them to this one */
+%let loadtype=;
+%let var_txfrom=;
+%let var_txto=;
+%let var_processed=;
+%let filter_text=;
+%let pk=;
+%let coltype=;
+%let sortpk=;
+%let var_busfrom=;
+%let var_busto=;
+%let rk_underlying=;
+%let buskey=;
+%let pre_edit_hook=;
 
 %put getting table attributes;
 proc sql noprint;
@@ -340,8 +360,10 @@ select upcase(loadtype)
                           );
   %put sourcevars=&sourcevars;
   /* the payload is capped at &maxrows - whichever limit that works out to be
-    - this is what goes to the browser */
+    - this is what goes to the browser.  The cap is a data set option on the
-  options obs=&maxrows;
+    SET, not `options obs=`, so no later step in this session can be cut by it:
+    a session option left set would truncate the metadata steps below, and a
+    table wide enough to hit the cell limit would lose columns rather than rows. */
   data outdata;
     /* pk fields come first */
     %if %length(&pk)>0 %then %do;
@@ -355,9 +377,8 @@ select upcase(loadtype)
       /* send empty row if empty table to help with hot rendering */
       output;
     %end;
-    set work.out ;
+    set work.out(obs=&maxrows) ;
   run;
-  options obs=max;
 
 
 
@@ -531,7 +552,7 @@ select upcase(loadtype)
 %mpestp_getdata()
 
 /* mp_abort cannot exit cleanly from inside a %include, so an abort there
-  (eg in a hook script run via mp_include) only records the error in
+  (eg in a hook script run via mp_include) only records the failure in
   work.mp_abort_errds.  mode=INCLUDE picks that up and aborts properly here. */
 %mp_abort(mode=INCLUDE)
 
@@ -628,8 +649,19 @@ proc sql noprint;
 select count(*) into: ismap from &mpelib..mpe_xlmap_info
   where XLMAP_TARGETLIBDS="&orig_libds" and &dc_dttmtfmt. le TX_TO;
 
+/* the derived columns, named for the EDIT grid so it can mark them as special.
+  They are read-only through the READONLY DQ rule below, but that rule does not
+  say WHY the column cannot be edited - a computed column is a different thing
+  from an ordinary read-only one.  work.derived_vars exists on every path, empty
+  when the table has no derived variables. */
+%let derivedvars=;
+proc sql noprint;
+select upcase(name) into: derivedvars separated by ' '
+  from work.derived_vars;
+quit;
+
 data sasparams;
-  length colHeaders $20000 filter_text $32767 trunc_msg $200;
+  length colHeaders $20000 filter_text $32767 trunc_msg $200 derivedvars $32767;
   colHeaders=cats(upcase("%mf_getvarlist(sasdata1,dlm=%str(,))"));
   pkCnt=countw("&pk");
   pk="&pk";
@@ -640,6 +672,7 @@ data sasparams;
   if trim(symget('rk_underlying')) ne '' then rk_flag=1;
   else rk_flag=0;
   filter_text=symget('filter_text');
+  derivedvars=symget('derivedvars');
   /* set when the selection was cut down to the row / cell limit, so the EDIT
     screen can say so against the table name */
   truncated=&truncated;
@@ -661,7 +694,16 @@ proc sort data=&mpelib..mpe_validations
   by base_col rule_type rule_value;
 run;
 
-/* merge with NOTNULL constraints in the physical table */
+/* merge with NOTNULL constraints in the physical table, and mark the derived
+  variables read-only: a derived column is computed at read time, so the grid
+  must not accept a value for it.  COLS carries the same READ (a derived column
+  has no CLS rule row) but the client only applies COLS.CLS_RULE when CLS_FLAG
+  is set, which is driven by the table's column-level security - so the rule is
+  declared here, where it holds regardless.
+
+  work.derived_vars exists on every path that reaches this step: %mpe_derived
+  writes it before returning, empty when the table has no derived variables,
+  so the union below needs no guard */
 proc sql;
 create table _data_ as
   select * from dqrules
@@ -674,6 +716,11 @@ union
     and upcase(memname)="%scan(&orig_libds,2,.)"
     and upcase(name) in (select name from vars3)
     and notnull='yes'
+union
+  select upcase(name) as base_col
+    ,'READONLY' as rule_type
+    ,'' as rule_value
+  from work.derived_vars
   order by 1,2,3;
 data dqrules;
   set &syslast;

sas/sasjs/services/editors/getdata.test.1.sas

@@ -14,7 +14,8 @@
   rows x 1004 columns, created and registered by testsetup) filtered to 200
   rows: that is under the row limit but 200 x 1004 = 200,800 cells, over the
   200,000 cell limit, so the cell limit decides the cap -
-  floor(200000 / 1004) = 199 rows.
+  floor(200000 / 1004) = 199 rows.  It also checks the column list is whole:
+  the cap applies to the rows the browser is sent, never to the COLS metadata.
 
   <h4> SAS Macros </h4>
   @li mp_assert.sas
@@ -98,7 +99,7 @@ run;
 
 %mp_assert(
   iftrue=(&t1_trunc=1),
-  desc=Test 1 - a selection over the row limit is served, flagged as truncated,
+  desc=%str(Test 1 - a selection over the row limit is served, flagged as truncated),
   outds=work.test_results
 )
 %mp_assert(
@@ -133,7 +134,7 @@ run;
 
 %mp_assert(
   iftrue=(&t2_trunc=1),
-  desc=Test 2 - a selection over the cell limit is served, flagged as truncated,
+  desc=%str(Test 2 - a selection over the cell limit is served, flagged as truncated),
   outds=work.test_results
 )
 %mp_assert(
@@ -147,6 +148,27 @@ run;
   outds=work.test_results
 )
 
+/**
+  * The row cap must not reach the metadata: the COLS payload for this table
+  * describes every column, not the 199 rows the cell limit admits.  A row cap
+  * left set as a session option would cut the column list to 199 entries, and
+  * a grid losing 800 columns looks like a missing table rather than a limit.
+  */
+%let t2_cols=-1;
+%let t2_phys=-1;
+proc sql noprint;
+select count(*) into: t2_cols from web2.cols;
+select count(*) into: t2_phys
+  from dictionary.columns
+  where libname='DCTEST' and memname='WIDEBOY';
+quit;
+
+%mp_assert(
+  iftrue=(&t2_cols=&t2_phys),
+  desc=%str(Test 2 - every column is described, not just the rows the cap admits),
+  outds=work.test_results
+)
+
 /**
   * dump results to the log for offline inspection
   */

sas/sasjs/services/editors/getdata.test.2.sas

@@ -65,8 +65,8 @@ select count(*) into: admin_check
   where groupname="&dc_admin_group";
 quit;
 %if &admin_check>0 %then %do;
-  %put WARNING: %mf_getuser() is in the &dc_admin_group group;
+  %put %str(WARN)ING: %mf_getuser() is in the &dc_admin_group group;
-  %put WARNING: Row Level Security is skipped for admins - test skipped;
+  %put %str(WARN)ING: Row Level Security is skipped for admins - test skipped;
   data work.test_results;
     length test_result $4 test_description $256 test_comments $256;
     test_result='PASS';

sas/sasjs/services/editors/getdata.test.sas

@@ -3,15 +3,14 @@
   @brief testing gethistory service
 
   <h4> SAS Macros </h4>
-  @li mp_assert.sas
+  @li mpe_execute.sas
-  @li mp_assertcols.sas
-  @li mp_assertcolvals.sas
   @li mp_assertdsobs.sas
   @li mf_getuniquefileref.sas
   @li mf_getuniquename.sas
   @li mp_filterstore.sas
   @li mpe_derived.sas
-  @li mx_execute.sas
+  @li mp_assert.sas
+  @li mp_assertcols.sas
 
 **/
 
@@ -57,7 +56,7 @@ data _null_;
   put 'LIBDS:$43. FILTER_RK:8.';
   put "&dclib..MPE_TABLES,&filter_rk1";
 run;
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputfiles=&f1:sascontroltable,
   outlib=web1,
@@ -112,8 +111,8 @@ data work.inquery;
   input GROUP_LOGIC:$3. SUBGROUP_LOGIC:$3. SUBGROUP_ID:8. VARIABLE_NM:$32.
     OPERATOR_NM:$10. RAW_VALUE:$4000.;
 datalines4;
-AND,AND,1,LIBREF,CONTAINS,"'DC'"
+AND,AND,1,LIBREF,=,"'&dc_libref'"
-AND,OR,2,DSN,=,"'MPE_LOCK_ANYTABLE'"
+AND,AND,2,DSN,=,"'MPE_LOCK_ANYTABLE'"
 ;;;;
 run;
 %mp_filterstore(
@@ -138,7 +137,7 @@ data _null_;
   put "&dclib..MPE_TABLES,&filter_rk";
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputfiles=&f2:sascontroltable,
   outlib=web2
@@ -218,7 +217,7 @@ data _null_;
   put "DCTEST.DCFMTS-FC,&filter_rk3";
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputfiles=&f3:sascontroltable,
   outlib=web3,
@@ -288,12 +287,16 @@ quit;
 )
 
 data work.inquery4;
-  infile datalines4 dsd;
+  length GROUP_LOGIC $3 SUBGROUP_LOGIC $3 SUBGROUP_ID 8 VARIABLE_NM $32
-  input GROUP_LOGIC:$3. SUBGROUP_LOGIC:$3. SUBGROUP_ID:8. VARIABLE_NM:$32.
+    OPERATOR_NM $10 RAW_VALUE $4000;
-    OPERATOR_NM:$10. RAW_VALUE:$4000.;
+  GROUP_LOGIC='AND'; SUBGROUP_LOGIC='AND'; SUBGROUP_ID=1;
-datalines4;
+  VARIABLE_NM='LIBREF'; OPERATOR_NM='=';
-AND,AND,1,DERIVED_NOTE,CONTAINS,"'MPE_LOCK'"
+  RAW_VALUE="'"!!trim(symget('dc_libref'))!!"'";
-;;;;
+  output;
+  GROUP_LOGIC='AND'; SUBGROUP_LOGIC='AND'; SUBGROUP_ID=2;
+  VARIABLE_NM='DERIVED_NOTE'; OPERATOR_NM='CONTAINS';
+  RAW_VALUE="'MPE_LOCK'";
+  output;
 run;
 %mp_filterstore(
   libds=&dc_libref..MPE_TABLES,
@@ -318,29 +321,38 @@ data _null_;
   put "&dclib..MPE_TABLES,&filter_rk4";
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputfiles=&f4:sascontroltable,
   outlib=web4,
+  outstatus=work.ab4,
+  failfast=0,
   mdebug=&sasjs_mdebug
 )
 
-/* a cancelled child returns no webout, so the outlib keeps whatever an
+%let state4=NONE;
-  earlier call left in it - check the job state before reading its members,
-  otherwise the assertion failures below describe the wrong problem */
-%let abort4=0;
 data _null_;
-  set work.results;
+  if not exist('work.ab4') then stop;
-  if state='canceled' then call symputx('abort4',1);
+  set work.ab4;
+  call symputx('state4',state);
 run;
 
 %mp_assert(
-  iftrue=(&abort4=0),
+  iftrue=("&state4"="completed"),
   desc=Test4 - the derived variable call completes,
   outds=work.test_results
 )
 
-%if &abort4=0 %then %do;
+/* a cancelled child returns no webout, so the outlib keeps whatever an earlier
+  * call left in it - read its members only when the child completed */
+%let completed4=0;
+data _null_;
+  if not exist('work.ab4') then stop;
+  set work.ab4;
+  if state='completed' then call symputx('completed4',1);
+run;
+
+%if &completed4=1 %then %do;
   data sasdata4;
     set web4.sasdata;
   run;
@@ -355,10 +367,30 @@ run;
   run;
 %end;
 
-%mp_assertcols(work.cols4,
+%if &completed4=1 and "&sasjs_mdebug"="1" %then %do;
+  /* the payload the assertions read, so a missing column or an unapplied filter
+    can be seen without guessing */
+  data _null_;
+    set work.cols4;
+    if _n_=1 then putlog "NOTE: &sysmacroname: Test4 cols payload:";
+    putlog (_all_)(=);
+  run;
+  data _null_;
+    set sasdata4;
+    if _n_=1 then putlog "NOTE: &sysmacroname: Test4 data payload:";
+    putlog (_all_)(=);
+  run;
+%end;
+
+proc sql noprint;
+select count(*) into: derived4
+  from work.cols4
+  where upcase(name)='DERIVED_NOTE';
+quit;
+
+%mp_assert(
+  iftrue=(&derived4>0),
   desc=Test4 - derived column is in the COLS payload,
-  cols=DERIVED_NOTE,
-  test=ALL,
   outds=work.test_results
 )
 
@@ -372,6 +404,19 @@ run;
   outds=work.test_results
 )
 
+/* COLS.CLS_RULE only reaches the grid when CLS_FLAG is set, which a table
+  with no column level security does not do - the read-only promise is
+  carried by this rule instead */
+data work.chk4e;
+  set web4.dqrules;
+  where upcase(base_col)='DERIVED_NOTE' and rule_type='READONLY';
+run;
+%mp_assertdsobs(work.chk4e,
+  desc=Test4 - derived column carries a READONLY rule,
+  test=EQUALS 1,
+  outds=work.test_results
+)
+
 data work.chk4b;
   set work.cols4;
   where name='DERIVED_NOTE' and label=:'DERIVED_NOTE=';
@@ -390,7 +435,7 @@ run;
 
 data work.chk4c;
   set work.sasdata4;
-  if derived_note=cats("%scan(&dclib..MPE_TABLES,1,.)",'.','MPE_LOCK_ANYTABLE');
+  if derived_note=cats("%scan(&dclib..MPE_TABLES,1,.)",'.','MPE_LOCKANYTABLE');
 run;
 %mp_assertdsobs(work.chk4c,
   desc=Test4 - derived value is correct,

sas/sasjs/services/editors/stagedata.sas

@@ -294,7 +294,7 @@ run;
   ,dc_dttmtfmt=&dc_dttmtfmt
 )
 /* mp_abort cannot exit cleanly from inside a %include, so an abort there
-  (eg in a hook script run via mp_include) only records the error in
+  (eg in a hook script run via mp_include) only records the failure in
   work.mp_abort_errds.  mode=INCLUDE picks that up and aborts properly here. */
 %mp_abort(mode=INCLUDE)
 

sas/sasjs/services/editors/stagedata.test.3.sas

@@ -5,13 +5,14 @@
     1. loading row level EDIT security against a table with LOADTYPE=REPLACE
     2. applying LOADTYPE=REPLACE to a table with row level EDIT security
 
-  The staged loads are aborted by the post edit hooks inside the child
+  The staged loads are aborted by the post edit hooks.  The assertion fetches
-  service.  The abort is recorded as a FAILED load in MPE_LOADS (via
+  the sasjsAbort object that mp_abort writes to the webout before the hook
-  mpe_loadfail), so the assertions query MPE_LOADS rather than the child
+  aborts.
-  response - a canceled child does not always return a webout or log.
 
   <h4> SAS Macros </h4>
   @li mx_execute.sas
+  @li mpe_execute.sas
+  @li mpe_assert_abort.sas
   @li mp_assert.sas
 
 
@@ -71,7 +72,6 @@ quit;
 /**
   * Test 1 - submitting an RLS EDIT rule against a REPLACE table should abort
   */
-%let t1=%sysfunc(datetime());
 data work.sascontroltable;
   action='LOAD';
   message='RLS EDIT rule on REPLACE table should abort';
@@ -100,36 +100,19 @@ data work.jsdata;
   output;
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputdatasets=work.sascontroltable work.jsdata,
   outlib=web1,
   outref=wb1,
+  outstatus=work.ab1,
+  failfast=0,
   mdebug=&sasjs_mdebug
 )
 
-/* a hook-aborted load is terminated before mpe_loadfail can record a
+%mpe_assert_abort(
-  * FAILED status, so the load row stays IN PROGRESS - the abort is
-  * proved by the load NOT completing successfully.  Tests in this suite run
-  * in parallel sessions, so match the row for THIS test by its unique
-  * reason_txt (the submitted message) rather than any load in the time
-  * window - an unrelated aborted/IN PROGRESS load would otherwise satisfy
-  * the assertion even if this load completed.  The row id (csv_dir/mperef)
-  * is generated inside the aborted child service, which returns no webout,
-  * so reason_txt is the only stable identifier the test controls. */
-%let abort1=0;
-proc sql noprint;
-select count(*) into: abort1
-  from &dc_libref..mpe_loads
-  where processed_dttm>&t1
-    and reason_txt="RLS EDIT rule on REPLACE table should abort"
-    and upcase(status) ne 'SUCCESS';
-quit;
-
-%mp_assert(
-  iftrue=(&abort1>0),
   desc=Checking RLS EDIT rule is rejected for REPLACE loadtype table (#211),
-  outds=work.test_results
+  outref=wb1
 )
 
 /**
@@ -137,7 +120,6 @@ quit;
   * time: the rule is evaluated against the submitted data, where a derived
   * column does not exist
   */
-%let t3=%sysfunc(datetime());
 
 /* define a derived variable on the target table */
 proc sql noprint;
@@ -185,27 +167,19 @@ data work.jsdata;
   output;
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputdatasets=work.sascontroltable work.jsdata,
   outlib=web3,
   outref=wb3,
+  outstatus=work.ab3,
+  failfast=0,
   mdebug=&sasjs_mdebug
 )
 
-%let abort3=0;
+%mpe_assert_abort(
-proc sql noprint;
-select count(*) into: abort3
-  from &dc_libref..mpe_loads
-  where processed_dttm>&t3
-    and reason_txt="RLS rule on a derived variable should abort"
-    and upcase(status) ne 'SUCCESS';
-quit;
-
-%mp_assert(
-  iftrue=(&abort3>0),
   desc=Checking an RLS rule on a derived variable is rejected,
-  outds=work.test_results
+  outref=wb3
 )
 
 /* tidy up the derived configuration */
@@ -219,7 +193,6 @@ quit;
   * should abort.  The active EDIT rule was inserted in the prep section,
   * long before this child service reads it.
   */
-%let t2=%sysfunc(datetime());
 
 /* now stage a REPLACE loadtype for DCTEST.WIDEBOY */
 data work.sascontroltable;
@@ -237,27 +210,19 @@ data work.jsdata;
   _____DELETE__THIS__RECORD_____='No';
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputdatasets=work.sascontroltable work.jsdata,
   outlib=web2,
   outref=wb2,
+  outstatus=work.ab2,
+  failfast=0,
   mdebug=&sasjs_mdebug
 )
 
-%let abort2=0;
+%mpe_assert_abort(
-proc sql noprint;
-select count(*) into: abort2
-  from &dc_libref..mpe_loads
-  where processed_dttm>&t2
-    and reason_txt="REPLACE loadtype on RLS-secured table should abort"
-    and upcase(status) ne 'SUCCESS';
-quit;
-
-%mp_assert(
-  iftrue=(&abort2>0),
   desc=Checking REPLACE loadtype is rejected for RLS EDIT-secured table (#211),
-  outds=work.test_results
+  outref=wb2
 )
 
 /**

sas/sasjs/services/editors/stagedata.test.4.sas

@@ -6,25 +6,27 @@
       table - the engine reads only active rows, so the dependency can never
       resolve at run time
     2. a row is staged with active=0 while a surviving active row depends on it
+    3. a staged row breaks a field rule of the shared scan (core's
+      mp_derivedvarcheck), or carries an ACTIVE value the EDIT screen's picker
+      cannot produce
 
   They should NOT abort when:
-    3. a new active row is added to a table that also holds an inactive row -
+    4. a new active row is added to a table that also holds an inactive row -
       the inactive row is not part of the dependency graph, so it cannot
       create a cycle or a stale reference
-    4. a delete-marked RLS rule naming a derived variable is submitted - the
+    5. a delete-marked RLS rule naming a derived variable is submitted - the
       rule is being removed, so its variables are not validated
-    5. a delete and re-add of the same variable is submitted together - the
+    6. a delete and re-add of the same variable is submitted together - the
       re-added active row resolves the dependency
-    6. an inactive row is set active again while another active row depends on
+    7. an inactive row is set active again while another active row depends on
       it - the re-activation is what makes the dependency resolvable
 
-  The staged loads are aborted by the post edit hook inside the child service.
+  The staged loads are aborted by the post edit hook. The assertion fetches the
-  An aborted load leaves its MPE_LOADS row IN PROGRESS, so each assertion
+  sasjsAbort object that mp_abort writes to the webout before the hook aborts.
-  matches on the unique reason_txt (the submitted message) rather than any load
-  in the time window - the suite runs in parallel sessions.
 
   <h4> SAS Macros </h4>
-  @li mx_execute.sas
+  @li mpe_execute.sas
+  @li mpe_assert_abort.sas
   @li mp_assert.sas
 
 
@@ -89,27 +91,19 @@ data work.jsdata;
   output;
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputdatasets=work.sascontroltable work.jsdata,
   outlib=web1,
   outref=wb1,
+  outstatus=work.ab1,
+  failfast=0,
   mdebug=&sasjs_mdebug
 )
 
-%let abort1=0;
+%mpe_assert_abort(
-proc sql noprint;
-select count(*) into: abort1
-  from &dc_libref..mpe_loads
-  where processed_dttm>&t1
-    and reason_txt='Derived var depending on an inactive row should abort'
-    and upcase(status) ne 'SUCCESS';
-quit;
-
-%mp_assert(
-  iftrue=(&abort1>0),
   desc=Checking a dependency on an inactive derived row is rejected,
-  outds=work.test_results
+  outref=wb1
 )
 
 /**
@@ -117,7 +111,6 @@ quit;
   * rejected: the control table still shows the row as active, so the check has
   * to read the staged row too
   */
-%let t2=%sysfunc(datetime());
 
 proc sql noprint;
 delete from &dc_libref..mpe_derived
@@ -164,27 +157,19 @@ data work.jsdata;
   _____DELETE__THIS__RECORD_____='No';
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputdatasets=work.sascontroltable work.jsdata,
   outlib=web2,
   outref=wb2,
+  outstatus=work.ab2,
+  failfast=0,
   mdebug=&sasjs_mdebug
 )
 
-%let abort2=0;
+%mpe_assert_abort(
-proc sql noprint;
-select count(*) into: abort2
-  from &dc_libref..mpe_loads
-  where processed_dttm>&t2
-    and reason_txt='Deactivating a relied-on derived row should abort'
-    and upcase(status) ne 'SUCCESS';
-quit;
-
-%mp_assert(
-  iftrue=(&abort2>0),
   desc=Checking deactivating a relied-on derived row is rejected,
-  outds=work.test_results
+  outref=wb2
 )
 
 /**
@@ -235,7 +220,7 @@ data work.jsdata;
   output;
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputdatasets=work.sascontroltable work.jsdata,
   outlib=web3,
@@ -313,7 +298,7 @@ data work.jsdata;
   _____DELETE__THIS__RECORD_____='Yes';
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputdatasets=work.sascontroltable work.jsdata,
   outlib=web4,
@@ -385,27 +370,19 @@ data work.jsdata;
   _____DELETE__THIS__RECORD_____='Yes';
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputdatasets=work.sascontroltable work.jsdata,
   outlib=web5,
   outref=wb5,
+  outstatus=work.ab5,
+  failfast=0,
   mdebug=&sasjs_mdebug
 )
 
-%let abort5=0;
+%mpe_assert_abort(
-proc sql noprint;
-select count(*) into: abort5
-  from &dc_libref..mpe_loads
-  where processed_dttm>&t5
-    and reason_txt='Deleting a relied-on derived row should abort'
-    and upcase(status) ne 'SUCCESS';
-quit;
-
-%mp_assert(
-  iftrue=(&abort5>0),
   desc=Checking deleting a relied-on derived row is rejected,
-  outds=work.test_results
+  outref=wb5
 )
 
 /**
@@ -475,7 +452,7 @@ data work.jsdata;
   output;
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputdatasets=work.sascontroltable work.jsdata,
   outlib=web6,
@@ -557,7 +534,7 @@ data work.jsdata;
   _____DELETE__THIS__RECORD_____='No';
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputdatasets=work.sascontroltable work.jsdata,
   outlib=web7,
@@ -578,6 +555,104 @@ run;
   outds=work.test_results
 )
 
+
+/**
+  * Test 8 - a field rule of the shared scan (mp_derivedvarcheck, core) is
+  * applied at submit time: an expression that could submit code of its own is
+  * refused by name, with the reason the scan gives. This is the same scan the
+  * engine runs at read time, so the case pins that the hook calls it rather
+  * than carrying a rule list of its own.
+  */
+data work.sascontroltable;
+  action='LOAD';
+  message='A refused function in VAR_LOGIC should abort';
+  libds="&dc_libref..MPE_DERIVED";
+  output;
+  stop;
+run;
+
+data work.jsdata;
+  length LIBREF $8 DSN $32 VAR_NM $32 VAR_DEPS $4000 VAR_TYPE $1
+    VAR_LOGIC $4000 VAR_FMT $32
+    _____DELETE__THIS__RECORD_____ $3;
+  LIBREF='DCTEST';
+  DSN='DC_DERIV';
+  VAR_NM='CALLER';
+  VAR_DEPS='';
+  VAR_TYPE='N';
+  VAR_LEN=8;
+  VAR_FMT='';
+  /* no semicolon in the payload: the single-expression rule is tested before
+    the function-name rule, so a quoted semicolon would abort on that first */
+  VAR_LOGIC='dosubl("data _null_")';
+  ACTIVE=1;
+  _____DELETE__THIS__RECORD_____='No';
+run;
+
+%mpe_execute(&_program,
+  viyacontext=&defaultcontext,
+  inputdatasets=work.sascontroltable work.jsdata,
+  outlib=web8,
+  outref=wb8,
+  mdebug=&sasjs_mdebug
+)
+
+/* the assertion compares the abort's MSG exactly, so the expected value is
+  assembled on one line: a literal wrapped across two source lines carries the
+  newline and the continuation indent with it */
+%let exp8=MPE_DERIVED: CALLER: VAR_LOGIC may not call a function that;
+%let exp8=&exp8 submits code or reaches the file system,;
+%let exp8=&exp8 the environment or the macro symbol table;
+
+%mpe_assert_abort(
+  desc=Checking a refused function in VAR_LOGIC is rejected,
+  outref=wb8,
+  msg=&exp8
+)
+
+/**
+  * Test 9 - the fields the hook owns rather than the scan: ACTIVE is a picker
+  * in the EDIT screen (HARDSELECT over the MPE_SELECTBOX pair), so a submission
+  * that bypasses the grid cannot land a value the screen would never offer.
+  */
+data work.sascontroltable;
+  action='LOAD';
+  message='An ACTIVE value the screen cannot offer should abort';
+  libds="&dc_libref..MPE_DERIVED";
+  output;
+  stop;
+run;
+
+data work.jsdata;
+  length LIBREF $8 DSN $32 VAR_NM $32 VAR_DEPS $4000 VAR_TYPE $1
+    VAR_LOGIC $4000 VAR_FMT $32
+    _____DELETE__THIS__RECORD_____ $3;
+  LIBREF='DCTEST';
+  DSN='DC_DERIV';
+  VAR_NM='BADACT';
+  VAR_DEPS='';
+  VAR_TYPE='N';
+  VAR_LEN=8;
+  VAR_FMT='';
+  VAR_LOGIC='1';
+  ACTIVE=2;
+  _____DELETE__THIS__RECORD_____='No';
+run;
+
+%mpe_execute(&_program,
+  viyacontext=&defaultcontext,
+  inputdatasets=work.sascontroltable work.jsdata,
+  outlib=web9,
+  outref=wb9,
+  mdebug=&sasjs_mdebug
+)
+
+%mpe_assert_abort(
+  desc=Checking an ACTIVE value outside the picker pair is rejected,
+  outref=wb9,
+  msg=MPE_DERIVED: BADACT: ACTIVE must be 1 or 0
+)
+
 /* tidy up so repeated runs start clean */
 proc sql noprint;
 delete from &dc_libref..mpe_derived

sas/sasjs/services/hooks/mpe_derived_postedit.sas

@@ -34,6 +34,7 @@
   <h4> SAS Macros </h4>
   @li mf_nobs.sas
   @li mp_abort.sas
+  @li mp_derivedvarcheck.sas
 
   <h4> Related Macros </h4>
   @li mpe_derived.sas
@@ -99,51 +100,43 @@ run;
 )
 
 /**
-  * Field level checks - the same rules %mpe_derived applies at read time
+  * Field level checks.  The rules for the derived variable itself - the name,
+  * the type, length, format, label and the expression - are core's, in
+  * mp_derivedvarcheck, so this hook and the read time engine (%mpe_derived ->
+  * mp_derivedvars) scan a row with the same rules and report the same reasons.
+  * What is added here are the two fields that belong to Data Controller rather
+  * than to the expression: the control table key, which the loader normalises
+  * but does not require, and ACTIVE, which the EDIT screen offers as a picker
+  * over the MPE_SELECTBOX pair - so a submission that bypasses the grid cannot
+  * land a value the screen would never offer.
   */
-data work.derived_check;
+data work.derived_dcfield(keep=name reason);
   set work.staging_ds;
-  length reason $200;
   if upcase(_____DELETE__THIS__RECORD_____)='YES' then delete;
+  length name $32 reason $200;
   call missing(reason);
-
+  name=upcase(strip(var_nm));
   if missing(libref) or missing(dsn) then reason='LIBREF and DSN are required';
-  else if var_nm='' then reason='VAR_NM is required';
+  else if missing(active) or active not in (0,1)
-  else if verify(strip(var_nm),'ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_')>0
+    then reason='ACTIVE must be 1 or 0';
-    then reason='VAR_NM must be a valid SAS name';
-  else if verify(substr(var_nm,1,1),'0123456789')=0
-    then reason='VAR_NM must not start with a digit';
-  else if var_nm='_____DELETE__THIS__RECORD_____'
-    then reason='VAR_NM is reserved';
-  else if var_type not in ('C','N')
-    then reason='VAR_TYPE must be C or N';
-  else if missing(var_len) or var_len ne int(var_len) or var_len<1
-    then reason='VAR_LEN must be a positive integer';
-  else if var_type='C' and var_len>32767
-    then reason='VAR_LEN exceeds 32767 for a character variable';
-  else if var_type='N' and var_len>8
-    then reason='VAR_LEN exceeds 8 for a numeric variable';
-  else if index(var_fmt,';')>0 or indexc(var_fmt,'&%')>0
-    or index(var_fmt,'/*')>0 or index(var_fmt,'*/')>0
-    then reason='VAR_FMT must be a format name, without ; & % or //*';
-  else if verify(strip(var_fmt),'ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_$.')>0
-    then reason='VAR_FMT must be a format name';
-  else if missing(var_logic)
-    then reason='VAR_LOGIC is required';
-  else if index(var_logic,';')>0
-    then reason='VAR_LOGIC must be a single expression - remove the ;';
-  else if indexc(var_logic,'&%')>0
-    then reason='VAR_LOGIC must not contain macro triggers';
-  else if index(var_logic,'/*')>0 or index(var_logic,'*/')>0
-    then reason='VAR_LOGIC must not contain comments';
-
   if not missing(reason) then output;
 run;
 
+%mp_derivedvarcheck(inds=work.staging_ds
+  ,outds=work.derived_field
+  ,reserved=%str(_____DELETE__THIS__RECORD_____)
+)
+
+/* the control table fields are reported before the expression's own rules, so
+  which of the two passes found a row does not change the message it gets */
+data work.derived_check;
+  set work.derived_dcfield work.derived_field;
+run;
+
 %if %mf_nobs(work.derived_check)>0 %then %do;
   data _null_;
     set work.derived_check(obs=1);
-    call symputx('der_errmsg',cats('MPE_DERIVED: ',var_nm,': ',reason));
+    call symputx('der_errmsg',cats('MPE_DERIVED: ',name,': ',reason));
     call symputx('der_errflag',1);
   run;
 %end;
@@ -443,7 +436,7 @@ run;
 
 %mp_abort(iftrue= (&cyc_found=1)
   ,mac=mpe_derived_postedit
-  ,msg=%str(VAR_DEPS contains a cycle: &cyc_path)
+  ,msg=%str(MPE_DERIVED: VAR_DEPS contains a cycle: &cyc_path)
 )
 
 %mend _mpe_derived_postedit;

sas/sasjs/services/public/getcols.test.sas

@@ -12,8 +12,9 @@
   assertion cannot pass against a service that does not validate.
 
   <h4> SAS Macros </h4>
+  @li mpe_assert_abort.sas
   @li mp_assert.sas
-  @li mx_execute.sas
+  @li mpe_execute.sas
   @li mf_getuniquefileref.sas
 
 **/
@@ -30,23 +31,19 @@ data _null_;
   put '%sysfunc(coalescec(&dclib..MPE_X_TEST,))';
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputfiles=&f1:iwant,
   outref=web1,
+  outstatus=work.ab1,
+  failfast=0,
   viyaresult=WEBOUT_TXT
 )
 
-%let abort1=0;
+%mpe_assert_abort(
-data _null_;
-  set work.results;
-  if state='canceled' then call symputx('abort1',1);
-run;
-
-%mp_assert(
-  iftrue=(&abort1=1),
   desc=Macro content in libds aborts the service,
-  outds=work.test_results
+  outref=web1,
+  msg=Invalid libds
 )
 
 /**
@@ -59,7 +56,7 @@ data _null_;
   put "&dclib..MPE_X_TEST";
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputfiles=&f2:iwant,
   outlib=web2

sas/sasjs/services/public/getcolvals.sas

@@ -36,7 +36,8 @@
   <h4> SAS Macros </h4>
   @li mf_existds.sas
   @li mf_nobs.sas
-  @li mf_verifymacvars.sas
+  @li mpeinit.sas
+  @li mpeterm.sas
   @li dc_assignlib.sas
   @li mf_getvarformat.sas
   @li mp_abort.sas
@@ -111,7 +112,10 @@ run;
 %let startrow=1;
 %let rows=4000;
 
-%mp_abort(iftrue= (%mf_verifymacvars(libds col2)=0)
+/* a request with no rows never reached the validation below, so this is the
+  only case that is missing inputs - an invalid value is a different abort, and
+  saying "missing" for it would hide the reason */
+%mp_abort(iftrue= (%mf_nobs(work.iwant)<=0)
   ,mac=&_program..sas
   ,msg=%str(Missing inputs from iwant)
 )

sas/sasjs/services/public/getcolvals.test.4.sas

@@ -13,8 +13,9 @@
   validate.
 
   <h4> SAS Macros </h4>
+  @li mpe_assert_abort.sas
   @li mp_assert.sas
-  @li mx_execute.sas
+  @li mpe_execute.sas
   @li mf_getuniquefileref.sas
 
 **/
@@ -27,27 +28,23 @@
 %let f1=%mf_getuniquefileref();
 data _null_;
   file &f1 termstr=crlf;
-  put 'LIBDS:$19. COL:$9.';
+  put 'LIBDS:$100. COL:$100.';
   put '%sysfunc(coalescec(&dclib..MPE_X_TEST,)),SOME_TIME';
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputfiles=&f1:iwant,
   outref=web1,
+  outstatus=work.ab1,
+  failfast=0,
   viyaresult=WEBOUT_TXT
 )
 
-%let abort1=0;
+%mpe_assert_abort(
-data _null_;
-  set work.results;
-  if state='canceled' then call symputx('abort1',1);
-run;
-
-%mp_assert(
-  iftrue=(&abort1=1),
   desc=Macro content in libds aborts the service,
-  outds=work.test_results
+  outref=web1,
+  msg=Invalid libds
 )
 
 /**
@@ -56,27 +53,23 @@ run;
 %let f2=%mf_getuniquefileref();
 data _null_;
   file &f2 termstr=crlf;
-  put 'LIBDS:$19. COL:$9.';
+  put 'LIBDS:$100. COL:$100.';
-  put '&dclib..MPE_X_TEST,%sysfunc(coalescec(SOME_TIME,))';
+  put "&dclib..MPE_X_TEST,%nrstr(%sysfunc(coalescec(SOME_TIME,)))";
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputfiles=&f2:iwant,
   outref=web2,
+  outstatus=work.ab2,
+  failfast=0,
   viyaresult=WEBOUT_TXT
 )
 
-%let abort2=0;
+%mpe_assert_abort(
-data _null_;
-  set work.results;
-  if state='canceled' then call symputx('abort2',1);
-run;
-
-%mp_assert(
-  iftrue=(&abort2=1),
   desc=Macro content in col aborts the service,
-  outds=work.test_results
+  outref=web2,
+  msg=Invalid col
 )
 
 /**
@@ -85,11 +78,11 @@ run;
 %let f3=%mf_getuniquefileref();
 data _null_;
   file &f3 termstr=crlf;
-  put 'LIBDS:$19. COL:$9.';
+  put 'LIBDS:$100. COL:$100.';
   put "&dclib..MPE_X_TEST,SOME_TIME";
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputfiles=&f3:iwant,
   outlib=web3

sas/sasjs/services/public/getddl.test.sas

@@ -15,7 +15,7 @@
 data work.params;
   length name $32 value $1000;
   name='libref';value="&dclib";output;
-  name='ds';value='';output;
+  name='ds';value='MPE_TABLES';output;
   name='flavour';value='TSQL';output;
 run;
 %mx_execute(&_program,
@@ -31,7 +31,7 @@ data work.results;
   input;
   putlog _infile_;
   if _n_=2 then do;
-    if _infile_=:'/* TSQL Flavour DDL for' then do;
+    if _infile_=:'/* TSQL Flavour DDL for' and index(_infile_,'MPE_TABLES')>0 then do;
       putlog 'test passed';
       output;
       output;
@@ -42,7 +42,7 @@ data work.results;
 run;
 
 %mp_assertdsobs(work.results,
-  desc=DDL file is successfully returned,
+  desc=DDL file is successfully returned for a table,
   test=EQUALS 2,
   outds=work.test_results
 )

sas/sasjs/services/public/getrawdata.sas

@@ -15,10 +15,10 @@
   @li mf_existfeature.sas
   @li dc_assignlib.sas
   @li mp_abort.sas
-  @li mp_binarycopy.sas
   @li mp_cntlout.sas
   @li mp_ds2cards.sas
   @li mp_ds2csv.sas
+  @li mp_ds2md.sas
   @li mp_streamfile.sas
   @li mp_validatecol.sas
   @li mf_nobs.sas

sas/sasjs/services/public/getrawdata.test.1.sas

@@ -13,8 +13,9 @@
   validate.
 
   <h4> SAS Macros </h4>
+  @li mpe_assert_abort.sas
   @li mp_assert.sas
-  @li mx_execute.sas
+  @li mpe_execute.sas
 
 **/
 
@@ -31,23 +32,19 @@ data work.params1;
   name='filter';value='0';output;
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputparams=work.params1,
   outref=web1,
+  outstatus=work.ab1,
+  failfast=0,
   viyaresult=WEBOUT_TXT
 )
 
-%let abort1=0;
+%mpe_assert_abort(
-data _null_;
-  set work.results;
-  if state='canceled' then call symputx('abort1',1);
-run;
-
-%mp_assert(
-  iftrue=(&abort1=1),
   desc=Macro content in table aborts the service,
-  outds=work.test_results
+  outref=web1,
+  msg=Invalid table
 )
 
 /**
@@ -60,23 +57,19 @@ data work.params2;
   name='filter';value='0 or 1=1';output;
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputparams=work.params2,
   outref=web2,
+  outstatus=work.ab2,
+  failfast=0,
   viyaresult=WEBOUT_TXT
 )
 
-%let abort2=0;
+%mpe_assert_abort(
-data _null_;
-  set work.results;
-  if state='canceled' then call symputx('abort2',1);
-run;
-
-%mp_assert(
-  iftrue=(&abort2=1),
   desc=SQL injection in filter aborts the service,
-  outds=work.test_results
+  outref=web2,
+  msg=Invalid filter
 )
 
 /**
@@ -91,23 +84,19 @@ data work.params3;
   name='filter';value='0';output;
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputparams=work.params3,
   outref=web3,
+  outstatus=work.ab3,
+  failfast=0,
   viyaresult=WEBOUT_TXT
 )
 
-%let abort3=0;
+%mpe_assert_abort(
-data _null_;
-  set work.results;
-  if state='canceled' then call symputx('abort3',1);
-run;
-
-%mp_assert(
-  iftrue=(&abort3=1),
   desc=Path traversal in table aborts the service,
-  outds=work.test_results
+  outref=web3,
+  msg=Invalid table
 )
 
 /**
@@ -120,7 +109,7 @@ data work.params4;
   name='filter';value='0';output;
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputparams=work.params4,
   outref=web4,

sas/sasjs/services/public/validatefilter.sas

@@ -78,13 +78,33 @@ run;
   outview=work.vw_derived,
   outvars=work.derived_vars
 )
-%let derived_nobs=%mf_nobs(work.derived_vars);
+/* A format catalog (LIBREF.CATALOGNAME-FC) must be left for mp_filterstore to
-%let filterds=%sysfunc(ifc(&derived_nobs>0,work.vw_derived,&ds));
+  rewrite into a cntlout dataset, and a -FC name handed over as targetds is not
+  something it can read - so pass no targetds at all in that case.  An empty
+  targetds makes the engine fall back to libds, which is what a plain dataset
+  wants anyway.
+
+  No macro triggers in this comment: the macro processor resolves them even
+  inside a comment, so prose about conditions is itself executed and the parser
+  reports "Expected DO not found" from here.  Same reason the values below are
+  plain LET statements with no trailing comma. */
+%let derived_nobs=%sysfunc(coalesce(%mf_nobs(work.derived_vars),0));
+%let is_fmtcat=%eval(%length(&ds)>3 and "%upcase(%substr(&ds,%length(&ds)-2,3))"="-FC");
+%let targetds_val=&ds;
+%if &derived_nobs>0 %then %do;
+  %let targetds_val=work.vw_derived;
+%end;
+/* the derived view keeps precedence: a catalog is only blanked when there is no
+  derived configuration to validate against.  Two plain %if blocks rather than
+  %else %if - that construct is what the parser rejected here before. */
+%if &derived_nobs=0 and &is_fmtcat=1 %then %do;
+  %let targetds_val=;
+%end;
 
 %mp_filterstore(
-  libds=&ds,
+  libds=&ds
-  targetds=&filterds,
+  ,targetds=&targetds_val
-  queryds=work.filterquery,
+  ,queryds=work.filterquery,
   filter_summary=&dc_libref..mpe_filteranytable,
   filter_detail=&dc_libref..mpe_filtersource,
   lock_table=&dc_libref..mpe_lockanytable,

sas/sasjs/services/public/validatefilter.test.1.sas

@@ -12,8 +12,9 @@
   assertion cannot pass against a service that does not validate.
 
   <h4> SAS Macros </h4>
+  @li mpe_assert_abort.sas
   @li mp_assert.sas
-  @li mx_execute.sas
+  @li mpe_execute.sas
   @li mf_getuniquefileref.sas
 
 **/
@@ -30,23 +31,19 @@ data _null_;
   put '%sysfunc(coalescec(&dclib..MPE_TABLES,))';
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputfiles=&f1:iwant,
   outref=web1,
+  outstatus=work.ab1,
+  failfast=0,
   viyaresult=WEBOUT_TXT
 )
 
-%let abort1=0;
+%mpe_assert_abort(
-data _null_;
-  set work.results;
-  if state='canceled' then call symputx('abort1',1);
-run;
-
-%mp_assert(
-  iftrue=(&abort1=1),
   desc=Macro content in filter_table aborts the service,
-  outds=work.test_results
+  outref=web1,
+  msg=Invalid filter_table
 )
 
 /**
@@ -71,7 +68,7 @@ AND,OR,2,DSN,=,"'MPE_LOCK_ANYTABLE'"
 ;;;;
 run;
 
-%mx_execute(&_program,
+%mpe_execute(&_program,
   viyacontext=&defaultcontext,
   inputfiles=&f2:iwant &f3:filterquery,
   outlib=web2

sas/sasjs/services/public/validatefilter.test.sas

@@ -5,7 +5,6 @@
   <h4> SAS Macros </h4>
   @li mx_execute.sas
   @li mp_assertdsobs.sas
-  @li mp_assertcolvals.sas
   @li mf_getuniquefileref.sas
 
 **/
@@ -34,8 +33,15 @@ run;
 %mx_execute(&_program,
   viyacontext=&defaultcontext,
   inputfiles=&f1:iwant &f2:filterquery,
-  outlib=web1
+  outlib=web1,
+  outref=web1ref,
+  mdebug=&sasjs_mdebug
 )
+data _null_;
+  infile web1ref;
+  input;
+  putlog _infile_;
+run;
 data result;
   set web1.result;
 run;
@@ -64,7 +70,8 @@ run;
   viyacontext=&defaultcontext,
   inputdatasets=work.iwant work.filterquery,
   outlib=web2,
-  outref=web2ref
+  outref=web2ref,
+  mdebug=&sasjs_mdebug
 )
 data _null_;
   infile web2ref;
@@ -119,7 +126,8 @@ run;
   viyacontext=&defaultcontext,
   inputdatasets=work.iwant work.filterquery,
   outlib=web3,
-  outref=web3ref
+  outref=web3ref,
+  mdebug=&sasjs_mdebug
 )
 data _null_;
   infile web3ref;

sas/sasjs/services/public/viewdata.sas

@@ -55,7 +55,6 @@
   @li mp_cntlout.sas
   @li mp_getcols.sas
   @li mp_getpk.sas
-  @li mp_jsonout.sas
   @li mp_searchdata.sas
   @li mp_validatecol.sas
   @li mpe_columnlevelsecurity.sas

sas/sasjs/services/public/viewdata.test.3.sas

@@ -57,7 +57,7 @@ data _null_;
 run;
 %mp_assert(
   iftrue=(&nobs=&expected),
-  desc=&desc (&searchtype "%superq(searchval)" returned &nobs rows, expected &expected),
+  desc=%str(&desc (&searchtype "%superq(searchval)" returned &nobs rows, expected &expected)),
   outds=work.test_results
 )
 %mend search;
@@ -118,6 +118,6 @@ proc sql noprint;
 quit;
 %mp_assert(
   iftrue=(&caprows=&capnobs and &capnobs gt 0 and &capnobs le &capmax),
-  desc=Checking a search beyond the cap returns as many rows as it reports (&capnobs reported, &caprows returned, cap &capmax),
+  desc=%str(Checking a search beyond the cap returns as many rows as it reports (&capnobs reported, &caprows returned, cap &capmax)),
   outds=work.test_results
 )

sas/sasjs/targets/sas9/macros_meta/dc_getlibs.sas

@@ -44,4 +44,12 @@ run;
 
 options metarepository=&repo;
 
+%if "&sasjs_mdebug"="1" %then %do;
+  data _null_;
+    set &outds;
+    putlog "NOTE: &sysmacroname: &outds:";
+    putlog (_all_)(=);
+  run;
+%end;
+
 %mend dc_getlibs;

sas/sasjs/targets/server/macros_server/dc_getlibs.sas

@@ -20,7 +20,15 @@ create table &outds as
     ,engine
     ,'' as libraryid length=17
   from dictionary.libnames
-  where libname not in ('WORK','SASUSER');
+  where upcase(libname) not in ("WORK","SASUSER","%upcase(&dc_libref)");
 insert into &syslast values ("&DC_LIBREF", "&DC_LIBNAME",'','V9');
 
+%if "&sasjs_mdebug"="1" %then %do;
+  data _null_;
+    set &outds;
+    putlog "NOTE: &sysmacroname: &outds:";
+    putlog (_all_)(=);
+  run;
+%end;
+
 %mend dc_getlibs;

sas/sasjs/targets/server/macros_server/dc_refreshcatalog.sas

@@ -6,6 +6,11 @@
 
   The MPELIB should be pre-assigned
 
+  The refresh macros are called from the statements this macro writes into a
+  catalog, so they are not visible as calls to lint.
+
+  @sasjslint {"noUnusedMacros": false}
+
   <h4> SAS Macros </h4>
   @li mpe_refreshlibs.sas
   @li dc_assignlib.sas
@@ -32,12 +37,17 @@ data libraries;
   where also libref="&libref";
 %end;
   file executor;
-  str=cats('%mpe_refreshtables(',libref,')');
+  str=cats('%mpe_refreshtables(',libref,');');
   put str;
   putlog str;
-  str=cats('%mpe_refreshcatalogs(',libref,')');
+  str=cats('%mpe_refreshcatalogs(',libref,');');
   put str;
 run;
-%inc executor/source2;
+%if "&sasjs_mdebug"="1" %then %do;
+  /* echo the generated statements, so a failure inside one of them can be
+    traced back to the libref that produced it */
+  %inc executor/source2;
+%end;
+%else %inc executor;
 
 %mend dc_refreshcatalog;

sas/sasjs/targets/server/services_server/admin/makedata.sas

@@ -6,8 +6,12 @@
     It also adds the STAGING directory as subdirectory to the BASE
     library location.
 
-  @warning This STP self destructs! It will delete itself after a successful run
+  @warning This STP self destructs! It will delete itself (and the
-    to avoid being executed twice (and overwriting actual data)
+    configurator) after a successful run, to avoid being executed twice and
+    overwriting actual data.  Pass a KEEPME column in the input data to keep
+    them - test harnesses reseed before every run and cannot redeploy first.
+
+  @sasjslint {"noBannedStrings": false}
 
   <h4> SAS Macros </h4>
   @li mf_getapploc.sas
@@ -55,6 +59,24 @@ data _null_;
 run;
 %end;
 
+/* optional KEEPME column - keep this program and the configurator on the drive
+  * after a successful run.  A normal install self-destructs so the seeding
+  * cannot be run twice over live data; a test harness reseeds before every run
+  * and wants them to stay put. */
+%let keepme=0;
+%if &exist=1 %then %do;
+data _null_;
+  dsid=open('work.fromjs');
+  /* every row, not just the first: KEEPME may be set on any of them */
+  if varnum(dsid,'KEEPME')>0 then do until(eof);
+    set work.fromjs end=eof;
+    if upcase(cats(keepme)) in ('1','Y','YES','TRUE') then call symputx('keepme',1);
+  end;
+  rc=close(dsid);
+  stop;
+run;
+%end;
+
 %mp_abort(iftrue= (&syscc ne 0)
   ,mac=&_program
   ,msg=%str(Issue on makedata entry)
@@ -191,7 +213,15 @@ data _null_;
 run;
 %end;
 
-/* We ran successfully, now remove configurator and makedata STPs */
+/* We ran successfully, now remove configurator and makedata STPs, unless the
-%ms_deletefile(&root/services/admin/configurator.sas)
+  * caller asked to keep them (KEEPME in the input data).  Keeping them is for
-%ms_deletefile(&_program..sas)
+  * test harnesses, which reseed before every run and cannot redeploy first. */
+%if &keepme ne 1 %then %do;
+  %ms_deletefile(&root/services/admin/configurator.sas);
+  %ms_deletefile(&_program..sas);
+%end;
+%else %do;
+  %put NOTE: &_program retained, as KEEPME was requested;
+%end;
+
 

sas/sasjs/targets/viya/macros_viya/dc_getlibs.sas

@@ -21,7 +21,15 @@ create table &outds as
     ,engine
     ,'' as libraryid length=17
   from dictionary.libnames
-  where libname not in ('WORK','SASUSER','CASUSER');
+  where upcase(libname) not in ("WORK","SASUSER","CASUSER","%upcase(&dc_libref)");
 insert into &syslast values ("&DC_LIBREF", "&DC_LIBNAME",'','V9');
 
+%if "&sasjs_mdebug"="1" %then %do;
+  data _null_;
+    set &outds;
+    putlog "NOTE: &sysmacroname: &outds:";
+    putlog (_all_)(=);
+  run;
+%end;
+
 %mend dc_getlibs;

sas/sasjs/targets/viya/macros_viya/dc_refreshcatalog.sas

@@ -6,6 +6,11 @@
 
   The MPELIB should be pre-assigned
 
+  The refresh macros are called from the statements this macro writes into a
+  catalog, so they are not visible as calls to lint.
+
+  @sasjslint {"noUnusedMacros": false}
+
   <h4> SAS Macros </h4>
   @li mpe_refreshlibs.sas
   @li dc_assignlib.sas
@@ -36,13 +41,18 @@
     set &mpelib..mpe_datacatalog_libs;
     where &dc_dttmtfmt. le TX_TO;
     file executor;
-    str=cats('%mpe_refreshtables(',libref,')');
+    str=cats('%mpe_refreshtables(',libref,');');
     put str;
-    str=cats('%mpe_refreshcatalogs(',libref,')');
+    str=cats('%mpe_refreshcatalogs(',libref,');');
     put str;
     putlog str;
   run;
-  %inc executor;
+  %if "&sasjs_mdebug"="1" %then %do;
+    /* echo the generated statements, so a failure inside one of them can be
+      traced back to the libref that produced it */
+    %inc executor/source2;
+  %end;
+  %else %inc executor;
 %end;
 
 %mend dc_refreshcatalog;