Compare commits
73
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
97b4cd4d61 | ||
|
|
dd25be8e75 | ||
|
|
2e40749052 | ||
|
|
fc687f8643 | ||
|
|
c97e8866db | ||
|
|
ad061946c3 | ||
|
|
914a1606bb | ||
|
|
c803773d3c | ||
|
|
b8eea5063c | ||
|
|
0a172b0c37 | ||
|
|
888873421c | ||
|
|
af1813f829 | ||
|
|
27686d3aad | ||
|
|
d84a1d50f4 | ||
|
|
cf7b21a8b6 | ||
|
|
60589c78b6 | ||
|
|
8e2cce4150 | ||
|
|
f7e753863b | ||
|
|
febfea3e5e | ||
|
|
3580d6e9eb | ||
|
|
bc9e3ae50f | ||
|
|
330448c496 | ||
|
|
00257f9160 | ||
|
|
e51953ca8a | ||
|
|
aa91daf9f7 | ||
|
|
00df87f189 | ||
|
|
65699bdd39 | ||
|
|
459ff2da37 | ||
|
|
c0ea810766 | ||
|
|
62e43034e4 | ||
|
|
80e8911af8 | ||
|
|
bd3c220af2 | ||
|
|
5e8910f238 | ||
|
|
221dd8a15e | ||
|
|
e9216b0895 | ||
|
|
37d81c902a | ||
|
|
689b5b6709 | ||
|
|
10157c1159 | ||
|
|
541a61ba08 | ||
|
|
ad50acf647 | ||
|
|
cd1ce4a7e2 | ||
|
|
7e3045a21b | ||
|
|
88010c3980 | ||
|
|
c5808a16eb | ||
|
|
0011f7e933 | ||
|
|
4ed1e0684d | ||
|
|
2623a3fb4b | ||
|
|
388f418bf4 | ||
|
|
f83dc359e2 | ||
|
|
411f6c3354 | ||
|
|
f042ca82ea | ||
|
|
15a1f34d52 | ||
|
|
6486c80ed4 | ||
|
|
dd482c4069 | ||
|
|
f58925b299 | ||
|
|
25ffa71add | ||
|
|
424b111737 | ||
|
|
849153d4a2 | ||
|
|
bc9e0996ec | ||
|
|
66039c91d7 | ||
|
|
abb1ecbcab | ||
|
|
337ea8e5e0 | ||
|
|
709acfad73 | ||
|
|
42355f17bc | ||
|
|
6ef2eec011 | ||
|
|
1ef3e0a97b | ||
|
|
56b6620ef4 | ||
|
|
fd272eec89 | ||
|
|
c463a4e03c | ||
|
|
23015ac41f | ||
|
|
a8f62bdfe2 | ||
|
|
355c46a7b6 | ||
|
|
661ee19cfa |
No files matched your search
@@ -1,10 +1,11 @@
|
||||
# Testing
|
||||
|
||||
Backend (SAS) tests are run with the sasjs CLI from the `sas/` directory.
|
||||
Backend (SAS) tests are run with the sasjs CLI from the `sas/` directory. For a deploy-and-run on the shared state see the `dc-estate-testing` skill (`.agents/skills/dc-estate-testing/SKILL.md`), which covers the credentials, the `makedata` seed the suite needs and the estate's group name.
|
||||
|
||||
## Commands
|
||||
|
||||
- Full deploy + test cycle: `npm run 4gl` (compiles, deploys to the 4gl target, runs makedata) then `sasjs test -t 4gl`.
|
||||
- The shared estate under its `server` target: `npm run server` (compile, deploy, seed) then `sasjs test -t server`.
|
||||
- Run tests only (no rebuild/redeploy): `sasjs test -t 4gl`.
|
||||
- Run a subset of tests: `sasjs test -t 4gl SOMESTRING` — the positional argument filters to matching tests (e.g. `sasjs test -t 4gl stagedata.test.3`).
|
||||
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
---
|
||||
name: dc-estate-testing
|
||||
description: >
|
||||
Use when deploying Data Controller to a shared SASjs Server estate (sas.4gl.io) or running the SAS
|
||||
test suite against it. Covers the target and credential setup, the makedata seed the suite needs on
|
||||
a fresh appLoc and the group it must name, how to read the results, and how to deploy the frontend
|
||||
so the streamed app is the build you just made.
|
||||
---
|
||||
|
||||
# Deploying and testing on the shared SASjs Server estate
|
||||
|
||||
The whole cycle is two npm scripts plus `sasjs test`. Everything below explains what those scripts do and the steps that go wrong.
|
||||
|
||||
cd sas && npm ci # macro dependencies
|
||||
cd ../client && npm ci && npm run build # frontend only - see "Deploying the frontend"
|
||||
cd ../sas && npm run server # cpfavicon + sasjs cbd -t server + makedata
|
||||
npx sasjs test -t server
|
||||
|
||||
`npm run server` is `sasjs cbd -t server && npm run serverdata`, and the seed in the second half is not optional. `sasjs cbd` on its own deploys the services and the test programs but leaves the appLoc with no Data Controller configuration, and `testsetup` reads `<appLoc>/services/public/settings.sas` through the drive API. `makedata` is what creates that file, so without the seed `testsetup` gets a 404, aborts, and 59 of the 64 test programs fail to complete on setup alone - which reads like a broken deployment rather than a missing seed.
|
||||
|
||||
## The targets
|
||||
|
||||
`server` is the shared estate: `serverType: SASJS` on https://sas.4gl.io, `appLoc` `/Public/app/dc`, a real SAS 9 binary behind SASjs Server (`GET /SASjsApi/info` reports `mode: server`, `runTimes: ["sas"]`, `localLoginEnabled: true`). `4gl` is the same estate with a slightly different config, hence the older `npm run 4gl` / `4glmakedata` pair.
|
||||
|
||||
A run against `server` exercises the SASjs Server branch of every macro and the SAS 9 backend, and nothing else: no Viya branch, no SAS 9 STP branch. Say that when reporting, rather than implying the matrix is covered.
|
||||
|
||||
To deploy your own branch without replacing the app everyone else uses, add a target that copies `server` and changes `appLoc` and `streamServiceName` (for example `/Public/app/<you>/dc`), keep it out of the commit, and pass it with `-t`. `sasjs cbd` overwrites the whole appLoc it is pointed at.
|
||||
|
||||
## Credentials
|
||||
|
||||
The CLI logs in with a registered client, not a username and password: `.env.<target>` (gitignored, mode 600) needs `CLIENT`, `ACCESS_TOKEN` and `REFRESH_TOKEN`. With `CLIENT` alone it prints a `/SASjsLogon?client_id=...` URL and waits for an authorization code, which cannot be answered headless.
|
||||
|
||||
Mint the pair from a local login, with the password read from a file rather than argv:
|
||||
|
||||
1. `GET /` with a cookie jar - it sets `XSRF-TOKEN`, which every POST must echo in an `X-XSRF-TOKEN` header.
|
||||
2. `POST /SASLogon/login` with `{"username","password"}`.
|
||||
3. `GET /SASjsApi/client` on the same session - the registered client ids.
|
||||
4. `POST /SASLogon/authorize` with `{"clientId"}` -> `{"code"}`.
|
||||
5. `POST /SASjsApi/auth/token` with `{"clientId","code"}` -> `accessToken` / `refreshToken`.
|
||||
|
||||
An unmatched `/SASjsApi/...` path answers `200` with the Swagger page, so check the content type; HTML means the route does not exist. Delete the token file when the run is over.
|
||||
|
||||
## The makedata seed
|
||||
|
||||
`npm run serverdata` sends `deploy/makeDataServer.json`: one `fromjs` row with `ADMIN`, `DCPATH` and `OUTFMT`. Two things about it are worth knowing before you trust its result.
|
||||
|
||||
- **`ADMIN` must name a group the calling account is actually in.** The shipped value is `AllUsers`, which is not a group on the shared estate - the account belongs to `all-users` (check with a call to `services/public/getgroups`, or read the `MF_GETUSER` the response echoes back). With a group the account is not in, the seed still answers `SUCCESS` and every later `getdata`/`stagedata` call aborts with `<user> is not authorised to edit <lib>.<ds> in the <lib>.MPE_SECURITY table`. Override `ADMIN` to `all-users` for the estate rather than editing the shipped JSON.
|
||||
- **Each run creates a new library.** The response carries `STATUS`, `DCPATH` and `LIBREF` (for example `DC621835`), and the LIBREF is clock-derived, so a libref from an earlier run answers `dataset <lib>.<table> does not exist` - take the current one from the response.
|
||||
- **`KEEPME` controls whether the seed can be repeated.** Without it `makedata` deletes itself and the configurator after a successful run (deliberately, so it cannot be executed twice over real data), and a second seed fails with `The Program at (<appLoc>/services/admin/makedata) does not exist` until you run `sasjs cbd` again. A harness that reseeds between runs sets `KEEPME` in the input row.
|
||||
- **Seed once per deploy, not before every test run.** The suite builds its own `dctest` library from the seeded control tables; reseeding in between is wasted work, and there is no need to run `makedata` and `sasjs test` in the opposite order.
|
||||
|
||||
## Reading the results
|
||||
|
||||
Results land in `sas/sasjsresults/`: `testResults.json` (the counts worth quoting: `tests_with_results`, `tests_that_pass`, `failed_to_complete`, `completed_with_failures`), `testResults.csv` (one row per assertion, lower-case headers `test_target`, `test_loc`, `sasjs_test_id`, `test_suite_result`, `test_description`, `test_url`), and one SAS log per program under `logs/`. The CLI's closing lines are not the source of truth when they disagree with the JSON.
|
||||
|
||||
A current green run is 64 of 64 test programs with 241 assertions passing and nothing failing to complete.
|
||||
|
||||
`ERROR` lines in the per-program logs are expected and are not a failed run: the specs that assert a refusal get `ERROR: Execution canceled by an %ABORT CANCEL statement`, and the ones that feed bad payloads get the rejection message they are asserting on (`Invalid libds:...`, `Invalid libref:...`). Judge the run by the JSON counts, then by whether the errors you find are the ones the tests asked for.
|
||||
|
||||
## Deploying the frontend
|
||||
|
||||
The streamed web app is a separate artefact and `sasjs cbd` only streams it if `client/dist` exists:
|
||||
|
||||
1. `client/libraries/sheet-crypto.tgz.gpg` is an encrypted dependency (`file:libraries/sheet-crypto.tgz` in `client/package.json`) that is not on the public registry. Decrypt it with the passphrase the maintainer holds: `printf '%s' "$SHEET_PWD" | gpg --batch --yes --passphrase-fd 0 --output ./libraries/sheet-crypto.tgz --decrypt ./libraries/sheet-crypto.tgz.gpg`.
|
||||
2. `client/.npmrc` is gitignored and needs `legacy-peer-deps=true` (the committed lock is written with it) plus the `@sheet` registry lines.
|
||||
3. `cd client && npm ci && npm run build`.
|
||||
4. The build carries the repo default appLoc, so point it at the deployment before deploying: rewrite `apploc="/Public/app/dc"` in `dist/index.html` to the target appLoc.
|
||||
5. `cd ../sas && npx sasjs cbd -t <target>` - with `client/dist` present the log says `Copying web app files` and `Web app is available at <url>`.
|
||||
|
||||
Without step 1 the deploy prints `ERROR An error has occurred when compiling web app services ... webSourcePath: .../client/dist doesn't exist` and then deploys the services anyway, so the deploy looks successful while the app streams nothing. The AppStream URL is session-protected: an anonymous `curl` gets a `401`, so verify it with a logged-in cookie jar and fetch the `main.<hash>.js` the page names.
|
||||
|
||||
Never commit `client/.npmrc` or the decrypted tarball; both are ignored, and `git status` staying clean is the check.
|
||||
|
||||
## Reading a failed run
|
||||
|
||||
- `ERROR An error occurred while executing the request. Job location: <appLoc>/tests/...` means the response was not JSON, so the program broke before it could write results. Open `sasjsresults/logs/<suite>.log` and read the first `ERROR`; a single compile break cascades into hundreds of `ERROR 180-322: Statement is not valid` lines and the tail is noise.
|
||||
- `WARNING: Apparent symbolic reference TESTLOC not resolved` in `testsetup.log`, next to a `404 Not Found` for `services/public/settings.sas`, is the missing seed described above, not a macro problem.
|
||||
- `sasjs test` runs the DEPLOYED program, so after editing a service, hook or test, run `sasjs cbd` again before testing or you will debug code that is no longer in the tree.
|
||||
- A deployment can succeed and still be missing its web app, its data or its configuration - check the thing you changed rather than the exit code.
|
||||
Executable
+49
@@ -0,0 +1,49 @@
|
||||
#!/usr/bin/env bash
|
||||
# The Cypress specs CI runs, and the order they run in.
|
||||
#
|
||||
# ORDER MATTERS. These specs share one mock estate that is seeded once per job,
|
||||
# so several of them depend on the state they find:
|
||||
#
|
||||
# - csv-limited asserts the free-tier "Locked Feature" modal, which an earlier
|
||||
# spec can remove by planting a valid licence. It has always run first.
|
||||
# - hook-programs asserts that an edit raises an approval row against its
|
||||
# target table; it runs early, before other specs have added rows to the
|
||||
# approval tables.
|
||||
#
|
||||
# Specs added later are appended rather than inserted, so the sequence that was
|
||||
# known green keeps its shape. metanav is deliberately absent - see
|
||||
# https://git.datacontroller.io/dc/dc/issues/395.
|
||||
#
|
||||
# download is held out for a different reason - its download button is covered by
|
||||
# a fixed-position element, see
|
||||
# https://git.datacontroller.io/dc/dc/issues/398.
|
||||
set -euo pipefail
|
||||
|
||||
SPECS="csv-limited hook-programs liveness editor excel-multi-load excel csv filtering filter-panel licensing viewer-labels full-table-search viewbox stage browser-info ddl-export row-cell-limits derived-variables startupservice-launcher coltype-delete-record submitted-details embed-va"
|
||||
|
||||
# Specs that exist but are held out of CI, each with the reason. Listed here so
|
||||
# the coverage check below can tell a deliberate exclusion from a forgotten file.
|
||||
EXCLUDED="metanav download"
|
||||
|
||||
case "${1:-}" in
|
||||
all)
|
||||
# comma separated, the form `cypress run --spec` takes
|
||||
for name in $SPECS; do printf 'cypress/e2e/%s.cy.ts,' "$name"; done | sed 's/,$//'
|
||||
;;
|
||||
coverage)
|
||||
on_disk=$(cd "$2/client" && ls cypress/e2e/*.cy.ts | xargs -n1 basename | sed 's/\.cy\.ts$//' | sort)
|
||||
listed=$(printf '%s\n' $SPECS $EXCLUDED | sort)
|
||||
missing=$(comm -23 <(echo "$on_disk") <(echo "$listed"))
|
||||
if [ -n "$missing" ]; then
|
||||
echo "Specs that are neither run nor explicitly excluded:"
|
||||
echo "$missing"
|
||||
echo "Add them to SPECS (or EXCLUDED, with a reason) in .gitea/scripts/cypress-specs.sh"
|
||||
exit 1
|
||||
fi
|
||||
echo "All $(echo "$on_disk" | wc -l) specs are accounted for: $(echo $SPECS | wc -w) run, $(echo $EXCLUDED | wc -w) excluded."
|
||||
;;
|
||||
*)
|
||||
echo "usage: $0 {all|coverage <repo-root>}" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
@@ -75,11 +75,33 @@ jobs:
|
||||
cd client
|
||||
npm run test:headless
|
||||
|
||||
- name: Angular Tests in other timezones
|
||||
# The VA filter clauses are built from local calendar dates, so a
|
||||
# UTC-only run cannot see a date that slips a day west of UTC or east of
|
||||
# it. Only the VA specs carry that property, so both scripts run that
|
||||
# subset rather than the whole suite a second and third time.
|
||||
run: |
|
||||
cd client
|
||||
npm run test:tz-west
|
||||
npm run test:tz-east
|
||||
|
||||
- name: Every Cypress spec is in the CI list
|
||||
# The spec list used to be spelled out inline in --spec, and five files
|
||||
# quietly fell out of it - among them the VA embed suite. Fails when a
|
||||
# spec on disk is not listed.
|
||||
run: bash .gitea/scripts/cypress-specs.sh coverage .
|
||||
|
||||
- name: Production Build
|
||||
run: |
|
||||
cd client
|
||||
npm run build
|
||||
|
||||
|
||||
# Cypress shard 1 of 4. The setup repeats in every shard because each
|
||||
# one gets its own runner and its own SASjs server; that fixed cost is what
|
||||
# limits how far sharding helps. The specs it runs come from
|
||||
# .gitea/scripts/cypress-specs.sh.
|
||||
|
||||
Build-and-test-development:
|
||||
runs-on: ubuntu-latest
|
||||
needs: Build-and-ng-test
|
||||
@@ -182,9 +204,20 @@ jobs:
|
||||
sasjs request services/admin/makedata -t server-ci -d deploy/makedata.json -o ./makedata_out.json || true
|
||||
|
||||
- name: Prepare and run frontend and cypress
|
||||
# One job, one SASjs server, one ng serve, one Chrome: four of each on one
|
||||
# runner starved the app under test and every failure in that run was a
|
||||
# timeout, not an assertion.
|
||||
timeout-minutes: 50
|
||||
run: |
|
||||
cd ./client
|
||||
# The spec list lives in one place, and an empty list is a broken
|
||||
# workflow rather than a green run with no tests.
|
||||
SPECS="$(bash ../.gitea/scripts/cypress-specs.sh all)"
|
||||
if [ -z "$SPECS" ]; then
|
||||
echo "the spec list resolved to nothing"
|
||||
exit 1
|
||||
fi
|
||||
echo "specs: $SPECS"
|
||||
mv ./cypress.env.example.json ./cypress.env.json
|
||||
replace-in-files --regex='"username".*' --replacement='"username":"'${{ secrets.CYPRESS_USERNAME_SASJS }}'",' ./cypress.env.json
|
||||
replace-in-files --regex='"password".*' --replacement='"password":"'${{ secrets.CYPRESS_PWD_SASJS }}'" ' ./cypress.env.json
|
||||
@@ -199,16 +232,22 @@ jobs:
|
||||
# Start frontend and run cypress
|
||||
# timeout 2700: SIGTERM after 45 min so Cypress can flush video/screenshots
|
||||
# before the outer timeout-minutes hard-kills the step (avoids silent multi-hour hangs)
|
||||
npx ng serve --host 0.0.0.0 --port 4200 & npx wait-on http://localhost:4200 && timeout 2700 npx cypress run --browser chrome --spec "cypress/e2e/csv-limited.cy.ts,cypress/e2e/hook-programs.cy.ts,cypress/e2e/liveness.cy.ts,cypress/e2e/editor.cy.ts,cypress/e2e/excel-multi-load.cy.ts,cypress/e2e/excel.cy.ts,cypress/e2e/csv.cy.ts,cypress/e2e/filtering.cy.ts,cypress/e2e/filter-panel.cy.ts,cypress/e2e/licensing.cy.ts,cypress/e2e/viewer-labels.cy.ts,cypress/e2e/full-table-search.cy.ts,cypress/e2e/viewbox.cy.ts,cypress/e2e/stage.cy.ts,cypress/e2e/browser-info.cy.ts,cypress/e2e/ddl-export.cy.ts,cypress/e2e/row-cell-limits.cy.ts,cypress/e2e/derived-variables.cy.ts,cypress/e2e/startupservice-launcher.cy.ts"
|
||||
npx ng serve --host 0.0.0.0 --port 4200 & npx wait-on http://localhost:4200 && timeout 2700 npx cypress run --browser chrome --spec "$SPECS"
|
||||
|
||||
- name: Zip Cypress videos
|
||||
if: always()
|
||||
# Only on a failure: a green run has nothing to watch, and zipping and
|
||||
# uploading every run's videos is the part of the video cost that lands
|
||||
# on the clock.
|
||||
if: failure()
|
||||
run: |
|
||||
mkdir -p ./client/cypress/videos
|
||||
zip -r cypress-videos ./client/cypress/videos
|
||||
|
||||
- name: Add cypress videos artifacts
|
||||
if: always()
|
||||
if: failure()
|
||||
# v3, not v4: this runner reports "GHESNotSupportedError:
|
||||
# @actions/artifact v2.0.0+, upload-artifact@v4+ ... not supported on
|
||||
# GHES", so the current major cannot upload here at all.
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
name: cypress-videos.zip
|
||||
|
||||
@@ -0,0 +1,121 @@
|
||||
name: Publish code documentation
|
||||
run-name: Publishing typedoc and SAS docs
|
||||
|
||||
# The code documentation sites are published here rather than inline in the
|
||||
# release job, so that they can be republished on their own - after a secret
|
||||
# rotation, or for a release whose artefact upload failed (the release job
|
||||
# cannot simply be re-run: semantic-release refuses once the tag exists).
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
NODE_VERSION: '24.5.0'
|
||||
|
||||
jobs:
|
||||
typedoc:
|
||||
runs-on: ubuntu-latest
|
||||
# One deploy per site at a time: `surfer deploy` swaps the site atomically,
|
||||
# so two runs racing is not corruption, but it is a wasted build and the
|
||||
# last swap wins.
|
||||
concurrency:
|
||||
group: surfer-webdoc
|
||||
cancel-in-progress: false
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: ${{ env.NODE_VERSION }}
|
||||
|
||||
- name: Write .npmrc file
|
||||
run: |
|
||||
printf '%s\n' "$NPMRC" >> client/.npmrc
|
||||
echo "legacy-peer-deps=true" >> client/.npmrc
|
||||
shell: bash
|
||||
env:
|
||||
NPMRC: ${{ secrets.NPMRC }}
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
cd client
|
||||
# Decrypt and Install sheet
|
||||
echo "$SHEET_PWD" | gpg --batch --yes --passphrase-fd 0 ./libraries/sheet-crypto.tgz.gpg
|
||||
npm ci
|
||||
shell: bash
|
||||
env:
|
||||
SHEET_PWD: ${{ secrets.SHEET_PWD }}
|
||||
|
||||
- name: Build typedoc
|
||||
run: |
|
||||
cd client
|
||||
npm run compodoc:build
|
||||
|
||||
# Deploy tooling is pinned to an exact version: an unpinned `latest` would
|
||||
# let a breaking or compromised upstream release publish straight to the
|
||||
# live site. `deploy` (v7) replaces the site atomically, and it takes the
|
||||
# Cloudron username plus an app password - the v6 `put --token` flow this
|
||||
# replaces is gone.
|
||||
#
|
||||
# webdoc is a NAMED site on the app whose default site is the marketing
|
||||
# site, so the name and its alias domain are both passed. Surfer picks
|
||||
# the target only from `--site-name` (it travels as the Surfer-Deployment
|
||||
# header); without it `deploy` replaces the app's DEFAULT site, which is
|
||||
# what serves datacontroller.io - so an unnamed deploy published the
|
||||
# Typedoc over the top of the marketing site instead of to webdoc.
|
||||
- name: Deploy typedoc
|
||||
env:
|
||||
SURFER_PASSWORD: ${{ secrets.WEBDOC_DATACONTROLLER_IO }}
|
||||
REF_NAME: ${{ gitea.ref_name }}
|
||||
COMMIT_SHA: ${{ gitea.sha }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
npx --yes @cloudron/surfer@7.2.0 deploy client/documentation \
|
||||
--server webdoc.datacontroller.io \
|
||||
--site-name webdoc \
|
||||
--domain webdoc.datacontroller.io \
|
||||
--username allan \
|
||||
--password "$SURFER_PASSWORD" \
|
||||
--message "Typedoc from ${REF_NAME} @ ${COMMIT_SHA:0:8}"
|
||||
|
||||
sas-docs:
|
||||
runs-on: ubuntu-latest
|
||||
concurrency:
|
||||
group: surfer-code-datacontroller
|
||||
cancel-in-progress: false
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: ${{ env.NODE_VERSION }}
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
# `sasjs doc` shells out to doxygen
|
||||
apt-get update
|
||||
apt-get install -y doxygen
|
||||
cd sas
|
||||
npm ci
|
||||
CLI_VERSION=$(node -p "require('./package.json').dependencies['@sasjs/cli']")
|
||||
npm i -g "@sasjs/cli@${CLI_VERSION}"
|
||||
|
||||
- name: Build SAS docs
|
||||
run: |
|
||||
cd sas
|
||||
sasjs doc
|
||||
|
||||
# No `--site-name` here on purpose: code.datacontroller.io is its own app
|
||||
# and the SAS docs are its DEFAULT site, so naming one would create a
|
||||
# separate site and leave the served copy stale. Do not copy the named
|
||||
# form from the typedoc job above - that app's default site is the
|
||||
# marketing site, which is why it must not be the target.
|
||||
- name: Deploy SAS docs
|
||||
env:
|
||||
SURFER_PASSWORD: ${{ secrets.CODE_DATACONTROLLER_IO }}
|
||||
REF_NAME: ${{ gitea.ref_name }}
|
||||
COMMIT_SHA: ${{ gitea.sha }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
npx --yes @cloudron/surfer@7.2.0 deploy sas/sasjsbuild/sasdocs \
|
||||
--server code.datacontroller.io \
|
||||
--username allan \
|
||||
--password "$SURFER_PASSWORD" \
|
||||
--message "SAS docs from ${REF_NAME} @ ${COMMIT_SHA:0:8}"
|
||||
@@ -0,0 +1,209 @@
|
||||
name: Release assets
|
||||
run-name: Building and uploading the release assets
|
||||
|
||||
# Builds the downloadable artefacts for an existing release and uploads them.
|
||||
#
|
||||
# The release job cuts the tag before it builds, so a failure anywhere between
|
||||
# the semantic-release step and the asset upload leaves a published release with
|
||||
# no files - and that job cannot simply be re-run, because semantic-release
|
||||
# refuses once the tag exists. This workflow rebuilds the artefacts from the tag
|
||||
# and attaches them, so the release can be completed without cutting a new one.
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
tag:
|
||||
description: 'Release tag to build and upload (blank = the latest release)'
|
||||
required: false
|
||||
default: ''
|
||||
|
||||
env:
|
||||
NODE_VERSION: '24.5.0'
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
|
||||
jobs:
|
||||
assets:
|
||||
runs-on: ubuntu-latest
|
||||
# Two runs uploading the same asset names would race the delete-then-upload
|
||||
# below.
|
||||
concurrency:
|
||||
group: release-assets
|
||||
cancel-in-progress: false
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: ${{ env.NODE_VERSION }}
|
||||
|
||||
- name: Write .npmrc file
|
||||
run: |
|
||||
printf '%s\n' "$NPMRC" >> client/.npmrc
|
||||
echo "legacy-peer-deps=true" >> client/.npmrc
|
||||
shell: bash
|
||||
env:
|
||||
NPMRC: ${{ secrets.NPMRC }}
|
||||
|
||||
- name: Install system packages
|
||||
# jq parses the release JSON below, so it is installed before the tag
|
||||
# resolve.
|
||||
run: |
|
||||
apt-get update
|
||||
apt-get install -y zip jq
|
||||
|
||||
- name: Resolve the release tag
|
||||
env:
|
||||
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
||||
REQUESTED_TAG: ${{ inputs.tag }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
AUTH_HEADER="Authorization: token $GITEA_TOKEN"
|
||||
BASE="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"
|
||||
|
||||
if [ -n "${REQUESTED_TAG}" ]; then
|
||||
TAG="${REQUESTED_TAG}"
|
||||
# Fail here rather than after a ten-minute build if the tag is not
|
||||
# cut yet or has no release.
|
||||
curl --fail-with-body -sS -H "$AUTH_HEADER" "$BASE/releases/tags/${TAG}" > /dev/null
|
||||
else
|
||||
TAG=$(curl --fail-with-body -sS -H "$AUTH_HEADER" "$BASE/releases/latest" | jq -r '.tag_name')
|
||||
fi
|
||||
echo "Building assets for ${TAG}"
|
||||
echo "TAG=${TAG}" >> "$GITHUB_ENV"
|
||||
|
||||
- name: Check out the tag
|
||||
run: |
|
||||
set -euo pipefail
|
||||
git fetch --force --tags origin "+refs/tags/${TAG}:refs/tags/${TAG}"
|
||||
git checkout --force "refs/tags/${TAG}"
|
||||
|
||||
- name: Install the SASjs CLI the tag pins
|
||||
run: |
|
||||
cd sas
|
||||
CLI_VERSION=$(node -p "require('./package.json').dependencies['@sasjs/cli']")
|
||||
npm i -g "@sasjs/cli@${CLI_VERSION}"
|
||||
|
||||
- name: Frontend build
|
||||
run: |
|
||||
cd client
|
||||
# Decrypt and Install sheet
|
||||
echo "$SHEET_PWD" | gpg --batch --yes --passphrase-fd 0 ./libraries/sheet-crypto.tgz.gpg
|
||||
npm ci
|
||||
npm i webpack
|
||||
npm run build
|
||||
shell: bash
|
||||
env:
|
||||
SHEET_PWD: ${{ secrets.SHEET_PWD }}
|
||||
|
||||
- name: Build SAS9 EBI Release
|
||||
run: |
|
||||
cd sas
|
||||
npm i
|
||||
sasjs c -t sas9
|
||||
rm -rf sasjsbuild/tests
|
||||
sasjs b -t sas9
|
||||
cp sasjsbuild/mysas9deploy.sas ./demostream_sas9.sas
|
||||
# remove streamed component and rebuild SAS 9 services
|
||||
rm -rf sasjsbuild/services/web9
|
||||
rm sasjsbuild/services/clickme.sas
|
||||
sasjs b -t sas9
|
||||
cp sasjsbuild/mysas9deploy.sas ./sas9.sas
|
||||
|
||||
- name: Build SASjs Server Release
|
||||
run: |
|
||||
cd sas
|
||||
cp sasjs/utils/favicon.ico ../client/dist/favicon.ico
|
||||
sasjs c -t server
|
||||
rm -rf sasjsbuild/tests
|
||||
server_apploc="/Public/app/dc"
|
||||
sed -i "s|apploc=\"[^\"]*\"|apploc=\"${server_apploc}\"|g" sasjsbuild/services/web/index.html
|
||||
sasjs b -t server
|
||||
cp sasjsbuild/server.json.zip ./sasjs_server.json.zip
|
||||
|
||||
- name: Build Viya Release
|
||||
run: |
|
||||
cd sas
|
||||
sasjs c -t viya
|
||||
rm -rf sasjsbuild/tests
|
||||
sed -i -e 's/servertype="SASJS"/servertype="SASVIYA"/g' sasjsbuild/services/DC.html
|
||||
sasjs b -t viya
|
||||
cp sasjsbuild/viya.sas ./viya.sas
|
||||
# compile Viya Full deploy (without web)
|
||||
rm -rf sasjsbuild/services/web
|
||||
rm sasjsbuild/services/DC.html
|
||||
sasjs b -t viya
|
||||
cp sasjsbuild/viya.sas ./viya_noweb.sas
|
||||
cp sasjsbuild/viya.json ./viya_noweb.json
|
||||
|
||||
- name: Zip Frontend (including viya.json for full viya deploy)
|
||||
run: |
|
||||
cd sas
|
||||
cp sasjsbuild/viya.json ../client/dist/viya.json
|
||||
# Zip from *inside* dist so the archive holds the frontend files at its
|
||||
# root rather than under a client/dist/ prefix (see #147).
|
||||
cd ../client/dist
|
||||
zip -r ../../frontend.zip .
|
||||
|
||||
- name: Upload assets to the release
|
||||
env:
|
||||
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
AUTH_HEADER="Authorization: token $GITEA_TOKEN"
|
||||
BASE="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"
|
||||
|
||||
RELEASE_JSON=$(curl --fail-with-body -sS -H "$AUTH_HEADER" "$BASE/releases/tags/${TAG}")
|
||||
RELEASE_ID=$(echo "$RELEASE_JSON" | jq -r '.id')
|
||||
RELEASE_BODY=$(echo "$RELEASE_JSON" | jq -r '.body')
|
||||
|
||||
ASSETS=(frontend.zip
|
||||
sas/demostream_sas9.sas
|
||||
sas/viya.sas
|
||||
sas/sasjs_server.json.zip
|
||||
sas/sas9.sas
|
||||
sas/viya_noweb.sas
|
||||
sas/viya_noweb.json)
|
||||
|
||||
: > SHA256SUMS
|
||||
for f in "${ASSETS[@]}"; do
|
||||
sum=$(sha256sum "$f" | cut -d' ' -f1)
|
||||
printf '%s %s\n' "$sum" "${f##*/}" >> SHA256SUMS
|
||||
done
|
||||
cat SHA256SUMS
|
||||
|
||||
# Replacing the assets means removing the ones this workflow uploads,
|
||||
# and a second run must not append the verification section twice. An
|
||||
# asset the release carries from another source is left alone.
|
||||
OWNED=$(printf '%s\n' "${ASSETS[@]##*/}" SHA256SUMS)
|
||||
echo "$RELEASE_JSON" | jq -r '.assets[]? | "\(.id) \(.name)"' |
|
||||
while read -r asset_id asset_name; do
|
||||
if printf '%s\n' "$OWNED" | grep -qxF -- "$asset_name"; then
|
||||
echo "Removing previous ${asset_name}"
|
||||
curl --fail-with-body -sS -X DELETE -H "$AUTH_HEADER" "$BASE/releases/${RELEASE_ID}/assets/${asset_id}"
|
||||
else
|
||||
echo "Leaving ${asset_name} in place: this workflow did not upload it"
|
||||
fi
|
||||
done
|
||||
|
||||
if echo "$RELEASE_BODY" | grep -q "## Verifying the download"; then
|
||||
echo "Release body already carries the verification section"
|
||||
else
|
||||
VERIFY_SECTION=$(./.gitea/scripts/verify-section.sh)
|
||||
NEW_BODY=$(jq -nr --arg body "$RELEASE_BODY" --arg verify "$VERIFY_SECTION" \
|
||||
'$body + "\n" + $verify + "\n\nFor installation instructions, please visit https://docs.datacontroller.io/"')
|
||||
curl --fail-with-body -sS -X PATCH \
|
||||
-H "$AUTH_HEADER" \
|
||||
-H 'Content-Type: application/json' \
|
||||
--data "$(jq -n --arg body "$NEW_BODY" '{draft:false, body:$body}')" \
|
||||
"$BASE/releases/$RELEASE_ID"
|
||||
fi
|
||||
|
||||
URL="$BASE/releases/${RELEASE_ID}/assets"
|
||||
for f in "${ASSETS[@]}"; do
|
||||
echo "Uploading $f ..."
|
||||
curl --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@$f"
|
||||
done
|
||||
echo "Uploading SHA256SUMS ..."
|
||||
curl --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@SHA256SUMS"
|
||||
@@ -107,6 +107,16 @@ jobs:
|
||||
echo ${{ secrets.SHEET_PWD }} | gpg --batch --yes --passphrase-fd 0 ./libraries/sheet-crypto.tgz.gpg
|
||||
npm ci
|
||||
|
||||
- name: Build frontend for web streaming
|
||||
# The server-ci target streams client/dist as the 'clickme' web app, so
|
||||
# the production build has to exist before the mocked services are
|
||||
# deployed. Without it `sasjs cbd` reports
|
||||
# "webSourcePath: .../client/dist ... doesn't exist", streams no
|
||||
# frontend and the deploy step fails.
|
||||
run: |
|
||||
cd client
|
||||
npm run build
|
||||
|
||||
- name: Setup and start SASjs server
|
||||
run: |
|
||||
npm i -g pm2
|
||||
@@ -271,6 +281,11 @@ jobs:
|
||||
|
||||
semantic-release
|
||||
|
||||
# The tag this job cuts, for the steps that publish against it: a
|
||||
# `releases/latest` lookup would pick whichever release landed last
|
||||
# if two overlap.
|
||||
echo "RELEASE_TAG=v$(node -p "require('./package.json').version")" >> "$GITHUB_ENV"
|
||||
|
||||
- name: Frontend Build
|
||||
description: Must be created AFTER the release as the version (git tag) is used in the interface
|
||||
run: |
|
||||
@@ -331,19 +346,6 @@ jobs:
|
||||
cd ../client/dist
|
||||
zip -r ../../frontend.zip .
|
||||
|
||||
- name: Release Typedoc
|
||||
run: |
|
||||
cd client
|
||||
npm -g install cloudron-surfer
|
||||
npm run compodoc:build
|
||||
surfer put --token ${{ secrets.TSDOC_TOKEN }} --server webdoc.datacontroller.io documentation/* /
|
||||
|
||||
- name: Release code.datacontroller.io
|
||||
run: |
|
||||
cd sas
|
||||
sasjs doc
|
||||
surfer put --token ${{ secrets.CODE_DATACONTROLLER_IO }} --server code.datacontroller.io sasjsbuild/sasdocs/* /
|
||||
|
||||
- name: Upload assets to release
|
||||
env:
|
||||
# Use the same ephemeral per-job Gitea Actions token that the
|
||||
@@ -358,7 +360,7 @@ jobs:
|
||||
AUTH_HEADER="Authorization: token ${GITEA_TOKEN}"
|
||||
BASE="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"
|
||||
|
||||
RELEASE_JSON=$(curl -k --fail-with-body -sS -H "$AUTH_HEADER" "$BASE/releases/latest")
|
||||
RELEASE_JSON=$(curl --fail-with-body -sS -H "$AUTH_HEADER" "$BASE/releases/tags/${RELEASE_TAG}")
|
||||
RELEASE_ID=$(echo "$RELEASE_JSON" | jq -r '.id')
|
||||
RELEASE_BODY=$(echo "$RELEASE_JSON" | jq -r '.body')
|
||||
|
||||
@@ -392,7 +394,7 @@ jobs:
|
||||
VERIFY_SECTION=$(./.gitea/scripts/verify-section.sh)
|
||||
NEW_BODY=$(jq -nr --arg body "$RELEASE_BODY" --arg verify "$VERIFY_SECTION" \
|
||||
'$body + "\n" + $verify + "\n\nFor installation instructions, please visit https://docs.datacontroller.io/"')
|
||||
curl -k --fail-with-body -sS -X PATCH \
|
||||
curl --fail-with-body -sS -X PATCH \
|
||||
-H "$AUTH_HEADER" \
|
||||
-H 'Content-Type: application/json' \
|
||||
--data "$(jq -n --arg body "$NEW_BODY" '{draft:false, body:$body}')" \
|
||||
@@ -402,10 +404,37 @@ jobs:
|
||||
URL="$BASE/releases/$RELEASE_ID/assets"
|
||||
for f in "${ASSETS[@]}"; do
|
||||
echo "Uploading $f ..."
|
||||
curl -k --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@$f"
|
||||
curl --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@$f"
|
||||
done
|
||||
echo "Uploading SHA256SUMS ..."
|
||||
curl -k --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@SHA256SUMS"
|
||||
curl --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@SHA256SUMS"
|
||||
|
||||
- name: Publish the code documentation
|
||||
# Typedoc (webdoc) and the SAS docs (code) are published by docs.yaml,
|
||||
# not inline here. Two reasons: the v6 `surfer put --token` flow is gone
|
||||
# in Surfer 7, which takes a Cloudron username plus an app password; and
|
||||
# a dispatch means a stale or failed docs deploy can be republished
|
||||
# without re-running this job, which semantic-release refuses once the
|
||||
# tag exists.
|
||||
#
|
||||
# This runs AFTER the release assets are uploaded: the documentation is
|
||||
# a side channel, and a docs problem must not cost a cut release its
|
||||
# artefacts.
|
||||
env:
|
||||
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
AUTH_HEADER="Authorization: token $GITEA_TOKEN"
|
||||
BASE="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"
|
||||
|
||||
TAG="${RELEASE_TAG}"
|
||||
echo "Dispatching code documentation for ${TAG}"
|
||||
|
||||
curl --fail-with-body -sS -X POST \
|
||||
-H "$AUTH_HEADER" \
|
||||
-H 'Content-Type: application/json' \
|
||||
--data "$(jq -n --arg ref "refs/tags/$TAG" '{ref: $ref}')" \
|
||||
"$BASE/actions/workflows/docs.yaml/dispatches"
|
||||
|
||||
- name: Publish the demo site for this release
|
||||
# The tag is created by this job's semantic-release step, so it carries
|
||||
@@ -420,12 +449,12 @@ jobs:
|
||||
AUTH_HEADER="Authorization: token ${GITEA_TOKEN}"
|
||||
BASE="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"
|
||||
|
||||
TAG=$(curl -k --fail-with-body -sS -H "$AUTH_HEADER" "$BASE/releases/latest" | jq -r '.tag_name')
|
||||
TAG="${RELEASE_TAG}"
|
||||
echo "Dispatching Sites for ${TAG}"
|
||||
|
||||
# the dispatch endpoint resolves a bare name as a branch; a tag needs
|
||||
# the refs/tags/ prefix
|
||||
curl -k --fail-with-body -sS -X POST \
|
||||
curl --fail-with-body -sS -X POST \
|
||||
-H "$AUTH_HEADER" \
|
||||
-H 'Content-Type: application/json' \
|
||||
--data "$(jq -n --arg ref "refs/tags/$TAG" '{ref: $ref}')" \
|
||||
|
||||
@@ -75,6 +75,7 @@ jobs:
|
||||
touch .env
|
||||
echo RUN_TIMES=js >> .env
|
||||
echo NODE_PATH=node >> .env
|
||||
echo HTTP_PORT=5000 >> .env
|
||||
# The server refuses to run as root unless RUN_AS names an account
|
||||
echo RUN_AS=root >> .env
|
||||
nohup ./api-linux > server.log 2>&1 &
|
||||
|
||||
@@ -32,6 +32,8 @@ Never consider a change complete until the relevant linters pass on the files yo
|
||||
|
||||
- **Client (TypeScript/HTML/etc.)**: run `npm run lint:check` from the `client/` directory (prettier). Fix any failures with `npm run lint:fix`.
|
||||
- **SAS**: after creating or modifying any `.sas` files, run `sasjs lint` from the `sas/` directory and ensure the files you touched have no lint warnings (the repo currently has pre-existing warnings in other files, which can be ignored).
|
||||
`sas/.sasjslint` carries `ignoredMacros: ["webout", "mpeinit", "mpeterm"]`. `webout` is the linter's own default - `sasjs compile` injects `%webout` into every service - and an explicit list replaces that default, so it has to be restated there. `mpeinit` and `mpeterm` are listed because every service is compiled with a serviceInit program (`sasjs/utils/serviceinitsas9|viya|server.sas`) that declares both macros in its own header, so a service body may call them without declaring them. Add a macro to that list only when it reaches a file the same way - injected by the compiler, through serviceInit, testSetUp or another implicit provider - never to silence a genuinely missing declaration.
|
||||
`noBannedStrings` flags the literal words `error ` and `warning ` anywhere in a file, comments included. Reword prose; where the text has to stay (a real `%put`, a doxygen `@warning` tag), keep the log output identical with `%put %str(WARN)ING:` rather than a per-file override, and use `@sasjslint {"noBannedStrings": false}` only when the word itself is the content.
|
||||
|
||||
## No external assets
|
||||
|
||||
@@ -41,7 +43,7 @@ Data Controller must run entirely locally (offline / on-prem, no internet access
|
||||
|
||||
Agent-related content lives in `.agents/`: technical/agent-facing documentation goes in `.agents/docs/` (not `docs/`), and skills in `.agents/skills/`. When writing explanatory or technical docs about the codebase, put them in `.agents/docs/`.
|
||||
|
||||
Skills installed via `npx skills add` are tracked in `skills-lock.json` and live under `.agents/skills/`. Repo-local custom skills (e.g. `dc-sas` for Data Controller SAS development, `handsontable`, `hyperformula`) are kept here too. Avoid editing npx-managed skill files directly — use `npx skills update` instead.
|
||||
Skills installed via `npx skills add` are tracked in `skills-lock.json` and live under `.agents/skills/`. Repo-local custom skills (e.g. `dc-sas` for Data Controller SAS development, `dc-estate-testing` for estate deploys and test runs, `handsontable`, `hyperformula`) are kept here too. Avoid editing npx-managed skill files directly — use `npx skills update` instead.
|
||||
|
||||
## Code comments and test names
|
||||
|
||||
|
||||
+150
@@ -1,3 +1,153 @@
|
||||
# [7.20.0](https://git.datacontroller.io/dc/dc/compare/v7.19.0...v7.20.0) (2026-10-28)
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **ci:** install doxygen for the SAS docs job ([f58925b](https://git.datacontroller.io/dc/dc/commit/f58925b29956f1f753f0a367fdcd0afe7f3f51a0))
|
||||
* **ci:** keep foreign release assets, install jq first, pin the CLI ([27686d3](https://git.datacontroller.io/dc/dc/commit/27686d3aad5e34ace59ca1b735a17cca1a4025e1))
|
||||
* **ci:** publish against the tag this job cuts, not releases/latest ([8888734](https://git.datacontroller.io/dc/dc/commit/888873421c3aeaf312d5ec45e3c805317c66976a))
|
||||
* **ci:** publish docs after the assets, with the webdoc app password ([dd482c4](https://git.datacontroller.io/dc/dc/commit/dd482c4069202a0c6b9983129c19efc09bd160a7))
|
||||
* **ci:** publish the code documentation with surfer 7 and app passwords ([25ffa71](https://git.datacontroller.io/dc/dc/commit/25ffa71adddba5e334f4a8c166f284d1be498fcd))
|
||||
* **ci:** publish the typedoc to the webdoc site, not the app default ([c5808a1](https://git.datacontroller.io/dc/dc/commit/c5808a16eba0d29562fcc76f13ac4a33ab8da0b7))
|
||||
* **ci:** rebuild and upload release assets on demand ([6486c80](https://git.datacontroller.io/dc/dc/commit/6486c80ed4f368bbf116c0ecdecc0e5e5f02075f))
|
||||
* **ci:** use the lockfile and the pinned CLI in the docs job ([af1813f](https://git.datacontroller.io/dc/dc/commit/af1813f829861df41c89b512c824e8d3ebac7a4b))
|
||||
* **ci:** verify the forge certificate on the release API calls ([cf7b21a](https://git.datacontroller.io/dc/dc/commit/cf7b21a8b61ec3bf7ebe21968ccf13ade264bc64))
|
||||
* **cypress:** hold download out too, and retry once in CI ([c97e886](https://git.datacontroller.io/dc/dc/commit/c97e8866db9ee404b2118d24baf53fc7058debef)), closes [#395](https://git.datacontroller.io/dc/dc/issues/395) [#398](https://git.datacontroller.io/dc/dc/issues/398)
|
||||
* **cypress:** register the deleteFolder task, restore the spec order, hold metanav out ([c803773](https://git.datacontroller.io/dc/dc/commit/c803773d3c36476d8263f01fc3b8344aa3503d44)), closes [#395](https://git.datacontroller.io/dc/dc/issues/395)
|
||||
* **deploy:** do not offer the deploy wizard inside a report ([80e8911](https://git.datacontroller.io/dc/dc/commit/80e8911af8857fa2c417f028cd59b64a87233b9e))
|
||||
* **deploy:** keep the app query when opening the standalone URL ([e51953c](https://git.datacontroller.io/dc/dc/commit/e51953ca8a6734e1649bad68cf4ad2057d677334))
|
||||
* **deploy:** one definition of the standalone URL, and test the guards ([459ff2d](https://git.datacontroller.io/dc/dc/commit/459ff2da371375e996839d69deb40f681c356f1d)), closes [#TCS](https://git.datacontroller.io/dc/dc/issues/TCS)
|
||||
* **deploy:** strip the embed parameters by name, and scope the timezone runs to the VA specs ([f7e7538](https://git.datacontroller.io/dc/dc/commit/f7e753863b61d8545bd0777d80267661b3c58670))
|
||||
* **embed:** build quarter ends from the calendar, not minus 24 hours ([7e3045a](https://git.datacontroller.io/dc/dc/commit/7e3045a21b67053b76a81bfe836e31571ccbffbf))
|
||||
* **embed:** defer the column check until metadata loads, report AND-ed pairs ([2623a3f](https://git.datacontroller.io/dc/dc/commit/2623a3fb4ba3bf19c00636b1717b53d5a2d54f3a))
|
||||
* **embed:** log a parse-time mapping problem once ([0011f7e](https://git.datacontroller.io/dc/dc/commit/0011f7e933d85be8e33debe3a6ecb9000edaabe3))
|
||||
* **embed:** make the va_parmap diagnostics say what they mean ([541a61b](https://git.datacontroller.io/dc/dc/commit/541a61ba08bc7256a911c50de7078859cc46c84e))
|
||||
* **embed:** match a mapping on the parameter label only, and warn only when the values differ ([e9216b0](https://git.datacontroller.io/dc/dc/commit/e9216b08956d5599662259a2b05cc1524c458df5))
|
||||
* **embed:** never build a clause from a value that did not convert, and say so in the banner ([bd3c220](https://git.datacontroller.io/dc/dc/commit/bd3c220af2678a2a3f8394fe3084ee03c23d0b15))
|
||||
* **embed:** one parameter maps one column, and report typos on both sides ([f83dc35](https://git.datacontroller.io/dc/dc/commit/f83dc359e258f5d1cb9fc49f0eaab9748995d823))
|
||||
* **embed:** record the parameter label only, so a name-keyed va_parmap is reported ([aa91daf](https://git.datacontroller.io/dc/dc/commit/aa91daf9f78c00cea486d0f885f8e634366bd1a0))
|
||||
* **embed:** resolve transform values as local dates and give ranges whole days ([10157c1](https://git.datacontroller.io/dc/dc/commit/10157c11595685efdc206fe7b16a5cafa098f26d))
|
||||
* **embed:** warn when a va_parmap entry is ignored ([411f6c3](https://git.datacontroller.io/dc/dc/commit/411f6c3354b7fbfec11fef3b174ae1f8066c92ba))
|
||||
* **makedata:** pair the %else of the keepme block so it parses in open code ([c0ea810](https://git.datacontroller.io/dc/dc/commit/c0ea810766ebfda5ae5358ab14146fbcc07fcc8e))
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* **embed:** map VA parameters to DC columns from the URL ([15a1f34](https://git.datacontroller.io/dc/dc/commit/15a1f34d5263503cac83b4fe4e52482777dc2f1f))
|
||||
* **embed:** resolve va_parmap values through named date anchors ([88010c3](https://git.datacontroller.io/dc/dc/commit/88010c3980c2ea1c39bbb024ed32d63e495e61f0))
|
||||
|
||||
# [7.19.0](https://git.datacontroller.io/dc/dc/compare/v7.18.0...v7.19.0) (2026-10-28)
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* address the review findings on the harness and refresh changes ([36e4e88](https://git.datacontroller.io/dc/dc/commit/36e4e88eb134d3b48217d6dd185ea3c03f03fba5))
|
||||
* **ci:** dispatch the demo deploy with a tag ref and the actions scope ([698dea6](https://git.datacontroller.io/dc/dc/commit/698dea6d6b7a8635313f11acfcc7a4f39a02bc99))
|
||||
* **ci:** restore the gitleaks ignore entries for the mock licence pair ([0d58be9](https://git.datacontroller.io/dc/dc/commit/0d58be975d8eb2d7f1ab6a648f760151392c7973))
|
||||
* **client:** open the SAS log page with noopener ([072a133](https://git.datacontroller.io/dc/dc/commit/072a1337020970b5b4ff8aa31719cc5ffec7701e))
|
||||
* **client:** refuse a truncated table in the multi-table upload ([1fb4818](https://git.datacontroller.io/dc/dc/commit/1fb4818d215f538a0ac51772ab1f261aeb91f7e1))
|
||||
* **cypress:** apply the licence key when the estate has none ([be8f445](https://git.datacontroller.io/dc/dc/commit/be8f445ae1051e132fb6eb5a69b45264255326b5)), closes [input#file-upload](https://git.datacontroller.io/input/issues/file-upload)
|
||||
* **dc_execute:** use = comparisons, not IN, in the fail-fast guard ([fed43cb](https://git.datacontroller.io/dc/dc/commit/fed43cb07ef7f2afc685612a124dbc16d7618e21))
|
||||
* **dc_getlibs:** exclude the DC libref from the dictionary scan ([f5aade5](https://git.datacontroller.io/dc/dc/commit/f5aade59760cc376b0f96eb4c892c8bca82dde14))
|
||||
* **dc_refreshcatalog:** terminate the generated statements ([1e22476](https://git.datacontroller.io/dc/dc/commit/1e224763ba80500a112812a983801968e3751959))
|
||||
* **derived:** a fileref with nothing written is a FAIL, not an error ([9b3cf50](https://git.datacontroller.io/dc/dc/commit/9b3cf50b9c3889786605f0f5d9032d84c50f7107))
|
||||
* **derived:** address five review findings ([ccd468b](https://git.datacontroller.io/dc/dc/commit/ccd468b3358c1418ee6d8e198b088a0cf23b8c24))
|
||||
* **derived:** carry the placement sequence in mpe_derived ([be50cb5](https://git.datacontroller.io/dc/dc/commit/be50cb5d082e788c6ae106bc1d02e73a8c7b9646))
|
||||
* **derived:** close four gaps in the MPE_DERIVED hook ([9914790](https://git.datacontroller.io/dc/dc/commit/9914790777f92128c74ecce67e6c67ee48824b45))
|
||||
* **derived:** close the RLS and search gaps on derived columns ([9ef813d](https://git.datacontroller.io/dc/dc/commit/9ef813d439b48e7fefd35c426e62b99e5943002a))
|
||||
* **derived:** close the two gaps in the derived/RLS invariant ([422f766](https://git.datacontroller.io/dc/dc/commit/422f766cd43db4ce740e1890fd5df30790eaa7e3))
|
||||
* **derived:** compare the outref as text, prove the file is empty ([7830b7b](https://git.datacontroller.io/dc/dc/commit/7830b7bcfa9806dde840ce8dd3e6a1887d0b38be))
|
||||
* **derived:** do not drop a shadowed column from staging or downloads ([2d5de9a](https://git.datacontroller.io/dc/dc/commit/2d5de9a324a07a36edc0a4f74c8fad09fde07cbf))
|
||||
* **derived:** do not duplicate a shadowed column in the grid payload ([6a1128a](https://git.datacontroller.io/dc/dc/commit/6a1128a46352f77c95dd02df9309f0984a1db2d1))
|
||||
* **derived:** eight review findings ([561d7fc](https://git.datacontroller.io/dc/dc/commit/561d7fc3ed2a460835619962cf4dcb29d3dc87c7))
|
||||
* **derived:** enforce the ACTIVE picker pair in the extracted check ([1863076](https://git.datacontroller.io/dc/dc/commit/1863076bc956b4ff94bf7b85b3dde0d6e3b54d7a)), closes [#366](https://git.datacontroller.io/dc/dc/issues/366)
|
||||
* **derived:** enforce the documented limits, and handle derived filters in VIEW ([d3dec68](https://git.datacontroller.io/dc/dc/commit/d3dec682aafeb143fe983d6546337cb6490f5620))
|
||||
* **derived:** filtering a format catalog no longer aborts the service ([469229a](https://git.datacontroller.io/dc/dc/commit/469229aecbaad304a835a5f229df111c37ff7c0d))
|
||||
* **derived:** guard the bitemporal cut-off, coalesce the selectbox seed ([7fb981d](https://git.datacontroller.io/dc/dc/commit/7fb981daa91bce29c2d62c246185d01dcad08d6b))
|
||||
* **derived:** guard the empty physical column list in the dep check ([5f5d868](https://git.datacontroller.io/dc/dc/commit/5f5d8681b9d78280a5200309917415e135c3ce5b))
|
||||
* **derived:** hold a derived column read-only whatever the table's security ([5f443b0](https://git.datacontroller.io/dc/dc/commit/5f443b0b16a2abd3c11d96caa7a043911c29ef01))
|
||||
* **derived:** ignore a re-added name when collecting inactive rows ([6b5bf3e](https://git.datacontroller.io/dc/dc/commit/6b5bf3e6be89b22e5a633c34b9aecbc8c7d48ee2))
|
||||
* **derived:** ignore inactive rows and VIEW rules in the submit guards ([85c89ca](https://git.datacontroller.io/dc/dc/commit/85c89ca2a4e02112ba496ac36a222f33675544da))
|
||||
* **derived:** keep targetds for plain datasets, test the empty file ([f8f6568](https://git.datacontroller.io/dc/dc/commit/f8f65681e1489f4ce94d9127d5e4c0c82798e960))
|
||||
* **derived:** make the model enforce the required columns, and leave VAR_FMT optional ([7efbf31](https://git.datacontroller.io/dc/dc/commit/7efbf31ea5ac234bd8f924fdf89a95d101b3aff1))
|
||||
* **derived:** only the delete marker is a reserved name ([af31c36](https://git.datacontroller.io/dc/dc/commit/af31c36b670ec63a2a4ba056bf6c4b28f6096e08))
|
||||
* **derived:** open the statements file with a long record length ([ee31782](https://git.datacontroller.io/dc/dc/commit/ee31782fbff909b540162b1bdfca6f307208331b))
|
||||
* **derived:** pair the filterds %else with %do/%end ([88a38db](https://git.datacontroller.io/dc/dc/commit/88a38db877dba9b98ccf50f0cc7242674654586b))
|
||||
* **derived:** raise the VAR_ORDER ceiling to 10000 ([27ddcfb](https://git.datacontroller.io/dc/dc/commit/27ddcfb5759d6eb8f9880adc7c271a408e7741ad))
|
||||
* **derived:** refuse dependencies on inactive rows, and keep the RLS scope guard coherent ([7799a30](https://git.datacontroller.io/dc/dc/commit/7799a300df82339683eea21e4a89ce7a6471c575))
|
||||
* **derived:** reject a dependency on a row this submission deletes ([8806504](https://git.datacontroller.io/dc/dc/commit/88065049674c2cc0141eef22704b759f5b708b0f))
|
||||
* **derived:** repair the validation rows for MPE_DERIVED ([9a047ad](https://git.datacontroller.io/dc/dc/commit/9a047ad42ec86df7d1eaf5058c0edde56016e88f))
|
||||
* **derived:** review feedback - cover the empty response, assert the last row ([e0da9f1](https://git.datacontroller.io/dc/dc/commit/e0da9f1098db63185538398ca478696cc849da4b))
|
||||
* **derived:** stop the hooks aborting on their own macro variables ([688550a](https://git.datacontroller.io/dc/dc/commit/688550a28d9794ac6b878926a0e1e2ef8f6e9335))
|
||||
* **derived:** truncate the derived label on an even quote boundary ([1b5c4b3](https://git.datacontroller.io/dc/dc/commit/1b5c4b3ecd962a28e08fc5671b57dbf7ae0b8d9f))
|
||||
* **derived:** verify on stripped values, and label with the variable name ([4d9ad38](https://git.datacontroller.io/dc/dc/commit/4d9ad38df1fa89b0d5a12214d703d9c1fe5295f2))
|
||||
* **editor:** a blank truncation note is not a warning ([f182179](https://git.datacontroller.io/dc/dc/commit/f18217940365f27ae388fae087e6126931e1222c))
|
||||
* **editor:** keep decimal values when an uploaded file is edited manually ([04ce186](https://git.datacontroller.io/dc/dc/commit/04ce1861e7821063b1be42c22f92f62aed509602))
|
||||
* **getcolvals:** report an invalid value, not missing inputs ([4bcee70](https://git.datacontroller.io/dc/dc/commit/4bcee706ffaf1929bdd14bb2fb15111174ca3916))
|
||||
* **getdata.test:** pin the derived filter to a libref, and spell the table as it is ([45dce7b](https://git.datacontroller.io/dc/dc/commit/45dce7b11b7a4399c1d0dc9c3e7a283b1cf59df1))
|
||||
* **getdata:** clear the globals it declares before use ([359b570](https://git.datacontroller.io/dc/dc/commit/359b570b85aa2f66507e81ea2977243be4590f83))
|
||||
* **getddl.test:** ask for a table, not a whole library ([059cdf1](https://git.datacontroller.io/dc/dc/commit/059cdf176a7ec556b45ea1b21d880c048eba14ca))
|
||||
* **getrawdata:** declare mp_ds2md so the Markdown download can run ([661ee19](https://git.datacontroller.io/dc/dc/commit/661ee19cfa5f996bf01f2b11817039f38dbb9985))
|
||||
* **lint:** clear the sasjs lint warnings on the files 7.19 touched ([355c46a](https://git.datacontroller.io/dc/dc/commit/355c46a7b64994d0146a88657dc5635593249138))
|
||||
* **makedata:** read KEEPME from every input row, and keep it out of the Viya seed ([d626cb8](https://git.datacontroller.io/dc/dc/commit/d626cb8f2daf94fde7ff21d924b9a2ca4968d983))
|
||||
* **mocks:** address the audit-pack review findings ([489819b](https://git.datacontroller.io/dc/dc/commit/489819b05111323ae1e34edda22a5446cdbc2d87))
|
||||
* **mocks:** carry the zip flags into the central directory ([c4021a4](https://git.datacontroller.io/dc/dc/commit/c4021a42ffaed2af80c53aaa1455249190b339b8))
|
||||
* **mocks:** clear the licence for published sites, not for the estate ([62f55db](https://git.datacontroller.io/dc/dc/commit/62f55db170185900063f7a431cd8f375ebabf930))
|
||||
* **mocks:** copy the physical column list before deriving ([f411b7d](https://git.datacontroller.io/dc/dc/commit/f411b7d269381e3f7cab2d85b6d8f7d7d50ea4d1))
|
||||
* **mocks:** derive before filtering in the viewer, and drop the columns ([60ec470](https://git.datacontroller.io/dc/dc/commit/60ec470404e5c279b58ac6a01768dfc83413b197))
|
||||
* **mocks:** derive the retained key wherever rows are staged ([74060c0](https://git.datacontroller.io/dc/dc/commit/74060c0585fb8b217cb181ea747ad318f5238868))
|
||||
* **mocks:** drop derived variables before staging ([73d25b1](https://git.datacontroller.io/dc/dc/commit/73d25b1220d574ef2547fbd9e6e3cf1ceda375ee))
|
||||
* **mocks:** key rk_underlying tables on their business key ([347dbb5](https://git.datacontroller.io/dc/dc/commit/347dbb53b01c47bbd562de14d5ccbae7d96407c6))
|
||||
* **mocks:** register MPE_DERIVED and seed its field rules ([ee3c878](https://git.datacontroller.io/dc/dc/commit/ee3c8787efef9cd716bdd2bf80f6cb6751461db9))
|
||||
* **mocks:** scope the derived hook's graph to the submission ([6cad23f](https://git.datacontroller.io/dc/dc/commit/6cad23fce9ac0cf64c247c40f3b7eac875fd4cfb))
|
||||
* **mocks:** search the physical columns only in the viewer ([293022d](https://git.datacontroller.io/dc/dc/commit/293022d46b27e096fd514cdda49f688612df2f5b))
|
||||
* **mocks:** stop the mock advertising invalid licence keys ([fb36379](https://git.datacontroller.io/dc/dc/commit/fb36379520028baaf4bc3f08131bfc88deab3ac3))
|
||||
* **mpe_assert_abort:** drop the unused line variable ([9c294c0](https://git.datacontroller.io/dc/dc/commit/9c294c0ab5ca8fe30a235553a298c1c68a61442f))
|
||||
* **mpe_assert_abort:** take the abort's own MSG, tolerate a missing response ([0a52d20](https://git.datacontroller.io/dc/dc/commit/0a52d20c5fc27b4adde5fc8ed6f39a58c48f62ff))
|
||||
* **mpe_execute:** fail fast when the child left no status ([a036386](https://git.datacontroller.io/dc/dc/commit/a03638667f1968c5909f496d954eeaa13f316763))
|
||||
* **mpe_execute:** no apostrophe in the SAS 9 status note ([2204085](https://git.datacontroller.io/dc/dc/commit/220408538b26016bda45e48d619cdbc3e1cbcc59))
|
||||
* **mpe_execute:** read an unknown state as unmeasured, not failed ([f2b1428](https://git.datacontroller.io/dc/dc/commit/f2b1428ec3099b2b6e3c6da6627530ef81fab201))
|
||||
* name the startupservice launcher button "Show log" ([3445107](https://git.datacontroller.io/dc/dc/commit/3445107d9a91642b67e114bcfcd81c3ce47f3f0f))
|
||||
* offer the launcher on the globvars failure modal too ([ad2da45](https://git.datacontroller.io/dc/dc/commit/ad2da453896a51f2cc7e49cebb6c84f7b47d7395))
|
||||
* **refresh:** name each catalogue load uniquely in the audit trail ([46bacdd](https://git.datacontroller.io/dc/dc/commit/46bacdd6d9ea814bcd02a1ff8aa3f447baf02f64))
|
||||
* **refresh:** one audit folder per refresh call ([56b6620](https://git.datacontroller.io/dc/dc/commit/56b6620ef402b288ce78a2a77662b651e310753e))
|
||||
* **sites:** round-trip the committed licence before publishing ([4d462d3](https://git.datacontroller.io/dc/dc/commit/4d462d310f7df23e5d5229cb0702721b7ad41bbd))
|
||||
* **test:** getdata test ([da30266](https://git.datacontroller.io/dc/dc/commit/da30266278c4fe27a72810cd5ed647cf3bd823af))
|
||||
* **tests:** enable closeouts logging ([fdcd8fa](https://git.datacontroller.io/dc/dc/commit/fdcd8fa4b3e7a7a119c662974d70dc9eea80bdae))
|
||||
* **tests:** read the work location from _SASJS_WORK ([033ffd3](https://git.datacontroller.io/dc/dc/commit/033ffd302502c7530fc44a314364f5cdb3be6ad2))
|
||||
* **ui:** one error modal at a time ([81ccf75](https://git.datacontroller.io/dc/dc/commit/81ccf758e9dab0c7632ab50bf469e88b8c4c4420))
|
||||
* **validatefilter:** build targetds without macro triggers in comments ([42355f1](https://git.datacontroller.io/dc/dc/commit/42355f17bc07bf4d44ba858984d83b9ed72cc0ba))
|
||||
* **validatefilter:** drop the dead nobs assignment, keep targetds precedence ([337ea8e](https://git.datacontroller.io/dc/dc/commit/337ea8e5e021c6f2451fa3ad8396148744118fc9))
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* **dc_getlibs:** dump the library list when debug is on ([582523d](https://git.datacontroller.io/dc/dc/commit/582523dc87a253b71824359224070ccb4746a386))
|
||||
* **derived:** allow a variable to shadow a physical column ([12beeff](https://git.datacontroller.io/dc/dc/commit/12beeff69518376a042fbbc58332acc40d89d132))
|
||||
* **derived:** declare VAR_DEPENDENCIES instead of VAR_ORDER ([813bce6](https://git.datacontroller.io/dc/dc/commit/813bce685691ad252043eeab31eb5460de81b756))
|
||||
* **derived:** optional VAR_LABEL on MPE_DERIVED ([7c9ce9a](https://git.datacontroller.io/dc/dc/commit/7c9ce9a712137e9825a5f2ff52796d765423147b))
|
||||
* **derived:** pickers, and an ACTIVE default, for MPE_DERIVED ([87e794f](https://git.datacontroller.io/dc/dc/commit/87e794f769fe420f214f43b141f0b41cf88eefee))
|
||||
* **derived:** report the dependency cycle path ([cee1a95](https://git.datacontroller.io/dc/dc/commit/cee1a954ee97e17f297b7cf43ef0e5cfc6e393a6))
|
||||
* **derived:** take any text in VAR_LABEL, but not a line break ([78e88a4](https://git.datacontroller.io/dc/dc/commit/78e88a41128b047c0276ccf647f128229152604c))
|
||||
* **derived:** validate MPE_DERIVED at submit time ([0b010f2](https://git.datacontroller.io/dc/dc/commit/0b010f2c65ddada70d62ea83c706943c6c45756f))
|
||||
* **derived:** validation rules and a VAR_TYPE dropdown for MPE_DERIVED ([aa6d50e](https://git.datacontroller.io/dc/dc/commit/aa6d50e1ea5fc2fe6be8aa9fa859b92d0a1061c5))
|
||||
* **derived:** VARIABLE_ORDER, so a derived variable can build on another ([33d043e](https://git.datacontroller.io/dc/dc/commit/33d043ea92f007dca70c95d5363329d4447eeba9))
|
||||
* **editor:** derived variables in the EDIT screen ([b0fd354](https://git.datacontroller.io/dc/dc/commit/b0fd354ecdbd7bf29196ed983ff3cce691affeed))
|
||||
* **editor:** mark derived variables as computed in the EDIT grid ([6513d09](https://git.datacontroller.io/dc/dc/commit/6513d09f059de15c5304f99b3f8d898d46ea1be7))
|
||||
* **editor:** serve a truncated selection with a warning instead of aborting ([34691dc](https://git.datacontroller.io/dc/dc/commit/34691dceecb63f952b23ad8acc5bc93753c6cfee))
|
||||
* **editor:** show the derived column label in the header tooltip ([cfe6647](https://git.datacontroller.io/dc/dc/commit/cfe6647ca870c45a3d8abc2753ce7f070e3fdf4a))
|
||||
* **mocks:** add getauditfile and the audit-folder artefacts ([752d944](https://git.datacontroller.io/dc/dc/commit/752d944c2091ef6d2624a15ab2e2e7be35608798))
|
||||
* **mocks:** implement derived variables in the JS services ([b4fae88](https://git.datacontroller.io/dc/dc/commit/b4fae88ae13658607e4209449577cd42849d3a99))
|
||||
* **mocks:** validate a submitted MPE_DERIVED row like the hook does ([5d183d1](https://git.datacontroller.io/dc/dc/commit/5d183d135085ea023f32e5b1d4992d997394a8fb))
|
||||
* run startupservice in a new window from its error modal ([40f6133](https://git.datacontroller.io/dc/dc/commit/40f61331aad6bc8a52c551b924f0d1f1e6426cee))
|
||||
* **sites:** mint a licence for the published sites ([7d9df71](https://git.datacontroller.io/dc/dc/commit/7d9df71fdb0ab265e745ed3b708038983e855a87))
|
||||
|
||||
|
||||
### Reverts
|
||||
|
||||
* **derived:** no shadowing, and no self reference ([5af8e6f](https://git.datacontroller.io/dc/dc/commit/5af8e6fc359221586af74a685159dfd75a40ec45))
|
||||
* **stagedata.test.4:** back to the MPE_LOADS evidence for now ([31aa859](https://git.datacontroller.io/dc/dc/commit/31aa8591538409fc1a3ef97d7474120094ce202d))
|
||||
|
||||
# [7.18.0](https://git.datacontroller.io/dc/dc/compare/v7.17.0...v7.18.0) (2026-10-19)
|
||||
|
||||
|
||||
|
||||
+1
-1
@@ -8,7 +8,7 @@ This file is the shared glossary and orientation for the repo. When your output
|
||||
|
||||
- `client/` - the Angular frontend (TypeScript). Uses [Handsontable](.agents/skills/handsontable/SKILL.md) for the editable grid and [HyperFormula](.agents/skills/hyperformula/SKILL.md) for Excel-formula support. Lint with `npm run lint:check` from `client/`.
|
||||
- `sas/` - the SAS backend. Services, macros, hooks and tests live under `sas/sasjs/**`; `sas/sasjsbuild/` is generated output (never hand-edit). Lint `.sas` files with `sasjs lint` from `sas/`.
|
||||
- `.agents/docs/` - technical deep-dives (see below). `.agents/skills/` - task skills (handsontable, hyperformula, dc-sas for Data Controller SAS development, plus npx-installed sasjs/ sas skills).
|
||||
- `.agents/docs/` - technical deep-dives (see below). `.agents/skills/` - task skills (handsontable, hyperformula, dc-sas for Data Controller SAS development, dc-estate-testing for deploying to and testing on the shared estate, plus npx-installed sasjs/ sas skills).
|
||||
|
||||
## Roles
|
||||
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import fs from 'fs'
|
||||
import { defineConfig } from 'cypress'
|
||||
|
||||
export default defineConfig({
|
||||
@@ -34,7 +35,28 @@ export default defineConfig({
|
||||
|
||||
e2e: {
|
||||
video: true,
|
||||
// One retry in a CI run, none interactively. `editor.cy.ts` test 28
|
||||
// intermittently fails on "app-loading ... continuously found" - a slow load,
|
||||
// not an assertion - and a single retry costs a few seconds where a red
|
||||
// pipeline costs forty minutes. Failures that are real still fail twice, and
|
||||
// the retried attempt's output is in the run log.
|
||||
retries: { runMode: 1, openMode: 0 },
|
||||
setupNodeEvents(on, config) {
|
||||
// `download.cy.ts` clears the downloads folder before each test through
|
||||
// `cy.task('deleteFolder')`. That task was registered in
|
||||
// cypress/plugins/index.js, which Cypress 10+ never loads unless it is
|
||||
// required from here - so the task had no handler and the spec failed on
|
||||
// its first hook.
|
||||
on('task', {
|
||||
deleteFolder(folderName: string) {
|
||||
return new Promise((resolve, reject) => {
|
||||
fs.rm(folderName, { recursive: true, force: true }, (err) =>
|
||||
err ? reject(err) : resolve(null)
|
||||
)
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
// Pin the browser locale so locale-formatted cells (intl-date/time/datetime)
|
||||
// render deterministically regardless of the runner's system locale.
|
||||
on('before:browser:launch', (browser, launchOptions) => {
|
||||
|
||||
@@ -93,8 +93,29 @@ context('derived variables: ', function () {
|
||||
'not.have.class',
|
||||
'dc-derived-cell'
|
||||
)
|
||||
// REGION is the first derived column, so it opens the computed block
|
||||
headerCell('REGION').should('have.class', 'dc-derived-first')
|
||||
// The computed block opens at whichever derived column comes first in the
|
||||
// grid - MPE_DERIVED order, so LINE_VALUE here, not REGION. Assert the
|
||||
// invariant rather than a name: exactly one derived header carries the
|
||||
// marker, and it is the first one.
|
||||
cy.get('.ht_clone_top .htCore thead tr th').then(($ths) => {
|
||||
const derived = [...$ths].filter((th) =>
|
||||
th.classList.contains('dc-derived-header')
|
||||
)
|
||||
expect(derived.length, 'derived headers').to.be.greaterThan(0)
|
||||
expect(
|
||||
derived[0].classList.contains('dc-derived-first'),
|
||||
'the first derived header opens the block'
|
||||
).to.be.true
|
||||
derived
|
||||
.slice(1)
|
||||
.forEach(
|
||||
(th) =>
|
||||
expect(
|
||||
th.classList.contains('dc-derived-first'),
|
||||
'only the first derived header carries the marker'
|
||||
).to.be.false
|
||||
)
|
||||
})
|
||||
|
||||
// the hover text carries the label, which is what says what the column is:
|
||||
// the configured VAR_LABEL where there is one, otherwise the generated
|
||||
|
||||
@@ -17,9 +17,20 @@ context('download files test: ', function () {
|
||||
|
||||
this.beforeEach(() => {
|
||||
cy.visit(hostUrl + appLocation)
|
||||
cy.get('input.username').type(username)
|
||||
cy.get('input.password').type(password)
|
||||
cy.get('.login-group button').click()
|
||||
|
||||
// beforeAll already logged in. Typing into the login form unconditionally
|
||||
// failed with "the center of this element is hidden from view" whenever the
|
||||
// session was live and the form was not on screen, so only fill it when it
|
||||
// is actually shown.
|
||||
cy.get('body').then(($body) => {
|
||||
const usernameInput = $body.find('input.username')[0]
|
||||
|
||||
if (usernameInput && !Cypress.dom.isHidden(usernameInput)) {
|
||||
cy.get('input.username').type(username)
|
||||
cy.get('input.password').type(password)
|
||||
cy.get('.login-group button').click()
|
||||
}
|
||||
})
|
||||
|
||||
visitPage('home')
|
||||
})
|
||||
|
||||
@@ -6,44 +6,90 @@ const libraryToOpenIncludes = Cypress.env(`libraryToOpenIncludes_${serverType}`)
|
||||
|
||||
// SAS Visual Analytics data-driven content embed mode (`?embed=va`):
|
||||
// - chrome is hidden (same as embed=true),
|
||||
// - the editor opens in edit mode immediately (not read-only),
|
||||
// - the only action button is a single Submit below the grid (stubbed in v1),
|
||||
// - the top button bar and Add Record are hidden.
|
||||
// - the editor opens READ-ONLY (the user clicks Edit, as in normal use),
|
||||
// - the Filter button is hidden - VA filters replace the filter builder,
|
||||
// - the Add Row / Add Record buttons are hidden.
|
||||
//
|
||||
// The detailed PK+label data-merge logic is covered by the VaMessagingService
|
||||
// unit spec; here we only assert the deterministic UI layout and that pushing a
|
||||
// VA postMessage does not break the page.
|
||||
// unit spec; here we assert the deterministic UI layout, that pushing a VA
|
||||
// postMessage does not break the page, and (case 3) that a `va_parmap` URL
|
||||
// mapping filters the column it names end to end.
|
||||
/**
|
||||
* Assert on the row count the editor shows. The assertion runs inside a Cy
|
||||
* `should`, so it retries until the grid has loaded: reading the text once would
|
||||
* capture the count from before a filter is applied.
|
||||
*/
|
||||
const parseRows = (text: string): number => {
|
||||
const match = /\(([\d,]+) rows?/.exec(text)
|
||||
return match ? Number(match[1].replace(/,/g, '')) : NaN
|
||||
}
|
||||
|
||||
const expectRows = (check: (rows: number) => void) =>
|
||||
cy.get('h4', { timeout: longerCommandTimeout }).should(($h4) => {
|
||||
check(parseRows($h4.text()))
|
||||
})
|
||||
|
||||
/** Assert the grid shrank, without pinning a fixture-specific number. */
|
||||
const expectFewerRowsThan = (baseline: number) =>
|
||||
expectRows((rows) => {
|
||||
expect(rows, 'filtered row count').to.be.lessThan(baseline)
|
||||
})
|
||||
|
||||
/** Read the unfiltered row count from the app itself, retrying until it loads. */
|
||||
const readRowCount = (): Cypress.Chainable<number> =>
|
||||
expectRows((rows) => {
|
||||
expect(rows, 'row count').to.be.greaterThan(0)
|
||||
})
|
||||
.get('h4')
|
||||
.invoke('text')
|
||||
.then(parseRows)
|
||||
|
||||
context('embed=va (VA data-driven content) tests: ', function () {
|
||||
// How many rows the table has unfiltered. Captured once, from the app itself.
|
||||
let unfilteredRows = 0
|
||||
|
||||
this.beforeAll(() => {
|
||||
cy.visit(`${hostUrl}/SASLogon/logout`)
|
||||
cy.loginAndUpdateValidKey()
|
||||
// Baseline for the relative row-count assertions: open the table with no VA
|
||||
// message and record what the editor itself reports.
|
||||
cy.visit(`${hostUrl}${appLocation}/#/editor/DC_JSLIB.MPE_X_TEST`, {
|
||||
timeout: 60000
|
||||
})
|
||||
readRowCount().then((count) => {
|
||||
unfilteredRows = count
|
||||
})
|
||||
})
|
||||
|
||||
this.beforeEach(() => {
|
||||
cy.visit(hostUrl + appLocation)
|
||||
// A cy.visit that differs from the current URL only in the hash is a
|
||||
// same-document navigation, so the app is not re-bootstrapped. Reload so
|
||||
// each test starts from a fresh document (the previous test may have left
|
||||
// ?embed=va in the URL, which the app reads once at init).
|
||||
cy.reload()
|
||||
visitPage('home')
|
||||
})
|
||||
|
||||
it('1 | opens the editor in VA mode: chrome hidden, editable, single bottom Submit', () => {
|
||||
it('1 | opens the editor in VA mode: chrome hidden, read-only, no Filter button', () => {
|
||||
openEditorInVaMode('mpe_x_test', () => {
|
||||
// Chrome hidden (same as embed=true)
|
||||
cy.get('header.app-header').should('not.exist')
|
||||
|
||||
// Grid present and in edit mode (read-only Filter/Edit/Upload bar absent)
|
||||
// Grid present, read-only: the user enters edit mode with Edit, as usual
|
||||
cy.get('#hotTable', { timeout: longerCommandTimeout }).should('exist')
|
||||
cy.get('.btnCtrl button').contains('Edit').should('not.exist')
|
||||
cy.get('.btnCtrl button').contains('Edit').should('exist')
|
||||
|
||||
// The filter builder is replaced by VA filters
|
||||
cy.get('.btnCtrl button').contains('Filter').should('not.exist')
|
||||
|
||||
// Top edit-mode buttons hidden in VA mode
|
||||
// Not in edit mode, so no edit-mode controls
|
||||
cy.get('.btnCtrl button.btn-outline-danger').should('not.exist') // Cancel
|
||||
cy.get('.btnCtrl').contains('Add Row').should('not.exist')
|
||||
cy.get('.btnCtrl').contains('Submit').should('not.exist')
|
||||
|
||||
// Bottom Add Record hidden
|
||||
// Bottom Add Record hidden in VA mode
|
||||
cy.contains('button', 'Add Record').should('not.exist')
|
||||
|
||||
// Exactly one Submit button, below the grid, disabled in v1
|
||||
cy.get('button').filter(':contains("Submit")').should('have.length', 1)
|
||||
cy.get('button').contains('Submit').should('be.disabled')
|
||||
})
|
||||
})
|
||||
|
||||
@@ -69,14 +115,382 @@ context('embed=va (VA data-driven content) tests: ', function () {
|
||||
// Editor still loads in VA mode after the message (chrome hidden, grid up)
|
||||
cy.get('#hotTable', { timeout: longerCommandTimeout }).should('exist')
|
||||
cy.get('header.app-header').should('not.exist')
|
||||
cy.get('button').contains('Submit').should('exist')
|
||||
cy.get('.btnCtrl button').contains('Edit').should('exist')
|
||||
})
|
||||
})
|
||||
it('3 | va_parmap filters a column from a parameter that is not its name (tab 1)', () => {
|
||||
// Two DC instances over one table (one per report tab), each filtering the
|
||||
// same DC column from its own report parameter. Without the URL mapping the
|
||||
// parameter label is not a column name, so nothing would filter at all. The
|
||||
// mapping here is written in lower case against an upper-case label and
|
||||
// column, which pins the case-insensitivity of both sides end to end.
|
||||
openEditorInVaMode(
|
||||
'mpe_x_test',
|
||||
() => {
|
||||
pushVaMessage({
|
||||
name: 'pr1',
|
||||
label: 'DROPDOWN_TAB1',
|
||||
dataType: 'string',
|
||||
value: 'Option 2'
|
||||
})
|
||||
// The applied filter names the MAPPED column, and the grid shrinks to
|
||||
// the rows matching that value (Option 2 occurs twice in this table).
|
||||
cy.get('.infoBar-text', { timeout: longerCommandTimeout }).should(
|
||||
'contain.text',
|
||||
'SOME_DROPDOWN'
|
||||
)
|
||||
cy.get('.infoBar-text').should('contain.text', "'Option 2'")
|
||||
expectFewerRowsThan(unfilteredRows)
|
||||
},
|
||||
'va_parmap=dropdown_tab1-some_dropdown'
|
||||
)
|
||||
})
|
||||
|
||||
it('4 | the other tab maps its own parameter to the same column', () => {
|
||||
// Same table, same column, a different parameter label and value (Option 1
|
||||
// occurs four times) - the per-instance URL mapping is what keeps the two
|
||||
// tabs independent.
|
||||
openEditorInVaMode(
|
||||
'mpe_x_test',
|
||||
() => {
|
||||
pushVaMessage({
|
||||
name: 'pr1',
|
||||
label: 'DROPDOWN_TAB2',
|
||||
dataType: 'string',
|
||||
value: 'Option 1'
|
||||
})
|
||||
cy.get('.infoBar-text', { timeout: longerCommandTimeout }).should(
|
||||
'contain.text',
|
||||
'SOME_DROPDOWN'
|
||||
)
|
||||
cy.get('.infoBar-text').should('contain.text', "'Option 1'")
|
||||
expectFewerRowsThan(unfilteredRows)
|
||||
},
|
||||
'va_parmap=DROPDOWN_TAB2-SOME_DROPDOWN'
|
||||
)
|
||||
})
|
||||
|
||||
it('5 | warns in the grid when a mapping names a column the table lacks', () => {
|
||||
// Written by hand in the report's URL, so a typo must be visible to the
|
||||
// report author: the parameter is skipped, and the reason is shown rather
|
||||
// than leaving an unfiltered grid that looks like a working filter.
|
||||
openEditorInVaMode(
|
||||
'mpe_x_test',
|
||||
() => {
|
||||
pushVaMessage({
|
||||
name: 'pr1',
|
||||
label: 'DROPDOWN_TAB1',
|
||||
dataType: 'string',
|
||||
value: 'Option 2'
|
||||
})
|
||||
cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
|
||||
'contain.text',
|
||||
'VA parameter mapping problem'
|
||||
)
|
||||
cy.get('.va-parmap-warning').should('contain.text', 'DROPDOWN_TAB1')
|
||||
cy.get('.va-parmap-warning').should('contain.text', 'NOT_A_COLUMN')
|
||||
// VA mode hides the app header, so the banner must paint inside the
|
||||
// viewport on its own - assert the real box, not just its presence.
|
||||
cy.get('.va-parmap-warning').should(($alert) => {
|
||||
const rect = $alert[0].getBoundingClientRect()
|
||||
expect(rect.width, 'banner width').to.be.greaterThan(100)
|
||||
expect(rect.height, 'banner height').to.be.greaterThan(20)
|
||||
expect(rect.top, 'banner top').to.be.at.least(0)
|
||||
expect(rect.bottom, 'banner bottom').to.be.at.most(
|
||||
Cypress.config('viewportHeight')
|
||||
)
|
||||
})
|
||||
cy.screenshot('va-parmap-banner', { capture: 'viewport' })
|
||||
// Nothing was filtered: the row count matches the unfiltered baseline.
|
||||
expectRows((rows) => expect(rows).to.equal(unfilteredRows))
|
||||
},
|
||||
'va_parmap=DROPDOWN_TAB1-NOT_A_COLUMN'
|
||||
)
|
||||
})
|
||||
|
||||
it('6 | warns straight away when a mapping entry cannot be parsed', () => {
|
||||
// No separator, so the entry never becomes a mapping - reported before any
|
||||
// VA message arrives.
|
||||
openEditorInVaMode(
|
||||
'mpe_x_test',
|
||||
() => {
|
||||
cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
|
||||
'contain.text',
|
||||
'it needs a "-" between the VA parameter and the column name'
|
||||
)
|
||||
},
|
||||
'va_parmap=DROPDOWN_TAB1'
|
||||
)
|
||||
})
|
||||
|
||||
it('7 | warns when a mapping names a parameter VA is not sending', () => {
|
||||
// Parameter-side typo: the mapping never applies, so without the banner this
|
||||
// is indistinguishable from a working filter over every row.
|
||||
openEditorInVaMode(
|
||||
'mpe_x_test',
|
||||
() => {
|
||||
pushVaMessage({
|
||||
name: 'pr1',
|
||||
label: 'DROPDOWN_TAB1',
|
||||
dataType: 'string',
|
||||
value: 'Option 2'
|
||||
})
|
||||
// One message only tells us this object's parameter set is still being
|
||||
// discovered; the warning waits for a message that introduces no new name.
|
||||
cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
|
||||
'not.exist'
|
||||
)
|
||||
pushVaMessage({
|
||||
name: 'pr1',
|
||||
label: 'DROPDOWN_TAB1',
|
||||
dataType: 'string',
|
||||
value: 'Option 2'
|
||||
})
|
||||
cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
|
||||
'contain.text',
|
||||
'names VA parameter "dropdown_tab_1", which VA is not sending'
|
||||
)
|
||||
expectRows((rows) => expect(rows).to.equal(unfilteredRows))
|
||||
},
|
||||
'va_parmap=DROPDOWN_TAB_1-SOME_DROPDOWN'
|
||||
)
|
||||
})
|
||||
|
||||
it('8 | applies a mapping from the buffered message on the very first load', () => {
|
||||
// VA posts once on iframe load, before the app bootstraps, so the first
|
||||
// getdata is already filtered from that buffered message - with no column
|
||||
// metadata available, the mapping is what names the column.
|
||||
visitWithBufferedMessage(
|
||||
'va_parmap=DROPDOWN_TAB1-SOME_DROPDOWN',
|
||||
{
|
||||
name: 'pr1',
|
||||
label: 'DROPDOWN_TAB1',
|
||||
dataType: 'string',
|
||||
value: 'Option 2'
|
||||
},
|
||||
() => {
|
||||
cy.get('.infoBar-text', { timeout: longerCommandTimeout }).should(
|
||||
'contain.text',
|
||||
"SOME_DROPDOWN = 'Option 2'"
|
||||
)
|
||||
expectFewerRowsThan(unfilteredRows)
|
||||
cy.url().should('match', /\/editor\/[^/]+\/\d+/)
|
||||
}
|
||||
)
|
||||
})
|
||||
|
||||
it('9 | a bad mapping on the first load still ends at the banner', () => {
|
||||
// The deferred first load cannot check the column against metadata, so its
|
||||
// saveQuery may fail and fall back to an unfiltered load - the banner is
|
||||
// what stops that reading as a working filter.
|
||||
visitWithBufferedMessage(
|
||||
'va_parmap=DROPDOWN_TAB1-NOT_A_COLUMN',
|
||||
{
|
||||
name: 'pr1',
|
||||
label: 'DROPDOWN_TAB1',
|
||||
dataType: 'string',
|
||||
value: 'Option 2'
|
||||
},
|
||||
() => {
|
||||
cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
|
||||
'contain.text',
|
||||
'NOT_A_COLUMN'
|
||||
)
|
||||
cy.get('.va-parmap-warning').should('contain.text', 'DROPDOWN_TAB1')
|
||||
// Deliberately no row-count assertion: this path cannot validate the
|
||||
// column (no metadata yet), so the outcome depends on the platform - on
|
||||
// SAS, mp_filtercheck aborts the save and the load falls back to
|
||||
// unfiltered; the JS mock stores the filter and returns 0 rows. The
|
||||
// banner is the part that must hold either way, because both outcomes
|
||||
// are otherwise indistinguishable from a working filter.
|
||||
}
|
||||
)
|
||||
})
|
||||
|
||||
it('10 | warns when two parameters filter the same column (an empty grid)', () => {
|
||||
// Clauses are AND-ed, so two different values on one column produce no rows
|
||||
// at all - the last silent-empty-grid path. Reported rather than rewritten,
|
||||
// so the author decides whether to use one parameter carrying both values.
|
||||
openEditorInVaMode(
|
||||
'mpe_x_test',
|
||||
() => {
|
||||
pushVaMessage(
|
||||
{
|
||||
name: 'pr1',
|
||||
label: 'DROPDOWN_TAB1',
|
||||
dataType: 'string',
|
||||
value: 'Option 1'
|
||||
},
|
||||
{
|
||||
name: 'pr2',
|
||||
label: 'DROPDOWN_TAB2',
|
||||
dataType: 'string',
|
||||
value: 'Option 2'
|
||||
}
|
||||
)
|
||||
cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
|
||||
'contain.text',
|
||||
'combined with AND'
|
||||
)
|
||||
cy.get('.va-parmap-warning').should('contain.text', 'DROPDOWN_TAB1')
|
||||
cy.get('.va-parmap-warning').should('contain.text', 'DROPDOWN_TAB2')
|
||||
// The consequence: rows must match both clauses, and the two values
|
||||
// differ, so none can match.
|
||||
expectRows((rows) => expect(rows, 'matching rows').to.equal(0))
|
||||
},
|
||||
'va_parmap=DROPDOWN_TAB1-SOME_DROPDOWN&va_parmap=DROPDOWN_TAB2-SOME_DROPDOWN'
|
||||
)
|
||||
})
|
||||
|
||||
it('12 | a transform adds the previous quarter to the filter', () => {
|
||||
// ADD(QUARTER_START,-1) is the reported requirement: one clause carrying the
|
||||
// selected date and the same point one quarter earlier. 2026-10-01 = 24380,
|
||||
// 2026-07-01 = 24288 in SAS days; nothing in the fixture matches either, so
|
||||
// the assertion is on the clause rather than the row count.
|
||||
openEditorInVaMode(
|
||||
'mpe_x_test',
|
||||
() => {
|
||||
pushVaMessage({
|
||||
name: 'pr1',
|
||||
label: 'SELECTED_DATE',
|
||||
dataType: 'string',
|
||||
value: '01OCT2026'
|
||||
})
|
||||
cy.get('.infoBar-text', { timeout: longerCommandTimeout }).should(
|
||||
'contain.text',
|
||||
'SOME_DATE IN ('
|
||||
)
|
||||
cy.get('.infoBar-text').should('contain.text', '24380')
|
||||
cy.get('.infoBar-text').should('contain.text', '24288')
|
||||
cy.get('h4').should('contain.text', '(0 rows')
|
||||
},
|
||||
'va_parmap=SELECTED_DATE-SOME_DATE:ADD(QUARTER_START,-1)'
|
||||
)
|
||||
})
|
||||
|
||||
it('13 | warns when a transform lands on a column it cannot apply to', () => {
|
||||
openEditorInVaMode(
|
||||
'mpe_x_test',
|
||||
() => {
|
||||
pushVaMessage({
|
||||
name: 'pr1',
|
||||
label: 'SELECTED_DATE',
|
||||
dataType: 'string',
|
||||
value: 'x'
|
||||
})
|
||||
cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
|
||||
'contain.text',
|
||||
'is not a date or datetime column'
|
||||
)
|
||||
cy.get('h4').should('contain.text', '(7 rows')
|
||||
},
|
||||
'va_parmap=SELECTED_DATE-SOME_CHAR:ADD(QUARTER_START,-1)'
|
||||
)
|
||||
})
|
||||
|
||||
it('11 | logs a parse-time problem once, however many messages arrive', () => {
|
||||
// The parse problem is logged at app init; the editor's own loop must not log
|
||||
// it again. The second entry is valid-but-missing, so the loop definitely runs
|
||||
// and logs its own line - which is what makes this a real assertion.
|
||||
cy.visit(
|
||||
`${hostUrl}${appLocation}/index.html#/editor/DC_JSLIB.MPE_X_TEST?embed=va&va_parmap=DROPDOWN_TAB1&va_parmap=DROPDOWN_TAB2-NOT_A_COLUMN`,
|
||||
{
|
||||
onBeforeLoad(win: any) {
|
||||
win.__warns = []
|
||||
const original = win.console.warn.bind(win.console)
|
||||
win.console.warn = (...args: any[]) => {
|
||||
win.__warns.push(args.join(' '))
|
||||
original(...args)
|
||||
}
|
||||
}
|
||||
}
|
||||
)
|
||||
cy.get('.app-loading', { timeout: longerCommandTimeout }).should(
|
||||
'not.exist'
|
||||
)
|
||||
pushVaMessage({
|
||||
name: 'pr2',
|
||||
label: 'DROPDOWN_TAB2',
|
||||
dataType: 'string',
|
||||
value: 'Option 2'
|
||||
})
|
||||
cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
|
||||
'contain.text',
|
||||
'NOT_A_COLUMN'
|
||||
)
|
||||
cy.window().then((win: any) => {
|
||||
const warns: string[] = win.__warns || []
|
||||
const parseLines = warns.filter((w) =>
|
||||
w.includes('va_parmap "DROPDOWN_TAB1"')
|
||||
)
|
||||
const runtimeLines = warns.filter((w) => w.includes('NOT_A_COLUMN'))
|
||||
expect(parseLines.length, warns.join(' | ')).to.eq(1)
|
||||
expect(runtimeLines.length, warns.join(' | ')).to.eq(1)
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
/**
|
||||
* Loads the editor with a VA message already in the pre-bootstrap buffer, the
|
||||
* way VA's own postMessage arrives before Angular starts. A distinct pathname is
|
||||
* used so this is a real document load: a cy.visit that differs only in the hash
|
||||
* is a same-document navigation and would not re-bootstrap the app.
|
||||
*/
|
||||
const visitWithBufferedMessage = (
|
||||
extraParams: string,
|
||||
parameter: any,
|
||||
callback?: any
|
||||
) => {
|
||||
cy.visit(
|
||||
`${hostUrl}${appLocation}/index.html#/editor/DC_JSLIB.MPE_X_TEST?embed=va&${extraParams}`,
|
||||
{
|
||||
onBeforeLoad(win: any) {
|
||||
win.__vaLastMessage = {
|
||||
t: Date.now(),
|
||||
origin: win.location.origin,
|
||||
data: {
|
||||
version: '1',
|
||||
resultName: 'dd1',
|
||||
rowCount: 0,
|
||||
availableRowCount: 0,
|
||||
data: [],
|
||||
columns: [],
|
||||
parameters: [parameter]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
)
|
||||
cy.get('.app-loading', { timeout: longerCommandTimeout }).should('not.exist')
|
||||
if (callback) callback()
|
||||
}
|
||||
|
||||
/** Pushes one VA data-driven content message carrying the given parameters. */
|
||||
const pushVaMessage = (...parameters: any[]) => {
|
||||
cy.window().then((win) => {
|
||||
win.postMessage(
|
||||
{
|
||||
version: '1',
|
||||
resultName: 'dd1',
|
||||
rowCount: 0,
|
||||
availableRowCount: 0,
|
||||
data: [],
|
||||
columns: [],
|
||||
parameters
|
||||
},
|
||||
'*'
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
// Opens a table from the tree (which routes to #/editor/LIB.TABLE), then
|
||||
// re-visits the same route with ?embed=va so the app parses VA embed mode.
|
||||
const openEditorInVaMode = (tablename: string, callback?: any) => {
|
||||
// `extraParams` is appended to that query string (e.g. va_parmap entries).
|
||||
const openEditorInVaMode = (
|
||||
tablename: string,
|
||||
callback?: any,
|
||||
extraParams?: string
|
||||
) => {
|
||||
openTableFromTree(libraryToOpenIncludes, tablename)
|
||||
|
||||
cy.get('#hotTable', { timeout: longerCommandTimeout })
|
||||
@@ -84,7 +498,13 @@ const openEditorInVaMode = (tablename: string, callback?: any) => {
|
||||
.then(() => {
|
||||
cy.url().then((url) => {
|
||||
const separator = url.includes('?') ? '&' : '?'
|
||||
cy.visit(`${url}${separator}embed=va`)
|
||||
const extra = extraParams ? `&${extraParams}` : ''
|
||||
cy.visit(`${url}${separator}embed=va${extra}`)
|
||||
// A cy.visit that differs from the current URL only in the hash is a
|
||||
// same-document navigation, so the app is not re-bootstrapped and the
|
||||
// embed parameter (read once at app init) never takes effect. Reload to
|
||||
// bootstrap it, as the VA iframe load does.
|
||||
cy.reload()
|
||||
cy.get('.app-loading', { timeout: longerCommandTimeout }).should(
|
||||
'not.exist'
|
||||
)
|
||||
|
||||
@@ -14,11 +14,10 @@ context('metanav tests: ', function () {
|
||||
})
|
||||
|
||||
this.beforeEach(() => {
|
||||
cy.visit(hostUrl + appLocation)
|
||||
cy.get('input.username').type(username)
|
||||
cy.get('input.password').type(password)
|
||||
cy.get('.login-group button').click()
|
||||
|
||||
// beforeAll has already logged in. Typing into the login form here failed
|
||||
// once the session existed - the form is not on screen, so Cypress reported
|
||||
// "the center of this element is hidden from view" and the whole suite was
|
||||
// skipped behind that hook.
|
||||
visitPage('view/metadata')
|
||||
})
|
||||
|
||||
|
||||
+3
-1
@@ -37,7 +37,9 @@
|
||||
"compodoc:build-and-serve": "compodoc -p tsconfig.doc.json -s --name 'Data Controller Client'",
|
||||
"compodoc:serve": "compodoc -s --name 'Data Controller Client'",
|
||||
"lighthouse": "lhci autorun",
|
||||
"ng": "ng"
|
||||
"ng": "ng",
|
||||
"test:tz-west": "TZ=America/New_York npx ng test --no-watch --no-progress --browsers ChromeHeadlessCI --include='**/va-*.spec.ts'",
|
||||
"test:tz-east": "TZ=Pacific/Auckland npx ng test --no-watch --no-progress --browsers ChromeHeadlessCI --include='**/va-*.spec.ts'"
|
||||
},
|
||||
"private": true,
|
||||
"dependencies": {
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { QueryClause } from './models/TableData'
|
||||
import { VaParmapTarget } from './shared/utils/parse-va-parmap'
|
||||
|
||||
/**
|
||||
* Filtering cache info, to be reused when filtering modal is re-open
|
||||
@@ -61,6 +62,21 @@ export const globals: {
|
||||
* Defaults to 'live'; toggled by the Auto-apply checkbox in the editor.
|
||||
*/
|
||||
vaApplyMode: 'live' | 'confirm'
|
||||
/**
|
||||
* Explicit VA parameter -> DC column mappings, parsed from the `va_parmap`
|
||||
* URL parameter (see parseVaParmap). Empty unless configured; entries override
|
||||
* VA's default same-name (parameter label -> column) resolution, so two DC
|
||||
* instances over one table can filter the same column from differently-named
|
||||
* VA parameters.
|
||||
*/
|
||||
vaParamMap: Map<string, VaParmapTarget>
|
||||
/**
|
||||
* `va_parmap` entries that could not be used at all (not a plain SAS column
|
||||
* name, or missing a side of the `-`). Surfaced to the report author, because
|
||||
* these are hand-written in the report's URL and a typo otherwise looks like
|
||||
* a working, unfiltered view.
|
||||
*/
|
||||
vaParmapProblems: string[]
|
||||
rootParam: string
|
||||
dcLib: string
|
||||
xlmaps: XLMapListItem[]
|
||||
@@ -77,6 +93,8 @@ export const globals: {
|
||||
} = {
|
||||
embed: false,
|
||||
vaApplyMode: 'live',
|
||||
vaParamMap: new Map(),
|
||||
vaParmapProblems: [],
|
||||
rootParam: <string>'',
|
||||
dcLib: '',
|
||||
xlmaps: [],
|
||||
|
||||
@@ -12,6 +12,7 @@ import '@clr/icons'
|
||||
import '@clr/icons/shapes/all-shapes'
|
||||
import { globals } from './_globals'
|
||||
import { parseEmbedParam } from './shared/utils/parse-embed-param'
|
||||
import { parseVaParmap } from './shared/utils/parse-va-parmap'
|
||||
import moment from 'moment'
|
||||
import { EventService } from './services/event.service'
|
||||
import { AppService } from './services/app.service'
|
||||
@@ -163,6 +164,21 @@ export class AppComponent {
|
||||
globals.embed = embed
|
||||
this.embed = embed
|
||||
}
|
||||
// VA parameter -> DC column mappings for this instance, held on globals
|
||||
// (like `embed` and `vaApplyMode`) so in-app navigation that rebuilds the
|
||||
// editor keeps them - e.g. returning from the stage / approve pages.
|
||||
if (globals.embed === 'va') {
|
||||
const parmap = parseVaParmap(hashQuery ?? '')
|
||||
globals.vaParamMap = parmap.map
|
||||
globals.vaParmapProblems = parmap.problems
|
||||
// Visible in the browser console as well as the editor banner: the entry
|
||||
// is hand-written in the report's URL, so a typo has to be findable when
|
||||
// someone is looking at the report rather than at DC's UI.
|
||||
for (const problem of parmap.problems) {
|
||||
// eslint-disable-next-line no-console
|
||||
console.warn('Data Controller: ' + problem)
|
||||
}
|
||||
}
|
||||
|
||||
this.subscribeToShowAbortModal()
|
||||
this.subscribeToRequestsModal()
|
||||
|
||||
@@ -1,7 +1,36 @@
|
||||
<main class="content-area position-relative">
|
||||
<!-- An embedded instance (a report object) has no report author who can deploy:
|
||||
showing the wizard there only produces a screen whose buttons cannot work,
|
||||
because the deployment needs an authenticated, non-embedded session. Name
|
||||
the state instead, and keep the wizard for standalone use. -->
|
||||
@if (isEmbedded) {
|
||||
<div class="clr-row">
|
||||
<div id="not-configured" class="card">
|
||||
<div class="card-header">Data Controller is not configured</div>
|
||||
<div class="card-block">
|
||||
<div class="card-text">
|
||||
<p class="mt-0">
|
||||
This Data Controller instance has not been deployed yet, so it
|
||||
cannot be shown inside a report.
|
||||
</p>
|
||||
<p>
|
||||
Deployment has to be done once, in its own browser tab, by someone
|
||||
who can write to the application location:
|
||||
<a [href]="deployUrl" target="_blank">{{ deployUrl }}</a>
|
||||
</p>
|
||||
<p>
|
||||
When it is configured, reload the report and this object will show
|
||||
the editor.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
}
|
||||
|
||||
<div class="clr-row">
|
||||
<!-- T&C section -->
|
||||
@if (step === 0) {
|
||||
@if (!isEmbedded && step === 0) {
|
||||
<div id="TCS" class="card">
|
||||
<div class="card-header">Terms and Conditions</div>
|
||||
<div class="card-block">
|
||||
@@ -35,7 +64,7 @@
|
||||
</div>
|
||||
<!-- T&C section end -->
|
||||
|
||||
@if (step > 0) {
|
||||
@if (!isEmbedded && step > 0) {
|
||||
@switch (true) {
|
||||
@case (sasJsConfig.serverType === ServerType.SasViya) {
|
||||
<div class="autodeploy-section card">
|
||||
|
||||
@@ -0,0 +1,229 @@
|
||||
import { NO_ERRORS_SCHEMA } from '@angular/core'
|
||||
import { ComponentFixture, TestBed } from '@angular/core/testing'
|
||||
import { ServerType } from '@sasjs/utils/types/serverType'
|
||||
import { DeployComponent } from './deploy.component'
|
||||
import { standaloneUrl } from './standalone-url'
|
||||
import { globals } from '../_globals'
|
||||
import { AppStoreService } from '../services/app-store.service'
|
||||
import { SasService } from '../services/sas.service'
|
||||
|
||||
/**
|
||||
* An embedded instance cannot deploy itself: the deployment needs an
|
||||
* authenticated, standalone session, and the person looking at a report object
|
||||
* is not the person who installs the app. The component therefore withholds the
|
||||
* wizard and names the state instead.
|
||||
*
|
||||
* These cases render the template, so they fail if the `!isEmbedded` guards in
|
||||
* deploy.component.html are dropped - asserting the getter alone would not.
|
||||
*/
|
||||
describe('DeployComponent embed guard', () => {
|
||||
let fixture: ComponentFixture<DeployComponent>
|
||||
let component: DeployComponent
|
||||
|
||||
const originalEmbed = globals.embed
|
||||
|
||||
const configure = () =>
|
||||
TestBed.configureTestingModule({
|
||||
declarations: [DeployComponent],
|
||||
providers: [
|
||||
{
|
||||
provide: AppStoreService,
|
||||
useValue: {
|
||||
getDcAdapterSettings: () => ({
|
||||
appLoc: '/Public/app/dc',
|
||||
dcPath: '/tmp/dc'
|
||||
})
|
||||
}
|
||||
},
|
||||
{
|
||||
provide: SasService,
|
||||
useValue: {
|
||||
getSasjsInstance: () => ({}),
|
||||
getSasjsConfig: () => ({ serverType: ServerType.Sasjs })
|
||||
}
|
||||
}
|
||||
],
|
||||
// The wizard renders child components this spec does not exercise.
|
||||
schemas: [NO_ERRORS_SCHEMA]
|
||||
}).compileComponents()
|
||||
|
||||
const render = () => {
|
||||
fixture = TestBed.createComponent(DeployComponent)
|
||||
component = fixture.componentInstance
|
||||
fixture.detectChanges()
|
||||
}
|
||||
|
||||
const tcsSection = () =>
|
||||
fixture.nativeElement.querySelector('#TCS') as HTMLElement | null
|
||||
const notConfigured = () =>
|
||||
fixture.nativeElement.querySelector('#not-configured') as HTMLElement | null
|
||||
const deploySection = () =>
|
||||
fixture.nativeElement.querySelector(
|
||||
'.autodeploy-section'
|
||||
) as HTMLElement | null
|
||||
|
||||
afterEach(() => {
|
||||
globals.embed = originalEmbed
|
||||
TestBed.resetTestingModule()
|
||||
})
|
||||
|
||||
it('offers the wizard when running in its own tab', () => {
|
||||
globals.embed = false
|
||||
configure()
|
||||
render()
|
||||
|
||||
expect(tcsSection()).toBeTruthy()
|
||||
expect(notConfigured()).toBeNull()
|
||||
expect(component.isEmbedded).toBeFalse()
|
||||
})
|
||||
|
||||
it('replaces the wizard with the notice for a VA embed', () => {
|
||||
globals.embed = 'va'
|
||||
configure()
|
||||
render()
|
||||
|
||||
expect(tcsSection()).toBeNull()
|
||||
expect(notConfigured()).toBeTruthy()
|
||||
})
|
||||
|
||||
it('replaces the wizard with the notice for a plain embed too', () => {
|
||||
globals.embed = true
|
||||
configure()
|
||||
render()
|
||||
|
||||
expect(tcsSection()).toBeNull()
|
||||
expect(notConfigured()).toBeTruthy()
|
||||
})
|
||||
|
||||
it('names the standalone URL in the embedded notice', () => {
|
||||
globals.embed = 'va'
|
||||
configure()
|
||||
render()
|
||||
|
||||
const link = notConfigured()!.querySelector('a') as HTMLAnchorElement
|
||||
|
||||
expect(link.getAttribute('href')).toBe(component.deployUrl)
|
||||
expect(link.textContent).toBe(component.deployUrl)
|
||||
})
|
||||
|
||||
it('withholds the deploy section after the terms are accepted, when embedded', () => {
|
||||
globals.embed = 'va'
|
||||
configure()
|
||||
render()
|
||||
|
||||
component.step = 1
|
||||
fixture.detectChanges()
|
||||
|
||||
expect(deploySection()).toBeNull()
|
||||
})
|
||||
|
||||
it('shows the deploy section after the terms are accepted, standalone', () => {
|
||||
globals.embed = false
|
||||
configure()
|
||||
render()
|
||||
|
||||
component.step = 1
|
||||
fixture.detectChanges()
|
||||
|
||||
expect(deploySection()).toBeTruthy()
|
||||
})
|
||||
})
|
||||
|
||||
describe('DeployComponent onNavigateToHome', () => {
|
||||
const originalEmbed = globals.embed
|
||||
const originalHref = window.location.href
|
||||
|
||||
afterEach(() => {
|
||||
globals.embed = originalEmbed
|
||||
history.replaceState(null, '', originalHref)
|
||||
TestBed.resetTestingModule()
|
||||
})
|
||||
|
||||
it('opens the standalone URL, merging an embed tab into the app URL', () => {
|
||||
// Karma serves the runner with its own query, so the app URL is set here
|
||||
// rather than read from window.location: the assertion has to see the
|
||||
// app's query survive the embed parameters being dropped. The shape is
|
||||
// the real SAS 9 one - the app's `_program` query in the search part,
|
||||
// the embed switches in the hash query.
|
||||
history.replaceState(
|
||||
null,
|
||||
'',
|
||||
'/deploy-tc?_program=/Public/app/dc&_debug=131#/editor?embed=va&va_parmap=A-B'
|
||||
)
|
||||
const open = spyOn(window, 'open')
|
||||
TestBed.configureTestingModule({
|
||||
declarations: [DeployComponent],
|
||||
providers: [
|
||||
{
|
||||
provide: AppStoreService,
|
||||
useValue: { getDcAdapterSettings: () => ({ appLoc: '', dcPath: '' }) }
|
||||
},
|
||||
{
|
||||
provide: SasService,
|
||||
useValue: {
|
||||
getSasjsInstance: () => ({}),
|
||||
getSasjsConfig: () => ({ serverType: ServerType.Sasjs })
|
||||
}
|
||||
}
|
||||
],
|
||||
schemas: [NO_ERRORS_SCHEMA]
|
||||
})
|
||||
const fixture = TestBed.createComponent(DeployComponent)
|
||||
|
||||
fixture.componentInstance.onNavigateToHome()
|
||||
|
||||
const opened = open.calls.mostRecent().args[0] as string
|
||||
expect(opened).toBe(
|
||||
window.location.origin + '/deploy-tc?_program=/Public/app/dc&_debug=131'
|
||||
)
|
||||
expect(opened).not.toContain('#')
|
||||
expect(opened).not.toContain('embed=')
|
||||
})
|
||||
})
|
||||
|
||||
describe('standaloneUrl', () => {
|
||||
const origin = window.location.origin
|
||||
|
||||
it('keeps the path of a plain URL', () => {
|
||||
expect(standaloneUrl(`${origin}/SASJobExecution/dc`)).toBe(
|
||||
`${origin}/SASJobExecution/dc`
|
||||
)
|
||||
})
|
||||
|
||||
it('keeps a query the app needs, and drops the hash route', () => {
|
||||
expect(standaloneUrl(`${origin}/path?x=1#/deploy`)).toBe(
|
||||
`${origin}/path?x=1`
|
||||
)
|
||||
})
|
||||
|
||||
it('keeps `_program=`, which is how SAS 9 and Viya address the app', () => {
|
||||
// The whole app URL is the query here: /SASJobExecution/?_program=... is
|
||||
// the standalone app, so dropping the query would open a blank executor.
|
||||
const sas9 =
|
||||
`${origin}/SASJobExecution/?_program=/Public/app/dc/services/public/startupservice` +
|
||||
'&_debug=131'
|
||||
expect(standaloneUrl(sas9)).toBe(sas9)
|
||||
})
|
||||
|
||||
it('drops a bare embed query', () => {
|
||||
expect(standaloneUrl(`${origin}/app?embed=va`)).toBe(`${origin}/app`)
|
||||
})
|
||||
|
||||
it('drops the va_parmap entries along with the embed switch', () => {
|
||||
expect(
|
||||
standaloneUrl(
|
||||
`${origin}/app?embed=va&va_parmap=A-B&va_parmap=C-D#/editor/x`
|
||||
)
|
||||
).toBe(`${origin}/app`)
|
||||
})
|
||||
|
||||
it('keeps a query parameter that merely starts with va_', () => {
|
||||
expect(standaloneUrl(`${origin}/app?embed=va&va_rowlimit=100`)).toBe(
|
||||
`${origin}/app?va_rowlimit=100`
|
||||
)
|
||||
})
|
||||
|
||||
it('resolves a relative URL against the current origin', () => {
|
||||
expect(standaloneUrl('/app/#/deploy')).toBe(`${origin}/app/`)
|
||||
})
|
||||
})
|
||||
@@ -4,6 +4,8 @@ import { SASjsConfig } from '@sasjs/adapter'
|
||||
import { ServerType } from '@sasjs/utils/types/serverType'
|
||||
import { AppStoreService } from '../services/app-store.service'
|
||||
import { DcAdapterSettings } from '../models/DcAdapterSettings'
|
||||
import { globals } from '../_globals'
|
||||
import { standaloneUrl } from './standalone-url'
|
||||
|
||||
@Component({
|
||||
selector: 'app-deploy',
|
||||
@@ -44,6 +46,20 @@ export class DeployComponent implements OnInit {
|
||||
this.setDeployDefaults()
|
||||
}
|
||||
|
||||
/**
|
||||
* True when DC is running inside a report rather than in its own tab. The
|
||||
* deployment needs an authenticated, standalone session, so the wizard is not
|
||||
* offered here - a report object cannot deploy the app that it is displaying.
|
||||
*/
|
||||
get isEmbedded(): boolean {
|
||||
return !!globals.embed
|
||||
}
|
||||
|
||||
/** The standalone URL an administrator deploys from, for the embedded notice. */
|
||||
get deployUrl(): string {
|
||||
return standaloneUrl(window.location.href)
|
||||
}
|
||||
|
||||
/**
|
||||
* Setting default values used for deploy request
|
||||
*/
|
||||
@@ -60,6 +76,8 @@ export class DeployComponent implements OnInit {
|
||||
}
|
||||
|
||||
public onNavigateToHome() {
|
||||
window.open(location.href.split('#')[0], '_blank')
|
||||
// Same helper as deployUrl: splitting on '#' left the query string in
|
||||
// place, so a tab opened from `?embed=va#/deploy` stayed embedded.
|
||||
window.open(standaloneUrl(window.location.href), '_blank')
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
/**
|
||||
* The URL a Data Controller instance runs at when it is not embedded: the same
|
||||
* URL, without the parts that make it an embed - the `embed` switch, the
|
||||
* `va_parmap` entries, and the hash route.
|
||||
*
|
||||
* The rest of the query is kept. On SAS 9 and Viya the app is addressed
|
||||
* entirely through it (`/SASJobExecution/?_program=/.../startupservice`), so
|
||||
* dropping the query would open a blank executor rather than the app, and any
|
||||
* other parameter the deployment relies on would go the same way.
|
||||
*
|
||||
* Only these two parameters are embed parameters. A later parameter that
|
||||
* happens to start with `va_` belongs to the app, not the embed, so it stays.
|
||||
*
|
||||
* Parsing with URL also handles the relative hrefs the adapter can hand back,
|
||||
* which plain string splicing cannot.
|
||||
*/
|
||||
const EMBED_PARAMS = ['embed', 'va_parmap']
|
||||
|
||||
export const standaloneUrl = (href: string): string => {
|
||||
const url = new URL(href, window.location.origin)
|
||||
url.hash = ''
|
||||
for (const key of Array.from(url.searchParams.keys())) {
|
||||
if (EMBED_PARAMS.includes(key.toLowerCase())) {
|
||||
url.searchParams.delete(key)
|
||||
}
|
||||
}
|
||||
|
||||
return url.toString()
|
||||
}
|
||||
@@ -459,6 +459,43 @@
|
||||
<!--<hr>-->
|
||||
<!--{{ validationDone }}-->
|
||||
|
||||
<!-- VA embed: something about a `va_parmap` entry in this report
|
||||
object's URL needs the report author's attention - it could not be
|
||||
used, it was applied but AND-ed against another parameter, or it
|
||||
names a parameter VA never sends. Shown as a banner, not just a
|
||||
console warning, because
|
||||
the entries are written by the report author and a typo otherwise
|
||||
looks like a working filter over the whole table. -->
|
||||
@if (isVaEmbed && vaMappingWarnings.length) {
|
||||
<clr-alert
|
||||
class="va-parmap-warning"
|
||||
[clrAlertType]="'warning'"
|
||||
[clrAlertClosable]="false"
|
||||
>
|
||||
<clr-alert-item>
|
||||
<div class="alert-text">
|
||||
<strong>VA parameter mapping problem</strong>
|
||||
@for (warning of vaMappingWarnings; track warning) {
|
||||
<div>{{ warning }}</div>
|
||||
}
|
||||
<!-- The reader is the report author, who can only fix this in
|
||||
the report: name the parameter they edit, and say what a
|
||||
correct entry looks like, so the message is actionable
|
||||
without the docs. -->
|
||||
<div>
|
||||
Fix: edit the <code>va_parmap</code> entries in the URL of
|
||||
this Data Controller object in the report. An entry is
|
||||
<code>parameter-column</code>, optionally
|
||||
<code>:ADD(anchor,offset)</code> or
|
||||
<code>:RANGE(period,offset)</code> - for example
|
||||
<code>va_parmap=SEL_DATE-D_DATE:ADD(QUARTER_START,-1)</code
|
||||
>.
|
||||
</div>
|
||||
</div>
|
||||
</clr-alert-item>
|
||||
</clr-alert>
|
||||
}
|
||||
|
||||
<div class="hot-wrapper clr-flex-1">
|
||||
<hot-table
|
||||
#hotInstance
|
||||
|
||||
@@ -27,3 +27,15 @@
|
||||
font-style: italic;
|
||||
opacity: 0.7;
|
||||
}
|
||||
|
||||
/* VA embed: `va_parmap` diagnostic banner. Clarity's alert lays `.alert-title`
|
||||
and `.alert-text` out on one row, which pushes a long message off the right
|
||||
edge; this keeps the whole message in the flow and wraps it. */
|
||||
.va-parmap-warning {
|
||||
width: 100%;
|
||||
|
||||
.alert-text {
|
||||
white-space: normal;
|
||||
overflow-wrap: anywhere;
|
||||
}
|
||||
}
|
||||
@@ -73,6 +73,7 @@ import {
|
||||
} from '../shared/dc-validator/utils/autoEscapedCellTracker'
|
||||
import { HyperFormula } from 'hyperformula'
|
||||
import { parseLabelsParam } from '../shared/utils/parse-labels-param'
|
||||
import { VaParmapTarget } from '../shared/utils/parse-va-parmap'
|
||||
import { getDisplayColHeaders } from '../shared/utils/display-col-headers'
|
||||
import { buildColInfoHtml } from '../shared/utils/col-info-html'
|
||||
import { globals } from '../_globals'
|
||||
@@ -485,6 +486,46 @@ export class EditorComponent
|
||||
get isVaEmbed() {
|
||||
return globals.embed === 'va'
|
||||
}
|
||||
/**
|
||||
* Explicit VA parameter -> DC column mappings (`va_parmap`) for this instance.
|
||||
* Parsed once at app init (see parseVaParmap) and held on globals so the
|
||||
* editor reloads a filter apply performs keep using the same mapping.
|
||||
*/
|
||||
get vaParamMap(): Map<string, VaParmapTarget> {
|
||||
return globals.vaParamMap
|
||||
}
|
||||
/** `va_parmap` entries that name a column this table does not have. */
|
||||
public vaUnresolvedMappings: string[] = []
|
||||
/** `va_parmap` keys that no VA message has named (a parameter-side typo). */
|
||||
public vaUnmatchedMappings: string[] = []
|
||||
/** Parameters in this message that filter the same column (an AND-ed pair). */
|
||||
public vaSameColumnMappings: string[] = []
|
||||
/** Transform mappings this message cannot apply (non-temporal column or value). */
|
||||
public vaTransformProblems: string[] = []
|
||||
/** The warning text last written to the console, so repeated VA messages log once. */
|
||||
private vaWarningsReported = ''
|
||||
/**
|
||||
* Everything wrong with this instance's `va_parmap` entries, in the report
|
||||
* author's terms: entries that could not be parsed at all, plus entries that
|
||||
* name a column the open table does not have. Shown as a banner in the editor
|
||||
* - a report developer is usually not going to read the browser console.
|
||||
*/
|
||||
get vaMappingWarnings(): string[] {
|
||||
return [...globals.vaParmapProblems, ...this.vaRuntimeWarnings]
|
||||
}
|
||||
/**
|
||||
* The warnings this instance discovers for itself, i.e. everything except the
|
||||
* parse-time problems - those are logged once at app init (`app.component`),
|
||||
* so logging them again here would double every line in the console.
|
||||
*/
|
||||
get vaRuntimeWarnings(): string[] {
|
||||
return [
|
||||
...this.vaUnresolvedMappings,
|
||||
...this.vaUnmatchedMappings,
|
||||
...this.vaSameColumnMappings,
|
||||
...this.vaTransformProblems
|
||||
]
|
||||
}
|
||||
/**
|
||||
* VA filter mode. Live = auto-apply each VA change; confirm = stage the change
|
||||
* and wait for the user to click Apply. Backed by globals (toggled by the
|
||||
@@ -3602,7 +3643,7 @@ export class EditorComponent
|
||||
const msg = this.vaMessaging.latestMessage()
|
||||
if (!msg) return false
|
||||
|
||||
const clauses = this.vaFilter.buildInitialClauses(msg)
|
||||
const clauses = this.vaFilter.buildInitialClauses(msg, this.vaParamMap)
|
||||
if (clauses.length === 0) return false
|
||||
|
||||
try {
|
||||
@@ -5048,7 +5089,53 @@ export class EditorComponent
|
||||
|
||||
this.updateVaFilterText(msg)
|
||||
|
||||
const clauses = this.vaFilter.buildClauses(msg, labelToVar, this.cols)
|
||||
// A `va_parmap` entry naming a column this table does not have is skipped by
|
||||
// buildClauses. Report it (banner + console, once per distinct set) so the
|
||||
// report author sees a misconfiguration instead of an unfiltered grid.
|
||||
this.vaUnresolvedMappings = this.vaFilter.unresolvedMappings(
|
||||
msg,
|
||||
this.vaParamMap,
|
||||
this.cols,
|
||||
this.libds
|
||||
)
|
||||
// A key naming no parameter VA has ever sent is a typo on the parameter
|
||||
// side: the mapping never applies. Only judged once parameters are actually
|
||||
// arriving, since VA sends just the parameters its query uses.
|
||||
this.vaUnmatchedMappings = this.vaFilter.unmatchedMappings(
|
||||
this.vaParamMap,
|
||||
this.vaMessaging.seenParamKeys,
|
||||
this.vaMessaging.lastMessageParamsStable
|
||||
)
|
||||
// Two parameters filtering one column are AND-ed, so differing values leave
|
||||
// an empty grid. Reported, not rewritten: turning that AND into an OR is a
|
||||
// filter-semantics decision, taken in buildClauses and not in a diagnostic.
|
||||
this.vaSameColumnMappings = this.vaFilter.sameColumnMappings(
|
||||
msg,
|
||||
this.vaParamMap,
|
||||
this.cols
|
||||
)
|
||||
// A transform that cannot run here (char column, value that is not a date) is
|
||||
// skipped by the clause builder, so the banner has to say why.
|
||||
this.vaTransformProblems = this.vaFilter.unsupportedTransforms(
|
||||
msg,
|
||||
this.vaParamMap,
|
||||
this.cols
|
||||
)
|
||||
const warnings = this.vaRuntimeWarnings.join(' | ')
|
||||
if (warnings !== this.vaWarningsReported) {
|
||||
this.vaWarningsReported = warnings
|
||||
for (const warning of this.vaRuntimeWarnings) {
|
||||
// eslint-disable-next-line no-console
|
||||
console.warn('Data Controller: ' + warning)
|
||||
}
|
||||
}
|
||||
|
||||
const clauses = this.vaFilter.buildClauses(
|
||||
msg,
|
||||
labelToVar,
|
||||
this.cols,
|
||||
this.vaParamMap
|
||||
)
|
||||
const signature = this.vaFilter.signature(clauses)
|
||||
|
||||
if (signature === this.vaMessaging.filterSignature) {
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { VaFilterService } from './va-filter.service'
|
||||
import { VaParmapTarget } from '../shared/utils/parse-va-parmap'
|
||||
import { VaMessage } from './va-messaging.service'
|
||||
import { HelperService } from './helper.service'
|
||||
|
||||
@@ -14,6 +15,7 @@ function msg(parameters: any[]): VaMessage {
|
||||
|
||||
describe('VaFilterService', () => {
|
||||
let service: VaFilterService
|
||||
let helperService: HelperService
|
||||
|
||||
// DC column metadata used by the metadata path. cellValidation[0] is the
|
||||
// delete column and must be skipped by buildLabelToVarMap.
|
||||
@@ -42,7 +44,8 @@ describe('VaFilterService', () => {
|
||||
]
|
||||
|
||||
beforeEach(() => {
|
||||
service = new VaFilterService(new HelperService({} as any))
|
||||
helperService = new HelperService({} as any)
|
||||
service = new VaFilterService(helperService)
|
||||
})
|
||||
|
||||
const labelToVar = () =>
|
||||
@@ -199,7 +202,694 @@ describe('VaFilterService', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('buildClauses with an explicit va_parmap', () => {
|
||||
const build = (parameters: any[], paramMap: Map<string, VaParmapTarget>) =>
|
||||
service.buildClauses(msg(parameters), labelToVar(), cols, paramMap)
|
||||
|
||||
it('overrides the same-name match, including the value formatting', () => {
|
||||
// The VA label IS a DC column here (SOME_CHAR), but the mapping points at
|
||||
// a different one - the numeric column - so the value is formatted numeric.
|
||||
const map = new Map([['some_char', { column: 'SOME_NUM' }]])
|
||||
expect(build([{ label: 'SOME_CHAR', value: '10' }], map)).toEqual([
|
||||
{
|
||||
GROUP_LOGIC: 'AND',
|
||||
SUBGROUP_LOGIC: 'AND',
|
||||
SUBGROUP_ID: 0,
|
||||
VARIABLE_NM: 'SOME_NUM',
|
||||
OPERATOR_NM: '=',
|
||||
RAW_VALUE: '10'
|
||||
}
|
||||
])
|
||||
})
|
||||
|
||||
it('lets two differently-named params filter the same column', () => {
|
||||
// The multi-tab case: one DC instance per report tab, each with its own
|
||||
// VA parameter, both mapping to the same DC column.
|
||||
const map = new Map([
|
||||
['region_tab1', { column: 'SOME_CHAR' }],
|
||||
['region_tab2', { column: 'SOME_CHAR' }]
|
||||
])
|
||||
const clauses = build(
|
||||
[
|
||||
{ label: 'REGION_TAB1', value: 'north' },
|
||||
{ label: 'REGION_TAB2', value: 'south' }
|
||||
],
|
||||
map
|
||||
)
|
||||
expect(clauses.map((c) => [c.VARIABLE_NM, c.RAW_VALUE])).toEqual([
|
||||
['SOME_CHAR', "'north'"],
|
||||
['SOME_CHAR', "'south'"]
|
||||
])
|
||||
})
|
||||
|
||||
it('matches the parameter label only, not its internal name', () => {
|
||||
// The internal name is the report's own bookkeeping. Matching it as well
|
||||
// meant one parameter answered to two keys, so an entry written against the
|
||||
// name and one against the label silently fought over it.
|
||||
const map = new Map([['pr273', { column: 'SOME_NUM' }]])
|
||||
expect(
|
||||
build([{ name: 'pr273', label: 'Tab_1_Value', value: '7' }], map)
|
||||
).toEqual([])
|
||||
// The label still matches, case-insensitively.
|
||||
expect(
|
||||
build(
|
||||
[{ name: 'pr273', label: 'Tab_1_Value', value: 'x' }],
|
||||
new Map([['tab_1_value', { column: 'SOME_CHAR' }]])
|
||||
)[0].VARIABLE_NM
|
||||
).toBe('SOME_CHAR')
|
||||
})
|
||||
|
||||
it('uses the column case from the loaded metadata', () => {
|
||||
const map = new Map([['region_tab1', { column: 'some_char' }]])
|
||||
expect(
|
||||
build([{ label: 'REGION_TAB1', value: 'x' }], map)[0].VARIABLE_NM
|
||||
).toBe('SOME_CHAR')
|
||||
})
|
||||
|
||||
it('leaves unmapped params on the same-name path', () => {
|
||||
const map = new Map([['region_tab1', { column: 'SOME_NUM' }]])
|
||||
const clauses = build(
|
||||
[
|
||||
{ label: 'REGION_TAB1', value: '10' },
|
||||
{ label: 'SOME_CHAR', value: 'x' }
|
||||
],
|
||||
map
|
||||
)
|
||||
expect(clauses.map((c) => [c.VARIABLE_NM, c.RAW_VALUE])).toEqual([
|
||||
['SOME_NUM', '10'],
|
||||
['SOME_CHAR', "'x'"]
|
||||
])
|
||||
})
|
||||
|
||||
it('skips a mapping whose column is not in this table (no fallback)', () => {
|
||||
// Falling back to the same-name match would silently apply the filter the
|
||||
// mapping exists to override, so the param is dropped instead.
|
||||
const map = new Map([['some_char', { column: 'NOT_IN_THIS_TABLE' }]])
|
||||
expect(build([{ label: 'SOME_CHAR', value: 'x' }], map)).toEqual([])
|
||||
})
|
||||
|
||||
it('ANDs two parameters that map to the same column - it does not union them', () => {
|
||||
// What a report gets when one column's two values arrive in two
|
||||
// parameters: every clause is AND-ed, so the result is the INTERSECTION.
|
||||
// Two different dates therefore match no rows at all rather than both
|
||||
// dates' rows - a single parameter carrying both values is the shape that
|
||||
// ORs, via `IN (...)`.
|
||||
const map = new Map([
|
||||
['param_a', { column: 'SOME_DATE' }],
|
||||
['param_b', { column: 'SOME_DATE' }]
|
||||
])
|
||||
const one = build([{ label: 'PARAM_A', value: '01JUL1960' }], map)[0]
|
||||
const other = build([{ label: 'PARAM_B', value: '01JAN1961' }], map)[0]
|
||||
const clauses = build(
|
||||
[
|
||||
{ label: 'PARAM_A', value: '01JUL1960' },
|
||||
{ label: 'PARAM_B', value: '01JAN1961' }
|
||||
],
|
||||
map
|
||||
)
|
||||
expect(one.RAW_VALUE).not.toBe(other.RAW_VALUE)
|
||||
expect(clauses).toHaveSize(2)
|
||||
expect(clauses.map((c) => c.RAW_VALUE)).toEqual([
|
||||
one.RAW_VALUE,
|
||||
other.RAW_VALUE
|
||||
])
|
||||
for (const clause of clauses) {
|
||||
expect(clause.VARIABLE_NM).toBe('SOME_DATE')
|
||||
expect(clause.OPERATOR_NM).toBe('=')
|
||||
expect(clause.GROUP_LOGIC).toBe('AND')
|
||||
expect(clause.SUBGROUP_LOGIC).toBe('AND')
|
||||
expect(clause.SUBGROUP_ID).toBe(0)
|
||||
}
|
||||
})
|
||||
|
||||
it('applies a transform to the clause it builds', () => {
|
||||
// ADD(QUARTER_START,-1) is the reported requirement: the selected row plus
|
||||
// the row for the previous quarter, as one IN clause so the AND rule is
|
||||
// untouched. 2026-10-01 = 24380, 2026-07-01 = 24288 in SAS days.
|
||||
const map = new Map([
|
||||
[
|
||||
'selected',
|
||||
{
|
||||
column: 'SOME_DATE',
|
||||
transform: {
|
||||
op: 'ADD' as const,
|
||||
anchor: 'QUARTER_START' as const,
|
||||
offset: -1
|
||||
}
|
||||
}
|
||||
]
|
||||
])
|
||||
const [clause] = build([{ label: 'SELECTED', value: '01OCT2026' }], map)
|
||||
expect(clause.VARIABLE_NM).toBe('SOME_DATE')
|
||||
expect(clause.OPERATOR_NM).toBe('IN')
|
||||
expect(clause.RAW_VALUE).toBe('(24380,24288)')
|
||||
})
|
||||
|
||||
it('applies a RANGE as BETWEEN', () => {
|
||||
// RANGE(QUARTER,-1) for a mid-quarter value: 2026-04-01..2026-06-30
|
||||
const map = new Map([
|
||||
[
|
||||
'selected',
|
||||
{
|
||||
column: 'SOME_DATE',
|
||||
transform: {
|
||||
op: 'RANGE' as const,
|
||||
period: 'QUARTER' as const,
|
||||
offset: -1
|
||||
}
|
||||
}
|
||||
]
|
||||
])
|
||||
const [clause] = build([{ label: 'SELECTED', value: '1962-08-15' }], map)
|
||||
expect(clause.OPERATOR_NM).toBe('BETWEEN')
|
||||
expect(clause.RAW_VALUE).toContain(' AND ')
|
||||
expect(clause.RAW_VALUE.split(' AND ')).toHaveSize(2)
|
||||
})
|
||||
|
||||
it('keeps equality when the value already sits on the anchor', () => {
|
||||
const map = new Map([
|
||||
[
|
||||
'selected',
|
||||
{
|
||||
column: 'SOME_DATE',
|
||||
transform: {
|
||||
op: 'ADD' as const,
|
||||
anchor: 'QUARTER_START' as const,
|
||||
offset: 0
|
||||
}
|
||||
}
|
||||
]
|
||||
])
|
||||
const [clause] = build([{ label: 'SELECTED', value: '01OCT2026' }], map)
|
||||
expect(clause.OPERATOR_NM).toBe('=')
|
||||
expect(clause.RAW_VALUE).toBe('24380')
|
||||
})
|
||||
|
||||
it('drops a transform on a column it cannot apply to', () => {
|
||||
// A character column has no dates to shift; the parameter is skipped rather
|
||||
// than filtered on the raw value the author did not see.
|
||||
const map = new Map([
|
||||
[
|
||||
'selected',
|
||||
{
|
||||
column: 'SOME_CHAR',
|
||||
transform: {
|
||||
op: 'ADD' as const,
|
||||
anchor: 'QUARTER_START' as const,
|
||||
offset: -1
|
||||
}
|
||||
}
|
||||
]
|
||||
])
|
||||
expect(build([{ label: 'SELECTED', value: 'x' }], map)).toEqual([])
|
||||
})
|
||||
|
||||
it('adds nothing when the value is unparseable as a date', () => {
|
||||
const map = new Map([
|
||||
[
|
||||
'selected',
|
||||
{
|
||||
column: 'SOME_DATE',
|
||||
transform: {
|
||||
op: 'ADD' as const,
|
||||
anchor: 'QUARTER_START' as const,
|
||||
offset: -1
|
||||
}
|
||||
}
|
||||
]
|
||||
])
|
||||
expect(build([{ label: 'SELECTED', value: 'not a date' }], map)).toEqual(
|
||||
[]
|
||||
)
|
||||
})
|
||||
|
||||
it('round-trips a bare SAS day number unchanged', () => {
|
||||
// VA may send the SAS internal value rather than a formatted string. The
|
||||
// Date a transform works from is local, so the number must come back out of
|
||||
// the clause identical - a UTC-built Date shifts a day west of UTC.
|
||||
const map = new Map([
|
||||
[
|
||||
'selected',
|
||||
{
|
||||
column: 'SOME_DATE',
|
||||
transform: {
|
||||
op: 'ADD' as const,
|
||||
anchor: 'QUARTER_START' as const,
|
||||
offset: -1
|
||||
}
|
||||
}
|
||||
]
|
||||
])
|
||||
const [clause] = build([{ label: 'SELECTED', value: 24380 }], map)
|
||||
expect(clause.RAW_VALUE).toBe('(24380,24288)')
|
||||
})
|
||||
|
||||
it('reports a range mapping fed more than one value', () => {
|
||||
const map = new Map([
|
||||
[
|
||||
'selected',
|
||||
{
|
||||
column: 'SOME_DATE',
|
||||
transform: {
|
||||
op: 'RANGE' as const,
|
||||
period: 'QUARTER' as const,
|
||||
offset: -1
|
||||
}
|
||||
}
|
||||
]
|
||||
])
|
||||
const problems = service.unsupportedTransforms(
|
||||
msg([{ label: 'SELECTED', value: ['01APR2026', '01JUL2026'] }]),
|
||||
map,
|
||||
[{ NAME: 'SOME_DATE', DDTYPE: 'DATE', FMTNAME: 'DATE9' }]
|
||||
)
|
||||
expect(problems).toHaveSize(1)
|
||||
expect(problems[0]).toContain('is a range')
|
||||
expect(problems[0]).toContain('2 values')
|
||||
})
|
||||
|
||||
it('refuses a range fed more than one value, rather than picking one', () => {
|
||||
// Two values, one of which does not parse: building a BETWEEN from the
|
||||
// survivor would silently filter one period when VA selected two.
|
||||
const map = new Map([
|
||||
[
|
||||
'selected',
|
||||
{
|
||||
column: 'SOME_DATE',
|
||||
transform: {
|
||||
op: 'RANGE' as const,
|
||||
period: 'QUARTER' as const,
|
||||
offset: -1
|
||||
}
|
||||
}
|
||||
]
|
||||
])
|
||||
expect(
|
||||
build([{ label: 'SELECTED', value: ['01APR2026', 'not a date'] }], map)
|
||||
).toEqual([])
|
||||
})
|
||||
|
||||
it('never lets a non-finite date reach the clause', () => {
|
||||
// convertJsDateToSasDate returning NaN would otherwise put a literal NaN in
|
||||
// the WHERE clause.
|
||||
const map = new Map([
|
||||
[
|
||||
'selected',
|
||||
{
|
||||
column: 'SOME_DATE',
|
||||
transform: {
|
||||
op: 'ADD' as const,
|
||||
anchor: 'QUARTER_START' as const,
|
||||
offset: -1
|
||||
}
|
||||
}
|
||||
]
|
||||
])
|
||||
spyOn(helperService, 'convertJsDateToSasDate').and.returnValue(NaN as any)
|
||||
expect(build([{ label: 'SELECTED', value: '01OCT2026' }], map)).toEqual(
|
||||
[]
|
||||
)
|
||||
})
|
||||
|
||||
it('reports a partly unparseable value set instead of covering the rest', () => {
|
||||
// One good value and one that is not a date: the clause still filters on
|
||||
// the good one, so the author has to be told the other was dropped.
|
||||
const map = new Map([
|
||||
[
|
||||
'selected',
|
||||
{
|
||||
column: 'SOME_DATE',
|
||||
transform: {
|
||||
op: 'ADD' as const,
|
||||
anchor: 'QUARTER_START' as const,
|
||||
offset: -1
|
||||
}
|
||||
}
|
||||
]
|
||||
])
|
||||
const problems = service.unsupportedTransforms(
|
||||
msg([{ label: 'SELECTED', value: ['01OCT2026', 'not a date'] }]),
|
||||
map,
|
||||
cols
|
||||
)
|
||||
expect(problems).toHaveSize(1)
|
||||
expect(problems[0]).toContain('1 of its 2 values was not a date')
|
||||
})
|
||||
|
||||
it('reports a range holding a value that does not parse', () => {
|
||||
// Counted on what VA sent, not on what parsed: otherwise the second value
|
||||
// vanishes and the range looks like a clean single-value case.
|
||||
const map = new Map([
|
||||
[
|
||||
'selected',
|
||||
{
|
||||
column: 'SOME_DATE',
|
||||
transform: {
|
||||
op: 'RANGE' as const,
|
||||
period: 'QUARTER' as const,
|
||||
offset: -1
|
||||
}
|
||||
}
|
||||
]
|
||||
])
|
||||
const problems = service.unsupportedTransforms(
|
||||
msg([{ label: 'SELECTED', value: ['01APR2026', 'not a date'] }]),
|
||||
map,
|
||||
cols
|
||||
)
|
||||
expect(problems).toHaveSize(1)
|
||||
expect(problems[0]).toContain('is a range')
|
||||
expect(problems[0]).toContain('2 values')
|
||||
})
|
||||
|
||||
it('applies a transform on the deferred initial load too', () => {
|
||||
const map = new Map([
|
||||
[
|
||||
'selected',
|
||||
{
|
||||
column: 'SOME_DATE',
|
||||
transform: {
|
||||
op: 'ADD' as const,
|
||||
anchor: 'QUARTER_START' as const,
|
||||
offset: -1
|
||||
}
|
||||
}
|
||||
]
|
||||
])
|
||||
const [clause] = service.buildInitialClauses(
|
||||
msg([{ name: 'pr1', label: 'SELECTED', value: '01OCT2026' }]),
|
||||
map
|
||||
)
|
||||
expect(clause.VARIABLE_NM).toBe('SOME_DATE')
|
||||
expect(clause.RAW_VALUE).toBe('(24380,24288)')
|
||||
})
|
||||
|
||||
it('behaves exactly as before with an empty map', () => {
|
||||
expect(
|
||||
service.buildClauses(
|
||||
msg([{ label: 'SOME_CHAR', value: 'x' }]),
|
||||
labelToVar(),
|
||||
cols
|
||||
)
|
||||
).toEqual(build([{ label: 'SOME_CHAR', value: 'x' }], new Map()))
|
||||
})
|
||||
})
|
||||
|
||||
describe('unresolvedMappings (what the report author is warned about)', () => {
|
||||
const report = (
|
||||
parameters: any[],
|
||||
map: Map<string, VaParmapTarget>,
|
||||
libds?: string
|
||||
) => service.unresolvedMappings(msg(parameters), map, cols, libds)
|
||||
|
||||
it('reports a mapping whose column is not in the loaded table', () => {
|
||||
expect(
|
||||
report(
|
||||
[{ label: 'REGION_TAB1', value: 'x' }],
|
||||
new Map([['region_tab1', { column: 'NOT_A_COLUMN' }]]),
|
||||
'DC_JSLIB.MPE_X_TEST'
|
||||
)
|
||||
).toEqual([
|
||||
'VA parameter "REGION_TAB1" is mapped to NOT_A_COLUMN, which is not a column of DC_JSLIB.MPE_X_TEST - the filter was ignored.'
|
||||
])
|
||||
})
|
||||
|
||||
it('says "this table" when the table name is not supplied', () => {
|
||||
expect(
|
||||
report(
|
||||
[{ label: 'REGION_TAB1', value: 'x' }],
|
||||
new Map([['region_tab1', { column: 'NOPE' }]])
|
||||
)[0]
|
||||
).toContain('is not a column of this table')
|
||||
})
|
||||
|
||||
it('reports nothing when the mapping resolves, or the param is unmapped', () => {
|
||||
expect(
|
||||
report(
|
||||
[{ label: 'REGION_TAB1', value: 'x' }],
|
||||
new Map([['region_tab1', { column: 'some_char' }]])
|
||||
)
|
||||
).toEqual([])
|
||||
expect(report([{ label: 'SOME_CHAR', value: 'x' }], new Map())).toEqual(
|
||||
[]
|
||||
)
|
||||
// A mapping whose parameter this message does not carry is not an error.
|
||||
expect(
|
||||
report([], new Map([['region_tab1', { column: 'NOPE' }]]))
|
||||
).toEqual([])
|
||||
})
|
||||
|
||||
it('reports each mapping once, not once per message parameter', () => {
|
||||
const problems = report(
|
||||
[
|
||||
{ label: 'REGION_TAB1', value: '1' },
|
||||
{ name: 'pr99', label: 'REGION_TAB1', value: '2' }
|
||||
],
|
||||
new Map([['region_tab1', { column: 'NOPE' }]])
|
||||
)
|
||||
expect(problems).toHaveSize(1)
|
||||
})
|
||||
|
||||
it('treats an unloaded column list as "not known yet", not as a bad mapping', () => {
|
||||
// While the grid is still loading there is no metadata to judge against, so
|
||||
// reporting would flash the banner on every load.
|
||||
expect(
|
||||
service.unresolvedMappings(
|
||||
msg([{ label: 'REGION_TAB1', value: 'x' }]),
|
||||
new Map([['region_tab1', { column: 'NOPE' }]]),
|
||||
[],
|
||||
'DC_JSLIB.MPE_X_TEST'
|
||||
)
|
||||
).toEqual([])
|
||||
expect(
|
||||
service.unresolvedMappings(
|
||||
msg([{ label: 'REGION_TAB1', value: 'x' }]),
|
||||
new Map([['region_tab1', { column: 'NOPE' }]]),
|
||||
undefined as any,
|
||||
'DC_JSLIB.MPE_X_TEST'
|
||||
)
|
||||
).toEqual([])
|
||||
})
|
||||
})
|
||||
|
||||
describe('sameColumnMappings (two parameters, one column)', () => {
|
||||
it('reports an AND-ed pair, naming both parameters and the column', () => {
|
||||
const map = new Map([
|
||||
['dropdown_tab1', { column: 'SOME_DROPDOWN' }],
|
||||
['dropdown_tab2', { column: 'SOME_DROPDOWN' }]
|
||||
])
|
||||
const problems = service.sameColumnMappings(
|
||||
msg([
|
||||
{ label: 'DROPDOWN_TAB1', value: 'Option 1' },
|
||||
{ label: 'DROPDOWN_TAB2', value: 'Option 2' }
|
||||
]),
|
||||
map
|
||||
)
|
||||
expect(problems).toHaveSize(1)
|
||||
expect(problems[0]).toContain('"DROPDOWN_TAB1" and "DROPDOWN_TAB2"')
|
||||
expect(problems[0]).toContain('all filter SOME_DROPDOWN')
|
||||
expect(problems[0]).toContain('combined with AND')
|
||||
})
|
||||
|
||||
it('says nothing for one parameter per column', () => {
|
||||
const map = new Map([
|
||||
['dropdown_tab1', { column: 'SOME_DROPDOWN' }],
|
||||
['num_tab1', { column: 'SOME_NUM' }]
|
||||
])
|
||||
expect(
|
||||
service.sameColumnMappings(
|
||||
msg([
|
||||
{ label: 'DROPDOWN_TAB1', value: 'Option 1' },
|
||||
{ label: 'NUM_TAB1', value: '1' }
|
||||
]),
|
||||
map
|
||||
)
|
||||
).toEqual([])
|
||||
expect(
|
||||
service.sameColumnMappings(
|
||||
msg([{ label: 'DROPDOWN_TAB1', value: 'Option 1' }]),
|
||||
map
|
||||
)
|
||||
).toEqual([])
|
||||
expect(service.sameColumnMappings(msg([]), map)).toEqual([])
|
||||
})
|
||||
|
||||
it('says nothing when the AND-ed parameters carry the same value', () => {
|
||||
// col = 'x' AND col = 'x' filters exactly as one clause does, so a banner
|
||||
// claiming "different values will show no rows" would be wrong.
|
||||
const map = new Map([
|
||||
['dropdown_tab1', { column: 'SOME_CHAR' }],
|
||||
['dropdown_tab2', { column: 'SOME_CHAR' }]
|
||||
])
|
||||
expect(
|
||||
service.sameColumnMappings(
|
||||
msg([
|
||||
{ label: 'DROPDOWN_TAB1', value: 'x' },
|
||||
{ label: 'DROPDOWN_TAB2', value: 'x' }
|
||||
]),
|
||||
map,
|
||||
cols
|
||||
)
|
||||
).toEqual([])
|
||||
// One differing value and the intersection can be empty: reported.
|
||||
const problems = service.sameColumnMappings(
|
||||
msg([
|
||||
{ label: 'DROPDOWN_TAB1', value: 'x' },
|
||||
{ label: 'DROPDOWN_TAB2', value: 'y' }
|
||||
]),
|
||||
map,
|
||||
cols
|
||||
)
|
||||
expect(problems).toHaveSize(1)
|
||||
expect(problems[0]).toContain('"DROPDOWN_TAB1" and "DROPDOWN_TAB2"')
|
||||
})
|
||||
|
||||
it('leaves a column the table does not have to unresolvedMappings', () => {
|
||||
// One cause, one line: the column is missing, so the AND story is moot.
|
||||
const map = new Map([
|
||||
['a', { column: 'NOPE' }],
|
||||
['b', { column: 'NOPE' }]
|
||||
])
|
||||
const cols = [
|
||||
{ LIBREF: 'DC_JSLIB', TABLE: 'MPE_X_TEST', VARIABLE: 'SOME_CHAR' }
|
||||
]
|
||||
expect(
|
||||
service.sameColumnMappings(
|
||||
msg([
|
||||
{ label: 'A', value: '1' },
|
||||
{ label: 'B', value: '2' }
|
||||
]),
|
||||
map,
|
||||
cols
|
||||
)
|
||||
).toEqual([])
|
||||
// Without metadata the pair is still reported: the AND semantics hold
|
||||
// whatever the table turns out to be.
|
||||
expect(
|
||||
service.sameColumnMappings(
|
||||
msg([
|
||||
{ label: 'A', value: '1' },
|
||||
{ label: 'B', value: '2' }
|
||||
]),
|
||||
map
|
||||
)
|
||||
).toHaveSize(1)
|
||||
})
|
||||
|
||||
it('reports one line per contended column', () => {
|
||||
const map = new Map([
|
||||
['a', { column: 'SOME_DROPDOWN' }],
|
||||
['b', { column: 'SOME_DROPDOWN' }],
|
||||
['c', { column: 'SOME_NUM' }],
|
||||
['d', { column: 'SOME_NUM' }]
|
||||
])
|
||||
const problems = service.sameColumnMappings(
|
||||
msg([
|
||||
{ label: 'A', value: '1' },
|
||||
{ label: 'B', value: '2' },
|
||||
{ label: 'C', value: '3' },
|
||||
{ label: 'D', value: '4' }
|
||||
]),
|
||||
map
|
||||
)
|
||||
expect(problems).toHaveSize(2)
|
||||
})
|
||||
})
|
||||
|
||||
describe('unmatchedMappings (parameter-side typo)', () => {
|
||||
const seen = (...keys: string[]) => new Set(keys)
|
||||
|
||||
it('reports a key that no message has named', () => {
|
||||
expect(
|
||||
service.unmatchedMappings(
|
||||
new Map([['dropdown_tab1', { column: 'SOME_CHAR' }]]),
|
||||
seen('dropdown_tab_1', 'pr1'),
|
||||
true
|
||||
)
|
||||
).toEqual([
|
||||
'This object\'s va_parmap names VA parameter "dropdown_tab1", which VA is not sending - check the spelling.'
|
||||
])
|
||||
})
|
||||
|
||||
it('says nothing until a parameter has actually been seen', () => {
|
||||
// Without evidence that this object receives parameters at all, an absent
|
||||
// key proves nothing - VA sends only the parameters its query uses.
|
||||
expect(
|
||||
service.unmatchedMappings(
|
||||
new Map([['dropdown_tab1', { column: 'SOME_CHAR' }]]),
|
||||
seen()
|
||||
)
|
||||
).toEqual([])
|
||||
expect(
|
||||
service.unmatchedMappings(
|
||||
new Map([['dropdown_tab1', { column: 'SOME_CHAR' }]]),
|
||||
undefined
|
||||
)
|
||||
).toEqual([])
|
||||
})
|
||||
|
||||
it('says nothing when every key has been seen', () => {
|
||||
expect(
|
||||
service.unmatchedMappings(
|
||||
new Map([['dropdown_tab1', { column: 'SOME_CHAR' }]]),
|
||||
seen('dropdown_tab1'),
|
||||
true
|
||||
)
|
||||
).toEqual([])
|
||||
expect(
|
||||
service.unmatchedMappings(new Map(), seen('anything'), true)
|
||||
).toEqual([])
|
||||
})
|
||||
|
||||
it('says nothing while the parameter set is still growing', () => {
|
||||
// A message that introduces a new parameter name means this object's set
|
||||
// has not settled, so an absent key may arrive in the next message.
|
||||
expect(
|
||||
service.unmatchedMappings(
|
||||
new Map([['dropdown_tab1', { column: 'SOME_CHAR' }]]),
|
||||
seen('pr1', 'pr2'),
|
||||
false
|
||||
)
|
||||
).toEqual([])
|
||||
})
|
||||
})
|
||||
|
||||
describe('buildInitialClauses (no metadata, infer kind from value)', () => {
|
||||
it('uses an explicit mapping instead of the label', () => {
|
||||
// The deferred first load has no column metadata, so a mapped column is
|
||||
// used as given and the SAS kind still comes from the value shape.
|
||||
const map = new Map([
|
||||
['region_tab1', { column: 'REGION' }],
|
||||
['amt_tab1', { column: 'AMOUNT' }]
|
||||
])
|
||||
const clauses = service.buildInitialClauses(
|
||||
msg([
|
||||
{ name: 'pr1', label: 'REGION_TAB1', dataType: 'string', value: 'n' },
|
||||
{ name: 'pr2', label: 'AMT_TAB1', dataType: 'number', value: '10' }
|
||||
]),
|
||||
map
|
||||
)
|
||||
expect(clauses.map((c) => [c.VARIABLE_NM, c.RAW_VALUE])).toEqual([
|
||||
['REGION', "'n'"],
|
||||
['AMOUNT', '10']
|
||||
])
|
||||
})
|
||||
|
||||
it('falls back to the label for params the map does not name', () => {
|
||||
const map = new Map([['region_tab2', { column: 'REGION' }]])
|
||||
const clauses = service.buildInitialClauses(
|
||||
msg([
|
||||
{ name: 'pr1', label: 'REGION_TAB1', value: 'n' },
|
||||
{ name: 'pr2', label: 'REGION_TAB2', value: 's' }
|
||||
]),
|
||||
map
|
||||
)
|
||||
expect(clauses.map((c) => [c.VARIABLE_NM, c.RAW_VALUE])).toEqual([
|
||||
['REGION_TAB1', "'n'"],
|
||||
['REGION', "'s'"]
|
||||
])
|
||||
})
|
||||
|
||||
it('uses the VA label as the column and infers kinds by value shape', () => {
|
||||
const clauses = service.buildInitialClauses(
|
||||
msg([
|
||||
|
||||
@@ -2,6 +2,12 @@ import { Injectable } from '@angular/core'
|
||||
import { HelperService } from './helper.service'
|
||||
import { VaMessage } from './va-messaging.service'
|
||||
import { QueryClause } from '../models/TableData'
|
||||
import { VaParmapTarget } from '../shared/utils/parse-va-parmap'
|
||||
import {
|
||||
applyVaTransform,
|
||||
isRangeTransform,
|
||||
VaTransform
|
||||
} from '../shared/utils/va-parmap-transform'
|
||||
|
||||
/** SAS data-type kind used to format a VA filter value into a DC RAW_VALUE. */
|
||||
type VaColumnKind = 'char' | 'numeric' | 'time' | 'date' | 'datetime'
|
||||
@@ -54,11 +60,19 @@ export class VaFilterService {
|
||||
* Translates VA `parameters` into DC filter clauses. Each parameter is matched
|
||||
* to a DC column by label; a single value becomes `col = value`, multiple
|
||||
* values become `col IN (...)`. Values are formatted for the column's SAS kind.
|
||||
*
|
||||
* `paramMap` (from the URL `va_parmap` parameter, see parseVaParmap) overrides
|
||||
* that same-name resolution for the parameters it names, so two DC instances
|
||||
* over one table can filter the same column from differently-named VA
|
||||
* parameters. A mapped column must exist in `cols` - if it does not the
|
||||
* parameter is skipped rather than falling back to the same-name match, which
|
||||
* is exactly the behaviour the mapping exists to override.
|
||||
*/
|
||||
buildClauses(
|
||||
msg: VaMessage,
|
||||
labelToVar: Map<string, string>,
|
||||
cols: any[]
|
||||
cols: any[],
|
||||
paramMap?: Map<string, VaParmapTarget>
|
||||
): QueryClause[] {
|
||||
const clauses: QueryClause[] = []
|
||||
for (const param of msg.parameters || []) {
|
||||
@@ -67,60 +81,194 @@ export class VaFilterService {
|
||||
.toString()
|
||||
.trim()
|
||||
.toLowerCase()
|
||||
// A VA param whose label isn't a DC column (beyond case, which is handled)
|
||||
// simply doesn't map — skip it.
|
||||
const varName = labelToVar.get(key)
|
||||
// An explicit URL mapping wins over the label->column lookup. The mapped
|
||||
// column has to exist in THIS table, else the entry does not apply here
|
||||
// (and the same-name fallback would be the behaviour it exists to
|
||||
// override).
|
||||
const mapped = this.mappedColumn(param, paramMap)
|
||||
const mappedCol = mapped ? this.findCol(mapped, cols) : undefined
|
||||
if (mapped && !mappedCol) continue
|
||||
// The table's own case, so the clause matches what the metadata build
|
||||
// produces for a same-name match.
|
||||
const varName: string = mappedCol
|
||||
? mappedCol.NAME
|
||||
: (labelToVar.get(key) ?? '')
|
||||
if (!varName) continue
|
||||
|
||||
const col = this.findCol(varName, cols)
|
||||
const col = mappedCol ?? this.findCol(varName, cols)
|
||||
const kind = this.columnKind(col)
|
||||
const order = this.dateOrderFromFmt(col?.FMTNAME || col?.FORMAT)
|
||||
const rawValues = Array.isArray(param.value) ? param.value : [param.value]
|
||||
// Format/validate every value for the column's SAS type. Numeric columns
|
||||
// must receive a finite number (untrusted postMessage input is
|
||||
// interpolated into a SAS WHERE clause, so a non-numeric value here would
|
||||
// be an injection vector); temporal -> SAS internal number; char ->
|
||||
// escaped. Invalid values are dropped.
|
||||
const formatted = rawValues
|
||||
.filter((v) => v !== undefined && v !== null && v !== '')
|
||||
.map((v) => this.formatValueByKind(v, kind, order))
|
||||
.filter((v): v is string => v !== null)
|
||||
if (formatted.length === 0) continue
|
||||
clauses.push(this.clause(varName, formatted))
|
||||
const built = this.buildClauseValueSet(
|
||||
param,
|
||||
kind,
|
||||
order,
|
||||
this.mappedTarget(param, paramMap)?.transform
|
||||
)
|
||||
if (!built) continue
|
||||
clauses.push({
|
||||
GROUP_LOGIC: 'AND',
|
||||
SUBGROUP_LOGIC: 'AND',
|
||||
SUBGROUP_ID: 0,
|
||||
VARIABLE_NM: varName,
|
||||
OPERATOR_NM: built.operator,
|
||||
RAW_VALUE: built.rawValue
|
||||
})
|
||||
}
|
||||
return clauses
|
||||
}
|
||||
|
||||
/**
|
||||
* The operator and raw value for one parameter, after any transform.
|
||||
*
|
||||
* Values are formatted for the column's SAS kind: numeric columns must receive
|
||||
* a finite number (untrusted postMessage input is interpolated into a WHERE
|
||||
* clause, so anything else would be an injection vector); temporal values
|
||||
* become SAS internal numbers; char values are escaped. Invalid values are
|
||||
* dropped, and a transform that cannot be applied to the column's kind yields
|
||||
* nothing at all - the caller reports that separately rather than filtering on
|
||||
* a value the author did not choose.
|
||||
*/
|
||||
private buildClauseValueSet(
|
||||
param: any,
|
||||
kind: VaColumnKind,
|
||||
order: 'YMD' | 'DMY' | 'MDY' | undefined,
|
||||
transform?: VaTransform
|
||||
): { operator: '=' | 'IN' | 'BETWEEN'; rawValue: string } | null {
|
||||
const rawValues = (
|
||||
Array.isArray(param.value) ? param.value : [param.value]
|
||||
).filter((v: any) => v !== undefined && v !== null && v !== '')
|
||||
if (rawValues.length === 0) return null
|
||||
|
||||
if (!transform) {
|
||||
const formatted = rawValues
|
||||
.map((v: any) => this.formatValueByKind(v, kind, order))
|
||||
.filter((v: string | null): v is string => v !== null)
|
||||
if (formatted.length === 0) return null
|
||||
return formatted.length === 1
|
||||
? { operator: '=', rawValue: formatted[0] }
|
||||
: { operator: 'IN', rawValue: `(${formatted.join(',')})` }
|
||||
}
|
||||
|
||||
// Transforms resolve to dates, so only temporal columns can take one.
|
||||
if (kind !== 'date' && kind !== 'datetime') return null
|
||||
const unit = kind === 'datetime' ? 'seconds' : 'days'
|
||||
const values = new Set<string>()
|
||||
for (const raw of rawValues) {
|
||||
const resolved = this.valueAsDate(raw, kind, order)
|
||||
if (!resolved) continue
|
||||
for (const date of applyVaTransform(
|
||||
resolved,
|
||||
transform,
|
||||
kind === 'datetime'
|
||||
)) {
|
||||
// A non-finite result (an Invalid Date reaching the helper) must never be
|
||||
// interpolated into the clause: `NaN` is not a SAS value. Dropped here,
|
||||
// and reported by unsupportedTransforms so the parameter is not silently
|
||||
// filtered on nothing.
|
||||
const sasDate = this.helperService.convertJsDateToSasDate(date, unit)
|
||||
if (!Number.isFinite(Number(sasDate))) continue
|
||||
values.add(`${sasDate}`)
|
||||
}
|
||||
}
|
||||
const formatted = Array.from(values)
|
||||
if (formatted.length === 0) return null
|
||||
if (isRangeTransform(transform)) {
|
||||
// One BETWEEN carries one period, so more than one value has no clause -
|
||||
// dropping it here (rather than building a BETWEEN from whichever value
|
||||
// parsed) is what unsupportedTransforms' count warning then describes.
|
||||
if (rawValues.length > 1) return null
|
||||
if (formatted.length !== 2) return null
|
||||
return { operator: 'BETWEEN', rawValue: formatted.join(' AND ') }
|
||||
}
|
||||
return formatted.length === 1
|
||||
? { operator: '=', rawValue: formatted[0] }
|
||||
: { operator: 'IN', rawValue: `(${formatted.join(',')})` }
|
||||
}
|
||||
|
||||
/**
|
||||
* A VA value as a local Date, for a transform to work from. Formatted strings
|
||||
* are parsed by shape; a bare number is already the SAS internal value, so it
|
||||
* is converted back from days (date) or seconds (datetime) since 1960-01-01.
|
||||
*
|
||||
* Built with local calendar construction, matching `parseDate` and
|
||||
* `HelperService.convertJsDateToSasDate`: a UTC instant read back through local
|
||||
* accessors lands on the previous day west of UTC, and the round trip to a SAS
|
||||
* day number would be one out.
|
||||
*/
|
||||
private valueAsDate(
|
||||
value: any,
|
||||
kind: VaColumnKind,
|
||||
order?: 'YMD' | 'DMY' | 'MDY'
|
||||
): Date | null {
|
||||
const s = String(value).trim()
|
||||
if (s === '') return null
|
||||
if (this.isBareNumber(s)) {
|
||||
const internal = Number(s)
|
||||
if (kind !== 'datetime') return new Date(1960, 0, 1 + internal)
|
||||
// Seconds since midnight 1960-01-01, split into whole days and a remainder
|
||||
// so the constructor normalises the clock time locally.
|
||||
const days = Math.floor(internal / 86400)
|
||||
const seconds = internal - days * 86400
|
||||
return new Date(1960, 0, 1 + days, 0, 0, seconds)
|
||||
}
|
||||
return kind === 'datetime'
|
||||
? this.parseDatetime(s, order)
|
||||
: this.parseDate(s, order)
|
||||
}
|
||||
|
||||
/**
|
||||
* Pre-getdata variant used on the deferred initial load, when DC column
|
||||
* metadata isn't available. Uses the VA `label` as the column name and infers
|
||||
* the SAS kind from the value shape, so the clause matches what the metadata
|
||||
* build later produces (no reconcile reload).
|
||||
*
|
||||
* `paramMap` overrides the label-as-column rule for the parameters it names,
|
||||
* so the very first (already filtered) load also honours an explicit mapping.
|
||||
*/
|
||||
buildInitialClauses(msg: VaMessage): QueryClause[] {
|
||||
buildInitialClauses(
|
||||
msg: VaMessage,
|
||||
paramMap?: Map<string, VaParmapTarget>
|
||||
): QueryClause[] {
|
||||
const clauses: QueryClause[] = []
|
||||
for (const param of msg.parameters || []) {
|
||||
if (!param || param.value === undefined || param.value === null) continue
|
||||
// The DC column name is the VA `label`; `name` is VA's internal id (pr###).
|
||||
const varName = (param.label ?? param.name ?? '').toString().trim()
|
||||
const target = this.mappedTarget(param, paramMap)
|
||||
// Without a mapping the DC column name is the VA `label`; `name` is VA's
|
||||
// internal id (pr###). A `va_parmap` entry wins over both.
|
||||
const varName = (target?.column ?? param.label ?? param.name ?? '')
|
||||
.toString()
|
||||
.trim()
|
||||
if (!varName) continue
|
||||
|
||||
const rawValues = Array.isArray(param.value) ? param.value : [param.value]
|
||||
const rawValues = (
|
||||
Array.isArray(param.value) ? param.value : [param.value]
|
||||
) as any[]
|
||||
const nonEmpty = rawValues.filter(
|
||||
(v) => v !== undefined && v !== null && v !== ''
|
||||
)
|
||||
if (nonEmpty.length === 0) continue
|
||||
|
||||
const formatted = nonEmpty
|
||||
.map((v) =>
|
||||
this.formatValueByKind(
|
||||
v,
|
||||
this.inferKindFromValue(String(v).trim(), param.dataType)
|
||||
)
|
||||
)
|
||||
.filter((v): v is string => v !== null)
|
||||
if (formatted.length === 0) continue
|
||||
clauses.push(this.clause(varName, formatted))
|
||||
// A transform only ever produces dates, so it fixes the kind: inferring it
|
||||
// from the value would see a formatted date either way, but with the
|
||||
// transform the column is known to be temporal.
|
||||
const kind = target?.transform
|
||||
? ('date' as VaColumnKind)
|
||||
: this.inferKindFromValue(String(nonEmpty[0]).trim(), param.dataType)
|
||||
const built = this.buildClauseValueSet(
|
||||
param,
|
||||
kind,
|
||||
undefined,
|
||||
target?.transform
|
||||
)
|
||||
if (!built) continue
|
||||
clauses.push({
|
||||
GROUP_LOGIC: 'AND',
|
||||
SUBGROUP_LOGIC: 'AND',
|
||||
SUBGROUP_ID: 0,
|
||||
VARIABLE_NM: varName,
|
||||
OPERATOR_NM: built.operator,
|
||||
RAW_VALUE: built.rawValue
|
||||
})
|
||||
}
|
||||
return clauses
|
||||
}
|
||||
@@ -141,16 +289,254 @@ export class VaFilterService {
|
||||
)
|
||||
}
|
||||
|
||||
private clause(varName: string, formatted: string[]): QueryClause {
|
||||
return {
|
||||
GROUP_LOGIC: 'AND',
|
||||
SUBGROUP_LOGIC: 'AND',
|
||||
SUBGROUP_ID: 0,
|
||||
VARIABLE_NM: varName,
|
||||
OPERATOR_NM: formatted.length === 1 ? '=' : 'IN',
|
||||
RAW_VALUE:
|
||||
formatted.length === 1 ? formatted[0] : `(${formatted.join(',')})`
|
||||
/**
|
||||
* Transform mappings in this message that cannot be applied, one plain-English
|
||||
* line each: the column is not temporal, or the value is not a date the browser
|
||||
* can parse. `buildClauseValueSet` skips those parameters, so this is what stops
|
||||
* the misconfiguration reading as a filter that simply matched nothing.
|
||||
*
|
||||
* A missing column is left to `unresolvedMappings`, which already reports it.
|
||||
*/
|
||||
unsupportedTransforms(
|
||||
msg: VaMessage,
|
||||
paramMap: Map<string, VaParmapTarget> | undefined,
|
||||
cols: any[]
|
||||
): string[] {
|
||||
if (!paramMap || paramMap.size === 0) return []
|
||||
if (!cols || cols.length === 0) return []
|
||||
const problems = new Set<string>()
|
||||
for (const param of msg.parameters || []) {
|
||||
const target = this.mappedTarget(param, paramMap)
|
||||
if (!target?.transform) continue
|
||||
const transform = target.transform
|
||||
const col = this.findCol(target.column, cols)
|
||||
if (!col) continue
|
||||
const kind = this.columnKind(col)
|
||||
const paramName = (param.label ?? param.name ?? '').toString().trim()
|
||||
const spec =
|
||||
target.transform.op === 'ADD'
|
||||
? `ADD(${target.transform.anchor},${target.transform.offset})`
|
||||
: `RANGE(${target.transform.period},${target.transform.offset})`
|
||||
if (kind !== 'date' && kind !== 'datetime') {
|
||||
problems.add(
|
||||
`VA parameter "${paramName}" maps to ${target.column}:${spec}, but ${target.column} is not a date or datetime column - the filter was ignored.`
|
||||
)
|
||||
continue
|
||||
}
|
||||
const order = this.dateOrderFromFmt(col?.FMTNAME || col?.FORMAT)
|
||||
const sent = (
|
||||
Array.isArray(param.value) ? param.value : [param.value]
|
||||
).filter((v: any) => v !== undefined && v !== null && v !== '')
|
||||
const usable = sent.filter((v: any) => {
|
||||
const date = this.valueAsDate(v, kind, order)
|
||||
if (!date) return false
|
||||
// Same test the clause builder applies, so this count matches what it uses.
|
||||
return applyVaTransform(date, transform, kind === 'datetime').every(
|
||||
(resolved) => {
|
||||
const sasDate = this.helperService.convertJsDateToSasDate(
|
||||
resolved,
|
||||
kind === 'datetime' ? 'seconds' : 'days'
|
||||
)
|
||||
return Number.isFinite(Number(sasDate))
|
||||
}
|
||||
)
|
||||
})
|
||||
if (usable.length === 0) {
|
||||
problems.add(
|
||||
`VA parameter "${paramName}" maps to ${target.column}:${spec}, but its value was not a date - the filter was ignored.`
|
||||
)
|
||||
continue
|
||||
}
|
||||
// One BETWEEN can only express one period. A multi-select would have to
|
||||
// become an OR of ranges, which the clause model does not carry, so say so
|
||||
// rather than building a BETWEEN from whichever value happened to parse.
|
||||
// Counted on what VA sent: a second value that does not parse must not make
|
||||
// the single-value case look clean.
|
||||
if (isRangeTransform(target.transform) && sent.length > 1) {
|
||||
problems.add(
|
||||
`VA parameter "${paramName}" maps to ${target.column}:${spec}, which is a range, but the parameter has ${sent.length} values - a range needs one value, so the filter was ignored.`
|
||||
)
|
||||
continue
|
||||
}
|
||||
if (usable.length < sent.length) {
|
||||
problems.add(
|
||||
`VA parameter "${paramName}" maps to ${target.column}:${spec}, but ${sent.length - usable.length} of its ${sent.length} values was not a date - the filter covers only the rest.`
|
||||
)
|
||||
}
|
||||
}
|
||||
return Array.from(problems)
|
||||
}
|
||||
|
||||
/**
|
||||
* `va_parmap` entries that this message actually uses but that name a column
|
||||
* the loaded table does not have, one plain-English line each. `buildClauses`
|
||||
* skips those parameters (deliberately: falling back to the same-name match
|
||||
* would silently apply the filter the mapping exists to override), so this is
|
||||
* what makes the misconfiguration visible to the report author - otherwise the
|
||||
* grid simply shows every row, which looks like a working filter.
|
||||
*/
|
||||
unresolvedMappings(
|
||||
msg: VaMessage,
|
||||
paramMap: Map<string, VaParmapTarget> | undefined,
|
||||
cols: any[],
|
||||
libds?: string
|
||||
): string[] {
|
||||
const problems = new Set<string>()
|
||||
if (!paramMap || paramMap.size === 0) return []
|
||||
// No metadata yet is not the same as "the table lacks this column": an empty
|
||||
// `cols` means the grid is still loading, so there is nothing to judge yet.
|
||||
// Reporting here would flash a misconfiguration banner during every load.
|
||||
if (!cols || cols.length === 0) return []
|
||||
for (const param of msg.parameters || []) {
|
||||
const mapped = this.mappedColumn(param, paramMap)
|
||||
if (!mapped || this.findCol(mapped, cols)) continue
|
||||
const paramName = (param.label ?? param.name ?? '').toString().trim()
|
||||
problems.add(
|
||||
`VA parameter "${paramName}" is mapped to ${mapped}, which is not a column of ${
|
||||
libds || 'this table'
|
||||
} - the filter was ignored.`
|
||||
)
|
||||
}
|
||||
return Array.from(problems)
|
||||
}
|
||||
|
||||
/**
|
||||
* `va_parmap` keys that no VA message has named, one plain-English line each.
|
||||
* A key that matches nothing is a parameter-side typo: the mapping never
|
||||
* applies and the grid stays unfiltered, which looks the same as a working
|
||||
* filter.
|
||||
*
|
||||
* `seenKeys` must be every key observed across messages so far. `paramsStable`
|
||||
* says the latest message introduced no new parameter name, i.e. this object's
|
||||
* parameter set has settled - until then a missing name may simply not have
|
||||
* arrived, because VA sends only the parameters its query uses.
|
||||
*/
|
||||
unmatchedMappings(
|
||||
paramMap: Map<string, VaParmapTarget> | undefined,
|
||||
seenKeys: Set<string> | undefined,
|
||||
paramsStable = false
|
||||
): string[] {
|
||||
if (!paramMap || paramMap.size === 0) return []
|
||||
if (!seenKeys || seenKeys.size === 0) return []
|
||||
if (!paramsStable) return []
|
||||
const problems: string[] = []
|
||||
for (const key of Array.from(paramMap.keys())) {
|
||||
if (!seenKeys.has(key)) {
|
||||
problems.push(
|
||||
`This object's va_parmap names VA parameter "${key}", which VA is not sending - check the spelling.`
|
||||
)
|
||||
}
|
||||
}
|
||||
return problems
|
||||
}
|
||||
|
||||
/**
|
||||
* Parameters in this message that filter the same column with *different*
|
||||
* values, one plain-English line per column. Clauses are AND-ed, so two
|
||||
* parameters on one column are an intersection: with different values the grid
|
||||
* comes back empty, which looks like a working filter. Two parameters carrying
|
||||
* the same value intersect cleanly and filter correctly, so they are not
|
||||
* reported - a banner there would call a working configuration broken.
|
||||
*
|
||||
* Reported rather than rewritten - folding them into one `IN (...)` would
|
||||
* silently turn an AND into an OR, which is a semantics decision, not a
|
||||
* diagnostic.
|
||||
*/
|
||||
sameColumnMappings(
|
||||
msg: VaMessage,
|
||||
paramMap: Map<string, VaParmapTarget> | undefined,
|
||||
cols?: any[]
|
||||
): string[] {
|
||||
if (!paramMap || paramMap.size === 0) return []
|
||||
// column -> the parameters on it, each with the clause value it produces
|
||||
const byColumn = new Map<string, { name: string; value: string }[]>()
|
||||
for (const param of msg.parameters || []) {
|
||||
const mapped = this.mappedColumn(param, paramMap)
|
||||
if (!mapped) continue
|
||||
// A column the table lacks is unresolvedMappings' finding; counting it here
|
||||
// would report one cause as two symptoms.
|
||||
const col =
|
||||
cols && cols.length > 0 ? this.findCol(mapped, cols) : undefined
|
||||
if (cols && cols.length > 0 && !col) continue
|
||||
// A parameter that produces no clause cannot conflict with anything.
|
||||
const value = this.valueSignature(
|
||||
param,
|
||||
col,
|
||||
this.mappedTarget(param, paramMap)?.transform
|
||||
)
|
||||
if (value === null) continue
|
||||
const column = mapped.toUpperCase()
|
||||
const params = byColumn.get(column) || []
|
||||
const paramName = (param.label ?? '').toString().trim()
|
||||
if (paramName && !params.some((p) => p.name === paramName)) {
|
||||
params.push({ name: paramName, value })
|
||||
}
|
||||
byColumn.set(column, params)
|
||||
}
|
||||
const problems: string[] = []
|
||||
for (const [column, params] of Array.from(byColumn.entries())) {
|
||||
if (params.length < 2) continue
|
||||
// Same value on every parameter: the AND is a no-op, not an empty grid.
|
||||
const distinct = new Set(params.map((p) => p.value))
|
||||
if (distinct.size === 1) continue
|
||||
problems.push(
|
||||
`VA parameters ${params
|
||||
.map((p) => `"${p.name}"`)
|
||||
.join(
|
||||
' and '
|
||||
)} all filter ${column}. Parameters are combined with AND, so rows must match all of them - different values will show no rows. Use one parameter carrying both values to match either.`
|
||||
)
|
||||
}
|
||||
return problems
|
||||
}
|
||||
|
||||
/**
|
||||
* The DC column an explicit URL mapping (`va_parmap`) assigns to this
|
||||
* parameter, or undefined when the parameter is not mapped. The key is matched
|
||||
* case-insensitively against the parameter's `label`.
|
||||
*/
|
||||
private mappedTarget(
|
||||
param: any,
|
||||
paramMap?: Map<string, VaParmapTarget>
|
||||
): VaParmapTarget | undefined {
|
||||
if (!paramMap || paramMap.size === 0) return undefined
|
||||
// The label only. The internal name (pr273) is an implementation detail of
|
||||
// the report, and matching both made one parameter answer to two keys: a
|
||||
// mapping written against the name could never be distinguished from a typo
|
||||
// once the label was also mapped, and the second entry silently lost.
|
||||
const key = (param.label ?? '').toString().trim().toLowerCase()
|
||||
if (!key) return undefined
|
||||
return paramMap.get(key)
|
||||
}
|
||||
|
||||
/**
|
||||
* The clause value a parameter would filter its column with, as a comparable
|
||||
* string, or null when it produces no clause at all. Used to tell an AND of two
|
||||
* identical values (harmless) from an AND of two different ones (empty grid).
|
||||
*
|
||||
* Delegates to the clause builder so a transform's derived dates take part in
|
||||
* the comparison: two parameters adding the same quarter are the same clause.
|
||||
*/
|
||||
private valueSignature(
|
||||
param: any,
|
||||
col: any,
|
||||
transform?: VaTransform
|
||||
): string | null {
|
||||
const built = this.buildClauseValueSet(
|
||||
param,
|
||||
this.columnKind(col),
|
||||
this.dateOrderFromFmt(col?.FMTNAME || col?.FORMAT),
|
||||
transform
|
||||
)
|
||||
return built ? built.rawValue : null
|
||||
}
|
||||
|
||||
/** The mapped column name, for callers that do not need the transform. */
|
||||
private mappedColumn(
|
||||
param: any,
|
||||
paramMap?: Map<string, VaParmapTarget>
|
||||
): string | undefined {
|
||||
return this.mappedTarget(param, paramMap)?.column
|
||||
}
|
||||
|
||||
/** The loaded column spec whose NAME matches varName (case-insensitive). */
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
import { VaMessagingService, VaMessage } from './va-messaging.service'
|
||||
import { VaFilterService } from './va-filter.service'
|
||||
import { HelperService } from './helper.service'
|
||||
|
||||
// Sample message taken from the official SAS DDC docs (brush column included).
|
||||
const SAMPLE = {
|
||||
@@ -51,6 +53,57 @@ describe('VaMessagingService', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('parameter keys (what the va_parmap typo diagnostic is judged on)', () => {
|
||||
const PARAMS = [
|
||||
{ name: 'pr273', label: 'Tab_1_Date', value: '2025-03-31' },
|
||||
{ name: 'pr274', label: 'Tab 2 Region', value: 'north' }
|
||||
]
|
||||
const withParams = () => ({ ...SAMPLE, parameters: PARAMS })
|
||||
|
||||
it('records each parameter label once, across messages', () => {
|
||||
service.parseMessage({ data: withParams() } as MessageEvent)
|
||||
service.parseMessage({ data: withParams() } as MessageEvent)
|
||||
|
||||
expect(Array.from(service.seenParamKeys)).toEqual([
|
||||
'tab_1_date',
|
||||
'tab 2 region'
|
||||
])
|
||||
// The second message introduced no new label, so the set has settled.
|
||||
expect(service.lastMessageParamsStable).toBeTrue()
|
||||
})
|
||||
|
||||
it('records the label only, so a mapping written against the internal name is reported', () => {
|
||||
// `pr273` is VA's own bookkeeping. A va_parmap entry written against it
|
||||
// never matches a parameter, and recording the name here would make the
|
||||
// typo diagnostic believe VA is sending it - leaving the author with an
|
||||
// unfiltered grid and no warning at all.
|
||||
service.parseMessage({ data: withParams() } as MessageEvent)
|
||||
service.parseMessage({ data: withParams() } as MessageEvent)
|
||||
|
||||
const filter = new VaFilterService(new HelperService({} as any))
|
||||
const seen = service.seenParamKeys
|
||||
const stable = service.lastMessageParamsStable
|
||||
|
||||
expect(
|
||||
filter.unmatchedMappings(
|
||||
new Map([['pr273', { column: 'D_DATE' }]]),
|
||||
seen,
|
||||
stable
|
||||
)
|
||||
).toEqual([
|
||||
'This object\'s va_parmap names VA parameter "pr273", which VA is not sending - check the spelling.'
|
||||
])
|
||||
// The label is a real key, so the same map written that way is silent.
|
||||
expect(
|
||||
filter.unmatchedMappings(
|
||||
new Map([['tab_1_date', { column: 'D_DATE' }]]),
|
||||
seen,
|
||||
stable
|
||||
)
|
||||
).toEqual([])
|
||||
})
|
||||
})
|
||||
|
||||
describe('dataColumns', () => {
|
||||
it('excludes brush columns and keeps original indices', () => {
|
||||
const msg = service.parseMessage({ data: SAMPLE } as MessageEvent)!
|
||||
|
||||
@@ -82,6 +82,26 @@ export class VaMessagingService {
|
||||
/** DC variable names that should be visible (the rest are hidden). */
|
||||
visibleColumns: string[] | null = null
|
||||
|
||||
/**
|
||||
* Lower-cased label of every parameter any VA message has carried so far, kept
|
||||
* across editor reloads. Lets the editor tell a `va_parmap` key that names no
|
||||
* parameter in the report (a typo, so the filter would silently never apply)
|
||||
* from one that is merely absent from the current message - VA only sends the
|
||||
* parameters its query uses.
|
||||
*
|
||||
* Labels only: that is the key a mapping is matched on, and recording the
|
||||
* internal name as well would make a name-keyed entry - which never applies -
|
||||
* look like a parameter VA is sending.
|
||||
*/
|
||||
seenParamKeys = new Set<string>()
|
||||
/**
|
||||
* True when every parameter name in the latest message had already been seen.
|
||||
* VA sends only the parameters a report object's query uses, and sends them per
|
||||
* object, so a name that is absent from the first messages may simply not have
|
||||
* arrived yet. A stable parameter set is what makes absence meaningful.
|
||||
*/
|
||||
lastMessageParamsStable = false
|
||||
|
||||
/** True once the pre-bootstrap early buffer has been drained (drain-once). */
|
||||
private earlyDrained = false
|
||||
|
||||
@@ -187,6 +207,22 @@ export class VaMessagingService {
|
||||
if (typeof data.resultName !== 'string') return null
|
||||
if (!Array.isArray(data.columns) || !Array.isArray(data.data)) return null
|
||||
|
||||
const parameters = Array.isArray(data.parameters) ? data.parameters : []
|
||||
const keys: string[] = []
|
||||
for (const param of parameters) {
|
||||
// The label only, which is the key a `va_parmap` entry is matched on. The
|
||||
// internal name (`pr273`) is the report's own bookkeeping: recording it
|
||||
// here would let the typo diagnostic below believe a name-keyed entry is
|
||||
// being sent when no code path can ever apply it.
|
||||
const key = (param?.label ?? '').toString().trim().toLowerCase()
|
||||
if (key && !keys.includes(key)) keys.push(key)
|
||||
}
|
||||
// Judged before the keys are recorded: a message that introduces a new name
|
||||
// means the set is still growing, so a missing name proves nothing yet.
|
||||
this.lastMessageParamsStable =
|
||||
keys.length > 0 && keys.every((key) => this.seenParamKeys.has(key))
|
||||
for (const key of keys) this.seenParamKeys.add(key)
|
||||
|
||||
return {
|
||||
version: data.version,
|
||||
resultName: data.resultName,
|
||||
@@ -194,7 +230,7 @@ export class VaMessagingService {
|
||||
availableRowCount: data.availableRowCount,
|
||||
data: data.data,
|
||||
columns: data.columns,
|
||||
parameters: Array.isArray(data.parameters) ? data.parameters : []
|
||||
parameters
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,215 @@
|
||||
import { parseVaParmap } from './parse-va-parmap'
|
||||
|
||||
describe('parseVaParmap', () => {
|
||||
it('returns an empty map and no problems when the parameter is absent', () => {
|
||||
for (const query of ['', 'embed=va', 'embed=va&labels=true']) {
|
||||
const parsed = parseVaParmap(query)
|
||||
expect(parsed.map.size).toBe(0)
|
||||
expect(parsed.problems).toEqual([])
|
||||
}
|
||||
})
|
||||
|
||||
it('maps a VA parameter label to a DC column', () => {
|
||||
const { map, problems } = parseVaParmap(
|
||||
'embed=va&va_parmap=REGION_TAB1-REGION'
|
||||
)
|
||||
expect(map.get('region_tab1')?.column).toBe('REGION')
|
||||
expect(map.size).toBe(1)
|
||||
expect(problems).toEqual([])
|
||||
})
|
||||
|
||||
it('collects repeated entries, so two parameters can map to one column', () => {
|
||||
// The multi-tab case: each DC instance filters the same DC column from a
|
||||
// differently-named VA parameter.
|
||||
const { map } = parseVaParmap(
|
||||
'embed=va&va_parmap=REGION_TAB1-REGION&va_parmap=REGION_TAB2-REGION'
|
||||
)
|
||||
expect(map.get('region_tab1')?.column).toBe('REGION')
|
||||
expect(map.get('region_tab2')?.column).toBe('REGION')
|
||||
expect(map.size).toBe(2)
|
||||
})
|
||||
|
||||
it('is case-insensitive on the key and trims both sides', () => {
|
||||
const { map } = parseVaParmap('va_parmap=%20Region_Tab1%20-%20region%20')
|
||||
expect(map.get('region_tab1')?.column).toBe('region')
|
||||
})
|
||||
|
||||
it('decodes a key containing a space (VA labels are free text)', () => {
|
||||
expect(
|
||||
parseVaParmap('va_parmap=My+Region-DEPT').map.get('my region')?.column
|
||||
).toBe('DEPT')
|
||||
})
|
||||
|
||||
it('splits at the LAST hyphen, so a label may contain hyphens', () => {
|
||||
expect(
|
||||
parseVaParmap('va_parmap=Sales-2024-DEPT').map.get('sales-2024')?.column
|
||||
).toBe('DEPT')
|
||||
})
|
||||
|
||||
describe('entries that cannot be used', () => {
|
||||
it('reports an entry with no separator', () => {
|
||||
const { map, problems } = parseVaParmap('va_parmap=DEPT')
|
||||
expect(map.size).toBe(0)
|
||||
expect(problems).toEqual([
|
||||
'va_parmap "DEPT" was ignored: it needs a "-" between the VA parameter and the column name.'
|
||||
])
|
||||
})
|
||||
|
||||
it('reports an entry with no parameter name', () => {
|
||||
const { map, problems } = parseVaParmap('va_parmap=-DEPT')
|
||||
expect(map.size).toBe(0)
|
||||
expect(problems).toEqual([
|
||||
'va_parmap "-DEPT" was ignored: the VA parameter name is missing.'
|
||||
])
|
||||
})
|
||||
|
||||
it('reports an entry with no column name', () => {
|
||||
const { map, problems } = parseVaParmap('va_parmap=REGION_TAB1-')
|
||||
expect(map.size).toBe(0)
|
||||
expect(problems).toEqual([
|
||||
'va_parmap "REGION_TAB1-" was ignored: "" is not a valid SAS column name.'
|
||||
])
|
||||
})
|
||||
|
||||
it('reports a column that is not a plain SAS name (injection guard)', () => {
|
||||
// The column is interpolated into a SAS WHERE clause, so anything beyond a
|
||||
// SAS name is refused at parse time rather than carried into a clause.
|
||||
for (const entry of [
|
||||
'A-x;drop',
|
||||
'A-REGION%20OR%201=1',
|
||||
"A-RE'GION",
|
||||
'A-1ABC'
|
||||
]) {
|
||||
const { map, problems } = parseVaParmap(`va_parmap=${entry}`)
|
||||
expect(map.size).toBe(0)
|
||||
expect(problems.length).toBe(1)
|
||||
expect(problems[0]).toContain('is not a valid SAS column name')
|
||||
}
|
||||
expect(parseVaParmap('va_parmap=A-A_B9').map.get('a')?.column).toBe(
|
||||
'A_B9'
|
||||
)
|
||||
})
|
||||
|
||||
it('keeps the usable entries and reports only the bad one', () => {
|
||||
const { map, problems } = parseVaParmap(
|
||||
'va_parmap=GOOD-REGION&va_parmap=BROKEN'
|
||||
)
|
||||
expect(map.get('good')?.column).toBe('REGION')
|
||||
expect(problems.length).toBe(1)
|
||||
expect(problems[0]).toContain('va_parmap "BROKEN"')
|
||||
})
|
||||
})
|
||||
|
||||
describe('transforms', () => {
|
||||
it('parses an ADD anchor with an offset', () => {
|
||||
const { map, problems } = parseVaParmap(
|
||||
'va_parmap=SELECTED-D_DATE:ADD(QUARTER_START,-1)'
|
||||
)
|
||||
expect(problems).toEqual([])
|
||||
expect(map.get('selected')?.column).toBe('D_DATE')
|
||||
expect(map.get('selected')?.transform).toEqual({
|
||||
op: 'ADD',
|
||||
anchor: 'QUARTER_START',
|
||||
offset: -1
|
||||
})
|
||||
})
|
||||
|
||||
it('defaults the offset to zero and accepts lower case', () => {
|
||||
const { map, problems } = parseVaParmap(
|
||||
'va_parmap=sel-d_date:add(quarter_start)'
|
||||
)
|
||||
expect(problems).toEqual([])
|
||||
expect(map.get('sel')?.transform).toEqual({
|
||||
op: 'ADD',
|
||||
anchor: 'QUARTER_START',
|
||||
offset: 0
|
||||
})
|
||||
})
|
||||
|
||||
it('parses a RANGE period', () => {
|
||||
const { map, problems } = parseVaParmap(
|
||||
'va_parmap=sel-D_DATE:RANGE(quarter,-1)'
|
||||
)
|
||||
expect(problems).toEqual([])
|
||||
expect(map.get('sel')?.transform).toEqual({
|
||||
op: 'RANGE',
|
||||
period: 'QUARTER',
|
||||
offset: -1
|
||||
})
|
||||
})
|
||||
|
||||
it('leaves a mapping without a transform as a plain column', () => {
|
||||
const { map } = parseVaParmap('va_parmap=sel-D_DATE')
|
||||
expect(map.get('sel')?.column).toBe('D_DATE')
|
||||
expect(map.get('sel')?.transform).toBeUndefined()
|
||||
})
|
||||
|
||||
it('reports an unknown operation, anchor, period or offset', () => {
|
||||
const cases: [string, string][] = [
|
||||
['sel-D_DATE:SHIFT(QUARTER_START,-1)', 'is not a known transform'],
|
||||
['sel-D_DATE:ADD(FORTNIGHT_START,-1)', 'is not a known anchor'],
|
||||
['sel-D_DATE:RANGE(DECADE,-1)', 'is not a known period'],
|
||||
['sel-D_DATE:ADD(QUARTER_START,3.5)', 'is not a whole number'],
|
||||
['sel-D_DATE:ADD', 'is not a transform']
|
||||
]
|
||||
for (const [entry, expected] of cases) {
|
||||
const { map, problems } = parseVaParmap(`va_parmap=${entry}`)
|
||||
expect(`${entry}: ${map.size}`).toBe(`${entry}: 0`)
|
||||
expect(`${entry}: ${problems.length}`).toBe(`${entry}: 1`)
|
||||
expect(problems[0]).toContain(expected)
|
||||
}
|
||||
})
|
||||
|
||||
it('keeps the parameter label free to contain a colon', () => {
|
||||
// The key side is untouched by the transform grammar: only the column side
|
||||
// is split, so a label with a colon still maps.
|
||||
const { map, problems } = parseVaParmap(
|
||||
'va_parmap=Sales:2026-D_DATE:ADD(QUARTER_START,-1)'
|
||||
)
|
||||
expect(problems).toEqual([])
|
||||
expect(map.get('sales:2026')?.column).toBe('D_DATE')
|
||||
expect(map.get('sales:2026')?.transform).toEqual({
|
||||
op: 'ADD',
|
||||
anchor: 'QUARTER_START',
|
||||
offset: -1
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
describe('one parameter, one column', () => {
|
||||
it('keeps the first mapping and reports a conflicting duplicate', () => {
|
||||
const { map, problems } = parseVaParmap(
|
||||
'va_parmap=DEPT-REGION&va_parmap=DEPT-OTHER'
|
||||
)
|
||||
expect(map.get('dept')?.column).toBe('REGION')
|
||||
expect(problems).toEqual([
|
||||
'va_parmap "DEPT-OTHER" was ignored: VA parameter "dept" already maps to REGION, and a parameter can filter only one column.'
|
||||
])
|
||||
})
|
||||
|
||||
it('treats a duplicate key as a duplicate across case', () => {
|
||||
const { map, problems } = parseVaParmap(
|
||||
'va_parmap=Dept-REGION&va_parmap=DEPT-SOMETHING_ELSE'
|
||||
)
|
||||
expect(map.get('dept')?.column).toBe('REGION')
|
||||
expect(problems.length).toBe(1)
|
||||
})
|
||||
|
||||
it('accepts the same mapping written twice (no ambiguity)', () => {
|
||||
const { map, problems } = parseVaParmap(
|
||||
'va_parmap=DEPT-REGION&va_parmap=dept-region'
|
||||
)
|
||||
expect(map.get('dept')?.column).toBe('REGION')
|
||||
expect(problems).toEqual([])
|
||||
})
|
||||
|
||||
it('still allows two parameters to map to one column', () => {
|
||||
// The multi-object case: different parameters, same DC column.
|
||||
const { map, problems } = parseVaParmap(
|
||||
'va_parmap=REGION_TAB1-REGION&va_parmap=REGION_TAB2-REGION'
|
||||
)
|
||||
expect(map.size).toBe(2)
|
||||
expect(problems).toEqual([])
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,192 @@
|
||||
import {
|
||||
VA_ANCHORS,
|
||||
VA_PERIODS,
|
||||
VaAnchor,
|
||||
VaPeriod,
|
||||
VaTransform
|
||||
} from './va-parmap-transform'
|
||||
|
||||
/**
|
||||
* What a VA parameter's mapping resolves to: the DC column, and optionally a
|
||||
* transform that adds derived dates to the filter (e.g. the previous quarter's
|
||||
* row alongside the selected one).
|
||||
*/
|
||||
export interface VaParmapTarget {
|
||||
column: string
|
||||
transform?: VaTransform
|
||||
}
|
||||
|
||||
/** Result of `parseVaParmap` - the usable mappings, and what was ignored. */
|
||||
export interface VaParmap {
|
||||
/** Lower-cased VA parameter key -> DC column (and transform, if any). */
|
||||
map: Map<string, VaParmapTarget>
|
||||
/** One plain-English line per `va_parmap` entry that was ignored. */
|
||||
problems: string[]
|
||||
}
|
||||
|
||||
/**
|
||||
* Parses the `va_parmap` entries out of a hash query string (the part after `?`
|
||||
* in `window.location.hash`).
|
||||
*
|
||||
* Format - one mapping per occurrence, each `<VA parameter key>-<DC column>`:
|
||||
*
|
||||
* ?embed=va&va_parmap=REGION_TAB1-REGION&va_parmap=REGION_TAB2-REGION
|
||||
*
|
||||
* Repeating the parameter (rather than packing a comma-separated list into one
|
||||
* value) keeps the grammar unambiguous: VA parameter labels are free text, so
|
||||
* any list separator we picked could also appear inside a label.
|
||||
*
|
||||
* Each entry is split at its LAST `-`. A SAS column name can never contain a
|
||||
* hyphen, so the final one always delimits the column, and a label that does
|
||||
* contain hyphens (e.g. `Sales-2024`) still maps correctly.
|
||||
*
|
||||
* The key is matched case-insensitively, and trailing/leading blanks are
|
||||
* trimmed, against a VA parameter's `label`, which is the same key the
|
||||
* label->column match uses. The internal `name` (e.g. `pr273`) is not a key: it
|
||||
* is the report's own bookkeeping, so an entry written against it never applies
|
||||
* and is reported by `unmatchedMappings`.
|
||||
*
|
||||
* An entry that cannot be used is reported in `problems` rather than dropped in
|
||||
* silence: these entries are hand-written in the report's DDC URL, usually by
|
||||
* the report author, and a typo would otherwise degrade to "no filter" - which
|
||||
* looks exactly like a working filter over the whole table.
|
||||
*
|
||||
* Returns an empty map when the parameter is absent, which leaves the default
|
||||
* same-name resolution untouched (existing embeds are unaffected).
|
||||
*
|
||||
* Modeled on `parseEmbedParam` / `parseLabelsParam`, which read the same hash
|
||||
* query.
|
||||
*/
|
||||
export function parseVaParmap(hashQuery: string): VaParmap {
|
||||
const map = new Map<string, VaParmapTarget>()
|
||||
const problems: string[] = []
|
||||
if (!hashQuery) return { map, problems }
|
||||
|
||||
const entries = new URLSearchParams(hashQuery).getAll('va_parmap')
|
||||
for (const entry of entries) {
|
||||
// A transform sits at the end, in parentheses: `<param>-<column>:OP(args)`.
|
||||
// It is lifted off FIRST, because its arguments may contain a hyphen (a
|
||||
// negative offset) and the key/column split below uses the hyphen.
|
||||
let body = entry ?? ''
|
||||
let transformSpec = ''
|
||||
if (body.trim().endsWith(')')) {
|
||||
const open = body.lastIndexOf('(')
|
||||
const colon = open > 0 ? body.lastIndexOf(':', open) : -1
|
||||
if (colon > 0) {
|
||||
transformSpec = body.slice(colon + 1).trim()
|
||||
body = body.slice(0, colon)
|
||||
}
|
||||
}
|
||||
// Last hyphen delimits the DC column (see above).
|
||||
const separator = body.lastIndexOf('-')
|
||||
if (separator < 0) {
|
||||
problems.push(
|
||||
`va_parmap "${entry}" was ignored: it needs a "-" between the VA parameter and the column name.`
|
||||
)
|
||||
continue
|
||||
}
|
||||
const key = body.slice(0, separator).trim().toLowerCase()
|
||||
const column = body.slice(separator + 1).trim()
|
||||
if (!key) {
|
||||
problems.push(
|
||||
`va_parmap "${entry}" was ignored: the VA parameter name is missing.`
|
||||
)
|
||||
continue
|
||||
}
|
||||
if (column.includes(':')) {
|
||||
problems.push(
|
||||
`va_parmap "${entry}" was ignored: "${column}" is not a transform - expected e.g. ADD(QUARTER_START,-1).`
|
||||
)
|
||||
continue
|
||||
}
|
||||
// The column name is interpolated into a SAS WHERE clause, so only a plain
|
||||
// SAS name is accepted - anything else is refused rather than mapped.
|
||||
if (!/^[A-Za-z_][A-Za-z0-9_]{0,31}$/.test(column)) {
|
||||
problems.push(
|
||||
`va_parmap "${entry}" was ignored: "${column}" is not a valid SAS column name.`
|
||||
)
|
||||
continue
|
||||
}
|
||||
const transform = transformSpec
|
||||
? parseTransform(transformSpec, entry, problems)
|
||||
: undefined
|
||||
if (transformSpec && !transform) continue
|
||||
// One VA parameter filters one column. A second entry for the same parameter
|
||||
// is ambiguous, so the FIRST one wins and the conflict is reported - the
|
||||
// alternative (last one wins, silently) would apply a filter the author
|
||||
// never sees the origin of. Two parameters mapping to the same column is
|
||||
// fine and is how two report objects filter one column from their own
|
||||
// parameters.
|
||||
const existing = map.get(key)
|
||||
if (existing) {
|
||||
if (existing.column.toUpperCase() !== column.toUpperCase()) {
|
||||
problems.push(
|
||||
`va_parmap "${entry}" was ignored: VA parameter "${key}" already maps to ${existing.column}, and a parameter can filter only one column.`
|
||||
)
|
||||
}
|
||||
continue
|
||||
}
|
||||
map.set(key, { column, transform })
|
||||
}
|
||||
|
||||
return { map, problems }
|
||||
}
|
||||
|
||||
/**
|
||||
* Parses `OP(arg[, arg])`, returning undefined (and pushing a problem line) when
|
||||
* the operation or one of its arguments is not in the allow-list.
|
||||
*
|
||||
* ADD takes an anchor and an optional offset; RANGE takes a period and an
|
||||
* optional offset. The offset is whole periods, negative for earlier.
|
||||
*/
|
||||
function parseTransform(
|
||||
spec: string,
|
||||
entry: string,
|
||||
problems: string[]
|
||||
): VaTransform | undefined {
|
||||
const match = /^([A-Za-z]+)\s*\(([^)]*)\)$/.exec(spec)
|
||||
if (!match) {
|
||||
problems.push(
|
||||
`va_parmap "${entry}" was ignored: "${spec}" is not a transform - expected e.g. ADD(QUARTER_START,-1).`
|
||||
)
|
||||
return undefined
|
||||
}
|
||||
const op = match[1].toUpperCase()
|
||||
const args = match[2]
|
||||
.split(',')
|
||||
.map((arg) => arg.trim())
|
||||
.filter((arg) => arg.length > 0)
|
||||
const offsetArg = args[1] === undefined ? '0' : args[1]
|
||||
if (!/^-?\d+$/.test(offsetArg)) {
|
||||
problems.push(
|
||||
`va_parmap "${entry}" was ignored: the offset "${offsetArg}" is not a whole number of periods.`
|
||||
)
|
||||
return undefined
|
||||
}
|
||||
const offset = Number(offsetArg)
|
||||
|
||||
if (op === 'ADD') {
|
||||
const anchor = (args[0] || '').toUpperCase()
|
||||
if (!VA_ANCHORS.includes(anchor as VaAnchor)) {
|
||||
problems.push(
|
||||
`va_parmap "${entry}" was ignored: "${args[0] || ''}" is not a known anchor (${VA_ANCHORS.join(', ')}).`
|
||||
)
|
||||
return undefined
|
||||
}
|
||||
return { op: 'ADD', anchor: anchor as VaAnchor, offset }
|
||||
}
|
||||
if (op === 'RANGE') {
|
||||
const period = (args[0] || '').toUpperCase()
|
||||
if (!VA_PERIODS.includes(period as VaPeriod)) {
|
||||
problems.push(
|
||||
`va_parmap "${entry}" was ignored: "${args[0] || ''}" is not a known period (${VA_PERIODS.join(', ')}).`
|
||||
)
|
||||
return undefined
|
||||
}
|
||||
return { op: 'RANGE', period: period as VaPeriod, offset }
|
||||
}
|
||||
problems.push(
|
||||
`va_parmap "${entry}" was ignored: "${match[1]}" is not a known transform (ADD, RANGE).`
|
||||
)
|
||||
return undefined
|
||||
}
|
||||
@@ -0,0 +1,194 @@
|
||||
import {
|
||||
applyVaTransform,
|
||||
isRangeTransform,
|
||||
VA_ANCHORS,
|
||||
VA_PERIODS,
|
||||
VaTransform
|
||||
} from './va-parmap-transform'
|
||||
|
||||
/** Local y-m-d, so assertions do not depend on the runner's timezone offset. */
|
||||
const ymd = (d: Date) =>
|
||||
`${d.getFullYear()}-${String(d.getMonth() + 1).padStart(2, '0')}-${String(
|
||||
d.getDate()
|
||||
).padStart(2, '0')}`
|
||||
|
||||
const date = (y: number, m: number, d: number) => new Date(y, m - 1, d)
|
||||
|
||||
describe('va_parmap transforms', () => {
|
||||
it('exposes only allow-listed anchors and periods', () => {
|
||||
expect(VA_ANCHORS).toContain('QUARTER_START')
|
||||
expect(VA_PERIODS).toEqual(['QUARTER', 'MONTH', 'YEAR'])
|
||||
})
|
||||
|
||||
describe('ADD - the value plus one anchor beside it', () => {
|
||||
it('adds the previous quarter start, which is the reported requirement', () => {
|
||||
// intnx('qtr', dt, -1, 'b') for a value on a quarter boundary
|
||||
const dates = applyVaTransform(date(2026, 10, 1), {
|
||||
op: 'ADD',
|
||||
anchor: 'QUARTER_START',
|
||||
offset: -1
|
||||
})
|
||||
expect(dates.map(ymd)).toEqual(['2026-10-01', '2026-07-01'])
|
||||
})
|
||||
|
||||
it('resolves a mid-quarter value to its own value plus the previous quarter start', () => {
|
||||
const dates = applyVaTransform(date(2026, 8, 15), {
|
||||
op: 'ADD',
|
||||
anchor: 'QUARTER_START',
|
||||
offset: -1
|
||||
})
|
||||
expect(dates.map(ymd)).toEqual(['2026-08-15', '2026-04-01'])
|
||||
})
|
||||
|
||||
it('de-duplicates when the value already sits on the anchor', () => {
|
||||
// 2026-01-01 is a quarter start, so "0 quarters back" adds nothing
|
||||
const dates = applyVaTransform(date(2026, 1, 1), {
|
||||
op: 'ADD',
|
||||
anchor: 'QUARTER_START',
|
||||
offset: 0
|
||||
})
|
||||
expect(dates.map(ymd)).toEqual(['2026-01-01'])
|
||||
})
|
||||
|
||||
it('resolves the other anchors at their boundaries', () => {
|
||||
const at = date(2026, 8, 15)
|
||||
expect(
|
||||
ymd(
|
||||
applyVaTransform(at, {
|
||||
op: 'ADD',
|
||||
anchor: 'QUARTER_END',
|
||||
offset: 0
|
||||
})[1]
|
||||
)
|
||||
).toBe('2026-09-30')
|
||||
expect(
|
||||
ymd(
|
||||
applyVaTransform(at, { op: 'ADD', anchor: 'MONTH_END', offset: 0 })[1]
|
||||
)
|
||||
).toBe('2026-08-31')
|
||||
expect(
|
||||
ymd(
|
||||
applyVaTransform(at, {
|
||||
op: 'ADD',
|
||||
anchor: 'YEAR_START',
|
||||
offset: -1
|
||||
})[1]
|
||||
)
|
||||
).toBe('2025-01-01')
|
||||
expect(
|
||||
ymd(
|
||||
applyVaTransform(at, { op: 'ADD', anchor: 'YEAR_END', offset: 0 })[1]
|
||||
)
|
||||
).toBe('2026-12-31')
|
||||
})
|
||||
|
||||
it('handles February and leap years at month end', () => {
|
||||
expect(
|
||||
ymd(
|
||||
applyVaTransform(date(2026, 3, 1), {
|
||||
op: 'ADD',
|
||||
anchor: 'MONTH_END',
|
||||
offset: -1
|
||||
})[1]
|
||||
)
|
||||
).toBe('2026-02-28')
|
||||
expect(
|
||||
ymd(
|
||||
applyVaTransform(date(2024, 3, 15), {
|
||||
op: 'ADD',
|
||||
anchor: 'MONTH_END',
|
||||
offset: -1
|
||||
})[1]
|
||||
)
|
||||
).toBe('2024-02-29')
|
||||
})
|
||||
|
||||
it('crosses a year boundary when the offset takes it out of the year', () => {
|
||||
const dates = applyVaTransform(date(2027, 1, 1), {
|
||||
op: 'ADD',
|
||||
anchor: 'QUARTER_START',
|
||||
offset: -1
|
||||
})
|
||||
expect(dates.map(ymd)).toEqual(['2027-01-01', '2026-10-01'])
|
||||
})
|
||||
})
|
||||
|
||||
describe('RANGE - a closed period', () => {
|
||||
it('covers the previous quarter', () => {
|
||||
const dates = applyVaTransform(date(2026, 8, 15), {
|
||||
op: 'RANGE',
|
||||
period: 'QUARTER',
|
||||
offset: -1
|
||||
})
|
||||
expect(dates.map(ymd)).toEqual(['2026-04-01', '2026-06-30'])
|
||||
})
|
||||
|
||||
it('covers the current month and the previous year', () => {
|
||||
expect(
|
||||
applyVaTransform(date(2026, 8, 15), {
|
||||
op: 'RANGE',
|
||||
period: 'MONTH',
|
||||
offset: 0
|
||||
}).map(ymd)
|
||||
).toEqual(['2026-08-01', '2026-08-31'])
|
||||
expect(
|
||||
applyVaTransform(date(2026, 8, 15), {
|
||||
op: 'RANGE',
|
||||
period: 'YEAR',
|
||||
offset: -1
|
||||
}).map(ymd)
|
||||
).toEqual(['2025-01-01', '2025-12-31'])
|
||||
})
|
||||
|
||||
it('reports itself as a range so the caller picks BETWEEN', () => {
|
||||
expect(
|
||||
isRangeTransform({ op: 'RANGE', period: 'QUARTER', offset: 0 })
|
||||
).toBe(true)
|
||||
expect(
|
||||
isRangeTransform({
|
||||
op: 'ADD',
|
||||
anchor: 'QUARTER_START',
|
||||
offset: 0
|
||||
} as VaTransform)
|
||||
).toBe(false)
|
||||
})
|
||||
})
|
||||
|
||||
describe('time of day', () => {
|
||||
it('gives a datetime range whole days, not the value clock time', () => {
|
||||
// A BETWEEN that starts at 13:45 on day one loses the earlier part of that
|
||||
// day, and one that ends at 13:45 on the last day loses the rest of it.
|
||||
const value = new Date(2026, 7, 15, 13, 45, 30)
|
||||
const [start, end] = applyVaTransform(
|
||||
value,
|
||||
{ op: 'RANGE', period: 'QUARTER', offset: 0 },
|
||||
true
|
||||
)
|
||||
expect([
|
||||
start.getHours(),
|
||||
start.getMinutes(),
|
||||
start.getSeconds()
|
||||
]).toEqual([0, 0, 0])
|
||||
expect([end.getHours(), end.getMinutes(), end.getSeconds()]).toEqual([
|
||||
23, 59, 59
|
||||
])
|
||||
expect(ymd(start)).toBe('2026-07-01')
|
||||
expect(ymd(end)).toBe('2026-09-30')
|
||||
})
|
||||
|
||||
it('keeps the time for datetime columns and drops it otherwise', () => {
|
||||
const value = new Date(2026, 9, 1, 13, 45, 30)
|
||||
const add: VaTransform = {
|
||||
op: 'ADD',
|
||||
anchor: 'QUARTER_START',
|
||||
offset: -1
|
||||
}
|
||||
const kept = applyVaTransform(value, add, true)[1]
|
||||
expect([kept.getHours(), kept.getMinutes(), kept.getSeconds()]).toEqual([
|
||||
13, 45, 30
|
||||
])
|
||||
const dropped = applyVaTransform(value, add, false)[1]
|
||||
expect([dropped.getHours(), dropped.getMinutes()]).toEqual([0, 0])
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,151 @@
|
||||
/**
|
||||
* Anchors and periods a `va_parmap` transform can resolve a report value to.
|
||||
*
|
||||
* The names describe the result rather than mirroring SAS `intnx`, because the
|
||||
* arithmetic runs in the browser: emulating `intnx`'s alignment letters (`b`,
|
||||
* `e`, `s`, `m`, never mind week alignment and fiscal quarters) would invite
|
||||
* assumptions we could not honour. Each anchor below is one small function, so
|
||||
* adding one is a function plus a test row.
|
||||
*/
|
||||
export const VA_ANCHORS = [
|
||||
'QUARTER_START',
|
||||
'QUARTER_END',
|
||||
'MONTH_START',
|
||||
'MONTH_END',
|
||||
'YEAR_START',
|
||||
'YEAR_END'
|
||||
] as const
|
||||
export type VaAnchor = (typeof VA_ANCHORS)[number]
|
||||
|
||||
export const VA_PERIODS = ['QUARTER', 'MONTH', 'YEAR'] as const
|
||||
export type VaPeriod = (typeof VA_PERIODS)[number]
|
||||
|
||||
/**
|
||||
* A parsed transform: the operation plus its validated arguments.
|
||||
*
|
||||
* `ADD` yields the value and one anchor beside it (a list of dates), which is
|
||||
* how "this period's row plus the previous quarter's" is expressed. `RANGE`
|
||||
* yields a closed period (two dates) for a BETWEEN.
|
||||
*/
|
||||
export type VaTransform =
|
||||
| { op: 'ADD'; anchor: VaAnchor; offset: number }
|
||||
| { op: 'RANGE'; period: VaPeriod; offset: number }
|
||||
|
||||
/** True when the transform produces a closed range rather than a list. */
|
||||
export function isRangeTransform(transform: VaTransform): boolean {
|
||||
return transform.op === 'RANGE'
|
||||
}
|
||||
|
||||
/** Local midnight on the first day of the month `offset` months from `date`. */
|
||||
function monthStart(date: Date, offset: number): Date {
|
||||
return new Date(date.getFullYear(), date.getMonth() + offset, 1)
|
||||
}
|
||||
|
||||
/** Local midnight on the last day of the month `offset` months from `date`. */
|
||||
function monthEnd(date: Date, offset: number): Date {
|
||||
return new Date(date.getFullYear(), date.getMonth() + offset + 1, 0)
|
||||
}
|
||||
|
||||
/** First day of the quarter `offset` quarters from the one containing `date`. */
|
||||
function quarterStart(date: Date, offset: number): Date {
|
||||
const firstMonthOfQuarter = 3 * Math.floor(date.getMonth() / 3)
|
||||
return new Date(date.getFullYear(), firstMonthOfQuarter + offset * 3, 1)
|
||||
}
|
||||
|
||||
/** Last day of the quarter `offset` quarters from the one containing `date`. */
|
||||
function quarterEnd(date: Date, offset: number): Date {
|
||||
// Day 0 of a month is the last day of the month before it: calendar
|
||||
// construction, so a daylight-saving transition inside the quarter cannot
|
||||
// shift the result by an hour the way subtracting 24 hours would.
|
||||
const nextStart = quarterStart(date, offset + 1)
|
||||
return new Date(nextStart.getFullYear(), nextStart.getMonth(), 0)
|
||||
}
|
||||
|
||||
/** First day of the year `offset` years from the one containing `date`. */
|
||||
function yearStart(date: Date, offset: number): Date {
|
||||
return new Date(date.getFullYear() + offset, 0, 1)
|
||||
}
|
||||
|
||||
/** Last day of the year `offset` years from the one containing `date`. */
|
||||
function yearEnd(date: Date, offset: number): Date {
|
||||
return new Date(date.getFullYear() + offset, 11, 31)
|
||||
}
|
||||
|
||||
/** Resolve one anchor, keeping the time of day of `date` for datetime columns. */
|
||||
function resolveAnchor(
|
||||
anchor: VaAnchor,
|
||||
date: Date,
|
||||
offset: number,
|
||||
keepTime: boolean
|
||||
): Date {
|
||||
let resolved: Date
|
||||
switch (anchor) {
|
||||
case 'QUARTER_START':
|
||||
resolved = quarterStart(date, offset)
|
||||
break
|
||||
case 'QUARTER_END':
|
||||
resolved = quarterEnd(date, offset)
|
||||
break
|
||||
case 'MONTH_START':
|
||||
resolved = monthStart(date, offset)
|
||||
break
|
||||
case 'MONTH_END':
|
||||
resolved = monthEnd(date, offset)
|
||||
break
|
||||
case 'YEAR_START':
|
||||
resolved = yearStart(date, offset)
|
||||
break
|
||||
default:
|
||||
resolved = yearEnd(date, offset)
|
||||
break
|
||||
}
|
||||
if (keepTime) {
|
||||
resolved.setHours(date.getHours(), date.getMinutes(), date.getSeconds(), 0)
|
||||
}
|
||||
return resolved
|
||||
}
|
||||
|
||||
/**
|
||||
* The dates a transform adds to a mapping, in order.
|
||||
*
|
||||
* `ADD` returns the report's own value plus the anchor beside it, de-duplicated -
|
||||
* so a value that already sits on the anchor yields one date, and the clause
|
||||
* stays an equality rather than an `IN` with a repeat. `RANGE` returns the
|
||||
* period's first and last day.
|
||||
*
|
||||
* `keepTime` carries the time of day across for datetime columns, since a
|
||||
* DATETIME filter compares to the second; date columns use midnight.
|
||||
*/
|
||||
export function applyVaTransform(
|
||||
value: Date,
|
||||
transform: VaTransform,
|
||||
keepTime = false
|
||||
): Date[] {
|
||||
if (transform.op === 'RANGE') {
|
||||
// A range covers whole days: the bounds are the period's own, never the
|
||||
// value's clock time. Reusing keepTime here would start the range mid-morning
|
||||
// on day one and end it mid-morning on the last day, dropping the rest of
|
||||
// both boundary days from a BETWEEN.
|
||||
const start = resolveAnchor(
|
||||
(transform.period + '_START') as VaAnchor,
|
||||
value,
|
||||
transform.offset,
|
||||
false
|
||||
)
|
||||
const end = resolveAnchor(
|
||||
(transform.period + '_END') as VaAnchor,
|
||||
value,
|
||||
transform.offset,
|
||||
false
|
||||
)
|
||||
if (keepTime) end.setHours(23, 59, 59, 0)
|
||||
return [start, end]
|
||||
}
|
||||
const anchor = resolveAnchor(
|
||||
transform.anchor,
|
||||
value,
|
||||
transform.offset,
|
||||
keepTime
|
||||
)
|
||||
return anchor.getTime() === value.getTime() ? [value] : [value, anchor]
|
||||
}
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "dcfrontend",
|
||||
"version": "7.18.0",
|
||||
"version": "7.20.0",
|
||||
"description": "Data Controller",
|
||||
"devDependencies": {
|
||||
"@nogoo9/gitleaks": "8.30.1-post.2",
|
||||
|
||||
@@ -11,5 +11,6 @@
|
||||
"noNestedMacros": true,
|
||||
"hasMacroParentheses": true,
|
||||
"strictMacroDefinition": true,
|
||||
"ignoredMacros": ["webout", "mpeinit", "mpeterm"],
|
||||
"ignoreList":["sasjs/scratch"]
|
||||
}
|
||||
@@ -13,7 +13,7 @@
|
||||
than the abort's own.
|
||||
|
||||
An unassigned outref (a child that died before writing a response) records a
|
||||
FAIL rather than raising an unassigned-fileref error in the data step.
|
||||
FAIL rather than letting the data step fail on an unassigned fileref.
|
||||
|
||||
Usage:
|
||||
|
||||
@@ -47,13 +47,13 @@
|
||||
%let abortseen=0;
|
||||
|
||||
%if %sysfunc(fileref(&outref))>0 %then %do;
|
||||
%put WARNING: &sysmacroname: fileref &outref is not assigned - the child left no response to read;
|
||||
%put %str(WARN)ING: &sysmacroname: fileref &outref is not assigned - the child left no response to read;
|
||||
%end;
|
||||
%else %if %sysfunc(fileexist(%sysfunc(pathname(&outref)))) ne 1 %then %do;
|
||||
/* the fileref exists but nothing was written to it: a child that aborted
|
||||
before it produced a response. Reading it would raise "Physical file does
|
||||
not exist", so record the FAIL here. */
|
||||
%put WARNING: &sysmacroname: &outref has no response written to it - the child produced nothing;
|
||||
%put %str(WARN)ING: &sysmacroname: &outref has no response written to it - the child produced nothing;
|
||||
%end;
|
||||
%else %do;
|
||||
data _null_;
|
||||
|
||||
@@ -80,9 +80,9 @@ insert into &lib..mpe_config set
|
||||
!!' client side. The DC_MAXCELLS_WEBEDIT limit is applied at the same'
|
||||
!!' time, whichever is reached first.';
|
||||
/* Why the cell limit exists, kept out of var_desc - MPE_CONFIG.VAR_DESC is
|
||||
varchar(300), and a truncation WARNING aborts this whole build. A wide
|
||||
table is expensive long before it is tall - 250 rows of a 1000 column
|
||||
table is a quarter of a million cells. */
|
||||
varchar(300), and a truncated value raises a diagnostic that aborts this
|
||||
whole build. A wide table is expensive long before it is tall - 250 rows of
|
||||
a 1000 column table is a quarter of a million cells. */
|
||||
insert into &lib..mpe_config set
|
||||
tx_from=0
|
||||
,tx_to='31DEC9999:23:59:59'dt
|
||||
@@ -2214,8 +2214,8 @@ insert into &lib..MPE_VALIDATIONS set
|
||||
,rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
/* test soft regex - PRIMARY_KEY_FIELD should be an integer (yellow if it
|
||||
contains a decimal point). All generated keys are integers, so no
|
||||
warning shows until a demo user enters a decimal. */
|
||||
contains a decimal point). All generated keys are integers, so nothing is
|
||||
flagged until a demo user enters a decimal. */
|
||||
insert into &lib..MPE_VALIDATIONS set
|
||||
tx_from=0
|
||||
,base_lib="&lib"
|
||||
|
||||
@@ -3,14 +3,15 @@
|
||||
@brief Refreshes the data catalog
|
||||
@details Assumes library is already assigned.
|
||||
|
||||
@param [in] etlsource identifies this load in the MPE_DATALOADS audit row.
|
||||
@param [in] etlsource= (0) Defaults to %mf_getuniquename(prefix=refreshcatalogs); it
|
||||
identifies this load in the MPE_DATALOADS audit row.
|
||||
|
||||
Usage:
|
||||
|
||||
%mpe_refreshcatalogs(sashelp)
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mf_uid.sas
|
||||
@li mf_getuniquename.sas
|
||||
@li mp_abort.sas
|
||||
@li bitemporal_dataloader.sas
|
||||
|
||||
@@ -20,11 +21,11 @@
|
||||
**/
|
||||
|
||||
%macro mpe_refreshcatalogs(lib,cat=#all
|
||||
,etlsource=refreshcatalog:mpe_refreshcatalogs:%mf_uid()
|
||||
,etlsource=0
|
||||
);
|
||||
%let lib=%upcase(&lib);
|
||||
%let cat=%upcase(&cat);
|
||||
%if "&etlsource"="0" %then %let etlsource=refreshcatalog:%sysmacroname:%mf_uid();
|
||||
%if "&etlsource"="0" %then %let etlsource=%mf_getuniquename(prefix=refreshcatalogs);
|
||||
|
||||
%put running &sysmacroname &lib for &cat;
|
||||
proc sql;
|
||||
|
||||
@@ -3,12 +3,12 @@
|
||||
@brief Refreshes the library data catalog
|
||||
@details
|
||||
|
||||
@param [in] etlsource= (refreshcatalog:<step>:%mf_uid()) identifies this load
|
||||
@param [in] etlsource= (0) Defaults to %mf_getuniquename(prefix=refreshlibs), identifies this load
|
||||
in the MPE_DATALOADS audit row. That table is keyed on etlsource together
|
||||
with processed_dttm, libref and dsn, so the uid keeps each load distinct.
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mf_uid.sas
|
||||
@li mf_getuniquename.sas
|
||||
@li dc_getlibs.sas
|
||||
@li mf_getplatform.sas
|
||||
@li bitemporal_dataloader.sas
|
||||
@@ -22,8 +22,13 @@
|
||||
**/
|
||||
|
||||
%macro mpe_refreshlibs(lib=0
|
||||
,etlsource=refreshcatalog:mpe_refreshlibs:%mf_uid()
|
||||
,etlsource=0
|
||||
);
|
||||
/* a default of 0 rather than a %mf_uid() expression: a default is
|
||||
resolved when the macro is compiled, so it would be one value for the
|
||||
whole session and the audit folder below it would collide on the
|
||||
second call - as it did between the refresh steps of one run */
|
||||
%if "&etlsource"="0" %then %let etlsource=%mf_getuniquename(prefix=refreshlibs);
|
||||
|
||||
%dc_getlibs(outds=work.mm_getLibs)
|
||||
proc sort data=mm_getlibs;
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
@brief Refreshes the data catalog
|
||||
@details
|
||||
|
||||
@param [in] etlsource= (refreshcatalog:<step>:%mf_uid()) identifies this load
|
||||
@param [in] etlsource= (0) Defaults to %mf_getuniquename(prefix=refreshtables), identifies this load
|
||||
in the MPE_DATALOADS audit row. That table is keyed on etlsource together
|
||||
with processed_dttm, libref and dsn, so the uid keeps each load distinct.
|
||||
Usage:
|
||||
@@ -11,7 +11,7 @@
|
||||
%mpe_refreshtables(sashelp)
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mf_uid.sas
|
||||
@li mf_getuniquename.sas
|
||||
@li mp_abort.sas
|
||||
@li bitemporal_dataloader.sas
|
||||
@li mf_existfeature.sas
|
||||
@@ -24,8 +24,13 @@
|
||||
**/
|
||||
|
||||
%macro mpe_refreshtables(lib,ds=#all
|
||||
,etlsource=refreshcatalog:mpe_refreshtables:%mf_uid()
|
||||
,etlsource=0
|
||||
);
|
||||
/* a default of 0 rather than a %mf_uid() expression: a default is
|
||||
resolved when the macro is compiled, so it would be one value for the
|
||||
whole session and the audit folder below it would collide on the
|
||||
second call - as it did between the refresh steps of one run */
|
||||
%if "&etlsource"="0" %then %let etlsource=%mf_getuniquename(prefix=refreshtables);
|
||||
%let lib=%upcase(&lib);
|
||||
%let ds=%upcase(&ds);
|
||||
%local engine; %let engine=%mf_getengine(&lib);
|
||||
|
||||
@@ -186,9 +186,9 @@ run;
|
||||
|
||||
%global loadtype var_txfrom var_txto var_processed filter_text pk coltype
|
||||
sortpk var_busfrom var_busto rk_underlying buskey pre_edit_hook;
|
||||
/* clear first: a global survives the call, so in a reused session it would still
|
||||
* hold the previous request's values, and any path that skips an assignment
|
||||
* would apply them to this one */
|
||||
/* clear first: a global survives the call, so in a reused session it would
|
||||
still hold the previous request's values, and any path that skips an
|
||||
assignment would apply them to this one */
|
||||
%let loadtype=;
|
||||
%let var_txfrom=;
|
||||
%let var_txto=;
|
||||
@@ -552,7 +552,7 @@ select upcase(loadtype)
|
||||
%mpestp_getdata()
|
||||
|
||||
/* mp_abort cannot exit cleanly from inside a %include, so an abort there
|
||||
(eg in a hook script run via mp_include) only records the error in
|
||||
(eg in a hook script run via mp_include) only records the failure in
|
||||
work.mp_abort_errds. mode=INCLUDE picks that up and aborts properly here. */
|
||||
%mp_abort(mode=INCLUDE)
|
||||
|
||||
|
||||
@@ -65,8 +65,8 @@ select count(*) into: admin_check
|
||||
where groupname="&dc_admin_group";
|
||||
quit;
|
||||
%if &admin_check>0 %then %do;
|
||||
%put WARNING: %mf_getuser() is in the &dc_admin_group group;
|
||||
%put WARNING: Row Level Security is skipped for admins - test skipped;
|
||||
%put %str(WARN)ING: %mf_getuser() is in the &dc_admin_group group;
|
||||
%put %str(WARN)ING: Row Level Security is skipped for admins - test skipped;
|
||||
data work.test_results;
|
||||
length test_result $4 test_description $256 test_comments $256;
|
||||
test_result='PASS';
|
||||
|
||||
@@ -294,7 +294,7 @@ run;
|
||||
,dc_dttmtfmt=&dc_dttmtfmt
|
||||
)
|
||||
/* mp_abort cannot exit cleanly from inside a %include, so an abort there
|
||||
(eg in a hook script run via mp_include) only records the error in
|
||||
(eg in a hook script run via mp_include) only records the failure in
|
||||
work.mp_abort_errds. mode=INCLUDE picks that up and aborts properly here. */
|
||||
%mp_abort(mode=INCLUDE)
|
||||
|
||||
|
||||
@@ -15,10 +15,10 @@
|
||||
@li mf_existfeature.sas
|
||||
@li dc_assignlib.sas
|
||||
@li mp_abort.sas
|
||||
@li mp_binarycopy.sas
|
||||
@li mp_cntlout.sas
|
||||
@li mp_ds2cards.sas
|
||||
@li mp_ds2csv.sas
|
||||
@li mp_ds2md.sas
|
||||
@li mp_streamfile.sas
|
||||
@li mp_validatecol.sas
|
||||
@li mf_nobs.sas
|
||||
|
||||
@@ -78,25 +78,33 @@ run;
|
||||
outview=work.vw_derived,
|
||||
outvars=work.derived_vars
|
||||
)
|
||||
%let derived_nobs=%mf_nobs(work.derived_vars);
|
||||
/* A format catalog (LIBREF.CATALOGNAME-FC) must be left for mp_filterstore to
|
||||
rewrite into a cntlout dataset, and a -FC name handed over as targetds is not
|
||||
something it can read - so pass no targetds at all in that case. An empty
|
||||
targetds makes the engine fall back to libds, which is what a plain dataset
|
||||
wants anyway.
|
||||
|
||||
/* targetds is passed as before for a plain dataset, and as the view when the
|
||||
filter can reference derived columns. A format catalog is the one case that
|
||||
must be left alone: mp_filterstore rewrites a -FC libds to a cntlout dataset
|
||||
itself, and a -FC name handed over as targetds is not something it can read.
|
||||
The trailing comma lets the call below read targetds=..., queryds=... */
|
||||
%let targetds_opt=;
|
||||
No macro triggers in this comment: the macro processor resolves them even
|
||||
inside a comment, so prose about conditions is itself executed and the parser
|
||||
reports "Expected DO not found" from here. Same reason the values below are
|
||||
plain LET statements with no trailing comma. */
|
||||
%let derived_nobs=%sysfunc(coalesce(%mf_nobs(work.derived_vars),0));
|
||||
%let is_fmtcat=%eval(%length(&ds)>3 and "%upcase(%substr(&ds,%length(&ds)-2,3))"="-FC");
|
||||
%let targetds_val=&ds;
|
||||
%if &derived_nobs>0 %then %do;
|
||||
%let targetds_opt=targetds=work.vw_derived,;
|
||||
%let targetds_val=work.vw_derived;
|
||||
%end;
|
||||
%else %if %length(&ds)>3 and "%upcase(%substr(&ds,%length(&ds)-2,3))" ne "-FC" %then %do;
|
||||
%let targetds_opt=targetds=&ds,;
|
||||
/* the derived view keeps precedence: a catalog is only blanked when there is no
|
||||
derived configuration to validate against. Two plain %if blocks rather than
|
||||
%else %if - that construct is what the parser rejected here before. */
|
||||
%if &derived_nobs=0 and &is_fmtcat=1 %then %do;
|
||||
%let targetds_val=;
|
||||
%end;
|
||||
|
||||
%mp_filterstore(
|
||||
libds=&ds,
|
||||
&targetds_opt
|
||||
queryds=work.filterquery,
|
||||
libds=&ds
|
||||
,targetds=&targetds_val
|
||||
,queryds=work.filterquery,
|
||||
filter_summary=&dc_libref..mpe_filteranytable,
|
||||
filter_detail=&dc_libref..mpe_filtersource,
|
||||
lock_table=&dc_libref..mpe_lockanytable,
|
||||
|
||||
@@ -5,7 +5,6 @@
|
||||
<h4> SAS Macros </h4>
|
||||
@li mx_execute.sas
|
||||
@li mp_assertdsobs.sas
|
||||
@li mp_assertcolvals.sas
|
||||
@li mf_getuniquefileref.sas
|
||||
|
||||
**/
|
||||
@@ -34,8 +33,15 @@ run;
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputfiles=&f1:iwant &f2:filterquery,
|
||||
outlib=web1
|
||||
outlib=web1,
|
||||
outref=web1ref,
|
||||
mdebug=&sasjs_mdebug
|
||||
)
|
||||
data _null_;
|
||||
infile web1ref;
|
||||
input;
|
||||
putlog _infile_;
|
||||
run;
|
||||
data result;
|
||||
set web1.result;
|
||||
run;
|
||||
@@ -64,7 +70,8 @@ run;
|
||||
viyacontext=&defaultcontext,
|
||||
inputdatasets=work.iwant work.filterquery,
|
||||
outlib=web2,
|
||||
outref=web2ref
|
||||
outref=web2ref,
|
||||
mdebug=&sasjs_mdebug
|
||||
)
|
||||
data _null_;
|
||||
infile web2ref;
|
||||
@@ -119,7 +126,8 @@ run;
|
||||
viyacontext=&defaultcontext,
|
||||
inputdatasets=work.iwant work.filterquery,
|
||||
outlib=web3,
|
||||
outref=web3ref
|
||||
outref=web3ref,
|
||||
mdebug=&sasjs_mdebug
|
||||
)
|
||||
data _null_;
|
||||
infile web3ref;
|
||||
|
||||
@@ -55,7 +55,6 @@
|
||||
@li mp_cntlout.sas
|
||||
@li mp_getcols.sas
|
||||
@li mp_getpk.sas
|
||||
@li mp_jsonout.sas
|
||||
@li mp_searchdata.sas
|
||||
@li mp_validatecol.sas
|
||||
@li mpe_columnlevelsecurity.sas
|
||||
|
||||
@@ -11,6 +11,8 @@
|
||||
overwriting actual data. Pass a KEEPME column in the input data to keep
|
||||
them - test harnesses reseed before every run and cannot redeploy first.
|
||||
|
||||
@sasjslint {"noBannedStrings": false}
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mf_getapploc.sas
|
||||
@li mf_mkdir.sas
|
||||
@@ -215,9 +217,11 @@ run;
|
||||
* caller asked to keep them (KEEPME in the input data). Keeping them is for
|
||||
* test harnesses, which reseed before every run and cannot redeploy first. */
|
||||
%if &keepme ne 1 %then %do;
|
||||
%ms_deletefile(&root/services/admin/configurator.sas)
|
||||
%ms_deletefile(&_program..sas)
|
||||
%ms_deletefile(&root/services/admin/configurator.sas);
|
||||
%ms_deletefile(&_program..sas);
|
||||
%end;
|
||||
%else %do;
|
||||
%put NOTE: &_program retained, as KEEPME was requested;
|
||||
%end;
|
||||
%else %put NOTE: &_program retained, as KEEPME was requested;
|
||||
|
||||
|
||||
@@ -298,6 +298,8 @@ run;
|
||||
/* keep the program when KEEPME was requested, so a reseed does not need a
|
||||
* redeploy first */
|
||||
%if &keepme ne 1 %then %do;
|
||||
%mv_deletejes(path=&root/admin, name=makedata)
|
||||
%mv_deletejes(path=&root/admin, name=makedata);
|
||||
%end;
|
||||
%else %do;
|
||||
%put NOTE: &_program retained, as KEEPME was requested;
|
||||
%end;
|
||||
%else %put NOTE: &_program retained, as KEEPME was requested;
|
||||
@@ -17,7 +17,6 @@
|
||||
@li mddl_dc_difftable.sas
|
||||
@li mddl_dc_locktable.sas
|
||||
@li mp_init.sas
|
||||
@li mx_execute.sas
|
||||
@li ms_createfile.sas
|
||||
@li ms_deletefile.sas
|
||||
@li mx_getcode.sas
|
||||
|
||||
Reference in new issue
Block a user