Compare commits
68
Commits
424b111737
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
428e70a4c1 | ||
|
|
94e040c006 | ||
|
|
cdc0ab2a91 | ||
|
|
8449ab3987 | ||
|
|
405b28dccc | ||
|
|
559221fad4 | ||
|
|
999e16b8fc | ||
|
|
ce5b1aa961 | ||
|
|
f74100a818 | ||
|
|
a4a0f9cb2f | ||
|
|
e6a5cce475 | ||
|
|
97b4cd4d61 | ||
|
|
eeffa92f9e | ||
|
|
dd25be8e75 | ||
|
|
2e40749052 | ||
|
|
fc687f8643 | ||
|
|
c97e8866db | ||
|
|
ad061946c3 | ||
|
|
914a1606bb | ||
|
|
c803773d3c | ||
|
|
b8eea5063c | ||
|
|
0a172b0c37 | ||
|
|
888873421c | ||
|
|
af1813f829 | ||
|
|
27686d3aad | ||
|
|
d84a1d50f4 | ||
|
|
cf7b21a8b6 | ||
|
|
60589c78b6 | ||
|
|
8e2cce4150 | ||
|
|
f7e753863b | ||
|
|
febfea3e5e | ||
|
|
3580d6e9eb | ||
|
|
bc9e3ae50f | ||
|
|
330448c496 | ||
|
|
00257f9160 | ||
|
|
e51953ca8a | ||
|
|
aa91daf9f7 | ||
|
|
00df87f189 | ||
|
|
65699bdd39 | ||
|
|
459ff2da37 | ||
|
|
c0ea810766 | ||
|
|
62e43034e4 | ||
|
|
80e8911af8 | ||
|
|
bd3c220af2 | ||
|
|
5e8910f238 | ||
|
|
221dd8a15e | ||
|
|
e9216b0895 | ||
|
|
37d81c902a | ||
|
|
689b5b6709 | ||
|
|
10157c1159 | ||
|
|
541a61ba08 | ||
|
|
ad50acf647 | ||
|
|
cd1ce4a7e2 | ||
|
|
7e3045a21b | ||
|
|
88010c3980 | ||
|
|
c5808a16eb | ||
|
|
0011f7e933 | ||
|
|
4ed1e0684d | ||
|
|
2623a3fb4b | ||
|
|
388f418bf4 | ||
|
|
f83dc359e2 | ||
|
|
411f6c3354 | ||
|
|
f042ca82ea | ||
|
|
15a1f34d52 | ||
|
|
6486c80ed4 | ||
|
|
dd482c4069 | ||
|
|
f58925b299 | ||
|
|
25ffa71add |
No files matched your search
@@ -1,10 +1,11 @@
|
|||||||
# Testing
|
# Testing
|
||||||
|
|
||||||
Backend (SAS) tests are run with the sasjs CLI from the `sas/` directory.
|
Backend (SAS) tests are run with the sasjs CLI from the `sas/` directory. For a deploy-and-run on the shared state see the `dc-estate-testing` skill (`.agents/skills/dc-estate-testing/SKILL.md`), which covers the credentials, the `makedata` seed the suite needs and the estate's group name.
|
||||||
|
|
||||||
## Commands
|
## Commands
|
||||||
|
|
||||||
- Full deploy + test cycle: `npm run 4gl` (compiles, deploys to the 4gl target, runs makedata) then `sasjs test -t 4gl`.
|
- Full deploy + test cycle: `npm run 4gl` (compiles, deploys to the 4gl target, runs makedata) then `sasjs test -t 4gl`.
|
||||||
|
- The shared estate under its `server` target: `npm run server` (compile, deploy, seed) then `sasjs test -t server`.
|
||||||
- Run tests only (no rebuild/redeploy): `sasjs test -t 4gl`.
|
- Run tests only (no rebuild/redeploy): `sasjs test -t 4gl`.
|
||||||
- Run a subset of tests: `sasjs test -t 4gl SOMESTRING` — the positional argument filters to matching tests (e.g. `sasjs test -t 4gl stagedata.test.3`).
|
- Run a subset of tests: `sasjs test -t 4gl SOMESTRING` — the positional argument filters to matching tests (e.g. `sasjs test -t 4gl stagedata.test.3`).
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,79 @@
|
|||||||
|
---
|
||||||
|
name: dc-estate-testing
|
||||||
|
description: >
|
||||||
|
Use when deploying Data Controller to a shared SASjs Server estate (sas.4gl.io) or running the SAS
|
||||||
|
test suite against it. Covers the target and credential setup, the makedata seed the suite needs on
|
||||||
|
a fresh appLoc and the group it must name, how to read the results, and how to deploy the frontend
|
||||||
|
so the streamed app is the build you just made.
|
||||||
|
---
|
||||||
|
|
||||||
|
# Deploying and testing on the shared SASjs Server estate
|
||||||
|
|
||||||
|
The whole cycle is two npm scripts plus `sasjs test`. Everything below explains what those scripts do and the steps that go wrong.
|
||||||
|
|
||||||
|
cd sas && npm ci # macro dependencies
|
||||||
|
cd ../client && npm ci && npm run build # frontend only - see "Deploying the frontend"
|
||||||
|
cd ../sas && npm run server # cpfavicon + sasjs cbd -t server + makedata
|
||||||
|
npx sasjs test -t server
|
||||||
|
|
||||||
|
`npm run server` is `sasjs cbd -t server && npm run serverdata`, and the seed in the second half is not optional. `sasjs cbd` on its own deploys the services and the test programs but leaves the appLoc with no Data Controller configuration, and `testsetup` reads `<appLoc>/services/public/settings.sas` through the drive API. `makedata` is what creates that file, so without the seed `testsetup` gets a 404, aborts, and 59 of the 64 test programs fail to complete on setup alone - which reads like a broken deployment rather than a missing seed.
|
||||||
|
|
||||||
|
## The targets
|
||||||
|
|
||||||
|
`server` is the shared estate: `serverType: SASJS` on https://sas.4gl.io, `appLoc` `/Public/app/dc`, a real SAS 9 binary behind SASjs Server (`GET /SASjsApi/info` reports `mode: server`, `runTimes: ["sas"]`, `localLoginEnabled: true`). `4gl` is the same estate with a slightly different config, hence the older `npm run 4gl` / `4glmakedata` pair.
|
||||||
|
|
||||||
|
A run against `server` exercises the SASjs Server branch of every macro and the SAS 9 backend, and nothing else: no Viya branch, no SAS 9 STP branch. Say that when reporting, rather than implying the matrix is covered.
|
||||||
|
|
||||||
|
To deploy your own branch without replacing the app everyone else uses, add a target that copies `server` and changes `appLoc` and `streamServiceName` (for example `/Public/app/<you>/dc`), keep it out of the commit, and pass it with `-t`. `sasjs cbd` overwrites the whole appLoc it is pointed at.
|
||||||
|
|
||||||
|
## Credentials
|
||||||
|
|
||||||
|
The CLI logs in with a registered client, not a username and password: `.env.<target>` (gitignored, mode 600) needs `CLIENT`, `ACCESS_TOKEN` and `REFRESH_TOKEN`. With `CLIENT` alone it prints a `/SASjsLogon?client_id=...` URL and waits for an authorization code, which cannot be answered headless.
|
||||||
|
|
||||||
|
Mint the pair from a local login, with the password read from a file rather than argv:
|
||||||
|
|
||||||
|
1. `GET /` with a cookie jar - it sets `XSRF-TOKEN`, which every POST must echo in an `X-XSRF-TOKEN` header.
|
||||||
|
2. `POST /SASLogon/login` with `{"username","password"}`.
|
||||||
|
3. `GET /SASjsApi/client` on the same session - the registered client ids.
|
||||||
|
4. `POST /SASLogon/authorize` with `{"clientId"}` -> `{"code"}`.
|
||||||
|
5. `POST /SASjsApi/auth/token` with `{"clientId","code"}` -> `accessToken` / `refreshToken`.
|
||||||
|
|
||||||
|
An unmatched `/SASjsApi/...` path answers `200` with the Swagger page, so check the content type; HTML means the route does not exist. Delete the token file when the run is over.
|
||||||
|
|
||||||
|
## The makedata seed
|
||||||
|
|
||||||
|
`npm run serverdata` sends `deploy/makeDataServer.json`: one `fromjs` row with `ADMIN`, `DCPATH` and `OUTFMT`. Two things about it are worth knowing before you trust its result.
|
||||||
|
|
||||||
|
- **`ADMIN` must name a group the calling account is actually in.** The shipped value is `all-users`, the SASjs Server group that every logged-in user is a member of, which is what the shared estate needs. A name the account is not in - a Viya-style group such as `AllUsers`, or one estate's own group carried into another - still answers `SUCCESS` from the seed, and then every later `getdata`/`stagedata` call aborts with `<user> is not authorised to edit <lib>.<ds> in the <lib>.MPE_SECURITY table`. Check what the account belongs to with a call to `services/public/getgroups` before changing the value.
|
||||||
|
- **Each run creates a new library.** The response carries `STATUS`, `DCPATH` and `LIBREF` (for example `DC621835`), and the LIBREF is clock-derived, so a libref from an earlier run answers `dataset <lib>.<table> does not exist` - take the current one from the response.
|
||||||
|
- **`KEEPME` controls whether the seed can be repeated.** Without it `makedata` deletes itself and the configurator after a successful run (deliberately, so it cannot be executed twice over real data), and a second seed fails with `The Program at (<appLoc>/services/admin/makedata) does not exist` until you run `sasjs cbd` again. A harness that reseeds between runs sets `KEEPME` in the input row.
|
||||||
|
- **Seed once per deploy, not before every test run.** The suite builds its own `dctest` library from the seeded control tables; reseeding in between is wasted work, and there is no need to run `makedata` and `sasjs test` in the opposite order.
|
||||||
|
|
||||||
|
## Reading the results
|
||||||
|
|
||||||
|
Results land in `sas/sasjsresults/`: `testResults.json` (the counts worth quoting: `tests_with_results`, `tests_that_pass`, `failed_to_complete`, `completed_with_failures`), `testResults.csv` (one row per assertion, lower-case headers `test_target`, `test_loc`, `sasjs_test_id`, `test_suite_result`, `test_description`, `test_url`), and one SAS log per program under `logs/`. The CLI's closing lines are not the source of truth when they disagree with the JSON.
|
||||||
|
|
||||||
|
A current green run is 64 of 64 test programs with 241 assertions passing and nothing failing to complete.
|
||||||
|
|
||||||
|
`ERROR` lines in the per-program logs are expected and are not a failed run: the specs that assert a refusal get `ERROR: Execution canceled by an %ABORT CANCEL statement`, and the ones that feed bad payloads get the rejection message they are asserting on (`Invalid libds:...`, `Invalid libref:...`). Judge the run by the JSON counts, then by whether the errors you find are the ones the tests asked for.
|
||||||
|
|
||||||
|
## Deploying the frontend
|
||||||
|
|
||||||
|
The streamed web app is a separate artefact and `sasjs cbd` only streams it if `client/dist` exists:
|
||||||
|
|
||||||
|
1. `client/libraries/sheet-crypto.tgz.gpg` is an encrypted dependency (`file:libraries/sheet-crypto.tgz` in `client/package.json`) that is not on the public registry. Decrypt it with the passphrase the maintainer holds: `printf '%s' "$SHEET_PWD" | gpg --batch --yes --passphrase-fd 0 --output ./libraries/sheet-crypto.tgz --decrypt ./libraries/sheet-crypto.tgz.gpg`.
|
||||||
|
2. `client/.npmrc` is gitignored and needs `legacy-peer-deps=true` (the committed lock is written with it) plus the `@sheet` registry lines.
|
||||||
|
3. `cd client && npm ci && npm run build`.
|
||||||
|
4. The build carries the repo default appLoc, so point it at the deployment before deploying: rewrite `apploc="/Public/app/dc"` in `dist/index.html` to the target appLoc.
|
||||||
|
5. `cd ../sas && npx sasjs cbd -t <target>` - with `client/dist` present the log says `Copying web app files` and `Web app is available at <url>`.
|
||||||
|
|
||||||
|
Without step 1 the deploy prints `ERROR An error has occurred when compiling web app services ... webSourcePath: .../client/dist doesn't exist` and then deploys the services anyway, so the deploy looks successful while the app streams nothing. The AppStream URL is session-protected: an anonymous `curl` gets a `401`, so verify it with a logged-in cookie jar and fetch the `main.<hash>.js` the page names.
|
||||||
|
|
||||||
|
Never commit `client/.npmrc` or the decrypted tarball; both are ignored, and `git status` staying clean is the check.
|
||||||
|
|
||||||
|
## Reading a failed run
|
||||||
|
|
||||||
|
- `ERROR An error occurred while executing the request. Job location: <appLoc>/tests/...` means the response was not JSON, so the program broke before it could write results. Open `sasjsresults/logs/<suite>.log` and read the first `ERROR`; a single compile break cascades into hundreds of `ERROR 180-322: Statement is not valid` lines and the tail is noise.
|
||||||
|
- `WARNING: Apparent symbolic reference TESTLOC not resolved` in `testsetup.log`, next to a `404 Not Found` for `services/public/settings.sas`, is the missing seed described above, not a macro problem.
|
||||||
|
- `sasjs test` runs the DEPLOYED program, so after editing a service, hook or test, run `sasjs cbd` again before testing or you will debug code that is no longer in the tree.
|
||||||
|
- A deployment can succeed and still be missing its web app, its data or its configuration - check the thing you changed rather than the exit code.
|
||||||
Executable
+49
@@ -0,0 +1,49 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# The Cypress specs CI runs, and the order they run in.
|
||||||
|
#
|
||||||
|
# ORDER MATTERS. These specs share one mock estate that is seeded once per job,
|
||||||
|
# so several of them depend on the state they find:
|
||||||
|
#
|
||||||
|
# - csv-limited asserts the free-tier "Locked Feature" modal, which an earlier
|
||||||
|
# spec can remove by planting a valid licence. It has always run first.
|
||||||
|
# - hook-programs asserts that an edit raises an approval row against its
|
||||||
|
# target table; it runs early, before other specs have added rows to the
|
||||||
|
# approval tables.
|
||||||
|
#
|
||||||
|
# Specs added later are appended rather than inserted, so the sequence that was
|
||||||
|
# known green keeps its shape. metanav is deliberately absent - see
|
||||||
|
# https://git.datacontroller.io/dc/dc/issues/395.
|
||||||
|
#
|
||||||
|
# download is held out for a different reason - its download button is covered by
|
||||||
|
# a fixed-position element, see
|
||||||
|
# https://git.datacontroller.io/dc/dc/issues/398.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SPECS="csv-limited hook-programs liveness editor excel-multi-load excel csv filtering filter-panel licensing viewer-labels full-table-search viewbox stage browser-info ddl-export row-cell-limits derived-variables startupservice-launcher coltype-delete-record submitted-details embed-va"
|
||||||
|
|
||||||
|
# Specs that exist but are held out of CI, each with the reason. Listed here so
|
||||||
|
# the coverage check below can tell a deliberate exclusion from a forgotten file.
|
||||||
|
EXCLUDED="metanav download"
|
||||||
|
|
||||||
|
case "${1:-}" in
|
||||||
|
all)
|
||||||
|
# comma separated, the form `cypress run --spec` takes
|
||||||
|
for name in $SPECS; do printf 'cypress/e2e/%s.cy.ts,' "$name"; done | sed 's/,$//'
|
||||||
|
;;
|
||||||
|
coverage)
|
||||||
|
on_disk=$(cd "$2/client" && ls cypress/e2e/*.cy.ts | xargs -n1 basename | sed 's/\.cy\.ts$//' | sort)
|
||||||
|
listed=$(printf '%s\n' $SPECS $EXCLUDED | sort)
|
||||||
|
missing=$(comm -23 <(echo "$on_disk") <(echo "$listed"))
|
||||||
|
if [ -n "$missing" ]; then
|
||||||
|
echo "Specs that are neither run nor explicitly excluded:"
|
||||||
|
echo "$missing"
|
||||||
|
echo "Add them to SPECS (or EXCLUDED, with a reason) in .gitea/scripts/cypress-specs.sh"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "All $(echo "$on_disk" | wc -l) specs are accounted for: $(echo $SPECS | wc -w) run, $(echo $EXCLUDED | wc -w) excluded."
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
echo "usage: $0 {all|coverage <repo-root>}" >&2
|
||||||
|
exit 2
|
||||||
|
;;
|
||||||
|
esac
|
||||||
@@ -75,11 +75,33 @@ jobs:
|
|||||||
cd client
|
cd client
|
||||||
npm run test:headless
|
npm run test:headless
|
||||||
|
|
||||||
|
- name: Angular Tests in other timezones
|
||||||
|
# The VA filter clauses are built from local calendar dates, so a
|
||||||
|
# UTC-only run cannot see a date that slips a day west of UTC or east of
|
||||||
|
# it. Only the VA specs carry that property, so both scripts run that
|
||||||
|
# subset rather than the whole suite a second and third time.
|
||||||
|
run: |
|
||||||
|
cd client
|
||||||
|
npm run test:tz-west
|
||||||
|
npm run test:tz-east
|
||||||
|
|
||||||
|
- name: Every Cypress spec is in the CI list
|
||||||
|
# The spec list used to be spelled out inline in --spec, and five files
|
||||||
|
# quietly fell out of it - among them the VA embed suite. Fails when a
|
||||||
|
# spec on disk is not listed.
|
||||||
|
run: bash .gitea/scripts/cypress-specs.sh coverage .
|
||||||
|
|
||||||
- name: Production Build
|
- name: Production Build
|
||||||
run: |
|
run: |
|
||||||
cd client
|
cd client
|
||||||
npm run build
|
npm run build
|
||||||
|
|
||||||
|
|
||||||
|
# Cypress shard 1 of 4. The setup repeats in every shard because each
|
||||||
|
# one gets its own runner and its own SASjs server; that fixed cost is what
|
||||||
|
# limits how far sharding helps. The specs it runs come from
|
||||||
|
# .gitea/scripts/cypress-specs.sh.
|
||||||
|
|
||||||
Build-and-test-development:
|
Build-and-test-development:
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
needs: Build-and-ng-test
|
needs: Build-and-ng-test
|
||||||
@@ -182,9 +204,20 @@ jobs:
|
|||||||
sasjs request services/admin/makedata -t server-ci -d deploy/makedata.json -o ./makedata_out.json || true
|
sasjs request services/admin/makedata -t server-ci -d deploy/makedata.json -o ./makedata_out.json || true
|
||||||
|
|
||||||
- name: Prepare and run frontend and cypress
|
- name: Prepare and run frontend and cypress
|
||||||
|
# One job, one SASjs server, one ng serve, one Chrome: four of each on one
|
||||||
|
# runner starved the app under test and every failure in that run was a
|
||||||
|
# timeout, not an assertion.
|
||||||
timeout-minutes: 50
|
timeout-minutes: 50
|
||||||
run: |
|
run: |
|
||||||
cd ./client
|
cd ./client
|
||||||
|
# The spec list lives in one place, and an empty list is a broken
|
||||||
|
# workflow rather than a green run with no tests.
|
||||||
|
SPECS="$(bash ../.gitea/scripts/cypress-specs.sh all)"
|
||||||
|
if [ -z "$SPECS" ]; then
|
||||||
|
echo "the spec list resolved to nothing"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "specs: $SPECS"
|
||||||
mv ./cypress.env.example.json ./cypress.env.json
|
mv ./cypress.env.example.json ./cypress.env.json
|
||||||
replace-in-files --regex='"username".*' --replacement='"username":"'${{ secrets.CYPRESS_USERNAME_SASJS }}'",' ./cypress.env.json
|
replace-in-files --regex='"username".*' --replacement='"username":"'${{ secrets.CYPRESS_USERNAME_SASJS }}'",' ./cypress.env.json
|
||||||
replace-in-files --regex='"password".*' --replacement='"password":"'${{ secrets.CYPRESS_PWD_SASJS }}'" ' ./cypress.env.json
|
replace-in-files --regex='"password".*' --replacement='"password":"'${{ secrets.CYPRESS_PWD_SASJS }}'" ' ./cypress.env.json
|
||||||
@@ -199,16 +232,22 @@ jobs:
|
|||||||
# Start frontend and run cypress
|
# Start frontend and run cypress
|
||||||
# timeout 2700: SIGTERM after 45 min so Cypress can flush video/screenshots
|
# timeout 2700: SIGTERM after 45 min so Cypress can flush video/screenshots
|
||||||
# before the outer timeout-minutes hard-kills the step (avoids silent multi-hour hangs)
|
# before the outer timeout-minutes hard-kills the step (avoids silent multi-hour hangs)
|
||||||
npx ng serve --host 0.0.0.0 --port 4200 & npx wait-on http://localhost:4200 && timeout 2700 npx cypress run --browser chrome --spec "cypress/e2e/csv-limited.cy.ts,cypress/e2e/hook-programs.cy.ts,cypress/e2e/liveness.cy.ts,cypress/e2e/editor.cy.ts,cypress/e2e/excel-multi-load.cy.ts,cypress/e2e/excel.cy.ts,cypress/e2e/csv.cy.ts,cypress/e2e/filtering.cy.ts,cypress/e2e/filter-panel.cy.ts,cypress/e2e/licensing.cy.ts,cypress/e2e/viewer-labels.cy.ts,cypress/e2e/full-table-search.cy.ts,cypress/e2e/viewbox.cy.ts,cypress/e2e/stage.cy.ts,cypress/e2e/browser-info.cy.ts,cypress/e2e/ddl-export.cy.ts,cypress/e2e/row-cell-limits.cy.ts,cypress/e2e/derived-variables.cy.ts,cypress/e2e/startupservice-launcher.cy.ts"
|
npx ng serve --host 0.0.0.0 --port 4200 & npx wait-on http://localhost:4200 && timeout 2700 npx cypress run --browser chrome --spec "$SPECS"
|
||||||
|
|
||||||
- name: Zip Cypress videos
|
- name: Zip Cypress videos
|
||||||
if: always()
|
# Only on a failure: a green run has nothing to watch, and zipping and
|
||||||
|
# uploading every run's videos is the part of the video cost that lands
|
||||||
|
# on the clock.
|
||||||
|
if: failure()
|
||||||
run: |
|
run: |
|
||||||
mkdir -p ./client/cypress/videos
|
mkdir -p ./client/cypress/videos
|
||||||
zip -r cypress-videos ./client/cypress/videos
|
zip -r cypress-videos ./client/cypress/videos
|
||||||
|
|
||||||
- name: Add cypress videos artifacts
|
- name: Add cypress videos artifacts
|
||||||
if: always()
|
if: failure()
|
||||||
|
# v3, not v4: this runner reports "GHESNotSupportedError:
|
||||||
|
# @actions/artifact v2.0.0+, upload-artifact@v4+ ... not supported on
|
||||||
|
# GHES", so the current major cannot upload here at all.
|
||||||
uses: actions/upload-artifact@v3
|
uses: actions/upload-artifact@v3
|
||||||
with:
|
with:
|
||||||
name: cypress-videos.zip
|
name: cypress-videos.zip
|
||||||
|
|||||||
@@ -0,0 +1,121 @@
|
|||||||
|
name: Publish code documentation
|
||||||
|
run-name: Publishing typedoc and SAS docs
|
||||||
|
|
||||||
|
# The code documentation sites are published here rather than inline in the
|
||||||
|
# release job, so that they can be republished on their own - after a secret
|
||||||
|
# rotation, or for a release whose artefact upload failed (the release job
|
||||||
|
# cannot simply be re-run: semantic-release refuses once the tag exists).
|
||||||
|
on:
|
||||||
|
workflow_dispatch:
|
||||||
|
|
||||||
|
env:
|
||||||
|
NODE_VERSION: '24.5.0'
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
typedoc:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
# One deploy per site at a time: `surfer deploy` swaps the site atomically,
|
||||||
|
# so two runs racing is not corruption, but it is a wasted build and the
|
||||||
|
# last swap wins.
|
||||||
|
concurrency:
|
||||||
|
group: surfer-webdoc
|
||||||
|
cancel-in-progress: false
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
- uses: actions/setup-node@v4
|
||||||
|
with:
|
||||||
|
node-version: ${{ env.NODE_VERSION }}
|
||||||
|
|
||||||
|
- name: Write .npmrc file
|
||||||
|
run: |
|
||||||
|
printf '%s\n' "$NPMRC" >> client/.npmrc
|
||||||
|
echo "legacy-peer-deps=true" >> client/.npmrc
|
||||||
|
shell: bash
|
||||||
|
env:
|
||||||
|
NPMRC: ${{ secrets.NPMRC }}
|
||||||
|
|
||||||
|
- name: Install dependencies
|
||||||
|
run: |
|
||||||
|
cd client
|
||||||
|
# Decrypt and Install sheet
|
||||||
|
echo "$SHEET_PWD" | gpg --batch --yes --passphrase-fd 0 ./libraries/sheet-crypto.tgz.gpg
|
||||||
|
npm ci
|
||||||
|
shell: bash
|
||||||
|
env:
|
||||||
|
SHEET_PWD: ${{ secrets.SHEET_PWD }}
|
||||||
|
|
||||||
|
- name: Build typedoc
|
||||||
|
run: |
|
||||||
|
cd client
|
||||||
|
npm run compodoc:build
|
||||||
|
|
||||||
|
# Deploy tooling is pinned to an exact version: an unpinned `latest` would
|
||||||
|
# let a breaking or compromised upstream release publish straight to the
|
||||||
|
# live site. `deploy` (v7) replaces the site atomically, and it takes the
|
||||||
|
# Cloudron username plus an app password - the v6 `put --token` flow this
|
||||||
|
# replaces is gone.
|
||||||
|
#
|
||||||
|
# webdoc is a NAMED site on the app whose default site is the marketing
|
||||||
|
# site, so the name and its alias domain are both passed. Surfer picks
|
||||||
|
# the target only from `--site-name` (it travels as the Surfer-Deployment
|
||||||
|
# header); without it `deploy` replaces the app's DEFAULT site, which is
|
||||||
|
# what serves datacontroller.io - so an unnamed deploy published the
|
||||||
|
# Typedoc over the top of the marketing site instead of to webdoc.
|
||||||
|
- name: Deploy typedoc
|
||||||
|
env:
|
||||||
|
SURFER_PASSWORD: ${{ secrets.WEBDOC_DATACONTROLLER_IO }}
|
||||||
|
REF_NAME: ${{ gitea.ref_name }}
|
||||||
|
COMMIT_SHA: ${{ gitea.sha }}
|
||||||
|
run: |
|
||||||
|
set -euo pipefail
|
||||||
|
npx --yes @cloudron/surfer@7.2.0 deploy client/documentation \
|
||||||
|
--server webdoc.datacontroller.io \
|
||||||
|
--site-name webdoc \
|
||||||
|
--domain webdoc.datacontroller.io \
|
||||||
|
--username allan \
|
||||||
|
--password "$SURFER_PASSWORD" \
|
||||||
|
--message "Typedoc from ${REF_NAME} @ ${COMMIT_SHA:0:8}"
|
||||||
|
|
||||||
|
sas-docs:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
concurrency:
|
||||||
|
group: surfer-code-datacontroller
|
||||||
|
cancel-in-progress: false
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
- uses: actions/setup-node@v4
|
||||||
|
with:
|
||||||
|
node-version: ${{ env.NODE_VERSION }}
|
||||||
|
|
||||||
|
- name: Install dependencies
|
||||||
|
run: |
|
||||||
|
# `sasjs doc` shells out to doxygen
|
||||||
|
apt-get update
|
||||||
|
apt-get install -y doxygen
|
||||||
|
cd sas
|
||||||
|
npm ci
|
||||||
|
CLI_VERSION=$(node -p "require('./package.json').dependencies['@sasjs/cli']")
|
||||||
|
npm i -g "@sasjs/cli@${CLI_VERSION}"
|
||||||
|
|
||||||
|
- name: Build SAS docs
|
||||||
|
run: |
|
||||||
|
cd sas
|
||||||
|
sasjs doc
|
||||||
|
|
||||||
|
# No `--site-name` here on purpose: code.datacontroller.io is its own app
|
||||||
|
# and the SAS docs are its DEFAULT site, so naming one would create a
|
||||||
|
# separate site and leave the served copy stale. Do not copy the named
|
||||||
|
# form from the typedoc job above - that app's default site is the
|
||||||
|
# marketing site, which is why it must not be the target.
|
||||||
|
- name: Deploy SAS docs
|
||||||
|
env:
|
||||||
|
SURFER_PASSWORD: ${{ secrets.CODE_DATACONTROLLER_IO }}
|
||||||
|
REF_NAME: ${{ gitea.ref_name }}
|
||||||
|
COMMIT_SHA: ${{ gitea.sha }}
|
||||||
|
run: |
|
||||||
|
set -euo pipefail
|
||||||
|
npx --yes @cloudron/surfer@7.2.0 deploy sas/sasjsbuild/sasdocs \
|
||||||
|
--server code.datacontroller.io \
|
||||||
|
--username allan \
|
||||||
|
--password "$SURFER_PASSWORD" \
|
||||||
|
--message "SAS docs from ${REF_NAME} @ ${COMMIT_SHA:0:8}"
|
||||||
@@ -0,0 +1,209 @@
|
|||||||
|
name: Release assets
|
||||||
|
run-name: Building and uploading the release assets
|
||||||
|
|
||||||
|
# Builds the downloadable artefacts for an existing release and uploads them.
|
||||||
|
#
|
||||||
|
# The release job cuts the tag before it builds, so a failure anywhere between
|
||||||
|
# the semantic-release step and the asset upload leaves a published release with
|
||||||
|
# no files - and that job cannot simply be re-run, because semantic-release
|
||||||
|
# refuses once the tag exists. This workflow rebuilds the artefacts from the tag
|
||||||
|
# and attaches them, so the release can be completed without cutting a new one.
|
||||||
|
on:
|
||||||
|
workflow_dispatch:
|
||||||
|
inputs:
|
||||||
|
tag:
|
||||||
|
description: 'Release tag to build and upload (blank = the latest release)'
|
||||||
|
required: false
|
||||||
|
default: ''
|
||||||
|
|
||||||
|
env:
|
||||||
|
NODE_VERSION: '24.5.0'
|
||||||
|
|
||||||
|
permissions:
|
||||||
|
contents: write
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
assets:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
# Two runs uploading the same asset names would race the delete-then-upload
|
||||||
|
# below.
|
||||||
|
concurrency:
|
||||||
|
group: release-assets
|
||||||
|
cancel-in-progress: false
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
with:
|
||||||
|
fetch-depth: 0
|
||||||
|
- uses: actions/setup-node@v4
|
||||||
|
with:
|
||||||
|
node-version: ${{ env.NODE_VERSION }}
|
||||||
|
|
||||||
|
- name: Write .npmrc file
|
||||||
|
run: |
|
||||||
|
printf '%s\n' "$NPMRC" >> client/.npmrc
|
||||||
|
echo "legacy-peer-deps=true" >> client/.npmrc
|
||||||
|
shell: bash
|
||||||
|
env:
|
||||||
|
NPMRC: ${{ secrets.NPMRC }}
|
||||||
|
|
||||||
|
- name: Install system packages
|
||||||
|
# jq parses the release JSON below, so it is installed before the tag
|
||||||
|
# resolve.
|
||||||
|
run: |
|
||||||
|
apt-get update
|
||||||
|
apt-get install -y zip jq
|
||||||
|
|
||||||
|
- name: Resolve the release tag
|
||||||
|
env:
|
||||||
|
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
||||||
|
REQUESTED_TAG: ${{ inputs.tag }}
|
||||||
|
run: |
|
||||||
|
set -euo pipefail
|
||||||
|
AUTH_HEADER="Authorization: token $GITEA_TOKEN"
|
||||||
|
BASE="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"
|
||||||
|
|
||||||
|
if [ -n "${REQUESTED_TAG}" ]; then
|
||||||
|
TAG="${REQUESTED_TAG}"
|
||||||
|
# Fail here rather than after a ten-minute build if the tag is not
|
||||||
|
# cut yet or has no release.
|
||||||
|
curl --fail-with-body -sS -H "$AUTH_HEADER" "$BASE/releases/tags/${TAG}" > /dev/null
|
||||||
|
else
|
||||||
|
TAG=$(curl --fail-with-body -sS -H "$AUTH_HEADER" "$BASE/releases/latest" | jq -r '.tag_name')
|
||||||
|
fi
|
||||||
|
echo "Building assets for ${TAG}"
|
||||||
|
echo "TAG=${TAG}" >> "$GITHUB_ENV"
|
||||||
|
|
||||||
|
- name: Check out the tag
|
||||||
|
run: |
|
||||||
|
set -euo pipefail
|
||||||
|
git fetch --force --tags origin "+refs/tags/${TAG}:refs/tags/${TAG}"
|
||||||
|
git checkout --force "refs/tags/${TAG}"
|
||||||
|
|
||||||
|
- name: Install the SASjs CLI the tag pins
|
||||||
|
run: |
|
||||||
|
cd sas
|
||||||
|
CLI_VERSION=$(node -p "require('./package.json').dependencies['@sasjs/cli']")
|
||||||
|
npm i -g "@sasjs/cli@${CLI_VERSION}"
|
||||||
|
|
||||||
|
- name: Frontend build
|
||||||
|
run: |
|
||||||
|
cd client
|
||||||
|
# Decrypt and Install sheet
|
||||||
|
echo "$SHEET_PWD" | gpg --batch --yes --passphrase-fd 0 ./libraries/sheet-crypto.tgz.gpg
|
||||||
|
npm ci
|
||||||
|
npm i webpack
|
||||||
|
npm run build
|
||||||
|
shell: bash
|
||||||
|
env:
|
||||||
|
SHEET_PWD: ${{ secrets.SHEET_PWD }}
|
||||||
|
|
||||||
|
- name: Build SAS9 EBI Release
|
||||||
|
run: |
|
||||||
|
cd sas
|
||||||
|
npm i
|
||||||
|
sasjs c -t sas9
|
||||||
|
rm -rf sasjsbuild/tests
|
||||||
|
sasjs b -t sas9
|
||||||
|
cp sasjsbuild/mysas9deploy.sas ./demostream_sas9.sas
|
||||||
|
# remove streamed component and rebuild SAS 9 services
|
||||||
|
rm -rf sasjsbuild/services/web9
|
||||||
|
rm sasjsbuild/services/clickme.sas
|
||||||
|
sasjs b -t sas9
|
||||||
|
cp sasjsbuild/mysas9deploy.sas ./sas9.sas
|
||||||
|
|
||||||
|
- name: Build SASjs Server Release
|
||||||
|
run: |
|
||||||
|
cd sas
|
||||||
|
cp sasjs/utils/favicon.ico ../client/dist/favicon.ico
|
||||||
|
sasjs c -t server
|
||||||
|
rm -rf sasjsbuild/tests
|
||||||
|
server_apploc="/Public/app/dc"
|
||||||
|
sed -i "s|apploc=\"[^\"]*\"|apploc=\"${server_apploc}\"|g" sasjsbuild/services/web/index.html
|
||||||
|
sasjs b -t server
|
||||||
|
cp sasjsbuild/server.json.zip ./sasjs_server.json.zip
|
||||||
|
|
||||||
|
- name: Build Viya Release
|
||||||
|
run: |
|
||||||
|
cd sas
|
||||||
|
sasjs c -t viya
|
||||||
|
rm -rf sasjsbuild/tests
|
||||||
|
sed -i -e 's/servertype="SASJS"/servertype="SASVIYA"/g' sasjsbuild/services/DC.html
|
||||||
|
sasjs b -t viya
|
||||||
|
cp sasjsbuild/viya.sas ./viya.sas
|
||||||
|
# compile Viya Full deploy (without web)
|
||||||
|
rm -rf sasjsbuild/services/web
|
||||||
|
rm sasjsbuild/services/DC.html
|
||||||
|
sasjs b -t viya
|
||||||
|
cp sasjsbuild/viya.sas ./viya_noweb.sas
|
||||||
|
cp sasjsbuild/viya.json ./viya_noweb.json
|
||||||
|
|
||||||
|
- name: Zip Frontend (including viya.json for full viya deploy)
|
||||||
|
run: |
|
||||||
|
cd sas
|
||||||
|
cp sasjsbuild/viya.json ../client/dist/viya.json
|
||||||
|
# Zip from *inside* dist so the archive holds the frontend files at its
|
||||||
|
# root rather than under a client/dist/ prefix (see #147).
|
||||||
|
cd ../client/dist
|
||||||
|
zip -r ../../frontend.zip .
|
||||||
|
|
||||||
|
- name: Upload assets to the release
|
||||||
|
env:
|
||||||
|
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
||||||
|
run: |
|
||||||
|
set -euo pipefail
|
||||||
|
AUTH_HEADER="Authorization: token $GITEA_TOKEN"
|
||||||
|
BASE="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"
|
||||||
|
|
||||||
|
RELEASE_JSON=$(curl --fail-with-body -sS -H "$AUTH_HEADER" "$BASE/releases/tags/${TAG}")
|
||||||
|
RELEASE_ID=$(echo "$RELEASE_JSON" | jq -r '.id')
|
||||||
|
RELEASE_BODY=$(echo "$RELEASE_JSON" | jq -r '.body')
|
||||||
|
|
||||||
|
ASSETS=(frontend.zip
|
||||||
|
sas/demostream_sas9.sas
|
||||||
|
sas/viya.sas
|
||||||
|
sas/sasjs_server.json.zip
|
||||||
|
sas/sas9.sas
|
||||||
|
sas/viya_noweb.sas
|
||||||
|
sas/viya_noweb.json)
|
||||||
|
|
||||||
|
: > SHA256SUMS
|
||||||
|
for f in "${ASSETS[@]}"; do
|
||||||
|
sum=$(sha256sum "$f" | cut -d' ' -f1)
|
||||||
|
printf '%s %s\n' "$sum" "${f##*/}" >> SHA256SUMS
|
||||||
|
done
|
||||||
|
cat SHA256SUMS
|
||||||
|
|
||||||
|
# Replacing the assets means removing the ones this workflow uploads,
|
||||||
|
# and a second run must not append the verification section twice. An
|
||||||
|
# asset the release carries from another source is left alone.
|
||||||
|
OWNED=$(printf '%s\n' "${ASSETS[@]##*/}" SHA256SUMS)
|
||||||
|
echo "$RELEASE_JSON" | jq -r '.assets[]? | "\(.id) \(.name)"' |
|
||||||
|
while read -r asset_id asset_name; do
|
||||||
|
if printf '%s\n' "$OWNED" | grep -qxF -- "$asset_name"; then
|
||||||
|
echo "Removing previous ${asset_name}"
|
||||||
|
curl --fail-with-body -sS -X DELETE -H "$AUTH_HEADER" "$BASE/releases/${RELEASE_ID}/assets/${asset_id}"
|
||||||
|
else
|
||||||
|
echo "Leaving ${asset_name} in place: this workflow did not upload it"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
if echo "$RELEASE_BODY" | grep -q "## Verifying the download"; then
|
||||||
|
echo "Release body already carries the verification section"
|
||||||
|
else
|
||||||
|
VERIFY_SECTION=$(./.gitea/scripts/verify-section.sh)
|
||||||
|
NEW_BODY=$(jq -nr --arg body "$RELEASE_BODY" --arg verify "$VERIFY_SECTION" \
|
||||||
|
'$body + "\n" + $verify + "\n\nFor installation instructions, please visit https://docs.datacontroller.io/"')
|
||||||
|
curl --fail-with-body -sS -X PATCH \
|
||||||
|
-H "$AUTH_HEADER" \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
--data "$(jq -n --arg body "$NEW_BODY" '{draft:false, body:$body}')" \
|
||||||
|
"$BASE/releases/$RELEASE_ID"
|
||||||
|
fi
|
||||||
|
|
||||||
|
URL="$BASE/releases/${RELEASE_ID}/assets"
|
||||||
|
for f in "${ASSETS[@]}"; do
|
||||||
|
echo "Uploading $f ..."
|
||||||
|
curl --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@$f"
|
||||||
|
done
|
||||||
|
echo "Uploading SHA256SUMS ..."
|
||||||
|
curl --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@SHA256SUMS"
|
||||||
@@ -281,6 +281,11 @@ jobs:
|
|||||||
|
|
||||||
semantic-release
|
semantic-release
|
||||||
|
|
||||||
|
# The tag this job cuts, for the steps that publish against it: a
|
||||||
|
# `releases/latest` lookup would pick whichever release landed last
|
||||||
|
# if two overlap.
|
||||||
|
echo "RELEASE_TAG=v$(node -p "require('./package.json').version")" >> "$GITHUB_ENV"
|
||||||
|
|
||||||
- name: Frontend Build
|
- name: Frontend Build
|
||||||
description: Must be created AFTER the release as the version (git tag) is used in the interface
|
description: Must be created AFTER the release as the version (git tag) is used in the interface
|
||||||
run: |
|
run: |
|
||||||
@@ -341,19 +346,6 @@ jobs:
|
|||||||
cd ../client/dist
|
cd ../client/dist
|
||||||
zip -r ../../frontend.zip .
|
zip -r ../../frontend.zip .
|
||||||
|
|
||||||
- name: Release Typedoc
|
|
||||||
run: |
|
|
||||||
cd client
|
|
||||||
npm -g install cloudron-surfer
|
|
||||||
npm run compodoc:build
|
|
||||||
surfer put --token ${{ secrets.TSDOC_TOKEN }} --server webdoc.datacontroller.io documentation/* /
|
|
||||||
|
|
||||||
- name: Release code.datacontroller.io
|
|
||||||
run: |
|
|
||||||
cd sas
|
|
||||||
sasjs doc
|
|
||||||
surfer put --token ${{ secrets.CODE_DATACONTROLLER_IO }} --server code.datacontroller.io sasjsbuild/sasdocs/* /
|
|
||||||
|
|
||||||
- name: Upload assets to release
|
- name: Upload assets to release
|
||||||
env:
|
env:
|
||||||
# Use the same ephemeral per-job Gitea Actions token that the
|
# Use the same ephemeral per-job Gitea Actions token that the
|
||||||
@@ -368,7 +360,7 @@ jobs:
|
|||||||
AUTH_HEADER="Authorization: token ${GITEA_TOKEN}"
|
AUTH_HEADER="Authorization: token ${GITEA_TOKEN}"
|
||||||
BASE="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"
|
BASE="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"
|
||||||
|
|
||||||
RELEASE_JSON=$(curl -k --fail-with-body -sS -H "$AUTH_HEADER" "$BASE/releases/latest")
|
RELEASE_JSON=$(curl --fail-with-body -sS -H "$AUTH_HEADER" "$BASE/releases/tags/${RELEASE_TAG}")
|
||||||
RELEASE_ID=$(echo "$RELEASE_JSON" | jq -r '.id')
|
RELEASE_ID=$(echo "$RELEASE_JSON" | jq -r '.id')
|
||||||
RELEASE_BODY=$(echo "$RELEASE_JSON" | jq -r '.body')
|
RELEASE_BODY=$(echo "$RELEASE_JSON" | jq -r '.body')
|
||||||
|
|
||||||
@@ -402,7 +394,7 @@ jobs:
|
|||||||
VERIFY_SECTION=$(./.gitea/scripts/verify-section.sh)
|
VERIFY_SECTION=$(./.gitea/scripts/verify-section.sh)
|
||||||
NEW_BODY=$(jq -nr --arg body "$RELEASE_BODY" --arg verify "$VERIFY_SECTION" \
|
NEW_BODY=$(jq -nr --arg body "$RELEASE_BODY" --arg verify "$VERIFY_SECTION" \
|
||||||
'$body + "\n" + $verify + "\n\nFor installation instructions, please visit https://docs.datacontroller.io/"')
|
'$body + "\n" + $verify + "\n\nFor installation instructions, please visit https://docs.datacontroller.io/"')
|
||||||
curl -k --fail-with-body -sS -X PATCH \
|
curl --fail-with-body -sS -X PATCH \
|
||||||
-H "$AUTH_HEADER" \
|
-H "$AUTH_HEADER" \
|
||||||
-H 'Content-Type: application/json' \
|
-H 'Content-Type: application/json' \
|
||||||
--data "$(jq -n --arg body "$NEW_BODY" '{draft:false, body:$body}')" \
|
--data "$(jq -n --arg body "$NEW_BODY" '{draft:false, body:$body}')" \
|
||||||
@@ -412,10 +404,37 @@ jobs:
|
|||||||
URL="$BASE/releases/$RELEASE_ID/assets"
|
URL="$BASE/releases/$RELEASE_ID/assets"
|
||||||
for f in "${ASSETS[@]}"; do
|
for f in "${ASSETS[@]}"; do
|
||||||
echo "Uploading $f ..."
|
echo "Uploading $f ..."
|
||||||
curl -k --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@$f"
|
curl --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@$f"
|
||||||
done
|
done
|
||||||
echo "Uploading SHA256SUMS ..."
|
echo "Uploading SHA256SUMS ..."
|
||||||
curl -k --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@SHA256SUMS"
|
curl --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@SHA256SUMS"
|
||||||
|
|
||||||
|
- name: Publish the code documentation
|
||||||
|
# Typedoc (webdoc) and the SAS docs (code) are published by docs.yaml,
|
||||||
|
# not inline here. Two reasons: the v6 `surfer put --token` flow is gone
|
||||||
|
# in Surfer 7, which takes a Cloudron username plus an app password; and
|
||||||
|
# a dispatch means a stale or failed docs deploy can be republished
|
||||||
|
# without re-running this job, which semantic-release refuses once the
|
||||||
|
# tag exists.
|
||||||
|
#
|
||||||
|
# This runs AFTER the release assets are uploaded: the documentation is
|
||||||
|
# a side channel, and a docs problem must not cost a cut release its
|
||||||
|
# artefacts.
|
||||||
|
env:
|
||||||
|
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
||||||
|
run: |
|
||||||
|
set -euo pipefail
|
||||||
|
AUTH_HEADER="Authorization: token $GITEA_TOKEN"
|
||||||
|
BASE="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"
|
||||||
|
|
||||||
|
TAG="${RELEASE_TAG}"
|
||||||
|
echo "Dispatching code documentation for ${TAG}"
|
||||||
|
|
||||||
|
curl --fail-with-body -sS -X POST \
|
||||||
|
-H "$AUTH_HEADER" \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
--data "$(jq -n --arg ref "refs/tags/$TAG" '{ref: $ref}')" \
|
||||||
|
"$BASE/actions/workflows/docs.yaml/dispatches"
|
||||||
|
|
||||||
- name: Publish the demo site for this release
|
- name: Publish the demo site for this release
|
||||||
# The tag is created by this job's semantic-release step, so it carries
|
# The tag is created by this job's semantic-release step, so it carries
|
||||||
@@ -430,12 +449,12 @@ jobs:
|
|||||||
AUTH_HEADER="Authorization: token ${GITEA_TOKEN}"
|
AUTH_HEADER="Authorization: token ${GITEA_TOKEN}"
|
||||||
BASE="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"
|
BASE="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"
|
||||||
|
|
||||||
TAG=$(curl -k --fail-with-body -sS -H "$AUTH_HEADER" "$BASE/releases/latest" | jq -r '.tag_name')
|
TAG="${RELEASE_TAG}"
|
||||||
echo "Dispatching Sites for ${TAG}"
|
echo "Dispatching Sites for ${TAG}"
|
||||||
|
|
||||||
# the dispatch endpoint resolves a bare name as a branch; a tag needs
|
# the dispatch endpoint resolves a bare name as a branch; a tag needs
|
||||||
# the refs/tags/ prefix
|
# the refs/tags/ prefix
|
||||||
curl -k --fail-with-body -sS -X POST \
|
curl --fail-with-body -sS -X POST \
|
||||||
-H "$AUTH_HEADER" \
|
-H "$AUTH_HEADER" \
|
||||||
-H 'Content-Type: application/json' \
|
-H 'Content-Type: application/json' \
|
||||||
--data "$(jq -n --arg ref "refs/tags/$TAG" '{ref: $ref}')" \
|
--data "$(jq -n --arg ref "refs/tags/$TAG" '{ref: $ref}')" \
|
||||||
|
|||||||
@@ -43,7 +43,7 @@ Data Controller must run entirely locally (offline / on-prem, no internet access
|
|||||||
|
|
||||||
Agent-related content lives in `.agents/`: technical/agent-facing documentation goes in `.agents/docs/` (not `docs/`), and skills in `.agents/skills/`. When writing explanatory or technical docs about the codebase, put them in `.agents/docs/`.
|
Agent-related content lives in `.agents/`: technical/agent-facing documentation goes in `.agents/docs/` (not `docs/`), and skills in `.agents/skills/`. When writing explanatory or technical docs about the codebase, put them in `.agents/docs/`.
|
||||||
|
|
||||||
Skills installed via `npx skills add` are tracked in `skills-lock.json` and live under `.agents/skills/`. Repo-local custom skills (e.g. `dc-sas` for Data Controller SAS development, `handsontable`, `hyperformula`) are kept here too. Avoid editing npx-managed skill files directly — use `npx skills update` instead.
|
Skills installed via `npx skills add` are tracked in `skills-lock.json` and live under `.agents/skills/`. Repo-local custom skills (e.g. `dc-sas` for Data Controller SAS development, `dc-estate-testing` for estate deploys and test runs, `handsontable`, `hyperformula`) are kept here too. Avoid editing npx-managed skill files directly — use `npx skills update` instead.
|
||||||
|
|
||||||
## Code comments and test names
|
## Code comments and test names
|
||||||
|
|
||||||
|
|||||||
@@ -1,3 +1,55 @@
|
|||||||
|
# [7.21.0](https://git.datacontroller.io/dc/dc/compare/v7.20.0...v7.21.0) (2026-10-29)
|
||||||
|
|
||||||
|
|
||||||
|
### Bug Fixes
|
||||||
|
|
||||||
|
* **client:** parse word-date values from VA parameters ([405b28d](https://git.datacontroller.io/dc/dc/commit/405b28dcccfe5994521bf63f0270d4877f544294))
|
||||||
|
* **client:** parse word-date values from VA parameters ([ce5b1aa](https://git.datacontroller.io/dc/dc/commit/ce5b1aa96175d0fd1f048fea6d127cbc743804f2))
|
||||||
|
* **makedata:** name the sasjs server group in the estate seed ([eeffa92](https://git.datacontroller.io/dc/dc/commit/eeffa92f9e973b323171bbefe2ef1cbc27e3aee7))
|
||||||
|
|
||||||
|
|
||||||
|
### Features
|
||||||
|
|
||||||
|
* **derived:** apply the core field scan in the MPE_DERIVED hook ([a4a0f9c](https://git.datacontroller.io/dc/dc/commit/a4a0f9cb2f7c0b6252405b71df64aae912d1b075))
|
||||||
|
|
||||||
|
# [7.20.0](https://git.datacontroller.io/dc/dc/compare/v7.19.0...v7.20.0) (2026-10-28)
|
||||||
|
|
||||||
|
|
||||||
|
### Bug Fixes
|
||||||
|
|
||||||
|
* **ci:** install doxygen for the SAS docs job ([f58925b](https://git.datacontroller.io/dc/dc/commit/f58925b29956f1f753f0a367fdcd0afe7f3f51a0))
|
||||||
|
* **ci:** keep foreign release assets, install jq first, pin the CLI ([27686d3](https://git.datacontroller.io/dc/dc/commit/27686d3aad5e34ace59ca1b735a17cca1a4025e1))
|
||||||
|
* **ci:** publish against the tag this job cuts, not releases/latest ([8888734](https://git.datacontroller.io/dc/dc/commit/888873421c3aeaf312d5ec45e3c805317c66976a))
|
||||||
|
* **ci:** publish docs after the assets, with the webdoc app password ([dd482c4](https://git.datacontroller.io/dc/dc/commit/dd482c4069202a0c6b9983129c19efc09bd160a7))
|
||||||
|
* **ci:** publish the code documentation with surfer 7 and app passwords ([25ffa71](https://git.datacontroller.io/dc/dc/commit/25ffa71adddba5e334f4a8c166f284d1be498fcd))
|
||||||
|
* **ci:** publish the typedoc to the webdoc site, not the app default ([c5808a1](https://git.datacontroller.io/dc/dc/commit/c5808a16eba0d29562fcc76f13ac4a33ab8da0b7))
|
||||||
|
* **ci:** rebuild and upload release assets on demand ([6486c80](https://git.datacontroller.io/dc/dc/commit/6486c80ed4f368bbf116c0ecdecc0e5e5f02075f))
|
||||||
|
* **ci:** use the lockfile and the pinned CLI in the docs job ([af1813f](https://git.datacontroller.io/dc/dc/commit/af1813f829861df41c89b512c824e8d3ebac7a4b))
|
||||||
|
* **ci:** verify the forge certificate on the release API calls ([cf7b21a](https://git.datacontroller.io/dc/dc/commit/cf7b21a8b61ec3bf7ebe21968ccf13ade264bc64))
|
||||||
|
* **cypress:** hold download out too, and retry once in CI ([c97e886](https://git.datacontroller.io/dc/dc/commit/c97e8866db9ee404b2118d24baf53fc7058debef)), closes [#395](https://git.datacontroller.io/dc/dc/issues/395) [#398](https://git.datacontroller.io/dc/dc/issues/398)
|
||||||
|
* **cypress:** register the deleteFolder task, restore the spec order, hold metanav out ([c803773](https://git.datacontroller.io/dc/dc/commit/c803773d3c36476d8263f01fc3b8344aa3503d44)), closes [#395](https://git.datacontroller.io/dc/dc/issues/395)
|
||||||
|
* **deploy:** do not offer the deploy wizard inside a report ([80e8911](https://git.datacontroller.io/dc/dc/commit/80e8911af8857fa2c417f028cd59b64a87233b9e))
|
||||||
|
* **deploy:** keep the app query when opening the standalone URL ([e51953c](https://git.datacontroller.io/dc/dc/commit/e51953ca8a6734e1649bad68cf4ad2057d677334))
|
||||||
|
* **deploy:** one definition of the standalone URL, and test the guards ([459ff2d](https://git.datacontroller.io/dc/dc/commit/459ff2da371375e996839d69deb40f681c356f1d)), closes [#TCS](https://git.datacontroller.io/dc/dc/issues/TCS)
|
||||||
|
* **deploy:** strip the embed parameters by name, and scope the timezone runs to the VA specs ([f7e7538](https://git.datacontroller.io/dc/dc/commit/f7e753863b61d8545bd0777d80267661b3c58670))
|
||||||
|
* **embed:** build quarter ends from the calendar, not minus 24 hours ([7e3045a](https://git.datacontroller.io/dc/dc/commit/7e3045a21b67053b76a81bfe836e31571ccbffbf))
|
||||||
|
* **embed:** defer the column check until metadata loads, report AND-ed pairs ([2623a3f](https://git.datacontroller.io/dc/dc/commit/2623a3fb4ba3bf19c00636b1717b53d5a2d54f3a))
|
||||||
|
* **embed:** log a parse-time mapping problem once ([0011f7e](https://git.datacontroller.io/dc/dc/commit/0011f7e933d85be8e33debe3a6ecb9000edaabe3))
|
||||||
|
* **embed:** make the va_parmap diagnostics say what they mean ([541a61b](https://git.datacontroller.io/dc/dc/commit/541a61ba08bc7256a911c50de7078859cc46c84e))
|
||||||
|
* **embed:** match a mapping on the parameter label only, and warn only when the values differ ([e9216b0](https://git.datacontroller.io/dc/dc/commit/e9216b08956d5599662259a2b05cc1524c458df5))
|
||||||
|
* **embed:** never build a clause from a value that did not convert, and say so in the banner ([bd3c220](https://git.datacontroller.io/dc/dc/commit/bd3c220af2678a2a3f8394fe3084ee03c23d0b15))
|
||||||
|
* **embed:** one parameter maps one column, and report typos on both sides ([f83dc35](https://git.datacontroller.io/dc/dc/commit/f83dc359e258f5d1cb9fc49f0eaab9748995d823))
|
||||||
|
* **embed:** record the parameter label only, so a name-keyed va_parmap is reported ([aa91daf](https://git.datacontroller.io/dc/dc/commit/aa91daf9f78c00cea486d0f885f8e634366bd1a0))
|
||||||
|
* **embed:** resolve transform values as local dates and give ranges whole days ([10157c1](https://git.datacontroller.io/dc/dc/commit/10157c11595685efdc206fe7b16a5cafa098f26d))
|
||||||
|
* **embed:** warn when a va_parmap entry is ignored ([411f6c3](https://git.datacontroller.io/dc/dc/commit/411f6c3354b7fbfec11fef3b174ae1f8066c92ba))
|
||||||
|
* **makedata:** pair the %else of the keepme block so it parses in open code ([c0ea810](https://git.datacontroller.io/dc/dc/commit/c0ea810766ebfda5ae5358ab14146fbcc07fcc8e))
|
||||||
|
|
||||||
|
|
||||||
|
### Features
|
||||||
|
|
||||||
|
* **embed:** map VA parameters to DC columns from the URL ([15a1f34](https://git.datacontroller.io/dc/dc/commit/15a1f34d5263503cac83b4fe4e52482777dc2f1f))
|
||||||
|
* **embed:** resolve va_parmap values through named date anchors ([88010c3](https://git.datacontroller.io/dc/dc/commit/88010c3980c2ea1c39bbb024ed32d63e495e61f0))
|
||||||
|
|
||||||
# [7.19.0](https://git.datacontroller.io/dc/dc/compare/v7.18.0...v7.19.0) (2026-10-28)
|
# [7.19.0](https://git.datacontroller.io/dc/dc/compare/v7.18.0...v7.19.0) (2026-10-28)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -8,7 +8,7 @@ This file is the shared glossary and orientation for the repo. When your output
|
|||||||
|
|
||||||
- `client/` - the Angular frontend (TypeScript). Uses [Handsontable](.agents/skills/handsontable/SKILL.md) for the editable grid and [HyperFormula](.agents/skills/hyperformula/SKILL.md) for Excel-formula support. Lint with `npm run lint:check` from `client/`.
|
- `client/` - the Angular frontend (TypeScript). Uses [Handsontable](.agents/skills/handsontable/SKILL.md) for the editable grid and [HyperFormula](.agents/skills/hyperformula/SKILL.md) for Excel-formula support. Lint with `npm run lint:check` from `client/`.
|
||||||
- `sas/` - the SAS backend. Services, macros, hooks and tests live under `sas/sasjs/**`; `sas/sasjsbuild/` is generated output (never hand-edit). Lint `.sas` files with `sasjs lint` from `sas/`.
|
- `sas/` - the SAS backend. Services, macros, hooks and tests live under `sas/sasjs/**`; `sas/sasjsbuild/` is generated output (never hand-edit). Lint `.sas` files with `sasjs lint` from `sas/`.
|
||||||
- `.agents/docs/` - technical deep-dives (see below). `.agents/skills/` - task skills (handsontable, hyperformula, dc-sas for Data Controller SAS development, plus npx-installed sasjs/ sas skills).
|
- `.agents/docs/` - technical deep-dives (see below). `.agents/skills/` - task skills (handsontable, hyperformula, dc-sas for Data Controller SAS development, dc-estate-testing for deploying to and testing on the shared estate, plus npx-installed sasjs/ sas skills).
|
||||||
|
|
||||||
## Roles
|
## Roles
|
||||||
|
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import fs from 'fs'
|
||||||
import { defineConfig } from 'cypress'
|
import { defineConfig } from 'cypress'
|
||||||
|
|
||||||
export default defineConfig({
|
export default defineConfig({
|
||||||
@@ -34,7 +35,28 @@ export default defineConfig({
|
|||||||
|
|
||||||
e2e: {
|
e2e: {
|
||||||
video: true,
|
video: true,
|
||||||
|
// One retry in a CI run, none interactively. `editor.cy.ts` test 28
|
||||||
|
// intermittently fails on "app-loading ... continuously found" - a slow load,
|
||||||
|
// not an assertion - and a single retry costs a few seconds where a red
|
||||||
|
// pipeline costs forty minutes. Failures that are real still fail twice, and
|
||||||
|
// the retried attempt's output is in the run log.
|
||||||
|
retries: { runMode: 1, openMode: 0 },
|
||||||
setupNodeEvents(on, config) {
|
setupNodeEvents(on, config) {
|
||||||
|
// `download.cy.ts` clears the downloads folder before each test through
|
||||||
|
// `cy.task('deleteFolder')`. That task was registered in
|
||||||
|
// cypress/plugins/index.js, which Cypress 10+ never loads unless it is
|
||||||
|
// required from here - so the task had no handler and the spec failed on
|
||||||
|
// its first hook.
|
||||||
|
on('task', {
|
||||||
|
deleteFolder(folderName: string) {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
fs.rm(folderName, { recursive: true, force: true }, (err) =>
|
||||||
|
err ? reject(err) : resolve(null)
|
||||||
|
)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
// Pin the browser locale so locale-formatted cells (intl-date/time/datetime)
|
// Pin the browser locale so locale-formatted cells (intl-date/time/datetime)
|
||||||
// render deterministically regardless of the runner's system locale.
|
// render deterministically regardless of the runner's system locale.
|
||||||
on('before:browser:launch', (browser, launchOptions) => {
|
on('before:browser:launch', (browser, launchOptions) => {
|
||||||
|
|||||||
@@ -17,9 +17,20 @@ context('download files test: ', function () {
|
|||||||
|
|
||||||
this.beforeEach(() => {
|
this.beforeEach(() => {
|
||||||
cy.visit(hostUrl + appLocation)
|
cy.visit(hostUrl + appLocation)
|
||||||
cy.get('input.username').type(username)
|
|
||||||
cy.get('input.password').type(password)
|
// beforeAll already logged in. Typing into the login form unconditionally
|
||||||
cy.get('.login-group button').click()
|
// failed with "the center of this element is hidden from view" whenever the
|
||||||
|
// session was live and the form was not on screen, so only fill it when it
|
||||||
|
// is actually shown.
|
||||||
|
cy.get('body').then(($body) => {
|
||||||
|
const usernameInput = $body.find('input.username')[0]
|
||||||
|
|
||||||
|
if (usernameInput && !Cypress.dom.isHidden(usernameInput)) {
|
||||||
|
cy.get('input.username').type(username)
|
||||||
|
cy.get('input.password').type(password)
|
||||||
|
cy.get('.login-group button').click()
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
visitPage('home')
|
visitPage('home')
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -6,44 +6,90 @@ const libraryToOpenIncludes = Cypress.env(`libraryToOpenIncludes_${serverType}`)
|
|||||||
|
|
||||||
// SAS Visual Analytics data-driven content embed mode (`?embed=va`):
|
// SAS Visual Analytics data-driven content embed mode (`?embed=va`):
|
||||||
// - chrome is hidden (same as embed=true),
|
// - chrome is hidden (same as embed=true),
|
||||||
// - the editor opens in edit mode immediately (not read-only),
|
// - the editor opens READ-ONLY (the user clicks Edit, as in normal use),
|
||||||
// - the only action button is a single Submit below the grid (stubbed in v1),
|
// - the Filter button is hidden - VA filters replace the filter builder,
|
||||||
// - the top button bar and Add Record are hidden.
|
// - the Add Row / Add Record buttons are hidden.
|
||||||
//
|
//
|
||||||
// The detailed PK+label data-merge logic is covered by the VaMessagingService
|
// The detailed PK+label data-merge logic is covered by the VaMessagingService
|
||||||
// unit spec; here we only assert the deterministic UI layout and that pushing a
|
// unit spec; here we assert the deterministic UI layout, that pushing a VA
|
||||||
// VA postMessage does not break the page.
|
// postMessage does not break the page, and (case 3) that a `va_parmap` URL
|
||||||
|
// mapping filters the column it names end to end.
|
||||||
|
/**
|
||||||
|
* Assert on the row count the editor shows. The assertion runs inside a Cy
|
||||||
|
* `should`, so it retries until the grid has loaded: reading the text once would
|
||||||
|
* capture the count from before a filter is applied.
|
||||||
|
*/
|
||||||
|
const parseRows = (text: string): number => {
|
||||||
|
const match = /\(([\d,]+) rows?/.exec(text)
|
||||||
|
return match ? Number(match[1].replace(/,/g, '')) : NaN
|
||||||
|
}
|
||||||
|
|
||||||
|
const expectRows = (check: (rows: number) => void) =>
|
||||||
|
cy.get('h4', { timeout: longerCommandTimeout }).should(($h4) => {
|
||||||
|
check(parseRows($h4.text()))
|
||||||
|
})
|
||||||
|
|
||||||
|
/** Assert the grid shrank, without pinning a fixture-specific number. */
|
||||||
|
const expectFewerRowsThan = (baseline: number) =>
|
||||||
|
expectRows((rows) => {
|
||||||
|
expect(rows, 'filtered row count').to.be.lessThan(baseline)
|
||||||
|
})
|
||||||
|
|
||||||
|
/** Read the unfiltered row count from the app itself, retrying until it loads. */
|
||||||
|
const readRowCount = (): Cypress.Chainable<number> =>
|
||||||
|
expectRows((rows) => {
|
||||||
|
expect(rows, 'row count').to.be.greaterThan(0)
|
||||||
|
})
|
||||||
|
.get('h4')
|
||||||
|
.invoke('text')
|
||||||
|
.then(parseRows)
|
||||||
|
|
||||||
context('embed=va (VA data-driven content) tests: ', function () {
|
context('embed=va (VA data-driven content) tests: ', function () {
|
||||||
|
// How many rows the table has unfiltered. Captured once, from the app itself.
|
||||||
|
let unfilteredRows = 0
|
||||||
|
|
||||||
this.beforeAll(() => {
|
this.beforeAll(() => {
|
||||||
cy.visit(`${hostUrl}/SASLogon/logout`)
|
cy.visit(`${hostUrl}/SASLogon/logout`)
|
||||||
cy.loginAndUpdateValidKey()
|
cy.loginAndUpdateValidKey()
|
||||||
|
// Baseline for the relative row-count assertions: open the table with no VA
|
||||||
|
// message and record what the editor itself reports.
|
||||||
|
cy.visit(`${hostUrl}${appLocation}/#/editor/DC_JSLIB.MPE_X_TEST`, {
|
||||||
|
timeout: 60000
|
||||||
|
})
|
||||||
|
readRowCount().then((count) => {
|
||||||
|
unfilteredRows = count
|
||||||
|
})
|
||||||
})
|
})
|
||||||
|
|
||||||
this.beforeEach(() => {
|
this.beforeEach(() => {
|
||||||
cy.visit(hostUrl + appLocation)
|
cy.visit(hostUrl + appLocation)
|
||||||
|
// A cy.visit that differs from the current URL only in the hash is a
|
||||||
|
// same-document navigation, so the app is not re-bootstrapped. Reload so
|
||||||
|
// each test starts from a fresh document (the previous test may have left
|
||||||
|
// ?embed=va in the URL, which the app reads once at init).
|
||||||
|
cy.reload()
|
||||||
visitPage('home')
|
visitPage('home')
|
||||||
})
|
})
|
||||||
|
|
||||||
it('1 | opens the editor in VA mode: chrome hidden, editable, single bottom Submit', () => {
|
it('1 | opens the editor in VA mode: chrome hidden, read-only, no Filter button', () => {
|
||||||
openEditorInVaMode('mpe_x_test', () => {
|
openEditorInVaMode('mpe_x_test', () => {
|
||||||
// Chrome hidden (same as embed=true)
|
// Chrome hidden (same as embed=true)
|
||||||
cy.get('header.app-header').should('not.exist')
|
cy.get('header.app-header').should('not.exist')
|
||||||
|
|
||||||
// Grid present and in edit mode (read-only Filter/Edit/Upload bar absent)
|
// Grid present, read-only: the user enters edit mode with Edit, as usual
|
||||||
cy.get('#hotTable', { timeout: longerCommandTimeout }).should('exist')
|
cy.get('#hotTable', { timeout: longerCommandTimeout }).should('exist')
|
||||||
cy.get('.btnCtrl button').contains('Edit').should('not.exist')
|
cy.get('.btnCtrl button').contains('Edit').should('exist')
|
||||||
|
|
||||||
|
// The filter builder is replaced by VA filters
|
||||||
cy.get('.btnCtrl button').contains('Filter').should('not.exist')
|
cy.get('.btnCtrl button').contains('Filter').should('not.exist')
|
||||||
|
|
||||||
// Top edit-mode buttons hidden in VA mode
|
// Not in edit mode, so no edit-mode controls
|
||||||
cy.get('.btnCtrl button.btn-outline-danger').should('not.exist') // Cancel
|
cy.get('.btnCtrl button.btn-outline-danger').should('not.exist') // Cancel
|
||||||
cy.get('.btnCtrl').contains('Add Row').should('not.exist')
|
cy.get('.btnCtrl').contains('Add Row').should('not.exist')
|
||||||
|
cy.get('.btnCtrl').contains('Submit').should('not.exist')
|
||||||
|
|
||||||
// Bottom Add Record hidden
|
// Bottom Add Record hidden in VA mode
|
||||||
cy.contains('button', 'Add Record').should('not.exist')
|
cy.contains('button', 'Add Record').should('not.exist')
|
||||||
|
|
||||||
// Exactly one Submit button, below the grid, disabled in v1
|
|
||||||
cy.get('button').filter(':contains("Submit")').should('have.length', 1)
|
|
||||||
cy.get('button').contains('Submit').should('be.disabled')
|
|
||||||
})
|
})
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -69,14 +115,382 @@ context('embed=va (VA data-driven content) tests: ', function () {
|
|||||||
// Editor still loads in VA mode after the message (chrome hidden, grid up)
|
// Editor still loads in VA mode after the message (chrome hidden, grid up)
|
||||||
cy.get('#hotTable', { timeout: longerCommandTimeout }).should('exist')
|
cy.get('#hotTable', { timeout: longerCommandTimeout }).should('exist')
|
||||||
cy.get('header.app-header').should('not.exist')
|
cy.get('header.app-header').should('not.exist')
|
||||||
cy.get('button').contains('Submit').should('exist')
|
cy.get('.btnCtrl button').contains('Edit').should('exist')
|
||||||
|
})
|
||||||
|
})
|
||||||
|
it('3 | va_parmap filters a column from a parameter that is not its name (tab 1)', () => {
|
||||||
|
// Two DC instances over one table (one per report tab), each filtering the
|
||||||
|
// same DC column from its own report parameter. Without the URL mapping the
|
||||||
|
// parameter label is not a column name, so nothing would filter at all. The
|
||||||
|
// mapping here is written in lower case against an upper-case label and
|
||||||
|
// column, which pins the case-insensitivity of both sides end to end.
|
||||||
|
openEditorInVaMode(
|
||||||
|
'mpe_x_test',
|
||||||
|
() => {
|
||||||
|
pushVaMessage({
|
||||||
|
name: 'pr1',
|
||||||
|
label: 'DROPDOWN_TAB1',
|
||||||
|
dataType: 'string',
|
||||||
|
value: 'Option 2'
|
||||||
|
})
|
||||||
|
// The applied filter names the MAPPED column, and the grid shrinks to
|
||||||
|
// the rows matching that value (Option 2 occurs twice in this table).
|
||||||
|
cy.get('.infoBar-text', { timeout: longerCommandTimeout }).should(
|
||||||
|
'contain.text',
|
||||||
|
'SOME_DROPDOWN'
|
||||||
|
)
|
||||||
|
cy.get('.infoBar-text').should('contain.text', "'Option 2'")
|
||||||
|
expectFewerRowsThan(unfilteredRows)
|
||||||
|
},
|
||||||
|
'va_parmap=dropdown_tab1-some_dropdown'
|
||||||
|
)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('4 | the other tab maps its own parameter to the same column', () => {
|
||||||
|
// Same table, same column, a different parameter label and value (Option 1
|
||||||
|
// occurs four times) - the per-instance URL mapping is what keeps the two
|
||||||
|
// tabs independent.
|
||||||
|
openEditorInVaMode(
|
||||||
|
'mpe_x_test',
|
||||||
|
() => {
|
||||||
|
pushVaMessage({
|
||||||
|
name: 'pr1',
|
||||||
|
label: 'DROPDOWN_TAB2',
|
||||||
|
dataType: 'string',
|
||||||
|
value: 'Option 1'
|
||||||
|
})
|
||||||
|
cy.get('.infoBar-text', { timeout: longerCommandTimeout }).should(
|
||||||
|
'contain.text',
|
||||||
|
'SOME_DROPDOWN'
|
||||||
|
)
|
||||||
|
cy.get('.infoBar-text').should('contain.text', "'Option 1'")
|
||||||
|
expectFewerRowsThan(unfilteredRows)
|
||||||
|
},
|
||||||
|
'va_parmap=DROPDOWN_TAB2-SOME_DROPDOWN'
|
||||||
|
)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('5 | warns in the grid when a mapping names a column the table lacks', () => {
|
||||||
|
// Written by hand in the report's URL, so a typo must be visible to the
|
||||||
|
// report author: the parameter is skipped, and the reason is shown rather
|
||||||
|
// than leaving an unfiltered grid that looks like a working filter.
|
||||||
|
openEditorInVaMode(
|
||||||
|
'mpe_x_test',
|
||||||
|
() => {
|
||||||
|
pushVaMessage({
|
||||||
|
name: 'pr1',
|
||||||
|
label: 'DROPDOWN_TAB1',
|
||||||
|
dataType: 'string',
|
||||||
|
value: 'Option 2'
|
||||||
|
})
|
||||||
|
cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
|
||||||
|
'contain.text',
|
||||||
|
'VA parameter mapping problem'
|
||||||
|
)
|
||||||
|
cy.get('.va-parmap-warning').should('contain.text', 'DROPDOWN_TAB1')
|
||||||
|
cy.get('.va-parmap-warning').should('contain.text', 'NOT_A_COLUMN')
|
||||||
|
// VA mode hides the app header, so the banner must paint inside the
|
||||||
|
// viewport on its own - assert the real box, not just its presence.
|
||||||
|
cy.get('.va-parmap-warning').should(($alert) => {
|
||||||
|
const rect = $alert[0].getBoundingClientRect()
|
||||||
|
expect(rect.width, 'banner width').to.be.greaterThan(100)
|
||||||
|
expect(rect.height, 'banner height').to.be.greaterThan(20)
|
||||||
|
expect(rect.top, 'banner top').to.be.at.least(0)
|
||||||
|
expect(rect.bottom, 'banner bottom').to.be.at.most(
|
||||||
|
Cypress.config('viewportHeight')
|
||||||
|
)
|
||||||
|
})
|
||||||
|
cy.screenshot('va-parmap-banner', { capture: 'viewport' })
|
||||||
|
// Nothing was filtered: the row count matches the unfiltered baseline.
|
||||||
|
expectRows((rows) => expect(rows).to.equal(unfilteredRows))
|
||||||
|
},
|
||||||
|
'va_parmap=DROPDOWN_TAB1-NOT_A_COLUMN'
|
||||||
|
)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('6 | warns straight away when a mapping entry cannot be parsed', () => {
|
||||||
|
// No separator, so the entry never becomes a mapping - reported before any
|
||||||
|
// VA message arrives.
|
||||||
|
openEditorInVaMode(
|
||||||
|
'mpe_x_test',
|
||||||
|
() => {
|
||||||
|
cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
|
||||||
|
'contain.text',
|
||||||
|
'it needs a "-" between the VA parameter and the column name'
|
||||||
|
)
|
||||||
|
},
|
||||||
|
'va_parmap=DROPDOWN_TAB1'
|
||||||
|
)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('7 | warns when a mapping names a parameter VA is not sending', () => {
|
||||||
|
// Parameter-side typo: the mapping never applies, so without the banner this
|
||||||
|
// is indistinguishable from a working filter over every row.
|
||||||
|
openEditorInVaMode(
|
||||||
|
'mpe_x_test',
|
||||||
|
() => {
|
||||||
|
pushVaMessage({
|
||||||
|
name: 'pr1',
|
||||||
|
label: 'DROPDOWN_TAB1',
|
||||||
|
dataType: 'string',
|
||||||
|
value: 'Option 2'
|
||||||
|
})
|
||||||
|
// One message only tells us this object's parameter set is still being
|
||||||
|
// discovered; the warning waits for a message that introduces no new name.
|
||||||
|
cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
|
||||||
|
'not.exist'
|
||||||
|
)
|
||||||
|
pushVaMessage({
|
||||||
|
name: 'pr1',
|
||||||
|
label: 'DROPDOWN_TAB1',
|
||||||
|
dataType: 'string',
|
||||||
|
value: 'Option 2'
|
||||||
|
})
|
||||||
|
cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
|
||||||
|
'contain.text',
|
||||||
|
'names VA parameter "dropdown_tab_1", which VA is not sending'
|
||||||
|
)
|
||||||
|
expectRows((rows) => expect(rows).to.equal(unfilteredRows))
|
||||||
|
},
|
||||||
|
'va_parmap=DROPDOWN_TAB_1-SOME_DROPDOWN'
|
||||||
|
)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('8 | applies a mapping from the buffered message on the very first load', () => {
|
||||||
|
// VA posts once on iframe load, before the app bootstraps, so the first
|
||||||
|
// getdata is already filtered from that buffered message - with no column
|
||||||
|
// metadata available, the mapping is what names the column.
|
||||||
|
visitWithBufferedMessage(
|
||||||
|
'va_parmap=DROPDOWN_TAB1-SOME_DROPDOWN',
|
||||||
|
{
|
||||||
|
name: 'pr1',
|
||||||
|
label: 'DROPDOWN_TAB1',
|
||||||
|
dataType: 'string',
|
||||||
|
value: 'Option 2'
|
||||||
|
},
|
||||||
|
() => {
|
||||||
|
cy.get('.infoBar-text', { timeout: longerCommandTimeout }).should(
|
||||||
|
'contain.text',
|
||||||
|
"SOME_DROPDOWN = 'Option 2'"
|
||||||
|
)
|
||||||
|
expectFewerRowsThan(unfilteredRows)
|
||||||
|
cy.url().should('match', /\/editor\/[^/]+\/\d+/)
|
||||||
|
}
|
||||||
|
)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('9 | a bad mapping on the first load still ends at the banner', () => {
|
||||||
|
// The deferred first load cannot check the column against metadata, so its
|
||||||
|
// saveQuery may fail and fall back to an unfiltered load - the banner is
|
||||||
|
// what stops that reading as a working filter.
|
||||||
|
visitWithBufferedMessage(
|
||||||
|
'va_parmap=DROPDOWN_TAB1-NOT_A_COLUMN',
|
||||||
|
{
|
||||||
|
name: 'pr1',
|
||||||
|
label: 'DROPDOWN_TAB1',
|
||||||
|
dataType: 'string',
|
||||||
|
value: 'Option 2'
|
||||||
|
},
|
||||||
|
() => {
|
||||||
|
cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
|
||||||
|
'contain.text',
|
||||||
|
'NOT_A_COLUMN'
|
||||||
|
)
|
||||||
|
cy.get('.va-parmap-warning').should('contain.text', 'DROPDOWN_TAB1')
|
||||||
|
// Deliberately no row-count assertion: this path cannot validate the
|
||||||
|
// column (no metadata yet), so the outcome depends on the platform - on
|
||||||
|
// SAS, mp_filtercheck aborts the save and the load falls back to
|
||||||
|
// unfiltered; the JS mock stores the filter and returns 0 rows. The
|
||||||
|
// banner is the part that must hold either way, because both outcomes
|
||||||
|
// are otherwise indistinguishable from a working filter.
|
||||||
|
}
|
||||||
|
)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('10 | warns when two parameters filter the same column (an empty grid)', () => {
|
||||||
|
// Clauses are AND-ed, so two different values on one column produce no rows
|
||||||
|
// at all - the last silent-empty-grid path. Reported rather than rewritten,
|
||||||
|
// so the author decides whether to use one parameter carrying both values.
|
||||||
|
openEditorInVaMode(
|
||||||
|
'mpe_x_test',
|
||||||
|
() => {
|
||||||
|
pushVaMessage(
|
||||||
|
{
|
||||||
|
name: 'pr1',
|
||||||
|
label: 'DROPDOWN_TAB1',
|
||||||
|
dataType: 'string',
|
||||||
|
value: 'Option 1'
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: 'pr2',
|
||||||
|
label: 'DROPDOWN_TAB2',
|
||||||
|
dataType: 'string',
|
||||||
|
value: 'Option 2'
|
||||||
|
}
|
||||||
|
)
|
||||||
|
cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
|
||||||
|
'contain.text',
|
||||||
|
'combined with AND'
|
||||||
|
)
|
||||||
|
cy.get('.va-parmap-warning').should('contain.text', 'DROPDOWN_TAB1')
|
||||||
|
cy.get('.va-parmap-warning').should('contain.text', 'DROPDOWN_TAB2')
|
||||||
|
// The consequence: rows must match both clauses, and the two values
|
||||||
|
// differ, so none can match.
|
||||||
|
expectRows((rows) => expect(rows, 'matching rows').to.equal(0))
|
||||||
|
},
|
||||||
|
'va_parmap=DROPDOWN_TAB1-SOME_DROPDOWN&va_parmap=DROPDOWN_TAB2-SOME_DROPDOWN'
|
||||||
|
)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('12 | a transform adds the previous quarter to the filter', () => {
|
||||||
|
// ADD(QUARTER_START,-1) is the reported requirement: one clause carrying the
|
||||||
|
// selected date and the same point one quarter earlier. 2026-10-01 = 24380,
|
||||||
|
// 2026-07-01 = 24288 in SAS days; nothing in the fixture matches either, so
|
||||||
|
// the assertion is on the clause rather than the row count.
|
||||||
|
openEditorInVaMode(
|
||||||
|
'mpe_x_test',
|
||||||
|
() => {
|
||||||
|
pushVaMessage({
|
||||||
|
name: 'pr1',
|
||||||
|
label: 'SELECTED_DATE',
|
||||||
|
dataType: 'string',
|
||||||
|
value: '01OCT2026'
|
||||||
|
})
|
||||||
|
cy.get('.infoBar-text', { timeout: longerCommandTimeout }).should(
|
||||||
|
'contain.text',
|
||||||
|
'SOME_DATE IN ('
|
||||||
|
)
|
||||||
|
cy.get('.infoBar-text').should('contain.text', '24380')
|
||||||
|
cy.get('.infoBar-text').should('contain.text', '24288')
|
||||||
|
cy.get('h4').should('contain.text', '(0 rows')
|
||||||
|
},
|
||||||
|
'va_parmap=SELECTED_DATE-SOME_DATE:ADD(QUARTER_START,-1)'
|
||||||
|
)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('13 | warns when a transform lands on a column it cannot apply to', () => {
|
||||||
|
openEditorInVaMode(
|
||||||
|
'mpe_x_test',
|
||||||
|
() => {
|
||||||
|
pushVaMessage({
|
||||||
|
name: 'pr1',
|
||||||
|
label: 'SELECTED_DATE',
|
||||||
|
dataType: 'string',
|
||||||
|
value: 'x'
|
||||||
|
})
|
||||||
|
cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
|
||||||
|
'contain.text',
|
||||||
|
'is not a date or datetime column'
|
||||||
|
)
|
||||||
|
cy.get('h4').should('contain.text', '(7 rows')
|
||||||
|
},
|
||||||
|
'va_parmap=SELECTED_DATE-SOME_CHAR:ADD(QUARTER_START,-1)'
|
||||||
|
)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('11 | logs a parse-time problem once, however many messages arrive', () => {
|
||||||
|
// The parse problem is logged at app init; the editor's own loop must not log
|
||||||
|
// it again. The second entry is valid-but-missing, so the loop definitely runs
|
||||||
|
// and logs its own line - which is what makes this a real assertion.
|
||||||
|
cy.visit(
|
||||||
|
`${hostUrl}${appLocation}/index.html#/editor/DC_JSLIB.MPE_X_TEST?embed=va&va_parmap=DROPDOWN_TAB1&va_parmap=DROPDOWN_TAB2-NOT_A_COLUMN`,
|
||||||
|
{
|
||||||
|
onBeforeLoad(win: any) {
|
||||||
|
win.__warns = []
|
||||||
|
const original = win.console.warn.bind(win.console)
|
||||||
|
win.console.warn = (...args: any[]) => {
|
||||||
|
win.__warns.push(args.join(' '))
|
||||||
|
original(...args)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
)
|
||||||
|
cy.get('.app-loading', { timeout: longerCommandTimeout }).should(
|
||||||
|
'not.exist'
|
||||||
|
)
|
||||||
|
pushVaMessage({
|
||||||
|
name: 'pr2',
|
||||||
|
label: 'DROPDOWN_TAB2',
|
||||||
|
dataType: 'string',
|
||||||
|
value: 'Option 2'
|
||||||
|
})
|
||||||
|
cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
|
||||||
|
'contain.text',
|
||||||
|
'NOT_A_COLUMN'
|
||||||
|
)
|
||||||
|
cy.window().then((win: any) => {
|
||||||
|
const warns: string[] = win.__warns || []
|
||||||
|
const parseLines = warns.filter((w) =>
|
||||||
|
w.includes('va_parmap "DROPDOWN_TAB1"')
|
||||||
|
)
|
||||||
|
const runtimeLines = warns.filter((w) => w.includes('NOT_A_COLUMN'))
|
||||||
|
expect(parseLines.length, warns.join(' | ')).to.eq(1)
|
||||||
|
expect(runtimeLines.length, warns.join(' | ')).to.eq(1)
|
||||||
})
|
})
|
||||||
})
|
})
|
||||||
})
|
})
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Loads the editor with a VA message already in the pre-bootstrap buffer, the
|
||||||
|
* way VA's own postMessage arrives before Angular starts. A distinct pathname is
|
||||||
|
* used so this is a real document load: a cy.visit that differs only in the hash
|
||||||
|
* is a same-document navigation and would not re-bootstrap the app.
|
||||||
|
*/
|
||||||
|
const visitWithBufferedMessage = (
|
||||||
|
extraParams: string,
|
||||||
|
parameter: any,
|
||||||
|
callback?: any
|
||||||
|
) => {
|
||||||
|
cy.visit(
|
||||||
|
`${hostUrl}${appLocation}/index.html#/editor/DC_JSLIB.MPE_X_TEST?embed=va&${extraParams}`,
|
||||||
|
{
|
||||||
|
onBeforeLoad(win: any) {
|
||||||
|
win.__vaLastMessage = {
|
||||||
|
t: Date.now(),
|
||||||
|
origin: win.location.origin,
|
||||||
|
data: {
|
||||||
|
version: '1',
|
||||||
|
resultName: 'dd1',
|
||||||
|
rowCount: 0,
|
||||||
|
availableRowCount: 0,
|
||||||
|
data: [],
|
||||||
|
columns: [],
|
||||||
|
parameters: [parameter]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
)
|
||||||
|
cy.get('.app-loading', { timeout: longerCommandTimeout }).should('not.exist')
|
||||||
|
if (callback) callback()
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Pushes one VA data-driven content message carrying the given parameters. */
|
||||||
|
const pushVaMessage = (...parameters: any[]) => {
|
||||||
|
cy.window().then((win) => {
|
||||||
|
win.postMessage(
|
||||||
|
{
|
||||||
|
version: '1',
|
||||||
|
resultName: 'dd1',
|
||||||
|
rowCount: 0,
|
||||||
|
availableRowCount: 0,
|
||||||
|
data: [],
|
||||||
|
columns: [],
|
||||||
|
parameters
|
||||||
|
},
|
||||||
|
'*'
|
||||||
|
)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
// Opens a table from the tree (which routes to #/editor/LIB.TABLE), then
|
// Opens a table from the tree (which routes to #/editor/LIB.TABLE), then
|
||||||
// re-visits the same route with ?embed=va so the app parses VA embed mode.
|
// re-visits the same route with ?embed=va so the app parses VA embed mode.
|
||||||
const openEditorInVaMode = (tablename: string, callback?: any) => {
|
// `extraParams` is appended to that query string (e.g. va_parmap entries).
|
||||||
|
const openEditorInVaMode = (
|
||||||
|
tablename: string,
|
||||||
|
callback?: any,
|
||||||
|
extraParams?: string
|
||||||
|
) => {
|
||||||
openTableFromTree(libraryToOpenIncludes, tablename)
|
openTableFromTree(libraryToOpenIncludes, tablename)
|
||||||
|
|
||||||
cy.get('#hotTable', { timeout: longerCommandTimeout })
|
cy.get('#hotTable', { timeout: longerCommandTimeout })
|
||||||
@@ -84,7 +498,13 @@ const openEditorInVaMode = (tablename: string, callback?: any) => {
|
|||||||
.then(() => {
|
.then(() => {
|
||||||
cy.url().then((url) => {
|
cy.url().then((url) => {
|
||||||
const separator = url.includes('?') ? '&' : '?'
|
const separator = url.includes('?') ? '&' : '?'
|
||||||
cy.visit(`${url}${separator}embed=va`)
|
const extra = extraParams ? `&${extraParams}` : ''
|
||||||
|
cy.visit(`${url}${separator}embed=va${extra}`)
|
||||||
|
// A cy.visit that differs from the current URL only in the hash is a
|
||||||
|
// same-document navigation, so the app is not re-bootstrapped and the
|
||||||
|
// embed parameter (read once at app init) never takes effect. Reload to
|
||||||
|
// bootstrap it, as the VA iframe load does.
|
||||||
|
cy.reload()
|
||||||
cy.get('.app-loading', { timeout: longerCommandTimeout }).should(
|
cy.get('.app-loading', { timeout: longerCommandTimeout }).should(
|
||||||
'not.exist'
|
'not.exist'
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -14,11 +14,10 @@ context('metanav tests: ', function () {
|
|||||||
})
|
})
|
||||||
|
|
||||||
this.beforeEach(() => {
|
this.beforeEach(() => {
|
||||||
cy.visit(hostUrl + appLocation)
|
// beforeAll has already logged in. Typing into the login form here failed
|
||||||
cy.get('input.username').type(username)
|
// once the session existed - the form is not on screen, so Cypress reported
|
||||||
cy.get('input.password').type(password)
|
// "the center of this element is hidden from view" and the whole suite was
|
||||||
cy.get('.login-group button').click()
|
// skipped behind that hook.
|
||||||
|
|
||||||
visitPage('view/metadata')
|
visitPage('view/metadata')
|
||||||
})
|
})
|
||||||
|
|
||||||
|
|||||||
+3
-1
@@ -37,7 +37,9 @@
|
|||||||
"compodoc:build-and-serve": "compodoc -p tsconfig.doc.json -s --name 'Data Controller Client'",
|
"compodoc:build-and-serve": "compodoc -p tsconfig.doc.json -s --name 'Data Controller Client'",
|
||||||
"compodoc:serve": "compodoc -s --name 'Data Controller Client'",
|
"compodoc:serve": "compodoc -s --name 'Data Controller Client'",
|
||||||
"lighthouse": "lhci autorun",
|
"lighthouse": "lhci autorun",
|
||||||
"ng": "ng"
|
"ng": "ng",
|
||||||
|
"test:tz-west": "TZ=America/New_York npx ng test --no-watch --no-progress --browsers ChromeHeadlessCI --include='**/va-*.spec.ts'",
|
||||||
|
"test:tz-east": "TZ=Pacific/Auckland npx ng test --no-watch --no-progress --browsers ChromeHeadlessCI --include='**/va-*.spec.ts'"
|
||||||
},
|
},
|
||||||
"private": true,
|
"private": true,
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
import { QueryClause } from './models/TableData'
|
import { QueryClause } from './models/TableData'
|
||||||
|
import { VaParmapTarget } from './shared/utils/parse-va-parmap'
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Filtering cache info, to be reused when filtering modal is re-open
|
* Filtering cache info, to be reused when filtering modal is re-open
|
||||||
@@ -61,6 +62,21 @@ export const globals: {
|
|||||||
* Defaults to 'live'; toggled by the Auto-apply checkbox in the editor.
|
* Defaults to 'live'; toggled by the Auto-apply checkbox in the editor.
|
||||||
*/
|
*/
|
||||||
vaApplyMode: 'live' | 'confirm'
|
vaApplyMode: 'live' | 'confirm'
|
||||||
|
/**
|
||||||
|
* Explicit VA parameter -> DC column mappings, parsed from the `va_parmap`
|
||||||
|
* URL parameter (see parseVaParmap). Empty unless configured; entries override
|
||||||
|
* VA's default same-name (parameter label -> column) resolution, so two DC
|
||||||
|
* instances over one table can filter the same column from differently-named
|
||||||
|
* VA parameters.
|
||||||
|
*/
|
||||||
|
vaParamMap: Map<string, VaParmapTarget>
|
||||||
|
/**
|
||||||
|
* `va_parmap` entries that could not be used at all (not a plain SAS column
|
||||||
|
* name, or missing a side of the `-`). Surfaced to the report author, because
|
||||||
|
* these are hand-written in the report's URL and a typo otherwise looks like
|
||||||
|
* a working, unfiltered view.
|
||||||
|
*/
|
||||||
|
vaParmapProblems: string[]
|
||||||
rootParam: string
|
rootParam: string
|
||||||
dcLib: string
|
dcLib: string
|
||||||
xlmaps: XLMapListItem[]
|
xlmaps: XLMapListItem[]
|
||||||
@@ -77,6 +93,8 @@ export const globals: {
|
|||||||
} = {
|
} = {
|
||||||
embed: false,
|
embed: false,
|
||||||
vaApplyMode: 'live',
|
vaApplyMode: 'live',
|
||||||
|
vaParamMap: new Map(),
|
||||||
|
vaParmapProblems: [],
|
||||||
rootParam: <string>'',
|
rootParam: <string>'',
|
||||||
dcLib: '',
|
dcLib: '',
|
||||||
xlmaps: [],
|
xlmaps: [],
|
||||||
|
|||||||
@@ -12,6 +12,7 @@ import '@clr/icons'
|
|||||||
import '@clr/icons/shapes/all-shapes'
|
import '@clr/icons/shapes/all-shapes'
|
||||||
import { globals } from './_globals'
|
import { globals } from './_globals'
|
||||||
import { parseEmbedParam } from './shared/utils/parse-embed-param'
|
import { parseEmbedParam } from './shared/utils/parse-embed-param'
|
||||||
|
import { parseVaParmap } from './shared/utils/parse-va-parmap'
|
||||||
import moment from 'moment'
|
import moment from 'moment'
|
||||||
import { EventService } from './services/event.service'
|
import { EventService } from './services/event.service'
|
||||||
import { AppService } from './services/app.service'
|
import { AppService } from './services/app.service'
|
||||||
@@ -163,6 +164,21 @@ export class AppComponent {
|
|||||||
globals.embed = embed
|
globals.embed = embed
|
||||||
this.embed = embed
|
this.embed = embed
|
||||||
}
|
}
|
||||||
|
// VA parameter -> DC column mappings for this instance, held on globals
|
||||||
|
// (like `embed` and `vaApplyMode`) so in-app navigation that rebuilds the
|
||||||
|
// editor keeps them - e.g. returning from the stage / approve pages.
|
||||||
|
if (globals.embed === 'va') {
|
||||||
|
const parmap = parseVaParmap(hashQuery ?? '')
|
||||||
|
globals.vaParamMap = parmap.map
|
||||||
|
globals.vaParmapProblems = parmap.problems
|
||||||
|
// Visible in the browser console as well as the editor banner: the entry
|
||||||
|
// is hand-written in the report's URL, so a typo has to be findable when
|
||||||
|
// someone is looking at the report rather than at DC's UI.
|
||||||
|
for (const problem of parmap.problems) {
|
||||||
|
// eslint-disable-next-line no-console
|
||||||
|
console.warn('Data Controller: ' + problem)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
this.subscribeToShowAbortModal()
|
this.subscribeToShowAbortModal()
|
||||||
this.subscribeToRequestsModal()
|
this.subscribeToRequestsModal()
|
||||||
|
|||||||
@@ -1,7 +1,36 @@
|
|||||||
<main class="content-area position-relative">
|
<main class="content-area position-relative">
|
||||||
|
<!-- An embedded instance (a report object) has no report author who can deploy:
|
||||||
|
showing the wizard there only produces a screen whose buttons cannot work,
|
||||||
|
because the deployment needs an authenticated, non-embedded session. Name
|
||||||
|
the state instead, and keep the wizard for standalone use. -->
|
||||||
|
@if (isEmbedded) {
|
||||||
|
<div class="clr-row">
|
||||||
|
<div id="not-configured" class="card">
|
||||||
|
<div class="card-header">Data Controller is not configured</div>
|
||||||
|
<div class="card-block">
|
||||||
|
<div class="card-text">
|
||||||
|
<p class="mt-0">
|
||||||
|
This Data Controller instance has not been deployed yet, so it
|
||||||
|
cannot be shown inside a report.
|
||||||
|
</p>
|
||||||
|
<p>
|
||||||
|
Deployment has to be done once, in its own browser tab, by someone
|
||||||
|
who can write to the application location:
|
||||||
|
<a [href]="deployUrl" target="_blank">{{ deployUrl }}</a>
|
||||||
|
</p>
|
||||||
|
<p>
|
||||||
|
When it is configured, reload the report and this object will show
|
||||||
|
the editor.
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
}
|
||||||
|
|
||||||
<div class="clr-row">
|
<div class="clr-row">
|
||||||
<!-- T&C section -->
|
<!-- T&C section -->
|
||||||
@if (step === 0) {
|
@if (!isEmbedded && step === 0) {
|
||||||
<div id="TCS" class="card">
|
<div id="TCS" class="card">
|
||||||
<div class="card-header">Terms and Conditions</div>
|
<div class="card-header">Terms and Conditions</div>
|
||||||
<div class="card-block">
|
<div class="card-block">
|
||||||
@@ -35,7 +64,7 @@
|
|||||||
</div>
|
</div>
|
||||||
<!-- T&C section end -->
|
<!-- T&C section end -->
|
||||||
|
|
||||||
@if (step > 0) {
|
@if (!isEmbedded && step > 0) {
|
||||||
@switch (true) {
|
@switch (true) {
|
||||||
@case (sasJsConfig.serverType === ServerType.SasViya) {
|
@case (sasJsConfig.serverType === ServerType.SasViya) {
|
||||||
<div class="autodeploy-section card">
|
<div class="autodeploy-section card">
|
||||||
|
|||||||
@@ -0,0 +1,229 @@
|
|||||||
|
import { NO_ERRORS_SCHEMA } from '@angular/core'
|
||||||
|
import { ComponentFixture, TestBed } from '@angular/core/testing'
|
||||||
|
import { ServerType } from '@sasjs/utils/types/serverType'
|
||||||
|
import { DeployComponent } from './deploy.component'
|
||||||
|
import { standaloneUrl } from './standalone-url'
|
||||||
|
import { globals } from '../_globals'
|
||||||
|
import { AppStoreService } from '../services/app-store.service'
|
||||||
|
import { SasService } from '../services/sas.service'
|
||||||
|
|
||||||
|
/**
|
||||||
|
* An embedded instance cannot deploy itself: the deployment needs an
|
||||||
|
* authenticated, standalone session, and the person looking at a report object
|
||||||
|
* is not the person who installs the app. The component therefore withholds the
|
||||||
|
* wizard and names the state instead.
|
||||||
|
*
|
||||||
|
* These cases render the template, so they fail if the `!isEmbedded` guards in
|
||||||
|
* deploy.component.html are dropped - asserting the getter alone would not.
|
||||||
|
*/
|
||||||
|
describe('DeployComponent embed guard', () => {
|
||||||
|
let fixture: ComponentFixture<DeployComponent>
|
||||||
|
let component: DeployComponent
|
||||||
|
|
||||||
|
const originalEmbed = globals.embed
|
||||||
|
|
||||||
|
const configure = () =>
|
||||||
|
TestBed.configureTestingModule({
|
||||||
|
declarations: [DeployComponent],
|
||||||
|
providers: [
|
||||||
|
{
|
||||||
|
provide: AppStoreService,
|
||||||
|
useValue: {
|
||||||
|
getDcAdapterSettings: () => ({
|
||||||
|
appLoc: '/Public/app/dc',
|
||||||
|
dcPath: '/tmp/dc'
|
||||||
|
})
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
provide: SasService,
|
||||||
|
useValue: {
|
||||||
|
getSasjsInstance: () => ({}),
|
||||||
|
getSasjsConfig: () => ({ serverType: ServerType.Sasjs })
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
// The wizard renders child components this spec does not exercise.
|
||||||
|
schemas: [NO_ERRORS_SCHEMA]
|
||||||
|
}).compileComponents()
|
||||||
|
|
||||||
|
const render = () => {
|
||||||
|
fixture = TestBed.createComponent(DeployComponent)
|
||||||
|
component = fixture.componentInstance
|
||||||
|
fixture.detectChanges()
|
||||||
|
}
|
||||||
|
|
||||||
|
const tcsSection = () =>
|
||||||
|
fixture.nativeElement.querySelector('#TCS') as HTMLElement | null
|
||||||
|
const notConfigured = () =>
|
||||||
|
fixture.nativeElement.querySelector('#not-configured') as HTMLElement | null
|
||||||
|
const deploySection = () =>
|
||||||
|
fixture.nativeElement.querySelector(
|
||||||
|
'.autodeploy-section'
|
||||||
|
) as HTMLElement | null
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
globals.embed = originalEmbed
|
||||||
|
TestBed.resetTestingModule()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('offers the wizard when running in its own tab', () => {
|
||||||
|
globals.embed = false
|
||||||
|
configure()
|
||||||
|
render()
|
||||||
|
|
||||||
|
expect(tcsSection()).toBeTruthy()
|
||||||
|
expect(notConfigured()).toBeNull()
|
||||||
|
expect(component.isEmbedded).toBeFalse()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('replaces the wizard with the notice for a VA embed', () => {
|
||||||
|
globals.embed = 'va'
|
||||||
|
configure()
|
||||||
|
render()
|
||||||
|
|
||||||
|
expect(tcsSection()).toBeNull()
|
||||||
|
expect(notConfigured()).toBeTruthy()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('replaces the wizard with the notice for a plain embed too', () => {
|
||||||
|
globals.embed = true
|
||||||
|
configure()
|
||||||
|
render()
|
||||||
|
|
||||||
|
expect(tcsSection()).toBeNull()
|
||||||
|
expect(notConfigured()).toBeTruthy()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('names the standalone URL in the embedded notice', () => {
|
||||||
|
globals.embed = 'va'
|
||||||
|
configure()
|
||||||
|
render()
|
||||||
|
|
||||||
|
const link = notConfigured()!.querySelector('a') as HTMLAnchorElement
|
||||||
|
|
||||||
|
expect(link.getAttribute('href')).toBe(component.deployUrl)
|
||||||
|
expect(link.textContent).toBe(component.deployUrl)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('withholds the deploy section after the terms are accepted, when embedded', () => {
|
||||||
|
globals.embed = 'va'
|
||||||
|
configure()
|
||||||
|
render()
|
||||||
|
|
||||||
|
component.step = 1
|
||||||
|
fixture.detectChanges()
|
||||||
|
|
||||||
|
expect(deploySection()).toBeNull()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('shows the deploy section after the terms are accepted, standalone', () => {
|
||||||
|
globals.embed = false
|
||||||
|
configure()
|
||||||
|
render()
|
||||||
|
|
||||||
|
component.step = 1
|
||||||
|
fixture.detectChanges()
|
||||||
|
|
||||||
|
expect(deploySection()).toBeTruthy()
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
describe('DeployComponent onNavigateToHome', () => {
|
||||||
|
const originalEmbed = globals.embed
|
||||||
|
const originalHref = window.location.href
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
globals.embed = originalEmbed
|
||||||
|
history.replaceState(null, '', originalHref)
|
||||||
|
TestBed.resetTestingModule()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('opens the standalone URL, merging an embed tab into the app URL', () => {
|
||||||
|
// Karma serves the runner with its own query, so the app URL is set here
|
||||||
|
// rather than read from window.location: the assertion has to see the
|
||||||
|
// app's query survive the embed parameters being dropped. The shape is
|
||||||
|
// the real SAS 9 one - the app's `_program` query in the search part,
|
||||||
|
// the embed switches in the hash query.
|
||||||
|
history.replaceState(
|
||||||
|
null,
|
||||||
|
'',
|
||||||
|
'/deploy-tc?_program=/Public/app/dc&_debug=131#/editor?embed=va&va_parmap=A-B'
|
||||||
|
)
|
||||||
|
const open = spyOn(window, 'open')
|
||||||
|
TestBed.configureTestingModule({
|
||||||
|
declarations: [DeployComponent],
|
||||||
|
providers: [
|
||||||
|
{
|
||||||
|
provide: AppStoreService,
|
||||||
|
useValue: { getDcAdapterSettings: () => ({ appLoc: '', dcPath: '' }) }
|
||||||
|
},
|
||||||
|
{
|
||||||
|
provide: SasService,
|
||||||
|
useValue: {
|
||||||
|
getSasjsInstance: () => ({}),
|
||||||
|
getSasjsConfig: () => ({ serverType: ServerType.Sasjs })
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
schemas: [NO_ERRORS_SCHEMA]
|
||||||
|
})
|
||||||
|
const fixture = TestBed.createComponent(DeployComponent)
|
||||||
|
|
||||||
|
fixture.componentInstance.onNavigateToHome()
|
||||||
|
|
||||||
|
const opened = open.calls.mostRecent().args[0] as string
|
||||||
|
expect(opened).toBe(
|
||||||
|
window.location.origin + '/deploy-tc?_program=/Public/app/dc&_debug=131'
|
||||||
|
)
|
||||||
|
expect(opened).not.toContain('#')
|
||||||
|
expect(opened).not.toContain('embed=')
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
describe('standaloneUrl', () => {
|
||||||
|
const origin = window.location.origin
|
||||||
|
|
||||||
|
it('keeps the path of a plain URL', () => {
|
||||||
|
expect(standaloneUrl(`${origin}/SASJobExecution/dc`)).toBe(
|
||||||
|
`${origin}/SASJobExecution/dc`
|
||||||
|
)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('keeps a query the app needs, and drops the hash route', () => {
|
||||||
|
expect(standaloneUrl(`${origin}/path?x=1#/deploy`)).toBe(
|
||||||
|
`${origin}/path?x=1`
|
||||||
|
)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('keeps `_program=`, which is how SAS 9 and Viya address the app', () => {
|
||||||
|
// The whole app URL is the query here: /SASJobExecution/?_program=... is
|
||||||
|
// the standalone app, so dropping the query would open a blank executor.
|
||||||
|
const sas9 =
|
||||||
|
`${origin}/SASJobExecution/?_program=/Public/app/dc/services/public/startupservice` +
|
||||||
|
'&_debug=131'
|
||||||
|
expect(standaloneUrl(sas9)).toBe(sas9)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('drops a bare embed query', () => {
|
||||||
|
expect(standaloneUrl(`${origin}/app?embed=va`)).toBe(`${origin}/app`)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('drops the va_parmap entries along with the embed switch', () => {
|
||||||
|
expect(
|
||||||
|
standaloneUrl(
|
||||||
|
`${origin}/app?embed=va&va_parmap=A-B&va_parmap=C-D#/editor/x`
|
||||||
|
)
|
||||||
|
).toBe(`${origin}/app`)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('keeps a query parameter that merely starts with va_', () => {
|
||||||
|
expect(standaloneUrl(`${origin}/app?embed=va&va_rowlimit=100`)).toBe(
|
||||||
|
`${origin}/app?va_rowlimit=100`
|
||||||
|
)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('resolves a relative URL against the current origin', () => {
|
||||||
|
expect(standaloneUrl('/app/#/deploy')).toBe(`${origin}/app/`)
|
||||||
|
})
|
||||||
|
})
|
||||||
@@ -4,6 +4,8 @@ import { SASjsConfig } from '@sasjs/adapter'
|
|||||||
import { ServerType } from '@sasjs/utils/types/serverType'
|
import { ServerType } from '@sasjs/utils/types/serverType'
|
||||||
import { AppStoreService } from '../services/app-store.service'
|
import { AppStoreService } from '../services/app-store.service'
|
||||||
import { DcAdapterSettings } from '../models/DcAdapterSettings'
|
import { DcAdapterSettings } from '../models/DcAdapterSettings'
|
||||||
|
import { globals } from '../_globals'
|
||||||
|
import { standaloneUrl } from './standalone-url'
|
||||||
|
|
||||||
@Component({
|
@Component({
|
||||||
selector: 'app-deploy',
|
selector: 'app-deploy',
|
||||||
@@ -44,6 +46,20 @@ export class DeployComponent implements OnInit {
|
|||||||
this.setDeployDefaults()
|
this.setDeployDefaults()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* True when DC is running inside a report rather than in its own tab. The
|
||||||
|
* deployment needs an authenticated, standalone session, so the wizard is not
|
||||||
|
* offered here - a report object cannot deploy the app that it is displaying.
|
||||||
|
*/
|
||||||
|
get isEmbedded(): boolean {
|
||||||
|
return !!globals.embed
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The standalone URL an administrator deploys from, for the embedded notice. */
|
||||||
|
get deployUrl(): string {
|
||||||
|
return standaloneUrl(window.location.href)
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Setting default values used for deploy request
|
* Setting default values used for deploy request
|
||||||
*/
|
*/
|
||||||
@@ -60,6 +76,8 @@ export class DeployComponent implements OnInit {
|
|||||||
}
|
}
|
||||||
|
|
||||||
public onNavigateToHome() {
|
public onNavigateToHome() {
|
||||||
window.open(location.href.split('#')[0], '_blank')
|
// Same helper as deployUrl: splitting on '#' left the query string in
|
||||||
|
// place, so a tab opened from `?embed=va#/deploy` stayed embedded.
|
||||||
|
window.open(standaloneUrl(window.location.href), '_blank')
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
/**
|
||||||
|
* The URL a Data Controller instance runs at when it is not embedded: the same
|
||||||
|
* URL, without the parts that make it an embed - the `embed` switch, the
|
||||||
|
* `va_parmap` entries, and the hash route.
|
||||||
|
*
|
||||||
|
* The rest of the query is kept. On SAS 9 and Viya the app is addressed
|
||||||
|
* entirely through it (`/SASJobExecution/?_program=/.../startupservice`), so
|
||||||
|
* dropping the query would open a blank executor rather than the app, and any
|
||||||
|
* other parameter the deployment relies on would go the same way.
|
||||||
|
*
|
||||||
|
* Only these two parameters are embed parameters. A later parameter that
|
||||||
|
* happens to start with `va_` belongs to the app, not the embed, so it stays.
|
||||||
|
*
|
||||||
|
* Parsing with URL also handles the relative hrefs the adapter can hand back,
|
||||||
|
* which plain string splicing cannot.
|
||||||
|
*/
|
||||||
|
const EMBED_PARAMS = ['embed', 'va_parmap']
|
||||||
|
|
||||||
|
export const standaloneUrl = (href: string): string => {
|
||||||
|
const url = new URL(href, window.location.origin)
|
||||||
|
url.hash = ''
|
||||||
|
for (const key of Array.from(url.searchParams.keys())) {
|
||||||
|
if (EMBED_PARAMS.includes(key.toLowerCase())) {
|
||||||
|
url.searchParams.delete(key)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return url.toString()
|
||||||
|
}
|
||||||
@@ -459,6 +459,43 @@
|
|||||||
<!--<hr>-->
|
<!--<hr>-->
|
||||||
<!--{{ validationDone }}-->
|
<!--{{ validationDone }}-->
|
||||||
|
|
||||||
|
<!-- VA embed: something about a `va_parmap` entry in this report
|
||||||
|
object's URL needs the report author's attention - it could not be
|
||||||
|
used, it was applied but AND-ed against another parameter, or it
|
||||||
|
names a parameter VA never sends. Shown as a banner, not just a
|
||||||
|
console warning, because
|
||||||
|
the entries are written by the report author and a typo otherwise
|
||||||
|
looks like a working filter over the whole table. -->
|
||||||
|
@if (isVaEmbed && vaMappingWarnings.length) {
|
||||||
|
<clr-alert
|
||||||
|
class="va-parmap-warning"
|
||||||
|
[clrAlertType]="'warning'"
|
||||||
|
[clrAlertClosable]="false"
|
||||||
|
>
|
||||||
|
<clr-alert-item>
|
||||||
|
<div class="alert-text">
|
||||||
|
<strong>VA parameter mapping problem</strong>
|
||||||
|
@for (warning of vaMappingWarnings; track warning) {
|
||||||
|
<div>{{ warning }}</div>
|
||||||
|
}
|
||||||
|
<!-- The reader is the report author, who can only fix this in
|
||||||
|
the report: name the parameter they edit, and say what a
|
||||||
|
correct entry looks like, so the message is actionable
|
||||||
|
without the docs. -->
|
||||||
|
<div>
|
||||||
|
Fix: edit the <code>va_parmap</code> entries in the URL of
|
||||||
|
this Data Controller object in the report. An entry is
|
||||||
|
<code>parameter-column</code>, optionally
|
||||||
|
<code>:ADD(anchor,offset)</code> or
|
||||||
|
<code>:RANGE(period,offset)</code> - for example
|
||||||
|
<code>va_parmap=SEL_DATE-D_DATE:ADD(QUARTER_START,-1)</code
|
||||||
|
>.
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</clr-alert-item>
|
||||||
|
</clr-alert>
|
||||||
|
}
|
||||||
|
|
||||||
<div class="hot-wrapper clr-flex-1">
|
<div class="hot-wrapper clr-flex-1">
|
||||||
<hot-table
|
<hot-table
|
||||||
#hotInstance
|
#hotInstance
|
||||||
|
|||||||
@@ -27,3 +27,15 @@
|
|||||||
font-style: italic;
|
font-style: italic;
|
||||||
opacity: 0.7;
|
opacity: 0.7;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* VA embed: `va_parmap` diagnostic banner. Clarity's alert lays `.alert-title`
|
||||||
|
and `.alert-text` out on one row, which pushes a long message off the right
|
||||||
|
edge; this keeps the whole message in the flow and wraps it. */
|
||||||
|
.va-parmap-warning {
|
||||||
|
width: 100%;
|
||||||
|
|
||||||
|
.alert-text {
|
||||||
|
white-space: normal;
|
||||||
|
overflow-wrap: anywhere;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -73,6 +73,7 @@ import {
|
|||||||
} from '../shared/dc-validator/utils/autoEscapedCellTracker'
|
} from '../shared/dc-validator/utils/autoEscapedCellTracker'
|
||||||
import { HyperFormula } from 'hyperformula'
|
import { HyperFormula } from 'hyperformula'
|
||||||
import { parseLabelsParam } from '../shared/utils/parse-labels-param'
|
import { parseLabelsParam } from '../shared/utils/parse-labels-param'
|
||||||
|
import { VaParmapTarget } from '../shared/utils/parse-va-parmap'
|
||||||
import { getDisplayColHeaders } from '../shared/utils/display-col-headers'
|
import { getDisplayColHeaders } from '../shared/utils/display-col-headers'
|
||||||
import { buildColInfoHtml } from '../shared/utils/col-info-html'
|
import { buildColInfoHtml } from '../shared/utils/col-info-html'
|
||||||
import { globals } from '../_globals'
|
import { globals } from '../_globals'
|
||||||
@@ -485,6 +486,46 @@ export class EditorComponent
|
|||||||
get isVaEmbed() {
|
get isVaEmbed() {
|
||||||
return globals.embed === 'va'
|
return globals.embed === 'va'
|
||||||
}
|
}
|
||||||
|
/**
|
||||||
|
* Explicit VA parameter -> DC column mappings (`va_parmap`) for this instance.
|
||||||
|
* Parsed once at app init (see parseVaParmap) and held on globals so the
|
||||||
|
* editor reloads a filter apply performs keep using the same mapping.
|
||||||
|
*/
|
||||||
|
get vaParamMap(): Map<string, VaParmapTarget> {
|
||||||
|
return globals.vaParamMap
|
||||||
|
}
|
||||||
|
/** `va_parmap` entries that name a column this table does not have. */
|
||||||
|
public vaUnresolvedMappings: string[] = []
|
||||||
|
/** `va_parmap` keys that no VA message has named (a parameter-side typo). */
|
||||||
|
public vaUnmatchedMappings: string[] = []
|
||||||
|
/** Parameters in this message that filter the same column (an AND-ed pair). */
|
||||||
|
public vaSameColumnMappings: string[] = []
|
||||||
|
/** Transform mappings this message cannot apply (non-temporal column or value). */
|
||||||
|
public vaTransformProblems: string[] = []
|
||||||
|
/** The warning text last written to the console, so repeated VA messages log once. */
|
||||||
|
private vaWarningsReported = ''
|
||||||
|
/**
|
||||||
|
* Everything wrong with this instance's `va_parmap` entries, in the report
|
||||||
|
* author's terms: entries that could not be parsed at all, plus entries that
|
||||||
|
* name a column the open table does not have. Shown as a banner in the editor
|
||||||
|
* - a report developer is usually not going to read the browser console.
|
||||||
|
*/
|
||||||
|
get vaMappingWarnings(): string[] {
|
||||||
|
return [...globals.vaParmapProblems, ...this.vaRuntimeWarnings]
|
||||||
|
}
|
||||||
|
/**
|
||||||
|
* The warnings this instance discovers for itself, i.e. everything except the
|
||||||
|
* parse-time problems - those are logged once at app init (`app.component`),
|
||||||
|
* so logging them again here would double every line in the console.
|
||||||
|
*/
|
||||||
|
get vaRuntimeWarnings(): string[] {
|
||||||
|
return [
|
||||||
|
...this.vaUnresolvedMappings,
|
||||||
|
...this.vaUnmatchedMappings,
|
||||||
|
...this.vaSameColumnMappings,
|
||||||
|
...this.vaTransformProblems
|
||||||
|
]
|
||||||
|
}
|
||||||
/**
|
/**
|
||||||
* VA filter mode. Live = auto-apply each VA change; confirm = stage the change
|
* VA filter mode. Live = auto-apply each VA change; confirm = stage the change
|
||||||
* and wait for the user to click Apply. Backed by globals (toggled by the
|
* and wait for the user to click Apply. Backed by globals (toggled by the
|
||||||
@@ -3602,7 +3643,7 @@ export class EditorComponent
|
|||||||
const msg = this.vaMessaging.latestMessage()
|
const msg = this.vaMessaging.latestMessage()
|
||||||
if (!msg) return false
|
if (!msg) return false
|
||||||
|
|
||||||
const clauses = this.vaFilter.buildInitialClauses(msg)
|
const clauses = this.vaFilter.buildInitialClauses(msg, this.vaParamMap)
|
||||||
if (clauses.length === 0) return false
|
if (clauses.length === 0) return false
|
||||||
|
|
||||||
try {
|
try {
|
||||||
@@ -5048,7 +5089,53 @@ export class EditorComponent
|
|||||||
|
|
||||||
this.updateVaFilterText(msg)
|
this.updateVaFilterText(msg)
|
||||||
|
|
||||||
const clauses = this.vaFilter.buildClauses(msg, labelToVar, this.cols)
|
// A `va_parmap` entry naming a column this table does not have is skipped by
|
||||||
|
// buildClauses. Report it (banner + console, once per distinct set) so the
|
||||||
|
// report author sees a misconfiguration instead of an unfiltered grid.
|
||||||
|
this.vaUnresolvedMappings = this.vaFilter.unresolvedMappings(
|
||||||
|
msg,
|
||||||
|
this.vaParamMap,
|
||||||
|
this.cols,
|
||||||
|
this.libds
|
||||||
|
)
|
||||||
|
// A key naming no parameter VA has ever sent is a typo on the parameter
|
||||||
|
// side: the mapping never applies. Only judged once parameters are actually
|
||||||
|
// arriving, since VA sends just the parameters its query uses.
|
||||||
|
this.vaUnmatchedMappings = this.vaFilter.unmatchedMappings(
|
||||||
|
this.vaParamMap,
|
||||||
|
this.vaMessaging.seenParamKeys,
|
||||||
|
this.vaMessaging.lastMessageParamsStable
|
||||||
|
)
|
||||||
|
// Two parameters filtering one column are AND-ed, so differing values leave
|
||||||
|
// an empty grid. Reported, not rewritten: turning that AND into an OR is a
|
||||||
|
// filter-semantics decision, taken in buildClauses and not in a diagnostic.
|
||||||
|
this.vaSameColumnMappings = this.vaFilter.sameColumnMappings(
|
||||||
|
msg,
|
||||||
|
this.vaParamMap,
|
||||||
|
this.cols
|
||||||
|
)
|
||||||
|
// A transform that cannot run here (char column, value that is not a date) is
|
||||||
|
// skipped by the clause builder, so the banner has to say why.
|
||||||
|
this.vaTransformProblems = this.vaFilter.unsupportedTransforms(
|
||||||
|
msg,
|
||||||
|
this.vaParamMap,
|
||||||
|
this.cols
|
||||||
|
)
|
||||||
|
const warnings = this.vaRuntimeWarnings.join(' | ')
|
||||||
|
if (warnings !== this.vaWarningsReported) {
|
||||||
|
this.vaWarningsReported = warnings
|
||||||
|
for (const warning of this.vaRuntimeWarnings) {
|
||||||
|
// eslint-disable-next-line no-console
|
||||||
|
console.warn('Data Controller: ' + warning)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const clauses = this.vaFilter.buildClauses(
|
||||||
|
msg,
|
||||||
|
labelToVar,
|
||||||
|
this.cols,
|
||||||
|
this.vaParamMap
|
||||||
|
)
|
||||||
const signature = this.vaFilter.signature(clauses)
|
const signature = this.vaFilter.signature(clauses)
|
||||||
|
|
||||||
if (signature === this.vaMessaging.filterSignature) {
|
if (signature === this.vaMessaging.filterSignature) {
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
import { VaFilterService } from './va-filter.service'
|
import { VaFilterService } from './va-filter.service'
|
||||||
|
import { VaParmapTarget } from '../shared/utils/parse-va-parmap'
|
||||||
import { VaMessage } from './va-messaging.service'
|
import { VaMessage } from './va-messaging.service'
|
||||||
import { HelperService } from './helper.service'
|
import { HelperService } from './helper.service'
|
||||||
|
|
||||||
@@ -14,6 +15,7 @@ function msg(parameters: any[]): VaMessage {
|
|||||||
|
|
||||||
describe('VaFilterService', () => {
|
describe('VaFilterService', () => {
|
||||||
let service: VaFilterService
|
let service: VaFilterService
|
||||||
|
let helperService: HelperService
|
||||||
|
|
||||||
// DC column metadata used by the metadata path. cellValidation[0] is the
|
// DC column metadata used by the metadata path. cellValidation[0] is the
|
||||||
// delete column and must be skipped by buildLabelToVarMap.
|
// delete column and must be skipped by buildLabelToVarMap.
|
||||||
@@ -42,7 +44,8 @@ describe('VaFilterService', () => {
|
|||||||
]
|
]
|
||||||
|
|
||||||
beforeEach(() => {
|
beforeEach(() => {
|
||||||
service = new VaFilterService(new HelperService({} as any))
|
helperService = new HelperService({} as any)
|
||||||
|
service = new VaFilterService(helperService)
|
||||||
})
|
})
|
||||||
|
|
||||||
const labelToVar = () =>
|
const labelToVar = () =>
|
||||||
@@ -128,6 +131,41 @@ describe('VaFilterService', () => {
|
|||||||
expect(build([{ label: 'SOME_DATE', value: '2025/03/31' }])).toHaveSize(1)
|
expect(build([{ label: 'SOME_DATE', value: '2025/03/31' }])).toHaveSize(1)
|
||||||
})
|
})
|
||||||
|
|
||||||
|
it('parses word dates, in either order and with an optional comma', () => {
|
||||||
|
const expected = "'01JAN2026'd"
|
||||||
|
// The reported shape: a date parameter displayed as WORDDATE.
|
||||||
|
expect(
|
||||||
|
build([{ label: 'SOME_DATE', value: 'January 1, 2026' }])[0].RAW_VALUE
|
||||||
|
).toBe(expected)
|
||||||
|
expect(
|
||||||
|
build([{ label: 'SOME_DATE', value: 'Jan 1, 2026' }])[0].RAW_VALUE
|
||||||
|
).toBe(expected)
|
||||||
|
expect(
|
||||||
|
build([{ label: 'SOME_DATE', value: 'Jan. 1, 2026' }])[0].RAW_VALUE
|
||||||
|
).toBe(expected)
|
||||||
|
expect(
|
||||||
|
build([{ label: 'SOME_DATE', value: '1 January 2026' }])[0].RAW_VALUE
|
||||||
|
).toBe(expected)
|
||||||
|
expect(
|
||||||
|
build([{ label: 'SOME_DATE', value: '1-Jan-2026' }])[0].RAW_VALUE
|
||||||
|
).toBe(expected)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('matches a spelled-out month case-insensitively, and keeps DATE9 working', () => {
|
||||||
|
expect(
|
||||||
|
build([{ label: 'SOME_DATE', value: 'DECEMBER 31, 2025' }])[0].RAW_VALUE
|
||||||
|
).toBe("'31DEC2025'd")
|
||||||
|
expect(
|
||||||
|
build([{ label: 'SOME_DATE', value: '01AUG1962' }])[0].RAW_VALUE
|
||||||
|
).toBe("'01AUG1962'd")
|
||||||
|
})
|
||||||
|
|
||||||
|
it('still drops a month-name value that has no day', () => {
|
||||||
|
// A month and year alone is not a date - accepted shapes always carry a
|
||||||
|
// day, so this must not silently become the first of the month.
|
||||||
|
expect(build([{ label: 'SOME_DATE', value: 'January 2026' }])).toEqual([])
|
||||||
|
})
|
||||||
|
|
||||||
it('resolves ambiguous slash dates day-first (never US), disambiguating by value', () => {
|
it('resolves ambiguous slash dates day-first (never US), disambiguating by value', () => {
|
||||||
const iso = build([{ label: 'SOME_DATE', value: '2025-03-31' }])[0]
|
const iso = build([{ label: 'SOME_DATE', value: '2025-03-31' }])[0]
|
||||||
.RAW_VALUE
|
.RAW_VALUE
|
||||||
@@ -199,7 +237,694 @@ describe('VaFilterService', () => {
|
|||||||
})
|
})
|
||||||
})
|
})
|
||||||
|
|
||||||
|
describe('buildClauses with an explicit va_parmap', () => {
|
||||||
|
const build = (parameters: any[], paramMap: Map<string, VaParmapTarget>) =>
|
||||||
|
service.buildClauses(msg(parameters), labelToVar(), cols, paramMap)
|
||||||
|
|
||||||
|
it('overrides the same-name match, including the value formatting', () => {
|
||||||
|
// The VA label IS a DC column here (SOME_CHAR), but the mapping points at
|
||||||
|
// a different one - the numeric column - so the value is formatted numeric.
|
||||||
|
const map = new Map([['some_char', { column: 'SOME_NUM' }]])
|
||||||
|
expect(build([{ label: 'SOME_CHAR', value: '10' }], map)).toEqual([
|
||||||
|
{
|
||||||
|
GROUP_LOGIC: 'AND',
|
||||||
|
SUBGROUP_LOGIC: 'AND',
|
||||||
|
SUBGROUP_ID: 0,
|
||||||
|
VARIABLE_NM: 'SOME_NUM',
|
||||||
|
OPERATOR_NM: '=',
|
||||||
|
RAW_VALUE: '10'
|
||||||
|
}
|
||||||
|
])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('lets two differently-named params filter the same column', () => {
|
||||||
|
// The multi-tab case: one DC instance per report tab, each with its own
|
||||||
|
// VA parameter, both mapping to the same DC column.
|
||||||
|
const map = new Map([
|
||||||
|
['region_tab1', { column: 'SOME_CHAR' }],
|
||||||
|
['region_tab2', { column: 'SOME_CHAR' }]
|
||||||
|
])
|
||||||
|
const clauses = build(
|
||||||
|
[
|
||||||
|
{ label: 'REGION_TAB1', value: 'north' },
|
||||||
|
{ label: 'REGION_TAB2', value: 'south' }
|
||||||
|
],
|
||||||
|
map
|
||||||
|
)
|
||||||
|
expect(clauses.map((c) => [c.VARIABLE_NM, c.RAW_VALUE])).toEqual([
|
||||||
|
['SOME_CHAR', "'north'"],
|
||||||
|
['SOME_CHAR', "'south'"]
|
||||||
|
])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('matches the parameter label only, not its internal name', () => {
|
||||||
|
// The internal name is the report's own bookkeeping. Matching it as well
|
||||||
|
// meant one parameter answered to two keys, so an entry written against the
|
||||||
|
// name and one against the label silently fought over it.
|
||||||
|
const map = new Map([['pr273', { column: 'SOME_NUM' }]])
|
||||||
|
expect(
|
||||||
|
build([{ name: 'pr273', label: 'Tab_1_Value', value: '7' }], map)
|
||||||
|
).toEqual([])
|
||||||
|
// The label still matches, case-insensitively.
|
||||||
|
expect(
|
||||||
|
build(
|
||||||
|
[{ name: 'pr273', label: 'Tab_1_Value', value: 'x' }],
|
||||||
|
new Map([['tab_1_value', { column: 'SOME_CHAR' }]])
|
||||||
|
)[0].VARIABLE_NM
|
||||||
|
).toBe('SOME_CHAR')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('uses the column case from the loaded metadata', () => {
|
||||||
|
const map = new Map([['region_tab1', { column: 'some_char' }]])
|
||||||
|
expect(
|
||||||
|
build([{ label: 'REGION_TAB1', value: 'x' }], map)[0].VARIABLE_NM
|
||||||
|
).toBe('SOME_CHAR')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('leaves unmapped params on the same-name path', () => {
|
||||||
|
const map = new Map([['region_tab1', { column: 'SOME_NUM' }]])
|
||||||
|
const clauses = build(
|
||||||
|
[
|
||||||
|
{ label: 'REGION_TAB1', value: '10' },
|
||||||
|
{ label: 'SOME_CHAR', value: 'x' }
|
||||||
|
],
|
||||||
|
map
|
||||||
|
)
|
||||||
|
expect(clauses.map((c) => [c.VARIABLE_NM, c.RAW_VALUE])).toEqual([
|
||||||
|
['SOME_NUM', '10'],
|
||||||
|
['SOME_CHAR', "'x'"]
|
||||||
|
])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('skips a mapping whose column is not in this table (no fallback)', () => {
|
||||||
|
// Falling back to the same-name match would silently apply the filter the
|
||||||
|
// mapping exists to override, so the param is dropped instead.
|
||||||
|
const map = new Map([['some_char', { column: 'NOT_IN_THIS_TABLE' }]])
|
||||||
|
expect(build([{ label: 'SOME_CHAR', value: 'x' }], map)).toEqual([])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('ANDs two parameters that map to the same column - it does not union them', () => {
|
||||||
|
// What a report gets when one column's two values arrive in two
|
||||||
|
// parameters: every clause is AND-ed, so the result is the INTERSECTION.
|
||||||
|
// Two different dates therefore match no rows at all rather than both
|
||||||
|
// dates' rows - a single parameter carrying both values is the shape that
|
||||||
|
// ORs, via `IN (...)`.
|
||||||
|
const map = new Map([
|
||||||
|
['param_a', { column: 'SOME_DATE' }],
|
||||||
|
['param_b', { column: 'SOME_DATE' }]
|
||||||
|
])
|
||||||
|
const one = build([{ label: 'PARAM_A', value: '01JUL1960' }], map)[0]
|
||||||
|
const other = build([{ label: 'PARAM_B', value: '01JAN1961' }], map)[0]
|
||||||
|
const clauses = build(
|
||||||
|
[
|
||||||
|
{ label: 'PARAM_A', value: '01JUL1960' },
|
||||||
|
{ label: 'PARAM_B', value: '01JAN1961' }
|
||||||
|
],
|
||||||
|
map
|
||||||
|
)
|
||||||
|
expect(one.RAW_VALUE).not.toBe(other.RAW_VALUE)
|
||||||
|
expect(clauses).toHaveSize(2)
|
||||||
|
expect(clauses.map((c) => c.RAW_VALUE)).toEqual([
|
||||||
|
one.RAW_VALUE,
|
||||||
|
other.RAW_VALUE
|
||||||
|
])
|
||||||
|
for (const clause of clauses) {
|
||||||
|
expect(clause.VARIABLE_NM).toBe('SOME_DATE')
|
||||||
|
expect(clause.OPERATOR_NM).toBe('=')
|
||||||
|
expect(clause.GROUP_LOGIC).toBe('AND')
|
||||||
|
expect(clause.SUBGROUP_LOGIC).toBe('AND')
|
||||||
|
expect(clause.SUBGROUP_ID).toBe(0)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
it('applies a transform to the clause it builds', () => {
|
||||||
|
// ADD(QUARTER_START,-1) is the reported requirement: the selected row plus
|
||||||
|
// the row for the previous quarter, as one IN clause so the AND rule is
|
||||||
|
// untouched. 2026-10-01 = 24380, 2026-07-01 = 24288 in SAS days.
|
||||||
|
const map = new Map([
|
||||||
|
[
|
||||||
|
'selected',
|
||||||
|
{
|
||||||
|
column: 'SOME_DATE',
|
||||||
|
transform: {
|
||||||
|
op: 'ADD' as const,
|
||||||
|
anchor: 'QUARTER_START' as const,
|
||||||
|
offset: -1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
])
|
||||||
|
const [clause] = build([{ label: 'SELECTED', value: '01OCT2026' }], map)
|
||||||
|
expect(clause.VARIABLE_NM).toBe('SOME_DATE')
|
||||||
|
expect(clause.OPERATOR_NM).toBe('IN')
|
||||||
|
expect(clause.RAW_VALUE).toBe('(24380,24288)')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('applies a RANGE as BETWEEN', () => {
|
||||||
|
// RANGE(QUARTER,-1) for a mid-quarter value: 2026-04-01..2026-06-30
|
||||||
|
const map = new Map([
|
||||||
|
[
|
||||||
|
'selected',
|
||||||
|
{
|
||||||
|
column: 'SOME_DATE',
|
||||||
|
transform: {
|
||||||
|
op: 'RANGE' as const,
|
||||||
|
period: 'QUARTER' as const,
|
||||||
|
offset: -1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
])
|
||||||
|
const [clause] = build([{ label: 'SELECTED', value: '1962-08-15' }], map)
|
||||||
|
expect(clause.OPERATOR_NM).toBe('BETWEEN')
|
||||||
|
expect(clause.RAW_VALUE).toContain(' AND ')
|
||||||
|
expect(clause.RAW_VALUE.split(' AND ')).toHaveSize(2)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('keeps equality when the value already sits on the anchor', () => {
|
||||||
|
const map = new Map([
|
||||||
|
[
|
||||||
|
'selected',
|
||||||
|
{
|
||||||
|
column: 'SOME_DATE',
|
||||||
|
transform: {
|
||||||
|
op: 'ADD' as const,
|
||||||
|
anchor: 'QUARTER_START' as const,
|
||||||
|
offset: 0
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
])
|
||||||
|
const [clause] = build([{ label: 'SELECTED', value: '01OCT2026' }], map)
|
||||||
|
expect(clause.OPERATOR_NM).toBe('=')
|
||||||
|
expect(clause.RAW_VALUE).toBe('24380')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('drops a transform on a column it cannot apply to', () => {
|
||||||
|
// A character column has no dates to shift; the parameter is skipped rather
|
||||||
|
// than filtered on the raw value the author did not see.
|
||||||
|
const map = new Map([
|
||||||
|
[
|
||||||
|
'selected',
|
||||||
|
{
|
||||||
|
column: 'SOME_CHAR',
|
||||||
|
transform: {
|
||||||
|
op: 'ADD' as const,
|
||||||
|
anchor: 'QUARTER_START' as const,
|
||||||
|
offset: -1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
])
|
||||||
|
expect(build([{ label: 'SELECTED', value: 'x' }], map)).toEqual([])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('adds nothing when the value is unparseable as a date', () => {
|
||||||
|
const map = new Map([
|
||||||
|
[
|
||||||
|
'selected',
|
||||||
|
{
|
||||||
|
column: 'SOME_DATE',
|
||||||
|
transform: {
|
||||||
|
op: 'ADD' as const,
|
||||||
|
anchor: 'QUARTER_START' as const,
|
||||||
|
offset: -1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
])
|
||||||
|
expect(build([{ label: 'SELECTED', value: 'not a date' }], map)).toEqual(
|
||||||
|
[]
|
||||||
|
)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('round-trips a bare SAS day number unchanged', () => {
|
||||||
|
// VA may send the SAS internal value rather than a formatted string. The
|
||||||
|
// Date a transform works from is local, so the number must come back out of
|
||||||
|
// the clause identical - a UTC-built Date shifts a day west of UTC.
|
||||||
|
const map = new Map([
|
||||||
|
[
|
||||||
|
'selected',
|
||||||
|
{
|
||||||
|
column: 'SOME_DATE',
|
||||||
|
transform: {
|
||||||
|
op: 'ADD' as const,
|
||||||
|
anchor: 'QUARTER_START' as const,
|
||||||
|
offset: -1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
])
|
||||||
|
const [clause] = build([{ label: 'SELECTED', value: 24380 }], map)
|
||||||
|
expect(clause.RAW_VALUE).toBe('(24380,24288)')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('reports a range mapping fed more than one value', () => {
|
||||||
|
const map = new Map([
|
||||||
|
[
|
||||||
|
'selected',
|
||||||
|
{
|
||||||
|
column: 'SOME_DATE',
|
||||||
|
transform: {
|
||||||
|
op: 'RANGE' as const,
|
||||||
|
period: 'QUARTER' as const,
|
||||||
|
offset: -1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
])
|
||||||
|
const problems = service.unsupportedTransforms(
|
||||||
|
msg([{ label: 'SELECTED', value: ['01APR2026', '01JUL2026'] }]),
|
||||||
|
map,
|
||||||
|
[{ NAME: 'SOME_DATE', DDTYPE: 'DATE', FMTNAME: 'DATE9' }]
|
||||||
|
)
|
||||||
|
expect(problems).toHaveSize(1)
|
||||||
|
expect(problems[0]).toContain('is a range')
|
||||||
|
expect(problems[0]).toContain('2 values')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('refuses a range fed more than one value, rather than picking one', () => {
|
||||||
|
// Two values, one of which does not parse: building a BETWEEN from the
|
||||||
|
// survivor would silently filter one period when VA selected two.
|
||||||
|
const map = new Map([
|
||||||
|
[
|
||||||
|
'selected',
|
||||||
|
{
|
||||||
|
column: 'SOME_DATE',
|
||||||
|
transform: {
|
||||||
|
op: 'RANGE' as const,
|
||||||
|
period: 'QUARTER' as const,
|
||||||
|
offset: -1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
])
|
||||||
|
expect(
|
||||||
|
build([{ label: 'SELECTED', value: ['01APR2026', 'not a date'] }], map)
|
||||||
|
).toEqual([])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('never lets a non-finite date reach the clause', () => {
|
||||||
|
// convertJsDateToSasDate returning NaN would otherwise put a literal NaN in
|
||||||
|
// the WHERE clause.
|
||||||
|
const map = new Map([
|
||||||
|
[
|
||||||
|
'selected',
|
||||||
|
{
|
||||||
|
column: 'SOME_DATE',
|
||||||
|
transform: {
|
||||||
|
op: 'ADD' as const,
|
||||||
|
anchor: 'QUARTER_START' as const,
|
||||||
|
offset: -1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
])
|
||||||
|
spyOn(helperService, 'convertJsDateToSasDate').and.returnValue(NaN as any)
|
||||||
|
expect(build([{ label: 'SELECTED', value: '01OCT2026' }], map)).toEqual(
|
||||||
|
[]
|
||||||
|
)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('reports a partly unparseable value set instead of covering the rest', () => {
|
||||||
|
// One good value and one that is not a date: the clause still filters on
|
||||||
|
// the good one, so the author has to be told the other was dropped.
|
||||||
|
const map = new Map([
|
||||||
|
[
|
||||||
|
'selected',
|
||||||
|
{
|
||||||
|
column: 'SOME_DATE',
|
||||||
|
transform: {
|
||||||
|
op: 'ADD' as const,
|
||||||
|
anchor: 'QUARTER_START' as const,
|
||||||
|
offset: -1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
])
|
||||||
|
const problems = service.unsupportedTransforms(
|
||||||
|
msg([{ label: 'SELECTED', value: ['01OCT2026', 'not a date'] }]),
|
||||||
|
map,
|
||||||
|
cols
|
||||||
|
)
|
||||||
|
expect(problems).toHaveSize(1)
|
||||||
|
expect(problems[0]).toContain('1 of its 2 values was not a date')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('reports a range holding a value that does not parse', () => {
|
||||||
|
// Counted on what VA sent, not on what parsed: otherwise the second value
|
||||||
|
// vanishes and the range looks like a clean single-value case.
|
||||||
|
const map = new Map([
|
||||||
|
[
|
||||||
|
'selected',
|
||||||
|
{
|
||||||
|
column: 'SOME_DATE',
|
||||||
|
transform: {
|
||||||
|
op: 'RANGE' as const,
|
||||||
|
period: 'QUARTER' as const,
|
||||||
|
offset: -1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
])
|
||||||
|
const problems = service.unsupportedTransforms(
|
||||||
|
msg([{ label: 'SELECTED', value: ['01APR2026', 'not a date'] }]),
|
||||||
|
map,
|
||||||
|
cols
|
||||||
|
)
|
||||||
|
expect(problems).toHaveSize(1)
|
||||||
|
expect(problems[0]).toContain('is a range')
|
||||||
|
expect(problems[0]).toContain('2 values')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('applies a transform on the deferred initial load too', () => {
|
||||||
|
const map = new Map([
|
||||||
|
[
|
||||||
|
'selected',
|
||||||
|
{
|
||||||
|
column: 'SOME_DATE',
|
||||||
|
transform: {
|
||||||
|
op: 'ADD' as const,
|
||||||
|
anchor: 'QUARTER_START' as const,
|
||||||
|
offset: -1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
])
|
||||||
|
const [clause] = service.buildInitialClauses(
|
||||||
|
msg([{ name: 'pr1', label: 'SELECTED', value: '01OCT2026' }]),
|
||||||
|
map
|
||||||
|
)
|
||||||
|
expect(clause.VARIABLE_NM).toBe('SOME_DATE')
|
||||||
|
expect(clause.RAW_VALUE).toBe('(24380,24288)')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('behaves exactly as before with an empty map', () => {
|
||||||
|
expect(
|
||||||
|
service.buildClauses(
|
||||||
|
msg([{ label: 'SOME_CHAR', value: 'x' }]),
|
||||||
|
labelToVar(),
|
||||||
|
cols
|
||||||
|
)
|
||||||
|
).toEqual(build([{ label: 'SOME_CHAR', value: 'x' }], new Map()))
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
describe('unresolvedMappings (what the report author is warned about)', () => {
|
||||||
|
const report = (
|
||||||
|
parameters: any[],
|
||||||
|
map: Map<string, VaParmapTarget>,
|
||||||
|
libds?: string
|
||||||
|
) => service.unresolvedMappings(msg(parameters), map, cols, libds)
|
||||||
|
|
||||||
|
it('reports a mapping whose column is not in the loaded table', () => {
|
||||||
|
expect(
|
||||||
|
report(
|
||||||
|
[{ label: 'REGION_TAB1', value: 'x' }],
|
||||||
|
new Map([['region_tab1', { column: 'NOT_A_COLUMN' }]]),
|
||||||
|
'DC_JSLIB.MPE_X_TEST'
|
||||||
|
)
|
||||||
|
).toEqual([
|
||||||
|
'VA parameter "REGION_TAB1" is mapped to NOT_A_COLUMN, which is not a column of DC_JSLIB.MPE_X_TEST - the filter was ignored.'
|
||||||
|
])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('says "this table" when the table name is not supplied', () => {
|
||||||
|
expect(
|
||||||
|
report(
|
||||||
|
[{ label: 'REGION_TAB1', value: 'x' }],
|
||||||
|
new Map([['region_tab1', { column: 'NOPE' }]])
|
||||||
|
)[0]
|
||||||
|
).toContain('is not a column of this table')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('reports nothing when the mapping resolves, or the param is unmapped', () => {
|
||||||
|
expect(
|
||||||
|
report(
|
||||||
|
[{ label: 'REGION_TAB1', value: 'x' }],
|
||||||
|
new Map([['region_tab1', { column: 'some_char' }]])
|
||||||
|
)
|
||||||
|
).toEqual([])
|
||||||
|
expect(report([{ label: 'SOME_CHAR', value: 'x' }], new Map())).toEqual(
|
||||||
|
[]
|
||||||
|
)
|
||||||
|
// A mapping whose parameter this message does not carry is not an error.
|
||||||
|
expect(
|
||||||
|
report([], new Map([['region_tab1', { column: 'NOPE' }]]))
|
||||||
|
).toEqual([])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('reports each mapping once, not once per message parameter', () => {
|
||||||
|
const problems = report(
|
||||||
|
[
|
||||||
|
{ label: 'REGION_TAB1', value: '1' },
|
||||||
|
{ name: 'pr99', label: 'REGION_TAB1', value: '2' }
|
||||||
|
],
|
||||||
|
new Map([['region_tab1', { column: 'NOPE' }]])
|
||||||
|
)
|
||||||
|
expect(problems).toHaveSize(1)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('treats an unloaded column list as "not known yet", not as a bad mapping', () => {
|
||||||
|
// While the grid is still loading there is no metadata to judge against, so
|
||||||
|
// reporting would flash the banner on every load.
|
||||||
|
expect(
|
||||||
|
service.unresolvedMappings(
|
||||||
|
msg([{ label: 'REGION_TAB1', value: 'x' }]),
|
||||||
|
new Map([['region_tab1', { column: 'NOPE' }]]),
|
||||||
|
[],
|
||||||
|
'DC_JSLIB.MPE_X_TEST'
|
||||||
|
)
|
||||||
|
).toEqual([])
|
||||||
|
expect(
|
||||||
|
service.unresolvedMappings(
|
||||||
|
msg([{ label: 'REGION_TAB1', value: 'x' }]),
|
||||||
|
new Map([['region_tab1', { column: 'NOPE' }]]),
|
||||||
|
undefined as any,
|
||||||
|
'DC_JSLIB.MPE_X_TEST'
|
||||||
|
)
|
||||||
|
).toEqual([])
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
describe('sameColumnMappings (two parameters, one column)', () => {
|
||||||
|
it('reports an AND-ed pair, naming both parameters and the column', () => {
|
||||||
|
const map = new Map([
|
||||||
|
['dropdown_tab1', { column: 'SOME_DROPDOWN' }],
|
||||||
|
['dropdown_tab2', { column: 'SOME_DROPDOWN' }]
|
||||||
|
])
|
||||||
|
const problems = service.sameColumnMappings(
|
||||||
|
msg([
|
||||||
|
{ label: 'DROPDOWN_TAB1', value: 'Option 1' },
|
||||||
|
{ label: 'DROPDOWN_TAB2', value: 'Option 2' }
|
||||||
|
]),
|
||||||
|
map
|
||||||
|
)
|
||||||
|
expect(problems).toHaveSize(1)
|
||||||
|
expect(problems[0]).toContain('"DROPDOWN_TAB1" and "DROPDOWN_TAB2"')
|
||||||
|
expect(problems[0]).toContain('all filter SOME_DROPDOWN')
|
||||||
|
expect(problems[0]).toContain('combined with AND')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('says nothing for one parameter per column', () => {
|
||||||
|
const map = new Map([
|
||||||
|
['dropdown_tab1', { column: 'SOME_DROPDOWN' }],
|
||||||
|
['num_tab1', { column: 'SOME_NUM' }]
|
||||||
|
])
|
||||||
|
expect(
|
||||||
|
service.sameColumnMappings(
|
||||||
|
msg([
|
||||||
|
{ label: 'DROPDOWN_TAB1', value: 'Option 1' },
|
||||||
|
{ label: 'NUM_TAB1', value: '1' }
|
||||||
|
]),
|
||||||
|
map
|
||||||
|
)
|
||||||
|
).toEqual([])
|
||||||
|
expect(
|
||||||
|
service.sameColumnMappings(
|
||||||
|
msg([{ label: 'DROPDOWN_TAB1', value: 'Option 1' }]),
|
||||||
|
map
|
||||||
|
)
|
||||||
|
).toEqual([])
|
||||||
|
expect(service.sameColumnMappings(msg([]), map)).toEqual([])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('says nothing when the AND-ed parameters carry the same value', () => {
|
||||||
|
// col = 'x' AND col = 'x' filters exactly as one clause does, so a banner
|
||||||
|
// claiming "different values will show no rows" would be wrong.
|
||||||
|
const map = new Map([
|
||||||
|
['dropdown_tab1', { column: 'SOME_CHAR' }],
|
||||||
|
['dropdown_tab2', { column: 'SOME_CHAR' }]
|
||||||
|
])
|
||||||
|
expect(
|
||||||
|
service.sameColumnMappings(
|
||||||
|
msg([
|
||||||
|
{ label: 'DROPDOWN_TAB1', value: 'x' },
|
||||||
|
{ label: 'DROPDOWN_TAB2', value: 'x' }
|
||||||
|
]),
|
||||||
|
map,
|
||||||
|
cols
|
||||||
|
)
|
||||||
|
).toEqual([])
|
||||||
|
// One differing value and the intersection can be empty: reported.
|
||||||
|
const problems = service.sameColumnMappings(
|
||||||
|
msg([
|
||||||
|
{ label: 'DROPDOWN_TAB1', value: 'x' },
|
||||||
|
{ label: 'DROPDOWN_TAB2', value: 'y' }
|
||||||
|
]),
|
||||||
|
map,
|
||||||
|
cols
|
||||||
|
)
|
||||||
|
expect(problems).toHaveSize(1)
|
||||||
|
expect(problems[0]).toContain('"DROPDOWN_TAB1" and "DROPDOWN_TAB2"')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('leaves a column the table does not have to unresolvedMappings', () => {
|
||||||
|
// One cause, one line: the column is missing, so the AND story is moot.
|
||||||
|
const map = new Map([
|
||||||
|
['a', { column: 'NOPE' }],
|
||||||
|
['b', { column: 'NOPE' }]
|
||||||
|
])
|
||||||
|
const cols = [
|
||||||
|
{ LIBREF: 'DC_JSLIB', TABLE: 'MPE_X_TEST', VARIABLE: 'SOME_CHAR' }
|
||||||
|
]
|
||||||
|
expect(
|
||||||
|
service.sameColumnMappings(
|
||||||
|
msg([
|
||||||
|
{ label: 'A', value: '1' },
|
||||||
|
{ label: 'B', value: '2' }
|
||||||
|
]),
|
||||||
|
map,
|
||||||
|
cols
|
||||||
|
)
|
||||||
|
).toEqual([])
|
||||||
|
// Without metadata the pair is still reported: the AND semantics hold
|
||||||
|
// whatever the table turns out to be.
|
||||||
|
expect(
|
||||||
|
service.sameColumnMappings(
|
||||||
|
msg([
|
||||||
|
{ label: 'A', value: '1' },
|
||||||
|
{ label: 'B', value: '2' }
|
||||||
|
]),
|
||||||
|
map
|
||||||
|
)
|
||||||
|
).toHaveSize(1)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('reports one line per contended column', () => {
|
||||||
|
const map = new Map([
|
||||||
|
['a', { column: 'SOME_DROPDOWN' }],
|
||||||
|
['b', { column: 'SOME_DROPDOWN' }],
|
||||||
|
['c', { column: 'SOME_NUM' }],
|
||||||
|
['d', { column: 'SOME_NUM' }]
|
||||||
|
])
|
||||||
|
const problems = service.sameColumnMappings(
|
||||||
|
msg([
|
||||||
|
{ label: 'A', value: '1' },
|
||||||
|
{ label: 'B', value: '2' },
|
||||||
|
{ label: 'C', value: '3' },
|
||||||
|
{ label: 'D', value: '4' }
|
||||||
|
]),
|
||||||
|
map
|
||||||
|
)
|
||||||
|
expect(problems).toHaveSize(2)
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
describe('unmatchedMappings (parameter-side typo)', () => {
|
||||||
|
const seen = (...keys: string[]) => new Set(keys)
|
||||||
|
|
||||||
|
it('reports a key that no message has named', () => {
|
||||||
|
expect(
|
||||||
|
service.unmatchedMappings(
|
||||||
|
new Map([['dropdown_tab1', { column: 'SOME_CHAR' }]]),
|
||||||
|
seen('dropdown_tab_1', 'pr1'),
|
||||||
|
true
|
||||||
|
)
|
||||||
|
).toEqual([
|
||||||
|
'This object\'s va_parmap names VA parameter "dropdown_tab1", which VA is not sending - check the spelling.'
|
||||||
|
])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('says nothing until a parameter has actually been seen', () => {
|
||||||
|
// Without evidence that this object receives parameters at all, an absent
|
||||||
|
// key proves nothing - VA sends only the parameters its query uses.
|
||||||
|
expect(
|
||||||
|
service.unmatchedMappings(
|
||||||
|
new Map([['dropdown_tab1', { column: 'SOME_CHAR' }]]),
|
||||||
|
seen()
|
||||||
|
)
|
||||||
|
).toEqual([])
|
||||||
|
expect(
|
||||||
|
service.unmatchedMappings(
|
||||||
|
new Map([['dropdown_tab1', { column: 'SOME_CHAR' }]]),
|
||||||
|
undefined
|
||||||
|
)
|
||||||
|
).toEqual([])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('says nothing when every key has been seen', () => {
|
||||||
|
expect(
|
||||||
|
service.unmatchedMappings(
|
||||||
|
new Map([['dropdown_tab1', { column: 'SOME_CHAR' }]]),
|
||||||
|
seen('dropdown_tab1'),
|
||||||
|
true
|
||||||
|
)
|
||||||
|
).toEqual([])
|
||||||
|
expect(
|
||||||
|
service.unmatchedMappings(new Map(), seen('anything'), true)
|
||||||
|
).toEqual([])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('says nothing while the parameter set is still growing', () => {
|
||||||
|
// A message that introduces a new parameter name means this object's set
|
||||||
|
// has not settled, so an absent key may arrive in the next message.
|
||||||
|
expect(
|
||||||
|
service.unmatchedMappings(
|
||||||
|
new Map([['dropdown_tab1', { column: 'SOME_CHAR' }]]),
|
||||||
|
seen('pr1', 'pr2'),
|
||||||
|
false
|
||||||
|
)
|
||||||
|
).toEqual([])
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
describe('buildInitialClauses (no metadata, infer kind from value)', () => {
|
describe('buildInitialClauses (no metadata, infer kind from value)', () => {
|
||||||
|
it('uses an explicit mapping instead of the label', () => {
|
||||||
|
// The deferred first load has no column metadata, so a mapped column is
|
||||||
|
// used as given and the SAS kind still comes from the value shape.
|
||||||
|
const map = new Map([
|
||||||
|
['region_tab1', { column: 'REGION' }],
|
||||||
|
['amt_tab1', { column: 'AMOUNT' }]
|
||||||
|
])
|
||||||
|
const clauses = service.buildInitialClauses(
|
||||||
|
msg([
|
||||||
|
{ name: 'pr1', label: 'REGION_TAB1', dataType: 'string', value: 'n' },
|
||||||
|
{ name: 'pr2', label: 'AMT_TAB1', dataType: 'number', value: '10' }
|
||||||
|
]),
|
||||||
|
map
|
||||||
|
)
|
||||||
|
expect(clauses.map((c) => [c.VARIABLE_NM, c.RAW_VALUE])).toEqual([
|
||||||
|
['REGION', "'n'"],
|
||||||
|
['AMOUNT', '10']
|
||||||
|
])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('falls back to the label for params the map does not name', () => {
|
||||||
|
const map = new Map([['region_tab2', { column: 'REGION' }]])
|
||||||
|
const clauses = service.buildInitialClauses(
|
||||||
|
msg([
|
||||||
|
{ name: 'pr1', label: 'REGION_TAB1', value: 'n' },
|
||||||
|
{ name: 'pr2', label: 'REGION_TAB2', value: 's' }
|
||||||
|
]),
|
||||||
|
map
|
||||||
|
)
|
||||||
|
expect(clauses.map((c) => [c.VARIABLE_NM, c.RAW_VALUE])).toEqual([
|
||||||
|
['REGION_TAB1', "'n'"],
|
||||||
|
['REGION', "'s'"]
|
||||||
|
])
|
||||||
|
})
|
||||||
|
|
||||||
it('uses the VA label as the column and infers kinds by value shape', () => {
|
it('uses the VA label as the column and infers kinds by value shape', () => {
|
||||||
const clauses = service.buildInitialClauses(
|
const clauses = service.buildInitialClauses(
|
||||||
msg([
|
msg([
|
||||||
|
|||||||
@@ -2,6 +2,12 @@ import { Injectable } from '@angular/core'
|
|||||||
import { HelperService } from './helper.service'
|
import { HelperService } from './helper.service'
|
||||||
import { VaMessage } from './va-messaging.service'
|
import { VaMessage } from './va-messaging.service'
|
||||||
import { QueryClause } from '../models/TableData'
|
import { QueryClause } from '../models/TableData'
|
||||||
|
import { VaParmapTarget } from '../shared/utils/parse-va-parmap'
|
||||||
|
import {
|
||||||
|
applyVaTransform,
|
||||||
|
isRangeTransform,
|
||||||
|
VaTransform
|
||||||
|
} from '../shared/utils/va-parmap-transform'
|
||||||
|
|
||||||
/** SAS data-type kind used to format a VA filter value into a DC RAW_VALUE. */
|
/** SAS data-type kind used to format a VA filter value into a DC RAW_VALUE. */
|
||||||
type VaColumnKind = 'char' | 'numeric' | 'time' | 'date' | 'datetime'
|
type VaColumnKind = 'char' | 'numeric' | 'time' | 'date' | 'datetime'
|
||||||
@@ -54,11 +60,19 @@ export class VaFilterService {
|
|||||||
* Translates VA `parameters` into DC filter clauses. Each parameter is matched
|
* Translates VA `parameters` into DC filter clauses. Each parameter is matched
|
||||||
* to a DC column by label; a single value becomes `col = value`, multiple
|
* to a DC column by label; a single value becomes `col = value`, multiple
|
||||||
* values become `col IN (...)`. Values are formatted for the column's SAS kind.
|
* values become `col IN (...)`. Values are formatted for the column's SAS kind.
|
||||||
|
*
|
||||||
|
* `paramMap` (from the URL `va_parmap` parameter, see parseVaParmap) overrides
|
||||||
|
* that same-name resolution for the parameters it names, so two DC instances
|
||||||
|
* over one table can filter the same column from differently-named VA
|
||||||
|
* parameters. A mapped column must exist in `cols` - if it does not the
|
||||||
|
* parameter is skipped rather than falling back to the same-name match, which
|
||||||
|
* is exactly the behaviour the mapping exists to override.
|
||||||
*/
|
*/
|
||||||
buildClauses(
|
buildClauses(
|
||||||
msg: VaMessage,
|
msg: VaMessage,
|
||||||
labelToVar: Map<string, string>,
|
labelToVar: Map<string, string>,
|
||||||
cols: any[]
|
cols: any[],
|
||||||
|
paramMap?: Map<string, VaParmapTarget>
|
||||||
): QueryClause[] {
|
): QueryClause[] {
|
||||||
const clauses: QueryClause[] = []
|
const clauses: QueryClause[] = []
|
||||||
for (const param of msg.parameters || []) {
|
for (const param of msg.parameters || []) {
|
||||||
@@ -67,60 +81,194 @@ export class VaFilterService {
|
|||||||
.toString()
|
.toString()
|
||||||
.trim()
|
.trim()
|
||||||
.toLowerCase()
|
.toLowerCase()
|
||||||
// A VA param whose label isn't a DC column (beyond case, which is handled)
|
// An explicit URL mapping wins over the label->column lookup. The mapped
|
||||||
// simply doesn't map — skip it.
|
// column has to exist in THIS table, else the entry does not apply here
|
||||||
const varName = labelToVar.get(key)
|
// (and the same-name fallback would be the behaviour it exists to
|
||||||
|
// override).
|
||||||
|
const mapped = this.mappedColumn(param, paramMap)
|
||||||
|
const mappedCol = mapped ? this.findCol(mapped, cols) : undefined
|
||||||
|
if (mapped && !mappedCol) continue
|
||||||
|
// The table's own case, so the clause matches what the metadata build
|
||||||
|
// produces for a same-name match.
|
||||||
|
const varName: string = mappedCol
|
||||||
|
? mappedCol.NAME
|
||||||
|
: (labelToVar.get(key) ?? '')
|
||||||
if (!varName) continue
|
if (!varName) continue
|
||||||
|
|
||||||
const col = this.findCol(varName, cols)
|
const col = mappedCol ?? this.findCol(varName, cols)
|
||||||
const kind = this.columnKind(col)
|
const kind = this.columnKind(col)
|
||||||
const order = this.dateOrderFromFmt(col?.FMTNAME || col?.FORMAT)
|
const order = this.dateOrderFromFmt(col?.FMTNAME || col?.FORMAT)
|
||||||
const rawValues = Array.isArray(param.value) ? param.value : [param.value]
|
const built = this.buildClauseValueSet(
|
||||||
// Format/validate every value for the column's SAS type. Numeric columns
|
param,
|
||||||
// must receive a finite number (untrusted postMessage input is
|
kind,
|
||||||
// interpolated into a SAS WHERE clause, so a non-numeric value here would
|
order,
|
||||||
// be an injection vector); temporal -> SAS internal number; char ->
|
this.mappedTarget(param, paramMap)?.transform
|
||||||
// escaped. Invalid values are dropped.
|
)
|
||||||
const formatted = rawValues
|
if (!built) continue
|
||||||
.filter((v) => v !== undefined && v !== null && v !== '')
|
clauses.push({
|
||||||
.map((v) => this.formatValueByKind(v, kind, order))
|
GROUP_LOGIC: 'AND',
|
||||||
.filter((v): v is string => v !== null)
|
SUBGROUP_LOGIC: 'AND',
|
||||||
if (formatted.length === 0) continue
|
SUBGROUP_ID: 0,
|
||||||
clauses.push(this.clause(varName, formatted))
|
VARIABLE_NM: varName,
|
||||||
|
OPERATOR_NM: built.operator,
|
||||||
|
RAW_VALUE: built.rawValue
|
||||||
|
})
|
||||||
}
|
}
|
||||||
return clauses
|
return clauses
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The operator and raw value for one parameter, after any transform.
|
||||||
|
*
|
||||||
|
* Values are formatted for the column's SAS kind: numeric columns must receive
|
||||||
|
* a finite number (untrusted postMessage input is interpolated into a WHERE
|
||||||
|
* clause, so anything else would be an injection vector); temporal values
|
||||||
|
* become SAS internal numbers; char values are escaped. Invalid values are
|
||||||
|
* dropped, and a transform that cannot be applied to the column's kind yields
|
||||||
|
* nothing at all - the caller reports that separately rather than filtering on
|
||||||
|
* a value the author did not choose.
|
||||||
|
*/
|
||||||
|
private buildClauseValueSet(
|
||||||
|
param: any,
|
||||||
|
kind: VaColumnKind,
|
||||||
|
order: 'YMD' | 'DMY' | 'MDY' | undefined,
|
||||||
|
transform?: VaTransform
|
||||||
|
): { operator: '=' | 'IN' | 'BETWEEN'; rawValue: string } | null {
|
||||||
|
const rawValues = (
|
||||||
|
Array.isArray(param.value) ? param.value : [param.value]
|
||||||
|
).filter((v: any) => v !== undefined && v !== null && v !== '')
|
||||||
|
if (rawValues.length === 0) return null
|
||||||
|
|
||||||
|
if (!transform) {
|
||||||
|
const formatted = rawValues
|
||||||
|
.map((v: any) => this.formatValueByKind(v, kind, order))
|
||||||
|
.filter((v: string | null): v is string => v !== null)
|
||||||
|
if (formatted.length === 0) return null
|
||||||
|
return formatted.length === 1
|
||||||
|
? { operator: '=', rawValue: formatted[0] }
|
||||||
|
: { operator: 'IN', rawValue: `(${formatted.join(',')})` }
|
||||||
|
}
|
||||||
|
|
||||||
|
// Transforms resolve to dates, so only temporal columns can take one.
|
||||||
|
if (kind !== 'date' && kind !== 'datetime') return null
|
||||||
|
const unit = kind === 'datetime' ? 'seconds' : 'days'
|
||||||
|
const values = new Set<string>()
|
||||||
|
for (const raw of rawValues) {
|
||||||
|
const resolved = this.valueAsDate(raw, kind, order)
|
||||||
|
if (!resolved) continue
|
||||||
|
for (const date of applyVaTransform(
|
||||||
|
resolved,
|
||||||
|
transform,
|
||||||
|
kind === 'datetime'
|
||||||
|
)) {
|
||||||
|
// A non-finite result (an Invalid Date reaching the helper) must never be
|
||||||
|
// interpolated into the clause: `NaN` is not a SAS value. Dropped here,
|
||||||
|
// and reported by unsupportedTransforms so the parameter is not silently
|
||||||
|
// filtered on nothing.
|
||||||
|
const sasDate = this.helperService.convertJsDateToSasDate(date, unit)
|
||||||
|
if (!Number.isFinite(Number(sasDate))) continue
|
||||||
|
values.add(`${sasDate}`)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const formatted = Array.from(values)
|
||||||
|
if (formatted.length === 0) return null
|
||||||
|
if (isRangeTransform(transform)) {
|
||||||
|
// One BETWEEN carries one period, so more than one value has no clause -
|
||||||
|
// dropping it here (rather than building a BETWEEN from whichever value
|
||||||
|
// parsed) is what unsupportedTransforms' count warning then describes.
|
||||||
|
if (rawValues.length > 1) return null
|
||||||
|
if (formatted.length !== 2) return null
|
||||||
|
return { operator: 'BETWEEN', rawValue: formatted.join(' AND ') }
|
||||||
|
}
|
||||||
|
return formatted.length === 1
|
||||||
|
? { operator: '=', rawValue: formatted[0] }
|
||||||
|
: { operator: 'IN', rawValue: `(${formatted.join(',')})` }
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A VA value as a local Date, for a transform to work from. Formatted strings
|
||||||
|
* are parsed by shape; a bare number is already the SAS internal value, so it
|
||||||
|
* is converted back from days (date) or seconds (datetime) since 1960-01-01.
|
||||||
|
*
|
||||||
|
* Built with local calendar construction, matching `parseDate` and
|
||||||
|
* `HelperService.convertJsDateToSasDate`: a UTC instant read back through local
|
||||||
|
* accessors lands on the previous day west of UTC, and the round trip to a SAS
|
||||||
|
* day number would be one out.
|
||||||
|
*/
|
||||||
|
private valueAsDate(
|
||||||
|
value: any,
|
||||||
|
kind: VaColumnKind,
|
||||||
|
order?: 'YMD' | 'DMY' | 'MDY'
|
||||||
|
): Date | null {
|
||||||
|
const s = String(value).trim()
|
||||||
|
if (s === '') return null
|
||||||
|
if (this.isBareNumber(s)) {
|
||||||
|
const internal = Number(s)
|
||||||
|
if (kind !== 'datetime') return new Date(1960, 0, 1 + internal)
|
||||||
|
// Seconds since midnight 1960-01-01, split into whole days and a remainder
|
||||||
|
// so the constructor normalises the clock time locally.
|
||||||
|
const days = Math.floor(internal / 86400)
|
||||||
|
const seconds = internal - days * 86400
|
||||||
|
return new Date(1960, 0, 1 + days, 0, 0, seconds)
|
||||||
|
}
|
||||||
|
return kind === 'datetime'
|
||||||
|
? this.parseDatetime(s, order)
|
||||||
|
: this.parseDate(s, order)
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Pre-getdata variant used on the deferred initial load, when DC column
|
* Pre-getdata variant used on the deferred initial load, when DC column
|
||||||
* metadata isn't available. Uses the VA `label` as the column name and infers
|
* metadata isn't available. Uses the VA `label` as the column name and infers
|
||||||
* the SAS kind from the value shape, so the clause matches what the metadata
|
* the SAS kind from the value shape, so the clause matches what the metadata
|
||||||
* build later produces (no reconcile reload).
|
* build later produces (no reconcile reload).
|
||||||
|
*
|
||||||
|
* `paramMap` overrides the label-as-column rule for the parameters it names,
|
||||||
|
* so the very first (already filtered) load also honours an explicit mapping.
|
||||||
*/
|
*/
|
||||||
buildInitialClauses(msg: VaMessage): QueryClause[] {
|
buildInitialClauses(
|
||||||
|
msg: VaMessage,
|
||||||
|
paramMap?: Map<string, VaParmapTarget>
|
||||||
|
): QueryClause[] {
|
||||||
const clauses: QueryClause[] = []
|
const clauses: QueryClause[] = []
|
||||||
for (const param of msg.parameters || []) {
|
for (const param of msg.parameters || []) {
|
||||||
if (!param || param.value === undefined || param.value === null) continue
|
if (!param || param.value === undefined || param.value === null) continue
|
||||||
// The DC column name is the VA `label`; `name` is VA's internal id (pr###).
|
const target = this.mappedTarget(param, paramMap)
|
||||||
const varName = (param.label ?? param.name ?? '').toString().trim()
|
// Without a mapping the DC column name is the VA `label`; `name` is VA's
|
||||||
|
// internal id (pr###). A `va_parmap` entry wins over both.
|
||||||
|
const varName = (target?.column ?? param.label ?? param.name ?? '')
|
||||||
|
.toString()
|
||||||
|
.trim()
|
||||||
if (!varName) continue
|
if (!varName) continue
|
||||||
|
|
||||||
const rawValues = Array.isArray(param.value) ? param.value : [param.value]
|
const rawValues = (
|
||||||
|
Array.isArray(param.value) ? param.value : [param.value]
|
||||||
|
) as any[]
|
||||||
const nonEmpty = rawValues.filter(
|
const nonEmpty = rawValues.filter(
|
||||||
(v) => v !== undefined && v !== null && v !== ''
|
(v) => v !== undefined && v !== null && v !== ''
|
||||||
)
|
)
|
||||||
if (nonEmpty.length === 0) continue
|
if (nonEmpty.length === 0) continue
|
||||||
|
|
||||||
const formatted = nonEmpty
|
// A transform only ever produces dates, so it fixes the kind: inferring it
|
||||||
.map((v) =>
|
// from the value would see a formatted date either way, but with the
|
||||||
this.formatValueByKind(
|
// transform the column is known to be temporal.
|
||||||
v,
|
const kind = target?.transform
|
||||||
this.inferKindFromValue(String(v).trim(), param.dataType)
|
? ('date' as VaColumnKind)
|
||||||
)
|
: this.inferKindFromValue(String(nonEmpty[0]).trim(), param.dataType)
|
||||||
)
|
const built = this.buildClauseValueSet(
|
||||||
.filter((v): v is string => v !== null)
|
param,
|
||||||
if (formatted.length === 0) continue
|
kind,
|
||||||
clauses.push(this.clause(varName, formatted))
|
undefined,
|
||||||
|
target?.transform
|
||||||
|
)
|
||||||
|
if (!built) continue
|
||||||
|
clauses.push({
|
||||||
|
GROUP_LOGIC: 'AND',
|
||||||
|
SUBGROUP_LOGIC: 'AND',
|
||||||
|
SUBGROUP_ID: 0,
|
||||||
|
VARIABLE_NM: varName,
|
||||||
|
OPERATOR_NM: built.operator,
|
||||||
|
RAW_VALUE: built.rawValue
|
||||||
|
})
|
||||||
}
|
}
|
||||||
return clauses
|
return clauses
|
||||||
}
|
}
|
||||||
@@ -141,16 +289,254 @@ export class VaFilterService {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
private clause(varName: string, formatted: string[]): QueryClause {
|
/**
|
||||||
return {
|
* Transform mappings in this message that cannot be applied, one plain-English
|
||||||
GROUP_LOGIC: 'AND',
|
* line each: the column is not temporal, or the value is not a date the browser
|
||||||
SUBGROUP_LOGIC: 'AND',
|
* can parse. `buildClauseValueSet` skips those parameters, so this is what stops
|
||||||
SUBGROUP_ID: 0,
|
* the misconfiguration reading as a filter that simply matched nothing.
|
||||||
VARIABLE_NM: varName,
|
*
|
||||||
OPERATOR_NM: formatted.length === 1 ? '=' : 'IN',
|
* A missing column is left to `unresolvedMappings`, which already reports it.
|
||||||
RAW_VALUE:
|
*/
|
||||||
formatted.length === 1 ? formatted[0] : `(${formatted.join(',')})`
|
unsupportedTransforms(
|
||||||
|
msg: VaMessage,
|
||||||
|
paramMap: Map<string, VaParmapTarget> | undefined,
|
||||||
|
cols: any[]
|
||||||
|
): string[] {
|
||||||
|
if (!paramMap || paramMap.size === 0) return []
|
||||||
|
if (!cols || cols.length === 0) return []
|
||||||
|
const problems = new Set<string>()
|
||||||
|
for (const param of msg.parameters || []) {
|
||||||
|
const target = this.mappedTarget(param, paramMap)
|
||||||
|
if (!target?.transform) continue
|
||||||
|
const transform = target.transform
|
||||||
|
const col = this.findCol(target.column, cols)
|
||||||
|
if (!col) continue
|
||||||
|
const kind = this.columnKind(col)
|
||||||
|
const paramName = (param.label ?? param.name ?? '').toString().trim()
|
||||||
|
const spec =
|
||||||
|
target.transform.op === 'ADD'
|
||||||
|
? `ADD(${target.transform.anchor},${target.transform.offset})`
|
||||||
|
: `RANGE(${target.transform.period},${target.transform.offset})`
|
||||||
|
if (kind !== 'date' && kind !== 'datetime') {
|
||||||
|
problems.add(
|
||||||
|
`VA parameter "${paramName}" maps to ${target.column}:${spec}, but ${target.column} is not a date or datetime column - the filter was ignored.`
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
const order = this.dateOrderFromFmt(col?.FMTNAME || col?.FORMAT)
|
||||||
|
const sent = (
|
||||||
|
Array.isArray(param.value) ? param.value : [param.value]
|
||||||
|
).filter((v: any) => v !== undefined && v !== null && v !== '')
|
||||||
|
const usable = sent.filter((v: any) => {
|
||||||
|
const date = this.valueAsDate(v, kind, order)
|
||||||
|
if (!date) return false
|
||||||
|
// Same test the clause builder applies, so this count matches what it uses.
|
||||||
|
return applyVaTransform(date, transform, kind === 'datetime').every(
|
||||||
|
(resolved) => {
|
||||||
|
const sasDate = this.helperService.convertJsDateToSasDate(
|
||||||
|
resolved,
|
||||||
|
kind === 'datetime' ? 'seconds' : 'days'
|
||||||
|
)
|
||||||
|
return Number.isFinite(Number(sasDate))
|
||||||
|
}
|
||||||
|
)
|
||||||
|
})
|
||||||
|
if (usable.length === 0) {
|
||||||
|
problems.add(
|
||||||
|
`VA parameter "${paramName}" maps to ${target.column}:${spec}, but its value was not a date - the filter was ignored.`
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
// One BETWEEN can only express one period. A multi-select would have to
|
||||||
|
// become an OR of ranges, which the clause model does not carry, so say so
|
||||||
|
// rather than building a BETWEEN from whichever value happened to parse.
|
||||||
|
// Counted on what VA sent: a second value that does not parse must not make
|
||||||
|
// the single-value case look clean.
|
||||||
|
if (isRangeTransform(target.transform) && sent.length > 1) {
|
||||||
|
problems.add(
|
||||||
|
`VA parameter "${paramName}" maps to ${target.column}:${spec}, which is a range, but the parameter has ${sent.length} values - a range needs one value, so the filter was ignored.`
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if (usable.length < sent.length) {
|
||||||
|
problems.add(
|
||||||
|
`VA parameter "${paramName}" maps to ${target.column}:${spec}, but ${sent.length - usable.length} of its ${sent.length} values was not a date - the filter covers only the rest.`
|
||||||
|
)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
return Array.from(problems)
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* `va_parmap` entries that this message actually uses but that name a column
|
||||||
|
* the loaded table does not have, one plain-English line each. `buildClauses`
|
||||||
|
* skips those parameters (deliberately: falling back to the same-name match
|
||||||
|
* would silently apply the filter the mapping exists to override), so this is
|
||||||
|
* what makes the misconfiguration visible to the report author - otherwise the
|
||||||
|
* grid simply shows every row, which looks like a working filter.
|
||||||
|
*/
|
||||||
|
unresolvedMappings(
|
||||||
|
msg: VaMessage,
|
||||||
|
paramMap: Map<string, VaParmapTarget> | undefined,
|
||||||
|
cols: any[],
|
||||||
|
libds?: string
|
||||||
|
): string[] {
|
||||||
|
const problems = new Set<string>()
|
||||||
|
if (!paramMap || paramMap.size === 0) return []
|
||||||
|
// No metadata yet is not the same as "the table lacks this column": an empty
|
||||||
|
// `cols` means the grid is still loading, so there is nothing to judge yet.
|
||||||
|
// Reporting here would flash a misconfiguration banner during every load.
|
||||||
|
if (!cols || cols.length === 0) return []
|
||||||
|
for (const param of msg.parameters || []) {
|
||||||
|
const mapped = this.mappedColumn(param, paramMap)
|
||||||
|
if (!mapped || this.findCol(mapped, cols)) continue
|
||||||
|
const paramName = (param.label ?? param.name ?? '').toString().trim()
|
||||||
|
problems.add(
|
||||||
|
`VA parameter "${paramName}" is mapped to ${mapped}, which is not a column of ${
|
||||||
|
libds || 'this table'
|
||||||
|
} - the filter was ignored.`
|
||||||
|
)
|
||||||
|
}
|
||||||
|
return Array.from(problems)
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* `va_parmap` keys that no VA message has named, one plain-English line each.
|
||||||
|
* A key that matches nothing is a parameter-side typo: the mapping never
|
||||||
|
* applies and the grid stays unfiltered, which looks the same as a working
|
||||||
|
* filter.
|
||||||
|
*
|
||||||
|
* `seenKeys` must be every key observed across messages so far. `paramsStable`
|
||||||
|
* says the latest message introduced no new parameter name, i.e. this object's
|
||||||
|
* parameter set has settled - until then a missing name may simply not have
|
||||||
|
* arrived, because VA sends only the parameters its query uses.
|
||||||
|
*/
|
||||||
|
unmatchedMappings(
|
||||||
|
paramMap: Map<string, VaParmapTarget> | undefined,
|
||||||
|
seenKeys: Set<string> | undefined,
|
||||||
|
paramsStable = false
|
||||||
|
): string[] {
|
||||||
|
if (!paramMap || paramMap.size === 0) return []
|
||||||
|
if (!seenKeys || seenKeys.size === 0) return []
|
||||||
|
if (!paramsStable) return []
|
||||||
|
const problems: string[] = []
|
||||||
|
for (const key of Array.from(paramMap.keys())) {
|
||||||
|
if (!seenKeys.has(key)) {
|
||||||
|
problems.push(
|
||||||
|
`This object's va_parmap names VA parameter "${key}", which VA is not sending - check the spelling.`
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return problems
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Parameters in this message that filter the same column with *different*
|
||||||
|
* values, one plain-English line per column. Clauses are AND-ed, so two
|
||||||
|
* parameters on one column are an intersection: with different values the grid
|
||||||
|
* comes back empty, which looks like a working filter. Two parameters carrying
|
||||||
|
* the same value intersect cleanly and filter correctly, so they are not
|
||||||
|
* reported - a banner there would call a working configuration broken.
|
||||||
|
*
|
||||||
|
* Reported rather than rewritten - folding them into one `IN (...)` would
|
||||||
|
* silently turn an AND into an OR, which is a semantics decision, not a
|
||||||
|
* diagnostic.
|
||||||
|
*/
|
||||||
|
sameColumnMappings(
|
||||||
|
msg: VaMessage,
|
||||||
|
paramMap: Map<string, VaParmapTarget> | undefined,
|
||||||
|
cols?: any[]
|
||||||
|
): string[] {
|
||||||
|
if (!paramMap || paramMap.size === 0) return []
|
||||||
|
// column -> the parameters on it, each with the clause value it produces
|
||||||
|
const byColumn = new Map<string, { name: string; value: string }[]>()
|
||||||
|
for (const param of msg.parameters || []) {
|
||||||
|
const mapped = this.mappedColumn(param, paramMap)
|
||||||
|
if (!mapped) continue
|
||||||
|
// A column the table lacks is unresolvedMappings' finding; counting it here
|
||||||
|
// would report one cause as two symptoms.
|
||||||
|
const col =
|
||||||
|
cols && cols.length > 0 ? this.findCol(mapped, cols) : undefined
|
||||||
|
if (cols && cols.length > 0 && !col) continue
|
||||||
|
// A parameter that produces no clause cannot conflict with anything.
|
||||||
|
const value = this.valueSignature(
|
||||||
|
param,
|
||||||
|
col,
|
||||||
|
this.mappedTarget(param, paramMap)?.transform
|
||||||
|
)
|
||||||
|
if (value === null) continue
|
||||||
|
const column = mapped.toUpperCase()
|
||||||
|
const params = byColumn.get(column) || []
|
||||||
|
const paramName = (param.label ?? '').toString().trim()
|
||||||
|
if (paramName && !params.some((p) => p.name === paramName)) {
|
||||||
|
params.push({ name: paramName, value })
|
||||||
|
}
|
||||||
|
byColumn.set(column, params)
|
||||||
|
}
|
||||||
|
const problems: string[] = []
|
||||||
|
for (const [column, params] of Array.from(byColumn.entries())) {
|
||||||
|
if (params.length < 2) continue
|
||||||
|
// Same value on every parameter: the AND is a no-op, not an empty grid.
|
||||||
|
const distinct = new Set(params.map((p) => p.value))
|
||||||
|
if (distinct.size === 1) continue
|
||||||
|
problems.push(
|
||||||
|
`VA parameters ${params
|
||||||
|
.map((p) => `"${p.name}"`)
|
||||||
|
.join(
|
||||||
|
' and '
|
||||||
|
)} all filter ${column}. Parameters are combined with AND, so rows must match all of them - different values will show no rows. Use one parameter carrying both values to match either.`
|
||||||
|
)
|
||||||
|
}
|
||||||
|
return problems
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The DC column an explicit URL mapping (`va_parmap`) assigns to this
|
||||||
|
* parameter, or undefined when the parameter is not mapped. The key is matched
|
||||||
|
* case-insensitively against the parameter's `label`.
|
||||||
|
*/
|
||||||
|
private mappedTarget(
|
||||||
|
param: any,
|
||||||
|
paramMap?: Map<string, VaParmapTarget>
|
||||||
|
): VaParmapTarget | undefined {
|
||||||
|
if (!paramMap || paramMap.size === 0) return undefined
|
||||||
|
// The label only. The internal name (pr273) is an implementation detail of
|
||||||
|
// the report, and matching both made one parameter answer to two keys: a
|
||||||
|
// mapping written against the name could never be distinguished from a typo
|
||||||
|
// once the label was also mapped, and the second entry silently lost.
|
||||||
|
const key = (param.label ?? '').toString().trim().toLowerCase()
|
||||||
|
if (!key) return undefined
|
||||||
|
return paramMap.get(key)
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The clause value a parameter would filter its column with, as a comparable
|
||||||
|
* string, or null when it produces no clause at all. Used to tell an AND of two
|
||||||
|
* identical values (harmless) from an AND of two different ones (empty grid).
|
||||||
|
*
|
||||||
|
* Delegates to the clause builder so a transform's derived dates take part in
|
||||||
|
* the comparison: two parameters adding the same quarter are the same clause.
|
||||||
|
*/
|
||||||
|
private valueSignature(
|
||||||
|
param: any,
|
||||||
|
col: any,
|
||||||
|
transform?: VaTransform
|
||||||
|
): string | null {
|
||||||
|
const built = this.buildClauseValueSet(
|
||||||
|
param,
|
||||||
|
this.columnKind(col),
|
||||||
|
this.dateOrderFromFmt(col?.FMTNAME || col?.FORMAT),
|
||||||
|
transform
|
||||||
|
)
|
||||||
|
return built ? built.rawValue : null
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The mapped column name, for callers that do not need the transform. */
|
||||||
|
private mappedColumn(
|
||||||
|
param: any,
|
||||||
|
paramMap?: Map<string, VaParmapTarget>
|
||||||
|
): string | undefined {
|
||||||
|
return this.mappedTarget(param, paramMap)?.column
|
||||||
}
|
}
|
||||||
|
|
||||||
/** The loaded column spec whose NAME matches varName (case-insensitive). */
|
/** The loaded column spec whose NAME matches varName (case-insensitive). */
|
||||||
@@ -321,14 +707,42 @@ export class VaFilterService {
|
|||||||
return this.parseMonthNameDate(value) ?? this.parseNumericDate(value, order)
|
return this.parseMonthNameDate(value) ?? this.parseNumericDate(value, order)
|
||||||
}
|
}
|
||||||
|
|
||||||
/** DATE9 "ddMMM[yy]yy" (e.g. 01AUG1962, 01AUG62) -> local Date, or null. */
|
/**
|
||||||
|
* Month-name date shapes -> local Date, or null. Three families:
|
||||||
|
* - DATE9 "ddMMM[yy]yy" (e.g. 01AUG1962, 01AUG62);
|
||||||
|
* - word date with the month first, comma optional: "January 1, 2026",
|
||||||
|
* "Jan 1 2026", "Jan. 1, 2026";
|
||||||
|
* - word date with the day first: "1 January 2026", "1-Jan-2026".
|
||||||
|
*
|
||||||
|
* A date parameter's display format decides which of these the report author
|
||||||
|
* sees, and VA sends the formatted text, so a report using WORDDATE (or a
|
||||||
|
* long month name in any order) has to parse here rather than being dropped.
|
||||||
|
* The month name may be abbreviated or spelled out; only its first three
|
||||||
|
* letters are matched.
|
||||||
|
*/
|
||||||
private parseMonthNameDate(value: string): Date | null {
|
private parseMonthNameDate(value: string): Date | null {
|
||||||
const m = /^(\d{1,2})([A-Za-z]{3})(\d{2}|\d{4})$/.exec(value)
|
const monthByName = (name: string): number =>
|
||||||
if (!m) return null
|
VaFilterService.SAS_MONTHS.indexOf(
|
||||||
const [, day, monthAbbr, year] = m
|
name.replace(/\./g, '').slice(0, 3).toUpperCase()
|
||||||
const monthIndex = VaFilterService.SAS_MONTHS.indexOf(
|
)
|
||||||
monthAbbr.toUpperCase()
|
|
||||||
)
|
const date9 = /^(\d{1,2})([A-Za-z]{3})(\d{2}|\d{4})$/.exec(value)
|
||||||
|
if (date9) {
|
||||||
|
const [, day, monthAbbr, year] = date9
|
||||||
|
const monthIndex = monthByName(monthAbbr)
|
||||||
|
if (monthIndex < 0) return null
|
||||||
|
return this.buildDate(this.fullYear(year), monthIndex + 1, Number(day))
|
||||||
|
}
|
||||||
|
|
||||||
|
const monthFirst =
|
||||||
|
/^([A-Za-z]{3,9})\.?[\s-]+(\d{1,2}),?[\s-]+(\d{2}|\d{4})$/.exec(value)
|
||||||
|
const dayFirst =
|
||||||
|
/^(\d{1,2})[\s-]+([A-Za-z]{3,9})\.?,?[\s-]+(\d{2}|\d{4})$/.exec(value)
|
||||||
|
const name = monthFirst ? monthFirst[1] : dayFirst ? dayFirst[2] : null
|
||||||
|
if (!name) return null
|
||||||
|
const day = monthFirst ? monthFirst[2] : dayFirst![1]
|
||||||
|
const year = monthFirst ? monthFirst[3] : dayFirst![3]
|
||||||
|
const monthIndex = monthByName(name)
|
||||||
if (monthIndex < 0) return null
|
if (monthIndex < 0) return null
|
||||||
return this.buildDate(this.fullYear(year), monthIndex + 1, Number(day))
|
return this.buildDate(this.fullYear(year), monthIndex + 1, Number(day))
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,4 +1,6 @@
|
|||||||
import { VaMessagingService, VaMessage } from './va-messaging.service'
|
import { VaMessagingService, VaMessage } from './va-messaging.service'
|
||||||
|
import { VaFilterService } from './va-filter.service'
|
||||||
|
import { HelperService } from './helper.service'
|
||||||
|
|
||||||
// Sample message taken from the official SAS DDC docs (brush column included).
|
// Sample message taken from the official SAS DDC docs (brush column included).
|
||||||
const SAMPLE = {
|
const SAMPLE = {
|
||||||
@@ -51,6 +53,57 @@ describe('VaMessagingService', () => {
|
|||||||
})
|
})
|
||||||
})
|
})
|
||||||
|
|
||||||
|
describe('parameter keys (what the va_parmap typo diagnostic is judged on)', () => {
|
||||||
|
const PARAMS = [
|
||||||
|
{ name: 'pr273', label: 'Tab_1_Date', value: '2025-03-31' },
|
||||||
|
{ name: 'pr274', label: 'Tab 2 Region', value: 'north' }
|
||||||
|
]
|
||||||
|
const withParams = () => ({ ...SAMPLE, parameters: PARAMS })
|
||||||
|
|
||||||
|
it('records each parameter label once, across messages', () => {
|
||||||
|
service.parseMessage({ data: withParams() } as MessageEvent)
|
||||||
|
service.parseMessage({ data: withParams() } as MessageEvent)
|
||||||
|
|
||||||
|
expect(Array.from(service.seenParamKeys)).toEqual([
|
||||||
|
'tab_1_date',
|
||||||
|
'tab 2 region'
|
||||||
|
])
|
||||||
|
// The second message introduced no new label, so the set has settled.
|
||||||
|
expect(service.lastMessageParamsStable).toBeTrue()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('records the label only, so a mapping written against the internal name is reported', () => {
|
||||||
|
// `pr273` is VA's own bookkeeping. A va_parmap entry written against it
|
||||||
|
// never matches a parameter, and recording the name here would make the
|
||||||
|
// typo diagnostic believe VA is sending it - leaving the author with an
|
||||||
|
// unfiltered grid and no warning at all.
|
||||||
|
service.parseMessage({ data: withParams() } as MessageEvent)
|
||||||
|
service.parseMessage({ data: withParams() } as MessageEvent)
|
||||||
|
|
||||||
|
const filter = new VaFilterService(new HelperService({} as any))
|
||||||
|
const seen = service.seenParamKeys
|
||||||
|
const stable = service.lastMessageParamsStable
|
||||||
|
|
||||||
|
expect(
|
||||||
|
filter.unmatchedMappings(
|
||||||
|
new Map([['pr273', { column: 'D_DATE' }]]),
|
||||||
|
seen,
|
||||||
|
stable
|
||||||
|
)
|
||||||
|
).toEqual([
|
||||||
|
'This object\'s va_parmap names VA parameter "pr273", which VA is not sending - check the spelling.'
|
||||||
|
])
|
||||||
|
// The label is a real key, so the same map written that way is silent.
|
||||||
|
expect(
|
||||||
|
filter.unmatchedMappings(
|
||||||
|
new Map([['tab_1_date', { column: 'D_DATE' }]]),
|
||||||
|
seen,
|
||||||
|
stable
|
||||||
|
)
|
||||||
|
).toEqual([])
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
describe('dataColumns', () => {
|
describe('dataColumns', () => {
|
||||||
it('excludes brush columns and keeps original indices', () => {
|
it('excludes brush columns and keeps original indices', () => {
|
||||||
const msg = service.parseMessage({ data: SAMPLE } as MessageEvent)!
|
const msg = service.parseMessage({ data: SAMPLE } as MessageEvent)!
|
||||||
|
|||||||
@@ -82,6 +82,26 @@ export class VaMessagingService {
|
|||||||
/** DC variable names that should be visible (the rest are hidden). */
|
/** DC variable names that should be visible (the rest are hidden). */
|
||||||
visibleColumns: string[] | null = null
|
visibleColumns: string[] | null = null
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Lower-cased label of every parameter any VA message has carried so far, kept
|
||||||
|
* across editor reloads. Lets the editor tell a `va_parmap` key that names no
|
||||||
|
* parameter in the report (a typo, so the filter would silently never apply)
|
||||||
|
* from one that is merely absent from the current message - VA only sends the
|
||||||
|
* parameters its query uses.
|
||||||
|
*
|
||||||
|
* Labels only: that is the key a mapping is matched on, and recording the
|
||||||
|
* internal name as well would make a name-keyed entry - which never applies -
|
||||||
|
* look like a parameter VA is sending.
|
||||||
|
*/
|
||||||
|
seenParamKeys = new Set<string>()
|
||||||
|
/**
|
||||||
|
* True when every parameter name in the latest message had already been seen.
|
||||||
|
* VA sends only the parameters a report object's query uses, and sends them per
|
||||||
|
* object, so a name that is absent from the first messages may simply not have
|
||||||
|
* arrived yet. A stable parameter set is what makes absence meaningful.
|
||||||
|
*/
|
||||||
|
lastMessageParamsStable = false
|
||||||
|
|
||||||
/** True once the pre-bootstrap early buffer has been drained (drain-once). */
|
/** True once the pre-bootstrap early buffer has been drained (drain-once). */
|
||||||
private earlyDrained = false
|
private earlyDrained = false
|
||||||
|
|
||||||
@@ -187,6 +207,22 @@ export class VaMessagingService {
|
|||||||
if (typeof data.resultName !== 'string') return null
|
if (typeof data.resultName !== 'string') return null
|
||||||
if (!Array.isArray(data.columns) || !Array.isArray(data.data)) return null
|
if (!Array.isArray(data.columns) || !Array.isArray(data.data)) return null
|
||||||
|
|
||||||
|
const parameters = Array.isArray(data.parameters) ? data.parameters : []
|
||||||
|
const keys: string[] = []
|
||||||
|
for (const param of parameters) {
|
||||||
|
// The label only, which is the key a `va_parmap` entry is matched on. The
|
||||||
|
// internal name (`pr273`) is the report's own bookkeeping: recording it
|
||||||
|
// here would let the typo diagnostic below believe a name-keyed entry is
|
||||||
|
// being sent when no code path can ever apply it.
|
||||||
|
const key = (param?.label ?? '').toString().trim().toLowerCase()
|
||||||
|
if (key && !keys.includes(key)) keys.push(key)
|
||||||
|
}
|
||||||
|
// Judged before the keys are recorded: a message that introduces a new name
|
||||||
|
// means the set is still growing, so a missing name proves nothing yet.
|
||||||
|
this.lastMessageParamsStable =
|
||||||
|
keys.length > 0 && keys.every((key) => this.seenParamKeys.has(key))
|
||||||
|
for (const key of keys) this.seenParamKeys.add(key)
|
||||||
|
|
||||||
return {
|
return {
|
||||||
version: data.version,
|
version: data.version,
|
||||||
resultName: data.resultName,
|
resultName: data.resultName,
|
||||||
@@ -194,7 +230,7 @@ export class VaMessagingService {
|
|||||||
availableRowCount: data.availableRowCount,
|
availableRowCount: data.availableRowCount,
|
||||||
data: data.data,
|
data: data.data,
|
||||||
columns: data.columns,
|
columns: data.columns,
|
||||||
parameters: Array.isArray(data.parameters) ? data.parameters : []
|
parameters
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,215 @@
|
|||||||
|
import { parseVaParmap } from './parse-va-parmap'
|
||||||
|
|
||||||
|
describe('parseVaParmap', () => {
|
||||||
|
it('returns an empty map and no problems when the parameter is absent', () => {
|
||||||
|
for (const query of ['', 'embed=va', 'embed=va&labels=true']) {
|
||||||
|
const parsed = parseVaParmap(query)
|
||||||
|
expect(parsed.map.size).toBe(0)
|
||||||
|
expect(parsed.problems).toEqual([])
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
it('maps a VA parameter label to a DC column', () => {
|
||||||
|
const { map, problems } = parseVaParmap(
|
||||||
|
'embed=va&va_parmap=REGION_TAB1-REGION'
|
||||||
|
)
|
||||||
|
expect(map.get('region_tab1')?.column).toBe('REGION')
|
||||||
|
expect(map.size).toBe(1)
|
||||||
|
expect(problems).toEqual([])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('collects repeated entries, so two parameters can map to one column', () => {
|
||||||
|
// The multi-tab case: each DC instance filters the same DC column from a
|
||||||
|
// differently-named VA parameter.
|
||||||
|
const { map } = parseVaParmap(
|
||||||
|
'embed=va&va_parmap=REGION_TAB1-REGION&va_parmap=REGION_TAB2-REGION'
|
||||||
|
)
|
||||||
|
expect(map.get('region_tab1')?.column).toBe('REGION')
|
||||||
|
expect(map.get('region_tab2')?.column).toBe('REGION')
|
||||||
|
expect(map.size).toBe(2)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('is case-insensitive on the key and trims both sides', () => {
|
||||||
|
const { map } = parseVaParmap('va_parmap=%20Region_Tab1%20-%20region%20')
|
||||||
|
expect(map.get('region_tab1')?.column).toBe('region')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('decodes a key containing a space (VA labels are free text)', () => {
|
||||||
|
expect(
|
||||||
|
parseVaParmap('va_parmap=My+Region-DEPT').map.get('my region')?.column
|
||||||
|
).toBe('DEPT')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('splits at the LAST hyphen, so a label may contain hyphens', () => {
|
||||||
|
expect(
|
||||||
|
parseVaParmap('va_parmap=Sales-2024-DEPT').map.get('sales-2024')?.column
|
||||||
|
).toBe('DEPT')
|
||||||
|
})
|
||||||
|
|
||||||
|
describe('entries that cannot be used', () => {
|
||||||
|
it('reports an entry with no separator', () => {
|
||||||
|
const { map, problems } = parseVaParmap('va_parmap=DEPT')
|
||||||
|
expect(map.size).toBe(0)
|
||||||
|
expect(problems).toEqual([
|
||||||
|
'va_parmap "DEPT" was ignored: it needs a "-" between the VA parameter and the column name.'
|
||||||
|
])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('reports an entry with no parameter name', () => {
|
||||||
|
const { map, problems } = parseVaParmap('va_parmap=-DEPT')
|
||||||
|
expect(map.size).toBe(0)
|
||||||
|
expect(problems).toEqual([
|
||||||
|
'va_parmap "-DEPT" was ignored: the VA parameter name is missing.'
|
||||||
|
])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('reports an entry with no column name', () => {
|
||||||
|
const { map, problems } = parseVaParmap('va_parmap=REGION_TAB1-')
|
||||||
|
expect(map.size).toBe(0)
|
||||||
|
expect(problems).toEqual([
|
||||||
|
'va_parmap "REGION_TAB1-" was ignored: "" is not a valid SAS column name.'
|
||||||
|
])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('reports a column that is not a plain SAS name (injection guard)', () => {
|
||||||
|
// The column is interpolated into a SAS WHERE clause, so anything beyond a
|
||||||
|
// SAS name is refused at parse time rather than carried into a clause.
|
||||||
|
for (const entry of [
|
||||||
|
'A-x;drop',
|
||||||
|
'A-REGION%20OR%201=1',
|
||||||
|
"A-RE'GION",
|
||||||
|
'A-1ABC'
|
||||||
|
]) {
|
||||||
|
const { map, problems } = parseVaParmap(`va_parmap=${entry}`)
|
||||||
|
expect(map.size).toBe(0)
|
||||||
|
expect(problems.length).toBe(1)
|
||||||
|
expect(problems[0]).toContain('is not a valid SAS column name')
|
||||||
|
}
|
||||||
|
expect(parseVaParmap('va_parmap=A-A_B9').map.get('a')?.column).toBe(
|
||||||
|
'A_B9'
|
||||||
|
)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('keeps the usable entries and reports only the bad one', () => {
|
||||||
|
const { map, problems } = parseVaParmap(
|
||||||
|
'va_parmap=GOOD-REGION&va_parmap=BROKEN'
|
||||||
|
)
|
||||||
|
expect(map.get('good')?.column).toBe('REGION')
|
||||||
|
expect(problems.length).toBe(1)
|
||||||
|
expect(problems[0]).toContain('va_parmap "BROKEN"')
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
describe('transforms', () => {
|
||||||
|
it('parses an ADD anchor with an offset', () => {
|
||||||
|
const { map, problems } = parseVaParmap(
|
||||||
|
'va_parmap=SELECTED-D_DATE:ADD(QUARTER_START,-1)'
|
||||||
|
)
|
||||||
|
expect(problems).toEqual([])
|
||||||
|
expect(map.get('selected')?.column).toBe('D_DATE')
|
||||||
|
expect(map.get('selected')?.transform).toEqual({
|
||||||
|
op: 'ADD',
|
||||||
|
anchor: 'QUARTER_START',
|
||||||
|
offset: -1
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
it('defaults the offset to zero and accepts lower case', () => {
|
||||||
|
const { map, problems } = parseVaParmap(
|
||||||
|
'va_parmap=sel-d_date:add(quarter_start)'
|
||||||
|
)
|
||||||
|
expect(problems).toEqual([])
|
||||||
|
expect(map.get('sel')?.transform).toEqual({
|
||||||
|
op: 'ADD',
|
||||||
|
anchor: 'QUARTER_START',
|
||||||
|
offset: 0
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
it('parses a RANGE period', () => {
|
||||||
|
const { map, problems } = parseVaParmap(
|
||||||
|
'va_parmap=sel-D_DATE:RANGE(quarter,-1)'
|
||||||
|
)
|
||||||
|
expect(problems).toEqual([])
|
||||||
|
expect(map.get('sel')?.transform).toEqual({
|
||||||
|
op: 'RANGE',
|
||||||
|
period: 'QUARTER',
|
||||||
|
offset: -1
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
it('leaves a mapping without a transform as a plain column', () => {
|
||||||
|
const { map } = parseVaParmap('va_parmap=sel-D_DATE')
|
||||||
|
expect(map.get('sel')?.column).toBe('D_DATE')
|
||||||
|
expect(map.get('sel')?.transform).toBeUndefined()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('reports an unknown operation, anchor, period or offset', () => {
|
||||||
|
const cases: [string, string][] = [
|
||||||
|
['sel-D_DATE:SHIFT(QUARTER_START,-1)', 'is not a known transform'],
|
||||||
|
['sel-D_DATE:ADD(FORTNIGHT_START,-1)', 'is not a known anchor'],
|
||||||
|
['sel-D_DATE:RANGE(DECADE,-1)', 'is not a known period'],
|
||||||
|
['sel-D_DATE:ADD(QUARTER_START,3.5)', 'is not a whole number'],
|
||||||
|
['sel-D_DATE:ADD', 'is not a transform']
|
||||||
|
]
|
||||||
|
for (const [entry, expected] of cases) {
|
||||||
|
const { map, problems } = parseVaParmap(`va_parmap=${entry}`)
|
||||||
|
expect(`${entry}: ${map.size}`).toBe(`${entry}: 0`)
|
||||||
|
expect(`${entry}: ${problems.length}`).toBe(`${entry}: 1`)
|
||||||
|
expect(problems[0]).toContain(expected)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
it('keeps the parameter label free to contain a colon', () => {
|
||||||
|
// The key side is untouched by the transform grammar: only the column side
|
||||||
|
// is split, so a label with a colon still maps.
|
||||||
|
const { map, problems } = parseVaParmap(
|
||||||
|
'va_parmap=Sales:2026-D_DATE:ADD(QUARTER_START,-1)'
|
||||||
|
)
|
||||||
|
expect(problems).toEqual([])
|
||||||
|
expect(map.get('sales:2026')?.column).toBe('D_DATE')
|
||||||
|
expect(map.get('sales:2026')?.transform).toEqual({
|
||||||
|
op: 'ADD',
|
||||||
|
anchor: 'QUARTER_START',
|
||||||
|
offset: -1
|
||||||
|
})
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
describe('one parameter, one column', () => {
|
||||||
|
it('keeps the first mapping and reports a conflicting duplicate', () => {
|
||||||
|
const { map, problems } = parseVaParmap(
|
||||||
|
'va_parmap=DEPT-REGION&va_parmap=DEPT-OTHER'
|
||||||
|
)
|
||||||
|
expect(map.get('dept')?.column).toBe('REGION')
|
||||||
|
expect(problems).toEqual([
|
||||||
|
'va_parmap "DEPT-OTHER" was ignored: VA parameter "dept" already maps to REGION, and a parameter can filter only one column.'
|
||||||
|
])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('treats a duplicate key as a duplicate across case', () => {
|
||||||
|
const { map, problems } = parseVaParmap(
|
||||||
|
'va_parmap=Dept-REGION&va_parmap=DEPT-SOMETHING_ELSE'
|
||||||
|
)
|
||||||
|
expect(map.get('dept')?.column).toBe('REGION')
|
||||||
|
expect(problems.length).toBe(1)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('accepts the same mapping written twice (no ambiguity)', () => {
|
||||||
|
const { map, problems } = parseVaParmap(
|
||||||
|
'va_parmap=DEPT-REGION&va_parmap=dept-region'
|
||||||
|
)
|
||||||
|
expect(map.get('dept')?.column).toBe('REGION')
|
||||||
|
expect(problems).toEqual([])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('still allows two parameters to map to one column', () => {
|
||||||
|
// The multi-object case: different parameters, same DC column.
|
||||||
|
const { map, problems } = parseVaParmap(
|
||||||
|
'va_parmap=REGION_TAB1-REGION&va_parmap=REGION_TAB2-REGION'
|
||||||
|
)
|
||||||
|
expect(map.size).toBe(2)
|
||||||
|
expect(problems).toEqual([])
|
||||||
|
})
|
||||||
|
})
|
||||||
|
})
|
||||||
@@ -0,0 +1,192 @@
|
|||||||
|
import {
|
||||||
|
VA_ANCHORS,
|
||||||
|
VA_PERIODS,
|
||||||
|
VaAnchor,
|
||||||
|
VaPeriod,
|
||||||
|
VaTransform
|
||||||
|
} from './va-parmap-transform'
|
||||||
|
|
||||||
|
/**
|
||||||
|
* What a VA parameter's mapping resolves to: the DC column, and optionally a
|
||||||
|
* transform that adds derived dates to the filter (e.g. the previous quarter's
|
||||||
|
* row alongside the selected one).
|
||||||
|
*/
|
||||||
|
export interface VaParmapTarget {
|
||||||
|
column: string
|
||||||
|
transform?: VaTransform
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Result of `parseVaParmap` - the usable mappings, and what was ignored. */
|
||||||
|
export interface VaParmap {
|
||||||
|
/** Lower-cased VA parameter key -> DC column (and transform, if any). */
|
||||||
|
map: Map<string, VaParmapTarget>
|
||||||
|
/** One plain-English line per `va_parmap` entry that was ignored. */
|
||||||
|
problems: string[]
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Parses the `va_parmap` entries out of a hash query string (the part after `?`
|
||||||
|
* in `window.location.hash`).
|
||||||
|
*
|
||||||
|
* Format - one mapping per occurrence, each `<VA parameter key>-<DC column>`:
|
||||||
|
*
|
||||||
|
* ?embed=va&va_parmap=REGION_TAB1-REGION&va_parmap=REGION_TAB2-REGION
|
||||||
|
*
|
||||||
|
* Repeating the parameter (rather than packing a comma-separated list into one
|
||||||
|
* value) keeps the grammar unambiguous: VA parameter labels are free text, so
|
||||||
|
* any list separator we picked could also appear inside a label.
|
||||||
|
*
|
||||||
|
* Each entry is split at its LAST `-`. A SAS column name can never contain a
|
||||||
|
* hyphen, so the final one always delimits the column, and a label that does
|
||||||
|
* contain hyphens (e.g. `Sales-2024`) still maps correctly.
|
||||||
|
*
|
||||||
|
* The key is matched case-insensitively, and trailing/leading blanks are
|
||||||
|
* trimmed, against a VA parameter's `label`, which is the same key the
|
||||||
|
* label->column match uses. The internal `name` (e.g. `pr273`) is not a key: it
|
||||||
|
* is the report's own bookkeeping, so an entry written against it never applies
|
||||||
|
* and is reported by `unmatchedMappings`.
|
||||||
|
*
|
||||||
|
* An entry that cannot be used is reported in `problems` rather than dropped in
|
||||||
|
* silence: these entries are hand-written in the report's DDC URL, usually by
|
||||||
|
* the report author, and a typo would otherwise degrade to "no filter" - which
|
||||||
|
* looks exactly like a working filter over the whole table.
|
||||||
|
*
|
||||||
|
* Returns an empty map when the parameter is absent, which leaves the default
|
||||||
|
* same-name resolution untouched (existing embeds are unaffected).
|
||||||
|
*
|
||||||
|
* Modeled on `parseEmbedParam` / `parseLabelsParam`, which read the same hash
|
||||||
|
* query.
|
||||||
|
*/
|
||||||
|
export function parseVaParmap(hashQuery: string): VaParmap {
|
||||||
|
const map = new Map<string, VaParmapTarget>()
|
||||||
|
const problems: string[] = []
|
||||||
|
if (!hashQuery) return { map, problems }
|
||||||
|
|
||||||
|
const entries = new URLSearchParams(hashQuery).getAll('va_parmap')
|
||||||
|
for (const entry of entries) {
|
||||||
|
// A transform sits at the end, in parentheses: `<param>-<column>:OP(args)`.
|
||||||
|
// It is lifted off FIRST, because its arguments may contain a hyphen (a
|
||||||
|
// negative offset) and the key/column split below uses the hyphen.
|
||||||
|
let body = entry ?? ''
|
||||||
|
let transformSpec = ''
|
||||||
|
if (body.trim().endsWith(')')) {
|
||||||
|
const open = body.lastIndexOf('(')
|
||||||
|
const colon = open > 0 ? body.lastIndexOf(':', open) : -1
|
||||||
|
if (colon > 0) {
|
||||||
|
transformSpec = body.slice(colon + 1).trim()
|
||||||
|
body = body.slice(0, colon)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Last hyphen delimits the DC column (see above).
|
||||||
|
const separator = body.lastIndexOf('-')
|
||||||
|
if (separator < 0) {
|
||||||
|
problems.push(
|
||||||
|
`va_parmap "${entry}" was ignored: it needs a "-" between the VA parameter and the column name.`
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
const key = body.slice(0, separator).trim().toLowerCase()
|
||||||
|
const column = body.slice(separator + 1).trim()
|
||||||
|
if (!key) {
|
||||||
|
problems.push(
|
||||||
|
`va_parmap "${entry}" was ignored: the VA parameter name is missing.`
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if (column.includes(':')) {
|
||||||
|
problems.push(
|
||||||
|
`va_parmap "${entry}" was ignored: "${column}" is not a transform - expected e.g. ADD(QUARTER_START,-1).`
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
// The column name is interpolated into a SAS WHERE clause, so only a plain
|
||||||
|
// SAS name is accepted - anything else is refused rather than mapped.
|
||||||
|
if (!/^[A-Za-z_][A-Za-z0-9_]{0,31}$/.test(column)) {
|
||||||
|
problems.push(
|
||||||
|
`va_parmap "${entry}" was ignored: "${column}" is not a valid SAS column name.`
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
const transform = transformSpec
|
||||||
|
? parseTransform(transformSpec, entry, problems)
|
||||||
|
: undefined
|
||||||
|
if (transformSpec && !transform) continue
|
||||||
|
// One VA parameter filters one column. A second entry for the same parameter
|
||||||
|
// is ambiguous, so the FIRST one wins and the conflict is reported - the
|
||||||
|
// alternative (last one wins, silently) would apply a filter the author
|
||||||
|
// never sees the origin of. Two parameters mapping to the same column is
|
||||||
|
// fine and is how two report objects filter one column from their own
|
||||||
|
// parameters.
|
||||||
|
const existing = map.get(key)
|
||||||
|
if (existing) {
|
||||||
|
if (existing.column.toUpperCase() !== column.toUpperCase()) {
|
||||||
|
problems.push(
|
||||||
|
`va_parmap "${entry}" was ignored: VA parameter "${key}" already maps to ${existing.column}, and a parameter can filter only one column.`
|
||||||
|
)
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
map.set(key, { column, transform })
|
||||||
|
}
|
||||||
|
|
||||||
|
return { map, problems }
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Parses `OP(arg[, arg])`, returning undefined (and pushing a problem line) when
|
||||||
|
* the operation or one of its arguments is not in the allow-list.
|
||||||
|
*
|
||||||
|
* ADD takes an anchor and an optional offset; RANGE takes a period and an
|
||||||
|
* optional offset. The offset is whole periods, negative for earlier.
|
||||||
|
*/
|
||||||
|
function parseTransform(
|
||||||
|
spec: string,
|
||||||
|
entry: string,
|
||||||
|
problems: string[]
|
||||||
|
): VaTransform | undefined {
|
||||||
|
const match = /^([A-Za-z]+)\s*\(([^)]*)\)$/.exec(spec)
|
||||||
|
if (!match) {
|
||||||
|
problems.push(
|
||||||
|
`va_parmap "${entry}" was ignored: "${spec}" is not a transform - expected e.g. ADD(QUARTER_START,-1).`
|
||||||
|
)
|
||||||
|
return undefined
|
||||||
|
}
|
||||||
|
const op = match[1].toUpperCase()
|
||||||
|
const args = match[2]
|
||||||
|
.split(',')
|
||||||
|
.map((arg) => arg.trim())
|
||||||
|
.filter((arg) => arg.length > 0)
|
||||||
|
const offsetArg = args[1] === undefined ? '0' : args[1]
|
||||||
|
if (!/^-?\d+$/.test(offsetArg)) {
|
||||||
|
problems.push(
|
||||||
|
`va_parmap "${entry}" was ignored: the offset "${offsetArg}" is not a whole number of periods.`
|
||||||
|
)
|
||||||
|
return undefined
|
||||||
|
}
|
||||||
|
const offset = Number(offsetArg)
|
||||||
|
|
||||||
|
if (op === 'ADD') {
|
||||||
|
const anchor = (args[0] || '').toUpperCase()
|
||||||
|
if (!VA_ANCHORS.includes(anchor as VaAnchor)) {
|
||||||
|
problems.push(
|
||||||
|
`va_parmap "${entry}" was ignored: "${args[0] || ''}" is not a known anchor (${VA_ANCHORS.join(', ')}).`
|
||||||
|
)
|
||||||
|
return undefined
|
||||||
|
}
|
||||||
|
return { op: 'ADD', anchor: anchor as VaAnchor, offset }
|
||||||
|
}
|
||||||
|
if (op === 'RANGE') {
|
||||||
|
const period = (args[0] || '').toUpperCase()
|
||||||
|
if (!VA_PERIODS.includes(period as VaPeriod)) {
|
||||||
|
problems.push(
|
||||||
|
`va_parmap "${entry}" was ignored: "${args[0] || ''}" is not a known period (${VA_PERIODS.join(', ')}).`
|
||||||
|
)
|
||||||
|
return undefined
|
||||||
|
}
|
||||||
|
return { op: 'RANGE', period: period as VaPeriod, offset }
|
||||||
|
}
|
||||||
|
problems.push(
|
||||||
|
`va_parmap "${entry}" was ignored: "${match[1]}" is not a known transform (ADD, RANGE).`
|
||||||
|
)
|
||||||
|
return undefined
|
||||||
|
}
|
||||||
@@ -0,0 +1,194 @@
|
|||||||
|
import {
|
||||||
|
applyVaTransform,
|
||||||
|
isRangeTransform,
|
||||||
|
VA_ANCHORS,
|
||||||
|
VA_PERIODS,
|
||||||
|
VaTransform
|
||||||
|
} from './va-parmap-transform'
|
||||||
|
|
||||||
|
/** Local y-m-d, so assertions do not depend on the runner's timezone offset. */
|
||||||
|
const ymd = (d: Date) =>
|
||||||
|
`${d.getFullYear()}-${String(d.getMonth() + 1).padStart(2, '0')}-${String(
|
||||||
|
d.getDate()
|
||||||
|
).padStart(2, '0')}`
|
||||||
|
|
||||||
|
const date = (y: number, m: number, d: number) => new Date(y, m - 1, d)
|
||||||
|
|
||||||
|
describe('va_parmap transforms', () => {
|
||||||
|
it('exposes only allow-listed anchors and periods', () => {
|
||||||
|
expect(VA_ANCHORS).toContain('QUARTER_START')
|
||||||
|
expect(VA_PERIODS).toEqual(['QUARTER', 'MONTH', 'YEAR'])
|
||||||
|
})
|
||||||
|
|
||||||
|
describe('ADD - the value plus one anchor beside it', () => {
|
||||||
|
it('adds the previous quarter start, which is the reported requirement', () => {
|
||||||
|
// intnx('qtr', dt, -1, 'b') for a value on a quarter boundary
|
||||||
|
const dates = applyVaTransform(date(2026, 10, 1), {
|
||||||
|
op: 'ADD',
|
||||||
|
anchor: 'QUARTER_START',
|
||||||
|
offset: -1
|
||||||
|
})
|
||||||
|
expect(dates.map(ymd)).toEqual(['2026-10-01', '2026-07-01'])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('resolves a mid-quarter value to its own value plus the previous quarter start', () => {
|
||||||
|
const dates = applyVaTransform(date(2026, 8, 15), {
|
||||||
|
op: 'ADD',
|
||||||
|
anchor: 'QUARTER_START',
|
||||||
|
offset: -1
|
||||||
|
})
|
||||||
|
expect(dates.map(ymd)).toEqual(['2026-08-15', '2026-04-01'])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('de-duplicates when the value already sits on the anchor', () => {
|
||||||
|
// 2026-01-01 is a quarter start, so "0 quarters back" adds nothing
|
||||||
|
const dates = applyVaTransform(date(2026, 1, 1), {
|
||||||
|
op: 'ADD',
|
||||||
|
anchor: 'QUARTER_START',
|
||||||
|
offset: 0
|
||||||
|
})
|
||||||
|
expect(dates.map(ymd)).toEqual(['2026-01-01'])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('resolves the other anchors at their boundaries', () => {
|
||||||
|
const at = date(2026, 8, 15)
|
||||||
|
expect(
|
||||||
|
ymd(
|
||||||
|
applyVaTransform(at, {
|
||||||
|
op: 'ADD',
|
||||||
|
anchor: 'QUARTER_END',
|
||||||
|
offset: 0
|
||||||
|
})[1]
|
||||||
|
)
|
||||||
|
).toBe('2026-09-30')
|
||||||
|
expect(
|
||||||
|
ymd(
|
||||||
|
applyVaTransform(at, { op: 'ADD', anchor: 'MONTH_END', offset: 0 })[1]
|
||||||
|
)
|
||||||
|
).toBe('2026-08-31')
|
||||||
|
expect(
|
||||||
|
ymd(
|
||||||
|
applyVaTransform(at, {
|
||||||
|
op: 'ADD',
|
||||||
|
anchor: 'YEAR_START',
|
||||||
|
offset: -1
|
||||||
|
})[1]
|
||||||
|
)
|
||||||
|
).toBe('2025-01-01')
|
||||||
|
expect(
|
||||||
|
ymd(
|
||||||
|
applyVaTransform(at, { op: 'ADD', anchor: 'YEAR_END', offset: 0 })[1]
|
||||||
|
)
|
||||||
|
).toBe('2026-12-31')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('handles February and leap years at month end', () => {
|
||||||
|
expect(
|
||||||
|
ymd(
|
||||||
|
applyVaTransform(date(2026, 3, 1), {
|
||||||
|
op: 'ADD',
|
||||||
|
anchor: 'MONTH_END',
|
||||||
|
offset: -1
|
||||||
|
})[1]
|
||||||
|
)
|
||||||
|
).toBe('2026-02-28')
|
||||||
|
expect(
|
||||||
|
ymd(
|
||||||
|
applyVaTransform(date(2024, 3, 15), {
|
||||||
|
op: 'ADD',
|
||||||
|
anchor: 'MONTH_END',
|
||||||
|
offset: -1
|
||||||
|
})[1]
|
||||||
|
)
|
||||||
|
).toBe('2024-02-29')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('crosses a year boundary when the offset takes it out of the year', () => {
|
||||||
|
const dates = applyVaTransform(date(2027, 1, 1), {
|
||||||
|
op: 'ADD',
|
||||||
|
anchor: 'QUARTER_START',
|
||||||
|
offset: -1
|
||||||
|
})
|
||||||
|
expect(dates.map(ymd)).toEqual(['2027-01-01', '2026-10-01'])
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
describe('RANGE - a closed period', () => {
|
||||||
|
it('covers the previous quarter', () => {
|
||||||
|
const dates = applyVaTransform(date(2026, 8, 15), {
|
||||||
|
op: 'RANGE',
|
||||||
|
period: 'QUARTER',
|
||||||
|
offset: -1
|
||||||
|
})
|
||||||
|
expect(dates.map(ymd)).toEqual(['2026-04-01', '2026-06-30'])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('covers the current month and the previous year', () => {
|
||||||
|
expect(
|
||||||
|
applyVaTransform(date(2026, 8, 15), {
|
||||||
|
op: 'RANGE',
|
||||||
|
period: 'MONTH',
|
||||||
|
offset: 0
|
||||||
|
}).map(ymd)
|
||||||
|
).toEqual(['2026-08-01', '2026-08-31'])
|
||||||
|
expect(
|
||||||
|
applyVaTransform(date(2026, 8, 15), {
|
||||||
|
op: 'RANGE',
|
||||||
|
period: 'YEAR',
|
||||||
|
offset: -1
|
||||||
|
}).map(ymd)
|
||||||
|
).toEqual(['2025-01-01', '2025-12-31'])
|
||||||
|
})
|
||||||
|
|
||||||
|
it('reports itself as a range so the caller picks BETWEEN', () => {
|
||||||
|
expect(
|
||||||
|
isRangeTransform({ op: 'RANGE', period: 'QUARTER', offset: 0 })
|
||||||
|
).toBe(true)
|
||||||
|
expect(
|
||||||
|
isRangeTransform({
|
||||||
|
op: 'ADD',
|
||||||
|
anchor: 'QUARTER_START',
|
||||||
|
offset: 0
|
||||||
|
} as VaTransform)
|
||||||
|
).toBe(false)
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
describe('time of day', () => {
|
||||||
|
it('gives a datetime range whole days, not the value clock time', () => {
|
||||||
|
// A BETWEEN that starts at 13:45 on day one loses the earlier part of that
|
||||||
|
// day, and one that ends at 13:45 on the last day loses the rest of it.
|
||||||
|
const value = new Date(2026, 7, 15, 13, 45, 30)
|
||||||
|
const [start, end] = applyVaTransform(
|
||||||
|
value,
|
||||||
|
{ op: 'RANGE', period: 'QUARTER', offset: 0 },
|
||||||
|
true
|
||||||
|
)
|
||||||
|
expect([
|
||||||
|
start.getHours(),
|
||||||
|
start.getMinutes(),
|
||||||
|
start.getSeconds()
|
||||||
|
]).toEqual([0, 0, 0])
|
||||||
|
expect([end.getHours(), end.getMinutes(), end.getSeconds()]).toEqual([
|
||||||
|
23, 59, 59
|
||||||
|
])
|
||||||
|
expect(ymd(start)).toBe('2026-07-01')
|
||||||
|
expect(ymd(end)).toBe('2026-09-30')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('keeps the time for datetime columns and drops it otherwise', () => {
|
||||||
|
const value = new Date(2026, 9, 1, 13, 45, 30)
|
||||||
|
const add: VaTransform = {
|
||||||
|
op: 'ADD',
|
||||||
|
anchor: 'QUARTER_START',
|
||||||
|
offset: -1
|
||||||
|
}
|
||||||
|
const kept = applyVaTransform(value, add, true)[1]
|
||||||
|
expect([kept.getHours(), kept.getMinutes(), kept.getSeconds()]).toEqual([
|
||||||
|
13, 45, 30
|
||||||
|
])
|
||||||
|
const dropped = applyVaTransform(value, add, false)[1]
|
||||||
|
expect([dropped.getHours(), dropped.getMinutes()]).toEqual([0, 0])
|
||||||
|
})
|
||||||
|
})
|
||||||
|
})
|
||||||
@@ -0,0 +1,151 @@
|
|||||||
|
/**
|
||||||
|
* Anchors and periods a `va_parmap` transform can resolve a report value to.
|
||||||
|
*
|
||||||
|
* The names describe the result rather than mirroring SAS `intnx`, because the
|
||||||
|
* arithmetic runs in the browser: emulating `intnx`'s alignment letters (`b`,
|
||||||
|
* `e`, `s`, `m`, never mind week alignment and fiscal quarters) would invite
|
||||||
|
* assumptions we could not honour. Each anchor below is one small function, so
|
||||||
|
* adding one is a function plus a test row.
|
||||||
|
*/
|
||||||
|
export const VA_ANCHORS = [
|
||||||
|
'QUARTER_START',
|
||||||
|
'QUARTER_END',
|
||||||
|
'MONTH_START',
|
||||||
|
'MONTH_END',
|
||||||
|
'YEAR_START',
|
||||||
|
'YEAR_END'
|
||||||
|
] as const
|
||||||
|
export type VaAnchor = (typeof VA_ANCHORS)[number]
|
||||||
|
|
||||||
|
export const VA_PERIODS = ['QUARTER', 'MONTH', 'YEAR'] as const
|
||||||
|
export type VaPeriod = (typeof VA_PERIODS)[number]
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A parsed transform: the operation plus its validated arguments.
|
||||||
|
*
|
||||||
|
* `ADD` yields the value and one anchor beside it (a list of dates), which is
|
||||||
|
* how "this period's row plus the previous quarter's" is expressed. `RANGE`
|
||||||
|
* yields a closed period (two dates) for a BETWEEN.
|
||||||
|
*/
|
||||||
|
export type VaTransform =
|
||||||
|
| { op: 'ADD'; anchor: VaAnchor; offset: number }
|
||||||
|
| { op: 'RANGE'; period: VaPeriod; offset: number }
|
||||||
|
|
||||||
|
/** True when the transform produces a closed range rather than a list. */
|
||||||
|
export function isRangeTransform(transform: VaTransform): boolean {
|
||||||
|
return transform.op === 'RANGE'
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Local midnight on the first day of the month `offset` months from `date`. */
|
||||||
|
function monthStart(date: Date, offset: number): Date {
|
||||||
|
return new Date(date.getFullYear(), date.getMonth() + offset, 1)
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Local midnight on the last day of the month `offset` months from `date`. */
|
||||||
|
function monthEnd(date: Date, offset: number): Date {
|
||||||
|
return new Date(date.getFullYear(), date.getMonth() + offset + 1, 0)
|
||||||
|
}
|
||||||
|
|
||||||
|
/** First day of the quarter `offset` quarters from the one containing `date`. */
|
||||||
|
function quarterStart(date: Date, offset: number): Date {
|
||||||
|
const firstMonthOfQuarter = 3 * Math.floor(date.getMonth() / 3)
|
||||||
|
return new Date(date.getFullYear(), firstMonthOfQuarter + offset * 3, 1)
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Last day of the quarter `offset` quarters from the one containing `date`. */
|
||||||
|
function quarterEnd(date: Date, offset: number): Date {
|
||||||
|
// Day 0 of a month is the last day of the month before it: calendar
|
||||||
|
// construction, so a daylight-saving transition inside the quarter cannot
|
||||||
|
// shift the result by an hour the way subtracting 24 hours would.
|
||||||
|
const nextStart = quarterStart(date, offset + 1)
|
||||||
|
return new Date(nextStart.getFullYear(), nextStart.getMonth(), 0)
|
||||||
|
}
|
||||||
|
|
||||||
|
/** First day of the year `offset` years from the one containing `date`. */
|
||||||
|
function yearStart(date: Date, offset: number): Date {
|
||||||
|
return new Date(date.getFullYear() + offset, 0, 1)
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Last day of the year `offset` years from the one containing `date`. */
|
||||||
|
function yearEnd(date: Date, offset: number): Date {
|
||||||
|
return new Date(date.getFullYear() + offset, 11, 31)
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Resolve one anchor, keeping the time of day of `date` for datetime columns. */
|
||||||
|
function resolveAnchor(
|
||||||
|
anchor: VaAnchor,
|
||||||
|
date: Date,
|
||||||
|
offset: number,
|
||||||
|
keepTime: boolean
|
||||||
|
): Date {
|
||||||
|
let resolved: Date
|
||||||
|
switch (anchor) {
|
||||||
|
case 'QUARTER_START':
|
||||||
|
resolved = quarterStart(date, offset)
|
||||||
|
break
|
||||||
|
case 'QUARTER_END':
|
||||||
|
resolved = quarterEnd(date, offset)
|
||||||
|
break
|
||||||
|
case 'MONTH_START':
|
||||||
|
resolved = monthStart(date, offset)
|
||||||
|
break
|
||||||
|
case 'MONTH_END':
|
||||||
|
resolved = monthEnd(date, offset)
|
||||||
|
break
|
||||||
|
case 'YEAR_START':
|
||||||
|
resolved = yearStart(date, offset)
|
||||||
|
break
|
||||||
|
default:
|
||||||
|
resolved = yearEnd(date, offset)
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if (keepTime) {
|
||||||
|
resolved.setHours(date.getHours(), date.getMinutes(), date.getSeconds(), 0)
|
||||||
|
}
|
||||||
|
return resolved
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The dates a transform adds to a mapping, in order.
|
||||||
|
*
|
||||||
|
* `ADD` returns the report's own value plus the anchor beside it, de-duplicated -
|
||||||
|
* so a value that already sits on the anchor yields one date, and the clause
|
||||||
|
* stays an equality rather than an `IN` with a repeat. `RANGE` returns the
|
||||||
|
* period's first and last day.
|
||||||
|
*
|
||||||
|
* `keepTime` carries the time of day across for datetime columns, since a
|
||||||
|
* DATETIME filter compares to the second; date columns use midnight.
|
||||||
|
*/
|
||||||
|
export function applyVaTransform(
|
||||||
|
value: Date,
|
||||||
|
transform: VaTransform,
|
||||||
|
keepTime = false
|
||||||
|
): Date[] {
|
||||||
|
if (transform.op === 'RANGE') {
|
||||||
|
// A range covers whole days: the bounds are the period's own, never the
|
||||||
|
// value's clock time. Reusing keepTime here would start the range mid-morning
|
||||||
|
// on day one and end it mid-morning on the last day, dropping the rest of
|
||||||
|
// both boundary days from a BETWEEN.
|
||||||
|
const start = resolveAnchor(
|
||||||
|
(transform.period + '_START') as VaAnchor,
|
||||||
|
value,
|
||||||
|
transform.offset,
|
||||||
|
false
|
||||||
|
)
|
||||||
|
const end = resolveAnchor(
|
||||||
|
(transform.period + '_END') as VaAnchor,
|
||||||
|
value,
|
||||||
|
transform.offset,
|
||||||
|
false
|
||||||
|
)
|
||||||
|
if (keepTime) end.setHours(23, 59, 59, 0)
|
||||||
|
return [start, end]
|
||||||
|
}
|
||||||
|
const anchor = resolveAnchor(
|
||||||
|
transform.anchor,
|
||||||
|
value,
|
||||||
|
transform.offset,
|
||||||
|
keepTime
|
||||||
|
)
|
||||||
|
return anchor.getTime() === value.getTime() ? [value] : [value, anchor]
|
||||||
|
}
|
||||||
+1
-1
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "dcfrontend",
|
"name": "dcfrontend",
|
||||||
"version": "7.19.0",
|
"version": "7.21.0",
|
||||||
"description": "Data Controller",
|
"description": "Data Controller",
|
||||||
"devDependencies": {
|
"devDependencies": {
|
||||||
"@nogoo9/gitleaks": "8.30.1-post.2",
|
"@nogoo9/gitleaks": "8.30.1-post.2",
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
{
|
{
|
||||||
"fromjs": [
|
"fromjs": [
|
||||||
{
|
{
|
||||||
"ADMIN": "AllUsers",
|
"ADMIN": "all-users",
|
||||||
"DCPATH": "/tmp/dcdata",
|
"DCPATH": "/tmp/dcdata",
|
||||||
"OUTFMT": "JSON"
|
"OUTFMT": "JSON"
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -13,10 +13,12 @@
|
|||||||
// The rules mirror services/hooks/mpe_derived_postedit.sas, in its order:
|
// The rules mirror services/hooks/mpe_derived_postedit.sas, in its order:
|
||||||
// normalise the incoming rows, reject a duplicate name and a bad field, then
|
// normalise the incoming rows, reject a duplicate name and a bad field, then
|
||||||
// check the dependency graph that the resulting configuration would have (the
|
// check the dependency graph that the resulting configuration would have (the
|
||||||
// incoming rows plus the current rows they do not replace). As on the server,
|
// incoming rows plus the current rows they do not replace). The field rules
|
||||||
// the collision between VAR_NM and a physical column is left to the read-time
|
// are core's mp_derivedvarcheck, which the server hook calls and the read-time
|
||||||
// engine, which runs with the target library assigned - the hook does not have
|
// engine runs too - this file spells them out a second time, so a change there
|
||||||
// it here either.
|
// has to be mirrored here. As on the server, the collision between VAR_NM and
|
||||||
|
// a physical column is left to the read-time engine, which runs with the target
|
||||||
|
// library assigned - the hook does not have it here either.
|
||||||
// ─────────────────────────────────────────────────────────────────────────────
|
// ─────────────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
;(function mpeDerivedPostedit() {
|
;(function mpeDerivedPostedit() {
|
||||||
@@ -83,9 +85,45 @@
|
|||||||
seen.add(key)
|
seen.add(key)
|
||||||
}
|
}
|
||||||
|
|
||||||
// ─── Field level checks, the rules the engine applies at read time ─────────
|
// ─── Field level checks ────────────────────────────────────────────────────
|
||||||
// C0 controls and DEL: what a single-line label cannot carry
|
// Two passes, in the order the SAS hook runs them. The first is the fields
|
||||||
|
// this hook owns - the control table key, which the loader normalises but
|
||||||
|
// does not require, and ACTIVE, the picker pair the EDIT screen offers.
|
||||||
|
for (const row of rows) {
|
||||||
|
if (isDeleted(row)) continue
|
||||||
|
const name = field(row, 'VAR_NM')
|
||||||
|
let reason = ''
|
||||||
|
if (!field(row, 'LIBREF') || !field(row, 'DSN'))
|
||||||
|
reason = 'LIBREF and DSN are required'
|
||||||
|
// ACTIVE is a picker in the EDIT screen (HARDSELECT over the
|
||||||
|
// MPE_SELECTBOX pair) - the same pair the hook enforces
|
||||||
|
else if (!['0', '1'].includes(String(colVal(row, 'ACTIVE')).trim()))
|
||||||
|
reason = 'ACTIVE must be 1 or 0'
|
||||||
|
if (reason) return reject(`MPE_DERIVED: ${name}: ${reason}`)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The second pass is the rules for the derived variable itself, which are
|
||||||
|
// core's (mp_derivedvarcheck): the engine runs the same macro at read time,
|
||||||
|
// so a definition is measured against one rule list wherever it is checked.
|
||||||
|
|
||||||
|
// C0 controls and DEL: what a label cannot carry once a line break has been
|
||||||
|
// flattened to a space, which is what the engine does with it
|
||||||
const CONTROL_CHARS = /[\u0000-\u001F\u007F]/
|
const CONTROL_CHARS = /[\u0000-\u001F\u007F]/
|
||||||
|
const flatten = (v) => v.replace(/[\r\n]+/g, ' ')
|
||||||
|
const AUTOMATIC_VARS = ['_N_', '_ERROR_', '_IORC_']
|
||||||
|
|
||||||
|
// the capabilities refused by name - a function name cannot be assembled at
|
||||||
|
// run time, so naming them is a control the expression cannot evade
|
||||||
|
const DENY_FUNCS = new RegExp(
|
||||||
|
'\\b(DOSUBL|JAVAOBJ|JAVAMETA|SYMGET|SYMPUTX?|SYSGET|GETOPTION' +
|
||||||
|
'|PATHNAME|FILEEXIST|FOPEN|FREAD|FGET|FCLOSE' +
|
||||||
|
'|DOPEN|DREAD|DNUM|MOPEN|SLEEP' +
|
||||||
|
'|FILENAME|FDELETE|FCOPY|DCREATE|DLCREATE)\\s*\\('
|
||||||
|
)
|
||||||
|
|
||||||
|
// quoted literals are data, not calls
|
||||||
|
const unquoted = (v) =>
|
||||||
|
v.replace(/'([^']|'')*'/g, ' ').replace(/"([^"]|"")*"/g, ' ')
|
||||||
|
|
||||||
// true when the value carries a character outside the allowed set
|
// true when the value carries a character outside the allowed set
|
||||||
const hasInvalidChar = (value, chars) =>
|
const hasInvalidChar = (value, chars) =>
|
||||||
@@ -93,8 +131,6 @@
|
|||||||
|
|
||||||
for (const row of rows) {
|
for (const row of rows) {
|
||||||
if (isDeleted(row)) continue
|
if (isDeleted(row)) continue
|
||||||
const libref = field(row, 'LIBREF')
|
|
||||||
const dsn = field(row, 'DSN')
|
|
||||||
const name = field(row, 'VAR_NM')
|
const name = field(row, 'VAR_NM')
|
||||||
const type = field(row, 'VAR_TYPE')
|
const type = field(row, 'VAR_TYPE')
|
||||||
const len = Number(colVal(row, 'VAR_LEN'))
|
const len = Number(colVal(row, 'VAR_LEN'))
|
||||||
@@ -103,19 +139,16 @@
|
|||||||
const logic = strip(colVal(row, 'VAR_LOGIC'))
|
const logic = strip(colVal(row, 'VAR_LOGIC'))
|
||||||
|
|
||||||
let reason = ''
|
let reason = ''
|
||||||
if (!libref || !dsn) reason = 'LIBREF and DSN are required'
|
if (!name) reason = 'VAR_NM is required'
|
||||||
else if (!name) reason = 'VAR_NM is required'
|
|
||||||
else if (hasInvalidChar(name, NAME_CHARS))
|
else if (hasInvalidChar(name, NAME_CHARS))
|
||||||
reason = 'VAR_NM must be a valid SAS name'
|
reason = 'VAR_NM must be a valid SAS name'
|
||||||
else if ('0123456789'.includes(name[0]))
|
else if ('0123456789'.includes(name[0]))
|
||||||
reason = 'VAR_NM must not start with a digit'
|
reason = 'VAR_NM must not start with a digit'
|
||||||
|
else if (AUTOMATIC_VARS.includes(name))
|
||||||
|
reason = 'VAR_NM is an automatic variable of the data step'
|
||||||
else if (name === DELETE_MARK) reason = 'VAR_NM is reserved'
|
else if (name === DELETE_MARK) reason = 'VAR_NM is reserved'
|
||||||
else if (type !== 'C' && type !== 'N')
|
else if (type !== 'C' && type !== 'N')
|
||||||
reason = 'VAR_TYPE must be C or N'
|
reason = 'VAR_TYPE must be C or N'
|
||||||
// ACTIVE is a picker in the EDIT screen (HARDSELECT over the
|
|
||||||
// MPE_SELECTBOX pair) - the same pair mpe_derived_check.sas enforces
|
|
||||||
else if (!['0', '1'].includes(String(colVal(row, 'ACTIVE')).trim()))
|
|
||||||
reason = 'ACTIVE must be 1 or 0'
|
|
||||||
else if (!Number.isInteger(len) || len < 1)
|
else if (!Number.isInteger(len) || len < 1)
|
||||||
reason = 'VAR_LEN must be a positive integer'
|
reason = 'VAR_LEN must be a positive integer'
|
||||||
else if (type === 'C' && len > 32767)
|
else if (type === 'C' && len > 32767)
|
||||||
@@ -132,20 +165,24 @@
|
|||||||
reason = 'VAR_FMT must be a format name, without ; & % or /* */'
|
reason = 'VAR_FMT must be a format name, without ; & % or /* */'
|
||||||
else if (hasInvalidChar(fmt, FMT_CHARS))
|
else if (hasInvalidChar(fmt, FMT_CHARS))
|
||||||
reason = 'VAR_FMT must be a format name'
|
reason = 'VAR_FMT must be a format name'
|
||||||
// free text, in any script - what is refused is anything that cannot be
|
// free text, in any script - what is refused is a control character,
|
||||||
// a single-line column label, and a value the engine would cut to 256
|
// and a value the engine would cut to 256
|
||||||
else if (CONTROL_CHARS.test(label))
|
else if (CONTROL_CHARS.test(flatten(label)))
|
||||||
reason =
|
reason = 'VAR_LABEL must be plain text - no control characters'
|
||||||
'VAR_LABEL must be plain text - no line breaks or control characters'
|
|
||||||
else if ([...label].length > 256)
|
else if ([...label].length > 256)
|
||||||
reason = 'VAR_LABEL exceeds 256 characters'
|
reason = 'VAR_LABEL exceeds 256 characters'
|
||||||
else if (!logic) reason = 'VAR_LOGIC is required'
|
else if (!logic) reason = 'VAR_LOGIC is required'
|
||||||
|
else if (logic.length > 4000)
|
||||||
|
reason = 'VAR_LOGIC exceeds 4000 characters'
|
||||||
else if (logic.includes(';'))
|
else if (logic.includes(';'))
|
||||||
reason = 'VAR_LOGIC must be a single expression - remove the ;'
|
reason = 'VAR_LOGIC must be a single expression - remove the ;'
|
||||||
else if (logic.includes('&') || logic.includes('%'))
|
else if (logic.includes('&') || logic.includes('%'))
|
||||||
reason = 'VAR_LOGIC must not contain macro triggers'
|
reason = 'VAR_LOGIC must not contain macro triggers'
|
||||||
else if (logic.includes('/*') || logic.includes('*/'))
|
else if (logic.includes('/*') || logic.includes('*/'))
|
||||||
reason = 'VAR_LOGIC must not contain comments'
|
reason = 'VAR_LOGIC must not contain comments'
|
||||||
|
else if (DENY_FUNCS.test(unquoted(logic).toUpperCase()))
|
||||||
|
reason =
|
||||||
|
'VAR_LOGIC may not call a function that submits code or reaches the file system, the environment or the macro symbol table'
|
||||||
|
|
||||||
if (reason) return reject(`MPE_DERIVED: ${name}: ${reason}`)
|
if (reason) return reject(`MPE_DERIVED: ${name}: ${reason}`)
|
||||||
}
|
}
|
||||||
|
|||||||
Generated
+4
-4
@@ -7,7 +7,7 @@
|
|||||||
"name": "dc-sas",
|
"name": "dc-sas",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@sasjs/cli": "6.2.0",
|
"@sasjs/cli": "6.2.0",
|
||||||
"@sasjs/core": "6.1.0"
|
"@sasjs/core": "6.1.4"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"node_modules/@asamuzakjp/css-color": {
|
"node_modules/@asamuzakjp/css-color": {
|
||||||
@@ -235,9 +235,9 @@
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
"node_modules/@sasjs/core": {
|
"node_modules/@sasjs/core": {
|
||||||
"version": "6.1.0",
|
"version": "6.1.4",
|
||||||
"resolved": "https://registry.npmjs.org/@sasjs/core/-/core-6.1.0.tgz",
|
"resolved": "https://registry.npmjs.org/@sasjs/core/-/core-6.1.4.tgz",
|
||||||
"integrity": "sha512-U0Ekq/TXVMquYV+AG6urBQSUgqUvV37yduufy68QQVgDj3PJiLUJDcOXKOGbhWGD+Nv877VuPCviGYqrqbjXvg==",
|
"integrity": "sha512-vIF2LIftxPWg4CvO3i5raEfrHFHY/6S+MqVGpjRR71dgjLRJEYNyPJgGqD6BCZwHv5V6/2nDnBi/BIIMiLXfzA==",
|
||||||
"license": "MIT"
|
"license": "MIT"
|
||||||
},
|
},
|
||||||
"node_modules/@sasjs/lint": {
|
"node_modules/@sasjs/lint": {
|
||||||
|
|||||||
+1
-1
@@ -34,7 +34,7 @@
|
|||||||
"private": true,
|
"private": true,
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@sasjs/cli": "6.2.0",
|
"@sasjs/cli": "6.2.0",
|
||||||
"@sasjs/core": "6.1.0"
|
"@sasjs/core": "6.1.4"
|
||||||
},
|
},
|
||||||
"overrides": {
|
"overrides": {
|
||||||
"nanoid": "3.3.18"
|
"nanoid": "3.3.18"
|
||||||
|
|||||||
@@ -49,9 +49,19 @@
|
|||||||
being maintained. A cycle is rejected.
|
being maintained. A cycle is rejected.
|
||||||
|
|
||||||
Semicolons, macro triggers and comments are rejected, so a row can only ever
|
Semicolons, macro triggers and comments are rejected, so a row can only ever
|
||||||
be an expression. A derived variable may not reuse the name of a physical
|
be an expression, and an expression that would let the row submit code of its
|
||||||
column, and may not reference itself: an intra-row calculation would imply a
|
own - DOSUBL, or a function reaching the file system, the environment or the
|
||||||
retained value, which depends on the order rows happen to arrive in.
|
macro symbol table - is refused by name.
|
||||||
|
|
||||||
|
The field level rules themselves are core's: mp_derivedvarcheck scans the
|
||||||
|
definitions and mp_derivedvars calls it, which is the same macro the MPE_DERIVED
|
||||||
|
post edit hook runs at submit time (services/hooks/mpe_derived_postedit). A
|
||||||
|
row is therefore measured against one rule list, whether it is being staged
|
||||||
|
or read.
|
||||||
|
|
||||||
|
A derived variable may not reuse the name of a physical column, and may not
|
||||||
|
reference itself: an intra-row calculation would imply a retained value,
|
||||||
|
which depends on the order rows happen to arrive in.
|
||||||
|
|
||||||
<h4> SAS Macros </h4>
|
<h4> SAS Macros </h4>
|
||||||
@li mp_derivedvars.sas
|
@li mp_derivedvars.sas
|
||||||
|
|||||||
@@ -1,75 +0,0 @@
|
|||||||
/**
|
|
||||||
@file
|
|
||||||
@brief Field level checks for a staged MPE_DERIVED submission
|
|
||||||
@details Writes one row per staged row that breaks a rule, with the reason in
|
|
||||||
REASON, so a caller can report the first failure and abort. Rows marked for
|
|
||||||
deletion are skipped - they are being removed, so their fields are not loaded.
|
|
||||||
|
|
||||||
The rules are the ones %mpe_derived applies at read time, so a submission
|
|
||||||
cannot stage a row the engine would then refuse.
|
|
||||||
|
|
||||||
@param inds the staged rows, already normalised by the caller
|
|
||||||
@param outds the dataset to create: one row per failing row, plus REASON
|
|
||||||
|
|
||||||
@author 4GL Apps Ltd
|
|
||||||
@copyright 4GL Apps Ltd. This code may only be used within Data Controller
|
|
||||||
and may not be re-distributed or re-sold without the express permission of
|
|
||||||
4GL Apps Ltd.
|
|
||||||
|
|
||||||
**/
|
|
||||||
|
|
||||||
%macro mpe_derived_check(inds=work.staging_ds, outds=work.derived_check);
|
|
||||||
data &outds;
|
|
||||||
set &inds;
|
|
||||||
length reason $200;
|
|
||||||
if upcase(_____DELETE__THIS__RECORD_____)='YES' then delete;
|
|
||||||
call missing(reason);
|
|
||||||
|
|
||||||
if missing(libref) or missing(dsn) then reason='LIBREF and DSN are required';
|
|
||||||
else if var_nm='' then reason='VAR_NM is required';
|
|
||||||
else if verify(strip(var_nm),'ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_')>0
|
|
||||||
then reason='VAR_NM must be a valid SAS name';
|
|
||||||
else if verify(substr(var_nm,1,1),'0123456789')=0
|
|
||||||
then reason='VAR_NM must not start with a digit';
|
|
||||||
else if var_nm='_____DELETE__THIS__RECORD_____'
|
|
||||||
then reason='VAR_NM is reserved';
|
|
||||||
else if var_type not in ('C','N')
|
|
||||||
then reason='VAR_TYPE must be C or N';
|
|
||||||
/* ACTIVE is a picker in the EDIT screen (HARDSELECT over the MPE_SELECTBOX
|
|
||||||
pair) - the same pair is enforced here, so a submission that bypasses the
|
|
||||||
grid cannot land a value the screen would never offer. */
|
|
||||||
else if missing(active) or active not in (0,1)
|
|
||||||
then reason='ACTIVE must be 1 or 0';
|
|
||||||
else if missing(var_len) or var_len ne int(var_len) or var_len<1
|
|
||||||
then reason='VAR_LEN must be a positive integer';
|
|
||||||
else if var_type='C' and var_len>32767
|
|
||||||
then reason='VAR_LEN exceeds 32767 for a character variable';
|
|
||||||
else if var_type='N' and var_len>8
|
|
||||||
then reason='VAR_LEN exceeds 8 for a numeric variable';
|
|
||||||
else if index(var_fmt,';')>0 or indexc(var_fmt,'&%')>0
|
|
||||||
or index(var_fmt,'/*')>0 or index(var_fmt,'*/')>0
|
|
||||||
then reason='VAR_FMT must be a format name, without ; & % or /* */';
|
|
||||||
else if verify(strip(var_fmt),'ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_$.')>0
|
|
||||||
then reason='VAR_FMT must be a format name';
|
|
||||||
/* VAR_LABEL is free text: any character is allowed, in any script, and the
|
|
||||||
quotes in it are escaped by %mp_derivedvars when the attrib statement is
|
|
||||||
built. What is refused is anything that cannot be a single-line column
|
|
||||||
label - a control character or a line break - and a value long enough to
|
|
||||||
be cut down silently, since the engine truncates the label to 256. */
|
|
||||||
else if anycntrl(var_label)>0
|
|
||||||
then reason='VAR_LABEL must be plain text - no line breaks or control characters';
|
|
||||||
else if klength(var_label)>256
|
|
||||||
then reason='VAR_LABEL exceeds 256 characters';
|
|
||||||
else if missing(var_logic)
|
|
||||||
then reason='VAR_LOGIC is required';
|
|
||||||
else if index(var_logic,';')>0
|
|
||||||
then reason='VAR_LOGIC must be a single expression - remove the ;';
|
|
||||||
else if indexc(var_logic,'&%')>0
|
|
||||||
then reason='VAR_LOGIC must not contain macro triggers';
|
|
||||||
else if index(var_logic,'/*')>0 or index(var_logic,'*/')>0
|
|
||||||
then reason='VAR_LOGIC must not contain comments';
|
|
||||||
|
|
||||||
if not missing(reason) then output;
|
|
||||||
run;
|
|
||||||
|
|
||||||
%mend mpe_derived_check;
|
|
||||||
@@ -1,194 +0,0 @@
|
|||||||
/**
|
|
||||||
@file
|
|
||||||
@brief testing the mpe_derived_check macro
|
|
||||||
|
|
||||||
@details Checks each field level rule the MPE_DERIVED post edit hook applies
|
|
||||||
to a staged submission. The macro is exercised directly so the rules can be
|
|
||||||
probed with values a service cannot carry: VAR_LABEL is varchar(256) in the
|
|
||||||
table and a staging dataset declares the same length, so a longer label is
|
|
||||||
truncated before any check would see it.
|
|
||||||
|
|
||||||
<h4> SAS Macros </h4>
|
|
||||||
@li mf_nobs.sas
|
|
||||||
@li mp_assert.sas
|
|
||||||
@li mpe_derived_check.sas
|
|
||||||
|
|
||||||
@author 4GL Apps Ltd
|
|
||||||
@copyright 4GL Apps Ltd. This code may only be used within Data Controller
|
|
||||||
and may not be re-distributed or re-sold without the express permission of
|
|
||||||
4GL Apps Ltd.
|
|
||||||
|
|
||||||
**/
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Test 1 - one row per failing rule, and only failing rows
|
|
||||||
*/
|
|
||||||
data work.staging_ds;
|
|
||||||
length libref $8 dsn $32 var_nm $32 var_type $1 var_len 8
|
|
||||||
var_fmt $32 var_label $4000 var_deps $4000 var_logic $4000
|
|
||||||
active 8 _____DELETE__THIS__RECORD_____ $3;
|
|
||||||
libref='DCTEST';
|
|
||||||
dsn='DC_DERIV';
|
|
||||||
/* valid unless a row says otherwise - ACTIVE is mandatory in the screen and
|
|
||||||
the check enforces the same pair here */
|
|
||||||
active=1;
|
|
||||||
_____DELETE__THIS__RECORD_____='No';
|
|
||||||
|
|
||||||
/* meets every rule */
|
|
||||||
var_nm='GOOD';
|
|
||||||
var_type='N';
|
|
||||||
var_len=8;
|
|
||||||
var_fmt='';
|
|
||||||
var_label='a plain label';
|
|
||||||
var_deps='';
|
|
||||||
var_logic='1';
|
|
||||||
output;
|
|
||||||
|
|
||||||
/* delete marked rows are being removed, so their fields are not loaded */
|
|
||||||
var_nm='DELME';
|
|
||||||
var_label='first line'||'0A'x||'second line';
|
|
||||||
_____DELETE__THIS__RECORD_____='Yes';
|
|
||||||
output;
|
|
||||||
_____DELETE__THIS__RECORD_____='No';
|
|
||||||
|
|
||||||
/* free text is allowed, including quotes and punctuation */
|
|
||||||
var_nm='FREETXT';
|
|
||||||
var_label='It''s "quoted", plain text: 100 percent';
|
|
||||||
output;
|
|
||||||
|
|
||||||
/* a line break cannot be a single line column label */
|
|
||||||
var_nm='LBLBREAK';
|
|
||||||
var_label='first line'||'0A'x||'second line';
|
|
||||||
output;
|
|
||||||
|
|
||||||
/* longer than the table holds, so it would be cut down silently */
|
|
||||||
var_nm='LBLLONG';
|
|
||||||
var_label=repeat('x',300);
|
|
||||||
output;
|
|
||||||
|
|
||||||
/* a format name, not an expression */
|
|
||||||
var_nm='BADFMT';
|
|
||||||
var_fmt='DATE9;';
|
|
||||||
var_label='fmt';
|
|
||||||
output;
|
|
||||||
|
|
||||||
/* one expression, no statement separator, no macro triggers */
|
|
||||||
var_nm='BADLOGIC';
|
|
||||||
var_fmt='';
|
|
||||||
var_logic='x; y';
|
|
||||||
output;
|
|
||||||
|
|
||||||
/* C or N */
|
|
||||||
var_nm='BADTYPE';
|
|
||||||
var_logic='1';
|
|
||||||
var_type='X';
|
|
||||||
output;
|
|
||||||
|
|
||||||
/* ACTIVE is a picker in the screen: only 1 or 0 */
|
|
||||||
var_nm='BADACTIVE';
|
|
||||||
var_type='C';
|
|
||||||
var_logic='1';
|
|
||||||
active=2;
|
|
||||||
output;
|
|
||||||
|
|
||||||
stop;
|
|
||||||
run;
|
|
||||||
|
|
||||||
%mpe_derived_check(inds=work.staging_ds, outds=work.derived_check)
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Test 2 - the reported rows, rule by rule
|
|
||||||
*/
|
|
||||||
data work.expected;
|
|
||||||
length var_nm $32 reason $200;
|
|
||||||
var_nm='LBLBREAK';
|
|
||||||
reason='VAR_LABEL must be plain text - no line breaks or control characters';
|
|
||||||
output;
|
|
||||||
var_nm='LBLLONG';
|
|
||||||
reason='VAR_LABEL exceeds 256 characters';
|
|
||||||
output;
|
|
||||||
var_nm='BADFMT';
|
|
||||||
reason='VAR_FMT must be a format name, without ; & % or /* */';
|
|
||||||
output;
|
|
||||||
var_nm='BADLOGIC';
|
|
||||||
reason='VAR_LOGIC must be a single expression - remove the ;';
|
|
||||||
output;
|
|
||||||
var_nm='BADTYPE';
|
|
||||||
reason='VAR_TYPE must be C or N';
|
|
||||||
output;
|
|
||||||
var_nm='BADACTIVE';
|
|
||||||
reason='ACTIVE must be 1 or 0';
|
|
||||||
output;
|
|
||||||
run;
|
|
||||||
|
|
||||||
proc sql;
|
|
||||||
create table work.cmp as
|
|
||||||
select e.var_nm
|
|
||||||
,case when a.reason=e.reason then 1 else 0 end as same
|
|
||||||
from work.expected e
|
|
||||||
left join work.derived_check a
|
|
||||||
on upcase(strip(a.var_nm))=upcase(e.var_nm);
|
|
||||||
quit;
|
|
||||||
|
|
||||||
data _null_;
|
|
||||||
set work.cmp;
|
|
||||||
call symputx(cats('rule_',var_nm),same,'l');
|
|
||||||
run;
|
|
||||||
|
|
||||||
%mp_assert(
|
|
||||||
iftrue=(%mf_nobs(work.derived_check)=6),
|
|
||||||
desc=%str(Checking only the six failing rows are reported),
|
|
||||||
outds=work.test_results
|
|
||||||
)
|
|
||||||
|
|
||||||
%mp_assert(
|
|
||||||
iftrue=(&rule_LBLBREAK=1),
|
|
||||||
desc=%str(Checking a label with a line break is rejected),
|
|
||||||
outds=work.test_results
|
|
||||||
)
|
|
||||||
|
|
||||||
%mp_assert(
|
|
||||||
iftrue=(&rule_LBLLONG=1),
|
|
||||||
desc=%str(Checking a label over 256 characters is rejected),
|
|
||||||
outds=work.test_results
|
|
||||||
)
|
|
||||||
|
|
||||||
%mp_assert(
|
|
||||||
iftrue=(&rule_BADFMT=1),
|
|
||||||
desc=%str(Checking a format carrying a statement separator is rejected),
|
|
||||||
outds=work.test_results
|
|
||||||
)
|
|
||||||
|
|
||||||
%mp_assert(
|
|
||||||
iftrue=(&rule_BADLOGIC=1),
|
|
||||||
desc=%str(Checking a logic value with a statement separator is rejected),
|
|
||||||
outds=work.test_results
|
|
||||||
)
|
|
||||||
|
|
||||||
%mp_assert(
|
|
||||||
iftrue=(&rule_BADTYPE=1),
|
|
||||||
desc=%str(Checking a VAR_TYPE other than C or N is rejected),
|
|
||||||
outds=work.test_results
|
|
||||||
)
|
|
||||||
|
|
||||||
%mp_assert(
|
|
||||||
iftrue=(&rule_BADACTIVE=1),
|
|
||||||
desc=%str(Checking an ACTIVE outside 1 or 0 is rejected),
|
|
||||||
outds=work.test_results
|
|
||||||
)
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Test 3 - a row that meets every rule, a free text label, and a delete
|
|
||||||
* marked row are all left alone
|
|
||||||
*/
|
|
||||||
proc sql noprint;
|
|
||||||
select count(*) into: notchecked
|
|
||||||
from work.derived_check
|
|
||||||
where upcase(strip(var_nm)) in ('GOOD','FREETXT','DELME');
|
|
||||||
quit;
|
|
||||||
|
|
||||||
%mp_assert(
|
|
||||||
iftrue=(¬checked=0),
|
|
||||||
desc=%str(Checking a valid row, a free text label and a delete marked row are not reported),
|
|
||||||
outds=work.test_results
|
|
||||||
)
|
|
||||||
@@ -2355,7 +2355,9 @@ insert into &lib..MPE_VALIDATIONS set
|
|||||||
,some_char="&x bottles of beer on the wall"
|
,some_char="&x bottles of beer on the wall"
|
||||||
,some_dropdown='Option 1'
|
,some_dropdown='Option 1'
|
||||||
,some_num=ranuni(0)
|
,some_num=ranuni(0)
|
||||||
,some_date=round(ranuni(0)*1000,1)
|
/* First of the month, cycling through two years: a randomly spread date
|
||||||
|
gave every row its own day, so a date filter never returned many rows. */
|
||||||
|
,some_date=intnx('month','01JAN1960'd,mod(&x,24),'S')
|
||||||
,some_datetime=round(ranuni(0)*50000,1)
|
,some_datetime=round(ranuni(0)*50000,1)
|
||||||
,some_time=round(ranuni(0)*100,1)
|
,some_time=round(ranuni(0)*100,1)
|
||||||
,some_shortnum=6+round(ranuni(0)*94,1)
|
,some_shortnum=6+round(ranuni(0)*94,1)
|
||||||
|
|||||||
@@ -6,16 +6,19 @@
|
|||||||
table - the engine reads only active rows, so the dependency can never
|
table - the engine reads only active rows, so the dependency can never
|
||||||
resolve at run time
|
resolve at run time
|
||||||
2. a row is staged with active=0 while a surviving active row depends on it
|
2. a row is staged with active=0 while a surviving active row depends on it
|
||||||
|
3. a staged row breaks a field rule of the shared scan (core's
|
||||||
|
mp_derivedvarcheck), or carries an ACTIVE value the EDIT screen's picker
|
||||||
|
cannot produce
|
||||||
|
|
||||||
They should NOT abort when:
|
They should NOT abort when:
|
||||||
3. a new active row is added to a table that also holds an inactive row -
|
4. a new active row is added to a table that also holds an inactive row -
|
||||||
the inactive row is not part of the dependency graph, so it cannot
|
the inactive row is not part of the dependency graph, so it cannot
|
||||||
create a cycle or a stale reference
|
create a cycle or a stale reference
|
||||||
4. a delete-marked RLS rule naming a derived variable is submitted - the
|
5. a delete-marked RLS rule naming a derived variable is submitted - the
|
||||||
rule is being removed, so its variables are not validated
|
rule is being removed, so its variables are not validated
|
||||||
5. a delete and re-add of the same variable is submitted together - the
|
6. a delete and re-add of the same variable is submitted together - the
|
||||||
re-added active row resolves the dependency
|
re-added active row resolves the dependency
|
||||||
6. an inactive row is set active again while another active row depends on
|
7. an inactive row is set active again while another active row depends on
|
||||||
it - the re-activation is what makes the dependency resolvable
|
it - the re-activation is what makes the dependency resolvable
|
||||||
|
|
||||||
The staged loads are aborted by the post edit hook. The assertion fetches the
|
The staged loads are aborted by the post edit hook. The assertion fetches the
|
||||||
@@ -553,6 +556,103 @@ run;
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Test 8 - a field rule of the shared scan (mp_derivedvarcheck, core) is
|
||||||
|
* applied at submit time: an expression that could submit code of its own is
|
||||||
|
* refused by name, with the reason the scan gives. This is the same scan the
|
||||||
|
* engine runs at read time, so the case pins that the hook calls it rather
|
||||||
|
* than carrying a rule list of its own.
|
||||||
|
*/
|
||||||
|
data work.sascontroltable;
|
||||||
|
action='LOAD';
|
||||||
|
message='A refused function in VAR_LOGIC should abort';
|
||||||
|
libds="&dc_libref..MPE_DERIVED";
|
||||||
|
output;
|
||||||
|
stop;
|
||||||
|
run;
|
||||||
|
|
||||||
|
data work.jsdata;
|
||||||
|
length LIBREF $8 DSN $32 VAR_NM $32 VAR_DEPS $4000 VAR_TYPE $1
|
||||||
|
VAR_LOGIC $4000 VAR_FMT $32
|
||||||
|
_____DELETE__THIS__RECORD_____ $3;
|
||||||
|
LIBREF='DCTEST';
|
||||||
|
DSN='DC_DERIV';
|
||||||
|
VAR_NM='CALLER';
|
||||||
|
VAR_DEPS='';
|
||||||
|
VAR_TYPE='N';
|
||||||
|
VAR_LEN=8;
|
||||||
|
VAR_FMT='';
|
||||||
|
/* no semicolon in the payload: the single-expression rule is tested before
|
||||||
|
the function-name rule, so a quoted semicolon would abort on that first */
|
||||||
|
VAR_LOGIC='dosubl("data _null_")';
|
||||||
|
ACTIVE=1;
|
||||||
|
_____DELETE__THIS__RECORD_____='No';
|
||||||
|
run;
|
||||||
|
|
||||||
|
%mpe_execute(&_program,
|
||||||
|
viyacontext=&defaultcontext,
|
||||||
|
inputdatasets=work.sascontroltable work.jsdata,
|
||||||
|
outlib=web8,
|
||||||
|
outref=wb8,
|
||||||
|
mdebug=&sasjs_mdebug
|
||||||
|
)
|
||||||
|
|
||||||
|
/* the assertion compares the abort's MSG exactly, so the expected value is
|
||||||
|
assembled on one line: a literal wrapped across two source lines carries the
|
||||||
|
newline and the continuation indent with it */
|
||||||
|
%let exp8=MPE_DERIVED: CALLER: VAR_LOGIC may not call a function that;
|
||||||
|
%let exp8=&exp8 submits code or reaches the file system,;
|
||||||
|
%let exp8=&exp8 the environment or the macro symbol table;
|
||||||
|
|
||||||
|
%mpe_assert_abort(
|
||||||
|
desc=Checking a refused function in VAR_LOGIC is rejected,
|
||||||
|
outref=wb8,
|
||||||
|
msg=&exp8
|
||||||
|
)
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Test 9 - the fields the hook owns rather than the scan: ACTIVE is a picker
|
||||||
|
* in the EDIT screen (HARDSELECT over the MPE_SELECTBOX pair), so a submission
|
||||||
|
* that bypasses the grid cannot land a value the screen would never offer.
|
||||||
|
*/
|
||||||
|
data work.sascontroltable;
|
||||||
|
action='LOAD';
|
||||||
|
message='An ACTIVE value the screen cannot offer should abort';
|
||||||
|
libds="&dc_libref..MPE_DERIVED";
|
||||||
|
output;
|
||||||
|
stop;
|
||||||
|
run;
|
||||||
|
|
||||||
|
data work.jsdata;
|
||||||
|
length LIBREF $8 DSN $32 VAR_NM $32 VAR_DEPS $4000 VAR_TYPE $1
|
||||||
|
VAR_LOGIC $4000 VAR_FMT $32
|
||||||
|
_____DELETE__THIS__RECORD_____ $3;
|
||||||
|
LIBREF='DCTEST';
|
||||||
|
DSN='DC_DERIV';
|
||||||
|
VAR_NM='BADACT';
|
||||||
|
VAR_DEPS='';
|
||||||
|
VAR_TYPE='N';
|
||||||
|
VAR_LEN=8;
|
||||||
|
VAR_FMT='';
|
||||||
|
VAR_LOGIC='1';
|
||||||
|
ACTIVE=2;
|
||||||
|
_____DELETE__THIS__RECORD_____='No';
|
||||||
|
run;
|
||||||
|
|
||||||
|
%mpe_execute(&_program,
|
||||||
|
viyacontext=&defaultcontext,
|
||||||
|
inputdatasets=work.sascontroltable work.jsdata,
|
||||||
|
outlib=web9,
|
||||||
|
outref=wb9,
|
||||||
|
mdebug=&sasjs_mdebug
|
||||||
|
)
|
||||||
|
|
||||||
|
%mpe_assert_abort(
|
||||||
|
desc=Checking an ACTIVE value outside the picker pair is rejected,
|
||||||
|
outref=wb9,
|
||||||
|
msg=MPE_DERIVED: BADACT: ACTIVE must be 1 or 0
|
||||||
|
)
|
||||||
|
|
||||||
/* tidy up so repeated runs start clean */
|
/* tidy up so repeated runs start clean */
|
||||||
proc sql noprint;
|
proc sql noprint;
|
||||||
delete from &dc_libref..mpe_derived
|
delete from &dc_libref..mpe_derived
|
||||||
|
|||||||
@@ -34,7 +34,7 @@
|
|||||||
<h4> SAS Macros </h4>
|
<h4> SAS Macros </h4>
|
||||||
@li mf_nobs.sas
|
@li mf_nobs.sas
|
||||||
@li mp_abort.sas
|
@li mp_abort.sas
|
||||||
@li mpe_derived_check.sas
|
@li mp_derivedvarcheck.sas
|
||||||
|
|
||||||
<h4> Related Macros </h4>
|
<h4> Related Macros </h4>
|
||||||
@li mpe_derived.sas
|
@li mpe_derived.sas
|
||||||
@@ -100,14 +100,43 @@ run;
|
|||||||
)
|
)
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Field level checks - the same rules %mpe_derived applies at read time
|
* Field level checks. The rules for the derived variable itself - the name,
|
||||||
|
* the type, length, format, label and the expression - are core's, in
|
||||||
|
* mp_derivedvarcheck, so this hook and the read time engine (%mpe_derived ->
|
||||||
|
* mp_derivedvars) scan a row with the same rules and report the same reasons.
|
||||||
|
* What is added here are the two fields that belong to Data Controller rather
|
||||||
|
* than to the expression: the control table key, which the loader normalises
|
||||||
|
* but does not require, and ACTIVE, which the EDIT screen offers as a picker
|
||||||
|
* over the MPE_SELECTBOX pair - so a submission that bypasses the grid cannot
|
||||||
|
* land a value the screen would never offer.
|
||||||
*/
|
*/
|
||||||
%mpe_derived_check(inds=work.staging_ds, outds=work.derived_check)
|
data work.derived_dcfield(keep=name reason);
|
||||||
|
set work.staging_ds;
|
||||||
|
if upcase(_____DELETE__THIS__RECORD_____)='YES' then delete;
|
||||||
|
length name $32 reason $200;
|
||||||
|
call missing(reason);
|
||||||
|
name=upcase(strip(var_nm));
|
||||||
|
if missing(libref) or missing(dsn) then reason='LIBREF and DSN are required';
|
||||||
|
else if missing(active) or active not in (0,1)
|
||||||
|
then reason='ACTIVE must be 1 or 0';
|
||||||
|
if not missing(reason) then output;
|
||||||
|
run;
|
||||||
|
|
||||||
|
%mp_derivedvarcheck(inds=work.staging_ds
|
||||||
|
,outds=work.derived_field
|
||||||
|
,reserved=%str(_____DELETE__THIS__RECORD_____)
|
||||||
|
)
|
||||||
|
|
||||||
|
/* the control table fields are reported before the expression's own rules, so
|
||||||
|
which of the two passes found a row does not change the message it gets */
|
||||||
|
data work.derived_check;
|
||||||
|
set work.derived_dcfield work.derived_field;
|
||||||
|
run;
|
||||||
|
|
||||||
%if %mf_nobs(work.derived_check)>0 %then %do;
|
%if %mf_nobs(work.derived_check)>0 %then %do;
|
||||||
data _null_;
|
data _null_;
|
||||||
set work.derived_check(obs=1);
|
set work.derived_check(obs=1);
|
||||||
call symputx('der_errmsg',cats('MPE_DERIVED: ',var_nm,': ',reason));
|
call symputx('der_errmsg',cats('MPE_DERIVED: ',name,': ',reason));
|
||||||
call symputx('der_errflag',1);
|
call symputx('der_errflag',1);
|
||||||
run;
|
run;
|
||||||
%end;
|
%end;
|
||||||
|
|||||||
@@ -217,9 +217,11 @@ run;
|
|||||||
* caller asked to keep them (KEEPME in the input data). Keeping them is for
|
* caller asked to keep them (KEEPME in the input data). Keeping them is for
|
||||||
* test harnesses, which reseed before every run and cannot redeploy first. */
|
* test harnesses, which reseed before every run and cannot redeploy first. */
|
||||||
%if &keepme ne 1 %then %do;
|
%if &keepme ne 1 %then %do;
|
||||||
%ms_deletefile(&root/services/admin/configurator.sas)
|
%ms_deletefile(&root/services/admin/configurator.sas);
|
||||||
%ms_deletefile(&_program..sas)
|
%ms_deletefile(&_program..sas);
|
||||||
|
%end;
|
||||||
|
%else %do;
|
||||||
|
%put NOTE: &_program retained, as KEEPME was requested;
|
||||||
%end;
|
%end;
|
||||||
%else %put NOTE: &_program retained, as KEEPME was requested;
|
|
||||||
|
|
||||||
|
|
||||||
@@ -298,6 +298,8 @@ run;
|
|||||||
/* keep the program when KEEPME was requested, so a reseed does not need a
|
/* keep the program when KEEPME was requested, so a reseed does not need a
|
||||||
* redeploy first */
|
* redeploy first */
|
||||||
%if &keepme ne 1 %then %do;
|
%if &keepme ne 1 %then %do;
|
||||||
%mv_deletejes(path=&root/admin, name=makedata)
|
%mv_deletejes(path=&root/admin, name=makedata);
|
||||||
|
%end;
|
||||||
|
%else %do;
|
||||||
|
%put NOTE: &_program retained, as KEEPME was requested;
|
||||||
%end;
|
%end;
|
||||||
%else %put NOTE: &_program retained, as KEEPME was requested;
|
|
||||||
Reference in new issue
Block a user