Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 dc
|
180ca89e5a |
No files matched your search
@@ -1,11 +1,10 @@
|
||||
# Testing
|
||||
|
||||
Backend (SAS) tests are run with the sasjs CLI from the `sas/` directory. For a deploy-and-run on the shared state see the `dc-estate-testing` skill (`.agents/skills/dc-estate-testing/SKILL.md`), which covers the credentials, the `makedata` seed the suite needs and the estate's group name.
|
||||
Backend (SAS) tests are run with the sasjs CLI from the `sas/` directory.
|
||||
|
||||
## Commands
|
||||
|
||||
- Full deploy + test cycle: `npm run 4gl` (compiles, deploys to the 4gl target, runs makedata) then `sasjs test -t 4gl`.
|
||||
- The shared estate under its `server` target: `npm run server` (compile, deploy, seed) then `sasjs test -t server`.
|
||||
- Run tests only (no rebuild/redeploy): `sasjs test -t 4gl`.
|
||||
- Run a subset of tests: `sasjs test -t 4gl SOMESTRING` — the positional argument filters to matching tests (e.g. `sasjs test -t 4gl stagedata.test.3`).
|
||||
|
||||
|
||||
@@ -1,79 +0,0 @@
|
||||
---
|
||||
name: dc-estate-testing
|
||||
description: >
|
||||
Use when deploying Data Controller to a shared SASjs Server estate (sas.4gl.io) or running the SAS
|
||||
test suite against it. Covers the target and credential setup, the makedata seed the suite needs on
|
||||
a fresh appLoc and the group it must name, how to read the results, and how to deploy the frontend
|
||||
so the streamed app is the build you just made.
|
||||
---
|
||||
|
||||
# Deploying and testing on the shared SASjs Server estate
|
||||
|
||||
The whole cycle is two npm scripts plus `sasjs test`. Everything below explains what those scripts do and the steps that go wrong.
|
||||
|
||||
cd sas && npm ci # macro dependencies
|
||||
cd ../client && npm ci && npm run build # frontend only - see "Deploying the frontend"
|
||||
cd ../sas && npm run server # cpfavicon + sasjs cbd -t server + makedata
|
||||
npx sasjs test -t server
|
||||
|
||||
`npm run server` is `sasjs cbd -t server && npm run serverdata`, and the seed in the second half is not optional. `sasjs cbd` on its own deploys the services and the test programs but leaves the appLoc with no Data Controller configuration, and `testsetup` reads `<appLoc>/services/public/settings.sas` through the drive API. `makedata` is what creates that file, so without the seed `testsetup` gets a 404, aborts, and 59 of the 64 test programs fail to complete on setup alone - which reads like a broken deployment rather than a missing seed.
|
||||
|
||||
## The targets
|
||||
|
||||
`server` is the shared estate: `serverType: SASJS` on https://sas.4gl.io, `appLoc` `/Public/app/dc`, a real SAS 9 binary behind SASjs Server (`GET /SASjsApi/info` reports `mode: server`, `runTimes: ["sas"]`, `localLoginEnabled: true`). `4gl` is the same estate with a slightly different config, hence the older `npm run 4gl` / `4glmakedata` pair.
|
||||
|
||||
A run against `server` exercises the SASjs Server branch of every macro and the SAS 9 backend, and nothing else: no Viya branch, no SAS 9 STP branch. Say that when reporting, rather than implying the matrix is covered.
|
||||
|
||||
To deploy your own branch without replacing the app everyone else uses, add a target that copies `server` and changes `appLoc` and `streamServiceName` (for example `/Public/app/<you>/dc`), keep it out of the commit, and pass it with `-t`. `sasjs cbd` overwrites the whole appLoc it is pointed at.
|
||||
|
||||
## Credentials
|
||||
|
||||
The CLI logs in with a registered client, not a username and password: `.env.<target>` (gitignored, mode 600) needs `CLIENT`, `ACCESS_TOKEN` and `REFRESH_TOKEN`. With `CLIENT` alone it prints a `/SASjsLogon?client_id=...` URL and waits for an authorization code, which cannot be answered headless.
|
||||
|
||||
Mint the pair from a local login, with the password read from a file rather than argv:
|
||||
|
||||
1. `GET /` with a cookie jar - it sets `XSRF-TOKEN`, which every POST must echo in an `X-XSRF-TOKEN` header.
|
||||
2. `POST /SASLogon/login` with `{"username","password"}`.
|
||||
3. `GET /SASjsApi/client` on the same session - the registered client ids.
|
||||
4. `POST /SASLogon/authorize` with `{"clientId"}` -> `{"code"}`.
|
||||
5. `POST /SASjsApi/auth/token` with `{"clientId","code"}` -> `accessToken` / `refreshToken`.
|
||||
|
||||
An unmatched `/SASjsApi/...` path answers `200` with the Swagger page, so check the content type; HTML means the route does not exist. Delete the token file when the run is over.
|
||||
|
||||
## The makedata seed
|
||||
|
||||
`npm run serverdata` sends `deploy/makeDataServer.json`: one `fromjs` row with `ADMIN`, `DCPATH` and `OUTFMT`. Two things about it are worth knowing before you trust its result.
|
||||
|
||||
- **`ADMIN` must name a group the calling account is actually in.** The shipped value is `all-users`, the SASjs Server group that every logged-in user is a member of, which is what the shared estate needs. A name the account is not in - a Viya-style group such as `AllUsers`, or one estate's own group carried into another - still answers `SUCCESS` from the seed, and then every later `getdata`/`stagedata` call aborts with `<user> is not authorised to edit <lib>.<ds> in the <lib>.MPE_SECURITY table`. Check what the account belongs to with a call to `services/public/getgroups` before changing the value.
|
||||
- **Each run creates a new library.** The response carries `STATUS`, `DCPATH` and `LIBREF` (for example `DC621835`), and the LIBREF is clock-derived, so a libref from an earlier run answers `dataset <lib>.<table> does not exist` - take the current one from the response.
|
||||
- **`KEEPME` controls whether the seed can be repeated.** Without it `makedata` deletes itself and the configurator after a successful run (deliberately, so it cannot be executed twice over real data), and a second seed fails with `The Program at (<appLoc>/services/admin/makedata) does not exist` until you run `sasjs cbd` again. A harness that reseeds between runs sets `KEEPME` in the input row.
|
||||
- **Seed once per deploy, not before every test run.** The suite builds its own `dctest` library from the seeded control tables; reseeding in between is wasted work, and there is no need to run `makedata` and `sasjs test` in the opposite order.
|
||||
|
||||
## Reading the results
|
||||
|
||||
Results land in `sas/sasjsresults/`: `testResults.json` (the counts worth quoting: `tests_with_results`, `tests_that_pass`, `failed_to_complete`, `completed_with_failures`), `testResults.csv` (one row per assertion, lower-case headers `test_target`, `test_loc`, `sasjs_test_id`, `test_suite_result`, `test_description`, `test_url`), and one SAS log per program under `logs/`. The CLI's closing lines are not the source of truth when they disagree with the JSON.
|
||||
|
||||
A current green run is 64 of 64 test programs with 241 assertions passing and nothing failing to complete.
|
||||
|
||||
`ERROR` lines in the per-program logs are expected and are not a failed run: the specs that assert a refusal get `ERROR: Execution canceled by an %ABORT CANCEL statement`, and the ones that feed bad payloads get the rejection message they are asserting on (`Invalid libds:...`, `Invalid libref:...`). Judge the run by the JSON counts, then by whether the errors you find are the ones the tests asked for.
|
||||
|
||||
## Deploying the frontend
|
||||
|
||||
The streamed web app is a separate artefact and `sasjs cbd` only streams it if `client/dist` exists:
|
||||
|
||||
1. `client/libraries/sheet-crypto.tgz.gpg` is an encrypted dependency (`file:libraries/sheet-crypto.tgz` in `client/package.json`) that is not on the public registry. Decrypt it with the passphrase the maintainer holds: `printf '%s' "$SHEET_PWD" | gpg --batch --yes --passphrase-fd 0 --output ./libraries/sheet-crypto.tgz --decrypt ./libraries/sheet-crypto.tgz.gpg`.
|
||||
2. `client/.npmrc` is gitignored and needs `legacy-peer-deps=true` (the committed lock is written with it) plus the `@sheet` registry lines.
|
||||
3. `cd client && npm ci && npm run build`.
|
||||
4. The build carries the repo default appLoc, so point it at the deployment before deploying: rewrite `apploc="/Public/app/dc"` in `dist/index.html` to the target appLoc.
|
||||
5. `cd ../sas && npx sasjs cbd -t <target>` - with `client/dist` present the log says `Copying web app files` and `Web app is available at <url>`.
|
||||
|
||||
Without step 1 the deploy prints `ERROR An error has occurred when compiling web app services ... webSourcePath: .../client/dist doesn't exist` and then deploys the services anyway, so the deploy looks successful while the app streams nothing. The AppStream URL is session-protected: an anonymous `curl` gets a `401`, so verify it with a logged-in cookie jar and fetch the `main.<hash>.js` the page names.
|
||||
|
||||
Never commit `client/.npmrc` or the decrypted tarball; both are ignored, and `git status` staying clean is the check.
|
||||
|
||||
## Reading a failed run
|
||||
|
||||
- `ERROR An error occurred while executing the request. Job location: <appLoc>/tests/...` means the response was not JSON, so the program broke before it could write results. Open `sasjsresults/logs/<suite>.log` and read the first `ERROR`; a single compile break cascades into hundreds of `ERROR 180-322: Statement is not valid` lines and the tail is noise.
|
||||
- `WARNING: Apparent symbolic reference TESTLOC not resolved` in `testsetup.log`, next to a `404 Not Found` for `services/public/settings.sas`, is the missing seed described above, not a macro problem.
|
||||
- `sasjs test` runs the DEPLOYED program, so after editing a service, hook or test, run `sasjs cbd` again before testing or you will debug code that is no longer in the tree.
|
||||
- A deployment can succeed and still be missing its web app, its data or its configuration - check the thing you changed rather than the exit code.
|
||||
@@ -1,49 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# The Cypress specs CI runs, and the order they run in.
|
||||
#
|
||||
# ORDER MATTERS. These specs share one mock estate that is seeded once per job,
|
||||
# so several of them depend on the state they find:
|
||||
#
|
||||
# - csv-limited asserts the free-tier "Locked Feature" modal, which an earlier
|
||||
# spec can remove by planting a valid licence. It has always run first.
|
||||
# - hook-programs asserts that an edit raises an approval row against its
|
||||
# target table; it runs early, before other specs have added rows to the
|
||||
# approval tables.
|
||||
#
|
||||
# Specs added later are appended rather than inserted, so the sequence that was
|
||||
# known green keeps its shape. metanav is deliberately absent - see
|
||||
# https://git.datacontroller.io/dc/dc/issues/395.
|
||||
#
|
||||
# download is held out for a different reason - its download button is covered by
|
||||
# a fixed-position element, see
|
||||
# https://git.datacontroller.io/dc/dc/issues/398.
|
||||
set -euo pipefail
|
||||
|
||||
SPECS="csv-limited hook-programs liveness editor excel-multi-load excel csv filtering filter-panel licensing viewer-labels full-table-search viewbox stage browser-info ddl-export row-cell-limits derived-variables startupservice-launcher coltype-delete-record submitted-details embed-va"
|
||||
|
||||
# Specs that exist but are held out of CI, each with the reason. Listed here so
|
||||
# the coverage check below can tell a deliberate exclusion from a forgotten file.
|
||||
EXCLUDED="metanav download"
|
||||
|
||||
case "${1:-}" in
|
||||
all)
|
||||
# comma separated, the form `cypress run --spec` takes
|
||||
for name in $SPECS; do printf 'cypress/e2e/%s.cy.ts,' "$name"; done | sed 's/,$//'
|
||||
;;
|
||||
coverage)
|
||||
on_disk=$(cd "$2/client" && ls cypress/e2e/*.cy.ts | xargs -n1 basename | sed 's/\.cy\.ts$//' | sort)
|
||||
listed=$(printf '%s\n' $SPECS $EXCLUDED | sort)
|
||||
missing=$(comm -23 <(echo "$on_disk") <(echo "$listed"))
|
||||
if [ -n "$missing" ]; then
|
||||
echo "Specs that are neither run nor explicitly excluded:"
|
||||
echo "$missing"
|
||||
echo "Add them to SPECS (or EXCLUDED, with a reason) in .gitea/scripts/cypress-specs.sh"
|
||||
exit 1
|
||||
fi
|
||||
echo "All $(echo "$on_disk" | wc -l) specs are accounted for: $(echo $SPECS | wc -w) run, $(echo $EXCLUDED | wc -w) excluded."
|
||||
;;
|
||||
*)
|
||||
echo "usage: $0 {all|coverage <repo-root>}" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
@@ -75,33 +75,11 @@ jobs:
|
||||
cd client
|
||||
npm run test:headless
|
||||
|
||||
- name: Angular Tests in other timezones
|
||||
# The VA filter clauses are built from local calendar dates, so a
|
||||
# UTC-only run cannot see a date that slips a day west of UTC or east of
|
||||
# it. Only the VA specs carry that property, so both scripts run that
|
||||
# subset rather than the whole suite a second and third time.
|
||||
run: |
|
||||
cd client
|
||||
npm run test:tz-west
|
||||
npm run test:tz-east
|
||||
|
||||
- name: Every Cypress spec is in the CI list
|
||||
# The spec list used to be spelled out inline in --spec, and five files
|
||||
# quietly fell out of it - among them the VA embed suite. Fails when a
|
||||
# spec on disk is not listed.
|
||||
run: bash .gitea/scripts/cypress-specs.sh coverage .
|
||||
|
||||
- name: Production Build
|
||||
run: |
|
||||
cd client
|
||||
npm run build
|
||||
|
||||
|
||||
# Cypress shard 1 of 4. The setup repeats in every shard because each
|
||||
# one gets its own runner and its own SASjs server; that fixed cost is what
|
||||
# limits how far sharding helps. The specs it runs come from
|
||||
# .gitea/scripts/cypress-specs.sh.
|
||||
|
||||
Build-and-test-development:
|
||||
runs-on: ubuntu-latest
|
||||
needs: Build-and-ng-test
|
||||
@@ -164,7 +142,6 @@ jobs:
|
||||
echo NODE_PATH=node >> .env
|
||||
echo CORS=enable >> .env
|
||||
echo WHITELIST=http://localhost:4200 >> .env
|
||||
echo HTTP_PORT=5000 >> .env
|
||||
# The server refuses to run as root unless RUN_AS names an account
|
||||
# (a security change in the recent releases). This job runs as root
|
||||
# in a throwaway container, so opt in explicitly.
|
||||
@@ -204,20 +181,9 @@ jobs:
|
||||
sasjs request services/admin/makedata -t server-ci -d deploy/makedata.json -o ./makedata_out.json || true
|
||||
|
||||
- name: Prepare and run frontend and cypress
|
||||
# One job, one SASjs server, one ng serve, one Chrome: four of each on one
|
||||
# runner starved the app under test and every failure in that run was a
|
||||
# timeout, not an assertion.
|
||||
timeout-minutes: 50
|
||||
timeout-minutes: 35
|
||||
run: |
|
||||
cd ./client
|
||||
# The spec list lives in one place, and an empty list is a broken
|
||||
# workflow rather than a green run with no tests.
|
||||
SPECS="$(bash ../.gitea/scripts/cypress-specs.sh all)"
|
||||
if [ -z "$SPECS" ]; then
|
||||
echo "the spec list resolved to nothing"
|
||||
exit 1
|
||||
fi
|
||||
echo "specs: $SPECS"
|
||||
mv ./cypress.env.example.json ./cypress.env.json
|
||||
replace-in-files --regex='"username".*' --replacement='"username":"'${{ secrets.CYPRESS_USERNAME_SASJS }}'",' ./cypress.env.json
|
||||
replace-in-files --regex='"password".*' --replacement='"password":"'${{ secrets.CYPRESS_PWD_SASJS }}'" ' ./cypress.env.json
|
||||
@@ -230,24 +196,18 @@ jobs:
|
||||
replace-in-files --regex='"hosturl".*' --replacement='hosturl:"http://localhost:4200",' ./cypress.config.ts
|
||||
cat ./cypress.config.ts
|
||||
# Start frontend and run cypress
|
||||
# timeout 2700: SIGTERM after 45 min so Cypress can flush video/screenshots
|
||||
# timeout 1800: SIGTERM after 30 min so Cypress can flush video/screenshots
|
||||
# before the outer timeout-minutes hard-kills the step (avoids silent multi-hour hangs)
|
||||
npx ng serve --host 0.0.0.0 --port 4200 & npx wait-on http://localhost:4200 && timeout 2700 npx cypress run --browser chrome --spec "$SPECS"
|
||||
npx ng serve --host 0.0.0.0 --port 4200 & npx wait-on http://localhost:4200 && timeout 1800 npx cypress run --browser chrome --spec "cypress/e2e/csv-limited.cy.ts,cypress/e2e/hook-programs.cy.ts,cypress/e2e/liveness.cy.ts,cypress/e2e/editor.cy.ts,cypress/e2e/excel-multi-load.cy.ts,cypress/e2e/excel.cy.ts,cypress/e2e/csv.cy.ts,cypress/e2e/filtering.cy.ts,cypress/e2e/filter-panel.cy.ts,cypress/e2e/licensing.cy.ts,cypress/e2e/viewer-labels.cy.ts,cypress/e2e/full-table-search.cy.ts,cypress/e2e/viewbox.cy.ts,cypress/e2e/stage.cy.ts,cypress/e2e/browser-info.cy.ts,cypress/e2e/ddl-export.cy.ts,cypress/e2e/row-cell-limits.cy.ts,cypress/e2e/derived-variables.cy.ts,cypress/e2e/startupservice-launcher.cy.ts"
|
||||
|
||||
- name: Zip Cypress videos
|
||||
# Only on a failure: a green run has nothing to watch, and zipping and
|
||||
# uploading every run's videos is the part of the video cost that lands
|
||||
# on the clock.
|
||||
if: failure()
|
||||
if: always()
|
||||
run: |
|
||||
mkdir -p ./client/cypress/videos
|
||||
zip -r cypress-videos ./client/cypress/videos
|
||||
|
||||
- name: Add cypress videos artifacts
|
||||
if: failure()
|
||||
# v3, not v4: this runner reports "GHESNotSupportedError:
|
||||
# @actions/artifact v2.0.0+, upload-artifact@v4+ ... not supported on
|
||||
# GHES", so the current major cannot upload here at all.
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
name: cypress-videos.zip
|
||||
|
||||
@@ -1,121 +0,0 @@
|
||||
name: Publish code documentation
|
||||
run-name: Publishing typedoc and SAS docs
|
||||
|
||||
# The code documentation sites are published here rather than inline in the
|
||||
# release job, so that they can be republished on their own - after a secret
|
||||
# rotation, or for a release whose artefact upload failed (the release job
|
||||
# cannot simply be re-run: semantic-release refuses once the tag exists).
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
NODE_VERSION: '24.5.0'
|
||||
|
||||
jobs:
|
||||
typedoc:
|
||||
runs-on: ubuntu-latest
|
||||
# One deploy per site at a time: `surfer deploy` swaps the site atomically,
|
||||
# so two runs racing is not corruption, but it is a wasted build and the
|
||||
# last swap wins.
|
||||
concurrency:
|
||||
group: surfer-webdoc
|
||||
cancel-in-progress: false
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: ${{ env.NODE_VERSION }}
|
||||
|
||||
- name: Write .npmrc file
|
||||
run: |
|
||||
printf '%s\n' "$NPMRC" >> client/.npmrc
|
||||
echo "legacy-peer-deps=true" >> client/.npmrc
|
||||
shell: bash
|
||||
env:
|
||||
NPMRC: ${{ secrets.NPMRC }}
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
cd client
|
||||
# Decrypt and Install sheet
|
||||
echo "$SHEET_PWD" | gpg --batch --yes --passphrase-fd 0 ./libraries/sheet-crypto.tgz.gpg
|
||||
npm ci
|
||||
shell: bash
|
||||
env:
|
||||
SHEET_PWD: ${{ secrets.SHEET_PWD }}
|
||||
|
||||
- name: Build typedoc
|
||||
run: |
|
||||
cd client
|
||||
npm run compodoc:build
|
||||
|
||||
# Deploy tooling is pinned to an exact version: an unpinned `latest` would
|
||||
# let a breaking or compromised upstream release publish straight to the
|
||||
# live site. `deploy` (v7) replaces the site atomically, and it takes the
|
||||
# Cloudron username plus an app password - the v6 `put --token` flow this
|
||||
# replaces is gone.
|
||||
#
|
||||
# webdoc is a NAMED site on the app whose default site is the marketing
|
||||
# site, so the name and its alias domain are both passed. Surfer picks
|
||||
# the target only from `--site-name` (it travels as the Surfer-Deployment
|
||||
# header); without it `deploy` replaces the app's DEFAULT site, which is
|
||||
# what serves datacontroller.io - so an unnamed deploy published the
|
||||
# Typedoc over the top of the marketing site instead of to webdoc.
|
||||
- name: Deploy typedoc
|
||||
env:
|
||||
SURFER_PASSWORD: ${{ secrets.WEBDOC_DATACONTROLLER_IO }}
|
||||
REF_NAME: ${{ gitea.ref_name }}
|
||||
COMMIT_SHA: ${{ gitea.sha }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
npx --yes @cloudron/surfer@7.2.0 deploy client/documentation \
|
||||
--server webdoc.datacontroller.io \
|
||||
--site-name webdoc \
|
||||
--domain webdoc.datacontroller.io \
|
||||
--username allan \
|
||||
--password "$SURFER_PASSWORD" \
|
||||
--message "Typedoc from ${REF_NAME} @ ${COMMIT_SHA:0:8}"
|
||||
|
||||
sas-docs:
|
||||
runs-on: ubuntu-latest
|
||||
concurrency:
|
||||
group: surfer-code-datacontroller
|
||||
cancel-in-progress: false
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: ${{ env.NODE_VERSION }}
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
# `sasjs doc` shells out to doxygen
|
||||
apt-get update
|
||||
apt-get install -y doxygen
|
||||
cd sas
|
||||
npm ci
|
||||
CLI_VERSION=$(node -p "require('./package.json').dependencies['@sasjs/cli']")
|
||||
npm i -g "@sasjs/cli@${CLI_VERSION}"
|
||||
|
||||
- name: Build SAS docs
|
||||
run: |
|
||||
cd sas
|
||||
sasjs doc
|
||||
|
||||
# No `--site-name` here on purpose: code.datacontroller.io is its own app
|
||||
# and the SAS docs are its DEFAULT site, so naming one would create a
|
||||
# separate site and leave the served copy stale. Do not copy the named
|
||||
# form from the typedoc job above - that app's default site is the
|
||||
# marketing site, which is why it must not be the target.
|
||||
- name: Deploy SAS docs
|
||||
env:
|
||||
SURFER_PASSWORD: ${{ secrets.CODE_DATACONTROLLER_IO }}
|
||||
REF_NAME: ${{ gitea.ref_name }}
|
||||
COMMIT_SHA: ${{ gitea.sha }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
npx --yes @cloudron/surfer@7.2.0 deploy sas/sasjsbuild/sasdocs \
|
||||
--server code.datacontroller.io \
|
||||
--username allan \
|
||||
--password "$SURFER_PASSWORD" \
|
||||
--message "SAS docs from ${REF_NAME} @ ${COMMIT_SHA:0:8}"
|
||||
@@ -33,7 +33,6 @@ jobs:
|
||||
echo NODE_PATH=node >> .env
|
||||
echo CORS=enable >> .env
|
||||
echo WHITELIST=http://localhost:4200 >> .env
|
||||
echo HTTP_PORT=5000 >> .env
|
||||
# The server refuses to run as root unless RUN_AS names an account
|
||||
# (a security change in the recent releases). This job runs as root
|
||||
# in a throwaway container, so opt in explicitly.
|
||||
|
||||
@@ -1,209 +0,0 @@
|
||||
name: Release assets
|
||||
run-name: Building and uploading the release assets
|
||||
|
||||
# Builds the downloadable artefacts for an existing release and uploads them.
|
||||
#
|
||||
# The release job cuts the tag before it builds, so a failure anywhere between
|
||||
# the semantic-release step and the asset upload leaves a published release with
|
||||
# no files - and that job cannot simply be re-run, because semantic-release
|
||||
# refuses once the tag exists. This workflow rebuilds the artefacts from the tag
|
||||
# and attaches them, so the release can be completed without cutting a new one.
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
tag:
|
||||
description: 'Release tag to build and upload (blank = the latest release)'
|
||||
required: false
|
||||
default: ''
|
||||
|
||||
env:
|
||||
NODE_VERSION: '24.5.0'
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
|
||||
jobs:
|
||||
assets:
|
||||
runs-on: ubuntu-latest
|
||||
# Two runs uploading the same asset names would race the delete-then-upload
|
||||
# below.
|
||||
concurrency:
|
||||
group: release-assets
|
||||
cancel-in-progress: false
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: ${{ env.NODE_VERSION }}
|
||||
|
||||
- name: Write .npmrc file
|
||||
run: |
|
||||
printf '%s\n' "$NPMRC" >> client/.npmrc
|
||||
echo "legacy-peer-deps=true" >> client/.npmrc
|
||||
shell: bash
|
||||
env:
|
||||
NPMRC: ${{ secrets.NPMRC }}
|
||||
|
||||
- name: Install system packages
|
||||
# jq parses the release JSON below, so it is installed before the tag
|
||||
# resolve.
|
||||
run: |
|
||||
apt-get update
|
||||
apt-get install -y zip jq
|
||||
|
||||
- name: Resolve the release tag
|
||||
env:
|
||||
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
||||
REQUESTED_TAG: ${{ inputs.tag }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
AUTH_HEADER="Authorization: token $GITEA_TOKEN"
|
||||
BASE="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"
|
||||
|
||||
if [ -n "${REQUESTED_TAG}" ]; then
|
||||
TAG="${REQUESTED_TAG}"
|
||||
# Fail here rather than after a ten-minute build if the tag is not
|
||||
# cut yet or has no release.
|
||||
curl --fail-with-body -sS -H "$AUTH_HEADER" "$BASE/releases/tags/${TAG}" > /dev/null
|
||||
else
|
||||
TAG=$(curl --fail-with-body -sS -H "$AUTH_HEADER" "$BASE/releases/latest" | jq -r '.tag_name')
|
||||
fi
|
||||
echo "Building assets for ${TAG}"
|
||||
echo "TAG=${TAG}" >> "$GITHUB_ENV"
|
||||
|
||||
- name: Check out the tag
|
||||
run: |
|
||||
set -euo pipefail
|
||||
git fetch --force --tags origin "+refs/tags/${TAG}:refs/tags/${TAG}"
|
||||
git checkout --force "refs/tags/${TAG}"
|
||||
|
||||
- name: Install the SASjs CLI the tag pins
|
||||
run: |
|
||||
cd sas
|
||||
CLI_VERSION=$(node -p "require('./package.json').dependencies['@sasjs/cli']")
|
||||
npm i -g "@sasjs/cli@${CLI_VERSION}"
|
||||
|
||||
- name: Frontend build
|
||||
run: |
|
||||
cd client
|
||||
# Decrypt and Install sheet
|
||||
echo "$SHEET_PWD" | gpg --batch --yes --passphrase-fd 0 ./libraries/sheet-crypto.tgz.gpg
|
||||
npm ci
|
||||
npm i webpack
|
||||
npm run build
|
||||
shell: bash
|
||||
env:
|
||||
SHEET_PWD: ${{ secrets.SHEET_PWD }}
|
||||
|
||||
- name: Build SAS9 EBI Release
|
||||
run: |
|
||||
cd sas
|
||||
npm i
|
||||
sasjs c -t sas9
|
||||
rm -rf sasjsbuild/tests
|
||||
sasjs b -t sas9
|
||||
cp sasjsbuild/mysas9deploy.sas ./demostream_sas9.sas
|
||||
# remove streamed component and rebuild SAS 9 services
|
||||
rm -rf sasjsbuild/services/web9
|
||||
rm sasjsbuild/services/clickme.sas
|
||||
sasjs b -t sas9
|
||||
cp sasjsbuild/mysas9deploy.sas ./sas9.sas
|
||||
|
||||
- name: Build SASjs Server Release
|
||||
run: |
|
||||
cd sas
|
||||
cp sasjs/utils/favicon.ico ../client/dist/favicon.ico
|
||||
sasjs c -t server
|
||||
rm -rf sasjsbuild/tests
|
||||
server_apploc="/Public/app/dc"
|
||||
sed -i "s|apploc=\"[^\"]*\"|apploc=\"${server_apploc}\"|g" sasjsbuild/services/web/index.html
|
||||
sasjs b -t server
|
||||
cp sasjsbuild/server.json.zip ./sasjs_server.json.zip
|
||||
|
||||
- name: Build Viya Release
|
||||
run: |
|
||||
cd sas
|
||||
sasjs c -t viya
|
||||
rm -rf sasjsbuild/tests
|
||||
sed -i -e 's/servertype="SASJS"/servertype="SASVIYA"/g' sasjsbuild/services/DC.html
|
||||
sasjs b -t viya
|
||||
cp sasjsbuild/viya.sas ./viya.sas
|
||||
# compile Viya Full deploy (without web)
|
||||
rm -rf sasjsbuild/services/web
|
||||
rm sasjsbuild/services/DC.html
|
||||
sasjs b -t viya
|
||||
cp sasjsbuild/viya.sas ./viya_noweb.sas
|
||||
cp sasjsbuild/viya.json ./viya_noweb.json
|
||||
|
||||
- name: Zip Frontend (including viya.json for full viya deploy)
|
||||
run: |
|
||||
cd sas
|
||||
cp sasjsbuild/viya.json ../client/dist/viya.json
|
||||
# Zip from *inside* dist so the archive holds the frontend files at its
|
||||
# root rather than under a client/dist/ prefix (see #147).
|
||||
cd ../client/dist
|
||||
zip -r ../../frontend.zip .
|
||||
|
||||
- name: Upload assets to the release
|
||||
env:
|
||||
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
AUTH_HEADER="Authorization: token $GITEA_TOKEN"
|
||||
BASE="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"
|
||||
|
||||
RELEASE_JSON=$(curl --fail-with-body -sS -H "$AUTH_HEADER" "$BASE/releases/tags/${TAG}")
|
||||
RELEASE_ID=$(echo "$RELEASE_JSON" | jq -r '.id')
|
||||
RELEASE_BODY=$(echo "$RELEASE_JSON" | jq -r '.body')
|
||||
|
||||
ASSETS=(frontend.zip
|
||||
sas/demostream_sas9.sas
|
||||
sas/viya.sas
|
||||
sas/sasjs_server.json.zip
|
||||
sas/sas9.sas
|
||||
sas/viya_noweb.sas
|
||||
sas/viya_noweb.json)
|
||||
|
||||
: > SHA256SUMS
|
||||
for f in "${ASSETS[@]}"; do
|
||||
sum=$(sha256sum "$f" | cut -d' ' -f1)
|
||||
printf '%s %s\n' "$sum" "${f##*/}" >> SHA256SUMS
|
||||
done
|
||||
cat SHA256SUMS
|
||||
|
||||
# Replacing the assets means removing the ones this workflow uploads,
|
||||
# and a second run must not append the verification section twice. An
|
||||
# asset the release carries from another source is left alone.
|
||||
OWNED=$(printf '%s\n' "${ASSETS[@]##*/}" SHA256SUMS)
|
||||
echo "$RELEASE_JSON" | jq -r '.assets[]? | "\(.id) \(.name)"' |
|
||||
while read -r asset_id asset_name; do
|
||||
if printf '%s\n' "$OWNED" | grep -qxF -- "$asset_name"; then
|
||||
echo "Removing previous ${asset_name}"
|
||||
curl --fail-with-body -sS -X DELETE -H "$AUTH_HEADER" "$BASE/releases/${RELEASE_ID}/assets/${asset_id}"
|
||||
else
|
||||
echo "Leaving ${asset_name} in place: this workflow did not upload it"
|
||||
fi
|
||||
done
|
||||
|
||||
if echo "$RELEASE_BODY" | grep -q "## Verifying the download"; then
|
||||
echo "Release body already carries the verification section"
|
||||
else
|
||||
VERIFY_SECTION=$(./.gitea/scripts/verify-section.sh)
|
||||
NEW_BODY=$(jq -nr --arg body "$RELEASE_BODY" --arg verify "$VERIFY_SECTION" \
|
||||
'$body + "\n" + $verify + "\n\nFor installation instructions, please visit https://docs.datacontroller.io/"')
|
||||
curl --fail-with-body -sS -X PATCH \
|
||||
-H "$AUTH_HEADER" \
|
||||
-H 'Content-Type: application/json' \
|
||||
--data "$(jq -n --arg body "$NEW_BODY" '{draft:false, body:$body}')" \
|
||||
"$BASE/releases/$RELEASE_ID"
|
||||
fi
|
||||
|
||||
URL="$BASE/releases/${RELEASE_ID}/assets"
|
||||
for f in "${ASSETS[@]}"; do
|
||||
echo "Uploading $f ..."
|
||||
curl --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@$f"
|
||||
done
|
||||
echo "Uploading SHA256SUMS ..."
|
||||
curl --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@SHA256SUMS"
|
||||
@@ -107,16 +107,6 @@ jobs:
|
||||
echo ${{ secrets.SHEET_PWD }} | gpg --batch --yes --passphrase-fd 0 ./libraries/sheet-crypto.tgz.gpg
|
||||
npm ci
|
||||
|
||||
- name: Build frontend for web streaming
|
||||
# The server-ci target streams client/dist as the 'clickme' web app, so
|
||||
# the production build has to exist before the mocked services are
|
||||
# deployed. Without it `sasjs cbd` reports
|
||||
# "webSourcePath: .../client/dist ... doesn't exist", streams no
|
||||
# frontend and the deploy step fails.
|
||||
run: |
|
||||
cd client
|
||||
npm run build
|
||||
|
||||
- name: Setup and start SASjs server
|
||||
run: |
|
||||
npm i -g pm2
|
||||
@@ -127,7 +117,6 @@ jobs:
|
||||
echo NODE_PATH=node >> .env
|
||||
echo CORS=enable >> .env
|
||||
echo WHITELIST=http://localhost:4200 >> .env
|
||||
echo HTTP_PORT=5000 >> .env
|
||||
# The server refuses to run as root unless RUN_AS names an account
|
||||
# (a security change in the recent releases). This job runs as root
|
||||
# in a throwaway container, so opt in explicitly.
|
||||
@@ -204,15 +193,11 @@ jobs:
|
||||
# and its permissions are derived from this block.
|
||||
# contents: write -> push CHANGELOG.md / package.json commit + tag, and
|
||||
# create the Release via the Gitea API.
|
||||
# actions: write -> dispatch the Sites workflow for the new tag (that
|
||||
# route is scoped to the actions unit, which contents
|
||||
# does not grant).
|
||||
# The TSDOC_TOKEN and CODE_DATACONTROLLER_IO secrets continue to be used
|
||||
# because they target *different* servers (webdoc.datacontroller.io and
|
||||
# code.datacontroller.io), not the Gitea API.
|
||||
permissions:
|
||||
contents: write
|
||||
actions: write
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
@@ -281,11 +266,6 @@ jobs:
|
||||
|
||||
semantic-release
|
||||
|
||||
# The tag this job cuts, for the steps that publish against it: a
|
||||
# `releases/latest` lookup would pick whichever release landed last
|
||||
# if two overlap.
|
||||
echo "RELEASE_TAG=v$(node -p "require('./package.json').version")" >> "$GITHUB_ENV"
|
||||
|
||||
- name: Frontend Build
|
||||
description: Must be created AFTER the release as the version (git tag) is used in the interface
|
||||
run: |
|
||||
@@ -346,6 +326,19 @@ jobs:
|
||||
cd ../client/dist
|
||||
zip -r ../../frontend.zip .
|
||||
|
||||
- name: Release Typedoc
|
||||
run: |
|
||||
cd client
|
||||
npm -g install cloudron-surfer
|
||||
npm run compodoc:build
|
||||
surfer put --token ${{ secrets.TSDOC_TOKEN }} --server webdoc.datacontroller.io documentation/* /
|
||||
|
||||
- name: Release code.datacontroller.io
|
||||
run: |
|
||||
cd sas
|
||||
sasjs doc
|
||||
surfer put --token ${{ secrets.CODE_DATACONTROLLER_IO }} --server code.datacontroller.io sasjsbuild/sasdocs/* /
|
||||
|
||||
- name: Upload assets to release
|
||||
env:
|
||||
# Use the same ephemeral per-job Gitea Actions token that the
|
||||
@@ -360,7 +353,7 @@ jobs:
|
||||
AUTH_HEADER="Authorization: token ${GITEA_TOKEN}"
|
||||
BASE="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"
|
||||
|
||||
RELEASE_JSON=$(curl --fail-with-body -sS -H "$AUTH_HEADER" "$BASE/releases/tags/${RELEASE_TAG}")
|
||||
RELEASE_JSON=$(curl -k --fail-with-body -sS -H "$AUTH_HEADER" "$BASE/releases/latest")
|
||||
RELEASE_ID=$(echo "$RELEASE_JSON" | jq -r '.id')
|
||||
RELEASE_BODY=$(echo "$RELEASE_JSON" | jq -r '.body')
|
||||
|
||||
@@ -394,7 +387,7 @@ jobs:
|
||||
VERIFY_SECTION=$(./.gitea/scripts/verify-section.sh)
|
||||
NEW_BODY=$(jq -nr --arg body "$RELEASE_BODY" --arg verify "$VERIFY_SECTION" \
|
||||
'$body + "\n" + $verify + "\n\nFor installation instructions, please visit https://docs.datacontroller.io/"')
|
||||
curl --fail-with-body -sS -X PATCH \
|
||||
curl -k --fail-with-body -sS -X PATCH \
|
||||
-H "$AUTH_HEADER" \
|
||||
-H 'Content-Type: application/json' \
|
||||
--data "$(jq -n --arg body "$NEW_BODY" '{draft:false, body:$body}')" \
|
||||
@@ -404,58 +397,7 @@ jobs:
|
||||
URL="$BASE/releases/$RELEASE_ID/assets"
|
||||
for f in "${ASSETS[@]}"; do
|
||||
echo "Uploading $f ..."
|
||||
curl --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@$f"
|
||||
curl -k --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@$f"
|
||||
done
|
||||
echo "Uploading SHA256SUMS ..."
|
||||
curl --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@SHA256SUMS"
|
||||
|
||||
- name: Publish the code documentation
|
||||
# Typedoc (webdoc) and the SAS docs (code) are published by docs.yaml,
|
||||
# not inline here. Two reasons: the v6 `surfer put --token` flow is gone
|
||||
# in Surfer 7, which takes a Cloudron username plus an app password; and
|
||||
# a dispatch means a stale or failed docs deploy can be republished
|
||||
# without re-running this job, which semantic-release refuses once the
|
||||
# tag exists.
|
||||
#
|
||||
# This runs AFTER the release assets are uploaded: the documentation is
|
||||
# a side channel, and a docs problem must not cost a cut release its
|
||||
# artefacts.
|
||||
env:
|
||||
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
AUTH_HEADER="Authorization: token $GITEA_TOKEN"
|
||||
BASE="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"
|
||||
|
||||
TAG="${RELEASE_TAG}"
|
||||
echo "Dispatching code documentation for ${TAG}"
|
||||
|
||||
curl --fail-with-body -sS -X POST \
|
||||
-H "$AUTH_HEADER" \
|
||||
-H 'Content-Type: application/json' \
|
||||
--data "$(jq -n --arg ref "refs/tags/$TAG" '{ref: $ref}')" \
|
||||
"$BASE/actions/workflows/docs.yaml/dispatches"
|
||||
|
||||
- name: Publish the demo site for this release
|
||||
# The tag is created by this job's semantic-release step, so it carries
|
||||
# the Actions user as its author - and the forge suppresses ref events
|
||||
# from that user, which means `on: push: tags` in sites.yaml never fires.
|
||||
# Dispatch it instead: the API call is authenticated, and the run it
|
||||
# starts checks out the tag the same way.
|
||||
env:
|
||||
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
AUTH_HEADER="Authorization: token ${GITEA_TOKEN}"
|
||||
BASE="${{ gitea.server_url }}/api/v1/repos/${{ gitea.repository }}"
|
||||
|
||||
TAG="${RELEASE_TAG}"
|
||||
echo "Dispatching Sites for ${TAG}"
|
||||
|
||||
# the dispatch endpoint resolves a bare name as a branch; a tag needs
|
||||
# the refs/tags/ prefix
|
||||
curl --fail-with-body -sS -X POST \
|
||||
-H "$AUTH_HEADER" \
|
||||
-H 'Content-Type: application/json' \
|
||||
--data "$(jq -n --arg ref "refs/tags/$TAG" '{ref: $ref}')" \
|
||||
"$BASE/actions/workflows/sites.yaml/dispatches"
|
||||
curl -k --fail-with-body -sS -H "$AUTH_HEADER" "$URL" -F "attachment=@SHA256SUMS"
|
||||
@@ -75,7 +75,6 @@ jobs:
|
||||
touch .env
|
||||
echo RUN_TIMES=js >> .env
|
||||
echo NODE_PATH=node >> .env
|
||||
echo HTTP_PORT=5000 >> .env
|
||||
# The server refuses to run as root unless RUN_AS names an account
|
||||
echo RUN_AS=root >> .env
|
||||
nohup ./api-linux > server.log 2>&1 &
|
||||
@@ -114,39 +113,6 @@ jobs:
|
||||
|
||||
- name: Build the browser-hosted site
|
||||
run: |
|
||||
# The published sites ship a working licence so they are usable beyond
|
||||
# the free tier. The pair is committed in sas/mocks/mock-licence.json
|
||||
# rather than minted here, so every deploy of both sites carries the
|
||||
# same key; the CI estate stays unlicensed because that file is outside
|
||||
# the deployed sync folder, so the specs still exercise the free tier.
|
||||
# The pair must round trip through the same WebCrypto the app uses,
|
||||
# because a committed pair can drift (one half regenerated, a value
|
||||
# truncated on copy) and nothing else would notice before publishing.
|
||||
# Identical values in the two fields is the HTTP-format marker the app
|
||||
# refuses in a secure context, so refuse to publish one.
|
||||
node -e "
|
||||
const d = require('./sas/mocks/mock-licence.json')
|
||||
const b = (s) => Buffer.from(String(s || ''), 'base64')
|
||||
if (!d.licenceKey || !d.activationKey || d.licenceKey === d.activationKey) {
|
||||
console.error('sas/mocks/mock-licence.json must hold an HTTPS-format key pair')
|
||||
process.exit(1)
|
||||
}
|
||||
crypto.subtle
|
||||
.importKey('pkcs8', b(d.activationKey), { name: 'RSA-OAEP', hash: 'SHA-256' }, false, ['decrypt'])
|
||||
.then((key) => crypto.subtle.decrypt({ name: 'RSA-OAEP', hash: 'SHA-256' }, key, b(d.licenceKey)))
|
||||
.then((plain) => {
|
||||
const data = JSON.parse(new TextDecoder().decode(plain))
|
||||
console.log('licence pair round trips: site', data.site_id, 'until', data.valid_until)
|
||||
})
|
||||
.catch((e) => {
|
||||
console.error('sas/mocks/mock-licence.json does not decrypt:', e.message || e)
|
||||
process.exit(1)
|
||||
})
|
||||
"
|
||||
LICENCE_DIR="sasjs_root/drive/files$APP_LOC/mock-storage"
|
||||
mkdir -p "$LICENCE_DIR"
|
||||
cp sas/mocks/mock-licence.json "$LICENCE_DIR/licence.json"
|
||||
|
||||
# DC's data files exist only after makedata has run, so the drive is
|
||||
# captured first and shipped as the site's starting drive. services/web9
|
||||
# is the streamed frontend, which is already the site itself.
|
||||
|
||||
@@ -1,5 +0,0 @@
|
||||
# sas/mocks/mock-licence.json holds a licence key pair for the mock estate: the
|
||||
# activation key is RSA key material the browser demo decrypts with, not a
|
||||
# credential for any service.
|
||||
sas/mocks/mock-licence.json:generic-api-key:2
|
||||
sas/mocks/mock-licence.json:generic-api-key:3
|
||||
@@ -32,8 +32,6 @@ Never consider a change complete until the relevant linters pass on the files yo
|
||||
|
||||
- **Client (TypeScript/HTML/etc.)**: run `npm run lint:check` from the `client/` directory (prettier). Fix any failures with `npm run lint:fix`.
|
||||
- **SAS**: after creating or modifying any `.sas` files, run `sasjs lint` from the `sas/` directory and ensure the files you touched have no lint warnings (the repo currently has pre-existing warnings in other files, which can be ignored).
|
||||
`sas/.sasjslint` carries `ignoredMacros: ["webout", "mpeinit", "mpeterm"]`. `webout` is the linter's own default - `sasjs compile` injects `%webout` into every service - and an explicit list replaces that default, so it has to be restated there. `mpeinit` and `mpeterm` are listed because every service is compiled with a serviceInit program (`sasjs/utils/serviceinitsas9|viya|server.sas`) that declares both macros in its own header, so a service body may call them without declaring them. Add a macro to that list only when it reaches a file the same way - injected by the compiler, through serviceInit, testSetUp or another implicit provider - never to silence a genuinely missing declaration.
|
||||
`noBannedStrings` flags the literal words `error ` and `warning ` anywhere in a file, comments included. Reword prose; where the text has to stay (a real `%put`, a doxygen `@warning` tag), keep the log output identical with `%put %str(WARN)ING:` rather than a per-file override, and use `@sasjslint {"noBannedStrings": false}` only when the word itself is the content.
|
||||
|
||||
## No external assets
|
||||
|
||||
@@ -43,7 +41,7 @@ Data Controller must run entirely locally (offline / on-prem, no internet access
|
||||
|
||||
Agent-related content lives in `.agents/`: technical/agent-facing documentation goes in `.agents/docs/` (not `docs/`), and skills in `.agents/skills/`. When writing explanatory or technical docs about the codebase, put them in `.agents/docs/`.
|
||||
|
||||
Skills installed via `npx skills add` are tracked in `skills-lock.json` and live under `.agents/skills/`. Repo-local custom skills (e.g. `dc-sas` for Data Controller SAS development, `dc-estate-testing` for estate deploys and test runs, `handsontable`, `hyperformula`) are kept here too. Avoid editing npx-managed skill files directly — use `npx skills update` instead.
|
||||
Skills installed via `npx skills add` are tracked in `skills-lock.json` and live under `.agents/skills/`. Repo-local custom skills (e.g. `dc-sas` for Data Controller SAS development, `handsontable`, `hyperformula`) are kept here too. Avoid editing npx-managed skill files directly — use `npx skills update` instead.
|
||||
|
||||
## Code comments and test names
|
||||
|
||||
|
||||
-164
@@ -1,167 +1,3 @@
|
||||
# [7.21.0](https://git.datacontroller.io/dc/dc/compare/v7.20.0...v7.21.0) (2026-10-29)
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **client:** parse word-date values from VA parameters ([405b28d](https://git.datacontroller.io/dc/dc/commit/405b28dcccfe5994521bf63f0270d4877f544294))
|
||||
* **client:** parse word-date values from VA parameters ([ce5b1aa](https://git.datacontroller.io/dc/dc/commit/ce5b1aa96175d0fd1f048fea6d127cbc743804f2))
|
||||
* **makedata:** name the sasjs server group in the estate seed ([eeffa92](https://git.datacontroller.io/dc/dc/commit/eeffa92f9e973b323171bbefe2ef1cbc27e3aee7))
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* **derived:** apply the core field scan in the MPE_DERIVED hook ([a4a0f9c](https://git.datacontroller.io/dc/dc/commit/a4a0f9cb2f7c0b6252405b71df64aae912d1b075))
|
||||
|
||||
# [7.20.0](https://git.datacontroller.io/dc/dc/compare/v7.19.0...v7.20.0) (2026-10-28)
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **ci:** install doxygen for the SAS docs job ([f58925b](https://git.datacontroller.io/dc/dc/commit/f58925b29956f1f753f0a367fdcd0afe7f3f51a0))
|
||||
* **ci:** keep foreign release assets, install jq first, pin the CLI ([27686d3](https://git.datacontroller.io/dc/dc/commit/27686d3aad5e34ace59ca1b735a17cca1a4025e1))
|
||||
* **ci:** publish against the tag this job cuts, not releases/latest ([8888734](https://git.datacontroller.io/dc/dc/commit/888873421c3aeaf312d5ec45e3c805317c66976a))
|
||||
* **ci:** publish docs after the assets, with the webdoc app password ([dd482c4](https://git.datacontroller.io/dc/dc/commit/dd482c4069202a0c6b9983129c19efc09bd160a7))
|
||||
* **ci:** publish the code documentation with surfer 7 and app passwords ([25ffa71](https://git.datacontroller.io/dc/dc/commit/25ffa71adddba5e334f4a8c166f284d1be498fcd))
|
||||
* **ci:** publish the typedoc to the webdoc site, not the app default ([c5808a1](https://git.datacontroller.io/dc/dc/commit/c5808a16eba0d29562fcc76f13ac4a33ab8da0b7))
|
||||
* **ci:** rebuild and upload release assets on demand ([6486c80](https://git.datacontroller.io/dc/dc/commit/6486c80ed4f368bbf116c0ecdecc0e5e5f02075f))
|
||||
* **ci:** use the lockfile and the pinned CLI in the docs job ([af1813f](https://git.datacontroller.io/dc/dc/commit/af1813f829861df41c89b512c824e8d3ebac7a4b))
|
||||
* **ci:** verify the forge certificate on the release API calls ([cf7b21a](https://git.datacontroller.io/dc/dc/commit/cf7b21a8b61ec3bf7ebe21968ccf13ade264bc64))
|
||||
* **cypress:** hold download out too, and retry once in CI ([c97e886](https://git.datacontroller.io/dc/dc/commit/c97e8866db9ee404b2118d24baf53fc7058debef)), closes [#395](https://git.datacontroller.io/dc/dc/issues/395) [#398](https://git.datacontroller.io/dc/dc/issues/398)
|
||||
* **cypress:** register the deleteFolder task, restore the spec order, hold metanav out ([c803773](https://git.datacontroller.io/dc/dc/commit/c803773d3c36476d8263f01fc3b8344aa3503d44)), closes [#395](https://git.datacontroller.io/dc/dc/issues/395)
|
||||
* **deploy:** do not offer the deploy wizard inside a report ([80e8911](https://git.datacontroller.io/dc/dc/commit/80e8911af8857fa2c417f028cd59b64a87233b9e))
|
||||
* **deploy:** keep the app query when opening the standalone URL ([e51953c](https://git.datacontroller.io/dc/dc/commit/e51953ca8a6734e1649bad68cf4ad2057d677334))
|
||||
* **deploy:** one definition of the standalone URL, and test the guards ([459ff2d](https://git.datacontroller.io/dc/dc/commit/459ff2da371375e996839d69deb40f681c356f1d)), closes [#TCS](https://git.datacontroller.io/dc/dc/issues/TCS)
|
||||
* **deploy:** strip the embed parameters by name, and scope the timezone runs to the VA specs ([f7e7538](https://git.datacontroller.io/dc/dc/commit/f7e753863b61d8545bd0777d80267661b3c58670))
|
||||
* **embed:** build quarter ends from the calendar, not minus 24 hours ([7e3045a](https://git.datacontroller.io/dc/dc/commit/7e3045a21b67053b76a81bfe836e31571ccbffbf))
|
||||
* **embed:** defer the column check until metadata loads, report AND-ed pairs ([2623a3f](https://git.datacontroller.io/dc/dc/commit/2623a3fb4ba3bf19c00636b1717b53d5a2d54f3a))
|
||||
* **embed:** log a parse-time mapping problem once ([0011f7e](https://git.datacontroller.io/dc/dc/commit/0011f7e933d85be8e33debe3a6ecb9000edaabe3))
|
||||
* **embed:** make the va_parmap diagnostics say what they mean ([541a61b](https://git.datacontroller.io/dc/dc/commit/541a61ba08bc7256a911c50de7078859cc46c84e))
|
||||
* **embed:** match a mapping on the parameter label only, and warn only when the values differ ([e9216b0](https://git.datacontroller.io/dc/dc/commit/e9216b08956d5599662259a2b05cc1524c458df5))
|
||||
* **embed:** never build a clause from a value that did not convert, and say so in the banner ([bd3c220](https://git.datacontroller.io/dc/dc/commit/bd3c220af2678a2a3f8394fe3084ee03c23d0b15))
|
||||
* **embed:** one parameter maps one column, and report typos on both sides ([f83dc35](https://git.datacontroller.io/dc/dc/commit/f83dc359e258f5d1cb9fc49f0eaab9748995d823))
|
||||
* **embed:** record the parameter label only, so a name-keyed va_parmap is reported ([aa91daf](https://git.datacontroller.io/dc/dc/commit/aa91daf9f78c00cea486d0f885f8e634366bd1a0))
|
||||
* **embed:** resolve transform values as local dates and give ranges whole days ([10157c1](https://git.datacontroller.io/dc/dc/commit/10157c11595685efdc206fe7b16a5cafa098f26d))
|
||||
* **embed:** warn when a va_parmap entry is ignored ([411f6c3](https://git.datacontroller.io/dc/dc/commit/411f6c3354b7fbfec11fef3b174ae1f8066c92ba))
|
||||
* **makedata:** pair the %else of the keepme block so it parses in open code ([c0ea810](https://git.datacontroller.io/dc/dc/commit/c0ea810766ebfda5ae5358ab14146fbcc07fcc8e))
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* **embed:** map VA parameters to DC columns from the URL ([15a1f34](https://git.datacontroller.io/dc/dc/commit/15a1f34d5263503cac83b4fe4e52482777dc2f1f))
|
||||
* **embed:** resolve va_parmap values through named date anchors ([88010c3](https://git.datacontroller.io/dc/dc/commit/88010c3980c2ea1c39bbb024ed32d63e495e61f0))
|
||||
|
||||
# [7.19.0](https://git.datacontroller.io/dc/dc/compare/v7.18.0...v7.19.0) (2026-10-28)
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* address the review findings on the harness and refresh changes ([36e4e88](https://git.datacontroller.io/dc/dc/commit/36e4e88eb134d3b48217d6dd185ea3c03f03fba5))
|
||||
* **ci:** dispatch the demo deploy with a tag ref and the actions scope ([698dea6](https://git.datacontroller.io/dc/dc/commit/698dea6d6b7a8635313f11acfcc7a4f39a02bc99))
|
||||
* **ci:** restore the gitleaks ignore entries for the mock licence pair ([0d58be9](https://git.datacontroller.io/dc/dc/commit/0d58be975d8eb2d7f1ab6a648f760151392c7973))
|
||||
* **client:** open the SAS log page with noopener ([072a133](https://git.datacontroller.io/dc/dc/commit/072a1337020970b5b4ff8aa31719cc5ffec7701e))
|
||||
* **client:** refuse a truncated table in the multi-table upload ([1fb4818](https://git.datacontroller.io/dc/dc/commit/1fb4818d215f538a0ac51772ab1f261aeb91f7e1))
|
||||
* **cypress:** apply the licence key when the estate has none ([be8f445](https://git.datacontroller.io/dc/dc/commit/be8f445ae1051e132fb6eb5a69b45264255326b5)), closes [input#file-upload](https://git.datacontroller.io/input/issues/file-upload)
|
||||
* **dc_execute:** use = comparisons, not IN, in the fail-fast guard ([fed43cb](https://git.datacontroller.io/dc/dc/commit/fed43cb07ef7f2afc685612a124dbc16d7618e21))
|
||||
* **dc_getlibs:** exclude the DC libref from the dictionary scan ([f5aade5](https://git.datacontroller.io/dc/dc/commit/f5aade59760cc376b0f96eb4c892c8bca82dde14))
|
||||
* **dc_refreshcatalog:** terminate the generated statements ([1e22476](https://git.datacontroller.io/dc/dc/commit/1e224763ba80500a112812a983801968e3751959))
|
||||
* **derived:** a fileref with nothing written is a FAIL, not an error ([9b3cf50](https://git.datacontroller.io/dc/dc/commit/9b3cf50b9c3889786605f0f5d9032d84c50f7107))
|
||||
* **derived:** address five review findings ([ccd468b](https://git.datacontroller.io/dc/dc/commit/ccd468b3358c1418ee6d8e198b088a0cf23b8c24))
|
||||
* **derived:** carry the placement sequence in mpe_derived ([be50cb5](https://git.datacontroller.io/dc/dc/commit/be50cb5d082e788c6ae106bc1d02e73a8c7b9646))
|
||||
* **derived:** close four gaps in the MPE_DERIVED hook ([9914790](https://git.datacontroller.io/dc/dc/commit/9914790777f92128c74ecce67e6c67ee48824b45))
|
||||
* **derived:** close the RLS and search gaps on derived columns ([9ef813d](https://git.datacontroller.io/dc/dc/commit/9ef813d439b48e7fefd35c426e62b99e5943002a))
|
||||
* **derived:** close the two gaps in the derived/RLS invariant ([422f766](https://git.datacontroller.io/dc/dc/commit/422f766cd43db4ce740e1890fd5df30790eaa7e3))
|
||||
* **derived:** compare the outref as text, prove the file is empty ([7830b7b](https://git.datacontroller.io/dc/dc/commit/7830b7bcfa9806dde840ce8dd3e6a1887d0b38be))
|
||||
* **derived:** do not drop a shadowed column from staging or downloads ([2d5de9a](https://git.datacontroller.io/dc/dc/commit/2d5de9a324a07a36edc0a4f74c8fad09fde07cbf))
|
||||
* **derived:** do not duplicate a shadowed column in the grid payload ([6a1128a](https://git.datacontroller.io/dc/dc/commit/6a1128a46352f77c95dd02df9309f0984a1db2d1))
|
||||
* **derived:** eight review findings ([561d7fc](https://git.datacontroller.io/dc/dc/commit/561d7fc3ed2a460835619962cf4dcb29d3dc87c7))
|
||||
* **derived:** enforce the ACTIVE picker pair in the extracted check ([1863076](https://git.datacontroller.io/dc/dc/commit/1863076bc956b4ff94bf7b85b3dde0d6e3b54d7a)), closes [#366](https://git.datacontroller.io/dc/dc/issues/366)
|
||||
* **derived:** enforce the documented limits, and handle derived filters in VIEW ([d3dec68](https://git.datacontroller.io/dc/dc/commit/d3dec682aafeb143fe983d6546337cb6490f5620))
|
||||
* **derived:** filtering a format catalog no longer aborts the service ([469229a](https://git.datacontroller.io/dc/dc/commit/469229aecbaad304a835a5f229df111c37ff7c0d))
|
||||
* **derived:** guard the bitemporal cut-off, coalesce the selectbox seed ([7fb981d](https://git.datacontroller.io/dc/dc/commit/7fb981daa91bce29c2d62c246185d01dcad08d6b))
|
||||
* **derived:** guard the empty physical column list in the dep check ([5f5d868](https://git.datacontroller.io/dc/dc/commit/5f5d8681b9d78280a5200309917415e135c3ce5b))
|
||||
* **derived:** hold a derived column read-only whatever the table's security ([5f443b0](https://git.datacontroller.io/dc/dc/commit/5f443b0b16a2abd3c11d96caa7a043911c29ef01))
|
||||
* **derived:** ignore a re-added name when collecting inactive rows ([6b5bf3e](https://git.datacontroller.io/dc/dc/commit/6b5bf3e6be89b22e5a633c34b9aecbc8c7d48ee2))
|
||||
* **derived:** ignore inactive rows and VIEW rules in the submit guards ([85c89ca](https://git.datacontroller.io/dc/dc/commit/85c89ca2a4e02112ba496ac36a222f33675544da))
|
||||
* **derived:** keep targetds for plain datasets, test the empty file ([f8f6568](https://git.datacontroller.io/dc/dc/commit/f8f65681e1489f4ce94d9127d5e4c0c82798e960))
|
||||
* **derived:** make the model enforce the required columns, and leave VAR_FMT optional ([7efbf31](https://git.datacontroller.io/dc/dc/commit/7efbf31ea5ac234bd8f924fdf89a95d101b3aff1))
|
||||
* **derived:** only the delete marker is a reserved name ([af31c36](https://git.datacontroller.io/dc/dc/commit/af31c36b670ec63a2a4ba056bf6c4b28f6096e08))
|
||||
* **derived:** open the statements file with a long record length ([ee31782](https://git.datacontroller.io/dc/dc/commit/ee31782fbff909b540162b1bdfca6f307208331b))
|
||||
* **derived:** pair the filterds %else with %do/%end ([88a38db](https://git.datacontroller.io/dc/dc/commit/88a38db877dba9b98ccf50f0cc7242674654586b))
|
||||
* **derived:** raise the VAR_ORDER ceiling to 10000 ([27ddcfb](https://git.datacontroller.io/dc/dc/commit/27ddcfb5759d6eb8f9880adc7c271a408e7741ad))
|
||||
* **derived:** refuse dependencies on inactive rows, and keep the RLS scope guard coherent ([7799a30](https://git.datacontroller.io/dc/dc/commit/7799a300df82339683eea21e4a89ce7a6471c575))
|
||||
* **derived:** reject a dependency on a row this submission deletes ([8806504](https://git.datacontroller.io/dc/dc/commit/88065049674c2cc0141eef22704b759f5b708b0f))
|
||||
* **derived:** repair the validation rows for MPE_DERIVED ([9a047ad](https://git.datacontroller.io/dc/dc/commit/9a047ad42ec86df7d1eaf5058c0edde56016e88f))
|
||||
* **derived:** review feedback - cover the empty response, assert the last row ([e0da9f1](https://git.datacontroller.io/dc/dc/commit/e0da9f1098db63185538398ca478696cc849da4b))
|
||||
* **derived:** stop the hooks aborting on their own macro variables ([688550a](https://git.datacontroller.io/dc/dc/commit/688550a28d9794ac6b878926a0e1e2ef8f6e9335))
|
||||
* **derived:** truncate the derived label on an even quote boundary ([1b5c4b3](https://git.datacontroller.io/dc/dc/commit/1b5c4b3ecd962a28e08fc5671b57dbf7ae0b8d9f))
|
||||
* **derived:** verify on stripped values, and label with the variable name ([4d9ad38](https://git.datacontroller.io/dc/dc/commit/4d9ad38df1fa89b0d5a12214d703d9c1fe5295f2))
|
||||
* **editor:** a blank truncation note is not a warning ([f182179](https://git.datacontroller.io/dc/dc/commit/f18217940365f27ae388fae087e6126931e1222c))
|
||||
* **editor:** keep decimal values when an uploaded file is edited manually ([04ce186](https://git.datacontroller.io/dc/dc/commit/04ce1861e7821063b1be42c22f92f62aed509602))
|
||||
* **getcolvals:** report an invalid value, not missing inputs ([4bcee70](https://git.datacontroller.io/dc/dc/commit/4bcee706ffaf1929bdd14bb2fb15111174ca3916))
|
||||
* **getdata.test:** pin the derived filter to a libref, and spell the table as it is ([45dce7b](https://git.datacontroller.io/dc/dc/commit/45dce7b11b7a4399c1d0dc9c3e7a283b1cf59df1))
|
||||
* **getdata:** clear the globals it declares before use ([359b570](https://git.datacontroller.io/dc/dc/commit/359b570b85aa2f66507e81ea2977243be4590f83))
|
||||
* **getddl.test:** ask for a table, not a whole library ([059cdf1](https://git.datacontroller.io/dc/dc/commit/059cdf176a7ec556b45ea1b21d880c048eba14ca))
|
||||
* **getrawdata:** declare mp_ds2md so the Markdown download can run ([661ee19](https://git.datacontroller.io/dc/dc/commit/661ee19cfa5f996bf01f2b11817039f38dbb9985))
|
||||
* **lint:** clear the sasjs lint warnings on the files 7.19 touched ([355c46a](https://git.datacontroller.io/dc/dc/commit/355c46a7b64994d0146a88657dc5635593249138))
|
||||
* **makedata:** read KEEPME from every input row, and keep it out of the Viya seed ([d626cb8](https://git.datacontroller.io/dc/dc/commit/d626cb8f2daf94fde7ff21d924b9a2ca4968d983))
|
||||
* **mocks:** address the audit-pack review findings ([489819b](https://git.datacontroller.io/dc/dc/commit/489819b05111323ae1e34edda22a5446cdbc2d87))
|
||||
* **mocks:** carry the zip flags into the central directory ([c4021a4](https://git.datacontroller.io/dc/dc/commit/c4021a42ffaed2af80c53aaa1455249190b339b8))
|
||||
* **mocks:** clear the licence for published sites, not for the estate ([62f55db](https://git.datacontroller.io/dc/dc/commit/62f55db170185900063f7a431cd8f375ebabf930))
|
||||
* **mocks:** copy the physical column list before deriving ([f411b7d](https://git.datacontroller.io/dc/dc/commit/f411b7d269381e3f7cab2d85b6d8f7d7d50ea4d1))
|
||||
* **mocks:** derive before filtering in the viewer, and drop the columns ([60ec470](https://git.datacontroller.io/dc/dc/commit/60ec470404e5c279b58ac6a01768dfc83413b197))
|
||||
* **mocks:** derive the retained key wherever rows are staged ([74060c0](https://git.datacontroller.io/dc/dc/commit/74060c0585fb8b217cb181ea747ad318f5238868))
|
||||
* **mocks:** drop derived variables before staging ([73d25b1](https://git.datacontroller.io/dc/dc/commit/73d25b1220d574ef2547fbd9e6e3cf1ceda375ee))
|
||||
* **mocks:** key rk_underlying tables on their business key ([347dbb5](https://git.datacontroller.io/dc/dc/commit/347dbb53b01c47bbd562de14d5ccbae7d96407c6))
|
||||
* **mocks:** register MPE_DERIVED and seed its field rules ([ee3c878](https://git.datacontroller.io/dc/dc/commit/ee3c8787efef9cd716bdd2bf80f6cb6751461db9))
|
||||
* **mocks:** scope the derived hook's graph to the submission ([6cad23f](https://git.datacontroller.io/dc/dc/commit/6cad23fce9ac0cf64c247c40f3b7eac875fd4cfb))
|
||||
* **mocks:** search the physical columns only in the viewer ([293022d](https://git.datacontroller.io/dc/dc/commit/293022d46b27e096fd514cdda49f688612df2f5b))
|
||||
* **mocks:** stop the mock advertising invalid licence keys ([fb36379](https://git.datacontroller.io/dc/dc/commit/fb36379520028baaf4bc3f08131bfc88deab3ac3))
|
||||
* **mpe_assert_abort:** drop the unused line variable ([9c294c0](https://git.datacontroller.io/dc/dc/commit/9c294c0ab5ca8fe30a235553a298c1c68a61442f))
|
||||
* **mpe_assert_abort:** take the abort's own MSG, tolerate a missing response ([0a52d20](https://git.datacontroller.io/dc/dc/commit/0a52d20c5fc27b4adde5fc8ed6f39a58c48f62ff))
|
||||
* **mpe_execute:** fail fast when the child left no status ([a036386](https://git.datacontroller.io/dc/dc/commit/a03638667f1968c5909f496d954eeaa13f316763))
|
||||
* **mpe_execute:** no apostrophe in the SAS 9 status note ([2204085](https://git.datacontroller.io/dc/dc/commit/220408538b26016bda45e48d619cdbc3e1cbcc59))
|
||||
* **mpe_execute:** read an unknown state as unmeasured, not failed ([f2b1428](https://git.datacontroller.io/dc/dc/commit/f2b1428ec3099b2b6e3c6da6627530ef81fab201))
|
||||
* name the startupservice launcher button "Show log" ([3445107](https://git.datacontroller.io/dc/dc/commit/3445107d9a91642b67e114bcfcd81c3ce47f3f0f))
|
||||
* offer the launcher on the globvars failure modal too ([ad2da45](https://git.datacontroller.io/dc/dc/commit/ad2da453896a51f2cc7e49cebb6c84f7b47d7395))
|
||||
* **refresh:** name each catalogue load uniquely in the audit trail ([46bacdd](https://git.datacontroller.io/dc/dc/commit/46bacdd6d9ea814bcd02a1ff8aa3f447baf02f64))
|
||||
* **refresh:** one audit folder per refresh call ([56b6620](https://git.datacontroller.io/dc/dc/commit/56b6620ef402b288ce78a2a77662b651e310753e))
|
||||
* **sites:** round-trip the committed licence before publishing ([4d462d3](https://git.datacontroller.io/dc/dc/commit/4d462d310f7df23e5d5229cb0702721b7ad41bbd))
|
||||
* **test:** getdata test ([da30266](https://git.datacontroller.io/dc/dc/commit/da30266278c4fe27a72810cd5ed647cf3bd823af))
|
||||
* **tests:** enable closeouts logging ([fdcd8fa](https://git.datacontroller.io/dc/dc/commit/fdcd8fa4b3e7a7a119c662974d70dc9eea80bdae))
|
||||
* **tests:** read the work location from _SASJS_WORK ([033ffd3](https://git.datacontroller.io/dc/dc/commit/033ffd302502c7530fc44a314364f5cdb3be6ad2))
|
||||
* **ui:** one error modal at a time ([81ccf75](https://git.datacontroller.io/dc/dc/commit/81ccf758e9dab0c7632ab50bf469e88b8c4c4420))
|
||||
* **validatefilter:** build targetds without macro triggers in comments ([42355f1](https://git.datacontroller.io/dc/dc/commit/42355f17bc07bf4d44ba858984d83b9ed72cc0ba))
|
||||
* **validatefilter:** drop the dead nobs assignment, keep targetds precedence ([337ea8e](https://git.datacontroller.io/dc/dc/commit/337ea8e5e021c6f2451fa3ad8396148744118fc9))
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* **dc_getlibs:** dump the library list when debug is on ([582523d](https://git.datacontroller.io/dc/dc/commit/582523dc87a253b71824359224070ccb4746a386))
|
||||
* **derived:** allow a variable to shadow a physical column ([12beeff](https://git.datacontroller.io/dc/dc/commit/12beeff69518376a042fbbc58332acc40d89d132))
|
||||
* **derived:** declare VAR_DEPENDENCIES instead of VAR_ORDER ([813bce6](https://git.datacontroller.io/dc/dc/commit/813bce685691ad252043eeab31eb5460de81b756))
|
||||
* **derived:** optional VAR_LABEL on MPE_DERIVED ([7c9ce9a](https://git.datacontroller.io/dc/dc/commit/7c9ce9a712137e9825a5f2ff52796d765423147b))
|
||||
* **derived:** pickers, and an ACTIVE default, for MPE_DERIVED ([87e794f](https://git.datacontroller.io/dc/dc/commit/87e794f769fe420f214f43b141f0b41cf88eefee))
|
||||
* **derived:** report the dependency cycle path ([cee1a95](https://git.datacontroller.io/dc/dc/commit/cee1a954ee97e17f297b7cf43ef0e5cfc6e393a6))
|
||||
* **derived:** take any text in VAR_LABEL, but not a line break ([78e88a4](https://git.datacontroller.io/dc/dc/commit/78e88a41128b047c0276ccf647f128229152604c))
|
||||
* **derived:** validate MPE_DERIVED at submit time ([0b010f2](https://git.datacontroller.io/dc/dc/commit/0b010f2c65ddada70d62ea83c706943c6c45756f))
|
||||
* **derived:** validation rules and a VAR_TYPE dropdown for MPE_DERIVED ([aa6d50e](https://git.datacontroller.io/dc/dc/commit/aa6d50e1ea5fc2fe6be8aa9fa859b92d0a1061c5))
|
||||
* **derived:** VARIABLE_ORDER, so a derived variable can build on another ([33d043e](https://git.datacontroller.io/dc/dc/commit/33d043ea92f007dca70c95d5363329d4447eeba9))
|
||||
* **editor:** derived variables in the EDIT screen ([b0fd354](https://git.datacontroller.io/dc/dc/commit/b0fd354ecdbd7bf29196ed983ff3cce691affeed))
|
||||
* **editor:** mark derived variables as computed in the EDIT grid ([6513d09](https://git.datacontroller.io/dc/dc/commit/6513d09f059de15c5304f99b3f8d898d46ea1be7))
|
||||
* **editor:** serve a truncated selection with a warning instead of aborting ([34691dc](https://git.datacontroller.io/dc/dc/commit/34691dceecb63f952b23ad8acc5bc93753c6cfee))
|
||||
* **editor:** show the derived column label in the header tooltip ([cfe6647](https://git.datacontroller.io/dc/dc/commit/cfe6647ca870c45a3d8abc2753ce7f070e3fdf4a))
|
||||
* **mocks:** add getauditfile and the audit-folder artefacts ([752d944](https://git.datacontroller.io/dc/dc/commit/752d944c2091ef6d2624a15ab2e2e7be35608798))
|
||||
* **mocks:** implement derived variables in the JS services ([b4fae88](https://git.datacontroller.io/dc/dc/commit/b4fae88ae13658607e4209449577cd42849d3a99))
|
||||
* **mocks:** validate a submitted MPE_DERIVED row like the hook does ([5d183d1](https://git.datacontroller.io/dc/dc/commit/5d183d135085ea023f32e5b1d4992d997394a8fb))
|
||||
* run startupservice in a new window from its error modal ([40f6133](https://git.datacontroller.io/dc/dc/commit/40f61331aad6bc8a52c551b924f0d1f1e6426cee))
|
||||
* **sites:** mint a licence for the published sites ([7d9df71](https://git.datacontroller.io/dc/dc/commit/7d9df71fdb0ab265e745ed3b708038983e855a87))
|
||||
|
||||
|
||||
### Reverts
|
||||
|
||||
* **derived:** no shadowing, and no self reference ([5af8e6f](https://git.datacontroller.io/dc/dc/commit/5af8e6fc359221586af74a685159dfd75a40ec45))
|
||||
* **stagedata.test.4:** back to the MPE_LOADS evidence for now ([31aa859](https://git.datacontroller.io/dc/dc/commit/31aa8591538409fc1a3ef97d7474120094ce202d))
|
||||
|
||||
# [7.18.0](https://git.datacontroller.io/dc/dc/compare/v7.17.0...v7.18.0) (2026-10-19)
|
||||
|
||||
|
||||
|
||||
+1
-1
@@ -8,7 +8,7 @@ This file is the shared glossary and orientation for the repo. When your output
|
||||
|
||||
- `client/` - the Angular frontend (TypeScript). Uses [Handsontable](.agents/skills/handsontable/SKILL.md) for the editable grid and [HyperFormula](.agents/skills/hyperformula/SKILL.md) for Excel-formula support. Lint with `npm run lint:check` from `client/`.
|
||||
- `sas/` - the SAS backend. Services, macros, hooks and tests live under `sas/sasjs/**`; `sas/sasjsbuild/` is generated output (never hand-edit). Lint `.sas` files with `sasjs lint` from `sas/`.
|
||||
- `.agents/docs/` - technical deep-dives (see below). `.agents/skills/` - task skills (handsontable, hyperformula, dc-sas for Data Controller SAS development, dc-estate-testing for deploying to and testing on the shared estate, plus npx-installed sasjs/ sas skills).
|
||||
- `.agents/docs/` - technical deep-dives (see below). `.agents/skills/` - task skills (handsontable, hyperformula, dc-sas for Data Controller SAS development, plus npx-installed sasjs/ sas skills).
|
||||
|
||||
## Roles
|
||||
|
||||
|
||||
+2
-3
@@ -86,7 +86,6 @@ RUN_TIMES=js
|
||||
NODE_PATH=node
|
||||
CORS=enable
|
||||
WHITELIST=http://localhost:4200
|
||||
HTTP_PORT=5000
|
||||
```
|
||||
|
||||
`RUN_TIMES=js` means the server executes `.js` files as Stored Programs (no SAS binary needed). `CORS=enable` plus `WHITELIST` allows the Angular dev server on port 4200 to call the mock services on port 5000. Start the server:
|
||||
@@ -95,7 +94,7 @@ HTTP_PORT=5000
|
||||
./api-linux # or api-macos / api-win.exe depending on your platform
|
||||
```
|
||||
|
||||
`HTTP_PORT` is not optional in SASjs Server 3.x: `PORT` and `PROTOCOL` are deprecated, and a server started without `HTTP_PORT` or `HTTPS_PORT` exits with "no listener is enabled". With it set, the server runs in desktop mode (no auth) on `http://localhost:5000`.
|
||||
The server runs in desktop mode (no auth) on `http://localhost:5000` by default.
|
||||
|
||||
### 2. Deploy the mock services and streaming frontend
|
||||
|
||||
@@ -136,7 +135,7 @@ After changing mock services, re-run `sasjs cbd -t server-ci` from `sas/mocks/sa
|
||||
|
||||
### Demo and staging sites
|
||||
|
||||
A push to `release/**` publishes [staging.datacontroller.io](https://staging.datacontroller.io); a release tag (`v*`, created by the release job after its tests pass) publishes [demo.datacontroller.io](https://demo.datacontroller.io) - workflow file `.gitea/workflows/sites.yaml`, using the `DEMO_DATACONTROLLER_IO` Surfer app password (user `allan`). Both are sites on one multi-site Surfer instance; the demo is its default site. The job builds the mocks as a browser-hosted site (SASjs CLI >= 6.1, `streamConfig.mockWeb`), so the services run in a service worker and the drive lives in the visitor's IndexedDB - there is no server. It runs `makedata` on a throwaway SASjs Server first and ships that drive as `sasjsbuild/mock-snapshot.json`; without it the site opens with no data. The estate carries no licence key, so CI runs on the free tier until a key is registered through the licensing screen (`admin/registerkey`) - the Cypress setup does that for the specs that need one. The Sites job writes a licence into the published drive, so demo and staging are fully usable. The pair lives in `sas/mocks/mock-licence.json`: the two values are one RSA key and are regenerated together, and the `.gitleaksignore` entries for it are line-scoped, so inserting a line above the pair re-fires the scanner rather than revealing a new secret.
|
||||
A push to `release/**` publishes [staging.datacontroller.io](https://staging.datacontroller.io); a release tag (`v*`, created by the release job after its tests pass) publishes [demo.datacontroller.io](https://demo.datacontroller.io) - workflow file `.gitea/workflows/sites.yaml`, using the `DEMO_DATACONTROLLER_IO` Surfer app password (user `allan`). Both are sites on one multi-site Surfer instance; the demo is its default site. The job builds the mocks as a browser-hosted site (SASjs CLI >= 6.1, `streamConfig.mockWeb`), so the services run in a service worker and the drive lives in the visitor's IndexedDB - there is no server. It runs `makedata` on a throwaway SASjs Server first and ships that drive as `sasjsbuild/mock-snapshot.json`; without it the site opens with no data. The mock ships no licence key, so both sites run on the free tier.
|
||||
|
||||
## GUI Elements
|
||||
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
import fs from 'fs'
|
||||
import { defineConfig } from 'cypress'
|
||||
|
||||
export default defineConfig({
|
||||
@@ -35,28 +34,7 @@ export default defineConfig({
|
||||
|
||||
e2e: {
|
||||
video: true,
|
||||
// One retry in a CI run, none interactively. `editor.cy.ts` test 28
|
||||
// intermittently fails on "app-loading ... continuously found" - a slow load,
|
||||
// not an assertion - and a single retry costs a few seconds where a red
|
||||
// pipeline costs forty minutes. Failures that are real still fail twice, and
|
||||
// the retried attempt's output is in the run log.
|
||||
retries: { runMode: 1, openMode: 0 },
|
||||
setupNodeEvents(on, config) {
|
||||
// `download.cy.ts` clears the downloads folder before each test through
|
||||
// `cy.task('deleteFolder')`. That task was registered in
|
||||
// cypress/plugins/index.js, which Cypress 10+ never loads unless it is
|
||||
// required from here - so the task had no handler and the spec failed on
|
||||
// its first hook.
|
||||
on('task', {
|
||||
deleteFolder(folderName: string) {
|
||||
return new Promise((resolve, reject) => {
|
||||
fs.rm(folderName, { recursive: true, force: true }, (err) =>
|
||||
err ? reject(err) : resolve(null)
|
||||
)
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
// Pin the browser locale so locale-formatted cells (intl-date/time/datetime)
|
||||
// render deterministically regardless of the runner's system locale.
|
||||
on('before:browser:launch', (browser, launchOptions) => {
|
||||
|
||||
@@ -40,8 +40,8 @@ context('special missings clip (DEMO_01)', function () {
|
||||
this.beforeEach(() => {
|
||||
cy.visit(hostUrl + appLocation)
|
||||
|
||||
// Assumes the estate is already activated (the licensing spec registers a
|
||||
// valid key); otherwise a licensing page would sit in shot.
|
||||
// The mock estate carries a valid licence key (mock-storage/licence.json),
|
||||
// so the app activates directly with no free-tier banner in shot.
|
||||
cy.get('.nav-tree', { timeout: longerCommandTimeout }).should('exist')
|
||||
})
|
||||
|
||||
|
||||
@@ -75,76 +75,6 @@ context('derived variables: ', function () {
|
||||
|
||||
cy.screenshot('derived-3-filter-value', { capture: 'viewport' })
|
||||
})
|
||||
|
||||
it('4 | a derived column is marked read-only, and opens no editor', () => {
|
||||
openEditor('TESTDATA.DERIVED_DEMO')
|
||||
|
||||
// a derived column is read-only AND computed, so the grid marks it apart
|
||||
// from an ordinary read-only column - its own header/cell class, no
|
||||
// readonlyCell dimming (editor.component.ts, afterGetColHeader /
|
||||
// afterGetCellMeta). STATE is the control: same grid, a physical column.
|
||||
headerCell('REGION').should('have.class', 'dc-derived-header')
|
||||
headerCell('REGION').should('not.have.class', 'readonlyCell')
|
||||
headerCell('STATE')
|
||||
.should('not.have.class', 'dc-derived-header')
|
||||
.and('not.have.class', 'readonlyCell')
|
||||
getCellByHeaderAndRow(0, 'REGION').should('have.class', 'dc-derived-cell')
|
||||
getCellByHeaderAndRow(0, 'STATE').should(
|
||||
'not.have.class',
|
||||
'dc-derived-cell'
|
||||
)
|
||||
// The computed block opens at whichever derived column comes first in the
|
||||
// grid - MPE_DERIVED order, so LINE_VALUE here, not REGION. Assert the
|
||||
// invariant rather than a name: exactly one derived header carries the
|
||||
// marker, and it is the first one.
|
||||
cy.get('.ht_clone_top .htCore thead tr th').then(($ths) => {
|
||||
const derived = [...$ths].filter((th) =>
|
||||
th.classList.contains('dc-derived-header')
|
||||
)
|
||||
expect(derived.length, 'derived headers').to.be.greaterThan(0)
|
||||
expect(
|
||||
derived[0].classList.contains('dc-derived-first'),
|
||||
'the first derived header opens the block'
|
||||
).to.be.true
|
||||
derived
|
||||
.slice(1)
|
||||
.forEach(
|
||||
(th) =>
|
||||
expect(
|
||||
th.classList.contains('dc-derived-first'),
|
||||
'only the first derived header carries the marker'
|
||||
).to.be.false
|
||||
)
|
||||
})
|
||||
|
||||
// the hover text carries the label, which is what says what the column is:
|
||||
// the configured VAR_LABEL where there is one, otherwise the generated
|
||||
// name=expression
|
||||
headerCell('REGION_LABEL')
|
||||
.should('have.attr', 'title')
|
||||
.and('contain', 'Region and state')
|
||||
headerCell('REGION')
|
||||
.should('have.attr', 'title')
|
||||
.and('contain', 'REGION=ifc')
|
||||
|
||||
enterEditMode()
|
||||
|
||||
getCellByHeaderAndRow(0, 'REGION').dblclick({ force: true })
|
||||
cy.get('.handsontableInput').should('not.exist')
|
||||
getCellByHeaderAndRow(0, 'REGION').should('have.text', 'West')
|
||||
})
|
||||
|
||||
/**
|
||||
* The positive control for the case above: same screen, same edit mode, a
|
||||
* physical column - so its silence is about the column, not the screen.
|
||||
*/
|
||||
it('5 | a physical column in the same grid does accept an edit', () => {
|
||||
openEditor('TESTDATA.DERIVED_DEMO')
|
||||
enterEditMode()
|
||||
|
||||
getCellByHeaderAndRow(0, 'STATE').dblclick({ force: true })
|
||||
cy.get('.handsontableInput').should('exist')
|
||||
})
|
||||
})
|
||||
|
||||
/**
|
||||
@@ -173,23 +103,6 @@ const openEditor = (libMem: string) => {
|
||||
cy.wait(2500)
|
||||
}
|
||||
|
||||
/** The EDIT screen opens read-only; this is the button that starts an edit. */
|
||||
const enterEditMode = () => {
|
||||
cy.get('.btnCtrl button.btn-primary', { timeout: longerCommandTimeout })
|
||||
.click()
|
||||
.then(() => cy.wait(2500))
|
||||
}
|
||||
|
||||
/** The grid header cell for a column, by its rendered text. */
|
||||
const headerCell = (headerText: string) =>
|
||||
cy.get('.ht_clone_top .htCore thead tr th').then(($ths) => {
|
||||
const th = [...$ths].find(
|
||||
(candidate) => (candidate as HTMLElement).innerText.trim() === headerText
|
||||
)
|
||||
if (!th) throw new Error(`header not found: ${headerText}`)
|
||||
return cy.wrap(th)
|
||||
})
|
||||
|
||||
const openFilterPopup = () => {
|
||||
cy.contains('button.btnView', 'Filter', {
|
||||
timeout: longerCommandTimeout
|
||||
|
||||
@@ -17,20 +17,9 @@ context('download files test: ', function () {
|
||||
|
||||
this.beforeEach(() => {
|
||||
cy.visit(hostUrl + appLocation)
|
||||
|
||||
// beforeAll already logged in. Typing into the login form unconditionally
|
||||
// failed with "the center of this element is hidden from view" whenever the
|
||||
// session was live and the form was not on screen, so only fill it when it
|
||||
// is actually shown.
|
||||
cy.get('body').then(($body) => {
|
||||
const usernameInput = $body.find('input.username')[0]
|
||||
|
||||
if (usernameInput && !Cypress.dom.isHidden(usernameInput)) {
|
||||
cy.get('input.username').type(username)
|
||||
cy.get('input.password').type(password)
|
||||
cy.get('.login-group button').click()
|
||||
}
|
||||
})
|
||||
cy.get('input.username').type(username)
|
||||
cy.get('input.password').type(password)
|
||||
cy.get('.login-group button').click()
|
||||
|
||||
visitPage('home')
|
||||
})
|
||||
|
||||
@@ -6,90 +6,44 @@ const libraryToOpenIncludes = Cypress.env(`libraryToOpenIncludes_${serverType}`)
|
||||
|
||||
// SAS Visual Analytics data-driven content embed mode (`?embed=va`):
|
||||
// - chrome is hidden (same as embed=true),
|
||||
// - the editor opens READ-ONLY (the user clicks Edit, as in normal use),
|
||||
// - the Filter button is hidden - VA filters replace the filter builder,
|
||||
// - the Add Row / Add Record buttons are hidden.
|
||||
// - the editor opens in edit mode immediately (not read-only),
|
||||
// - the only action button is a single Submit below the grid (stubbed in v1),
|
||||
// - the top button bar and Add Record are hidden.
|
||||
//
|
||||
// The detailed PK+label data-merge logic is covered by the VaMessagingService
|
||||
// unit spec; here we assert the deterministic UI layout, that pushing a VA
|
||||
// postMessage does not break the page, and (case 3) that a `va_parmap` URL
|
||||
// mapping filters the column it names end to end.
|
||||
/**
|
||||
* Assert on the row count the editor shows. The assertion runs inside a Cy
|
||||
* `should`, so it retries until the grid has loaded: reading the text once would
|
||||
* capture the count from before a filter is applied.
|
||||
*/
|
||||
const parseRows = (text: string): number => {
|
||||
const match = /\(([\d,]+) rows?/.exec(text)
|
||||
return match ? Number(match[1].replace(/,/g, '')) : NaN
|
||||
}
|
||||
|
||||
const expectRows = (check: (rows: number) => void) =>
|
||||
cy.get('h4', { timeout: longerCommandTimeout }).should(($h4) => {
|
||||
check(parseRows($h4.text()))
|
||||
})
|
||||
|
||||
/** Assert the grid shrank, without pinning a fixture-specific number. */
|
||||
const expectFewerRowsThan = (baseline: number) =>
|
||||
expectRows((rows) => {
|
||||
expect(rows, 'filtered row count').to.be.lessThan(baseline)
|
||||
})
|
||||
|
||||
/** Read the unfiltered row count from the app itself, retrying until it loads. */
|
||||
const readRowCount = (): Cypress.Chainable<number> =>
|
||||
expectRows((rows) => {
|
||||
expect(rows, 'row count').to.be.greaterThan(0)
|
||||
})
|
||||
.get('h4')
|
||||
.invoke('text')
|
||||
.then(parseRows)
|
||||
|
||||
// unit spec; here we only assert the deterministic UI layout and that pushing a
|
||||
// VA postMessage does not break the page.
|
||||
context('embed=va (VA data-driven content) tests: ', function () {
|
||||
// How many rows the table has unfiltered. Captured once, from the app itself.
|
||||
let unfilteredRows = 0
|
||||
|
||||
this.beforeAll(() => {
|
||||
cy.visit(`${hostUrl}/SASLogon/logout`)
|
||||
cy.loginAndUpdateValidKey()
|
||||
// Baseline for the relative row-count assertions: open the table with no VA
|
||||
// message and record what the editor itself reports.
|
||||
cy.visit(`${hostUrl}${appLocation}/#/editor/DC_JSLIB.MPE_X_TEST`, {
|
||||
timeout: 60000
|
||||
})
|
||||
readRowCount().then((count) => {
|
||||
unfilteredRows = count
|
||||
})
|
||||
})
|
||||
|
||||
this.beforeEach(() => {
|
||||
cy.visit(hostUrl + appLocation)
|
||||
// A cy.visit that differs from the current URL only in the hash is a
|
||||
// same-document navigation, so the app is not re-bootstrapped. Reload so
|
||||
// each test starts from a fresh document (the previous test may have left
|
||||
// ?embed=va in the URL, which the app reads once at init).
|
||||
cy.reload()
|
||||
visitPage('home')
|
||||
})
|
||||
|
||||
it('1 | opens the editor in VA mode: chrome hidden, read-only, no Filter button', () => {
|
||||
it('1 | opens the editor in VA mode: chrome hidden, editable, single bottom Submit', () => {
|
||||
openEditorInVaMode('mpe_x_test', () => {
|
||||
// Chrome hidden (same as embed=true)
|
||||
cy.get('header.app-header').should('not.exist')
|
||||
|
||||
// Grid present, read-only: the user enters edit mode with Edit, as usual
|
||||
// Grid present and in edit mode (read-only Filter/Edit/Upload bar absent)
|
||||
cy.get('#hotTable', { timeout: longerCommandTimeout }).should('exist')
|
||||
cy.get('.btnCtrl button').contains('Edit').should('exist')
|
||||
|
||||
// The filter builder is replaced by VA filters
|
||||
cy.get('.btnCtrl button').contains('Edit').should('not.exist')
|
||||
cy.get('.btnCtrl button').contains('Filter').should('not.exist')
|
||||
|
||||
// Not in edit mode, so no edit-mode controls
|
||||
// Top edit-mode buttons hidden in VA mode
|
||||
cy.get('.btnCtrl button.btn-outline-danger').should('not.exist') // Cancel
|
||||
cy.get('.btnCtrl').contains('Add Row').should('not.exist')
|
||||
cy.get('.btnCtrl').contains('Submit').should('not.exist')
|
||||
|
||||
// Bottom Add Record hidden in VA mode
|
||||
// Bottom Add Record hidden
|
||||
cy.contains('button', 'Add Record').should('not.exist')
|
||||
|
||||
// Exactly one Submit button, below the grid, disabled in v1
|
||||
cy.get('button').filter(':contains("Submit")').should('have.length', 1)
|
||||
cy.get('button').contains('Submit').should('be.disabled')
|
||||
})
|
||||
})
|
||||
|
||||
@@ -115,382 +69,14 @@ context('embed=va (VA data-driven content) tests: ', function () {
|
||||
// Editor still loads in VA mode after the message (chrome hidden, grid up)
|
||||
cy.get('#hotTable', { timeout: longerCommandTimeout }).should('exist')
|
||||
cy.get('header.app-header').should('not.exist')
|
||||
cy.get('.btnCtrl button').contains('Edit').should('exist')
|
||||
})
|
||||
})
|
||||
it('3 | va_parmap filters a column from a parameter that is not its name (tab 1)', () => {
|
||||
// Two DC instances over one table (one per report tab), each filtering the
|
||||
// same DC column from its own report parameter. Without the URL mapping the
|
||||
// parameter label is not a column name, so nothing would filter at all. The
|
||||
// mapping here is written in lower case against an upper-case label and
|
||||
// column, which pins the case-insensitivity of both sides end to end.
|
||||
openEditorInVaMode(
|
||||
'mpe_x_test',
|
||||
() => {
|
||||
pushVaMessage({
|
||||
name: 'pr1',
|
||||
label: 'DROPDOWN_TAB1',
|
||||
dataType: 'string',
|
||||
value: 'Option 2'
|
||||
})
|
||||
// The applied filter names the MAPPED column, and the grid shrinks to
|
||||
// the rows matching that value (Option 2 occurs twice in this table).
|
||||
cy.get('.infoBar-text', { timeout: longerCommandTimeout }).should(
|
||||
'contain.text',
|
||||
'SOME_DROPDOWN'
|
||||
)
|
||||
cy.get('.infoBar-text').should('contain.text', "'Option 2'")
|
||||
expectFewerRowsThan(unfilteredRows)
|
||||
},
|
||||
'va_parmap=dropdown_tab1-some_dropdown'
|
||||
)
|
||||
})
|
||||
|
||||
it('4 | the other tab maps its own parameter to the same column', () => {
|
||||
// Same table, same column, a different parameter label and value (Option 1
|
||||
// occurs four times) - the per-instance URL mapping is what keeps the two
|
||||
// tabs independent.
|
||||
openEditorInVaMode(
|
||||
'mpe_x_test',
|
||||
() => {
|
||||
pushVaMessage({
|
||||
name: 'pr1',
|
||||
label: 'DROPDOWN_TAB2',
|
||||
dataType: 'string',
|
||||
value: 'Option 1'
|
||||
})
|
||||
cy.get('.infoBar-text', { timeout: longerCommandTimeout }).should(
|
||||
'contain.text',
|
||||
'SOME_DROPDOWN'
|
||||
)
|
||||
cy.get('.infoBar-text').should('contain.text', "'Option 1'")
|
||||
expectFewerRowsThan(unfilteredRows)
|
||||
},
|
||||
'va_parmap=DROPDOWN_TAB2-SOME_DROPDOWN'
|
||||
)
|
||||
})
|
||||
|
||||
it('5 | warns in the grid when a mapping names a column the table lacks', () => {
|
||||
// Written by hand in the report's URL, so a typo must be visible to the
|
||||
// report author: the parameter is skipped, and the reason is shown rather
|
||||
// than leaving an unfiltered grid that looks like a working filter.
|
||||
openEditorInVaMode(
|
||||
'mpe_x_test',
|
||||
() => {
|
||||
pushVaMessage({
|
||||
name: 'pr1',
|
||||
label: 'DROPDOWN_TAB1',
|
||||
dataType: 'string',
|
||||
value: 'Option 2'
|
||||
})
|
||||
cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
|
||||
'contain.text',
|
||||
'VA parameter mapping problem'
|
||||
)
|
||||
cy.get('.va-parmap-warning').should('contain.text', 'DROPDOWN_TAB1')
|
||||
cy.get('.va-parmap-warning').should('contain.text', 'NOT_A_COLUMN')
|
||||
// VA mode hides the app header, so the banner must paint inside the
|
||||
// viewport on its own - assert the real box, not just its presence.
|
||||
cy.get('.va-parmap-warning').should(($alert) => {
|
||||
const rect = $alert[0].getBoundingClientRect()
|
||||
expect(rect.width, 'banner width').to.be.greaterThan(100)
|
||||
expect(rect.height, 'banner height').to.be.greaterThan(20)
|
||||
expect(rect.top, 'banner top').to.be.at.least(0)
|
||||
expect(rect.bottom, 'banner bottom').to.be.at.most(
|
||||
Cypress.config('viewportHeight')
|
||||
)
|
||||
})
|
||||
cy.screenshot('va-parmap-banner', { capture: 'viewport' })
|
||||
// Nothing was filtered: the row count matches the unfiltered baseline.
|
||||
expectRows((rows) => expect(rows).to.equal(unfilteredRows))
|
||||
},
|
||||
'va_parmap=DROPDOWN_TAB1-NOT_A_COLUMN'
|
||||
)
|
||||
})
|
||||
|
||||
it('6 | warns straight away when a mapping entry cannot be parsed', () => {
|
||||
// No separator, so the entry never becomes a mapping - reported before any
|
||||
// VA message arrives.
|
||||
openEditorInVaMode(
|
||||
'mpe_x_test',
|
||||
() => {
|
||||
cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
|
||||
'contain.text',
|
||||
'it needs a "-" between the VA parameter and the column name'
|
||||
)
|
||||
},
|
||||
'va_parmap=DROPDOWN_TAB1'
|
||||
)
|
||||
})
|
||||
|
||||
it('7 | warns when a mapping names a parameter VA is not sending', () => {
|
||||
// Parameter-side typo: the mapping never applies, so without the banner this
|
||||
// is indistinguishable from a working filter over every row.
|
||||
openEditorInVaMode(
|
||||
'mpe_x_test',
|
||||
() => {
|
||||
pushVaMessage({
|
||||
name: 'pr1',
|
||||
label: 'DROPDOWN_TAB1',
|
||||
dataType: 'string',
|
||||
value: 'Option 2'
|
||||
})
|
||||
// One message only tells us this object's parameter set is still being
|
||||
// discovered; the warning waits for a message that introduces no new name.
|
||||
cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
|
||||
'not.exist'
|
||||
)
|
||||
pushVaMessage({
|
||||
name: 'pr1',
|
||||
label: 'DROPDOWN_TAB1',
|
||||
dataType: 'string',
|
||||
value: 'Option 2'
|
||||
})
|
||||
cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
|
||||
'contain.text',
|
||||
'names VA parameter "dropdown_tab_1", which VA is not sending'
|
||||
)
|
||||
expectRows((rows) => expect(rows).to.equal(unfilteredRows))
|
||||
},
|
||||
'va_parmap=DROPDOWN_TAB_1-SOME_DROPDOWN'
|
||||
)
|
||||
})
|
||||
|
||||
it('8 | applies a mapping from the buffered message on the very first load', () => {
|
||||
// VA posts once on iframe load, before the app bootstraps, so the first
|
||||
// getdata is already filtered from that buffered message - with no column
|
||||
// metadata available, the mapping is what names the column.
|
||||
visitWithBufferedMessage(
|
||||
'va_parmap=DROPDOWN_TAB1-SOME_DROPDOWN',
|
||||
{
|
||||
name: 'pr1',
|
||||
label: 'DROPDOWN_TAB1',
|
||||
dataType: 'string',
|
||||
value: 'Option 2'
|
||||
},
|
||||
() => {
|
||||
cy.get('.infoBar-text', { timeout: longerCommandTimeout }).should(
|
||||
'contain.text',
|
||||
"SOME_DROPDOWN = 'Option 2'"
|
||||
)
|
||||
expectFewerRowsThan(unfilteredRows)
|
||||
cy.url().should('match', /\/editor\/[^/]+\/\d+/)
|
||||
}
|
||||
)
|
||||
})
|
||||
|
||||
it('9 | a bad mapping on the first load still ends at the banner', () => {
|
||||
// The deferred first load cannot check the column against metadata, so its
|
||||
// saveQuery may fail and fall back to an unfiltered load - the banner is
|
||||
// what stops that reading as a working filter.
|
||||
visitWithBufferedMessage(
|
||||
'va_parmap=DROPDOWN_TAB1-NOT_A_COLUMN',
|
||||
{
|
||||
name: 'pr1',
|
||||
label: 'DROPDOWN_TAB1',
|
||||
dataType: 'string',
|
||||
value: 'Option 2'
|
||||
},
|
||||
() => {
|
||||
cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
|
||||
'contain.text',
|
||||
'NOT_A_COLUMN'
|
||||
)
|
||||
cy.get('.va-parmap-warning').should('contain.text', 'DROPDOWN_TAB1')
|
||||
// Deliberately no row-count assertion: this path cannot validate the
|
||||
// column (no metadata yet), so the outcome depends on the platform - on
|
||||
// SAS, mp_filtercheck aborts the save and the load falls back to
|
||||
// unfiltered; the JS mock stores the filter and returns 0 rows. The
|
||||
// banner is the part that must hold either way, because both outcomes
|
||||
// are otherwise indistinguishable from a working filter.
|
||||
}
|
||||
)
|
||||
})
|
||||
|
||||
it('10 | warns when two parameters filter the same column (an empty grid)', () => {
|
||||
// Clauses are AND-ed, so two different values on one column produce no rows
|
||||
// at all - the last silent-empty-grid path. Reported rather than rewritten,
|
||||
// so the author decides whether to use one parameter carrying both values.
|
||||
openEditorInVaMode(
|
||||
'mpe_x_test',
|
||||
() => {
|
||||
pushVaMessage(
|
||||
{
|
||||
name: 'pr1',
|
||||
label: 'DROPDOWN_TAB1',
|
||||
dataType: 'string',
|
||||
value: 'Option 1'
|
||||
},
|
||||
{
|
||||
name: 'pr2',
|
||||
label: 'DROPDOWN_TAB2',
|
||||
dataType: 'string',
|
||||
value: 'Option 2'
|
||||
}
|
||||
)
|
||||
cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
|
||||
'contain.text',
|
||||
'combined with AND'
|
||||
)
|
||||
cy.get('.va-parmap-warning').should('contain.text', 'DROPDOWN_TAB1')
|
||||
cy.get('.va-parmap-warning').should('contain.text', 'DROPDOWN_TAB2')
|
||||
// The consequence: rows must match both clauses, and the two values
|
||||
// differ, so none can match.
|
||||
expectRows((rows) => expect(rows, 'matching rows').to.equal(0))
|
||||
},
|
||||
'va_parmap=DROPDOWN_TAB1-SOME_DROPDOWN&va_parmap=DROPDOWN_TAB2-SOME_DROPDOWN'
|
||||
)
|
||||
})
|
||||
|
||||
it('12 | a transform adds the previous quarter to the filter', () => {
|
||||
// ADD(QUARTER_START,-1) is the reported requirement: one clause carrying the
|
||||
// selected date and the same point one quarter earlier. 2026-10-01 = 24380,
|
||||
// 2026-07-01 = 24288 in SAS days; nothing in the fixture matches either, so
|
||||
// the assertion is on the clause rather than the row count.
|
||||
openEditorInVaMode(
|
||||
'mpe_x_test',
|
||||
() => {
|
||||
pushVaMessage({
|
||||
name: 'pr1',
|
||||
label: 'SELECTED_DATE',
|
||||
dataType: 'string',
|
||||
value: '01OCT2026'
|
||||
})
|
||||
cy.get('.infoBar-text', { timeout: longerCommandTimeout }).should(
|
||||
'contain.text',
|
||||
'SOME_DATE IN ('
|
||||
)
|
||||
cy.get('.infoBar-text').should('contain.text', '24380')
|
||||
cy.get('.infoBar-text').should('contain.text', '24288')
|
||||
cy.get('h4').should('contain.text', '(0 rows')
|
||||
},
|
||||
'va_parmap=SELECTED_DATE-SOME_DATE:ADD(QUARTER_START,-1)'
|
||||
)
|
||||
})
|
||||
|
||||
it('13 | warns when a transform lands on a column it cannot apply to', () => {
|
||||
openEditorInVaMode(
|
||||
'mpe_x_test',
|
||||
() => {
|
||||
pushVaMessage({
|
||||
name: 'pr1',
|
||||
label: 'SELECTED_DATE',
|
||||
dataType: 'string',
|
||||
value: 'x'
|
||||
})
|
||||
cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
|
||||
'contain.text',
|
||||
'is not a date or datetime column'
|
||||
)
|
||||
cy.get('h4').should('contain.text', '(7 rows')
|
||||
},
|
||||
'va_parmap=SELECTED_DATE-SOME_CHAR:ADD(QUARTER_START,-1)'
|
||||
)
|
||||
})
|
||||
|
||||
it('11 | logs a parse-time problem once, however many messages arrive', () => {
|
||||
// The parse problem is logged at app init; the editor's own loop must not log
|
||||
// it again. The second entry is valid-but-missing, so the loop definitely runs
|
||||
// and logs its own line - which is what makes this a real assertion.
|
||||
cy.visit(
|
||||
`${hostUrl}${appLocation}/index.html#/editor/DC_JSLIB.MPE_X_TEST?embed=va&va_parmap=DROPDOWN_TAB1&va_parmap=DROPDOWN_TAB2-NOT_A_COLUMN`,
|
||||
{
|
||||
onBeforeLoad(win: any) {
|
||||
win.__warns = []
|
||||
const original = win.console.warn.bind(win.console)
|
||||
win.console.warn = (...args: any[]) => {
|
||||
win.__warns.push(args.join(' '))
|
||||
original(...args)
|
||||
}
|
||||
}
|
||||
}
|
||||
)
|
||||
cy.get('.app-loading', { timeout: longerCommandTimeout }).should(
|
||||
'not.exist'
|
||||
)
|
||||
pushVaMessage({
|
||||
name: 'pr2',
|
||||
label: 'DROPDOWN_TAB2',
|
||||
dataType: 'string',
|
||||
value: 'Option 2'
|
||||
})
|
||||
cy.get('.va-parmap-warning', { timeout: longerCommandTimeout }).should(
|
||||
'contain.text',
|
||||
'NOT_A_COLUMN'
|
||||
)
|
||||
cy.window().then((win: any) => {
|
||||
const warns: string[] = win.__warns || []
|
||||
const parseLines = warns.filter((w) =>
|
||||
w.includes('va_parmap "DROPDOWN_TAB1"')
|
||||
)
|
||||
const runtimeLines = warns.filter((w) => w.includes('NOT_A_COLUMN'))
|
||||
expect(parseLines.length, warns.join(' | ')).to.eq(1)
|
||||
expect(runtimeLines.length, warns.join(' | ')).to.eq(1)
|
||||
cy.get('button').contains('Submit').should('exist')
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
/**
|
||||
* Loads the editor with a VA message already in the pre-bootstrap buffer, the
|
||||
* way VA's own postMessage arrives before Angular starts. A distinct pathname is
|
||||
* used so this is a real document load: a cy.visit that differs only in the hash
|
||||
* is a same-document navigation and would not re-bootstrap the app.
|
||||
*/
|
||||
const visitWithBufferedMessage = (
|
||||
extraParams: string,
|
||||
parameter: any,
|
||||
callback?: any
|
||||
) => {
|
||||
cy.visit(
|
||||
`${hostUrl}${appLocation}/index.html#/editor/DC_JSLIB.MPE_X_TEST?embed=va&${extraParams}`,
|
||||
{
|
||||
onBeforeLoad(win: any) {
|
||||
win.__vaLastMessage = {
|
||||
t: Date.now(),
|
||||
origin: win.location.origin,
|
||||
data: {
|
||||
version: '1',
|
||||
resultName: 'dd1',
|
||||
rowCount: 0,
|
||||
availableRowCount: 0,
|
||||
data: [],
|
||||
columns: [],
|
||||
parameters: [parameter]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
)
|
||||
cy.get('.app-loading', { timeout: longerCommandTimeout }).should('not.exist')
|
||||
if (callback) callback()
|
||||
}
|
||||
|
||||
/** Pushes one VA data-driven content message carrying the given parameters. */
|
||||
const pushVaMessage = (...parameters: any[]) => {
|
||||
cy.window().then((win) => {
|
||||
win.postMessage(
|
||||
{
|
||||
version: '1',
|
||||
resultName: 'dd1',
|
||||
rowCount: 0,
|
||||
availableRowCount: 0,
|
||||
data: [],
|
||||
columns: [],
|
||||
parameters
|
||||
},
|
||||
'*'
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
// Opens a table from the tree (which routes to #/editor/LIB.TABLE), then
|
||||
// re-visits the same route with ?embed=va so the app parses VA embed mode.
|
||||
// `extraParams` is appended to that query string (e.g. va_parmap entries).
|
||||
const openEditorInVaMode = (
|
||||
tablename: string,
|
||||
callback?: any,
|
||||
extraParams?: string
|
||||
) => {
|
||||
const openEditorInVaMode = (tablename: string, callback?: any) => {
|
||||
openTableFromTree(libraryToOpenIncludes, tablename)
|
||||
|
||||
cy.get('#hotTable', { timeout: longerCommandTimeout })
|
||||
@@ -498,13 +84,7 @@ const openEditorInVaMode = (
|
||||
.then(() => {
|
||||
cy.url().then((url) => {
|
||||
const separator = url.includes('?') ? '&' : '?'
|
||||
const extra = extraParams ? `&${extraParams}` : ''
|
||||
cy.visit(`${url}${separator}embed=va${extra}`)
|
||||
// A cy.visit that differs from the current URL only in the hash is a
|
||||
// same-document navigation, so the app is not re-bootstrapped and the
|
||||
// embed parameter (read once at app init) never takes effect. Reload to
|
||||
// bootstrap it, as the VA iframe load does.
|
||||
cy.reload()
|
||||
cy.visit(`${url}${separator}embed=va`)
|
||||
cy.get('.app-loading', { timeout: longerCommandTimeout }).should(
|
||||
'not.exist'
|
||||
)
|
||||
|
||||
@@ -14,10 +14,11 @@ context('metanav tests: ', function () {
|
||||
})
|
||||
|
||||
this.beforeEach(() => {
|
||||
// beforeAll has already logged in. Typing into the login form here failed
|
||||
// once the session existed - the form is not on screen, so Cypress reported
|
||||
// "the center of this element is hidden from view" and the whole suite was
|
||||
// skipped behind that hook.
|
||||
cy.visit(hostUrl + appLocation)
|
||||
cy.get('input.username').type(username)
|
||||
cy.get('input.password').type(password)
|
||||
cy.get('.login-group button').click()
|
||||
|
||||
visitPage('view/metadata')
|
||||
})
|
||||
|
||||
|
||||
@@ -10,7 +10,7 @@ const check = (cwd) => {
|
||||
onlyAllow:
|
||||
'AFLv2.1;Apache 2.0;Apache-2.0;Apache*;Artistic-2.0;BlueOak-1.0.0;0BSD;BSD*;BSD-2-Clause;BSD-3-Clause;CC0-1.0;CC-BY-3.0;CC-BY-4.0;ISC;MIT;MPL-2.0;ODC-By-1.0;Python-2.0;Unlicense;',
|
||||
excludePackages:
|
||||
'@cds/city@1.1.0;@handsontable/angular-wrapper@16.0.1;@handsontable/angular-wrapper@17.1.0;@handsontable/angular-wrapper@18.0.0;@handsontable/angular-wrapper@18.1.1;@handsontable/angular-wrapper@18.1.2;handsontable@^16.0.1;handsontable@16.2.0;handsontable@17.1.0;handsontable@18.0.0;handsontable@18.1.1;handsontable@18.1.2;hyperformula@2.7.1;hyperformula@3.0.0;hyperformula@3.1.0;hyperformula@3.2.0;hyperformula@3.3.0;jackspeak@3.4.3;path-scurry@1.11.1;package-json-from-dist@1.0.1;buffers@0.1.1'
|
||||
'@cds/city@1.1.0;@handsontable/angular-wrapper@16.0.1;@handsontable/angular-wrapper@17.1.0;@handsontable/angular-wrapper@18.0.0;@handsontable/angular-wrapper@18.1.1;handsontable@^16.0.1;handsontable@16.2.0;handsontable@17.1.0;handsontable@18.0.0;handsontable@18.1.1;hyperformula@2.7.1;hyperformula@3.0.0;hyperformula@3.1.0;hyperformula@3.2.0;hyperformula@3.3.0;jackspeak@3.4.3;path-scurry@1.11.1;package-json-from-dist@1.0.1;buffers@0.1.1'
|
||||
},
|
||||
(error, json) => {
|
||||
if (error) {
|
||||
|
||||
Generated
+80
-54
@@ -20,7 +20,7 @@
|
||||
"@clr/angular": "file:libraries/clr-angular-17.9.0.tgz",
|
||||
"@clr/icons": "^13.0.2",
|
||||
"@clr/ui": "file:libraries/clr-ui-17.9.0.tgz",
|
||||
"@handsontable/angular-wrapper": "18.1.2",
|
||||
"@handsontable/angular-wrapper": "18.1.1",
|
||||
"@sasjs/adapter": "5.1.2",
|
||||
"@sasjs/utils": "4.0.0",
|
||||
"@sheet/crypto": "file:libraries/sheet-crypto.tgz",
|
||||
@@ -35,7 +35,7 @@
|
||||
"d3-graphviz": "^5.0.2",
|
||||
"exceljs": "^4.4.0",
|
||||
"fs-extra": "^7.0.1",
|
||||
"handsontable": "18.1.2",
|
||||
"handsontable": "18.1.1",
|
||||
"https-browserify": "1.0.0",
|
||||
"hyperformula": "^2.5.0",
|
||||
"iconv-lite": "^0.5.0",
|
||||
@@ -969,6 +969,7 @@
|
||||
"integrity": "sha512-pQiOg+se1AU/ncMlnJ9V6xYnMQ84qI1BGWuJpbU6A99VTXJg90scg0+T7DWmKssR1YjP5qmmBtrZfKsHEcLW/A==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@angular-eslint/bundled-angular-compiler": "19.8.1",
|
||||
"eslint-scope": "^8.0.2"
|
||||
@@ -999,6 +1000,7 @@
|
||||
"integrity": "sha512-HdDfbLwpxzx49wAWz4oOUQKSkZcFNRpWEGsDEgufZ3ExwxRk1/2QPJuvZ2CbhKeEcLPrLKUorQ8XlGrNsJmNiQ==",
|
||||
"deprecated": "@angular/animations is deprecated. Use `animate.enter` and `animate.leave` instead. For more information see: https://v22.angular.dev/guide/animations.",
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"tslib": "^2.3.0"
|
||||
},
|
||||
@@ -1144,6 +1146,7 @@
|
||||
"resolved": "https://registry.npmjs.org/@angular/cdk/-/cdk-20.2.14.tgz",
|
||||
"integrity": "sha512-7bZxc01URbiPiIBWThQ69XwOxVduqEKN4PhpbF2AAyfMc/W8Hcr4VoIJOwL0O1Nkq5beS8pCAqoOeIgFyXd/kg==",
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"parse5": "^8.0.0",
|
||||
"tslib": "^2.3.0"
|
||||
@@ -1440,6 +1443,7 @@
|
||||
"integrity": "sha512-nqhDw2ZcAUrKNPwhjinJny903bRhI0rQhiDz1LksjeRxqa36i3l75+4iXbOy0rlDpLJGxqtgoPavQjmmyS5UJw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@inquirer/checkbox": "^4.2.1",
|
||||
"@inquirer/confirm": "^5.1.14",
|
||||
@@ -1582,6 +1586,7 @@
|
||||
"resolved": "https://registry.npmjs.org/@angular/common/-/common-20.3.34.tgz",
|
||||
"integrity": "sha512-OhB/l0cNldOvQeICqbWqRUzK5fF6zNYy44Q4cIGJZql8HXL/invGiqWLE9f+kU8z4jRisofHIGTBtyJQYYOudg==",
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"tslib": "^2.3.0"
|
||||
},
|
||||
@@ -1598,6 +1603,7 @@
|
||||
"resolved": "https://registry.npmjs.org/@angular/compiler/-/compiler-20.3.34.tgz",
|
||||
"integrity": "sha512-8agPW384c/JER1ersPTmJJfcCOnlYcBbdj0MYRxNdiYe+7gtnspF3ORuJmxqCEpMS17AljSmk2skwUcYnHB6xA==",
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"tslib": "^2.3.0"
|
||||
},
|
||||
@@ -1611,6 +1617,7 @@
|
||||
"integrity": "sha512-PKHzhSOyDUFh64Ull3fUwgFlxnsQNEAuRyz4KqQ8FsTGZ4pOpUaZntO+2Xn1nX/gT2whSyGBJhD3H37zA8ZGEQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@babel/core": "7.29.7",
|
||||
"@jridgewell/sourcemap-codec": "^1.4.14",
|
||||
@@ -1643,6 +1650,7 @@
|
||||
"resolved": "https://registry.npmjs.org/@angular/core/-/core-20.3.34.tgz",
|
||||
"integrity": "sha512-xOvSmGoao3SVPz//R7zTrq7+WdrIGi+WNpAdxJEvJJXuSpLGS3kSUqukjY4kOMochg2v0NCEPIdCyH+yPs8JSw==",
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"tslib": "^2.3.0"
|
||||
},
|
||||
@@ -1686,6 +1694,7 @@
|
||||
"resolved": "https://registry.npmjs.org/@angular/platform-browser/-/platform-browser-20.3.34.tgz",
|
||||
"integrity": "sha512-0bd2Bj05WPoBTFM0A13IwbzxKVQPstx/N+7X0ZWKCifliyydIc7DBHu+r37ybJpl8ZsCVBe7DoR9Gon4lZ+zlw==",
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"tslib": "^2.3.0"
|
||||
},
|
||||
@@ -1781,6 +1790,7 @@
|
||||
"integrity": "sha512-RgHBCvtjbOK2gXSNBNIkNoEc9qoVEtau3hj8gEqKQuL3HZAibKarWFEI3Lfm6EYKkLalOh8eSrj9b+ch9H/VBA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@babel/code-frame": "^7.29.7",
|
||||
"@babel/generator": "^7.29.7",
|
||||
@@ -2307,7 +2317,6 @@
|
||||
"integrity": "sha512-oBNVCvnO5tND+xSopWvV8WNGfpTfgP4Zr/YXXSj8zfmcPktp5Ku/aZlsIowgSD4fjmgHn6sGmB9APVsU5zOdhA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@babel/helper-plugin-utils": "^7.29.7",
|
||||
"@babel/helper-skip-transparent-expression-wrappers": "^7.29.7"
|
||||
@@ -3353,7 +3362,6 @@
|
||||
"integrity": "sha512-GYzX36n1nsciIb0uyH0GHwxwtNwPQIcpxSeiVLDtG/B7jB5xXgchnmL1f/jCX5o+pwnaDBtO60ONSJhEBJfxYA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@babel/compat-data": "^7.29.7",
|
||||
"@babel/helper-compilation-targets": "^7.29.7",
|
||||
@@ -3440,7 +3448,6 @@
|
||||
"integrity": "sha512-d98gXZkgswvkyohMBABkhm3GeXhYj8psWfwQ2C7gtfrKGTykQa/iOIi+JJhwMjPlZ6Vm2XN+DCf3Es1EoG4ZLA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@babel/helper-plugin-utils": "^7.29.7",
|
||||
"@babel/helper-remap-async-to-generator": "^7.29.7",
|
||||
@@ -3459,7 +3466,6 @@
|
||||
"integrity": "sha512-pcUb2SS+RMo9TWVBwKGI5ShtoG7R+zBsFmCKDa6fe8c+hPr3XJlZgoE5j6i8W7gDjhyvy+85vmYexanvXh3d1w==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@babel/helper-module-imports": "^7.29.7",
|
||||
"@babel/helper-plugin-utils": "^7.29.7",
|
||||
@@ -3478,7 +3484,6 @@
|
||||
"integrity": "sha512-coWpDLJ410R781Npmn/SIBZEsAetR4xVi0SxLMXPaMO4lSf1MwnkGYMtkFxew0Dn8B3/CpbpYxN0JCgg8mn67g==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@babel/helper-define-polyfill-provider": "^0.6.8",
|
||||
"core-js-compat": "^3.48.0"
|
||||
@@ -3493,7 +3498,6 @@
|
||||
"integrity": "sha512-BR7VvDCVHO+q2xBEWskxS6DJE1qRnb7DxzUrogb71CWoSficBxYsiAGd+Kl0mmq/MprG9yArRkyrQxTO6XjMzA==",
|
||||
"dev": true,
|
||||
"license": "ISC",
|
||||
"peer": true,
|
||||
"bin": {
|
||||
"semver": "bin/semver.js"
|
||||
}
|
||||
@@ -3600,6 +3604,7 @@
|
||||
"resolved": "https://registry.npmjs.org/@cds/core/-/core-6.17.0.tgz",
|
||||
"integrity": "sha512-afG+KwNE4BDc6CAylgGbJrJUeGYFbOIfagylaqqbS7fgIX2h3ZqFcUOv/2hujSZ9GZiuCSa+A/BkJi7K83QEgg==",
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"lit": "^2.1.3",
|
||||
"ramda": "^0.29.0",
|
||||
@@ -3781,6 +3786,7 @@
|
||||
"integrity": "sha512-TQMmc3w+5AxjpL8iIiwebF73dRDF4fBIieAqGn9RGCWaEVwQ6Fb2cGe31Yns0RRIzii5goJ1Y7xbMwo1TxMplw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"readdirp": "^5.0.0"
|
||||
},
|
||||
@@ -4898,9 +4904,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@handsontable/angular-wrapper": {
|
||||
"version": "18.1.2",
|
||||
"resolved": "https://registry.npmjs.org/@handsontable/angular-wrapper/-/angular-wrapper-18.1.2.tgz",
|
||||
"integrity": "sha512-LoUc3jIO7Y5rkSzHW+njvQWA6x8RipkWmBkR7vmnEqgxcytejgd7RIy6/b3O0AqYuv2XUT+vXrNacn9wEekWhQ==",
|
||||
"version": "18.1.1",
|
||||
"resolved": "https://registry.npmjs.org/@handsontable/angular-wrapper/-/angular-wrapper-18.1.1.tgz",
|
||||
"integrity": "sha512-paiINuWOevybN6RFUrtSOgRv5SaFlMd5dRbBOzeLyJJzqhLW7Jg+QSy6+51FAL6FKHZPfWHYoP0F2YgNPEXYbg==",
|
||||
"license": "SEE LICENSE IN LICENSE.txt",
|
||||
"optionalDependencies": {
|
||||
"tslib": "^2.3.0"
|
||||
@@ -7170,6 +7176,7 @@
|
||||
"integrity": "sha512-gLyJlPHPZYdAk1JENA9LeHejZe1Ti77/pTeFm/nMXmQH/HFZlcS/O2XJB+L8fkbrNSqhdtlvjBVjxwUYanNH5Q==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"peer": true,
|
||||
"engines": {
|
||||
"node": ">=8.0.0"
|
||||
}
|
||||
@@ -7193,6 +7200,7 @@
|
||||
"integrity": "sha512-s5vvxXPVdjqS3kTLKMeBMvop9hbWkwzBpu+mUO2M7sZtlkyDJGwFe33wRKnbaYDo8ExRVBIIdwIGrqpxHuKttA==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"peer": true,
|
||||
"engines": {
|
||||
"node": ">=14"
|
||||
},
|
||||
@@ -7206,6 +7214,7 @@
|
||||
"integrity": "sha512-OOCM2C/QIURhJMuKaekP3TRBxBKxG/TWWA0TL2J6nXUtDnuCtccy49LUJF8xPFXMX+0LMcxFpCo8M9cGY1W6rQ==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@opentelemetry/semantic-conventions": "1.28.0"
|
||||
},
|
||||
@@ -7232,6 +7241,7 @@
|
||||
"integrity": "sha512-BdBGhQBh8IjZ2oIIX6F2/Q3LKm/FDDKi6ccYKcBTeilh6SNdNKveDOLk73BkSJjQLJk6qe4Yh+hHw1UPhCDdrg==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@opentelemetry/api-logs": "0.57.2",
|
||||
"@types/shimmer": "^1.2.0",
|
||||
@@ -7662,6 +7672,7 @@
|
||||
"integrity": "sha512-5UxZqiAgLYGFjS4s9qm5mBVo433u+dSPUFWVWXmLAD4wB65oMCoXaJP1KJa9DIYYMeHu3z4BZcStG3LC593cWA==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@opentelemetry/core": "1.30.1",
|
||||
"@opentelemetry/semantic-conventions": "1.28.0"
|
||||
@@ -7689,6 +7700,7 @@
|
||||
"integrity": "sha512-jVPgBbH1gCy2Lb7X0AVQ8XAfgg0pJ4nvl8/IiQA6nxOsPvS+0zMJaFSs2ltXe0J6C8dqjcnpyqINDJmU30+uOg==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@opentelemetry/core": "1.30.1",
|
||||
"@opentelemetry/resources": "1.30.1",
|
||||
@@ -7717,6 +7729,7 @@
|
||||
"integrity": "sha512-eSYWTm620tTk45EKSedaUL8MFYI8hW164hIXsgIHyxu3VobUB3fFCu5t0hQby6OoWRPsG1KkKUG2M5UadiLiVg==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"peer": true,
|
||||
"engines": {
|
||||
"node": ">=14"
|
||||
}
|
||||
@@ -9364,8 +9377,7 @@
|
||||
"resolved": "https://registry.npmjs.org/@types/hammerjs/-/hammerjs-2.0.46.tgz",
|
||||
"integrity": "sha512-ynRvcq6wvqexJ9brDMS4BnBLzmr0e14d6ZJTEShTBWKymQiHwlAyGu0ZPEFI2Fh1U53F7tN9ufClWM5KvqkKOw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@types/http-errors": {
|
||||
"version": "2.0.5",
|
||||
@@ -9709,6 +9721,7 @@
|
||||
"integrity": "sha512-CZ4nMxWwgu1HEEFNkeaCptra9QCtkmKdgf3sWh1rl1trIhmxLilgTV4cwcbQ4wemnT4sWQN8CaKOmdYx+g2gMA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@typescript-eslint/scope-manager": "8.65.0",
|
||||
"@typescript-eslint/types": "8.65.0",
|
||||
@@ -9882,6 +9895,7 @@
|
||||
"integrity": "sha512-Adfoy59kxNEjT5E5fnyrp7GpHYuWs8ONbdohu4ez6IU46qDq8LltG90tLmkucZB2uhYL8mqBjtXqGd8yhfjXqQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"engines": {
|
||||
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
|
||||
},
|
||||
@@ -9976,7 +9990,6 @@
|
||||
"integrity": "sha512-bzHngBbmIgzj6TsTjVUCntGx6FFLgE7mPjpO0PzHycOaZHXqS43XnrkFIHRS16LLr/xO6FHSXG3GJFo/F4bSBQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@typescript-eslint/tsconfig-utils": "^8.72.0",
|
||||
"@typescript-eslint/types": "^8.72.0",
|
||||
@@ -9999,7 +10012,6 @@
|
||||
"integrity": "sha512-JewPyJOYcXBSAS7PIgufscHOwZqhTGyBRiQJhhuDln66tMchxvvt7nJ90JKKIoOelmdHA3kAQbm9cEUHc4JSEA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@typescript-eslint/types": "8.72.0",
|
||||
"@typescript-eslint/visitor-keys": "8.72.0"
|
||||
@@ -10018,7 +10030,6 @@
|
||||
"integrity": "sha512-0vbxCf7xLVPSjcOdm6eQbgViH8EDJ8xnl1laLT9yoqSz2YmcSfSg5Vdl0JUvZf6rcuJ0ZivJIuMD4edTg8obaA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"engines": {
|
||||
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
|
||||
},
|
||||
@@ -10036,7 +10047,6 @@
|
||||
"integrity": "sha512-gwLfFgWcw15tN2KXDCJvxEOQjPn+P3j/n3eSqkZ5sX4VpW9Cosodr+tWwBZRvJT3TcKx6HnPRGZMCNAiP1Xhug==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@typescript-eslint/project-service": "8.72.0",
|
||||
"@typescript-eslint/tsconfig-utils": "8.72.0",
|
||||
@@ -10065,7 +10075,6 @@
|
||||
"integrity": "sha512-E2mroviasaWecDzTGiGMtuEguuLVOeRggnrIn93CJuxYJzQBHbyymlT08aVQydbBal5538o+JkBKXh2sRuCyIA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@typescript-eslint/types": "8.72.0",
|
||||
"eslint-visitor-keys": "^5.0.0"
|
||||
@@ -10084,7 +10093,6 @@
|
||||
"integrity": "sha512-tD40eHxA35h0PEIZNeIjkHoDR4YjjJp34biM0mDvplBe//mB+IHCqHDGV7pxF+7MklTvighcCPPZC7ynWyjdTA==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"peer": true,
|
||||
"engines": {
|
||||
"node": "^20.19.0 || ^22.13.0 || >=24"
|
||||
},
|
||||
@@ -10098,7 +10106,6 @@
|
||||
"integrity": "sha512-Y7/KDsb8LjooZpwaqGyulO6DQlksgCncchHGk+sZIY4SBvUocMBEFH5Ur1fI4dV+Jvl0w6cjvucaIi40puRioA==",
|
||||
"dev": true,
|
||||
"license": "ISC",
|
||||
"peer": true,
|
||||
"bin": {
|
||||
"semver": "bin/semver.js"
|
||||
},
|
||||
@@ -10391,6 +10398,7 @@
|
||||
"resolved": "https://registry.npmjs.org/acorn/-/acorn-8.18.0.tgz",
|
||||
"integrity": "sha512-lGq+9yr1/GuAWaVYIHRjvvySG5/4VfKIvC8EWxStPdcDh/Ka7FG3twP6v4d5BkravUilhIAsG4Qj83t02LWUPQ==",
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"bin": {
|
||||
"acorn": "bin/acorn"
|
||||
},
|
||||
@@ -10461,6 +10469,7 @@
|
||||
"resolved": "https://registry.npmjs.org/ajv/-/ajv-8.18.0.tgz",
|
||||
"integrity": "sha512-PlXPeEWMXMZ7sPYOHqmDyCJzcfNrUr3fGNKtezX14ykXOEIvyK81d+qydx89KY5O71FKMPaQ2vBfBFI5NHR63A==",
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"fast-deep-equal": "^3.1.3",
|
||||
"fast-uri": "^3.0.1",
|
||||
@@ -10924,6 +10933,7 @@
|
||||
"resolved": "https://registry.npmjs.org/axios/-/axios-1.20.0.tgz",
|
||||
"integrity": "sha512-r8aOh8j9cGKpgQAqpzrUHnSIc6a59Y3Xf/cv8sy1DrHCkZHzQGEuoq1tARk6qSyDdtQGSDgpb9kFlruzPvrgwg==",
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"follow-redirects": "^1.16.0",
|
||||
"form-data": "^4.0.6",
|
||||
@@ -11059,6 +11069,7 @@
|
||||
"resolved": "https://registry.npmjs.org/bare-events/-/bare-events-2.9.2.tgz",
|
||||
"integrity": "sha512-AIPKioV7/Y/8KfZ3AAhjPJxLLbY49S64Ym5DakZlUg75qQiTgUq9hEJoEwa4eUezPUlXRy/i5NpsKvo9jgKmoA==",
|
||||
"license": "Apache-2.0",
|
||||
"peer": true,
|
||||
"peerDependencies": {
|
||||
"bare-abort-controller": "*"
|
||||
},
|
||||
@@ -11430,8 +11441,7 @@
|
||||
"resolved": "https://registry.npmjs.org/browser-stdout/-/browser-stdout-1.3.1.tgz",
|
||||
"integrity": "sha512-qhAVI1+Av2X7qelOfAIYwXONood6XlZE/fXaBSmW/T5SzLAmCgzi+eiWE7fUvbHaeNBQH13UftjpXxsfLkMpgw==",
|
||||
"dev": true,
|
||||
"license": "ISC",
|
||||
"peer": true
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/browserify-aes": {
|
||||
"version": "1.2.0",
|
||||
@@ -11489,6 +11499,7 @@
|
||||
}
|
||||
],
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"baseline-browser-mapping": "^2.11.12",
|
||||
"caniuse-lite": "^1.0.30001809",
|
||||
@@ -11862,6 +11873,7 @@
|
||||
"integrity": "sha512-Qgzu8kfBvo+cA4962jnP1KkS6Dop5NS6g7R5LFYJr4b8Ub94PPQXUksCw9PvXoeXPRRddRNC5C1JQUR2SMGtnA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"readdirp": "^4.0.1"
|
||||
},
|
||||
@@ -12293,8 +12305,7 @@
|
||||
"resolved": "https://registry.npmjs.org/common-path-prefix/-/common-path-prefix-3.0.0.tgz",
|
||||
"integrity": "sha512-QE33hToZseCH3jS0qN96O/bSh3kaw/h+Tq7ngyY9eWDUnTlTNUyqfqvCXioLe5Na5jFsL78ra/wuBU4iuEgd4w==",
|
||||
"dev": true,
|
||||
"license": "ISC",
|
||||
"peer": true
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/common-tags": {
|
||||
"version": "1.8.2",
|
||||
@@ -12975,6 +12986,7 @@
|
||||
"dev": true,
|
||||
"hasInstallScript": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@cypress/request": "^4.0.0",
|
||||
"@cypress/xvfb": "^1.2.4",
|
||||
@@ -13376,6 +13388,7 @@
|
||||
"resolved": "https://registry.npmjs.org/d3-selection/-/d3-selection-3.0.0.tgz",
|
||||
"integrity": "sha512-fmTRWbNMmsmWq6xJV8D19U/gw/bwrHfNXxrIN+HfZgnzqTHp9jOmKMhsTUjXOJnZOdZY9Q28y4yebKzqDKlxlQ==",
|
||||
"license": "ISC",
|
||||
"peer": true,
|
||||
"engines": {
|
||||
"node": ">=12"
|
||||
}
|
||||
@@ -13651,7 +13664,8 @@
|
||||
"resolved": "https://registry.npmjs.org/devtools-protocol/-/devtools-protocol-0.0.1625959.tgz",
|
||||
"integrity": "sha512-wRBSU330hwOLLcb3N4NIe3eFs6MgT6ku3AiZONjnTSJ7f3dVchJfn6nE0Lfos9jK1na15bgp7xLhaCx40Y47NQ==",
|
||||
"dev": true,
|
||||
"license": "BSD-3-Clause"
|
||||
"license": "BSD-3-Clause",
|
||||
"peer": true
|
||||
},
|
||||
"node_modules/dezalgo": {
|
||||
"version": "1.0.4",
|
||||
@@ -13677,7 +13691,6 @@
|
||||
"integrity": "sha512-svtcdpS8CgJyqAjEQIXdb3OjhFVVYjzGAPO8WGCmRbrml64SPw/jJD4GoE98aR7r25A0XcgrK3F02yw9R/vhQw==",
|
||||
"dev": true,
|
||||
"license": "BSD-3-Clause",
|
||||
"peer": true,
|
||||
"engines": {
|
||||
"node": ">=0.3.1"
|
||||
}
|
||||
@@ -14298,6 +14311,7 @@
|
||||
"deprecated": "This version is no longer supported. Please see https://eslint.org/version-support for other options.",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@eslint-community/eslint-utils": "^4.2.0",
|
||||
"@eslint-community/regexpp": "^4.6.1",
|
||||
@@ -14743,6 +14757,7 @@
|
||||
"integrity": "sha512-Bdcs4+3qlpVlx2NRn6fgX2Ue2/gGRaPeawebgclM0ERSCqDpA+owF1fdPwjJUTAJWMTuAaxjDf+hzb0/4eKvvw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"accepts": "~1.3.8",
|
||||
"array-flatten": "1.1.1",
|
||||
@@ -15170,7 +15185,6 @@
|
||||
"integrity": "sha512-9ZonPT4ZAK4a+1pUPVPZJapbi7O5qbbJPdYw/NOQWZZbVLdDTYM3A4R9z/DpAM08IDaFGsvPgiGZ82WEwUDWjg==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"common-path-prefix": "^3.0.0",
|
||||
"pkg-dir": "^7.0.0"
|
||||
@@ -15205,7 +15219,6 @@
|
||||
"integrity": "sha512-afd4O7zpqHeRyg4PfDQsXmlDe2PfdHtJt6Akt8jOWaApLOZk5JXs6VMR29lz03pRe9mpykrRCYIYxaJYcfpncQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
},
|
||||
@@ -15731,10 +15744,11 @@
|
||||
}
|
||||
},
|
||||
"node_modules/handsontable": {
|
||||
"version": "18.1.2",
|
||||
"resolved": "https://registry.npmjs.org/handsontable/-/handsontable-18.1.2.tgz",
|
||||
"integrity": "sha512-iL4Cia0Fsl+md1oCqUS8PJqfft3583d5ptc5M7ewfp6JTUsNsrIcpF3GPYvZy57e7P5PACX8WboGlkwgHn0h9Q==",
|
||||
"license": "SEE LICENSE IN LICENSE.txt"
|
||||
"version": "18.1.1",
|
||||
"resolved": "https://registry.npmjs.org/handsontable/-/handsontable-18.1.1.tgz",
|
||||
"integrity": "sha512-8vz/jp96m0aJQPRepOBoImsC/bLe9mE/O4nOB6mll9DCjN5nqXHX9LV4zu/XDpD89TKJwtA+TNhVUkFQxW+q7w==",
|
||||
"license": "SEE LICENSE IN LICENSE.txt",
|
||||
"peer": true
|
||||
},
|
||||
"node_modules/has-flag": {
|
||||
"version": "4.0.0",
|
||||
@@ -15912,6 +15926,7 @@
|
||||
"integrity": "sha512-6E2QDAc9Ick9Sq77ZrGS/dk2WUYni91aufTw6LJKpV7w8kW5/GxVUc650FOADOmlwg3K+f7Pun6XlV+pYmW6gw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"engines": {
|
||||
"node": ">=16.9.0"
|
||||
}
|
||||
@@ -17109,7 +17124,8 @@
|
||||
"resolved": "https://registry.npmjs.org/jasmine-core/-/jasmine-core-5.1.2.tgz",
|
||||
"integrity": "sha512-2oIUMGn00FdUiqz6epiiJr7xcFyNYj3rDcfmnzfkBnHyBQ3cBQUs4mmyGsOb7TTLb9kxk7dBcmEmqhDKkBoDyA==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
"license": "MIT",
|
||||
"peer": true
|
||||
},
|
||||
"node_modules/jest-worker": {
|
||||
"version": "27.5.1",
|
||||
@@ -17146,6 +17162,7 @@
|
||||
"integrity": "sha512-/imKNG4EbWNrVjoNC/1H5/9GFy+tqjGBHCaSsN+P2RnPqjsLmv6UD3Ej+Kj8nBWaRAwyk7kK5ZUc+OEatnTR3A==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"bin": {
|
||||
"jiti": "bin/jiti.js"
|
||||
}
|
||||
@@ -17432,6 +17449,7 @@
|
||||
"integrity": "sha512-LrtUxbdvt1gOpo3gxG+VAJlJAEMhbWlM4YrFQgql98FwF7+K8K12LYO4hnDdUkNjeztYrOXEMqgTajSWgmtI/w==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@colors/colors": "1.5.0",
|
||||
"body-parser": "^1.19.0",
|
||||
@@ -17987,6 +18005,7 @@
|
||||
"integrity": "sha512-umRhrCH7fCi8Uj2RcwKjJdvUORTjeWqkdKx0LbcZvjIwsAVsnIAGcxHaqowPeBFBjQuWOeC/bve0AlpFzF/+SQ==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"copy-anything": "^3.0.5",
|
||||
"parse-node-version": "^1.0.1"
|
||||
@@ -18448,6 +18467,7 @@
|
||||
"integrity": "sha512-SL0JY3DaxylDuo/MecFeiC+7pedM0zia33zl0vcjgwcq1q1FWWF1To9EIauPbl8GbMCU0R2e0uJ8bZunhYKD2g==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"cli-truncate": "^4.0.0",
|
||||
"colorette": "^2.0.20",
|
||||
@@ -19574,7 +19594,6 @@
|
||||
"integrity": "sha512-beammieDdiWdkjANWeWPpkxUROYGi5FN88fPOkLiticz7HTg3Y+Pho3EYE6SUdcuWnZ3h2WD8EXCA55JF4o6wg==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"browser-stdout": "^1.3.1",
|
||||
"chokidar": "^5.0.0",
|
||||
@@ -19615,8 +19634,7 @@
|
||||
"url": "https://github.com/sponsors/nodeca"
|
||||
}
|
||||
],
|
||||
"license": "PSF-2.0",
|
||||
"peer": true
|
||||
"license": "PSF-2.0"
|
||||
},
|
||||
"node_modules/mocha/node_modules/chokidar": {
|
||||
"version": "5.0.0",
|
||||
@@ -19624,7 +19642,6 @@
|
||||
"integrity": "sha512-TQMmc3w+5AxjpL8iIiwebF73dRDF4fBIieAqGn9RGCWaEVwQ6Fb2cGe31Yns0RRIzii5goJ1Y7xbMwo1TxMplw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"readdirp": "^5.0.0"
|
||||
},
|
||||
@@ -19641,7 +19658,6 @@
|
||||
"integrity": "sha512-Wjlyrolmm8uDpm/ogGyXZXb1Z+Ca2B8NbJwqBVg0axK9GbBeoS7yGV6vjXnYdGm6X53iehEuxxbyiKp8QmN4Vw==",
|
||||
"dev": true,
|
||||
"license": "BlueOak-1.0.0",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"minimatch": "^10.2.2",
|
||||
"minipass": "^7.1.3",
|
||||
@@ -19660,7 +19676,6 @@
|
||||
"integrity": "sha512-lJJV/5dYS+RcL8uQdBDW9c9uWFLLBNRyFhnAKXw5tVqLlKZ4RMGZKv+YQ/IA3OhD+RpbJa1LLFM1FQPGyIXvOA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"engines": {
|
||||
"node": ">=12"
|
||||
},
|
||||
@@ -19684,7 +19699,6 @@
|
||||
}
|
||||
],
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"argparse": "^3.0.3"
|
||||
},
|
||||
@@ -19698,7 +19712,6 @@
|
||||
"integrity": "sha512-Kko+Y5XQ6fM+Ce3dq3m9YGxnacYZYl9cA1wZjaF3Vbry2L3i1qVg8+CAgNPsXRArPMUMCaOR7oa9Nqntc43JKA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"engines": {
|
||||
"node": ">= 20.19.0"
|
||||
},
|
||||
@@ -19713,7 +19726,6 @@
|
||||
"integrity": "sha512-1tB5mhVo7U+ETBKNf92xT4hrQa3pm0MZ0PQvuDnWgAAGHDsfp4lPSpiS6psrSiet87wyGPh9ft6wmhOMQ0hDiw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"engines": {
|
||||
"node": ">=14.16"
|
||||
},
|
||||
@@ -19727,7 +19739,6 @@
|
||||
"integrity": "sha512-MpUEN2OodtUzxvKQl72cUF7RQ5EiHsGvSsVG0ia9c5RbWGL2CI4C7EpPS8UTBIplnlzZiNuV56w+FuNxy3ty2Q==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"has-flag": "^4.0.0"
|
||||
},
|
||||
@@ -21435,7 +21446,6 @@
|
||||
"integrity": "sha512-Ie9z/WINcxxLp27BKOCHGde4ITq9UklYKDzVo1nhk5sqGEXU3FpkwP5GM2voTGJkGd9B3Otl+Q4uwSOeSUtOBA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"find-up": "^6.3.0"
|
||||
},
|
||||
@@ -21452,7 +21462,6 @@
|
||||
"integrity": "sha512-v2ZsoEuVHYy8ZIlYqwPe/39Cy+cFDzp4dXPaxNvkEuouymu+2Jbz0PxpKarJHYJTmv2HWT3O382qY8l4jMWthw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"locate-path": "^7.1.0",
|
||||
"path-exists": "^5.0.0"
|
||||
@@ -21470,7 +21479,6 @@
|
||||
"integrity": "sha512-gvVijfZvn7R+2qyPX8mAuKcFGDf6Nc61GdvGafQsHL0sBIxfKzA+usWn4GFC/bk+QdwPUD4kWFJLhElipq+0VA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"p-locate": "^6.0.0"
|
||||
},
|
||||
@@ -21487,7 +21495,6 @@
|
||||
"integrity": "sha512-5b0R4txpzjPWVw/cXXUResoD4hb6U/x9BH08L7nw+GN1sezDzPdxeRvpc9c433fZhBan/wusjbCsqwqm4EIBIQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"yocto-queue": "^1.0.0"
|
||||
},
|
||||
@@ -21504,7 +21511,6 @@
|
||||
"integrity": "sha512-wPrq66Llhl7/4AGC6I+cqxT07LhXvWL08LNXz1fENOw0Ap4sRZZ/gZpTTJ5jpurzzzfS2W/Ge9BY3LgLjCShcw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"p-limit": "^4.0.0"
|
||||
},
|
||||
@@ -21521,7 +21527,6 @@
|
||||
"integrity": "sha512-RjhtfwJOxzcFmNOi6ltcbcu4Iu+FL3zEj83dk4kAS+fVpTxXLO1b38RvJgT/0QwvV/L3aY9TAnyv0EOqW4GoMQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"engines": {
|
||||
"node": "^12.20.0 || ^14.13.1 || >=16.0.0"
|
||||
}
|
||||
@@ -21532,7 +21537,6 @@
|
||||
"integrity": "sha512-4LCcse/U2MHZ63HAJVE+v71o7yOdIe4cZ70Wpf8D/IyjDKYQLV5GD46B+hSTjJsvV5PztjvHoU580EftxjDZFQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"engines": {
|
||||
"node": ">=12.20"
|
||||
},
|
||||
@@ -21605,6 +21609,7 @@
|
||||
}
|
||||
],
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"nanoid": "^3.3.16",
|
||||
"picocolors": "^1.1.1",
|
||||
@@ -22840,6 +22845,7 @@
|
||||
"resolved": "https://registry.npmjs.org/rxjs/-/rxjs-7.8.2.tgz",
|
||||
"integrity": "sha512-dhKf903U/PQZY6boNNtAGdWbG85WAbjT/1xYoZIC7FAY0yWapOBQVsVrDl58W86//e1VpMNBtRV4MaXfdMySFA==",
|
||||
"license": "Apache-2.0",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"tslib": "^2.1.0"
|
||||
}
|
||||
@@ -22894,6 +22900,7 @@
|
||||
"integrity": "sha512-9GUyuksjw70uNpb1MTYWsH9MQHOHY6kwfnkafC24+7aOMZn9+rVMBxRbLvw756mrBFbIsFg6Xw9IkR2Fnn3k+Q==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"chokidar": "^4.0.0",
|
||||
"immutable": "^5.0.2",
|
||||
@@ -24202,6 +24209,7 @@
|
||||
"integrity": "sha512-+6erLbBm0+LROX2sPXlUYx/ux5PyE9K/a92Wrt6oA+WDAoFTdpHE5tCYCI5PNzq2y8df4rA+QgHLJuR4jNymsg==",
|
||||
"dev": true,
|
||||
"license": "BSD-2-Clause",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@jridgewell/source-map": "^0.3.3",
|
||||
"acorn": "^8.14.0",
|
||||
@@ -24430,6 +24438,7 @@
|
||||
"resolved": "https://registry.npmjs.org/tough-cookie/-/tough-cookie-5.1.2.tgz",
|
||||
"integrity": "sha512-FVDYdxtnj0G6Qm/DhNPSb8Ju59ULcup3tuJxkFb5K8Bv2pUXILbf0xZWU8PX8Ov19OXljbUyveOFwRMwkXzO+A==",
|
||||
"license": "BSD-3-Clause",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"tldts": "^6.1.32"
|
||||
},
|
||||
@@ -24692,7 +24701,8 @@
|
||||
"version": "2.8.1",
|
||||
"resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz",
|
||||
"integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==",
|
||||
"license": "0BSD"
|
||||
"license": "0BSD",
|
||||
"peer": true
|
||||
},
|
||||
"node_modules/tsyringe": {
|
||||
"version": "4.10.0",
|
||||
@@ -24859,6 +24869,7 @@
|
||||
"integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"peer": true,
|
||||
"bin": {
|
||||
"tsc": "bin/tsc",
|
||||
"tsserver": "bin/tsserver"
|
||||
@@ -24914,10 +24925,19 @@
|
||||
"integrity": "sha512-EvRlUt/QqAEvXdQeRMYUfwOQsfzpDztykrG2owGEE/rBOBoYv9uF39ZzsNDh4lTFHhTr2XOZjKkYgueD12ZmGA==",
|
||||
"devOptional": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"engines": {
|
||||
"node": ">=18.17"
|
||||
}
|
||||
},
|
||||
"node_modules/undici-types": {
|
||||
"version": "8.9.0",
|
||||
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-8.9.0.tgz",
|
||||
"integrity": "sha512-KTDyRTYX8sWmKXAikPHHSyc63CRPETMctyjKFupcC6OBLXT3xsN0e9aF7m+mIXutFWpUXuedtowG7iLOzp0kQg==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"optional": true
|
||||
},
|
||||
"node_modules/unicode-canonical-property-names-ecmascript": {
|
||||
"version": "2.0.1",
|
||||
"resolved": "https://registry.npmjs.org/unicode-canonical-property-names-ecmascript/-/unicode-canonical-property-names-ecmascript-2.0.1.tgz",
|
||||
@@ -25107,6 +25127,7 @@
|
||||
"https://github.com/sponsors/ctavan"
|
||||
],
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"bin": {
|
||||
"uuid": "dist/esm/bin/uuid"
|
||||
}
|
||||
@@ -25230,6 +25251,7 @@
|
||||
"integrity": "sha512-AHEzTORMbi7/7aXp7w7KZ5+DCUdSeLsJz0grY8xHpc9DHdYb0vQu6DqJfm2tOw3gDExNgPgnJTXxM6syYFHaCA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"esbuild": "^0.27.0 || ^0.28.0",
|
||||
"fdir": "^6.5.0",
|
||||
@@ -25420,6 +25442,7 @@
|
||||
"resolved": "https://registry.npmjs.org/webpack/-/webpack-5.112.0.tgz",
|
||||
"integrity": "sha512-NIBpsRuCll036haDDt/CfcTDfsWNR9Z782nMRMI9D377BsGcYzZDTNU5TyzhCJzPIcihAxMBaMAhq2WSaDrH8Q==",
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@types/estree": "^1.0.8",
|
||||
"@types/json-schema": "^7.0.15",
|
||||
@@ -25491,6 +25514,7 @@
|
||||
"integrity": "sha512-HNLRmamRvVavZQ+avceZifmv8hmdUjg43t6MI4SqJDwFdW7RPQwH5vzGhDRZSX59SgfbeHhLnq3g+uooWo7pVw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"@types/bonjour": "^3.5.13",
|
||||
"@types/connect-history-api-fallback": "^1.5.4",
|
||||
@@ -25882,8 +25906,7 @@
|
||||
"resolved": "https://registry.npmjs.org/workerpool/-/workerpool-10.0.3.tgz",
|
||||
"integrity": "sha512-6z2Iis68Wqth93/G/wJP9u+R3O+d2XTlgWChGCwuT1qLbBsOYueGRZuJ++v3mtDP5KjYdy+WzvWC+VWETSVXJA==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"peer": true
|
||||
"license": "Apache-2.0"
|
||||
},
|
||||
"node_modules/wrap-ansi": {
|
||||
"version": "9.0.2",
|
||||
@@ -26174,6 +26197,7 @@
|
||||
"integrity": "sha512-jIH9yLR9wqr0wOS0TpBvo/g/2UgZH5qePVbjgRliiF0BYvOZyaBknKsF+x9Iht0O6sqgnB93rCICdOZFecJuDw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"dependencies": {
|
||||
"pend": "~1.2.0"
|
||||
},
|
||||
@@ -26290,6 +26314,7 @@
|
||||
"integrity": "sha512-AvvthqfqrAhNH9dnfmrfKzX5upOdjUVJYFqNSlkmGf64gRaTzlPwz99IHYnVs28qYAybvAlBV+H7pn0saFY4Ig==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"peer": true,
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/colinhacks"
|
||||
}
|
||||
@@ -26308,7 +26333,8 @@
|
||||
"version": "0.15.1",
|
||||
"resolved": "https://registry.npmjs.org/zone.js/-/zone.js-0.15.1.tgz",
|
||||
"integrity": "sha512-XE96n56IQpJM7NAoXswY3XRLcWFW83xe0BiAOeMD7K5k5xecOeul3Qcpx6GqEeeHNkW5DWL5zOyTbEfB4eti8w==",
|
||||
"license": "MIT"
|
||||
"license": "MIT",
|
||||
"peer": true
|
||||
}
|
||||
}
|
||||
}
|
||||
+3
-5
@@ -37,9 +37,7 @@
|
||||
"compodoc:build-and-serve": "compodoc -p tsconfig.doc.json -s --name 'Data Controller Client'",
|
||||
"compodoc:serve": "compodoc -s --name 'Data Controller Client'",
|
||||
"lighthouse": "lhci autorun",
|
||||
"ng": "ng",
|
||||
"test:tz-west": "TZ=America/New_York npx ng test --no-watch --no-progress --browsers ChromeHeadlessCI --include='**/va-*.spec.ts'",
|
||||
"test:tz-east": "TZ=Pacific/Auckland npx ng test --no-watch --no-progress --browsers ChromeHeadlessCI --include='**/va-*.spec.ts'"
|
||||
"ng": "ng"
|
||||
},
|
||||
"private": true,
|
||||
"dependencies": {
|
||||
@@ -56,7 +54,7 @@
|
||||
"@clr/angular": "file:libraries/clr-angular-17.9.0.tgz",
|
||||
"@clr/icons": "^13.0.2",
|
||||
"@clr/ui": "file:libraries/clr-ui-17.9.0.tgz",
|
||||
"@handsontable/angular-wrapper": "18.1.2",
|
||||
"@handsontable/angular-wrapper": "18.1.1",
|
||||
"@sasjs/adapter": "5.1.2",
|
||||
"@sasjs/utils": "4.0.0",
|
||||
"@sheet/crypto": "file:libraries/sheet-crypto.tgz",
|
||||
@@ -71,7 +69,7 @@
|
||||
"d3-graphviz": "^5.0.2",
|
||||
"exceljs": "^4.4.0",
|
||||
"fs-extra": "^7.0.1",
|
||||
"handsontable": "18.1.2",
|
||||
"handsontable": "18.1.1",
|
||||
"https-browserify": "1.0.0",
|
||||
"hyperformula": "^2.5.0",
|
||||
"iconv-lite": "^0.5.0",
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import { QueryClause } from './models/TableData'
|
||||
import { VaParmapTarget } from './shared/utils/parse-va-parmap'
|
||||
|
||||
/**
|
||||
* Filtering cache info, to be reused when filtering modal is re-open
|
||||
@@ -62,21 +61,6 @@ export const globals: {
|
||||
* Defaults to 'live'; toggled by the Auto-apply checkbox in the editor.
|
||||
*/
|
||||
vaApplyMode: 'live' | 'confirm'
|
||||
/**
|
||||
* Explicit VA parameter -> DC column mappings, parsed from the `va_parmap`
|
||||
* URL parameter (see parseVaParmap). Empty unless configured; entries override
|
||||
* VA's default same-name (parameter label -> column) resolution, so two DC
|
||||
* instances over one table can filter the same column from differently-named
|
||||
* VA parameters.
|
||||
*/
|
||||
vaParamMap: Map<string, VaParmapTarget>
|
||||
/**
|
||||
* `va_parmap` entries that could not be used at all (not a plain SAS column
|
||||
* name, or missing a side of the `-`). Surfaced to the report author, because
|
||||
* these are hand-written in the report's URL and a typo otherwise looks like
|
||||
* a working, unfiltered view.
|
||||
*/
|
||||
vaParmapProblems: string[]
|
||||
rootParam: string
|
||||
dcLib: string
|
||||
xlmaps: XLMapListItem[]
|
||||
@@ -93,8 +77,6 @@ export const globals: {
|
||||
} = {
|
||||
embed: false,
|
||||
vaApplyMode: 'live',
|
||||
vaParamMap: new Map(),
|
||||
vaParmapProblems: [],
|
||||
rootParam: <string>'',
|
||||
dcLib: '',
|
||||
xlmaps: [],
|
||||
|
||||
@@ -12,7 +12,6 @@ import '@clr/icons'
|
||||
import '@clr/icons/shapes/all-shapes'
|
||||
import { globals } from './_globals'
|
||||
import { parseEmbedParam } from './shared/utils/parse-embed-param'
|
||||
import { parseVaParmap } from './shared/utils/parse-va-parmap'
|
||||
import moment from 'moment'
|
||||
import { EventService } from './services/event.service'
|
||||
import { AppService } from './services/app.service'
|
||||
@@ -164,21 +163,6 @@ export class AppComponent {
|
||||
globals.embed = embed
|
||||
this.embed = embed
|
||||
}
|
||||
// VA parameter -> DC column mappings for this instance, held on globals
|
||||
// (like `embed` and `vaApplyMode`) so in-app navigation that rebuilds the
|
||||
// editor keeps them - e.g. returning from the stage / approve pages.
|
||||
if (globals.embed === 'va') {
|
||||
const parmap = parseVaParmap(hashQuery ?? '')
|
||||
globals.vaParamMap = parmap.map
|
||||
globals.vaParmapProblems = parmap.problems
|
||||
// Visible in the browser console as well as the editor banner: the entry
|
||||
// is hand-written in the report's URL, so a typo has to be findable when
|
||||
// someone is looking at the report rather than at DC's UI.
|
||||
for (const problem of parmap.problems) {
|
||||
// eslint-disable-next-line no-console
|
||||
console.warn('Data Controller: ' + problem)
|
||||
}
|
||||
}
|
||||
|
||||
this.subscribeToShowAbortModal()
|
||||
this.subscribeToRequestsModal()
|
||||
@@ -330,12 +314,9 @@ export class AppComponent {
|
||||
*/
|
||||
public subscribeToShowAbortModal() {
|
||||
this.eventService.onShowAbortModal.subscribe((sasjsAbort: InfoModal) => {
|
||||
/* One message at a time. Pushing to the array stacked a second modal on
|
||||
top of the one already open - two dismissals for one failure, and the
|
||||
lower one was often a duplicate of the upper. Replacing keeps the
|
||||
newest message, which is the one the user has to act on. */
|
||||
sasjsAbort.id = Date.now()
|
||||
this.sasjsAborts = [sasjsAbort]
|
||||
let abortId = this.sasjsAborts.length + 1
|
||||
sasjsAbort.id = abortId
|
||||
this.sasjsAborts.push(sasjsAbort)
|
||||
this.cdr.detectChanges() //Changes were not triggered while hot is focused
|
||||
})
|
||||
}
|
||||
|
||||
@@ -1,36 +1,7 @@
|
||||
<main class="content-area position-relative">
|
||||
<!-- An embedded instance (a report object) has no report author who can deploy:
|
||||
showing the wizard there only produces a screen whose buttons cannot work,
|
||||
because the deployment needs an authenticated, non-embedded session. Name
|
||||
the state instead, and keep the wizard for standalone use. -->
|
||||
@if (isEmbedded) {
|
||||
<div class="clr-row">
|
||||
<div id="not-configured" class="card">
|
||||
<div class="card-header">Data Controller is not configured</div>
|
||||
<div class="card-block">
|
||||
<div class="card-text">
|
||||
<p class="mt-0">
|
||||
This Data Controller instance has not been deployed yet, so it
|
||||
cannot be shown inside a report.
|
||||
</p>
|
||||
<p>
|
||||
Deployment has to be done once, in its own browser tab, by someone
|
||||
who can write to the application location:
|
||||
<a [href]="deployUrl" target="_blank">{{ deployUrl }}</a>
|
||||
</p>
|
||||
<p>
|
||||
When it is configured, reload the report and this object will show
|
||||
the editor.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
}
|
||||
|
||||
<div class="clr-row">
|
||||
<!-- T&C section -->
|
||||
@if (!isEmbedded && step === 0) {
|
||||
@if (step === 0) {
|
||||
<div id="TCS" class="card">
|
||||
<div class="card-header">Terms and Conditions</div>
|
||||
<div class="card-block">
|
||||
@@ -64,7 +35,7 @@
|
||||
</div>
|
||||
<!-- T&C section end -->
|
||||
|
||||
@if (!isEmbedded && step > 0) {
|
||||
@if (step > 0) {
|
||||
@switch (true) {
|
||||
@case (sasJsConfig.serverType === ServerType.SasViya) {
|
||||
<div class="autodeploy-section card">
|
||||
|
||||
@@ -1,229 +0,0 @@
|
||||
import { NO_ERRORS_SCHEMA } from '@angular/core'
|
||||
import { ComponentFixture, TestBed } from '@angular/core/testing'
|
||||
import { ServerType } from '@sasjs/utils/types/serverType'
|
||||
import { DeployComponent } from './deploy.component'
|
||||
import { standaloneUrl } from './standalone-url'
|
||||
import { globals } from '../_globals'
|
||||
import { AppStoreService } from '../services/app-store.service'
|
||||
import { SasService } from '../services/sas.service'
|
||||
|
||||
/**
|
||||
* An embedded instance cannot deploy itself: the deployment needs an
|
||||
* authenticated, standalone session, and the person looking at a report object
|
||||
* is not the person who installs the app. The component therefore withholds the
|
||||
* wizard and names the state instead.
|
||||
*
|
||||
* These cases render the template, so they fail if the `!isEmbedded` guards in
|
||||
* deploy.component.html are dropped - asserting the getter alone would not.
|
||||
*/
|
||||
describe('DeployComponent embed guard', () => {
|
||||
let fixture: ComponentFixture<DeployComponent>
|
||||
let component: DeployComponent
|
||||
|
||||
const originalEmbed = globals.embed
|
||||
|
||||
const configure = () =>
|
||||
TestBed.configureTestingModule({
|
||||
declarations: [DeployComponent],
|
||||
providers: [
|
||||
{
|
||||
provide: AppStoreService,
|
||||
useValue: {
|
||||
getDcAdapterSettings: () => ({
|
||||
appLoc: '/Public/app/dc',
|
||||
dcPath: '/tmp/dc'
|
||||
})
|
||||
}
|
||||
},
|
||||
{
|
||||
provide: SasService,
|
||||
useValue: {
|
||||
getSasjsInstance: () => ({}),
|
||||
getSasjsConfig: () => ({ serverType: ServerType.Sasjs })
|
||||
}
|
||||
}
|
||||
],
|
||||
// The wizard renders child components this spec does not exercise.
|
||||
schemas: [NO_ERRORS_SCHEMA]
|
||||
}).compileComponents()
|
||||
|
||||
const render = () => {
|
||||
fixture = TestBed.createComponent(DeployComponent)
|
||||
component = fixture.componentInstance
|
||||
fixture.detectChanges()
|
||||
}
|
||||
|
||||
const tcsSection = () =>
|
||||
fixture.nativeElement.querySelector('#TCS') as HTMLElement | null
|
||||
const notConfigured = () =>
|
||||
fixture.nativeElement.querySelector('#not-configured') as HTMLElement | null
|
||||
const deploySection = () =>
|
||||
fixture.nativeElement.querySelector(
|
||||
'.autodeploy-section'
|
||||
) as HTMLElement | null
|
||||
|
||||
afterEach(() => {
|
||||
globals.embed = originalEmbed
|
||||
TestBed.resetTestingModule()
|
||||
})
|
||||
|
||||
it('offers the wizard when running in its own tab', () => {
|
||||
globals.embed = false
|
||||
configure()
|
||||
render()
|
||||
|
||||
expect(tcsSection()).toBeTruthy()
|
||||
expect(notConfigured()).toBeNull()
|
||||
expect(component.isEmbedded).toBeFalse()
|
||||
})
|
||||
|
||||
it('replaces the wizard with the notice for a VA embed', () => {
|
||||
globals.embed = 'va'
|
||||
configure()
|
||||
render()
|
||||
|
||||
expect(tcsSection()).toBeNull()
|
||||
expect(notConfigured()).toBeTruthy()
|
||||
})
|
||||
|
||||
it('replaces the wizard with the notice for a plain embed too', () => {
|
||||
globals.embed = true
|
||||
configure()
|
||||
render()
|
||||
|
||||
expect(tcsSection()).toBeNull()
|
||||
expect(notConfigured()).toBeTruthy()
|
||||
})
|
||||
|
||||
it('names the standalone URL in the embedded notice', () => {
|
||||
globals.embed = 'va'
|
||||
configure()
|
||||
render()
|
||||
|
||||
const link = notConfigured()!.querySelector('a') as HTMLAnchorElement
|
||||
|
||||
expect(link.getAttribute('href')).toBe(component.deployUrl)
|
||||
expect(link.textContent).toBe(component.deployUrl)
|
||||
})
|
||||
|
||||
it('withholds the deploy section after the terms are accepted, when embedded', () => {
|
||||
globals.embed = 'va'
|
||||
configure()
|
||||
render()
|
||||
|
||||
component.step = 1
|
||||
fixture.detectChanges()
|
||||
|
||||
expect(deploySection()).toBeNull()
|
||||
})
|
||||
|
||||
it('shows the deploy section after the terms are accepted, standalone', () => {
|
||||
globals.embed = false
|
||||
configure()
|
||||
render()
|
||||
|
||||
component.step = 1
|
||||
fixture.detectChanges()
|
||||
|
||||
expect(deploySection()).toBeTruthy()
|
||||
})
|
||||
})
|
||||
|
||||
describe('DeployComponent onNavigateToHome', () => {
|
||||
const originalEmbed = globals.embed
|
||||
const originalHref = window.location.href
|
||||
|
||||
afterEach(() => {
|
||||
globals.embed = originalEmbed
|
||||
history.replaceState(null, '', originalHref)
|
||||
TestBed.resetTestingModule()
|
||||
})
|
||||
|
||||
it('opens the standalone URL, merging an embed tab into the app URL', () => {
|
||||
// Karma serves the runner with its own query, so the app URL is set here
|
||||
// rather than read from window.location: the assertion has to see the
|
||||
// app's query survive the embed parameters being dropped. The shape is
|
||||
// the real SAS 9 one - the app's `_program` query in the search part,
|
||||
// the embed switches in the hash query.
|
||||
history.replaceState(
|
||||
null,
|
||||
'',
|
||||
'/deploy-tc?_program=/Public/app/dc&_debug=131#/editor?embed=va&va_parmap=A-B'
|
||||
)
|
||||
const open = spyOn(window, 'open')
|
||||
TestBed.configureTestingModule({
|
||||
declarations: [DeployComponent],
|
||||
providers: [
|
||||
{
|
||||
provide: AppStoreService,
|
||||
useValue: { getDcAdapterSettings: () => ({ appLoc: '', dcPath: '' }) }
|
||||
},
|
||||
{
|
||||
provide: SasService,
|
||||
useValue: {
|
||||
getSasjsInstance: () => ({}),
|
||||
getSasjsConfig: () => ({ serverType: ServerType.Sasjs })
|
||||
}
|
||||
}
|
||||
],
|
||||
schemas: [NO_ERRORS_SCHEMA]
|
||||
})
|
||||
const fixture = TestBed.createComponent(DeployComponent)
|
||||
|
||||
fixture.componentInstance.onNavigateToHome()
|
||||
|
||||
const opened = open.calls.mostRecent().args[0] as string
|
||||
expect(opened).toBe(
|
||||
window.location.origin + '/deploy-tc?_program=/Public/app/dc&_debug=131'
|
||||
)
|
||||
expect(opened).not.toContain('#')
|
||||
expect(opened).not.toContain('embed=')
|
||||
})
|
||||
})
|
||||
|
||||
describe('standaloneUrl', () => {
|
||||
const origin = window.location.origin
|
||||
|
||||
it('keeps the path of a plain URL', () => {
|
||||
expect(standaloneUrl(`${origin}/SASJobExecution/dc`)).toBe(
|
||||
`${origin}/SASJobExecution/dc`
|
||||
)
|
||||
})
|
||||
|
||||
it('keeps a query the app needs, and drops the hash route', () => {
|
||||
expect(standaloneUrl(`${origin}/path?x=1#/deploy`)).toBe(
|
||||
`${origin}/path?x=1`
|
||||
)
|
||||
})
|
||||
|
||||
it('keeps `_program=`, which is how SAS 9 and Viya address the app', () => {
|
||||
// The whole app URL is the query here: /SASJobExecution/?_program=... is
|
||||
// the standalone app, so dropping the query would open a blank executor.
|
||||
const sas9 =
|
||||
`${origin}/SASJobExecution/?_program=/Public/app/dc/services/public/startupservice` +
|
||||
'&_debug=131'
|
||||
expect(standaloneUrl(sas9)).toBe(sas9)
|
||||
})
|
||||
|
||||
it('drops a bare embed query', () => {
|
||||
expect(standaloneUrl(`${origin}/app?embed=va`)).toBe(`${origin}/app`)
|
||||
})
|
||||
|
||||
it('drops the va_parmap entries along with the embed switch', () => {
|
||||
expect(
|
||||
standaloneUrl(
|
||||
`${origin}/app?embed=va&va_parmap=A-B&va_parmap=C-D#/editor/x`
|
||||
)
|
||||
).toBe(`${origin}/app`)
|
||||
})
|
||||
|
||||
it('keeps a query parameter that merely starts with va_', () => {
|
||||
expect(standaloneUrl(`${origin}/app?embed=va&va_rowlimit=100`)).toBe(
|
||||
`${origin}/app?va_rowlimit=100`
|
||||
)
|
||||
})
|
||||
|
||||
it('resolves a relative URL against the current origin', () => {
|
||||
expect(standaloneUrl('/app/#/deploy')).toBe(`${origin}/app/`)
|
||||
})
|
||||
})
|
||||
@@ -4,8 +4,6 @@ import { SASjsConfig } from '@sasjs/adapter'
|
||||
import { ServerType } from '@sasjs/utils/types/serverType'
|
||||
import { AppStoreService } from '../services/app-store.service'
|
||||
import { DcAdapterSettings } from '../models/DcAdapterSettings'
|
||||
import { globals } from '../_globals'
|
||||
import { standaloneUrl } from './standalone-url'
|
||||
|
||||
@Component({
|
||||
selector: 'app-deploy',
|
||||
@@ -46,20 +44,6 @@ export class DeployComponent implements OnInit {
|
||||
this.setDeployDefaults()
|
||||
}
|
||||
|
||||
/**
|
||||
* True when DC is running inside a report rather than in its own tab. The
|
||||
* deployment needs an authenticated, standalone session, so the wizard is not
|
||||
* offered here - a report object cannot deploy the app that it is displaying.
|
||||
*/
|
||||
get isEmbedded(): boolean {
|
||||
return !!globals.embed
|
||||
}
|
||||
|
||||
/** The standalone URL an administrator deploys from, for the embedded notice. */
|
||||
get deployUrl(): string {
|
||||
return standaloneUrl(window.location.href)
|
||||
}
|
||||
|
||||
/**
|
||||
* Setting default values used for deploy request
|
||||
*/
|
||||
@@ -76,8 +60,6 @@ export class DeployComponent implements OnInit {
|
||||
}
|
||||
|
||||
public onNavigateToHome() {
|
||||
// Same helper as deployUrl: splitting on '#' left the query string in
|
||||
// place, so a tab opened from `?embed=va#/deploy` stayed embedded.
|
||||
window.open(standaloneUrl(window.location.href), '_blank')
|
||||
window.open(location.href.split('#')[0], '_blank')
|
||||
}
|
||||
}
|
||||
@@ -1,29 +0,0 @@
|
||||
/**
|
||||
* The URL a Data Controller instance runs at when it is not embedded: the same
|
||||
* URL, without the parts that make it an embed - the `embed` switch, the
|
||||
* `va_parmap` entries, and the hash route.
|
||||
*
|
||||
* The rest of the query is kept. On SAS 9 and Viya the app is addressed
|
||||
* entirely through it (`/SASJobExecution/?_program=/.../startupservice`), so
|
||||
* dropping the query would open a blank executor rather than the app, and any
|
||||
* other parameter the deployment relies on would go the same way.
|
||||
*
|
||||
* Only these two parameters are embed parameters. A later parameter that
|
||||
* happens to start with `va_` belongs to the app, not the embed, so it stays.
|
||||
*
|
||||
* Parsing with URL also handles the relative hrefs the adapter can hand back,
|
||||
* which plain string splicing cannot.
|
||||
*/
|
||||
const EMBED_PARAMS = ['embed', 'va_parmap']
|
||||
|
||||
export const standaloneUrl = (href: string): string => {
|
||||
const url = new URL(href, window.location.origin)
|
||||
url.hash = ''
|
||||
for (const key of Array.from(url.searchParams.keys())) {
|
||||
if (EMBED_PARAMS.includes(key.toLowerCase())) {
|
||||
url.searchParams.delete(key)
|
||||
}
|
||||
}
|
||||
|
||||
return url.toString()
|
||||
}
|
||||
@@ -459,43 +459,6 @@
|
||||
<!--<hr>-->
|
||||
<!--{{ validationDone }}-->
|
||||
|
||||
<!-- VA embed: something about a `va_parmap` entry in this report
|
||||
object's URL needs the report author's attention - it could not be
|
||||
used, it was applied but AND-ed against another parameter, or it
|
||||
names a parameter VA never sends. Shown as a banner, not just a
|
||||
console warning, because
|
||||
the entries are written by the report author and a typo otherwise
|
||||
looks like a working filter over the whole table. -->
|
||||
@if (isVaEmbed && vaMappingWarnings.length) {
|
||||
<clr-alert
|
||||
class="va-parmap-warning"
|
||||
[clrAlertType]="'warning'"
|
||||
[clrAlertClosable]="false"
|
||||
>
|
||||
<clr-alert-item>
|
||||
<div class="alert-text">
|
||||
<strong>VA parameter mapping problem</strong>
|
||||
@for (warning of vaMappingWarnings; track warning) {
|
||||
<div>{{ warning }}</div>
|
||||
}
|
||||
<!-- The reader is the report author, who can only fix this in
|
||||
the report: name the parameter they edit, and say what a
|
||||
correct entry looks like, so the message is actionable
|
||||
without the docs. -->
|
||||
<div>
|
||||
Fix: edit the <code>va_parmap</code> entries in the URL of
|
||||
this Data Controller object in the report. An entry is
|
||||
<code>parameter-column</code>, optionally
|
||||
<code>:ADD(anchor,offset)</code> or
|
||||
<code>:RANGE(period,offset)</code> - for example
|
||||
<code>va_parmap=SEL_DATE-D_DATE:ADD(QUARTER_START,-1)</code
|
||||
>.
|
||||
</div>
|
||||
</div>
|
||||
</clr-alert-item>
|
||||
</clr-alert>
|
||||
}
|
||||
|
||||
<div class="hot-wrapper clr-flex-1">
|
||||
<hot-table
|
||||
#hotInstance
|
||||
|
||||
@@ -27,15 +27,3 @@
|
||||
font-style: italic;
|
||||
opacity: 0.7;
|
||||
}
|
||||
|
||||
/* VA embed: `va_parmap` diagnostic banner. Clarity's alert lays `.alert-title`
|
||||
and `.alert-text` out on one row, which pushes a long message off the right
|
||||
edge; this keeps the whole message in the flow and wraps it. */
|
||||
.va-parmap-warning {
|
||||
width: 100%;
|
||||
|
||||
.alert-text {
|
||||
white-space: normal;
|
||||
overflow-wrap: anywhere;
|
||||
}
|
||||
}
|
||||
@@ -73,7 +73,6 @@ import {
|
||||
} from '../shared/dc-validator/utils/autoEscapedCellTracker'
|
||||
import { HyperFormula } from 'hyperformula'
|
||||
import { parseLabelsParam } from '../shared/utils/parse-labels-param'
|
||||
import { VaParmapTarget } from '../shared/utils/parse-va-parmap'
|
||||
import { getDisplayColHeaders } from '../shared/utils/display-col-headers'
|
||||
import { buildColInfoHtml } from '../shared/utils/col-info-html'
|
||||
import { globals } from '../_globals'
|
||||
@@ -486,46 +485,6 @@ export class EditorComponent
|
||||
get isVaEmbed() {
|
||||
return globals.embed === 'va'
|
||||
}
|
||||
/**
|
||||
* Explicit VA parameter -> DC column mappings (`va_parmap`) for this instance.
|
||||
* Parsed once at app init (see parseVaParmap) and held on globals so the
|
||||
* editor reloads a filter apply performs keep using the same mapping.
|
||||
*/
|
||||
get vaParamMap(): Map<string, VaParmapTarget> {
|
||||
return globals.vaParamMap
|
||||
}
|
||||
/** `va_parmap` entries that name a column this table does not have. */
|
||||
public vaUnresolvedMappings: string[] = []
|
||||
/** `va_parmap` keys that no VA message has named (a parameter-side typo). */
|
||||
public vaUnmatchedMappings: string[] = []
|
||||
/** Parameters in this message that filter the same column (an AND-ed pair). */
|
||||
public vaSameColumnMappings: string[] = []
|
||||
/** Transform mappings this message cannot apply (non-temporal column or value). */
|
||||
public vaTransformProblems: string[] = []
|
||||
/** The warning text last written to the console, so repeated VA messages log once. */
|
||||
private vaWarningsReported = ''
|
||||
/**
|
||||
* Everything wrong with this instance's `va_parmap` entries, in the report
|
||||
* author's terms: entries that could not be parsed at all, plus entries that
|
||||
* name a column the open table does not have. Shown as a banner in the editor
|
||||
* - a report developer is usually not going to read the browser console.
|
||||
*/
|
||||
get vaMappingWarnings(): string[] {
|
||||
return [...globals.vaParmapProblems, ...this.vaRuntimeWarnings]
|
||||
}
|
||||
/**
|
||||
* The warnings this instance discovers for itself, i.e. everything except the
|
||||
* parse-time problems - those are logged once at app init (`app.component`),
|
||||
* so logging them again here would double every line in the console.
|
||||
*/
|
||||
get vaRuntimeWarnings(): string[] {
|
||||
return [
|
||||
...this.vaUnresolvedMappings,
|
||||
...this.vaUnmatchedMappings,
|
||||
...this.vaSameColumnMappings,
|
||||
...this.vaTransformProblems
|
||||
]
|
||||
}
|
||||
/**
|
||||
* VA filter mode. Live = auto-apply each VA change; confirm = stage the change
|
||||
* and wait for the user to click Apply. Backed by globals (toggled by the
|
||||
@@ -616,8 +575,6 @@ export class EditorComponent
|
||||
public hasBaseDropZoneOver = false
|
||||
public hasAnotherDropZoneOver = false
|
||||
public headerPks: string[] = []
|
||||
/** Derived (computed) columns from MPE_DERIVED - read-only, and marked as such */
|
||||
public derivedVars: string[] = []
|
||||
public columnLevelSecurityFlag = false
|
||||
public dateTimeHeaders: string[] = []
|
||||
public timeHeaders: string[] = []
|
||||
@@ -1132,20 +1089,6 @@ export class EditorComponent
|
||||
return colRules?.readOnly
|
||||
}
|
||||
|
||||
/**
|
||||
* A derived (computed) column - configured in MPE_DERIVED, created at read
|
||||
* time. Read-only, and styled apart from an ordinary read-only column so the
|
||||
* user can see that there is nothing to edit rather than that they may not.
|
||||
*/
|
||||
isDerivedCol(col: string | number) {
|
||||
const name =
|
||||
typeof col === 'number'
|
||||
? (this.hotInstance?.colToProp(col) as string) || this.columnHeader[col]
|
||||
: col
|
||||
|
||||
return !!name && this.derivedVars.includes(name.toUpperCase())
|
||||
}
|
||||
|
||||
isColHeader(col: string) {
|
||||
return this.headerArray.indexOf(col.toUpperCase()) > -1
|
||||
}
|
||||
@@ -2814,24 +2757,17 @@ export class EditorComponent
|
||||
this.disableSubmit = true
|
||||
this.submit = false
|
||||
|
||||
/* sasService has already shown the SAS message when it rejected
|
||||
(the abort, or the adapter's error), so reporting it again here
|
||||
stacked a second modal on top of the first - and read
|
||||
'undefined' because it looked for a property that does not
|
||||
exist. Only speak up when nothing was shown. */
|
||||
if (!err?.errorShown) {
|
||||
const errorText =
|
||||
typeof err?.adapterResponse?.error === 'string'
|
||||
? err.adapterResponse.error
|
||||
: JSON.stringify(err?.adapterResponse ?? err)
|
||||
const errorText =
|
||||
typeof err.adapterRespnse === 'string'
|
||||
? err.adapterRespnse
|
||||
: JSON.stringify(err.adapterRespnse)
|
||||
|
||||
this.eventService.showAbortModal(
|
||||
'editors/stagedata',
|
||||
`Submit request failed\n${errorText}`,
|
||||
null,
|
||||
'Submit error'
|
||||
)
|
||||
}
|
||||
this.eventService.showAbortModal(
|
||||
'editors/stagedata',
|
||||
`Submit request failed\n${errorText}`,
|
||||
null,
|
||||
'Submit error'
|
||||
)
|
||||
|
||||
return false
|
||||
})
|
||||
@@ -3643,7 +3579,7 @@ export class EditorComponent
|
||||
const msg = this.vaMessaging.latestMessage()
|
||||
if (!msg) return false
|
||||
|
||||
const clauses = this.vaFilter.buildInitialClauses(msg, this.vaParamMap)
|
||||
const clauses = this.vaFilter.buildInitialClauses(msg)
|
||||
if (clauses.length === 0) return false
|
||||
|
||||
try {
|
||||
@@ -4005,11 +3941,8 @@ export class EditorComponent
|
||||
|
||||
// getdata serves a selection over DC_MAXOBS_WEBEDIT / DC_MAXCELLS_WEBEDIT
|
||||
// truncated to the first rows within the limit, and carries the reason
|
||||
// here. Empty means the whole selection is in the grid - and a blank
|
||||
// character arrives as a single blank (mp_jsonout cannot emit a zero-length
|
||||
// string, since SAS has none), so trim it: whitespace is "no note", not a
|
||||
// warning with an empty tooltip.
|
||||
this.truncationNote = (response.data.sasparams[0].TRUNC_MSG || '').trim()
|
||||
// here. Empty means the whole selection is in the grid.
|
||||
this.truncationNote = response.data.sasparams[0].TRUNC_MSG || ''
|
||||
|
||||
const hot = this.hotInstance
|
||||
|
||||
@@ -4031,12 +3964,6 @@ export class EditorComponent
|
||||
|
||||
this.headerColumns = response.data.sasparams[0].COLHEADERS.split(',')
|
||||
this.headerPks = response.data.sasparams[0].PK.split(' ')
|
||||
// Derived (computed) columns, named by getdata from MPE_DERIVED - read-only
|
||||
// through a READONLY DQ rule, and marked in the grid so they read as
|
||||
// computed rather than merely non-editable.
|
||||
this.derivedVars = (response.data.sasparams[0].DERIVEDVARS || '')
|
||||
.split(' ')
|
||||
.filter((col: string) => !!col)
|
||||
this.columnLevelSecurityFlag = !!response.data.sasparams[0].CLS_FLAG
|
||||
|
||||
if (this.columnLevelSecurityFlag)
|
||||
@@ -4411,37 +4338,9 @@ export class EditorComponent
|
||||
// header columns styling - primary keys
|
||||
const isPKCol = column && this.isColPk(column)
|
||||
const isReadonlyCol = column && this.isReadonlyCol(column)
|
||||
const isDerivedCol = column && this.isDerivedCol(column)
|
||||
|
||||
if (isPKCol) th.classList.add('primaryKeyHeaderStyle')
|
||||
if (isReadonlyCol && !isPKCol && !isDerivedCol) {
|
||||
th.classList.add('readonlyCell')
|
||||
}
|
||||
|
||||
// a derived column is computed, not merely locked - give it its own
|
||||
// hue and an 'fx' marker rather than the read-only dimming
|
||||
if (isDerivedCol) {
|
||||
th.classList.add('dc-derived-header')
|
||||
// the label says what the column is - for a derived variable that
|
||||
// is the configured VAR_LABEL, or the generated name=expression
|
||||
// when the administrator left it blank. A label that merely
|
||||
// repeats the name adds nothing, so it falls back to the rule.
|
||||
const derivedLabel = (this.cols || [])
|
||||
.filter((c: any) => c.NAME === column)
|
||||
.map((c: any) => String(c.LABEL || '').trim())
|
||||
.find(
|
||||
(l: string) =>
|
||||
l.length > 0 &&
|
||||
l.toUpperCase() !== String(column).toUpperCase()
|
||||
)
|
||||
th.title = derivedLabel
|
||||
? `${derivedLabel} - derived variable, computed at read time and cannot be edited`
|
||||
: 'Derived variable - computed at read time and cannot be edited'
|
||||
const firstDerivedCol = this.columnHeader.findIndex(
|
||||
(name: string) => this.isDerivedCol(name)
|
||||
)
|
||||
if (col === firstDerivedCol) th.classList.add('dc-derived-first')
|
||||
}
|
||||
if (isReadonlyCol && !isPKCol) th.classList.add('readonlyCell')
|
||||
|
||||
// Remove header arrow from Delete column
|
||||
if (col === 0) {
|
||||
@@ -4457,7 +4356,6 @@ export class EditorComponent
|
||||
cellProperties: Handsontable.CellProperties
|
||||
) => {
|
||||
const isReadonlyCol = col && this.isReadonlyCol(col)
|
||||
const isDerivedCol = col && this.isDerivedCol(col)
|
||||
|
||||
// Check if this cell should be marked as invalid due to duplicate primary key values
|
||||
// Only applies to primary key columns (col 1 through readOnlyFields)
|
||||
@@ -4481,26 +4379,7 @@ export class EditorComponent
|
||||
}
|
||||
|
||||
// Add readonlyCell class for readonly columns to maintain original styling
|
||||
if (isDerivedCol) {
|
||||
// a derived cell carries its own tint; the read-only dimming would
|
||||
// wash it out, and "greyed out" is a different message from
|
||||
// "computed here"
|
||||
classes = classes.filter((c: string) => c !== 'readonlyCell')
|
||||
if (!classes.includes('dc-derived-cell')) {
|
||||
classes.push('dc-derived-cell')
|
||||
}
|
||||
// derived columns are appended after the physical ones - mark the
|
||||
// start of the block so it reads as one group
|
||||
const firstDerivedCol = this.columnHeader.findIndex(
|
||||
(name: string) => this.isDerivedCol(name)
|
||||
)
|
||||
if (
|
||||
col === firstDerivedCol &&
|
||||
!classes.includes('dc-derived-first')
|
||||
) {
|
||||
classes.push('dc-derived-first')
|
||||
}
|
||||
} else if (isReadonlyCol && !classes.includes('readonlyCell')) {
|
||||
if (isReadonlyCol && !classes.includes('readonlyCell')) {
|
||||
classes.push('readonlyCell')
|
||||
}
|
||||
|
||||
@@ -5089,53 +4968,7 @@ export class EditorComponent
|
||||
|
||||
this.updateVaFilterText(msg)
|
||||
|
||||
// A `va_parmap` entry naming a column this table does not have is skipped by
|
||||
// buildClauses. Report it (banner + console, once per distinct set) so the
|
||||
// report author sees a misconfiguration instead of an unfiltered grid.
|
||||
this.vaUnresolvedMappings = this.vaFilter.unresolvedMappings(
|
||||
msg,
|
||||
this.vaParamMap,
|
||||
this.cols,
|
||||
this.libds
|
||||
)
|
||||
// A key naming no parameter VA has ever sent is a typo on the parameter
|
||||
// side: the mapping never applies. Only judged once parameters are actually
|
||||
// arriving, since VA sends just the parameters its query uses.
|
||||
this.vaUnmatchedMappings = this.vaFilter.unmatchedMappings(
|
||||
this.vaParamMap,
|
||||
this.vaMessaging.seenParamKeys,
|
||||
this.vaMessaging.lastMessageParamsStable
|
||||
)
|
||||
// Two parameters filtering one column are AND-ed, so differing values leave
|
||||
// an empty grid. Reported, not rewritten: turning that AND into an OR is a
|
||||
// filter-semantics decision, taken in buildClauses and not in a diagnostic.
|
||||
this.vaSameColumnMappings = this.vaFilter.sameColumnMappings(
|
||||
msg,
|
||||
this.vaParamMap,
|
||||
this.cols
|
||||
)
|
||||
// A transform that cannot run here (char column, value that is not a date) is
|
||||
// skipped by the clause builder, so the banner has to say why.
|
||||
this.vaTransformProblems = this.vaFilter.unsupportedTransforms(
|
||||
msg,
|
||||
this.vaParamMap,
|
||||
this.cols
|
||||
)
|
||||
const warnings = this.vaRuntimeWarnings.join(' | ')
|
||||
if (warnings !== this.vaWarningsReported) {
|
||||
this.vaWarningsReported = warnings
|
||||
for (const warning of this.vaRuntimeWarnings) {
|
||||
// eslint-disable-next-line no-console
|
||||
console.warn('Data Controller: ' + warning)
|
||||
}
|
||||
}
|
||||
|
||||
const clauses = this.vaFilter.buildClauses(
|
||||
msg,
|
||||
labelToVar,
|
||||
this.cols,
|
||||
this.vaParamMap
|
||||
)
|
||||
const clauses = this.vaFilter.buildClauses(msg, labelToVar, this.cols)
|
||||
const signature = this.vaFilter.signature(clauses)
|
||||
|
||||
if (signature === this.vaMessaging.filterSignature) {
|
||||
|
||||
@@ -51,11 +51,6 @@ export interface SASParam {
|
||||
TRUNCATED?: number
|
||||
/** Reason shown in the tooltip when TRUNCATED is 1, else empty */
|
||||
TRUNC_MSG?: string
|
||||
/**
|
||||
* Space separated list of the derived (computed) columns from MPE_DERIVED,
|
||||
* upper case. Empty when the table has none.
|
||||
*/
|
||||
DERIVEDVARS?: string
|
||||
}
|
||||
|
||||
export interface ExcelRule {
|
||||
|
||||
@@ -868,21 +868,6 @@ export class MultiDatasetComponent implements OnInit, AfterViewInit {
|
||||
return this.sasStoreService
|
||||
.callService(myParams, 'SASControlTable', 'editors/getdata', libds)
|
||||
.then((res: EditorsGetDataServiceResponse) => {
|
||||
// getdata serves a selection over DC_MAXOBS_WEBEDIT / DC_MAXCELLS_WEBEDIT
|
||||
// truncated, rather than aborting the way it used to. This flow compares
|
||||
// the whole table against the upload, so a partial read would compute
|
||||
// the changes against the wrong starting state - refuse the table
|
||||
// instead of submitting differences nobody asked for.
|
||||
const sasParams = res?.data?.sasparams?.[0]
|
||||
|
||||
if (sasParams && Number(sasParams.TRUNCATED) === 1) {
|
||||
this.eventService.showInfoModal(
|
||||
'Upload error',
|
||||
`${libds} holds more than the EDIT limit allows (${sasParams.TRUNC_MSG}), so it cannot be compared with the upload. Raise DC_MAXOBS_WEBEDIT / DC_MAXCELLS_WEBEDIT, or submit the table without the multi-table upload.`
|
||||
)
|
||||
return undefined
|
||||
}
|
||||
|
||||
return res
|
||||
})
|
||||
.catch((err: any) => {
|
||||
|
||||
@@ -186,8 +186,7 @@ describe('SasService openServiceInNewWindow', () => {
|
||||
|
||||
expect(openSpy).toHaveBeenCalledWith(
|
||||
'https://sas.example.com/SASJobExecution/?_program=/Public/app/dc/services/public/startupservice&_contextname=ctx&_debug=131',
|
||||
'_blank',
|
||||
'noopener'
|
||||
'_blank'
|
||||
)
|
||||
})
|
||||
|
||||
@@ -200,8 +199,7 @@ describe('SasService openServiceInNewWindow', () => {
|
||||
|
||||
expect(openSpy).toHaveBeenCalledWith(
|
||||
'https://sas.example.com/SASJobExecution/?_program=/Public/app/dc/services/public/startupservice&_debug=131',
|
||||
'_blank',
|
||||
'noopener'
|
||||
'_blank'
|
||||
)
|
||||
})
|
||||
})
|
||||
@@ -286,10 +286,7 @@ export class SasService {
|
||||
adapterResponse: {
|
||||
error: abortMsg
|
||||
},
|
||||
log: sasRequest?.logFile,
|
||||
/* the modal above already told the user: a caller that also
|
||||
reports the failure would stack a second one on top */
|
||||
errorShown: !wrapperOptions?.suppressSuccessAbortModal
|
||||
log: sasRequest?.logFile
|
||||
})
|
||||
}
|
||||
|
||||
@@ -332,8 +329,7 @@ export class SasService {
|
||||
adapterResponse: {
|
||||
error: errorMessage
|
||||
},
|
||||
log: sasRequest?.logFile,
|
||||
errorShown: !wrapperOptions?.suppressErrorAbortModal
|
||||
log: sasRequest?.logFile
|
||||
})
|
||||
}
|
||||
|
||||
@@ -871,10 +867,7 @@ export class SasService {
|
||||
params
|
||||
})
|
||||
|
||||
// noopener: the SAS log page is on a different origin (Viya's serverUrl is
|
||||
// not the app's), and without it the opened window keeps a handle on
|
||||
// window.opener and can navigate this tab.
|
||||
window.open(url, '_blank', 'noopener')
|
||||
window.open(url, '_blank')
|
||||
}
|
||||
|
||||
public getServerType(): string {
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import { VaFilterService } from './va-filter.service'
|
||||
import { VaParmapTarget } from '../shared/utils/parse-va-parmap'
|
||||
import { VaMessage } from './va-messaging.service'
|
||||
import { HelperService } from './helper.service'
|
||||
|
||||
@@ -15,7 +14,6 @@ function msg(parameters: any[]): VaMessage {
|
||||
|
||||
describe('VaFilterService', () => {
|
||||
let service: VaFilterService
|
||||
let helperService: HelperService
|
||||
|
||||
// DC column metadata used by the metadata path. cellValidation[0] is the
|
||||
// delete column and must be skipped by buildLabelToVarMap.
|
||||
@@ -44,8 +42,7 @@ describe('VaFilterService', () => {
|
||||
]
|
||||
|
||||
beforeEach(() => {
|
||||
helperService = new HelperService({} as any)
|
||||
service = new VaFilterService(helperService)
|
||||
service = new VaFilterService(new HelperService({} as any))
|
||||
})
|
||||
|
||||
const labelToVar = () =>
|
||||
@@ -131,41 +128,6 @@ describe('VaFilterService', () => {
|
||||
expect(build([{ label: 'SOME_DATE', value: '2025/03/31' }])).toHaveSize(1)
|
||||
})
|
||||
|
||||
it('parses word dates, in either order and with an optional comma', () => {
|
||||
const expected = "'01JAN2026'd"
|
||||
// The reported shape: a date parameter displayed as WORDDATE.
|
||||
expect(
|
||||
build([{ label: 'SOME_DATE', value: 'January 1, 2026' }])[0].RAW_VALUE
|
||||
).toBe(expected)
|
||||
expect(
|
||||
build([{ label: 'SOME_DATE', value: 'Jan 1, 2026' }])[0].RAW_VALUE
|
||||
).toBe(expected)
|
||||
expect(
|
||||
build([{ label: 'SOME_DATE', value: 'Jan. 1, 2026' }])[0].RAW_VALUE
|
||||
).toBe(expected)
|
||||
expect(
|
||||
build([{ label: 'SOME_DATE', value: '1 January 2026' }])[0].RAW_VALUE
|
||||
).toBe(expected)
|
||||
expect(
|
||||
build([{ label: 'SOME_DATE', value: '1-Jan-2026' }])[0].RAW_VALUE
|
||||
).toBe(expected)
|
||||
})
|
||||
|
||||
it('matches a spelled-out month case-insensitively, and keeps DATE9 working', () => {
|
||||
expect(
|
||||
build([{ label: 'SOME_DATE', value: 'DECEMBER 31, 2025' }])[0].RAW_VALUE
|
||||
).toBe("'31DEC2025'd")
|
||||
expect(
|
||||
build([{ label: 'SOME_DATE', value: '01AUG1962' }])[0].RAW_VALUE
|
||||
).toBe("'01AUG1962'd")
|
||||
})
|
||||
|
||||
it('still drops a month-name value that has no day', () => {
|
||||
// A month and year alone is not a date - accepted shapes always carry a
|
||||
// day, so this must not silently become the first of the month.
|
||||
expect(build([{ label: 'SOME_DATE', value: 'January 2026' }])).toEqual([])
|
||||
})
|
||||
|
||||
it('resolves ambiguous slash dates day-first (never US), disambiguating by value', () => {
|
||||
const iso = build([{ label: 'SOME_DATE', value: '2025-03-31' }])[0]
|
||||
.RAW_VALUE
|
||||
@@ -237,694 +199,7 @@ describe('VaFilterService', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('buildClauses with an explicit va_parmap', () => {
|
||||
const build = (parameters: any[], paramMap: Map<string, VaParmapTarget>) =>
|
||||
service.buildClauses(msg(parameters), labelToVar(), cols, paramMap)
|
||||
|
||||
it('overrides the same-name match, including the value formatting', () => {
|
||||
// The VA label IS a DC column here (SOME_CHAR), but the mapping points at
|
||||
// a different one - the numeric column - so the value is formatted numeric.
|
||||
const map = new Map([['some_char', { column: 'SOME_NUM' }]])
|
||||
expect(build([{ label: 'SOME_CHAR', value: '10' }], map)).toEqual([
|
||||
{
|
||||
GROUP_LOGIC: 'AND',
|
||||
SUBGROUP_LOGIC: 'AND',
|
||||
SUBGROUP_ID: 0,
|
||||
VARIABLE_NM: 'SOME_NUM',
|
||||
OPERATOR_NM: '=',
|
||||
RAW_VALUE: '10'
|
||||
}
|
||||
])
|
||||
})
|
||||
|
||||
it('lets two differently-named params filter the same column', () => {
|
||||
// The multi-tab case: one DC instance per report tab, each with its own
|
||||
// VA parameter, both mapping to the same DC column.
|
||||
const map = new Map([
|
||||
['region_tab1', { column: 'SOME_CHAR' }],
|
||||
['region_tab2', { column: 'SOME_CHAR' }]
|
||||
])
|
||||
const clauses = build(
|
||||
[
|
||||
{ label: 'REGION_TAB1', value: 'north' },
|
||||
{ label: 'REGION_TAB2', value: 'south' }
|
||||
],
|
||||
map
|
||||
)
|
||||
expect(clauses.map((c) => [c.VARIABLE_NM, c.RAW_VALUE])).toEqual([
|
||||
['SOME_CHAR', "'north'"],
|
||||
['SOME_CHAR', "'south'"]
|
||||
])
|
||||
})
|
||||
|
||||
it('matches the parameter label only, not its internal name', () => {
|
||||
// The internal name is the report's own bookkeeping. Matching it as well
|
||||
// meant one parameter answered to two keys, so an entry written against the
|
||||
// name and one against the label silently fought over it.
|
||||
const map = new Map([['pr273', { column: 'SOME_NUM' }]])
|
||||
expect(
|
||||
build([{ name: 'pr273', label: 'Tab_1_Value', value: '7' }], map)
|
||||
).toEqual([])
|
||||
// The label still matches, case-insensitively.
|
||||
expect(
|
||||
build(
|
||||
[{ name: 'pr273', label: 'Tab_1_Value', value: 'x' }],
|
||||
new Map([['tab_1_value', { column: 'SOME_CHAR' }]])
|
||||
)[0].VARIABLE_NM
|
||||
).toBe('SOME_CHAR')
|
||||
})
|
||||
|
||||
it('uses the column case from the loaded metadata', () => {
|
||||
const map = new Map([['region_tab1', { column: 'some_char' }]])
|
||||
expect(
|
||||
build([{ label: 'REGION_TAB1', value: 'x' }], map)[0].VARIABLE_NM
|
||||
).toBe('SOME_CHAR')
|
||||
})
|
||||
|
||||
it('leaves unmapped params on the same-name path', () => {
|
||||
const map = new Map([['region_tab1', { column: 'SOME_NUM' }]])
|
||||
const clauses = build(
|
||||
[
|
||||
{ label: 'REGION_TAB1', value: '10' },
|
||||
{ label: 'SOME_CHAR', value: 'x' }
|
||||
],
|
||||
map
|
||||
)
|
||||
expect(clauses.map((c) => [c.VARIABLE_NM, c.RAW_VALUE])).toEqual([
|
||||
['SOME_NUM', '10'],
|
||||
['SOME_CHAR', "'x'"]
|
||||
])
|
||||
})
|
||||
|
||||
it('skips a mapping whose column is not in this table (no fallback)', () => {
|
||||
// Falling back to the same-name match would silently apply the filter the
|
||||
// mapping exists to override, so the param is dropped instead.
|
||||
const map = new Map([['some_char', { column: 'NOT_IN_THIS_TABLE' }]])
|
||||
expect(build([{ label: 'SOME_CHAR', value: 'x' }], map)).toEqual([])
|
||||
})
|
||||
|
||||
it('ANDs two parameters that map to the same column - it does not union them', () => {
|
||||
// What a report gets when one column's two values arrive in two
|
||||
// parameters: every clause is AND-ed, so the result is the INTERSECTION.
|
||||
// Two different dates therefore match no rows at all rather than both
|
||||
// dates' rows - a single parameter carrying both values is the shape that
|
||||
// ORs, via `IN (...)`.
|
||||
const map = new Map([
|
||||
['param_a', { column: 'SOME_DATE' }],
|
||||
['param_b', { column: 'SOME_DATE' }]
|
||||
])
|
||||
const one = build([{ label: 'PARAM_A', value: '01JUL1960' }], map)[0]
|
||||
const other = build([{ label: 'PARAM_B', value: '01JAN1961' }], map)[0]
|
||||
const clauses = build(
|
||||
[
|
||||
{ label: 'PARAM_A', value: '01JUL1960' },
|
||||
{ label: 'PARAM_B', value: '01JAN1961' }
|
||||
],
|
||||
map
|
||||
)
|
||||
expect(one.RAW_VALUE).not.toBe(other.RAW_VALUE)
|
||||
expect(clauses).toHaveSize(2)
|
||||
expect(clauses.map((c) => c.RAW_VALUE)).toEqual([
|
||||
one.RAW_VALUE,
|
||||
other.RAW_VALUE
|
||||
])
|
||||
for (const clause of clauses) {
|
||||
expect(clause.VARIABLE_NM).toBe('SOME_DATE')
|
||||
expect(clause.OPERATOR_NM).toBe('=')
|
||||
expect(clause.GROUP_LOGIC).toBe('AND')
|
||||
expect(clause.SUBGROUP_LOGIC).toBe('AND')
|
||||
expect(clause.SUBGROUP_ID).toBe(0)
|
||||
}
|
||||
})
|
||||
|
||||
it('applies a transform to the clause it builds', () => {
|
||||
// ADD(QUARTER_START,-1) is the reported requirement: the selected row plus
|
||||
// the row for the previous quarter, as one IN clause so the AND rule is
|
||||
// untouched. 2026-10-01 = 24380, 2026-07-01 = 24288 in SAS days.
|
||||
const map = new Map([
|
||||
[
|
||||
'selected',
|
||||
{
|
||||
column: 'SOME_DATE',
|
||||
transform: {
|
||||
op: 'ADD' as const,
|
||||
anchor: 'QUARTER_START' as const,
|
||||
offset: -1
|
||||
}
|
||||
}
|
||||
]
|
||||
])
|
||||
const [clause] = build([{ label: 'SELECTED', value: '01OCT2026' }], map)
|
||||
expect(clause.VARIABLE_NM).toBe('SOME_DATE')
|
||||
expect(clause.OPERATOR_NM).toBe('IN')
|
||||
expect(clause.RAW_VALUE).toBe('(24380,24288)')
|
||||
})
|
||||
|
||||
it('applies a RANGE as BETWEEN', () => {
|
||||
// RANGE(QUARTER,-1) for a mid-quarter value: 2026-04-01..2026-06-30
|
||||
const map = new Map([
|
||||
[
|
||||
'selected',
|
||||
{
|
||||
column: 'SOME_DATE',
|
||||
transform: {
|
||||
op: 'RANGE' as const,
|
||||
period: 'QUARTER' as const,
|
||||
offset: -1
|
||||
}
|
||||
}
|
||||
]
|
||||
])
|
||||
const [clause] = build([{ label: 'SELECTED', value: '1962-08-15' }], map)
|
||||
expect(clause.OPERATOR_NM).toBe('BETWEEN')
|
||||
expect(clause.RAW_VALUE).toContain(' AND ')
|
||||
expect(clause.RAW_VALUE.split(' AND ')).toHaveSize(2)
|
||||
})
|
||||
|
||||
it('keeps equality when the value already sits on the anchor', () => {
|
||||
const map = new Map([
|
||||
[
|
||||
'selected',
|
||||
{
|
||||
column: 'SOME_DATE',
|
||||
transform: {
|
||||
op: 'ADD' as const,
|
||||
anchor: 'QUARTER_START' as const,
|
||||
offset: 0
|
||||
}
|
||||
}
|
||||
]
|
||||
])
|
||||
const [clause] = build([{ label: 'SELECTED', value: '01OCT2026' }], map)
|
||||
expect(clause.OPERATOR_NM).toBe('=')
|
||||
expect(clause.RAW_VALUE).toBe('24380')
|
||||
})
|
||||
|
||||
it('drops a transform on a column it cannot apply to', () => {
|
||||
// A character column has no dates to shift; the parameter is skipped rather
|
||||
// than filtered on the raw value the author did not see.
|
||||
const map = new Map([
|
||||
[
|
||||
'selected',
|
||||
{
|
||||
column: 'SOME_CHAR',
|
||||
transform: {
|
||||
op: 'ADD' as const,
|
||||
anchor: 'QUARTER_START' as const,
|
||||
offset: -1
|
||||
}
|
||||
}
|
||||
]
|
||||
])
|
||||
expect(build([{ label: 'SELECTED', value: 'x' }], map)).toEqual([])
|
||||
})
|
||||
|
||||
it('adds nothing when the value is unparseable as a date', () => {
|
||||
const map = new Map([
|
||||
[
|
||||
'selected',
|
||||
{
|
||||
column: 'SOME_DATE',
|
||||
transform: {
|
||||
op: 'ADD' as const,
|
||||
anchor: 'QUARTER_START' as const,
|
||||
offset: -1
|
||||
}
|
||||
}
|
||||
]
|
||||
])
|
||||
expect(build([{ label: 'SELECTED', value: 'not a date' }], map)).toEqual(
|
||||
[]
|
||||
)
|
||||
})
|
||||
|
||||
it('round-trips a bare SAS day number unchanged', () => {
|
||||
// VA may send the SAS internal value rather than a formatted string. The
|
||||
// Date a transform works from is local, so the number must come back out of
|
||||
// the clause identical - a UTC-built Date shifts a day west of UTC.
|
||||
const map = new Map([
|
||||
[
|
||||
'selected',
|
||||
{
|
||||
column: 'SOME_DATE',
|
||||
transform: {
|
||||
op: 'ADD' as const,
|
||||
anchor: 'QUARTER_START' as const,
|
||||
offset: -1
|
||||
}
|
||||
}
|
||||
]
|
||||
])
|
||||
const [clause] = build([{ label: 'SELECTED', value: 24380 }], map)
|
||||
expect(clause.RAW_VALUE).toBe('(24380,24288)')
|
||||
})
|
||||
|
||||
it('reports a range mapping fed more than one value', () => {
|
||||
const map = new Map([
|
||||
[
|
||||
'selected',
|
||||
{
|
||||
column: 'SOME_DATE',
|
||||
transform: {
|
||||
op: 'RANGE' as const,
|
||||
period: 'QUARTER' as const,
|
||||
offset: -1
|
||||
}
|
||||
}
|
||||
]
|
||||
])
|
||||
const problems = service.unsupportedTransforms(
|
||||
msg([{ label: 'SELECTED', value: ['01APR2026', '01JUL2026'] }]),
|
||||
map,
|
||||
[{ NAME: 'SOME_DATE', DDTYPE: 'DATE', FMTNAME: 'DATE9' }]
|
||||
)
|
||||
expect(problems).toHaveSize(1)
|
||||
expect(problems[0]).toContain('is a range')
|
||||
expect(problems[0]).toContain('2 values')
|
||||
})
|
||||
|
||||
it('refuses a range fed more than one value, rather than picking one', () => {
|
||||
// Two values, one of which does not parse: building a BETWEEN from the
|
||||
// survivor would silently filter one period when VA selected two.
|
||||
const map = new Map([
|
||||
[
|
||||
'selected',
|
||||
{
|
||||
column: 'SOME_DATE',
|
||||
transform: {
|
||||
op: 'RANGE' as const,
|
||||
period: 'QUARTER' as const,
|
||||
offset: -1
|
||||
}
|
||||
}
|
||||
]
|
||||
])
|
||||
expect(
|
||||
build([{ label: 'SELECTED', value: ['01APR2026', 'not a date'] }], map)
|
||||
).toEqual([])
|
||||
})
|
||||
|
||||
it('never lets a non-finite date reach the clause', () => {
|
||||
// convertJsDateToSasDate returning NaN would otherwise put a literal NaN in
|
||||
// the WHERE clause.
|
||||
const map = new Map([
|
||||
[
|
||||
'selected',
|
||||
{
|
||||
column: 'SOME_DATE',
|
||||
transform: {
|
||||
op: 'ADD' as const,
|
||||
anchor: 'QUARTER_START' as const,
|
||||
offset: -1
|
||||
}
|
||||
}
|
||||
]
|
||||
])
|
||||
spyOn(helperService, 'convertJsDateToSasDate').and.returnValue(NaN as any)
|
||||
expect(build([{ label: 'SELECTED', value: '01OCT2026' }], map)).toEqual(
|
||||
[]
|
||||
)
|
||||
})
|
||||
|
||||
it('reports a partly unparseable value set instead of covering the rest', () => {
|
||||
// One good value and one that is not a date: the clause still filters on
|
||||
// the good one, so the author has to be told the other was dropped.
|
||||
const map = new Map([
|
||||
[
|
||||
'selected',
|
||||
{
|
||||
column: 'SOME_DATE',
|
||||
transform: {
|
||||
op: 'ADD' as const,
|
||||
anchor: 'QUARTER_START' as const,
|
||||
offset: -1
|
||||
}
|
||||
}
|
||||
]
|
||||
])
|
||||
const problems = service.unsupportedTransforms(
|
||||
msg([{ label: 'SELECTED', value: ['01OCT2026', 'not a date'] }]),
|
||||
map,
|
||||
cols
|
||||
)
|
||||
expect(problems).toHaveSize(1)
|
||||
expect(problems[0]).toContain('1 of its 2 values was not a date')
|
||||
})
|
||||
|
||||
it('reports a range holding a value that does not parse', () => {
|
||||
// Counted on what VA sent, not on what parsed: otherwise the second value
|
||||
// vanishes and the range looks like a clean single-value case.
|
||||
const map = new Map([
|
||||
[
|
||||
'selected',
|
||||
{
|
||||
column: 'SOME_DATE',
|
||||
transform: {
|
||||
op: 'RANGE' as const,
|
||||
period: 'QUARTER' as const,
|
||||
offset: -1
|
||||
}
|
||||
}
|
||||
]
|
||||
])
|
||||
const problems = service.unsupportedTransforms(
|
||||
msg([{ label: 'SELECTED', value: ['01APR2026', 'not a date'] }]),
|
||||
map,
|
||||
cols
|
||||
)
|
||||
expect(problems).toHaveSize(1)
|
||||
expect(problems[0]).toContain('is a range')
|
||||
expect(problems[0]).toContain('2 values')
|
||||
})
|
||||
|
||||
it('applies a transform on the deferred initial load too', () => {
|
||||
const map = new Map([
|
||||
[
|
||||
'selected',
|
||||
{
|
||||
column: 'SOME_DATE',
|
||||
transform: {
|
||||
op: 'ADD' as const,
|
||||
anchor: 'QUARTER_START' as const,
|
||||
offset: -1
|
||||
}
|
||||
}
|
||||
]
|
||||
])
|
||||
const [clause] = service.buildInitialClauses(
|
||||
msg([{ name: 'pr1', label: 'SELECTED', value: '01OCT2026' }]),
|
||||
map
|
||||
)
|
||||
expect(clause.VARIABLE_NM).toBe('SOME_DATE')
|
||||
expect(clause.RAW_VALUE).toBe('(24380,24288)')
|
||||
})
|
||||
|
||||
it('behaves exactly as before with an empty map', () => {
|
||||
expect(
|
||||
service.buildClauses(
|
||||
msg([{ label: 'SOME_CHAR', value: 'x' }]),
|
||||
labelToVar(),
|
||||
cols
|
||||
)
|
||||
).toEqual(build([{ label: 'SOME_CHAR', value: 'x' }], new Map()))
|
||||
})
|
||||
})
|
||||
|
||||
describe('unresolvedMappings (what the report author is warned about)', () => {
|
||||
const report = (
|
||||
parameters: any[],
|
||||
map: Map<string, VaParmapTarget>,
|
||||
libds?: string
|
||||
) => service.unresolvedMappings(msg(parameters), map, cols, libds)
|
||||
|
||||
it('reports a mapping whose column is not in the loaded table', () => {
|
||||
expect(
|
||||
report(
|
||||
[{ label: 'REGION_TAB1', value: 'x' }],
|
||||
new Map([['region_tab1', { column: 'NOT_A_COLUMN' }]]),
|
||||
'DC_JSLIB.MPE_X_TEST'
|
||||
)
|
||||
).toEqual([
|
||||
'VA parameter "REGION_TAB1" is mapped to NOT_A_COLUMN, which is not a column of DC_JSLIB.MPE_X_TEST - the filter was ignored.'
|
||||
])
|
||||
})
|
||||
|
||||
it('says "this table" when the table name is not supplied', () => {
|
||||
expect(
|
||||
report(
|
||||
[{ label: 'REGION_TAB1', value: 'x' }],
|
||||
new Map([['region_tab1', { column: 'NOPE' }]])
|
||||
)[0]
|
||||
).toContain('is not a column of this table')
|
||||
})
|
||||
|
||||
it('reports nothing when the mapping resolves, or the param is unmapped', () => {
|
||||
expect(
|
||||
report(
|
||||
[{ label: 'REGION_TAB1', value: 'x' }],
|
||||
new Map([['region_tab1', { column: 'some_char' }]])
|
||||
)
|
||||
).toEqual([])
|
||||
expect(report([{ label: 'SOME_CHAR', value: 'x' }], new Map())).toEqual(
|
||||
[]
|
||||
)
|
||||
// A mapping whose parameter this message does not carry is not an error.
|
||||
expect(
|
||||
report([], new Map([['region_tab1', { column: 'NOPE' }]]))
|
||||
).toEqual([])
|
||||
})
|
||||
|
||||
it('reports each mapping once, not once per message parameter', () => {
|
||||
const problems = report(
|
||||
[
|
||||
{ label: 'REGION_TAB1', value: '1' },
|
||||
{ name: 'pr99', label: 'REGION_TAB1', value: '2' }
|
||||
],
|
||||
new Map([['region_tab1', { column: 'NOPE' }]])
|
||||
)
|
||||
expect(problems).toHaveSize(1)
|
||||
})
|
||||
|
||||
it('treats an unloaded column list as "not known yet", not as a bad mapping', () => {
|
||||
// While the grid is still loading there is no metadata to judge against, so
|
||||
// reporting would flash the banner on every load.
|
||||
expect(
|
||||
service.unresolvedMappings(
|
||||
msg([{ label: 'REGION_TAB1', value: 'x' }]),
|
||||
new Map([['region_tab1', { column: 'NOPE' }]]),
|
||||
[],
|
||||
'DC_JSLIB.MPE_X_TEST'
|
||||
)
|
||||
).toEqual([])
|
||||
expect(
|
||||
service.unresolvedMappings(
|
||||
msg([{ label: 'REGION_TAB1', value: 'x' }]),
|
||||
new Map([['region_tab1', { column: 'NOPE' }]]),
|
||||
undefined as any,
|
||||
'DC_JSLIB.MPE_X_TEST'
|
||||
)
|
||||
).toEqual([])
|
||||
})
|
||||
})
|
||||
|
||||
describe('sameColumnMappings (two parameters, one column)', () => {
|
||||
it('reports an AND-ed pair, naming both parameters and the column', () => {
|
||||
const map = new Map([
|
||||
['dropdown_tab1', { column: 'SOME_DROPDOWN' }],
|
||||
['dropdown_tab2', { column: 'SOME_DROPDOWN' }]
|
||||
])
|
||||
const problems = service.sameColumnMappings(
|
||||
msg([
|
||||
{ label: 'DROPDOWN_TAB1', value: 'Option 1' },
|
||||
{ label: 'DROPDOWN_TAB2', value: 'Option 2' }
|
||||
]),
|
||||
map
|
||||
)
|
||||
expect(problems).toHaveSize(1)
|
||||
expect(problems[0]).toContain('"DROPDOWN_TAB1" and "DROPDOWN_TAB2"')
|
||||
expect(problems[0]).toContain('all filter SOME_DROPDOWN')
|
||||
expect(problems[0]).toContain('combined with AND')
|
||||
})
|
||||
|
||||
it('says nothing for one parameter per column', () => {
|
||||
const map = new Map([
|
||||
['dropdown_tab1', { column: 'SOME_DROPDOWN' }],
|
||||
['num_tab1', { column: 'SOME_NUM' }]
|
||||
])
|
||||
expect(
|
||||
service.sameColumnMappings(
|
||||
msg([
|
||||
{ label: 'DROPDOWN_TAB1', value: 'Option 1' },
|
||||
{ label: 'NUM_TAB1', value: '1' }
|
||||
]),
|
||||
map
|
||||
)
|
||||
).toEqual([])
|
||||
expect(
|
||||
service.sameColumnMappings(
|
||||
msg([{ label: 'DROPDOWN_TAB1', value: 'Option 1' }]),
|
||||
map
|
||||
)
|
||||
).toEqual([])
|
||||
expect(service.sameColumnMappings(msg([]), map)).toEqual([])
|
||||
})
|
||||
|
||||
it('says nothing when the AND-ed parameters carry the same value', () => {
|
||||
// col = 'x' AND col = 'x' filters exactly as one clause does, so a banner
|
||||
// claiming "different values will show no rows" would be wrong.
|
||||
const map = new Map([
|
||||
['dropdown_tab1', { column: 'SOME_CHAR' }],
|
||||
['dropdown_tab2', { column: 'SOME_CHAR' }]
|
||||
])
|
||||
expect(
|
||||
service.sameColumnMappings(
|
||||
msg([
|
||||
{ label: 'DROPDOWN_TAB1', value: 'x' },
|
||||
{ label: 'DROPDOWN_TAB2', value: 'x' }
|
||||
]),
|
||||
map,
|
||||
cols
|
||||
)
|
||||
).toEqual([])
|
||||
// One differing value and the intersection can be empty: reported.
|
||||
const problems = service.sameColumnMappings(
|
||||
msg([
|
||||
{ label: 'DROPDOWN_TAB1', value: 'x' },
|
||||
{ label: 'DROPDOWN_TAB2', value: 'y' }
|
||||
]),
|
||||
map,
|
||||
cols
|
||||
)
|
||||
expect(problems).toHaveSize(1)
|
||||
expect(problems[0]).toContain('"DROPDOWN_TAB1" and "DROPDOWN_TAB2"')
|
||||
})
|
||||
|
||||
it('leaves a column the table does not have to unresolvedMappings', () => {
|
||||
// One cause, one line: the column is missing, so the AND story is moot.
|
||||
const map = new Map([
|
||||
['a', { column: 'NOPE' }],
|
||||
['b', { column: 'NOPE' }]
|
||||
])
|
||||
const cols = [
|
||||
{ LIBREF: 'DC_JSLIB', TABLE: 'MPE_X_TEST', VARIABLE: 'SOME_CHAR' }
|
||||
]
|
||||
expect(
|
||||
service.sameColumnMappings(
|
||||
msg([
|
||||
{ label: 'A', value: '1' },
|
||||
{ label: 'B', value: '2' }
|
||||
]),
|
||||
map,
|
||||
cols
|
||||
)
|
||||
).toEqual([])
|
||||
// Without metadata the pair is still reported: the AND semantics hold
|
||||
// whatever the table turns out to be.
|
||||
expect(
|
||||
service.sameColumnMappings(
|
||||
msg([
|
||||
{ label: 'A', value: '1' },
|
||||
{ label: 'B', value: '2' }
|
||||
]),
|
||||
map
|
||||
)
|
||||
).toHaveSize(1)
|
||||
})
|
||||
|
||||
it('reports one line per contended column', () => {
|
||||
const map = new Map([
|
||||
['a', { column: 'SOME_DROPDOWN' }],
|
||||
['b', { column: 'SOME_DROPDOWN' }],
|
||||
['c', { column: 'SOME_NUM' }],
|
||||
['d', { column: 'SOME_NUM' }]
|
||||
])
|
||||
const problems = service.sameColumnMappings(
|
||||
msg([
|
||||
{ label: 'A', value: '1' },
|
||||
{ label: 'B', value: '2' },
|
||||
{ label: 'C', value: '3' },
|
||||
{ label: 'D', value: '4' }
|
||||
]),
|
||||
map
|
||||
)
|
||||
expect(problems).toHaveSize(2)
|
||||
})
|
||||
})
|
||||
|
||||
describe('unmatchedMappings (parameter-side typo)', () => {
|
||||
const seen = (...keys: string[]) => new Set(keys)
|
||||
|
||||
it('reports a key that no message has named', () => {
|
||||
expect(
|
||||
service.unmatchedMappings(
|
||||
new Map([['dropdown_tab1', { column: 'SOME_CHAR' }]]),
|
||||
seen('dropdown_tab_1', 'pr1'),
|
||||
true
|
||||
)
|
||||
).toEqual([
|
||||
'This object\'s va_parmap names VA parameter "dropdown_tab1", which VA is not sending - check the spelling.'
|
||||
])
|
||||
})
|
||||
|
||||
it('says nothing until a parameter has actually been seen', () => {
|
||||
// Without evidence that this object receives parameters at all, an absent
|
||||
// key proves nothing - VA sends only the parameters its query uses.
|
||||
expect(
|
||||
service.unmatchedMappings(
|
||||
new Map([['dropdown_tab1', { column: 'SOME_CHAR' }]]),
|
||||
seen()
|
||||
)
|
||||
).toEqual([])
|
||||
expect(
|
||||
service.unmatchedMappings(
|
||||
new Map([['dropdown_tab1', { column: 'SOME_CHAR' }]]),
|
||||
undefined
|
||||
)
|
||||
).toEqual([])
|
||||
})
|
||||
|
||||
it('says nothing when every key has been seen', () => {
|
||||
expect(
|
||||
service.unmatchedMappings(
|
||||
new Map([['dropdown_tab1', { column: 'SOME_CHAR' }]]),
|
||||
seen('dropdown_tab1'),
|
||||
true
|
||||
)
|
||||
).toEqual([])
|
||||
expect(
|
||||
service.unmatchedMappings(new Map(), seen('anything'), true)
|
||||
).toEqual([])
|
||||
})
|
||||
|
||||
it('says nothing while the parameter set is still growing', () => {
|
||||
// A message that introduces a new parameter name means this object's set
|
||||
// has not settled, so an absent key may arrive in the next message.
|
||||
expect(
|
||||
service.unmatchedMappings(
|
||||
new Map([['dropdown_tab1', { column: 'SOME_CHAR' }]]),
|
||||
seen('pr1', 'pr2'),
|
||||
false
|
||||
)
|
||||
).toEqual([])
|
||||
})
|
||||
})
|
||||
|
||||
describe('buildInitialClauses (no metadata, infer kind from value)', () => {
|
||||
it('uses an explicit mapping instead of the label', () => {
|
||||
// The deferred first load has no column metadata, so a mapped column is
|
||||
// used as given and the SAS kind still comes from the value shape.
|
||||
const map = new Map([
|
||||
['region_tab1', { column: 'REGION' }],
|
||||
['amt_tab1', { column: 'AMOUNT' }]
|
||||
])
|
||||
const clauses = service.buildInitialClauses(
|
||||
msg([
|
||||
{ name: 'pr1', label: 'REGION_TAB1', dataType: 'string', value: 'n' },
|
||||
{ name: 'pr2', label: 'AMT_TAB1', dataType: 'number', value: '10' }
|
||||
]),
|
||||
map
|
||||
)
|
||||
expect(clauses.map((c) => [c.VARIABLE_NM, c.RAW_VALUE])).toEqual([
|
||||
['REGION', "'n'"],
|
||||
['AMOUNT', '10']
|
||||
])
|
||||
})
|
||||
|
||||
it('falls back to the label for params the map does not name', () => {
|
||||
const map = new Map([['region_tab2', { column: 'REGION' }]])
|
||||
const clauses = service.buildInitialClauses(
|
||||
msg([
|
||||
{ name: 'pr1', label: 'REGION_TAB1', value: 'n' },
|
||||
{ name: 'pr2', label: 'REGION_TAB2', value: 's' }
|
||||
]),
|
||||
map
|
||||
)
|
||||
expect(clauses.map((c) => [c.VARIABLE_NM, c.RAW_VALUE])).toEqual([
|
||||
['REGION_TAB1', "'n'"],
|
||||
['REGION', "'s'"]
|
||||
])
|
||||
})
|
||||
|
||||
it('uses the VA label as the column and infers kinds by value shape', () => {
|
||||
const clauses = service.buildInitialClauses(
|
||||
msg([
|
||||
|
||||
@@ -2,12 +2,6 @@ import { Injectable } from '@angular/core'
|
||||
import { HelperService } from './helper.service'
|
||||
import { VaMessage } from './va-messaging.service'
|
||||
import { QueryClause } from '../models/TableData'
|
||||
import { VaParmapTarget } from '../shared/utils/parse-va-parmap'
|
||||
import {
|
||||
applyVaTransform,
|
||||
isRangeTransform,
|
||||
VaTransform
|
||||
} from '../shared/utils/va-parmap-transform'
|
||||
|
||||
/** SAS data-type kind used to format a VA filter value into a DC RAW_VALUE. */
|
||||
type VaColumnKind = 'char' | 'numeric' | 'time' | 'date' | 'datetime'
|
||||
@@ -60,19 +54,11 @@ export class VaFilterService {
|
||||
* Translates VA `parameters` into DC filter clauses. Each parameter is matched
|
||||
* to a DC column by label; a single value becomes `col = value`, multiple
|
||||
* values become `col IN (...)`. Values are formatted for the column's SAS kind.
|
||||
*
|
||||
* `paramMap` (from the URL `va_parmap` parameter, see parseVaParmap) overrides
|
||||
* that same-name resolution for the parameters it names, so two DC instances
|
||||
* over one table can filter the same column from differently-named VA
|
||||
* parameters. A mapped column must exist in `cols` - if it does not the
|
||||
* parameter is skipped rather than falling back to the same-name match, which
|
||||
* is exactly the behaviour the mapping exists to override.
|
||||
*/
|
||||
buildClauses(
|
||||
msg: VaMessage,
|
||||
labelToVar: Map<string, string>,
|
||||
cols: any[],
|
||||
paramMap?: Map<string, VaParmapTarget>
|
||||
cols: any[]
|
||||
): QueryClause[] {
|
||||
const clauses: QueryClause[] = []
|
||||
for (const param of msg.parameters || []) {
|
||||
@@ -81,194 +67,60 @@ export class VaFilterService {
|
||||
.toString()
|
||||
.trim()
|
||||
.toLowerCase()
|
||||
// An explicit URL mapping wins over the label->column lookup. The mapped
|
||||
// column has to exist in THIS table, else the entry does not apply here
|
||||
// (and the same-name fallback would be the behaviour it exists to
|
||||
// override).
|
||||
const mapped = this.mappedColumn(param, paramMap)
|
||||
const mappedCol = mapped ? this.findCol(mapped, cols) : undefined
|
||||
if (mapped && !mappedCol) continue
|
||||
// The table's own case, so the clause matches what the metadata build
|
||||
// produces for a same-name match.
|
||||
const varName: string = mappedCol
|
||||
? mappedCol.NAME
|
||||
: (labelToVar.get(key) ?? '')
|
||||
// A VA param whose label isn't a DC column (beyond case, which is handled)
|
||||
// simply doesn't map — skip it.
|
||||
const varName = labelToVar.get(key)
|
||||
if (!varName) continue
|
||||
|
||||
const col = mappedCol ?? this.findCol(varName, cols)
|
||||
const col = this.findCol(varName, cols)
|
||||
const kind = this.columnKind(col)
|
||||
const order = this.dateOrderFromFmt(col?.FMTNAME || col?.FORMAT)
|
||||
const built = this.buildClauseValueSet(
|
||||
param,
|
||||
kind,
|
||||
order,
|
||||
this.mappedTarget(param, paramMap)?.transform
|
||||
)
|
||||
if (!built) continue
|
||||
clauses.push({
|
||||
GROUP_LOGIC: 'AND',
|
||||
SUBGROUP_LOGIC: 'AND',
|
||||
SUBGROUP_ID: 0,
|
||||
VARIABLE_NM: varName,
|
||||
OPERATOR_NM: built.operator,
|
||||
RAW_VALUE: built.rawValue
|
||||
})
|
||||
const rawValues = Array.isArray(param.value) ? param.value : [param.value]
|
||||
// Format/validate every value for the column's SAS type. Numeric columns
|
||||
// must receive a finite number (untrusted postMessage input is
|
||||
// interpolated into a SAS WHERE clause, so a non-numeric value here would
|
||||
// be an injection vector); temporal -> SAS internal number; char ->
|
||||
// escaped. Invalid values are dropped.
|
||||
const formatted = rawValues
|
||||
.filter((v) => v !== undefined && v !== null && v !== '')
|
||||
.map((v) => this.formatValueByKind(v, kind, order))
|
||||
.filter((v): v is string => v !== null)
|
||||
if (formatted.length === 0) continue
|
||||
clauses.push(this.clause(varName, formatted))
|
||||
}
|
||||
return clauses
|
||||
}
|
||||
|
||||
/**
|
||||
* The operator and raw value for one parameter, after any transform.
|
||||
*
|
||||
* Values are formatted for the column's SAS kind: numeric columns must receive
|
||||
* a finite number (untrusted postMessage input is interpolated into a WHERE
|
||||
* clause, so anything else would be an injection vector); temporal values
|
||||
* become SAS internal numbers; char values are escaped. Invalid values are
|
||||
* dropped, and a transform that cannot be applied to the column's kind yields
|
||||
* nothing at all - the caller reports that separately rather than filtering on
|
||||
* a value the author did not choose.
|
||||
*/
|
||||
private buildClauseValueSet(
|
||||
param: any,
|
||||
kind: VaColumnKind,
|
||||
order: 'YMD' | 'DMY' | 'MDY' | undefined,
|
||||
transform?: VaTransform
|
||||
): { operator: '=' | 'IN' | 'BETWEEN'; rawValue: string } | null {
|
||||
const rawValues = (
|
||||
Array.isArray(param.value) ? param.value : [param.value]
|
||||
).filter((v: any) => v !== undefined && v !== null && v !== '')
|
||||
if (rawValues.length === 0) return null
|
||||
|
||||
if (!transform) {
|
||||
const formatted = rawValues
|
||||
.map((v: any) => this.formatValueByKind(v, kind, order))
|
||||
.filter((v: string | null): v is string => v !== null)
|
||||
if (formatted.length === 0) return null
|
||||
return formatted.length === 1
|
||||
? { operator: '=', rawValue: formatted[0] }
|
||||
: { operator: 'IN', rawValue: `(${formatted.join(',')})` }
|
||||
}
|
||||
|
||||
// Transforms resolve to dates, so only temporal columns can take one.
|
||||
if (kind !== 'date' && kind !== 'datetime') return null
|
||||
const unit = kind === 'datetime' ? 'seconds' : 'days'
|
||||
const values = new Set<string>()
|
||||
for (const raw of rawValues) {
|
||||
const resolved = this.valueAsDate(raw, kind, order)
|
||||
if (!resolved) continue
|
||||
for (const date of applyVaTransform(
|
||||
resolved,
|
||||
transform,
|
||||
kind === 'datetime'
|
||||
)) {
|
||||
// A non-finite result (an Invalid Date reaching the helper) must never be
|
||||
// interpolated into the clause: `NaN` is not a SAS value. Dropped here,
|
||||
// and reported by unsupportedTransforms so the parameter is not silently
|
||||
// filtered on nothing.
|
||||
const sasDate = this.helperService.convertJsDateToSasDate(date, unit)
|
||||
if (!Number.isFinite(Number(sasDate))) continue
|
||||
values.add(`${sasDate}`)
|
||||
}
|
||||
}
|
||||
const formatted = Array.from(values)
|
||||
if (formatted.length === 0) return null
|
||||
if (isRangeTransform(transform)) {
|
||||
// One BETWEEN carries one period, so more than one value has no clause -
|
||||
// dropping it here (rather than building a BETWEEN from whichever value
|
||||
// parsed) is what unsupportedTransforms' count warning then describes.
|
||||
if (rawValues.length > 1) return null
|
||||
if (formatted.length !== 2) return null
|
||||
return { operator: 'BETWEEN', rawValue: formatted.join(' AND ') }
|
||||
}
|
||||
return formatted.length === 1
|
||||
? { operator: '=', rawValue: formatted[0] }
|
||||
: { operator: 'IN', rawValue: `(${formatted.join(',')})` }
|
||||
}
|
||||
|
||||
/**
|
||||
* A VA value as a local Date, for a transform to work from. Formatted strings
|
||||
* are parsed by shape; a bare number is already the SAS internal value, so it
|
||||
* is converted back from days (date) or seconds (datetime) since 1960-01-01.
|
||||
*
|
||||
* Built with local calendar construction, matching `parseDate` and
|
||||
* `HelperService.convertJsDateToSasDate`: a UTC instant read back through local
|
||||
* accessors lands on the previous day west of UTC, and the round trip to a SAS
|
||||
* day number would be one out.
|
||||
*/
|
||||
private valueAsDate(
|
||||
value: any,
|
||||
kind: VaColumnKind,
|
||||
order?: 'YMD' | 'DMY' | 'MDY'
|
||||
): Date | null {
|
||||
const s = String(value).trim()
|
||||
if (s === '') return null
|
||||
if (this.isBareNumber(s)) {
|
||||
const internal = Number(s)
|
||||
if (kind !== 'datetime') return new Date(1960, 0, 1 + internal)
|
||||
// Seconds since midnight 1960-01-01, split into whole days and a remainder
|
||||
// so the constructor normalises the clock time locally.
|
||||
const days = Math.floor(internal / 86400)
|
||||
const seconds = internal - days * 86400
|
||||
return new Date(1960, 0, 1 + days, 0, 0, seconds)
|
||||
}
|
||||
return kind === 'datetime'
|
||||
? this.parseDatetime(s, order)
|
||||
: this.parseDate(s, order)
|
||||
}
|
||||
|
||||
/**
|
||||
* Pre-getdata variant used on the deferred initial load, when DC column
|
||||
* metadata isn't available. Uses the VA `label` as the column name and infers
|
||||
* the SAS kind from the value shape, so the clause matches what the metadata
|
||||
* build later produces (no reconcile reload).
|
||||
*
|
||||
* `paramMap` overrides the label-as-column rule for the parameters it names,
|
||||
* so the very first (already filtered) load also honours an explicit mapping.
|
||||
*/
|
||||
buildInitialClauses(
|
||||
msg: VaMessage,
|
||||
paramMap?: Map<string, VaParmapTarget>
|
||||
): QueryClause[] {
|
||||
buildInitialClauses(msg: VaMessage): QueryClause[] {
|
||||
const clauses: QueryClause[] = []
|
||||
for (const param of msg.parameters || []) {
|
||||
if (!param || param.value === undefined || param.value === null) continue
|
||||
const target = this.mappedTarget(param, paramMap)
|
||||
// Without a mapping the DC column name is the VA `label`; `name` is VA's
|
||||
// internal id (pr###). A `va_parmap` entry wins over both.
|
||||
const varName = (target?.column ?? param.label ?? param.name ?? '')
|
||||
.toString()
|
||||
.trim()
|
||||
// The DC column name is the VA `label`; `name` is VA's internal id (pr###).
|
||||
const varName = (param.label ?? param.name ?? '').toString().trim()
|
||||
if (!varName) continue
|
||||
|
||||
const rawValues = (
|
||||
Array.isArray(param.value) ? param.value : [param.value]
|
||||
) as any[]
|
||||
const rawValues = Array.isArray(param.value) ? param.value : [param.value]
|
||||
const nonEmpty = rawValues.filter(
|
||||
(v) => v !== undefined && v !== null && v !== ''
|
||||
)
|
||||
if (nonEmpty.length === 0) continue
|
||||
|
||||
// A transform only ever produces dates, so it fixes the kind: inferring it
|
||||
// from the value would see a formatted date either way, but with the
|
||||
// transform the column is known to be temporal.
|
||||
const kind = target?.transform
|
||||
? ('date' as VaColumnKind)
|
||||
: this.inferKindFromValue(String(nonEmpty[0]).trim(), param.dataType)
|
||||
const built = this.buildClauseValueSet(
|
||||
param,
|
||||
kind,
|
||||
undefined,
|
||||
target?.transform
|
||||
)
|
||||
if (!built) continue
|
||||
clauses.push({
|
||||
GROUP_LOGIC: 'AND',
|
||||
SUBGROUP_LOGIC: 'AND',
|
||||
SUBGROUP_ID: 0,
|
||||
VARIABLE_NM: varName,
|
||||
OPERATOR_NM: built.operator,
|
||||
RAW_VALUE: built.rawValue
|
||||
})
|
||||
const formatted = nonEmpty
|
||||
.map((v) =>
|
||||
this.formatValueByKind(
|
||||
v,
|
||||
this.inferKindFromValue(String(v).trim(), param.dataType)
|
||||
)
|
||||
)
|
||||
.filter((v): v is string => v !== null)
|
||||
if (formatted.length === 0) continue
|
||||
clauses.push(this.clause(varName, formatted))
|
||||
}
|
||||
return clauses
|
||||
}
|
||||
@@ -289,254 +141,16 @@ export class VaFilterService {
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Transform mappings in this message that cannot be applied, one plain-English
|
||||
* line each: the column is not temporal, or the value is not a date the browser
|
||||
* can parse. `buildClauseValueSet` skips those parameters, so this is what stops
|
||||
* the misconfiguration reading as a filter that simply matched nothing.
|
||||
*
|
||||
* A missing column is left to `unresolvedMappings`, which already reports it.
|
||||
*/
|
||||
unsupportedTransforms(
|
||||
msg: VaMessage,
|
||||
paramMap: Map<string, VaParmapTarget> | undefined,
|
||||
cols: any[]
|
||||
): string[] {
|
||||
if (!paramMap || paramMap.size === 0) return []
|
||||
if (!cols || cols.length === 0) return []
|
||||
const problems = new Set<string>()
|
||||
for (const param of msg.parameters || []) {
|
||||
const target = this.mappedTarget(param, paramMap)
|
||||
if (!target?.transform) continue
|
||||
const transform = target.transform
|
||||
const col = this.findCol(target.column, cols)
|
||||
if (!col) continue
|
||||
const kind = this.columnKind(col)
|
||||
const paramName = (param.label ?? param.name ?? '').toString().trim()
|
||||
const spec =
|
||||
target.transform.op === 'ADD'
|
||||
? `ADD(${target.transform.anchor},${target.transform.offset})`
|
||||
: `RANGE(${target.transform.period},${target.transform.offset})`
|
||||
if (kind !== 'date' && kind !== 'datetime') {
|
||||
problems.add(
|
||||
`VA parameter "${paramName}" maps to ${target.column}:${spec}, but ${target.column} is not a date or datetime column - the filter was ignored.`
|
||||
)
|
||||
continue
|
||||
}
|
||||
const order = this.dateOrderFromFmt(col?.FMTNAME || col?.FORMAT)
|
||||
const sent = (
|
||||
Array.isArray(param.value) ? param.value : [param.value]
|
||||
).filter((v: any) => v !== undefined && v !== null && v !== '')
|
||||
const usable = sent.filter((v: any) => {
|
||||
const date = this.valueAsDate(v, kind, order)
|
||||
if (!date) return false
|
||||
// Same test the clause builder applies, so this count matches what it uses.
|
||||
return applyVaTransform(date, transform, kind === 'datetime').every(
|
||||
(resolved) => {
|
||||
const sasDate = this.helperService.convertJsDateToSasDate(
|
||||
resolved,
|
||||
kind === 'datetime' ? 'seconds' : 'days'
|
||||
)
|
||||
return Number.isFinite(Number(sasDate))
|
||||
}
|
||||
)
|
||||
})
|
||||
if (usable.length === 0) {
|
||||
problems.add(
|
||||
`VA parameter "${paramName}" maps to ${target.column}:${spec}, but its value was not a date - the filter was ignored.`
|
||||
)
|
||||
continue
|
||||
}
|
||||
// One BETWEEN can only express one period. A multi-select would have to
|
||||
// become an OR of ranges, which the clause model does not carry, so say so
|
||||
// rather than building a BETWEEN from whichever value happened to parse.
|
||||
// Counted on what VA sent: a second value that does not parse must not make
|
||||
// the single-value case look clean.
|
||||
if (isRangeTransform(target.transform) && sent.length > 1) {
|
||||
problems.add(
|
||||
`VA parameter "${paramName}" maps to ${target.column}:${spec}, which is a range, but the parameter has ${sent.length} values - a range needs one value, so the filter was ignored.`
|
||||
)
|
||||
continue
|
||||
}
|
||||
if (usable.length < sent.length) {
|
||||
problems.add(
|
||||
`VA parameter "${paramName}" maps to ${target.column}:${spec}, but ${sent.length - usable.length} of its ${sent.length} values was not a date - the filter covers only the rest.`
|
||||
)
|
||||
}
|
||||
private clause(varName: string, formatted: string[]): QueryClause {
|
||||
return {
|
||||
GROUP_LOGIC: 'AND',
|
||||
SUBGROUP_LOGIC: 'AND',
|
||||
SUBGROUP_ID: 0,
|
||||
VARIABLE_NM: varName,
|
||||
OPERATOR_NM: formatted.length === 1 ? '=' : 'IN',
|
||||
RAW_VALUE:
|
||||
formatted.length === 1 ? formatted[0] : `(${formatted.join(',')})`
|
||||
}
|
||||
return Array.from(problems)
|
||||
}
|
||||
|
||||
/**
|
||||
* `va_parmap` entries that this message actually uses but that name a column
|
||||
* the loaded table does not have, one plain-English line each. `buildClauses`
|
||||
* skips those parameters (deliberately: falling back to the same-name match
|
||||
* would silently apply the filter the mapping exists to override), so this is
|
||||
* what makes the misconfiguration visible to the report author - otherwise the
|
||||
* grid simply shows every row, which looks like a working filter.
|
||||
*/
|
||||
unresolvedMappings(
|
||||
msg: VaMessage,
|
||||
paramMap: Map<string, VaParmapTarget> | undefined,
|
||||
cols: any[],
|
||||
libds?: string
|
||||
): string[] {
|
||||
const problems = new Set<string>()
|
||||
if (!paramMap || paramMap.size === 0) return []
|
||||
// No metadata yet is not the same as "the table lacks this column": an empty
|
||||
// `cols` means the grid is still loading, so there is nothing to judge yet.
|
||||
// Reporting here would flash a misconfiguration banner during every load.
|
||||
if (!cols || cols.length === 0) return []
|
||||
for (const param of msg.parameters || []) {
|
||||
const mapped = this.mappedColumn(param, paramMap)
|
||||
if (!mapped || this.findCol(mapped, cols)) continue
|
||||
const paramName = (param.label ?? param.name ?? '').toString().trim()
|
||||
problems.add(
|
||||
`VA parameter "${paramName}" is mapped to ${mapped}, which is not a column of ${
|
||||
libds || 'this table'
|
||||
} - the filter was ignored.`
|
||||
)
|
||||
}
|
||||
return Array.from(problems)
|
||||
}
|
||||
|
||||
/**
|
||||
* `va_parmap` keys that no VA message has named, one plain-English line each.
|
||||
* A key that matches nothing is a parameter-side typo: the mapping never
|
||||
* applies and the grid stays unfiltered, which looks the same as a working
|
||||
* filter.
|
||||
*
|
||||
* `seenKeys` must be every key observed across messages so far. `paramsStable`
|
||||
* says the latest message introduced no new parameter name, i.e. this object's
|
||||
* parameter set has settled - until then a missing name may simply not have
|
||||
* arrived, because VA sends only the parameters its query uses.
|
||||
*/
|
||||
unmatchedMappings(
|
||||
paramMap: Map<string, VaParmapTarget> | undefined,
|
||||
seenKeys: Set<string> | undefined,
|
||||
paramsStable = false
|
||||
): string[] {
|
||||
if (!paramMap || paramMap.size === 0) return []
|
||||
if (!seenKeys || seenKeys.size === 0) return []
|
||||
if (!paramsStable) return []
|
||||
const problems: string[] = []
|
||||
for (const key of Array.from(paramMap.keys())) {
|
||||
if (!seenKeys.has(key)) {
|
||||
problems.push(
|
||||
`This object's va_parmap names VA parameter "${key}", which VA is not sending - check the spelling.`
|
||||
)
|
||||
}
|
||||
}
|
||||
return problems
|
||||
}
|
||||
|
||||
/**
|
||||
* Parameters in this message that filter the same column with *different*
|
||||
* values, one plain-English line per column. Clauses are AND-ed, so two
|
||||
* parameters on one column are an intersection: with different values the grid
|
||||
* comes back empty, which looks like a working filter. Two parameters carrying
|
||||
* the same value intersect cleanly and filter correctly, so they are not
|
||||
* reported - a banner there would call a working configuration broken.
|
||||
*
|
||||
* Reported rather than rewritten - folding them into one `IN (...)` would
|
||||
* silently turn an AND into an OR, which is a semantics decision, not a
|
||||
* diagnostic.
|
||||
*/
|
||||
sameColumnMappings(
|
||||
msg: VaMessage,
|
||||
paramMap: Map<string, VaParmapTarget> | undefined,
|
||||
cols?: any[]
|
||||
): string[] {
|
||||
if (!paramMap || paramMap.size === 0) return []
|
||||
// column -> the parameters on it, each with the clause value it produces
|
||||
const byColumn = new Map<string, { name: string; value: string }[]>()
|
||||
for (const param of msg.parameters || []) {
|
||||
const mapped = this.mappedColumn(param, paramMap)
|
||||
if (!mapped) continue
|
||||
// A column the table lacks is unresolvedMappings' finding; counting it here
|
||||
// would report one cause as two symptoms.
|
||||
const col =
|
||||
cols && cols.length > 0 ? this.findCol(mapped, cols) : undefined
|
||||
if (cols && cols.length > 0 && !col) continue
|
||||
// A parameter that produces no clause cannot conflict with anything.
|
||||
const value = this.valueSignature(
|
||||
param,
|
||||
col,
|
||||
this.mappedTarget(param, paramMap)?.transform
|
||||
)
|
||||
if (value === null) continue
|
||||
const column = mapped.toUpperCase()
|
||||
const params = byColumn.get(column) || []
|
||||
const paramName = (param.label ?? '').toString().trim()
|
||||
if (paramName && !params.some((p) => p.name === paramName)) {
|
||||
params.push({ name: paramName, value })
|
||||
}
|
||||
byColumn.set(column, params)
|
||||
}
|
||||
const problems: string[] = []
|
||||
for (const [column, params] of Array.from(byColumn.entries())) {
|
||||
if (params.length < 2) continue
|
||||
// Same value on every parameter: the AND is a no-op, not an empty grid.
|
||||
const distinct = new Set(params.map((p) => p.value))
|
||||
if (distinct.size === 1) continue
|
||||
problems.push(
|
||||
`VA parameters ${params
|
||||
.map((p) => `"${p.name}"`)
|
||||
.join(
|
||||
' and '
|
||||
)} all filter ${column}. Parameters are combined with AND, so rows must match all of them - different values will show no rows. Use one parameter carrying both values to match either.`
|
||||
)
|
||||
}
|
||||
return problems
|
||||
}
|
||||
|
||||
/**
|
||||
* The DC column an explicit URL mapping (`va_parmap`) assigns to this
|
||||
* parameter, or undefined when the parameter is not mapped. The key is matched
|
||||
* case-insensitively against the parameter's `label`.
|
||||
*/
|
||||
private mappedTarget(
|
||||
param: any,
|
||||
paramMap?: Map<string, VaParmapTarget>
|
||||
): VaParmapTarget | undefined {
|
||||
if (!paramMap || paramMap.size === 0) return undefined
|
||||
// The label only. The internal name (pr273) is an implementation detail of
|
||||
// the report, and matching both made one parameter answer to two keys: a
|
||||
// mapping written against the name could never be distinguished from a typo
|
||||
// once the label was also mapped, and the second entry silently lost.
|
||||
const key = (param.label ?? '').toString().trim().toLowerCase()
|
||||
if (!key) return undefined
|
||||
return paramMap.get(key)
|
||||
}
|
||||
|
||||
/**
|
||||
* The clause value a parameter would filter its column with, as a comparable
|
||||
* string, or null when it produces no clause at all. Used to tell an AND of two
|
||||
* identical values (harmless) from an AND of two different ones (empty grid).
|
||||
*
|
||||
* Delegates to the clause builder so a transform's derived dates take part in
|
||||
* the comparison: two parameters adding the same quarter are the same clause.
|
||||
*/
|
||||
private valueSignature(
|
||||
param: any,
|
||||
col: any,
|
||||
transform?: VaTransform
|
||||
): string | null {
|
||||
const built = this.buildClauseValueSet(
|
||||
param,
|
||||
this.columnKind(col),
|
||||
this.dateOrderFromFmt(col?.FMTNAME || col?.FORMAT),
|
||||
transform
|
||||
)
|
||||
return built ? built.rawValue : null
|
||||
}
|
||||
|
||||
/** The mapped column name, for callers that do not need the transform. */
|
||||
private mappedColumn(
|
||||
param: any,
|
||||
paramMap?: Map<string, VaParmapTarget>
|
||||
): string | undefined {
|
||||
return this.mappedTarget(param, paramMap)?.column
|
||||
}
|
||||
|
||||
/** The loaded column spec whose NAME matches varName (case-insensitive). */
|
||||
@@ -707,42 +321,14 @@ export class VaFilterService {
|
||||
return this.parseMonthNameDate(value) ?? this.parseNumericDate(value, order)
|
||||
}
|
||||
|
||||
/**
|
||||
* Month-name date shapes -> local Date, or null. Three families:
|
||||
* - DATE9 "ddMMM[yy]yy" (e.g. 01AUG1962, 01AUG62);
|
||||
* - word date with the month first, comma optional: "January 1, 2026",
|
||||
* "Jan 1 2026", "Jan. 1, 2026";
|
||||
* - word date with the day first: "1 January 2026", "1-Jan-2026".
|
||||
*
|
||||
* A date parameter's display format decides which of these the report author
|
||||
* sees, and VA sends the formatted text, so a report using WORDDATE (or a
|
||||
* long month name in any order) has to parse here rather than being dropped.
|
||||
* The month name may be abbreviated or spelled out; only its first three
|
||||
* letters are matched.
|
||||
*/
|
||||
/** DATE9 "ddMMM[yy]yy" (e.g. 01AUG1962, 01AUG62) -> local Date, or null. */
|
||||
private parseMonthNameDate(value: string): Date | null {
|
||||
const monthByName = (name: string): number =>
|
||||
VaFilterService.SAS_MONTHS.indexOf(
|
||||
name.replace(/\./g, '').slice(0, 3).toUpperCase()
|
||||
)
|
||||
|
||||
const date9 = /^(\d{1,2})([A-Za-z]{3})(\d{2}|\d{4})$/.exec(value)
|
||||
if (date9) {
|
||||
const [, day, monthAbbr, year] = date9
|
||||
const monthIndex = monthByName(monthAbbr)
|
||||
if (monthIndex < 0) return null
|
||||
return this.buildDate(this.fullYear(year), monthIndex + 1, Number(day))
|
||||
}
|
||||
|
||||
const monthFirst =
|
||||
/^([A-Za-z]{3,9})\.?[\s-]+(\d{1,2}),?[\s-]+(\d{2}|\d{4})$/.exec(value)
|
||||
const dayFirst =
|
||||
/^(\d{1,2})[\s-]+([A-Za-z]{3,9})\.?,?[\s-]+(\d{2}|\d{4})$/.exec(value)
|
||||
const name = monthFirst ? monthFirst[1] : dayFirst ? dayFirst[2] : null
|
||||
if (!name) return null
|
||||
const day = monthFirst ? monthFirst[2] : dayFirst![1]
|
||||
const year = monthFirst ? monthFirst[3] : dayFirst![3]
|
||||
const monthIndex = monthByName(name)
|
||||
const m = /^(\d{1,2})([A-Za-z]{3})(\d{2}|\d{4})$/.exec(value)
|
||||
if (!m) return null
|
||||
const [, day, monthAbbr, year] = m
|
||||
const monthIndex = VaFilterService.SAS_MONTHS.indexOf(
|
||||
monthAbbr.toUpperCase()
|
||||
)
|
||||
if (monthIndex < 0) return null
|
||||
return this.buildDate(this.fullYear(year), monthIndex + 1, Number(day))
|
||||
}
|
||||
|
||||
@@ -1,6 +1,4 @@
|
||||
import { VaMessagingService, VaMessage } from './va-messaging.service'
|
||||
import { VaFilterService } from './va-filter.service'
|
||||
import { HelperService } from './helper.service'
|
||||
|
||||
// Sample message taken from the official SAS DDC docs (brush column included).
|
||||
const SAMPLE = {
|
||||
@@ -53,57 +51,6 @@ describe('VaMessagingService', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('parameter keys (what the va_parmap typo diagnostic is judged on)', () => {
|
||||
const PARAMS = [
|
||||
{ name: 'pr273', label: 'Tab_1_Date', value: '2025-03-31' },
|
||||
{ name: 'pr274', label: 'Tab 2 Region', value: 'north' }
|
||||
]
|
||||
const withParams = () => ({ ...SAMPLE, parameters: PARAMS })
|
||||
|
||||
it('records each parameter label once, across messages', () => {
|
||||
service.parseMessage({ data: withParams() } as MessageEvent)
|
||||
service.parseMessage({ data: withParams() } as MessageEvent)
|
||||
|
||||
expect(Array.from(service.seenParamKeys)).toEqual([
|
||||
'tab_1_date',
|
||||
'tab 2 region'
|
||||
])
|
||||
// The second message introduced no new label, so the set has settled.
|
||||
expect(service.lastMessageParamsStable).toBeTrue()
|
||||
})
|
||||
|
||||
it('records the label only, so a mapping written against the internal name is reported', () => {
|
||||
// `pr273` is VA's own bookkeeping. A va_parmap entry written against it
|
||||
// never matches a parameter, and recording the name here would make the
|
||||
// typo diagnostic believe VA is sending it - leaving the author with an
|
||||
// unfiltered grid and no warning at all.
|
||||
service.parseMessage({ data: withParams() } as MessageEvent)
|
||||
service.parseMessage({ data: withParams() } as MessageEvent)
|
||||
|
||||
const filter = new VaFilterService(new HelperService({} as any))
|
||||
const seen = service.seenParamKeys
|
||||
const stable = service.lastMessageParamsStable
|
||||
|
||||
expect(
|
||||
filter.unmatchedMappings(
|
||||
new Map([['pr273', { column: 'D_DATE' }]]),
|
||||
seen,
|
||||
stable
|
||||
)
|
||||
).toEqual([
|
||||
'This object\'s va_parmap names VA parameter "pr273", which VA is not sending - check the spelling.'
|
||||
])
|
||||
// The label is a real key, so the same map written that way is silent.
|
||||
expect(
|
||||
filter.unmatchedMappings(
|
||||
new Map([['tab_1_date', { column: 'D_DATE' }]]),
|
||||
seen,
|
||||
stable
|
||||
)
|
||||
).toEqual([])
|
||||
})
|
||||
})
|
||||
|
||||
describe('dataColumns', () => {
|
||||
it('excludes brush columns and keeps original indices', () => {
|
||||
const msg = service.parseMessage({ data: SAMPLE } as MessageEvent)!
|
||||
|
||||
@@ -82,26 +82,6 @@ export class VaMessagingService {
|
||||
/** DC variable names that should be visible (the rest are hidden). */
|
||||
visibleColumns: string[] | null = null
|
||||
|
||||
/**
|
||||
* Lower-cased label of every parameter any VA message has carried so far, kept
|
||||
* across editor reloads. Lets the editor tell a `va_parmap` key that names no
|
||||
* parameter in the report (a typo, so the filter would silently never apply)
|
||||
* from one that is merely absent from the current message - VA only sends the
|
||||
* parameters its query uses.
|
||||
*
|
||||
* Labels only: that is the key a mapping is matched on, and recording the
|
||||
* internal name as well would make a name-keyed entry - which never applies -
|
||||
* look like a parameter VA is sending.
|
||||
*/
|
||||
seenParamKeys = new Set<string>()
|
||||
/**
|
||||
* True when every parameter name in the latest message had already been seen.
|
||||
* VA sends only the parameters a report object's query uses, and sends them per
|
||||
* object, so a name that is absent from the first messages may simply not have
|
||||
* arrived yet. A stable parameter set is what makes absence meaningful.
|
||||
*/
|
||||
lastMessageParamsStable = false
|
||||
|
||||
/** True once the pre-bootstrap early buffer has been drained (drain-once). */
|
||||
private earlyDrained = false
|
||||
|
||||
@@ -207,22 +187,6 @@ export class VaMessagingService {
|
||||
if (typeof data.resultName !== 'string') return null
|
||||
if (!Array.isArray(data.columns) || !Array.isArray(data.data)) return null
|
||||
|
||||
const parameters = Array.isArray(data.parameters) ? data.parameters : []
|
||||
const keys: string[] = []
|
||||
for (const param of parameters) {
|
||||
// The label only, which is the key a `va_parmap` entry is matched on. The
|
||||
// internal name (`pr273`) is the report's own bookkeeping: recording it
|
||||
// here would let the typo diagnostic below believe a name-keyed entry is
|
||||
// being sent when no code path can ever apply it.
|
||||
const key = (param?.label ?? '').toString().trim().toLowerCase()
|
||||
if (key && !keys.includes(key)) keys.push(key)
|
||||
}
|
||||
// Judged before the keys are recorded: a message that introduces a new name
|
||||
// means the set is still growing, so a missing name proves nothing yet.
|
||||
this.lastMessageParamsStable =
|
||||
keys.length > 0 && keys.every((key) => this.seenParamKeys.has(key))
|
||||
for (const key of keys) this.seenParamKeys.add(key)
|
||||
|
||||
return {
|
||||
version: data.version,
|
||||
resultName: data.resultName,
|
||||
@@ -230,7 +194,7 @@ export class VaMessagingService {
|
||||
availableRowCount: data.availableRowCount,
|
||||
data: data.data,
|
||||
columns: data.columns,
|
||||
parameters
|
||||
parameters: Array.isArray(data.parameters) ? data.parameters : []
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,215 +0,0 @@
|
||||
import { parseVaParmap } from './parse-va-parmap'
|
||||
|
||||
describe('parseVaParmap', () => {
|
||||
it('returns an empty map and no problems when the parameter is absent', () => {
|
||||
for (const query of ['', 'embed=va', 'embed=va&labels=true']) {
|
||||
const parsed = parseVaParmap(query)
|
||||
expect(parsed.map.size).toBe(0)
|
||||
expect(parsed.problems).toEqual([])
|
||||
}
|
||||
})
|
||||
|
||||
it('maps a VA parameter label to a DC column', () => {
|
||||
const { map, problems } = parseVaParmap(
|
||||
'embed=va&va_parmap=REGION_TAB1-REGION'
|
||||
)
|
||||
expect(map.get('region_tab1')?.column).toBe('REGION')
|
||||
expect(map.size).toBe(1)
|
||||
expect(problems).toEqual([])
|
||||
})
|
||||
|
||||
it('collects repeated entries, so two parameters can map to one column', () => {
|
||||
// The multi-tab case: each DC instance filters the same DC column from a
|
||||
// differently-named VA parameter.
|
||||
const { map } = parseVaParmap(
|
||||
'embed=va&va_parmap=REGION_TAB1-REGION&va_parmap=REGION_TAB2-REGION'
|
||||
)
|
||||
expect(map.get('region_tab1')?.column).toBe('REGION')
|
||||
expect(map.get('region_tab2')?.column).toBe('REGION')
|
||||
expect(map.size).toBe(2)
|
||||
})
|
||||
|
||||
it('is case-insensitive on the key and trims both sides', () => {
|
||||
const { map } = parseVaParmap('va_parmap=%20Region_Tab1%20-%20region%20')
|
||||
expect(map.get('region_tab1')?.column).toBe('region')
|
||||
})
|
||||
|
||||
it('decodes a key containing a space (VA labels are free text)', () => {
|
||||
expect(
|
||||
parseVaParmap('va_parmap=My+Region-DEPT').map.get('my region')?.column
|
||||
).toBe('DEPT')
|
||||
})
|
||||
|
||||
it('splits at the LAST hyphen, so a label may contain hyphens', () => {
|
||||
expect(
|
||||
parseVaParmap('va_parmap=Sales-2024-DEPT').map.get('sales-2024')?.column
|
||||
).toBe('DEPT')
|
||||
})
|
||||
|
||||
describe('entries that cannot be used', () => {
|
||||
it('reports an entry with no separator', () => {
|
||||
const { map, problems } = parseVaParmap('va_parmap=DEPT')
|
||||
expect(map.size).toBe(0)
|
||||
expect(problems).toEqual([
|
||||
'va_parmap "DEPT" was ignored: it needs a "-" between the VA parameter and the column name.'
|
||||
])
|
||||
})
|
||||
|
||||
it('reports an entry with no parameter name', () => {
|
||||
const { map, problems } = parseVaParmap('va_parmap=-DEPT')
|
||||
expect(map.size).toBe(0)
|
||||
expect(problems).toEqual([
|
||||
'va_parmap "-DEPT" was ignored: the VA parameter name is missing.'
|
||||
])
|
||||
})
|
||||
|
||||
it('reports an entry with no column name', () => {
|
||||
const { map, problems } = parseVaParmap('va_parmap=REGION_TAB1-')
|
||||
expect(map.size).toBe(0)
|
||||
expect(problems).toEqual([
|
||||
'va_parmap "REGION_TAB1-" was ignored: "" is not a valid SAS column name.'
|
||||
])
|
||||
})
|
||||
|
||||
it('reports a column that is not a plain SAS name (injection guard)', () => {
|
||||
// The column is interpolated into a SAS WHERE clause, so anything beyond a
|
||||
// SAS name is refused at parse time rather than carried into a clause.
|
||||
for (const entry of [
|
||||
'A-x;drop',
|
||||
'A-REGION%20OR%201=1',
|
||||
"A-RE'GION",
|
||||
'A-1ABC'
|
||||
]) {
|
||||
const { map, problems } = parseVaParmap(`va_parmap=${entry}`)
|
||||
expect(map.size).toBe(0)
|
||||
expect(problems.length).toBe(1)
|
||||
expect(problems[0]).toContain('is not a valid SAS column name')
|
||||
}
|
||||
expect(parseVaParmap('va_parmap=A-A_B9').map.get('a')?.column).toBe(
|
||||
'A_B9'
|
||||
)
|
||||
})
|
||||
|
||||
it('keeps the usable entries and reports only the bad one', () => {
|
||||
const { map, problems } = parseVaParmap(
|
||||
'va_parmap=GOOD-REGION&va_parmap=BROKEN'
|
||||
)
|
||||
expect(map.get('good')?.column).toBe('REGION')
|
||||
expect(problems.length).toBe(1)
|
||||
expect(problems[0]).toContain('va_parmap "BROKEN"')
|
||||
})
|
||||
})
|
||||
|
||||
describe('transforms', () => {
|
||||
it('parses an ADD anchor with an offset', () => {
|
||||
const { map, problems } = parseVaParmap(
|
||||
'va_parmap=SELECTED-D_DATE:ADD(QUARTER_START,-1)'
|
||||
)
|
||||
expect(problems).toEqual([])
|
||||
expect(map.get('selected')?.column).toBe('D_DATE')
|
||||
expect(map.get('selected')?.transform).toEqual({
|
||||
op: 'ADD',
|
||||
anchor: 'QUARTER_START',
|
||||
offset: -1
|
||||
})
|
||||
})
|
||||
|
||||
it('defaults the offset to zero and accepts lower case', () => {
|
||||
const { map, problems } = parseVaParmap(
|
||||
'va_parmap=sel-d_date:add(quarter_start)'
|
||||
)
|
||||
expect(problems).toEqual([])
|
||||
expect(map.get('sel')?.transform).toEqual({
|
||||
op: 'ADD',
|
||||
anchor: 'QUARTER_START',
|
||||
offset: 0
|
||||
})
|
||||
})
|
||||
|
||||
it('parses a RANGE period', () => {
|
||||
const { map, problems } = parseVaParmap(
|
||||
'va_parmap=sel-D_DATE:RANGE(quarter,-1)'
|
||||
)
|
||||
expect(problems).toEqual([])
|
||||
expect(map.get('sel')?.transform).toEqual({
|
||||
op: 'RANGE',
|
||||
period: 'QUARTER',
|
||||
offset: -1
|
||||
})
|
||||
})
|
||||
|
||||
it('leaves a mapping without a transform as a plain column', () => {
|
||||
const { map } = parseVaParmap('va_parmap=sel-D_DATE')
|
||||
expect(map.get('sel')?.column).toBe('D_DATE')
|
||||
expect(map.get('sel')?.transform).toBeUndefined()
|
||||
})
|
||||
|
||||
it('reports an unknown operation, anchor, period or offset', () => {
|
||||
const cases: [string, string][] = [
|
||||
['sel-D_DATE:SHIFT(QUARTER_START,-1)', 'is not a known transform'],
|
||||
['sel-D_DATE:ADD(FORTNIGHT_START,-1)', 'is not a known anchor'],
|
||||
['sel-D_DATE:RANGE(DECADE,-1)', 'is not a known period'],
|
||||
['sel-D_DATE:ADD(QUARTER_START,3.5)', 'is not a whole number'],
|
||||
['sel-D_DATE:ADD', 'is not a transform']
|
||||
]
|
||||
for (const [entry, expected] of cases) {
|
||||
const { map, problems } = parseVaParmap(`va_parmap=${entry}`)
|
||||
expect(`${entry}: ${map.size}`).toBe(`${entry}: 0`)
|
||||
expect(`${entry}: ${problems.length}`).toBe(`${entry}: 1`)
|
||||
expect(problems[0]).toContain(expected)
|
||||
}
|
||||
})
|
||||
|
||||
it('keeps the parameter label free to contain a colon', () => {
|
||||
// The key side is untouched by the transform grammar: only the column side
|
||||
// is split, so a label with a colon still maps.
|
||||
const { map, problems } = parseVaParmap(
|
||||
'va_parmap=Sales:2026-D_DATE:ADD(QUARTER_START,-1)'
|
||||
)
|
||||
expect(problems).toEqual([])
|
||||
expect(map.get('sales:2026')?.column).toBe('D_DATE')
|
||||
expect(map.get('sales:2026')?.transform).toEqual({
|
||||
op: 'ADD',
|
||||
anchor: 'QUARTER_START',
|
||||
offset: -1
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
describe('one parameter, one column', () => {
|
||||
it('keeps the first mapping and reports a conflicting duplicate', () => {
|
||||
const { map, problems } = parseVaParmap(
|
||||
'va_parmap=DEPT-REGION&va_parmap=DEPT-OTHER'
|
||||
)
|
||||
expect(map.get('dept')?.column).toBe('REGION')
|
||||
expect(problems).toEqual([
|
||||
'va_parmap "DEPT-OTHER" was ignored: VA parameter "dept" already maps to REGION, and a parameter can filter only one column.'
|
||||
])
|
||||
})
|
||||
|
||||
it('treats a duplicate key as a duplicate across case', () => {
|
||||
const { map, problems } = parseVaParmap(
|
||||
'va_parmap=Dept-REGION&va_parmap=DEPT-SOMETHING_ELSE'
|
||||
)
|
||||
expect(map.get('dept')?.column).toBe('REGION')
|
||||
expect(problems.length).toBe(1)
|
||||
})
|
||||
|
||||
it('accepts the same mapping written twice (no ambiguity)', () => {
|
||||
const { map, problems } = parseVaParmap(
|
||||
'va_parmap=DEPT-REGION&va_parmap=dept-region'
|
||||
)
|
||||
expect(map.get('dept')?.column).toBe('REGION')
|
||||
expect(problems).toEqual([])
|
||||
})
|
||||
|
||||
it('still allows two parameters to map to one column', () => {
|
||||
// The multi-object case: different parameters, same DC column.
|
||||
const { map, problems } = parseVaParmap(
|
||||
'va_parmap=REGION_TAB1-REGION&va_parmap=REGION_TAB2-REGION'
|
||||
)
|
||||
expect(map.size).toBe(2)
|
||||
expect(problems).toEqual([])
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -1,192 +0,0 @@
|
||||
import {
|
||||
VA_ANCHORS,
|
||||
VA_PERIODS,
|
||||
VaAnchor,
|
||||
VaPeriod,
|
||||
VaTransform
|
||||
} from './va-parmap-transform'
|
||||
|
||||
/**
|
||||
* What a VA parameter's mapping resolves to: the DC column, and optionally a
|
||||
* transform that adds derived dates to the filter (e.g. the previous quarter's
|
||||
* row alongside the selected one).
|
||||
*/
|
||||
export interface VaParmapTarget {
|
||||
column: string
|
||||
transform?: VaTransform
|
||||
}
|
||||
|
||||
/** Result of `parseVaParmap` - the usable mappings, and what was ignored. */
|
||||
export interface VaParmap {
|
||||
/** Lower-cased VA parameter key -> DC column (and transform, if any). */
|
||||
map: Map<string, VaParmapTarget>
|
||||
/** One plain-English line per `va_parmap` entry that was ignored. */
|
||||
problems: string[]
|
||||
}
|
||||
|
||||
/**
|
||||
* Parses the `va_parmap` entries out of a hash query string (the part after `?`
|
||||
* in `window.location.hash`).
|
||||
*
|
||||
* Format - one mapping per occurrence, each `<VA parameter key>-<DC column>`:
|
||||
*
|
||||
* ?embed=va&va_parmap=REGION_TAB1-REGION&va_parmap=REGION_TAB2-REGION
|
||||
*
|
||||
* Repeating the parameter (rather than packing a comma-separated list into one
|
||||
* value) keeps the grammar unambiguous: VA parameter labels are free text, so
|
||||
* any list separator we picked could also appear inside a label.
|
||||
*
|
||||
* Each entry is split at its LAST `-`. A SAS column name can never contain a
|
||||
* hyphen, so the final one always delimits the column, and a label that does
|
||||
* contain hyphens (e.g. `Sales-2024`) still maps correctly.
|
||||
*
|
||||
* The key is matched case-insensitively, and trailing/leading blanks are
|
||||
* trimmed, against a VA parameter's `label`, which is the same key the
|
||||
* label->column match uses. The internal `name` (e.g. `pr273`) is not a key: it
|
||||
* is the report's own bookkeeping, so an entry written against it never applies
|
||||
* and is reported by `unmatchedMappings`.
|
||||
*
|
||||
* An entry that cannot be used is reported in `problems` rather than dropped in
|
||||
* silence: these entries are hand-written in the report's DDC URL, usually by
|
||||
* the report author, and a typo would otherwise degrade to "no filter" - which
|
||||
* looks exactly like a working filter over the whole table.
|
||||
*
|
||||
* Returns an empty map when the parameter is absent, which leaves the default
|
||||
* same-name resolution untouched (existing embeds are unaffected).
|
||||
*
|
||||
* Modeled on `parseEmbedParam` / `parseLabelsParam`, which read the same hash
|
||||
* query.
|
||||
*/
|
||||
export function parseVaParmap(hashQuery: string): VaParmap {
|
||||
const map = new Map<string, VaParmapTarget>()
|
||||
const problems: string[] = []
|
||||
if (!hashQuery) return { map, problems }
|
||||
|
||||
const entries = new URLSearchParams(hashQuery).getAll('va_parmap')
|
||||
for (const entry of entries) {
|
||||
// A transform sits at the end, in parentheses: `<param>-<column>:OP(args)`.
|
||||
// It is lifted off FIRST, because its arguments may contain a hyphen (a
|
||||
// negative offset) and the key/column split below uses the hyphen.
|
||||
let body = entry ?? ''
|
||||
let transformSpec = ''
|
||||
if (body.trim().endsWith(')')) {
|
||||
const open = body.lastIndexOf('(')
|
||||
const colon = open > 0 ? body.lastIndexOf(':', open) : -1
|
||||
if (colon > 0) {
|
||||
transformSpec = body.slice(colon + 1).trim()
|
||||
body = body.slice(0, colon)
|
||||
}
|
||||
}
|
||||
// Last hyphen delimits the DC column (see above).
|
||||
const separator = body.lastIndexOf('-')
|
||||
if (separator < 0) {
|
||||
problems.push(
|
||||
`va_parmap "${entry}" was ignored: it needs a "-" between the VA parameter and the column name.`
|
||||
)
|
||||
continue
|
||||
}
|
||||
const key = body.slice(0, separator).trim().toLowerCase()
|
||||
const column = body.slice(separator + 1).trim()
|
||||
if (!key) {
|
||||
problems.push(
|
||||
`va_parmap "${entry}" was ignored: the VA parameter name is missing.`
|
||||
)
|
||||
continue
|
||||
}
|
||||
if (column.includes(':')) {
|
||||
problems.push(
|
||||
`va_parmap "${entry}" was ignored: "${column}" is not a transform - expected e.g. ADD(QUARTER_START,-1).`
|
||||
)
|
||||
continue
|
||||
}
|
||||
// The column name is interpolated into a SAS WHERE clause, so only a plain
|
||||
// SAS name is accepted - anything else is refused rather than mapped.
|
||||
if (!/^[A-Za-z_][A-Za-z0-9_]{0,31}$/.test(column)) {
|
||||
problems.push(
|
||||
`va_parmap "${entry}" was ignored: "${column}" is not a valid SAS column name.`
|
||||
)
|
||||
continue
|
||||
}
|
||||
const transform = transformSpec
|
||||
? parseTransform(transformSpec, entry, problems)
|
||||
: undefined
|
||||
if (transformSpec && !transform) continue
|
||||
// One VA parameter filters one column. A second entry for the same parameter
|
||||
// is ambiguous, so the FIRST one wins and the conflict is reported - the
|
||||
// alternative (last one wins, silently) would apply a filter the author
|
||||
// never sees the origin of. Two parameters mapping to the same column is
|
||||
// fine and is how two report objects filter one column from their own
|
||||
// parameters.
|
||||
const existing = map.get(key)
|
||||
if (existing) {
|
||||
if (existing.column.toUpperCase() !== column.toUpperCase()) {
|
||||
problems.push(
|
||||
`va_parmap "${entry}" was ignored: VA parameter "${key}" already maps to ${existing.column}, and a parameter can filter only one column.`
|
||||
)
|
||||
}
|
||||
continue
|
||||
}
|
||||
map.set(key, { column, transform })
|
||||
}
|
||||
|
||||
return { map, problems }
|
||||
}
|
||||
|
||||
/**
|
||||
* Parses `OP(arg[, arg])`, returning undefined (and pushing a problem line) when
|
||||
* the operation or one of its arguments is not in the allow-list.
|
||||
*
|
||||
* ADD takes an anchor and an optional offset; RANGE takes a period and an
|
||||
* optional offset. The offset is whole periods, negative for earlier.
|
||||
*/
|
||||
function parseTransform(
|
||||
spec: string,
|
||||
entry: string,
|
||||
problems: string[]
|
||||
): VaTransform | undefined {
|
||||
const match = /^([A-Za-z]+)\s*\(([^)]*)\)$/.exec(spec)
|
||||
if (!match) {
|
||||
problems.push(
|
||||
`va_parmap "${entry}" was ignored: "${spec}" is not a transform - expected e.g. ADD(QUARTER_START,-1).`
|
||||
)
|
||||
return undefined
|
||||
}
|
||||
const op = match[1].toUpperCase()
|
||||
const args = match[2]
|
||||
.split(',')
|
||||
.map((arg) => arg.trim())
|
||||
.filter((arg) => arg.length > 0)
|
||||
const offsetArg = args[1] === undefined ? '0' : args[1]
|
||||
if (!/^-?\d+$/.test(offsetArg)) {
|
||||
problems.push(
|
||||
`va_parmap "${entry}" was ignored: the offset "${offsetArg}" is not a whole number of periods.`
|
||||
)
|
||||
return undefined
|
||||
}
|
||||
const offset = Number(offsetArg)
|
||||
|
||||
if (op === 'ADD') {
|
||||
const anchor = (args[0] || '').toUpperCase()
|
||||
if (!VA_ANCHORS.includes(anchor as VaAnchor)) {
|
||||
problems.push(
|
||||
`va_parmap "${entry}" was ignored: "${args[0] || ''}" is not a known anchor (${VA_ANCHORS.join(', ')}).`
|
||||
)
|
||||
return undefined
|
||||
}
|
||||
return { op: 'ADD', anchor: anchor as VaAnchor, offset }
|
||||
}
|
||||
if (op === 'RANGE') {
|
||||
const period = (args[0] || '').toUpperCase()
|
||||
if (!VA_PERIODS.includes(period as VaPeriod)) {
|
||||
problems.push(
|
||||
`va_parmap "${entry}" was ignored: "${args[0] || ''}" is not a known period (${VA_PERIODS.join(', ')}).`
|
||||
)
|
||||
return undefined
|
||||
}
|
||||
return { op: 'RANGE', period: period as VaPeriod, offset }
|
||||
}
|
||||
problems.push(
|
||||
`va_parmap "${entry}" was ignored: "${match[1]}" is not a known transform (ADD, RANGE).`
|
||||
)
|
||||
return undefined
|
||||
}
|
||||
@@ -1,194 +0,0 @@
|
||||
import {
|
||||
applyVaTransform,
|
||||
isRangeTransform,
|
||||
VA_ANCHORS,
|
||||
VA_PERIODS,
|
||||
VaTransform
|
||||
} from './va-parmap-transform'
|
||||
|
||||
/** Local y-m-d, so assertions do not depend on the runner's timezone offset. */
|
||||
const ymd = (d: Date) =>
|
||||
`${d.getFullYear()}-${String(d.getMonth() + 1).padStart(2, '0')}-${String(
|
||||
d.getDate()
|
||||
).padStart(2, '0')}`
|
||||
|
||||
const date = (y: number, m: number, d: number) => new Date(y, m - 1, d)
|
||||
|
||||
describe('va_parmap transforms', () => {
|
||||
it('exposes only allow-listed anchors and periods', () => {
|
||||
expect(VA_ANCHORS).toContain('QUARTER_START')
|
||||
expect(VA_PERIODS).toEqual(['QUARTER', 'MONTH', 'YEAR'])
|
||||
})
|
||||
|
||||
describe('ADD - the value plus one anchor beside it', () => {
|
||||
it('adds the previous quarter start, which is the reported requirement', () => {
|
||||
// intnx('qtr', dt, -1, 'b') for a value on a quarter boundary
|
||||
const dates = applyVaTransform(date(2026, 10, 1), {
|
||||
op: 'ADD',
|
||||
anchor: 'QUARTER_START',
|
||||
offset: -1
|
||||
})
|
||||
expect(dates.map(ymd)).toEqual(['2026-10-01', '2026-07-01'])
|
||||
})
|
||||
|
||||
it('resolves a mid-quarter value to its own value plus the previous quarter start', () => {
|
||||
const dates = applyVaTransform(date(2026, 8, 15), {
|
||||
op: 'ADD',
|
||||
anchor: 'QUARTER_START',
|
||||
offset: -1
|
||||
})
|
||||
expect(dates.map(ymd)).toEqual(['2026-08-15', '2026-04-01'])
|
||||
})
|
||||
|
||||
it('de-duplicates when the value already sits on the anchor', () => {
|
||||
// 2026-01-01 is a quarter start, so "0 quarters back" adds nothing
|
||||
const dates = applyVaTransform(date(2026, 1, 1), {
|
||||
op: 'ADD',
|
||||
anchor: 'QUARTER_START',
|
||||
offset: 0
|
||||
})
|
||||
expect(dates.map(ymd)).toEqual(['2026-01-01'])
|
||||
})
|
||||
|
||||
it('resolves the other anchors at their boundaries', () => {
|
||||
const at = date(2026, 8, 15)
|
||||
expect(
|
||||
ymd(
|
||||
applyVaTransform(at, {
|
||||
op: 'ADD',
|
||||
anchor: 'QUARTER_END',
|
||||
offset: 0
|
||||
})[1]
|
||||
)
|
||||
).toBe('2026-09-30')
|
||||
expect(
|
||||
ymd(
|
||||
applyVaTransform(at, { op: 'ADD', anchor: 'MONTH_END', offset: 0 })[1]
|
||||
)
|
||||
).toBe('2026-08-31')
|
||||
expect(
|
||||
ymd(
|
||||
applyVaTransform(at, {
|
||||
op: 'ADD',
|
||||
anchor: 'YEAR_START',
|
||||
offset: -1
|
||||
})[1]
|
||||
)
|
||||
).toBe('2025-01-01')
|
||||
expect(
|
||||
ymd(
|
||||
applyVaTransform(at, { op: 'ADD', anchor: 'YEAR_END', offset: 0 })[1]
|
||||
)
|
||||
).toBe('2026-12-31')
|
||||
})
|
||||
|
||||
it('handles February and leap years at month end', () => {
|
||||
expect(
|
||||
ymd(
|
||||
applyVaTransform(date(2026, 3, 1), {
|
||||
op: 'ADD',
|
||||
anchor: 'MONTH_END',
|
||||
offset: -1
|
||||
})[1]
|
||||
)
|
||||
).toBe('2026-02-28')
|
||||
expect(
|
||||
ymd(
|
||||
applyVaTransform(date(2024, 3, 15), {
|
||||
op: 'ADD',
|
||||
anchor: 'MONTH_END',
|
||||
offset: -1
|
||||
})[1]
|
||||
)
|
||||
).toBe('2024-02-29')
|
||||
})
|
||||
|
||||
it('crosses a year boundary when the offset takes it out of the year', () => {
|
||||
const dates = applyVaTransform(date(2027, 1, 1), {
|
||||
op: 'ADD',
|
||||
anchor: 'QUARTER_START',
|
||||
offset: -1
|
||||
})
|
||||
expect(dates.map(ymd)).toEqual(['2027-01-01', '2026-10-01'])
|
||||
})
|
||||
})
|
||||
|
||||
describe('RANGE - a closed period', () => {
|
||||
it('covers the previous quarter', () => {
|
||||
const dates = applyVaTransform(date(2026, 8, 15), {
|
||||
op: 'RANGE',
|
||||
period: 'QUARTER',
|
||||
offset: -1
|
||||
})
|
||||
expect(dates.map(ymd)).toEqual(['2026-04-01', '2026-06-30'])
|
||||
})
|
||||
|
||||
it('covers the current month and the previous year', () => {
|
||||
expect(
|
||||
applyVaTransform(date(2026, 8, 15), {
|
||||
op: 'RANGE',
|
||||
period: 'MONTH',
|
||||
offset: 0
|
||||
}).map(ymd)
|
||||
).toEqual(['2026-08-01', '2026-08-31'])
|
||||
expect(
|
||||
applyVaTransform(date(2026, 8, 15), {
|
||||
op: 'RANGE',
|
||||
period: 'YEAR',
|
||||
offset: -1
|
||||
}).map(ymd)
|
||||
).toEqual(['2025-01-01', '2025-12-31'])
|
||||
})
|
||||
|
||||
it('reports itself as a range so the caller picks BETWEEN', () => {
|
||||
expect(
|
||||
isRangeTransform({ op: 'RANGE', period: 'QUARTER', offset: 0 })
|
||||
).toBe(true)
|
||||
expect(
|
||||
isRangeTransform({
|
||||
op: 'ADD',
|
||||
anchor: 'QUARTER_START',
|
||||
offset: 0
|
||||
} as VaTransform)
|
||||
).toBe(false)
|
||||
})
|
||||
})
|
||||
|
||||
describe('time of day', () => {
|
||||
it('gives a datetime range whole days, not the value clock time', () => {
|
||||
// A BETWEEN that starts at 13:45 on day one loses the earlier part of that
|
||||
// day, and one that ends at 13:45 on the last day loses the rest of it.
|
||||
const value = new Date(2026, 7, 15, 13, 45, 30)
|
||||
const [start, end] = applyVaTransform(
|
||||
value,
|
||||
{ op: 'RANGE', period: 'QUARTER', offset: 0 },
|
||||
true
|
||||
)
|
||||
expect([
|
||||
start.getHours(),
|
||||
start.getMinutes(),
|
||||
start.getSeconds()
|
||||
]).toEqual([0, 0, 0])
|
||||
expect([end.getHours(), end.getMinutes(), end.getSeconds()]).toEqual([
|
||||
23, 59, 59
|
||||
])
|
||||
expect(ymd(start)).toBe('2026-07-01')
|
||||
expect(ymd(end)).toBe('2026-09-30')
|
||||
})
|
||||
|
||||
it('keeps the time for datetime columns and drops it otherwise', () => {
|
||||
const value = new Date(2026, 9, 1, 13, 45, 30)
|
||||
const add: VaTransform = {
|
||||
op: 'ADD',
|
||||
anchor: 'QUARTER_START',
|
||||
offset: -1
|
||||
}
|
||||
const kept = applyVaTransform(value, add, true)[1]
|
||||
expect([kept.getHours(), kept.getMinutes(), kept.getSeconds()]).toEqual([
|
||||
13, 45, 30
|
||||
])
|
||||
const dropped = applyVaTransform(value, add, false)[1]
|
||||
expect([dropped.getHours(), dropped.getMinutes()]).toEqual([0, 0])
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -1,151 +0,0 @@
|
||||
/**
|
||||
* Anchors and periods a `va_parmap` transform can resolve a report value to.
|
||||
*
|
||||
* The names describe the result rather than mirroring SAS `intnx`, because the
|
||||
* arithmetic runs in the browser: emulating `intnx`'s alignment letters (`b`,
|
||||
* `e`, `s`, `m`, never mind week alignment and fiscal quarters) would invite
|
||||
* assumptions we could not honour. Each anchor below is one small function, so
|
||||
* adding one is a function plus a test row.
|
||||
*/
|
||||
export const VA_ANCHORS = [
|
||||
'QUARTER_START',
|
||||
'QUARTER_END',
|
||||
'MONTH_START',
|
||||
'MONTH_END',
|
||||
'YEAR_START',
|
||||
'YEAR_END'
|
||||
] as const
|
||||
export type VaAnchor = (typeof VA_ANCHORS)[number]
|
||||
|
||||
export const VA_PERIODS = ['QUARTER', 'MONTH', 'YEAR'] as const
|
||||
export type VaPeriod = (typeof VA_PERIODS)[number]
|
||||
|
||||
/**
|
||||
* A parsed transform: the operation plus its validated arguments.
|
||||
*
|
||||
* `ADD` yields the value and one anchor beside it (a list of dates), which is
|
||||
* how "this period's row plus the previous quarter's" is expressed. `RANGE`
|
||||
* yields a closed period (two dates) for a BETWEEN.
|
||||
*/
|
||||
export type VaTransform =
|
||||
| { op: 'ADD'; anchor: VaAnchor; offset: number }
|
||||
| { op: 'RANGE'; period: VaPeriod; offset: number }
|
||||
|
||||
/** True when the transform produces a closed range rather than a list. */
|
||||
export function isRangeTransform(transform: VaTransform): boolean {
|
||||
return transform.op === 'RANGE'
|
||||
}
|
||||
|
||||
/** Local midnight on the first day of the month `offset` months from `date`. */
|
||||
function monthStart(date: Date, offset: number): Date {
|
||||
return new Date(date.getFullYear(), date.getMonth() + offset, 1)
|
||||
}
|
||||
|
||||
/** Local midnight on the last day of the month `offset` months from `date`. */
|
||||
function monthEnd(date: Date, offset: number): Date {
|
||||
return new Date(date.getFullYear(), date.getMonth() + offset + 1, 0)
|
||||
}
|
||||
|
||||
/** First day of the quarter `offset` quarters from the one containing `date`. */
|
||||
function quarterStart(date: Date, offset: number): Date {
|
||||
const firstMonthOfQuarter = 3 * Math.floor(date.getMonth() / 3)
|
||||
return new Date(date.getFullYear(), firstMonthOfQuarter + offset * 3, 1)
|
||||
}
|
||||
|
||||
/** Last day of the quarter `offset` quarters from the one containing `date`. */
|
||||
function quarterEnd(date: Date, offset: number): Date {
|
||||
// Day 0 of a month is the last day of the month before it: calendar
|
||||
// construction, so a daylight-saving transition inside the quarter cannot
|
||||
// shift the result by an hour the way subtracting 24 hours would.
|
||||
const nextStart = quarterStart(date, offset + 1)
|
||||
return new Date(nextStart.getFullYear(), nextStart.getMonth(), 0)
|
||||
}
|
||||
|
||||
/** First day of the year `offset` years from the one containing `date`. */
|
||||
function yearStart(date: Date, offset: number): Date {
|
||||
return new Date(date.getFullYear() + offset, 0, 1)
|
||||
}
|
||||
|
||||
/** Last day of the year `offset` years from the one containing `date`. */
|
||||
function yearEnd(date: Date, offset: number): Date {
|
||||
return new Date(date.getFullYear() + offset, 11, 31)
|
||||
}
|
||||
|
||||
/** Resolve one anchor, keeping the time of day of `date` for datetime columns. */
|
||||
function resolveAnchor(
|
||||
anchor: VaAnchor,
|
||||
date: Date,
|
||||
offset: number,
|
||||
keepTime: boolean
|
||||
): Date {
|
||||
let resolved: Date
|
||||
switch (anchor) {
|
||||
case 'QUARTER_START':
|
||||
resolved = quarterStart(date, offset)
|
||||
break
|
||||
case 'QUARTER_END':
|
||||
resolved = quarterEnd(date, offset)
|
||||
break
|
||||
case 'MONTH_START':
|
||||
resolved = monthStart(date, offset)
|
||||
break
|
||||
case 'MONTH_END':
|
||||
resolved = monthEnd(date, offset)
|
||||
break
|
||||
case 'YEAR_START':
|
||||
resolved = yearStart(date, offset)
|
||||
break
|
||||
default:
|
||||
resolved = yearEnd(date, offset)
|
||||
break
|
||||
}
|
||||
if (keepTime) {
|
||||
resolved.setHours(date.getHours(), date.getMinutes(), date.getSeconds(), 0)
|
||||
}
|
||||
return resolved
|
||||
}
|
||||
|
||||
/**
|
||||
* The dates a transform adds to a mapping, in order.
|
||||
*
|
||||
* `ADD` returns the report's own value plus the anchor beside it, de-duplicated -
|
||||
* so a value that already sits on the anchor yields one date, and the clause
|
||||
* stays an equality rather than an `IN` with a repeat. `RANGE` returns the
|
||||
* period's first and last day.
|
||||
*
|
||||
* `keepTime` carries the time of day across for datetime columns, since a
|
||||
* DATETIME filter compares to the second; date columns use midnight.
|
||||
*/
|
||||
export function applyVaTransform(
|
||||
value: Date,
|
||||
transform: VaTransform,
|
||||
keepTime = false
|
||||
): Date[] {
|
||||
if (transform.op === 'RANGE') {
|
||||
// A range covers whole days: the bounds are the period's own, never the
|
||||
// value's clock time. Reusing keepTime here would start the range mid-morning
|
||||
// on day one and end it mid-morning on the last day, dropping the rest of
|
||||
// both boundary days from a BETWEEN.
|
||||
const start = resolveAnchor(
|
||||
(transform.period + '_START') as VaAnchor,
|
||||
value,
|
||||
transform.offset,
|
||||
false
|
||||
)
|
||||
const end = resolveAnchor(
|
||||
(transform.period + '_END') as VaAnchor,
|
||||
value,
|
||||
transform.offset,
|
||||
false
|
||||
)
|
||||
if (keepTime) end.setHours(23, 59, 59, 0)
|
||||
return [start, end]
|
||||
}
|
||||
const anchor = resolveAnchor(
|
||||
transform.anchor,
|
||||
value,
|
||||
transform.offset,
|
||||
keepTime
|
||||
)
|
||||
return anchor.getTime() === value.getTime() ? [value] : [value, anchor]
|
||||
}
|
||||
@@ -5019,57 +5019,6 @@ body[cds-theme='dark'] {
|
||||
border: 1px solid var(--cds-alias-status-warning) !important;
|
||||
}
|
||||
|
||||
/* Derived (computed) columns: read-only, but computed at read time rather than
|
||||
locked by policy. Tinted and marked with an 'fx' glyph instead of the
|
||||
read-only dimming, so the two do not read as the same thing. Blue is the
|
||||
info family - the primary key green and the warning ochre keep their meaning. */
|
||||
.handsontable th.dc-derived-header {
|
||||
background: #d3e5f2 !important;
|
||||
box-shadow: inset 0 -2px 0 #1a6ea3;
|
||||
}
|
||||
|
||||
.handsontable th.dc-derived-header div::after {
|
||||
content: 'fx';
|
||||
margin-left: 4px;
|
||||
font-size: 9px;
|
||||
font-style: italic;
|
||||
opacity: 0.7;
|
||||
}
|
||||
|
||||
.handsontable td.dc-derived-cell {
|
||||
background: #eef5fa !important;
|
||||
font-style: italic;
|
||||
}
|
||||
|
||||
/* the first derived column starts the computed block */
|
||||
.handsontable th.dc-derived-first {
|
||||
border-left: 2px solid #1a6ea3 !important;
|
||||
}
|
||||
|
||||
.handsontable td.dc-derived-cell.dc-derived-first {
|
||||
box-shadow: inset 2px 0 0 #1a6ea3;
|
||||
}
|
||||
|
||||
body[cds-theme='dark'] {
|
||||
.handsontable th.dc-derived-header {
|
||||
background: #2b4a5c !important;
|
||||
box-shadow: inset 0 -2px 0 #4b9cc9;
|
||||
}
|
||||
|
||||
.handsontable td.dc-derived-cell {
|
||||
background: #24343d !important;
|
||||
color: #dbe7ee;
|
||||
}
|
||||
|
||||
.handsontable th.dc-derived-first {
|
||||
border-left-color: #4b9cc9 !important;
|
||||
}
|
||||
|
||||
.handsontable td.dc-derived-cell.dc-derived-first {
|
||||
box-shadow: inset 2px 0 0 #4b9cc9;
|
||||
}
|
||||
}
|
||||
|
||||
.handsontable .numericListbox {
|
||||
text-align: right;
|
||||
}
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "dcfrontend",
|
||||
"version": "7.21.0",
|
||||
"version": "7.18.0",
|
||||
"description": "Data Controller",
|
||||
"devDependencies": {
|
||||
"@nogoo9/gitleaks": "8.30.1-post.2",
|
||||
|
||||
@@ -11,6 +11,5 @@
|
||||
"noNestedMacros": true,
|
||||
"hasMacroParentheses": true,
|
||||
"strictMacroDefinition": true,
|
||||
"ignoredMacros": ["webout", "mpeinit", "mpeterm"],
|
||||
"ignoreList":["sasjs/scratch"]
|
||||
}
|
||||
@@ -1,10 +0,0 @@
|
||||
{
|
||||
"fromjs": [
|
||||
{
|
||||
"ADMIN": "all-users",
|
||||
"DCPATH": "/tmp/dcdata",
|
||||
"OUTFMT": "JSON",
|
||||
"KEEPME": "Y"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"fromjs": [
|
||||
{
|
||||
"ADMIN": "all-users",
|
||||
"ADMIN": "AllUsers",
|
||||
"DCPATH": "/tmp/dcdata",
|
||||
"OUTFMT": "JSON"
|
||||
}
|
||||
|
||||
@@ -1,4 +0,0 @@
|
||||
{
|
||||
"licenceKey": "G4H/DNTdEMkmCIxOrYkRYTQ1P4sWg0gs8Zk9rakxFNYDZcLyEPMJJo82YJQ/S24P2E/YPxaWsDZXgOrWKfeCDS7NrLcwsoFNa8KRF2WTlRgSMiG4sjBRX8zNEIvszj5DAq4LEgOUUAdfctNfpUcygbxT1dhqy6gU3hBu/xRuXGd36rhxQwfahaKkfaCg/rFplkrvqnjoJ6622BxWP/HG1dxZUE7HNJyT4tNaXzlC9U6u/aaJJ66+V+9511BwjM9ss4nyHuHoogVE1yrzLMlL94P3QoM1hwkGHnEHmFzg3zHbQG1f3pUs9YzK0ytsFKU7AK948gJrGbf37Zqa+Mz7EZf6XzqqXx0zQ4y9zAhLV9s4QG8JdhrByhfG5kefrs4L/BpgKq3ElgYL5EMkrLJUrlqWYI2ouxJuIGCQz41VCWXJ2S7pd39ljifc8Zj+t2Mn5EEkRVsrbdxjCkyoZOQ1ALofa1+auUI26Rm5rwEgrfhEocDmphJpO2F8lZgwDxbepSE6v/vEV/JQWBQTyrX10W+qOEu8OhvZrYJo+eIJMaO7q1pjfdrn3XhoH+BeB3Sy1sevGLXQZC+CzOtcvajetGGCDAabtTLc49ouWPUiUOAWPE3uYymsMoRZgpLIMxaUM86hoK0r9T6wSaP0bddK754sXN1jWwIDAHSGz5WO1ko=",
|
||||
"activationKey": "MIIJRAIBADANBgkqhkiG9w0BAQEFAASCCS4wggkqAgEAAoICAQD2w8Ge6tMI82vaQ7x5ZdlwnTJycn0UFZoSgRwObaM0NC5wp9Y8lwkVzN8FUhg2oYSTpvyJpix+VGRj0AlNBwFAVcJG9VDPYh2LE3Us9ghRfkzFUiM2PqorHnzL5DwpailRtmFHoDQPGwwSmKncTo2QHql/K6tWe7g3Nv9uqQNks283hvDQ6PVGU4TaFTSRbjGX4j+JAR651J6wI6nY+9kNxOBwnL/yKTnJGmL8oqZ+oC45quZzgRW4dLIeTODcB4IZe2ruxZ9tvXpSuAESiOv5bBZyMJEKdKwsRdt6vk+bezkZBCzPq+ly892Ncq7x5u4/8dGcGxRxxp5TnIIx3hFoiXZ7svXJ7st6uHuKkuwYq+ebWh6bcvQAwH2wCP40twMjQgmU8nHHo6+pulxu1wVZBxtoyM4m3Lu0ZncTi6mF0duiPvpkNNAZzKOF98aUq3hxYp5PP28qrtWuPsgSdTNWJwnV2R288yAPx4WLUAGB9h889d/m67HUiKadmfG4oyUnChCU/mqeJ/5nQkK1n76ezRPGsTcMZIv2mmPLShA/j38lGMjR/3pzM2Ha6UFjJKpurpeDiyt5pUdmRBpQRoBUx/PHnbXBsO9PBXa0mjdq3UFbgb5iBD1Fz2YClZ/h0d4YlYUMSeharDrLfR15g3fGnzxrkR03zYeHkxMUi5vYHQIDAQABAoICAFxUg5pcH8lOF1FMHfXIY5MM5mbHKsC8knERCPfnEvVB/wxc6D60fow/I0Vg4blzNuFuTjiMykIcfpGQMiK1l8TEMdSODoYCHq6+gY3ootUnLj94i79MtCw9FJHPnKkcVUiDNqPvr/rk8l/06j2J0NMXffEc3FeYJCXYuLtlluTzdI4wueNY/wkoz+EvpQMzWXAMkhxHpIEAqr2aIn+TQDQ0MYxBQlqwDTSPYHFJxIVFlvYx2VdnecKSO2E1/76kOowhTzH3FL+hcWcI/TG7tJuXU1C3llV0sUVBgC1NjwjMkuEm3EN5/ffqgj/eJNJTR1r6tqSzxzMJD42WVmqlPhqEmmRE1yeaio6Ulm5EJLREYEY4/1EotdIO6PCXF1Xeo+10VJLcwi19vzMlz7vFB6vSJzJNYOaiwm4rgmF12+nNt18cHoFvegGPAoCJurUhXU+iV6LlKYB8J/+kpITleZwm4dEaa6EWxgFf4nZ4N+qRVqk4JfC+rN3DQgbPcII+z7axAb8xutq5rvzskGAmKcTEagMG+pnk9DI+0m1C2J6+mC/YruVlpSlPB7Ecu/OF++vJoSCLuDXVuxMTQ3tQXoicOG/S8d1aa3nq4fvLIgZTOZoffUvNvjCyy/Dlv6vaDVCjgqt2kFZVSZv3C6ttyrNVBSjhucASGTq/7q82JKVrAoIBAQD7e7SYqDM8SAjOfpRAHCbIVVSaHw1tul16B6JAF67OIs1uD//Y+maL4GHlpFxnib0tg6z3+sG10NMUhK7elB6SkJ8voT+Ztg/+on659vmaqwTlkGjgks+8DmIw1b0ZbXuIqhnkOeSsll2V9NWItIrbzNwvhtMEQsOtFvmrLlKQvx9Bo3lGmbfdU1/9XpwW4wiA5ONcFCl1xllX9nItB+ActPoAeQgnuejWAISQkITDTOFcGu0AjMjJ66HUlY/WDbf7wY/s2aMbOiVx7B6YzQnK+lZdjjNfoaL8nfN8Z8ZzJFAgmTQ9ztZCJ8fLTfZDsYD6XZnVEIG401+6de0Ea4MHAoIBAQD7Mlr++keNDxYy3k6kwTXDOSKMDfXIDverFy/tiNorcVU5nIpuhjowl4HIQVAhzlXhATp938LM/PvJIeu9HtgsY9BFNL52fr7iGEWknqZQVE+8Um35tzV3vtPH9NVXHIYmcRCAQQJ5a6oVQRJKAM3E1+htM2AeZKMaQSCC6VHGBXJG9nEltdMKTet09pd26Z5y/YIfTqbIekyjRomGdInyWBkpJLJ/uROVb9OGnsYWadDktn7y9+oH+VZoJxW5QwXV0OL7wMzymlrKhhBNMziJRBk4rVztclYzUShpwE8Xfjt5WYkOoieXtmkdpF7TMvRYy6cdflh39l8s7THq4U67AoIBAQDPShR+kIZK9gVsUCHbBllhYBSwRVBIVgmfAJPYyKyp9T3nrwBw1VWZjoajR2tzXLG2xobYkLUDLPIQQ53wkZ7xqIvOMte917gvZ7mVh+IOf1E1+Ga6VcktdZ2yKzwHrZrZ93wIB0BakkbpyDoKyOZ8bsw/0J43MOEh14JMe45LCX62GA1rCGHp6UOCEW4P32uu8vwFdgkGr0wNmXJbc9ZjUzmPEjSKqk8ltp+OmdEVOlW4GempPmJzLeri07UP95bjyZTc4jfzxWYAFWTTBo8/5EE0NHZcKVuzZm7A6LlHR3NUq3PITqmFNk0VY1nOSo3J0UVvvn9u3axWgcWwkv4ZAoIBAQD5GaB/RfZ6fcGHxEi8vRTxedE2jglz0NPuCFpw9ctUVVksfn4QyG8gDL6YtZwfShyDutIkTSdCXIXbmRXLP7Ejo/+/AWT24Q2t7F/ztZg8BL/lguCoR/RvwP8HRVXiwtiNeoXgiZdIjWnnFdZrKIzlbS6CS/Y/bZEI0hjJeVKsSm5ezLi7qqmTiF28aWUeeOG2KotFPJvMJuuk5qgue5boilC5KjkQy5oOEQz0bLMHOvv6MreFfi/IpIv5UPkL9NjuhezxtuoGkucB+xxjUi34gV15hQGoPkAZN5wZ8qj8JSQZXJ67sjYNDskCqiOW0Orau2oHp2lX3SvYw4dC4xdNAoIBAQDSXF8OnWu08smGDs4S57yfqe59N0ULIBDOZdEQcLXRaDeAoOwQj8szUf0pycXUZLvXxv29iF61FJg70iGGCp3lW+9Oe0P+dGsKvzphRGsQPZy4h/qZDoQ9dE667ejI4aDb/C1GaOZ8TT1QfpQcVKbgLjdC19IGQPhFljGM1wMQojU5lp4/iKmA3J3ahOuV4dvEUr6xkqzq+PhH405XdURXsVc1J6qYX2wQXRoB0oDsrCbqidZi14DaeE8UhZRA6m9haUOb/3Y+OkCs2QIW4MNZ+zg8XtHjiUhMxAaJ6zdEFltWPoVFsVfgoecquAHb0uDJJRz/+RfR0AiG0XG0NW+C"
|
||||
}
|
||||
@@ -1159,26 +1159,6 @@ const tables = {
|
||||
selectbox_value: 'FORMULA',
|
||||
selectbox_order: 1,
|
||||
ver_to_dttm: 127490111999
|
||||
},
|
||||
{
|
||||
selectbox_rk: 73,
|
||||
ver_from_dttm: 0,
|
||||
select_lib: dcLibref,
|
||||
select_ds: 'MPE_DERIVED',
|
||||
base_column: 'VAR_TYPE',
|
||||
selectbox_value: 'C',
|
||||
selectbox_order: 1,
|
||||
ver_to_dttm: 127490111999
|
||||
},
|
||||
{
|
||||
selectbox_rk: 74,
|
||||
ver_from_dttm: 0,
|
||||
select_lib: dcLibref,
|
||||
select_ds: 'MPE_DERIVED',
|
||||
base_column: 'VAR_TYPE',
|
||||
selectbox_value: 'N',
|
||||
selectbox_order: 2,
|
||||
ver_to_dttm: 127490111999
|
||||
}
|
||||
],
|
||||
MPE_SUBMIT: [],
|
||||
@@ -1391,19 +1371,6 @@ const tables = {
|
||||
'RLS_SCOPE RLS_GROUP RLS_LIBREF RLS_TABLE RLS_GROUP_LOGIC RLS_SUBGROUP_LOGIC RLS_SUBGROUP_ID RLS_VARIABLE_NM RLS_OPERATOR_NM RLS_RAW_VALUE',
|
||||
post_edit_hook: 'services/hooks/mpe_row_level_security_postedit'
|
||||
},
|
||||
{
|
||||
tx_from: 0,
|
||||
tx_to: 127490111999,
|
||||
libref: dcLibref,
|
||||
dsn: 'MPE_DERIVED',
|
||||
num_of_approvals_required: 1,
|
||||
loadtype: 'TXTEMPORAL',
|
||||
buskey: 'LIBREF DSN VAR_NM',
|
||||
notes: 'Derived (computed) variables for the EDIT screen',
|
||||
var_txfrom: 'TX_FROM',
|
||||
var_txto: 'TX_TO',
|
||||
post_edit_hook: 'services/hooks/mpe_derived_postedit'
|
||||
},
|
||||
{
|
||||
tx_from: 0,
|
||||
tx_to: 127490111999,
|
||||
@@ -2035,206 +2002,6 @@ const tables = {
|
||||
rule_value: 'services/validations/mpe_x_test.some_num',
|
||||
rule_active: 1,
|
||||
tx_to: 127490111999
|
||||
},
|
||||
{
|
||||
tx_from: 0,
|
||||
base_lib: dcLibref,
|
||||
base_ds: 'MPE_DERIVED',
|
||||
base_col: 'LIBREF',
|
||||
rule_type: 'NOTNULL',
|
||||
rule_value: ' ',
|
||||
rule_active: 1,
|
||||
tx_to: 127490111999
|
||||
},
|
||||
{
|
||||
tx_from: 0,
|
||||
base_lib: dcLibref,
|
||||
base_ds: 'MPE_DERIVED',
|
||||
base_col: 'LIBREF',
|
||||
rule_type: 'CASE',
|
||||
rule_value: 'UPCASE',
|
||||
rule_active: 1,
|
||||
tx_to: 127490111999
|
||||
},
|
||||
{
|
||||
tx_from: 0,
|
||||
base_lib: dcLibref,
|
||||
base_ds: 'MPE_DERIVED',
|
||||
base_col: 'DSN',
|
||||
rule_type: 'NOTNULL',
|
||||
rule_value: ' ',
|
||||
rule_active: 1,
|
||||
tx_to: 127490111999
|
||||
},
|
||||
{
|
||||
tx_from: 0,
|
||||
base_lib: dcLibref,
|
||||
base_ds: 'MPE_DERIVED',
|
||||
base_col: 'DSN',
|
||||
rule_type: 'CASE',
|
||||
rule_value: 'UPCASE',
|
||||
rule_active: 1,
|
||||
tx_to: 127490111999
|
||||
},
|
||||
{
|
||||
tx_from: 0,
|
||||
base_lib: dcLibref,
|
||||
base_ds: 'MPE_DERIVED',
|
||||
base_col: 'VAR_NM',
|
||||
rule_type: 'NOTNULL',
|
||||
rule_value: ' ',
|
||||
rule_active: 1,
|
||||
tx_to: 127490111999
|
||||
},
|
||||
{
|
||||
tx_from: 0,
|
||||
base_lib: dcLibref,
|
||||
base_ds: 'MPE_DERIVED',
|
||||
base_col: 'VAR_NM',
|
||||
rule_type: 'CASE',
|
||||
rule_value: 'UPCASE',
|
||||
rule_active: 1,
|
||||
tx_to: 127490111999
|
||||
},
|
||||
{
|
||||
tx_from: 0,
|
||||
base_lib: dcLibref,
|
||||
base_ds: 'MPE_DERIVED',
|
||||
base_col: 'VAR_NM',
|
||||
rule_type: 'HARDREGEX',
|
||||
rule_value: '^[A-Za-z_][A-Za-z0-9_]*$',
|
||||
rule_active: 1,
|
||||
tx_to: 127490111999
|
||||
},
|
||||
{
|
||||
tx_from: 0,
|
||||
base_lib: dcLibref,
|
||||
base_ds: 'MPE_DERIVED',
|
||||
base_col: 'VAR_DEPS',
|
||||
rule_type: 'CASE',
|
||||
rule_value: 'UPCASE',
|
||||
rule_active: 1,
|
||||
tx_to: 127490111999
|
||||
},
|
||||
{
|
||||
tx_from: 0,
|
||||
base_lib: dcLibref,
|
||||
base_ds: 'MPE_DERIVED',
|
||||
base_col: 'VAR_DEPS',
|
||||
rule_type: 'HARDREGEX',
|
||||
rule_value: '^[A-Za-z0-9_ ]*$',
|
||||
rule_active: 1,
|
||||
tx_to: 127490111999
|
||||
},
|
||||
{
|
||||
tx_from: 0,
|
||||
base_lib: dcLibref,
|
||||
base_ds: 'MPE_DERIVED',
|
||||
base_col: 'VAR_TYPE',
|
||||
rule_type: 'NOTNULL',
|
||||
rule_value: ' ',
|
||||
rule_active: 1,
|
||||
tx_to: 127490111999
|
||||
},
|
||||
{
|
||||
tx_from: 0,
|
||||
base_lib: dcLibref,
|
||||
base_ds: 'MPE_DERIVED',
|
||||
base_col: 'VAR_TYPE',
|
||||
rule_type: 'CASE',
|
||||
rule_value: 'UPCASE',
|
||||
rule_active: 1,
|
||||
tx_to: 127490111999
|
||||
},
|
||||
{
|
||||
tx_from: 0,
|
||||
base_lib: dcLibref,
|
||||
base_ds: 'MPE_DERIVED',
|
||||
base_col: 'VAR_TYPE',
|
||||
rule_type: 'HARDSELECT',
|
||||
rule_value: ' ',
|
||||
rule_active: 1,
|
||||
tx_to: 127490111999
|
||||
},
|
||||
{
|
||||
tx_from: 0,
|
||||
base_lib: dcLibref,
|
||||
base_ds: 'MPE_DERIVED',
|
||||
base_col: 'VAR_LEN',
|
||||
rule_type: 'NOTNULL',
|
||||
rule_value: ' ',
|
||||
rule_active: 1,
|
||||
tx_to: 127490111999
|
||||
},
|
||||
{
|
||||
tx_from: 0,
|
||||
base_lib: dcLibref,
|
||||
base_ds: 'MPE_DERIVED',
|
||||
base_col: 'VAR_LEN',
|
||||
rule_type: 'MINVAL',
|
||||
rule_value: '1',
|
||||
rule_active: 1,
|
||||
tx_to: 127490111999
|
||||
},
|
||||
{
|
||||
tx_from: 0,
|
||||
base_lib: dcLibref,
|
||||
base_ds: 'MPE_DERIVED',
|
||||
base_col: 'VAR_LEN',
|
||||
rule_type: 'MAXVAL',
|
||||
rule_value: '32767',
|
||||
rule_active: 1,
|
||||
tx_to: 127490111999
|
||||
},
|
||||
{
|
||||
tx_from: 0,
|
||||
base_lib: dcLibref,
|
||||
base_ds: 'MPE_DERIVED',
|
||||
base_col: 'VAR_FMT',
|
||||
rule_type: 'CASE',
|
||||
rule_value: 'UPCASE',
|
||||
rule_active: 1,
|
||||
tx_to: 127490111999
|
||||
},
|
||||
{
|
||||
tx_from: 0,
|
||||
base_lib: dcLibref,
|
||||
base_ds: 'MPE_DERIVED',
|
||||
base_col: 'VAR_FMT',
|
||||
rule_type: 'HARDREGEX',
|
||||
rule_value: '^(\$?[A-Za-z][A-Za-z0-9_]*(\.[0-9]+)?\.?|\$?[0-9]+(\.[0-9]+)?\.?)?$',
|
||||
rule_active: 1,
|
||||
tx_to: 127490111999
|
||||
},
|
||||
{
|
||||
tx_from: 0,
|
||||
base_lib: dcLibref,
|
||||
base_ds: 'MPE_DERIVED',
|
||||
base_col: 'VAR_LOGIC',
|
||||
rule_type: 'NOTNULL',
|
||||
rule_value: ' ',
|
||||
rule_active: 1,
|
||||
tx_to: 127490111999
|
||||
},
|
||||
{
|
||||
tx_from: 0,
|
||||
base_lib: dcLibref,
|
||||
base_ds: 'MPE_DERIVED',
|
||||
base_col: 'VAR_LABEL',
|
||||
rule_type: 'HARDREGEX',
|
||||
rule_value: '^[^\\x00-\\x1F\\x7F]*$',
|
||||
rule_active: 1,
|
||||
tx_to: 127490111999
|
||||
},
|
||||
{
|
||||
tx_from: 0,
|
||||
base_lib: dcLibref,
|
||||
base_ds: 'MPE_DERIVED',
|
||||
base_col: 'VAR_LOGIC',
|
||||
rule_type: 'HARDREGEX',
|
||||
rule_value: '^(?!.*(/\\*|\\*/))[^;&%]*$',
|
||||
rule_active: 1,
|
||||
tx_to: 127490111999
|
||||
}
|
||||
],
|
||||
MPE_X_TEST: [
|
||||
|
||||
@@ -47,8 +47,7 @@ if (!tableId || !libds) {
|
||||
webOutClose()
|
||||
} else {
|
||||
// The SAS service also checks EDIT or APPROVE access (mpe_accesscheck). The
|
||||
// mock seeds MPE_SECURITY but no mock service enforces it, so access is not
|
||||
// emulated here.
|
||||
// mock has no permission store, so access is not emulated.
|
||||
|
||||
// The approval folder is zipped as it stands. getauditfile.sas lists it with
|
||||
// mp_dirlist and adds each file to the package under contents/.
|
||||
|
||||
@@ -641,27 +641,6 @@ function zipUtf8(value) {
|
||||
return Uint8Array.from(out)
|
||||
}
|
||||
|
||||
/**
|
||||
* Renders rows as CSV, the counterpart of %mp_ds2csv: the header row, then one
|
||||
* line per row, with values quoted only when they carry a delimiter, quote or
|
||||
* line break.
|
||||
*
|
||||
* @param {string[]} headers column names, in order
|
||||
* @param {Array<Object>} rows objects keyed by column name
|
||||
* @returns {string} CSV text with a trailing newline
|
||||
*/
|
||||
function toCsv(headers, rows) {
|
||||
const cell = (value) => {
|
||||
const text = value === undefined || value === null ? '' : String(value)
|
||||
return /[",\n\r]/.test(text) ? '"' + text.replace(/"/g, '""') + '"' : text
|
||||
}
|
||||
const lines = [headers.map(cell).join(',')]
|
||||
for (const row of rows) {
|
||||
lines.push(headers.map((header) => cell(row[header])).join(','))
|
||||
}
|
||||
return lines.join('\n') + '\n'
|
||||
}
|
||||
|
||||
/**
|
||||
* Builds a zip archive in memory, entries stored rather than deflated - the JS
|
||||
* counterpart of the ODS package getauditfile.sas publishes. The zip format
|
||||
@@ -682,19 +661,16 @@ function mpZipPackage(entries) {
|
||||
const name = zipUtf8(entry.name)
|
||||
const data = zipUtf8(entry.content)
|
||||
const crc = zipCrc32(data)
|
||||
// Bit 11 marks the name as UTF-8. Without it an extractor decodes a
|
||||
// non-ASCII name as CP437, so the entry cannot be read back by name.
|
||||
const flags = name.some((byte) => byte >= 0x80) ? 0x0800 : 0
|
||||
|
||||
// 1980-01-01 00:00, the earliest DOS timestamp: a stable value keeps
|
||||
// the archive byte-identical for identical inputs.
|
||||
const header = [].concat(
|
||||
u32(0x04034b50), u16(20), u16(flags), u16(0), u16(0), u16(0x21),
|
||||
u32(0x04034b50), u16(20), u16(0), u16(0), u16(0), u16(0x21),
|
||||
u32(crc), u32(data.length), u32(data.length), u16(name.length), u16(0)
|
||||
)
|
||||
|
||||
local.push(Uint8Array.from(header), name, data)
|
||||
central.push({ name, crc, size: data.length, offset, flags })
|
||||
central.push({ name, crc, size: data.length, offset })
|
||||
offset += header.length + name.length + data.length
|
||||
}
|
||||
|
||||
@@ -702,7 +678,7 @@ function mpZipPackage(entries) {
|
||||
let directorySize = 0
|
||||
for (const entry of central) {
|
||||
const header = [].concat(
|
||||
u32(0x02014b50), u16(20), u16(20), u16(entry.flags), u16(0), u16(0), u16(0x21),
|
||||
u32(0x02014b50), u16(20), u16(20), u16(0), u16(0), u16(0), u16(0x21),
|
||||
u32(entry.crc), u32(entry.size), u32(entry.size),
|
||||
u16(entry.name.length), u16(0), u16(0), u16(0), u16(0), u32(0),
|
||||
u32(entry.offset)
|
||||
|
||||
@@ -101,7 +101,7 @@ function pkFirst(columns, keys) {
|
||||
// ─── Derived (computed) variables (mpe_derived.sas) ─────────────────────────
|
||||
// Applied before the filter, so a filter can reference them. The derived
|
||||
// columns are added to the schema too, which is what puts them in COLS and in
|
||||
// the grid; they are held read-only by a READONLY rule below.
|
||||
// the grid (read only, since the mock marks every column READ).
|
||||
|
||||
const derived = mpeDerived({
|
||||
libref: requestedLibref,
|
||||
@@ -160,16 +160,6 @@ if (tableData && tableData.columns) {
|
||||
}
|
||||
}
|
||||
|
||||
// A derived variable is computed at read time, so the grid must not accept a
|
||||
// value for it. COLS carries READ for the same reason (a derived column has no
|
||||
// CLS rule row), but the client only applies COLS.CLS_RULE when CLS_FLAG is
|
||||
// set, which column-level security drives - so the rule is declared here too,
|
||||
// as getdata.sas does. dqrules is the request-local array the filter/map above
|
||||
// built, so appending to it leaves the loaded MPE_VALIDATIONS rows untouched.
|
||||
for (const v of derived) {
|
||||
dqrules.push({ BASE_COL: v.name, RULE_TYPE: 'READONLY', RULE_VALUE: '' })
|
||||
}
|
||||
|
||||
// ─── DQ dropdown data (from MPE_SELECTBOX + SOFTSELECT from table data) ──────
|
||||
// MPE_SELECTBOX lives in the control library.
|
||||
|
||||
@@ -436,11 +426,7 @@ if (tableData && tableData.columns && tableData.rows) {
|
||||
CLS_FLAG: 0,
|
||||
ISMAP: 0,
|
||||
TRUNCATED: truncated,
|
||||
TRUNC_MSG: truncMsg,
|
||||
// names the derived columns for the grid, as getdata.sas does - the
|
||||
// READONLY rule above is why they cannot be edited, this is how the
|
||||
// EDIT screen knows they are computed rather than locked
|
||||
DERIVEDVARS: derived.map(d => d.name).join(' ')
|
||||
TRUNC_MSG: truncMsg
|
||||
}],
|
||||
versions: [],
|
||||
xl_rules: []
|
||||
@@ -455,7 +441,7 @@ if (tableData && tableData.columns && tableData.rows) {
|
||||
],
|
||||
maxvarlengths: [], query: [], sasdata: [],
|
||||
$sasdata: { vars: {} },
|
||||
sasparams: [{ COLHEADERS: '', FILTER_TEXT: ' ', PKCNT: 0, PK: '', DTVARS: '', DTTMVARS: '', TMVARS: '', LOADTYPE: 'UPDATE', RK_FLAG: 0, CLS_FLAG: 0, ISMAP: 0, TRUNCATED: 0, TRUNC_MSG: '', DERIVEDVARS: '' }],
|
||||
sasparams: [{ COLHEADERS: '', FILTER_TEXT: ' ', PKCNT: 0, PK: '', DTVARS: '', DTTMVARS: '', TMVARS: '', LOADTYPE: 'UPDATE', RK_FLAG: 0, CLS_FLAG: 0, ISMAP: 0, TRUNCATED: 0, TRUNC_MSG: '' }],
|
||||
versions: [], xl_rules: []
|
||||
}
|
||||
}
|
||||
|
||||
@@ -59,9 +59,6 @@ for (let i = 1; i <= 10; i++) {
|
||||
|
||||
let libds = ''
|
||||
if (typeof table !== 'undefined' && table) libds = String(table).toUpperCase()
|
||||
// captured before any hook can re-point the submission: the cleaned CSV is
|
||||
// named after the original table, as loadfile.sas names it
|
||||
const origLibds = libds
|
||||
let libref = libds.split('.')[0] || dcLibref
|
||||
let dsn = libds.split('.')[1] || ''
|
||||
|
||||
@@ -181,9 +178,6 @@ if (!msg) {
|
||||
// written, while LIBREF / DS are still ordinary macro variables, so it
|
||||
// can re-point the changeset at a different table. The mock eval()s the
|
||||
// hook's .js counterpart in this scope so it can reassign libref / dsn.
|
||||
// A validating hook rejects the submission by setting hookErrmsg (see
|
||||
// hooks/mpe_derived_postedit.js), which stops the staging below.
|
||||
let hookErrmsg = ''
|
||||
const hookTables = mpeLoadTableData('MPE_TABLES')
|
||||
const hookReg = (hookTables && hookTables.rows)
|
||||
? hookTables.rows.find(r => r.libref === libref && r.dsn === dsn)
|
||||
@@ -191,21 +185,16 @@ if (!msg) {
|
||||
if (hookReg && hookReg.post_edit_hook) {
|
||||
const hookSrc = mockHookSource(hookReg.post_edit_hook)
|
||||
if (hookSrc) {
|
||||
// a hook reads and normalises the submitted rows as jsDataRows
|
||||
// (stagedata.js names them that); here they are `rows`
|
||||
const jsDataRows = rows
|
||||
try {
|
||||
eval(hookSrc)
|
||||
} catch (err) {
|
||||
// a hook that throws has not validated anything, so the
|
||||
// submission is refused rather than staged unchecked
|
||||
hookErrmsg =
|
||||
console.log(
|
||||
'post_edit_hook ' + hookReg.post_edit_hook + ' failed: ' + err.message
|
||||
)
|
||||
}
|
||||
libds = libref + '.' + dsn
|
||||
}
|
||||
}
|
||||
if (hookErrmsg) msg = hookErrmsg
|
||||
|
||||
tableId = makeTableId()
|
||||
// ─── RK assignment (mpe_loader) ──────────────────────────────────────
|
||||
@@ -225,7 +214,7 @@ if (!msg) {
|
||||
if (rkName && !headers.some((h) => h.toUpperCase() === rkName.toUpperCase())) {
|
||||
headers.push(rkName)
|
||||
}
|
||||
if (!msg) stageSubmission({
|
||||
stageSubmission({
|
||||
dataDir: libDataDir(libref),
|
||||
stagingDir,
|
||||
tableId,
|
||||
@@ -235,11 +224,13 @@ if (!msg) {
|
||||
message: 'File upload: ' + csvName,
|
||||
headers,
|
||||
rows,
|
||||
// loadfile.sas leaves the cleaned CSV (%mp_cleancsv) in the approval
|
||||
// folder, named after the table as the request found it. It copies
|
||||
// the upload as BKP_<name> only when a second, non-CSV fileref came
|
||||
// with the request, which the mock does not accept - so no BKP here.
|
||||
auditFiles: [{ name: origLibds + '.csv', content: csvText }]
|
||||
// loadfile.sas leaves two files in the approval folder: the cleaned
|
||||
// CSV (%mp_cleancsv) and a copy of the upload (BKP_<name>). The mock
|
||||
// only accepts CSV, so both hold the uploaded text.
|
||||
auditFiles: [
|
||||
{ name: libds + '.csv', content: csvText },
|
||||
{ name: 'BKP_' + csvName, content: csvText }
|
||||
]
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -22,9 +22,6 @@ let message = _sctRow.MESSAGE || ''
|
||||
let libds = _sctRow.LIBDS || ''
|
||||
let libref = libds.split('.')[0] || dcLibref
|
||||
let dsn = libds.split('.')[1] || ''
|
||||
// stagedata.sas names the CSV it stages after the table as the request found
|
||||
// it, before a hook can re-point the submission
|
||||
const origLibds = libds
|
||||
|
||||
// The base table lives in its own library (DC_JSLIB for control tables,
|
||||
// TESTDATA for demo user tables); the staging dir stays in the control lib.
|
||||
@@ -66,10 +63,6 @@ if (schema.length > 0) {
|
||||
// eval()s the hook's .js counterpart in this scope so it can reassign
|
||||
// libref / dsn / libds; the submit record then names the new base table, exactly
|
||||
// as the SAS hook does via call symputx.
|
||||
// A validating hook rejects the submission by setting hookErrmsg (see
|
||||
// hooks/mpe_derived_postedit.js); the response below turns it into an abort,
|
||||
// and nothing is staged.
|
||||
let hookErrmsg = ''
|
||||
const hookTables = makeTableLoader(mpeDataDir)('MPE_TABLES')
|
||||
const hookReg = (hookTables && hookTables.rows)
|
||||
? hookTables.rows.find(r => r.libref === libref && r.dsn === dsn)
|
||||
@@ -80,10 +73,9 @@ if (hookReg && hookReg.post_edit_hook) {
|
||||
try {
|
||||
eval(hookSrc)
|
||||
} catch (err) {
|
||||
// a hook that throws has not validated anything, so the submission
|
||||
// is refused rather than staged unchecked
|
||||
hookErrmsg =
|
||||
console.log(
|
||||
'post_edit_hook ' + hookReg.post_edit_hook + ' failed: ' + err.message
|
||||
)
|
||||
}
|
||||
libds = libref + '.' + dsn
|
||||
}
|
||||
@@ -109,29 +101,6 @@ if (rkName && !jsDataHeaders.some((h) => h.toUpperCase() === rkName.toUpperCase(
|
||||
jsDataHeaders.push(rkName)
|
||||
}
|
||||
|
||||
// ─── Derived variables ───────────────────────────────────────────────────────
|
||||
// stagedata.sas drops the MPE_DERIVED droplist from the submitted data before
|
||||
// anything is staged (they are display-only), and the approval CSV is written
|
||||
// from that stripped dataset - so neither the staged rows nor the pack may
|
||||
// carry them.
|
||||
|
||||
const derivedVars = mpeDerived({
|
||||
libref,
|
||||
table: dsn,
|
||||
dataDir: mpeDataDir,
|
||||
columns: schema
|
||||
})
|
||||
if (derivedVars.length > 0) {
|
||||
const derivedNames = new Set(derivedVars.map((d) => d.name.toUpperCase()))
|
||||
const keep = (key) => !derivedNames.has(key.toUpperCase())
|
||||
jsDataHeaders = jsDataHeaders.filter(keep)
|
||||
jsDataRows = jsDataRows.map((row) => {
|
||||
const copy = {}
|
||||
for (const key of Object.keys(row)) if (keep(key)) copy[key] = row[key]
|
||||
return copy
|
||||
})
|
||||
}
|
||||
|
||||
// ─── Stage the data ──────────────────────────────────────────────────────────
|
||||
// One subfolder per submission (DSID) under the staging directory, so that the
|
||||
// staged data can sit alongside related artifacts (submit/approval logs,
|
||||
@@ -139,7 +108,7 @@ if (derivedVars.length > 0) {
|
||||
|
||||
const tableId = makeTableId()
|
||||
|
||||
if (!hookErrmsg) stageSubmission({
|
||||
stageSubmission({
|
||||
dataDir,
|
||||
stagingDir,
|
||||
tableId,
|
||||
@@ -148,17 +117,9 @@ if (!hookErrmsg) stageSubmission({
|
||||
dsn,
|
||||
message,
|
||||
headers: jsDataHeaders,
|
||||
rows: jsDataRows,
|
||||
// stagedata.sas writes the staged rows as <orig_libds>.csv alongside them
|
||||
// (%mp_ds2csv into the approval folder), so the audit pack carries it
|
||||
auditFiles: [{ name: origLibds + '.csv', content: toCsv(jsDataHeaders, jsDataRows) }]
|
||||
rows: jsDataRows
|
||||
})
|
||||
|
||||
webOutOpen()
|
||||
if (hookErrmsg) {
|
||||
webOutObj([{ MSG: hookErrmsg, MAC: 'stagedata' }], 'sasjsAbort')
|
||||
webOutObj([{ STATUS: 'ERROR', DSID: '' }], 'sasparams')
|
||||
} else {
|
||||
webOutObj([{ STATUS: 'SUCCESS', DSID: tableId, URL: '' }], 'sasparams')
|
||||
}
|
||||
webOutObj([{ STATUS: 'SUCCESS', DSID: tableId, URL: '' }], 'sasparams')
|
||||
webOutClose()
|
||||
@@ -1,410 +0,0 @@
|
||||
// ─────────────────────────────────────────────────────────────────────────────
|
||||
// Mock of services/hooks/mpe_derived_postedit
|
||||
//
|
||||
// Registered against DC_JSLIB.MPE_DERIVED (MPE_TABLES.post_edit_hook). This
|
||||
// file is eval'd by editors/stagedata.js and editors/loadfile.js, mirroring how
|
||||
// mpe_loader.sas %includes the hook program via %mpe_runhook, so it shares the
|
||||
// caller's scope: jsDataRows (the submitted rows - work.staging_ds on the
|
||||
// server), libref, dsn, mpeDataDir and the dcMockUtils helpers.
|
||||
//
|
||||
// A hook rejects a submission by setting hookErrmsg, which the caller turns
|
||||
// into a sasjsAbort - the same outcome as the SAS hook's %mp_abort.
|
||||
//
|
||||
// The rules mirror services/hooks/mpe_derived_postedit.sas, in its order:
|
||||
// normalise the incoming rows, reject a duplicate name and a bad field, then
|
||||
// check the dependency graph that the resulting configuration would have (the
|
||||
// incoming rows plus the current rows they do not replace). The field rules
|
||||
// are core's mp_derivedvarcheck, which the server hook calls and the read-time
|
||||
// engine runs too - this file spells them out a second time, so a change there
|
||||
// has to be mirrored here. As on the server, the collision between VAR_NM and
|
||||
// a physical column is left to the read-time engine, which runs with the target
|
||||
// library assigned - the hook does not have it here either.
|
||||
// ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
;(function mpeDerivedPostedit() {
|
||||
const DELETE_MARK = '_____DELETE__THIS__RECORD_____'
|
||||
const NAME_CHARS = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_'
|
||||
const FMT_CHARS = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_$.'
|
||||
|
||||
const text = (v) => (v === undefined || v === null ? '' : String(v))
|
||||
const up = (v) => text(v).trim().toUpperCase()
|
||||
const strip = (v) => text(v).trim()
|
||||
|
||||
/** Reads a field as the hook normalised it. */
|
||||
const field = (row, name) => up(colVal(row, name))
|
||||
|
||||
/** Writes a field back under the key the row already carries. */
|
||||
const setField = (row, name, value) => {
|
||||
const key = Object.keys(row).find((k) => k.toUpperCase() === name)
|
||||
if (key) row[key] = value
|
||||
else row[name] = value
|
||||
}
|
||||
|
||||
const rows = Array.isArray(jsDataRows) ? jsDataRows : []
|
||||
const isDeleted = (row) => field(row, DELETE_MARK) === 'YES'
|
||||
const isLive = (row) =>
|
||||
!isDeleted(row) && Number(colVal(row, 'ACTIVE')) === 1
|
||||
const keyOf = (row) =>
|
||||
[field(row, 'LIBREF'), field(row, 'DSN'), field(row, 'VAR_NM')].join(
|
||||
'.'
|
||||
)
|
||||
|
||||
const reject = (msg) => {
|
||||
hookErrmsg = msg
|
||||
}
|
||||
|
||||
// ─── Normalise the incoming values ─────────────────────────────────────────
|
||||
for (const row of rows) {
|
||||
if (isDeleted(row)) continue
|
||||
for (const name of [
|
||||
'LIBREF',
|
||||
'DSN',
|
||||
'VAR_NM',
|
||||
'VAR_TYPE',
|
||||
'VAR_FMT',
|
||||
'VAR_DEPS'
|
||||
]) {
|
||||
setField(row, name, up(colVal(row, name)))
|
||||
}
|
||||
for (const name of ['VAR_LABEL', 'VAR_LOGIC']) {
|
||||
setField(row, name, strip(colVal(row, name)))
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Duplicate names after normalisation ───────────────────────────────────
|
||||
// mpe_loader compares the raw values, so a case variant reaches this point
|
||||
const seen = new Set()
|
||||
for (const row of rows) {
|
||||
if (isDeleted(row)) continue
|
||||
const key = keyOf(row)
|
||||
if (seen.has(key)) {
|
||||
return reject(
|
||||
`MPE_DERIVED: ${field(row, 'VAR_NM')} is declared more than once for ${field(row, 'LIBREF')}.${field(row, 'DSN')}`
|
||||
)
|
||||
}
|
||||
seen.add(key)
|
||||
}
|
||||
|
||||
// ─── Field level checks ────────────────────────────────────────────────────
|
||||
// Two passes, in the order the SAS hook runs them. The first is the fields
|
||||
// this hook owns - the control table key, which the loader normalises but
|
||||
// does not require, and ACTIVE, the picker pair the EDIT screen offers.
|
||||
for (const row of rows) {
|
||||
if (isDeleted(row)) continue
|
||||
const name = field(row, 'VAR_NM')
|
||||
let reason = ''
|
||||
if (!field(row, 'LIBREF') || !field(row, 'DSN'))
|
||||
reason = 'LIBREF and DSN are required'
|
||||
// ACTIVE is a picker in the EDIT screen (HARDSELECT over the
|
||||
// MPE_SELECTBOX pair) - the same pair the hook enforces
|
||||
else if (!['0', '1'].includes(String(colVal(row, 'ACTIVE')).trim()))
|
||||
reason = 'ACTIVE must be 1 or 0'
|
||||
if (reason) return reject(`MPE_DERIVED: ${name}: ${reason}`)
|
||||
}
|
||||
|
||||
// The second pass is the rules for the derived variable itself, which are
|
||||
// core's (mp_derivedvarcheck): the engine runs the same macro at read time,
|
||||
// so a definition is measured against one rule list wherever it is checked.
|
||||
|
||||
// C0 controls and DEL: what a label cannot carry once a line break has been
|
||||
// flattened to a space, which is what the engine does with it
|
||||
const CONTROL_CHARS = /[\u0000-\u001F\u007F]/
|
||||
const flatten = (v) => v.replace(/[\r\n]+/g, ' ')
|
||||
const AUTOMATIC_VARS = ['_N_', '_ERROR_', '_IORC_']
|
||||
|
||||
// the capabilities refused by name - a function name cannot be assembled at
|
||||
// run time, so naming them is a control the expression cannot evade
|
||||
const DENY_FUNCS = new RegExp(
|
||||
'\\b(DOSUBL|JAVAOBJ|JAVAMETA|SYMGET|SYMPUTX?|SYSGET|GETOPTION' +
|
||||
'|PATHNAME|FILEEXIST|FOPEN|FREAD|FGET|FCLOSE' +
|
||||
'|DOPEN|DREAD|DNUM|MOPEN|SLEEP' +
|
||||
'|FILENAME|FDELETE|FCOPY|DCREATE|DLCREATE)\\s*\\('
|
||||
)
|
||||
|
||||
// quoted literals are data, not calls
|
||||
const unquoted = (v) =>
|
||||
v.replace(/'([^']|'')*'/g, ' ').replace(/"([^"]|"")*"/g, ' ')
|
||||
|
||||
// true when the value carries a character outside the allowed set
|
||||
const hasInvalidChar = (value, chars) =>
|
||||
[...value].some((c) => !chars.includes(c))
|
||||
|
||||
for (const row of rows) {
|
||||
if (isDeleted(row)) continue
|
||||
const name = field(row, 'VAR_NM')
|
||||
const type = field(row, 'VAR_TYPE')
|
||||
const len = Number(colVal(row, 'VAR_LEN'))
|
||||
const fmt = field(row, 'VAR_FMT')
|
||||
const label = strip(colVal(row, 'VAR_LABEL'))
|
||||
const logic = strip(colVal(row, 'VAR_LOGIC'))
|
||||
|
||||
let reason = ''
|
||||
if (!name) reason = 'VAR_NM is required'
|
||||
else if (hasInvalidChar(name, NAME_CHARS))
|
||||
reason = 'VAR_NM must be a valid SAS name'
|
||||
else if ('0123456789'.includes(name[0]))
|
||||
reason = 'VAR_NM must not start with a digit'
|
||||
else if (AUTOMATIC_VARS.includes(name))
|
||||
reason = 'VAR_NM is an automatic variable of the data step'
|
||||
else if (name === DELETE_MARK) reason = 'VAR_NM is reserved'
|
||||
else if (type !== 'C' && type !== 'N')
|
||||
reason = 'VAR_TYPE must be C or N'
|
||||
else if (!Number.isInteger(len) || len < 1)
|
||||
reason = 'VAR_LEN must be a positive integer'
|
||||
else if (type === 'C' && len > 32767)
|
||||
reason = 'VAR_LEN exceeds 32767 for a character variable'
|
||||
else if (type === 'N' && len > 8)
|
||||
reason = 'VAR_LEN exceeds 8 for a numeric variable'
|
||||
else if (
|
||||
fmt.includes(';') ||
|
||||
fmt.includes('&') ||
|
||||
fmt.includes('%') ||
|
||||
fmt.includes('/*') ||
|
||||
fmt.includes('*/')
|
||||
)
|
||||
reason = 'VAR_FMT must be a format name, without ; & % or /* */'
|
||||
else if (hasInvalidChar(fmt, FMT_CHARS))
|
||||
reason = 'VAR_FMT must be a format name'
|
||||
// free text, in any script - what is refused is a control character,
|
||||
// and a value the engine would cut to 256
|
||||
else if (CONTROL_CHARS.test(flatten(label)))
|
||||
reason = 'VAR_LABEL must be plain text - no control characters'
|
||||
else if ([...label].length > 256)
|
||||
reason = 'VAR_LABEL exceeds 256 characters'
|
||||
else if (!logic) reason = 'VAR_LOGIC is required'
|
||||
else if (logic.length > 4000)
|
||||
reason = 'VAR_LOGIC exceeds 4000 characters'
|
||||
else if (logic.includes(';'))
|
||||
reason = 'VAR_LOGIC must be a single expression - remove the ;'
|
||||
else if (logic.includes('&') || logic.includes('%'))
|
||||
reason = 'VAR_LOGIC must not contain macro triggers'
|
||||
else if (logic.includes('/*') || logic.includes('*/'))
|
||||
reason = 'VAR_LOGIC must not contain comments'
|
||||
else if (DENY_FUNCS.test(unquoted(logic).toUpperCase()))
|
||||
reason =
|
||||
'VAR_LOGIC may not call a function that submits code or reaches the file system, the environment or the macro symbol table'
|
||||
|
||||
if (reason) return reject(`MPE_DERIVED: ${name}: ${reason}`)
|
||||
}
|
||||
|
||||
// ─── The graph of the resulting configuration ──────────────────────────────
|
||||
// The incoming live rows plus the current rows they do not replace. Only
|
||||
// active rows are in it, because the engine reads nothing else - an inactive
|
||||
// row can neither resolve a dependency nor be resolved by one.
|
||||
const controlDir = mpeDataDir
|
||||
const loadControl = makeTableLoader(controlDir)
|
||||
const current = loadControl('MPE_DERIVED')
|
||||
const nowSeconds = Date.now() / 1000
|
||||
|
||||
// mpeDerived reads a row as current when TX_TO is missing, zero or in the
|
||||
// future (dcMockUtils.js, the read-time engine), so the same test is used
|
||||
// here - a seeded row with no TX_TO is live for the engine, and a dependency
|
||||
// on it has to validate
|
||||
const isCurrent = (row) =>
|
||||
!Number(colVal(row, 'TX_TO')) || Number(colVal(row, 'TX_TO')) > nowSeconds
|
||||
|
||||
const stagedKeys = new Set(rows.map(keyOf))
|
||||
const stagedLive = rows.filter(isLive)
|
||||
|
||||
// the control-table arm of the SAS hook is limited to the tables this
|
||||
// submission touches, so another table's configuration cannot block it
|
||||
const stagedTables = new Set(
|
||||
rows.map((row) => [field(row, 'LIBREF'), field(row, 'DSN')].join('.'))
|
||||
)
|
||||
|
||||
/** libref.dsn.name - the key every graph lookup uses. */
|
||||
const nodeKey = (n) => [n.libref, n.dsn, n.name].join('.')
|
||||
|
||||
const graph = new Map()
|
||||
for (const row of stagedLive) {
|
||||
graph.set(keyOf(row), {
|
||||
libref: field(row, 'LIBREF'),
|
||||
dsn: field(row, 'DSN'),
|
||||
name: field(row, 'VAR_NM'),
|
||||
deps: field(row, 'VAR_DEPS').split(/\s+/).filter(Boolean),
|
||||
logic: strip(colVal(row, 'VAR_LOGIC')).toUpperCase()
|
||||
})
|
||||
}
|
||||
const currentRows =
|
||||
current && Array.isArray(current.rows)
|
||||
? current.rows.filter(
|
||||
(r) =>
|
||||
isCurrent(r) &&
|
||||
Number(colVal(r, 'ACTIVE')) === 1 &&
|
||||
stagedTables.has(
|
||||
[up(colVal(r, 'LIBREF')), up(colVal(r, 'DSN'))].join(
|
||||
'.'
|
||||
)
|
||||
) &&
|
||||
!stagedKeys.has(
|
||||
[
|
||||
up(colVal(r, 'LIBREF')),
|
||||
up(colVal(r, 'DSN')),
|
||||
up(colVal(r, 'VAR_NM'))
|
||||
].join('.')
|
||||
)
|
||||
)
|
||||
: []
|
||||
for (const row of currentRows) {
|
||||
graph.set(
|
||||
[
|
||||
up(colVal(row, 'LIBREF')),
|
||||
up(colVal(row, 'DSN')),
|
||||
up(colVal(row, 'VAR_NM'))
|
||||
].join('.'),
|
||||
{
|
||||
libref: up(colVal(row, 'LIBREF')),
|
||||
dsn: up(colVal(row, 'DSN')),
|
||||
name: up(colVal(row, 'VAR_NM')),
|
||||
deps: up(colVal(row, 'VAR_DEPS')).split(/\s+/).filter(Boolean),
|
||||
logic: strip(colVal(row, 'VAR_LOGIC')).toUpperCase()
|
||||
}
|
||||
)
|
||||
}
|
||||
|
||||
/** Names this load leaves inactive, which therefore cannot resolve a dep. */
|
||||
const inactive = new Set()
|
||||
const liveKeys = new Set(stagedLive.map(keyOf))
|
||||
for (const row of rows) {
|
||||
const key = keyOf(row)
|
||||
if (!liveKeys.has(key)) inactive.add(key)
|
||||
}
|
||||
if (current && Array.isArray(current.rows)) {
|
||||
for (const row of current.rows) {
|
||||
const key = [
|
||||
up(colVal(row, 'LIBREF')),
|
||||
up(colVal(row, 'DSN')),
|
||||
up(colVal(row, 'VAR_NM'))
|
||||
].join('.')
|
||||
if (
|
||||
isCurrent(row) &&
|
||||
Number(colVal(row, 'ACTIVE')) !== 1 &&
|
||||
!liveKeys.has(key)
|
||||
) {
|
||||
inactive.add(key)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for (const node of graph.values()) {
|
||||
for (const dep of node.deps) {
|
||||
const key = [node.libref, node.dsn, dep].join('.')
|
||||
if (inactive.has(key) && !graph.has(key)) {
|
||||
return reject(
|
||||
`MPE_DERIVED: ${node.name}: VAR_DEPS lists ${dep}, which is not active after this load - the engine cannot resolve it`
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Self reference, undeclared and stale dependencies ─────────────────────
|
||||
/** The identifiers an expression uses, with quoted runs removed. */
|
||||
const identifiers = (logic) =>
|
||||
logic
|
||||
.replace(/'([^']|'')*'/g, ' ')
|
||||
.replace(/"([^"]|"")*"/g, ' ')
|
||||
.replace(/[^A-Za-z0-9_]/g, ' ')
|
||||
.toUpperCase()
|
||||
.split(/\s+/)
|
||||
.filter(Boolean)
|
||||
|
||||
// scoped per table, as the server does: a name that is a derived variable
|
||||
// on another table says nothing about this one
|
||||
const graphKeys = new Set([...graph.values()].map((n) => nodeKey(n)))
|
||||
|
||||
// one check at a time, in the order the server runs them, so the message a
|
||||
// row gets is the one it would get there
|
||||
for (const node of graph.values()) {
|
||||
if (node.deps.includes(node.name)) {
|
||||
return reject(
|
||||
`MPE_DERIVED: ${node.name}: VAR_DEPS may not list itself`
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
for (const node of graph.values()) {
|
||||
for (const token of identifiers(node.logic)) {
|
||||
if (
|
||||
graphKeys.has([node.libref, node.dsn, token].join('.')) &&
|
||||
token !== node.name &&
|
||||
!node.deps.includes(token)
|
||||
) {
|
||||
return reject(
|
||||
`MPE_DERIVED: ${node.name}: VAR_LOGIC references ${token}, which is not in VAR_DEPS`
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for (const node of graph.values()) {
|
||||
const tokens = identifiers(node.logic)
|
||||
for (const dep of node.deps) {
|
||||
if (!tokens.includes(dep)) {
|
||||
return reject(
|
||||
`MPE_DERIVED: ${node.name}: VAR_DEPS lists ${dep}, which VAR_LOGIC does not reference`
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ─── A derived variable may not be named by a row level security rule ──────
|
||||
const rls = loadControl('MPE_ROW_LEVEL_SECURITY')
|
||||
if (rls && Array.isArray(rls.rows)) {
|
||||
for (const node of graph.values()) {
|
||||
const match = rls.rows.find(
|
||||
(r) =>
|
||||
up(colVal(r, 'RLS_LIBREF')) === node.libref &&
|
||||
up(colVal(r, 'RLS_TABLE')) === node.dsn &&
|
||||
up(colVal(r, 'RLS_VARIABLE_NM')) === node.name &&
|
||||
Number(colVal(r, 'RLS_ACTIVE')) === 1 &&
|
||||
isCurrent(r)
|
||||
)
|
||||
if (match) {
|
||||
return reject(
|
||||
`MPE_DERIVED: ${node.name} is named by a row level security rule on ${node.libref}.${node.dsn} - derived variables cannot be used there`
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Cycles ────────────────────────────────────────────────────────────────
|
||||
// Repeatedly place the variables whose derived inputs are already placed; a
|
||||
// round that places nothing leaves a cycle. The path is walked so the
|
||||
// message names it, as the SAS hook does.
|
||||
const nodeKeys = new Map(
|
||||
[...graph.values()].map((n) => [n, [n.libref, n.dsn, n.name].join('.')])
|
||||
)
|
||||
const placed = new Set()
|
||||
let remaining = [...graph.values()]
|
||||
|
||||
while (remaining.length > 0) {
|
||||
const ready = remaining.filter((n) =>
|
||||
n.deps.every((d) => {
|
||||
const key = [n.libref, n.dsn, d].join('.')
|
||||
return !graph.has(key) || placed.has(key)
|
||||
})
|
||||
)
|
||||
if (ready.length === 0) {
|
||||
// walk the unplaced edges so the message names the cycle
|
||||
const start = remaining[0]
|
||||
const path = [start.name]
|
||||
const visited = new Set([nodeKeys.get(start)])
|
||||
let cursor = start
|
||||
for (let i = 0; i < remaining.length; i++) {
|
||||
const parentKey = cursor.deps
|
||||
.map((d) => [cursor.libref, cursor.dsn, d].join('.'))
|
||||
.find((key) => graph.has(key) && !placed.has(key))
|
||||
if (!parentKey) break
|
||||
path.push(graph.get(parentKey).name)
|
||||
if (visited.has(parentKey)) break
|
||||
visited.add(parentKey)
|
||||
cursor = graph.get(parentKey)
|
||||
}
|
||||
return reject(
|
||||
`MPE_DERIVED: VAR_DEPS contains a cycle: ${path.join(' -> ')}`
|
||||
)
|
||||
}
|
||||
for (const node of ready) placed.add(nodeKeys.get(node))
|
||||
remaining = remaining.filter((n) => !placed.has(nodeKeys.get(n)))
|
||||
}
|
||||
})()
|
||||
@@ -49,23 +49,6 @@ if (!tableData && dsn) {
|
||||
if (found) tableData = found.data
|
||||
}
|
||||
|
||||
// ─── Derived variables (mpe_derived.sas) ─────────────────────────────────────
|
||||
// A stored filter can reference a derived column, so the derivation has to run
|
||||
// before the filter. The columns themselves are dropped from the output - they
|
||||
// belong to the EDIT screen, as viewdata.sas drops them.
|
||||
// a copy: the derivation below returns a new column list, and every use here
|
||||
// (cols, search, rows, vars) must keep looking at the physical columns only
|
||||
const physicalColumns = [...(tableData && tableData.columns ? tableData.columns : [])]
|
||||
const derived = mpeDerived({
|
||||
libref: libref,
|
||||
table: dsn,
|
||||
dataDir: mpeDataDir,
|
||||
columns: physicalColumns
|
||||
})
|
||||
if (tableData && derived.length > 0) {
|
||||
tableData = mpeDeriveRows(tableData, derived)
|
||||
}
|
||||
|
||||
// ─── Apply the stored filter (mpe_filtermaster, VIEW mode) ───────────────────
|
||||
// Mirrors viewdata.sas: %mpe_filtermaster(VIEW,&libds,filter_rk=&filter_rk)
|
||||
// Row Level Security and any saved filter are applied here, before the search.
|
||||
@@ -120,7 +103,7 @@ let nobs = 0
|
||||
if (tableData && tableData.columns) {
|
||||
// mpGetcols, the JS counterpart of %mp_getcols.sas - the macro viewdata.sas
|
||||
// calls for its own column list.
|
||||
cols = mpGetcols(physicalColumns).map(c => {
|
||||
cols = mpGetcols(tableData.columns).map(c => {
|
||||
const ddtype = c.DDTYPE === 'CHARACTER' ? 'C' : c.DDTYPE === 'NUMERIC' ? 'N' : c.DDTYPE
|
||||
const formatStr = c.TYPE === 'N' ? (c.FORMAT || '8.') : ('$' + c.LENGTH + '.')
|
||||
return {
|
||||
@@ -136,10 +119,7 @@ if (tableData && tableData.columns) {
|
||||
|
||||
let rows
|
||||
if (searchType) {
|
||||
// the derived columns are dropped before the search, so only the
|
||||
// physical columns are searched (viewdata.sas drops them in the
|
||||
// vwsearch data step mp_searchdata then reads)
|
||||
const matches = filteredRows.filter(row => rowMatchesSearch(row, physicalColumns, searchType, searchVal))
|
||||
const matches = filteredRows.filter(row => rowMatchesSearch(row, tableData.columns, searchType, searchVal))
|
||||
// mp_searchdata caps its output at outobs=&DC_MAXOBS_WEBVIEW, and
|
||||
// viewdata.sas then keeps only `if _n_<&DC_MAXOBS_WEBVIEW` of them.
|
||||
nobs = Math.min(matches.length, maxObsWebView)
|
||||
@@ -154,7 +134,7 @@ if (tableData && tableData.columns) {
|
||||
// viewdata rows: all values as strings (formatted, SAS proc-print style)
|
||||
viewdata = rows.map(r => {
|
||||
const row = {}
|
||||
for (const col of physicalColumns) {
|
||||
for (const col of tableData.columns) {
|
||||
const lcName = col.name.toLowerCase()
|
||||
let val = r[col.name]
|
||||
if (val === undefined) val = r[lcName]
|
||||
@@ -194,15 +174,15 @@ if (tableData && tableData.columns) {
|
||||
// (SAS missing), character columns get "".
|
||||
// A search that matches nothing sends no rows at all (the client shows
|
||||
// its "no data found" panel instead of an empty grid).
|
||||
if (viewdata.length === 0 && !searchType && physicalColumns.length > 0) {
|
||||
if (viewdata.length === 0 && !searchType && tableData.columns.length > 0) {
|
||||
const emptyRow = {}
|
||||
for (const col of physicalColumns) emptyRow[col.name] = col.type === 'N' ? '.' : ''
|
||||
for (const col of tableData.columns) emptyRow[col.name] = col.type === 'N' ? '.' : ''
|
||||
viewdata = [emptyRow]
|
||||
}
|
||||
|
||||
// $viewdata vars: keep original SAS types (num stays num, char stays char)
|
||||
vars = {}
|
||||
for (const col of physicalColumns) {
|
||||
for (const col of tableData.columns) {
|
||||
vars[col.name] = sasVarsEntry(col)
|
||||
}
|
||||
}
|
||||
|
||||
Generated
+8
-8
@@ -6,8 +6,8 @@
|
||||
"": {
|
||||
"name": "dc-sas",
|
||||
"dependencies": {
|
||||
"@sasjs/cli": "6.2.0",
|
||||
"@sasjs/core": "6.1.4"
|
||||
"@sasjs/cli": "6.1.0",
|
||||
"@sasjs/core": "6.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@asamuzakjp/css-color": {
|
||||
@@ -165,9 +165,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@sasjs/cli": {
|
||||
"version": "6.2.0",
|
||||
"resolved": "https://registry.npmjs.org/@sasjs/cli/-/cli-6.2.0.tgz",
|
||||
"integrity": "sha512-zovAOCmyPqcm+gooAZgwvgGH8n5cas3sY0lA80+hyFoFtmZ8VjW/bh19LM8OVwheT9dBCRCx4fnHwjy5XGpzvQ==",
|
||||
"version": "6.1.0",
|
||||
"resolved": "https://registry.npmjs.org/@sasjs/cli/-/cli-6.1.0.tgz",
|
||||
"integrity": "sha512-i2CdGS69EM6Sn7XfR1ltBk9Bxz6gEV9xTGSWUmk4eVe+XnJBlYHyg93VgMcx7sWCtzRF9Dv7N2p7eXEc2DHfbQ==",
|
||||
"license": "ISC",
|
||||
"dependencies": {
|
||||
"@sasjs/adapter": "^5.1.1",
|
||||
@@ -235,9 +235,9 @@
|
||||
}
|
||||
},
|
||||
"node_modules/@sasjs/core": {
|
||||
"version": "6.1.4",
|
||||
"resolved": "https://registry.npmjs.org/@sasjs/core/-/core-6.1.4.tgz",
|
||||
"integrity": "sha512-vIF2LIftxPWg4CvO3i5raEfrHFHY/6S+MqVGpjRR71dgjLRJEYNyPJgGqD6BCZwHv5V6/2nDnBi/BIIMiLXfzA==",
|
||||
"version": "6.0.0",
|
||||
"resolved": "https://registry.npmjs.org/@sasjs/core/-/core-6.0.0.tgz",
|
||||
"integrity": "sha512-E9iXPvNgCux80xHhfopCsQE/CUNp1NEF1Z7NRj089A1T8WXP9AWJYcfFfOqoljNh8rnMo1fbIgoNrIaaAgW0MA==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@sasjs/lint": {
|
||||
|
||||
+3
-6
@@ -26,15 +26,12 @@
|
||||
"viyacbd": "sasjs folder delete /Public/app/viya && sasjs cbd -t viya && npm run viyadata",
|
||||
"viyadata": "sasjs request services/admin/makedata -d deploy/makeDataViya.json -l sasjsresults/makedata.log -o sasjsresults/output.json -t viya",
|
||||
"v4data": "sasjs request services/admin/makedata -d deploy/makeDataV4.json -l sasjsresults/makedatav4.log -o sasjsresults/outputv4.json -t v4",
|
||||
"sasdocs": "sasjs doc && ./sasjs/utils/deploydocs.sh",
|
||||
"sas719setup": "sasjs cbd -t sas719 && npm run sas719data",
|
||||
"sas719data": "sasjs request services/admin/makedata -d deploy/makeDataSas719.json -l sasjsresults/makedata_sas719.log -o sasjsresults/makedata_sas719.json -t sas719",
|
||||
"sas719test": "sasjs test -t sas719"
|
||||
"sasdocs": "sasjs doc && ./sasjs/utils/deploydocs.sh"
|
||||
},
|
||||
"private": true,
|
||||
"dependencies": {
|
||||
"@sasjs/cli": "6.2.0",
|
||||
"@sasjs/core": "6.1.4"
|
||||
"@sasjs/cli": "6.1.0",
|
||||
"@sasjs/core": "6.0.0"
|
||||
},
|
||||
"overrides": {
|
||||
"nanoid": "3.3.18"
|
||||
|
||||
@@ -17,16 +17,16 @@
|
||||
create table &curlib..MPE_DERIVED
|
||||
(
|
||||
TX_FROM num not null format=datetime19.3,
|
||||
LIBREF varchar(8) not null,
|
||||
DSN varchar(32) not null,
|
||||
VAR_NM varchar(32) not null,
|
||||
LIBREF varchar(8),
|
||||
DSN varchar(32),
|
||||
VAR_NM varchar(32),
|
||||
VAR_DEPS varchar(4000),
|
||||
VAR_TYPE varchar(1) not null,
|
||||
VAR_LEN int not null,
|
||||
VAR_TYPE varchar(1),
|
||||
VAR_LEN int,
|
||||
VAR_FMT varchar(32),
|
||||
VAR_LABEL varchar(256),
|
||||
VAR_LOGIC varchar(4000) not null,
|
||||
ACTIVE int not null,
|
||||
VAR_LOGIC varchar(4000),
|
||||
ACTIVE int,
|
||||
TX_TO num not null format=datetime19.3,
|
||||
constraint pk_mpe_derived
|
||||
primary key(tx_from, libref, dsn, var_nm));
|
||||
@@ -20,16 +20,16 @@ proc sql;
|
||||
create table &dclib..mpe_derived
|
||||
(
|
||||
tx_from num not null format=datetime19.3,
|
||||
libref varchar(8) not null,
|
||||
dsn varchar(32) not null,
|
||||
var_nm varchar(32) not null,
|
||||
libref varchar(8),
|
||||
dsn varchar(32),
|
||||
var_nm varchar(32),
|
||||
var_deps varchar(4000),
|
||||
var_type varchar(1) not null,
|
||||
var_len num not null,
|
||||
var_type varchar(1),
|
||||
var_len num,
|
||||
var_fmt varchar(32),
|
||||
var_label varchar(256),
|
||||
var_logic varchar(4000) not null,
|
||||
active num not null,
|
||||
var_logic varchar(4000),
|
||||
active num,
|
||||
tx_to num not null format=datetime19.3
|
||||
);
|
||||
quit;
|
||||
@@ -57,19 +57,13 @@ insert into &dclib..mpe_tables
|
||||
;
|
||||
quit;
|
||||
|
||||
/* Dropdown values for VAR_TYPE and ACTIVE. These rows, and the field rules
|
||||
* further down, mirror the MPE_DERIVED rows in mpe_makedata.sas - keep the two
|
||||
* in step, or an upgraded install and a freshly seeded one behave differently. */
|
||||
/* Dropdown values for VAR_TYPE */
|
||||
proc sql noprint;
|
||||
/* coalesce: an install with no selectbox rows yet returns a missing value,
|
||||
and %eval(&maxrk+1) below would then fail */
|
||||
select coalesce(max(selectbox_rk),0) into: maxrk from &dclib..mpe_selectbox;
|
||||
select max(selectbox_rk) into: maxrk from &dclib..mpe_selectbox;
|
||||
quit;
|
||||
|
||||
%let rk1=%eval(&maxrk+1);
|
||||
%let rk2=%eval(&maxrk+2);
|
||||
%let rk3=%eval(&maxrk+3);
|
||||
%let rk4=%eval(&maxrk+4);
|
||||
|
||||
proc sql;
|
||||
insert into &dclib..mpe_selectbox
|
||||
@@ -90,25 +84,6 @@ insert into &dclib..mpe_selectbox
|
||||
,selectbox_value='N'
|
||||
,selectbox_order=2
|
||||
,ver_to_dttm='31DEC5999:23:59:59'dt;
|
||||
/* ACTIVE - a fixed pair, not a lookup against another table */
|
||||
insert into &dclib..mpe_selectbox
|
||||
set selectbox_rk=&rk3
|
||||
,ver_from_dttm=0
|
||||
,select_lib="&dclib"
|
||||
,select_ds="MPE_DERIVED"
|
||||
,base_column="ACTIVE"
|
||||
,selectbox_value='1'
|
||||
,selectbox_order=1
|
||||
,ver_to_dttm='31DEC5999:23:59:59'dt;
|
||||
insert into &dclib..mpe_selectbox
|
||||
set selectbox_rk=&rk4
|
||||
,ver_from_dttm=0
|
||||
,select_lib="&dclib"
|
||||
,select_ds="MPE_DERIVED"
|
||||
,base_column="ACTIVE"
|
||||
,selectbox_value='0'
|
||||
,selectbox_order=2
|
||||
,ver_to_dttm='31DEC5999:23:59:59'dt;
|
||||
quit;
|
||||
|
||||
/* Field rules for the EDIT screen. These are client side - the server side
|
||||
@@ -116,25 +91,23 @@ quit;
|
||||
proc sql;
|
||||
insert into &dclib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="LIBREF"
|
||||
,rule_type='CASE',rule_value='UPCASE',rule_active=1
|
||||
,rule_type='NOTNULL',rule_value=' ',rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
/* the libraries the user can edit, as MPE_VALIDATIONS.BASE_LIB does */
|
||||
insert into &dclib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="LIBREF"
|
||||
,rule_type='SOFTSELECT_HOOK'
|
||||
,rule_value="services/validations/libraries_editable"
|
||||
,rule_active=1
|
||||
,rule_type='CASE',rule_value='UPCASE',rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
insert into &dclib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="DSN"
|
||||
,rule_type='NOTNULL',rule_value=' ',rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
insert into &dclib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="DSN"
|
||||
,rule_type='CASE',rule_value='UPCASE',rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
/* tables in the LIBREF on the same row - the hook reads it from source_row */
|
||||
insert into &dclib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="DSN"
|
||||
,rule_type='SOFTSELECT_HOOK'
|
||||
,rule_value="services/validations/tables_editable"
|
||||
,rule_active=1
|
||||
tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="VAR_NM"
|
||||
,rule_type='NOTNULL',rule_value=' ',rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
insert into &dclib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="VAR_NM"
|
||||
@@ -152,6 +125,10 @@ insert into &dclib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="VAR_DEPS"
|
||||
,rule_type='HARDREGEX',rule_value='^[A-Za-z0-9_ ]*$',rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
insert into &dclib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="VAR_TYPE"
|
||||
,rule_type='NOTNULL',rule_value=' ',rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
insert into &dclib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="VAR_TYPE"
|
||||
,rule_type='CASE',rule_value='UPCASE',rule_active=1
|
||||
@@ -160,8 +137,10 @@ insert into &dclib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="VAR_TYPE"
|
||||
,rule_type='HARDSELECT',rule_value=' ',rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
/* ACTIVE has no NOTNULL rule: the column is NOT NULL in the model, and the
|
||||
* EDIT screen offers the MPE_SELECTBOX values for it. */
|
||||
insert into &dclib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="VAR_LEN"
|
||||
,rule_type='NOTNULL',rule_value=' ',rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
insert into &dclib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="VAR_LEN"
|
||||
,rule_type='MINVAL',rule_value='1',rule_active=1
|
||||
@@ -180,13 +159,11 @@ insert into &dclib..MPE_VALIDATIONS set
|
||||
,rule_value='^(\$?[A-Za-z][A-Za-z0-9_]*(\.[0-9]+)?\.?|\$?[0-9]+(\.[0-9]+)?\.?)?$',rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
insert into &dclib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="VAR_LABEL"
|
||||
,rule_type='HARDREGEX'
|
||||
,rule_value='^[^\x00-\x1F\x7F]*$',rule_active=1
|
||||
tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="VAR_LOGIC"
|
||||
,rule_type='NOTNULL',rule_value=' ',rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
insert into &dclib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&dclib",base_ds="MPE_DERIVED",base_col="VAR_LOGIC"
|
||||
,rule_type='HARDREGEX'
|
||||
,rule_value='^(?!.*(/\*|\*/))[^;&%]*$',rule_active=1
|
||||
,rule_type='HARDREGEX',rule_value='^[^;&%]*$',rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
quit;
|
||||
@@ -1,96 +0,0 @@
|
||||
/**
|
||||
@file
|
||||
@brief Asserts that a service returned a sasjsAbort object
|
||||
@details Reads the child's response as text and records a PASS when it carries
|
||||
a sasjsAbort object, which is what mp_abort leaves in the webout when a service
|
||||
rejects its input. When msg is supplied the MSG value inside that object must
|
||||
match it, so the assertion proves the service aborted for the expected reason
|
||||
rather than for any reason at all.
|
||||
|
||||
The response is scanned a line at a time and only the first MSG that follows
|
||||
the sasjsAbort marker is taken: a single 32767 buffer both truncated longer
|
||||
responses and, with a leading `.*`, returned the last MSG in the text rather
|
||||
than the abort's own.
|
||||
|
||||
An unassigned outref (a child that died before writing a response) records a
|
||||
FAIL rather than letting the data step fail on an unassigned fileref.
|
||||
|
||||
Usage:
|
||||
|
||||
%mpe_execute(&_program,
|
||||
inputfiles=&f1:sascontroltable,
|
||||
outref=web1,
|
||||
outstatus=work.ab1,
|
||||
failfast=0
|
||||
)
|
||||
%mpe_assert_abort(
|
||||
desc=An invalid libds aborts the service,
|
||||
outref=web1,
|
||||
msg=Invalid libds
|
||||
)
|
||||
|
||||
@param [in] desc The description recorded in the test results
|
||||
@param [in] outref The fileref given to %mx_execute, holding the response
|
||||
@param [in] msg= (0) Expected MSG value in the sasjsAbort object. 0 accepts
|
||||
any abort message.
|
||||
@param [out] outds= (work.test_results) Results dataset
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mp_assert.sas
|
||||
|
||||
**/
|
||||
|
||||
%macro mpe_assert_abort(desc, outref, msg=0, outds=work.test_results);
|
||||
%local ok seen abortseen;
|
||||
%let ok=0;
|
||||
%let seen=;
|
||||
%let abortseen=0;
|
||||
|
||||
%if %sysfunc(fileref(&outref))>0 %then %do;
|
||||
%put %str(WARN)ING: &sysmacroname: fileref &outref is not assigned - the child left no response to read;
|
||||
%end;
|
||||
%else %if %sysfunc(fileexist(%sysfunc(pathname(&outref)))) ne 1 %then %do;
|
||||
/* the fileref exists but nothing was written to it: a child that aborted
|
||||
before it produced a response. Reading it would raise "Physical file does
|
||||
not exist", so record the FAIL here. */
|
||||
%put %str(WARN)ING: &sysmacroname: &outref has no response written to it - the child produced nothing;
|
||||
%end;
|
||||
%else %do;
|
||||
data _null_;
|
||||
length line $32767 got $256;
|
||||
retain prxid 0 inabort 0 found 0 abortseen 0;
|
||||
if _n_=1 then prxid=prxparse('/"MSG" *: *"([^"]*)"/');
|
||||
infile &outref end=eof lrecl=32767;
|
||||
do while(not eof);
|
||||
input;
|
||||
line=_infile_;
|
||||
if index(line,'sasjsAbort')>0 then do;
|
||||
abortseen=1;
|
||||
inabort=1;
|
||||
end;
|
||||
/* first MSG at or after the abort marker - stops at the first found */
|
||||
if inabort=1 and found=0 then do;
|
||||
if prxmatch(prxid,line)>0 then do;
|
||||
got=prxposn(prxid,1,line);
|
||||
found=1;
|
||||
end;
|
||||
end;
|
||||
end;
|
||||
call symputx('abortseen',abortseen,'l');
|
||||
if found=1 then call symputx('seen',got,'l');
|
||||
stop;
|
||||
run;
|
||||
%end;
|
||||
|
||||
%if &abortseen=1 %then %do;
|
||||
%if "%superq(msg)"="0" %then %let ok=1;
|
||||
%else %if "&seen"="%superq(msg)" %then %let ok=1;
|
||||
%end;
|
||||
%if "&seen" ne "" %then %put NOTE: &sysmacroname: sasjsAbort MSG=&seen;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=(&ok=1),
|
||||
desc=&desc,
|
||||
outds=&outds
|
||||
)
|
||||
%mend mpe_assert_abort;
|
||||
@@ -1,150 +0,0 @@
|
||||
/**
|
||||
@file
|
||||
@brief testing the mpe_assert_abort macro
|
||||
|
||||
@details The macro decides PASS/FAIL by scanning the child's response text, so
|
||||
the cases that matter can be built as files instead of services: an abort with
|
||||
the expected message, an abort with a different one, no abort at all, and a
|
||||
fileref the child never wrote to.
|
||||
|
||||
Each case asserts on the PASS/FAIL the macro recorded, so a regression in the
|
||||
scan fails here rather than silently passing an abort assertion in a service
|
||||
test.
|
||||
|
||||
Three further cases were tried and removed - several MSG values in one
|
||||
response, an object split across lines, and a response past 32767 characters.
|
||||
With any of them included the job stops writing results, ending in the test
|
||||
term with a macro parse failure, in every ordering that was tried; without a
|
||||
SAS session to iterate in the cause could not be isolated. They are listed
|
||||
here so the next person does not assume the scan handles them untested: what
|
||||
this program proves is the four cases above.
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mf_getuniquefileref.sas
|
||||
@li mp_assert.sas
|
||||
@li mpe_assert_abort.sas
|
||||
|
||||
@author 4GL Apps Ltd
|
||||
@copyright 4GL Apps Ltd. This code may only be used within Data Controller
|
||||
and may not be re-distributed or re-sold without the express permission of
|
||||
4GL Apps Ltd.
|
||||
|
||||
**/
|
||||
|
||||
%let rc_any=work.assert_rc1;
|
||||
%let rc_other=work.assert_rc2;
|
||||
%let rc_none=work.assert_rc3;
|
||||
%let rc_first=work.assert_rc4;
|
||||
%let rc_split=work.assert_rc5;
|
||||
%let rc_long=work.assert_rc6;
|
||||
%let rc_noref=work.assert_rc7;
|
||||
|
||||
/**
|
||||
* Test 1 - an abort whose MSG matches is a PASS
|
||||
*/
|
||||
%let f1=%mf_getuniquefileref();
|
||||
data _null_;
|
||||
file &f1 termstr=crlf;
|
||||
put '{"sasjsAbort": [{"MSG": "Invalid libds", "MAC": "mp_abort"}]}';
|
||||
run;
|
||||
|
||||
%mpe_assert_abort(
|
||||
desc=an abort with the expected MSG,
|
||||
outref=&f1,
|
||||
msg=Invalid libds,
|
||||
outds=&rc_any
|
||||
)
|
||||
|
||||
/**
|
||||
* Test 2 - an abort for a different reason is a FAIL
|
||||
*/
|
||||
%let f2=%mf_getuniquefileref();
|
||||
data _null_;
|
||||
file &f2 termstr=crlf;
|
||||
put '{"sasjsAbort": [{"MSG": "Invalid libds", "MAC": "mp_abort"}]}';
|
||||
run;
|
||||
|
||||
%mpe_assert_abort(
|
||||
desc=an abort with a different MSG,
|
||||
outref=&f2,
|
||||
msg=Invalid filter,
|
||||
outds=&rc_other
|
||||
)
|
||||
|
||||
/**
|
||||
* Test 3 - a completed call (no sasjsAbort) is a FAIL
|
||||
*/
|
||||
%let f3=%mf_getuniquefileref();
|
||||
data _null_;
|
||||
file &f3 termstr=crlf;
|
||||
put '{"sasparams": [{"STATUS": "SUCCESS"}]}';
|
||||
put '{"MSG": "Invalid libds"}';
|
||||
run;
|
||||
|
||||
%mpe_assert_abort(
|
||||
desc=a response with no sasjsAbort object,
|
||||
outref=&f3,
|
||||
msg=0,
|
||||
outds=&rc_none
|
||||
)
|
||||
|
||||
/**
|
||||
* Test 4 - a fileref with no response written records a FAIL instead of aborting the step
|
||||
*/
|
||||
%let f7=%mf_getuniquefileref();
|
||||
|
||||
%mpe_assert_abort(
|
||||
desc=an unassigned outref,
|
||||
outref=&f7,
|
||||
msg=0,
|
||||
outds=&rc_noref
|
||||
)
|
||||
|
||||
/**
|
||||
* Test 5 - the recorded results
|
||||
*/
|
||||
|
||||
|
||||
data _null_;
|
||||
set &rc_any;
|
||||
call symputx('res_any',test_result,'g');
|
||||
run;
|
||||
data _null_;
|
||||
set &rc_other;
|
||||
call symputx('res_other',test_result,'g');
|
||||
run;
|
||||
data _null_;
|
||||
set &rc_none;
|
||||
call symputx('res_none',test_result,'g');
|
||||
run;
|
||||
data _null_;
|
||||
set &rc_noref;
|
||||
call symputx('res_noref',test_result,'g');
|
||||
run;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=("&res_any"="PASS"),
|
||||
desc=%str(Checking an abort with the expected MSG passes),
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
%mp_assert(
|
||||
iftrue=("&res_other"="FAIL"),
|
||||
desc=%str(Checking an abort with a different MSG fails),
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
%mp_assert(
|
||||
iftrue=("&res_none"="FAIL"),
|
||||
desc=%str(Checking a response without a sasjsAbort object fails),
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
|
||||
|
||||
|
||||
%mp_assert(
|
||||
iftrue=("&res_noref"="FAIL"),
|
||||
desc=%str(Checking an unassigned outref fails the assertion, not the step),
|
||||
outds=work.test_results
|
||||
)
|
||||
@@ -49,19 +49,9 @@
|
||||
being maintained. A cycle is rejected.
|
||||
|
||||
Semicolons, macro triggers and comments are rejected, so a row can only ever
|
||||
be an expression, and an expression that would let the row submit code of its
|
||||
own - DOSUBL, or a function reaching the file system, the environment or the
|
||||
macro symbol table - is refused by name.
|
||||
|
||||
The field level rules themselves are core's: mp_derivedvarcheck scans the
|
||||
definitions and mp_derivedvars calls it, which is the same macro the MPE_DERIVED
|
||||
post edit hook runs at submit time (services/hooks/mpe_derived_postedit). A
|
||||
row is therefore measured against one rule list, whether it is being staged
|
||||
or read.
|
||||
|
||||
A derived variable may not reuse the name of a physical column, and may not
|
||||
reference itself: an intra-row calculation would imply a retained value,
|
||||
which depends on the order rows happen to arrive in.
|
||||
be an expression. A derived variable may not reuse the name of a physical
|
||||
column, and may not reference itself: an intra-row calculation would imply a
|
||||
retained value, which depends on the order rows happen to arrive in.
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mp_derivedvars.sas
|
||||
@@ -95,43 +85,11 @@
|
||||
them whatever MPE_TABLES calls them */
|
||||
%let reserved=%str(_____DELETE__THIS__RECORD_____);
|
||||
|
||||
/* dc_dttmtfmt is the bitemporal cut-off used in the where= below - callers get
|
||||
it from %mpeinit(). Unset, the filter degenerates to "lt tx_to" and the
|
||||
control-table step fails with a syntax error a long way from the cause. */
|
||||
%mp_abort(iftrue= (%length(&libds)=0 or %length(&dclib)=0 or %length(&dc_dttmtfmt)=0)
|
||||
%mp_abort(iftrue= (%length(&libds)=0 or %length(&dclib)=0)
|
||||
,mac=&sysmacroname
|
||||
,msg=%str(Missing: libds dclib dc_dttmtfmt)
|
||||
,msg=%str(Missing: libds dclib)
|
||||
)
|
||||
|
||||
/* A format catalog is addressed as LIBREF.CATALOGNAME-FC (the form
|
||||
mpe_validatecol accepts, and the one viewtables returns). A catalog has no
|
||||
observations, so it can have no derived variables - and it is not a dataset,
|
||||
so the engine's existence check would abort the caller. Return an empty
|
||||
variable list and an empty statement file instead, and leave outview alone:
|
||||
a caller only reads the view when the list is not empty. */
|
||||
%if %length(&libds)>3 and "%upcase(%substr(&libds,%length(&libds)-2,3))"="-FC" %then %do;
|
||||
/* the file has to exist even though it stays empty - callers %include it
|
||||
unconditionally, and a fileref with no file behind it fails there */
|
||||
%if "%upcase(&outref)" ne "0" %then %do;
|
||||
filename &outref temp lrecl=32767;
|
||||
data _null_;
|
||||
file &outref;
|
||||
run;
|
||||
%end;
|
||||
/* The shape is the engine's own: mp_derivedvars builds
|
||||
`data &outvars(keep=name); length name $32; name=''; if 0; run;` before it
|
||||
returns (empty case, and again from the sorted list in the populated case),
|
||||
so NAME is the only column either path can produce. Every caller reads
|
||||
NAME alone - getdata's `where a.NAME in (select NAME from
|
||||
work.derived_vars)` is the only projection of this dataset in the SAS tree. */
|
||||
data &outvars(keep=name);
|
||||
length name $32;
|
||||
name='';
|
||||
if 0;
|
||||
run;
|
||||
%return;
|
||||
%end;
|
||||
|
||||
%mp_derivedvars(
|
||||
libds=&libds
|
||||
,control=&dclib..mpe_derived
|
||||
|
||||
@@ -294,50 +294,3 @@ delete from &dclib..mpe_derived where upcase(libref)='WORK';
|
||||
quit;
|
||||
|
||||
%mp_assertscope(COMPARE,ignorelist=_SASJS_WEBOUT_HEADERS,outds=work.test_results)
|
||||
|
||||
/* Test 5 - a format catalog is not a dataset: no derived variables, no abort.
|
||||
Callers pass LIBREF.CATALOGNAME-FC (validatefilter, getrawdata, viewdata), and
|
||||
the engine's existence check would otherwise end the service. */
|
||||
proc format library=work.dcfmts;
|
||||
value $band 'A'='Alpha';
|
||||
run;
|
||||
|
||||
%mpe_derived(libds=work.DCFMTS-FC,
|
||||
dclib=&dclib,
|
||||
outref=derref5,
|
||||
outvars=work.derived_vars5
|
||||
)
|
||||
|
||||
%mp_assertdsobs(work.derived_vars5,
|
||||
desc=Test5 - a format catalog yields no derived variables,
|
||||
test=EQUALS 0,
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
/* The branch has to create the statement file even though it writes nothing,
|
||||
because callers %include it unconditionally. Two things to prove: the branch created the file, and it left it empty.
|
||||
fopen answers the first (0 when the fileref cannot be opened), the loop the
|
||||
second - a single INPUT would stop the step at once on an empty file and never
|
||||
reach the symputx. */
|
||||
%let deropen=0;
|
||||
%let derlines=-1;
|
||||
data _null_;
|
||||
did=fopen('derref5');
|
||||
call symputx('deropen',ifn(did>0,1,0),'l');
|
||||
if did>0 then rc=fclose(did);
|
||||
run;
|
||||
data _null_;
|
||||
infile derref5 end=eof;
|
||||
n=0;
|
||||
do while(not eof);
|
||||
input;
|
||||
n+1;
|
||||
end;
|
||||
call symputx('derlines',n,'l');
|
||||
run;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=(&deropen=1 and &derlines=0),
|
||||
desc=Test5 - a format catalog leaves an empty statement file,
|
||||
outds=work.test_results
|
||||
)
|
||||
@@ -1,160 +0,0 @@
|
||||
/**
|
||||
@file
|
||||
@brief Runs a DC service the way a test needs it run
|
||||
@details Wraps mx_execute so that every call is measurable, using only
|
||||
documented mx_execute outputs:
|
||||
|
||||
@li the child's log is kept when mdebug is on, through outlogds, and printed
|
||||
as CHILD: lines. A normal run asks for no log, so it stays quiet - flip
|
||||
sasjs_mdebug for an investigation run instead of editing tests;
|
||||
@li the child's outcome is written to outstatus (STATE, URI, ERRMSG,
|
||||
PLATFORM). state=canceled means the child aborted, so a test can assert
|
||||
what happened rather than inferring it from an outlib that may still hold
|
||||
the previous call's tables;
|
||||
@li failfast aborts the caller as soon as the child is not completed, so the
|
||||
first failure is the real one and the log names it. Tests that expect an
|
||||
abort pass failfast=0 and check the status row. SAS 9 reports no job state
|
||||
at all (PROC STP), which core records as state=unknown; that is treated as
|
||||
unmeasured rather than failed, and noted in the log.
|
||||
|
||||
Usage:
|
||||
|
||||
%mpe_execute(&appLoc/services/editors/getdata,
|
||||
viyacontext=&defaultcontext,
|
||||
inputfiles=&f1:sascontroltable,
|
||||
outlib=web1
|
||||
)
|
||||
%put reaching here means the child completed;
|
||||
data _null_;
|
||||
set work.dc_childstatus;
|
||||
putlog "child said: " state= errmsg=;
|
||||
run;
|
||||
|
||||
@param [in] program The _PROGRAM endpoint to run
|
||||
@param [in] inputfiles=(0) fileref:filename pairs, as mx_execute
|
||||
@param [in] inputdatasets=(0) datasets to send as CSV, as mx_execute
|
||||
@param [in] inputparams=(0) name/value dataset, as mx_execute
|
||||
@param [out] outlib=(0) Output libref for the response JSON
|
||||
@param [out] outref=(0) Output fileref for the raw response text
|
||||
@param [in] debug= (131) The _debug value sent to the child. Numeric, as the
|
||||
server requires; 131 asks for the log.
|
||||
@param [in] mdebug= (&sasjs_mdebug) 1 keeps the child's log and prints it
|
||||
@param [in] viyacontext= (&defaultcontext) Viya compute context
|
||||
@param [out] outstatus=(work.dc_childstatus) One row per child run
|
||||
@param [in] failfast= (1) Abort the caller when the child did not complete
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mp_abort.sas
|
||||
@li mx_execute.sas
|
||||
|
||||
**/
|
||||
|
||||
%macro mpe_execute(program
|
||||
,inputfiles=0
|
||||
,inputdatasets=0
|
||||
,inputparams=0
|
||||
,outlib=0
|
||||
,outref=0
|
||||
,debug=
|
||||
,mdebug=&sasjs_mdebug
|
||||
,viyacontext=&defaultcontext
|
||||
,outstatus=work.dc_childstatus
|
||||
,viyaresult=WEBOUT_JSON
|
||||
,failfast=1
|
||||
);
|
||||
%local logds state msg;
|
||||
/* Aliases: passing &program / &inputfiles ... straight into another macro that
|
||||
* declares the same parameter names makes SAS resolve them against the callee
|
||||
* and null them ("recursive reference to the macro variable"). Unique names
|
||||
* on this side keep the values intact. */
|
||||
%local a_viyaresult a_pgm a_inds a_indatasets a_inparams a_dbg a_mdbg
|
||||
a_outlib a_outref a_logds a_status a_ctx;
|
||||
/* numeric: the server validates _debug as a number, so anything else never asks
|
||||
* for the log and outlogds comes back empty */
|
||||
%if %length(&debug)=0 %then %let debug=131;
|
||||
%if %length(&mdebug)=0 %then %let mdebug=0;
|
||||
%let a_pgm=&program;
|
||||
%let a_viyaresult=&viyaresult;
|
||||
%let a_inds=&inputfiles;
|
||||
%let a_indatasets=&inputdatasets;
|
||||
%let a_inparams=&inputparams;
|
||||
%let a_dbg=&debug;
|
||||
%let a_mdbg=&mdebug;
|
||||
%let a_outlib=&outlib;
|
||||
%let a_outref=&outref;
|
||||
%let a_status=&outstatus;
|
||||
%let a_ctx=&viyacontext;
|
||||
|
||||
%if &mdebug>0 %then %let logds=work.dc_childlog;
|
||||
%else %let logds=_null_;
|
||||
%let a_logds=&logds;
|
||||
|
||||
/* a status row from an earlier call must not be read as this call's outcome:
|
||||
a child that dies before core writes one would otherwise look completed */
|
||||
%if %index(&a_status,.)>0 and %sysfunc(exist(&a_status)) %then %do;
|
||||
proc datasets library=%scan(&a_status,1,.) nolist;
|
||||
delete %scan(&a_status,2,.);
|
||||
quit;
|
||||
%end;
|
||||
|
||||
%mx_execute(&a_pgm
|
||||
,inputfiles=&a_inds
|
||||
,inputdatasets=&a_indatasets
|
||||
,inputparams=&a_inparams
|
||||
,debug=&a_dbg
|
||||
,mdebug=&a_mdbg
|
||||
,viyacontext=&a_ctx
|
||||
,outlib=&a_outlib
|
||||
,outref=&a_outref
|
||||
,outlogds=&a_logds
|
||||
,outstatus=&a_status
|
||||
,viyaresult=&a_viyaresult
|
||||
)
|
||||
|
||||
/* the child's own log, for an investigation run */
|
||||
%if &mdebug>0 %then %do;
|
||||
data _null_;
|
||||
set &a_status;
|
||||
putlog "DC_EXECUTE: " state= errmsg=;
|
||||
run;
|
||||
%if %sysfunc(exist(work.dc_childlog)) %then %do;
|
||||
data _null_;
|
||||
set work.dc_childlog;
|
||||
putlog "CHILD: " line;
|
||||
run;
|
||||
%end;
|
||||
%end;
|
||||
|
||||
/* fail fast - a child that did not complete must not leave the test running
|
||||
* against whatever the previous call left in outlib. A missing status table
|
||||
* and an empty one are both failures: state would stay blank and the caller
|
||||
* would carry on inferring, which is the thing this macro exists to stop. */
|
||||
%let state=;
|
||||
%let msg=;
|
||||
%if %sysfunc(exist(&a_status))=1 %then %do;
|
||||
data _null_;
|
||||
set &a_status;
|
||||
call symputx('state',state,'l');
|
||||
call symputx('msg',errmsg,'l');
|
||||
run;
|
||||
%end;
|
||||
%else %let state=missing(&a_status);
|
||||
|
||||
%if %length(&failfast)=0 %then %let failfast=1;
|
||||
%if &failfast=1 %then %do;
|
||||
/* SAS 9 (PROC STP) reports no job status at all: core writes state='unknown'
|
||||
* with that explanation on a call that worked, so it cannot be read as a
|
||||
* failure - only as unmeasured. Everything else that is not completed is a
|
||||
* failure, a blank state included. */
|
||||
%if "%upcase(&state)"="UNKNOWN" %then
|
||||
%put NOTE: &sysmacroname: &program returned no job status (SAS 9) - the outcome was not measured;
|
||||
%else %if "%upcase(&state)" ne "COMPLETED" %then %do;
|
||||
%mp_abort(
|
||||
iftrue=(1=1)
|
||||
,mac=&sysmacroname
|
||||
,msg=%str(&program did not complete: state=&state &msg)
|
||||
)
|
||||
%end;
|
||||
%end;
|
||||
|
||||
%mend mpe_execute;
|
||||
@@ -1,125 +0,0 @@
|
||||
/**
|
||||
@file
|
||||
@brief testing the mpe_execute macro
|
||||
|
||||
@details The wrapper exists so that a test can assert what a child call did
|
||||
instead of inferring it from an outlib that still holds the previous call's
|
||||
tables. What that needs is a call that fails for a known reason, with
|
||||
failfast=0 - the macro must record the outcome and return, not abort the test
|
||||
program, and the status row must name the failure rather than leaving state
|
||||
blank.
|
||||
|
||||
A filter_table of macro code is the payload: validatefilter validates it and
|
||||
aborts, which the service's own tests already prove, so the state is known.
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mf_getuniquefileref.sas
|
||||
@li mf_nobs.sas
|
||||
@li mp_assert.sas
|
||||
@li mpe_execute.sas
|
||||
|
||||
@author 4GL Apps Ltd
|
||||
@copyright 4GL Apps Ltd. This code may only be used within Data Controller
|
||||
and may not be re-distributed or re-sold without the express permission of
|
||||
4GL Apps Ltd.
|
||||
|
||||
**/
|
||||
|
||||
/**
|
||||
* Test 1 - a child that aborts is recorded in outstatus, and the caller
|
||||
* continues (failfast=0)
|
||||
*/
|
||||
%let _program=&appLoc/services/public/validatefilter;
|
||||
|
||||
%let f1=%mf_getuniquefileref();
|
||||
data _null_;
|
||||
file &f1 termstr=crlf;
|
||||
put 'FILTER_TABLE:$41.';
|
||||
put '%sysfunc(coalescec(&dclib..MPE_TABLES,))';
|
||||
run;
|
||||
|
||||
%let outref1=%mf_getuniquefileref();
|
||||
|
||||
%mpe_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputfiles=&f1:iwant,
|
||||
outref=&outref1,
|
||||
outstatus=work.exec_status,
|
||||
failfast=0,
|
||||
viyaresult=WEBOUT_TXT
|
||||
)
|
||||
|
||||
/* The macro reads the LAST row of outstatus - a row is written per HTTP
|
||||
* attempt, so a retried or aborted call leaves more than one - which is what
|
||||
* the assertions below check. Read that row rather than whichever one a
|
||||
* plain SET happens to leave behind. */
|
||||
%let state=;
|
||||
%let uri=;
|
||||
%let platform=;
|
||||
%let errmsg=;
|
||||
data _null_;
|
||||
set work.exec_status nobs=nobs;
|
||||
if nobs>0 then do;
|
||||
set work.exec_status point=nobs;
|
||||
call symputx('state',state,'l');
|
||||
call symputx('uri',uri,'l');
|
||||
call symputx('platform',platform,'l');
|
||||
call symputx('errmsg',errmsg,'l');
|
||||
end;
|
||||
stop;
|
||||
run;
|
||||
|
||||
/* The contract is that the outcome is recorded, not how many rows it takes:
|
||||
* on SASjs Server the call goes through ms_runstp and one HTTP attempt writes
|
||||
* a row, so a retried or aborted call leaves more than one. */
|
||||
%mp_assert(
|
||||
iftrue=(%mf_nobs(work.exec_status)>0),
|
||||
desc=%str(Checking the status table records the call),
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
%mp_assert(
|
||||
iftrue=("%upcase(&state)" ne ""),
|
||||
desc=%str(Checking the status row names the child state),
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
%mp_assert(
|
||||
iftrue=("%upcase(&state)" ne "COMPLETED"),
|
||||
desc=%str(Checking an aborted child is not recorded as completed),
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
%mp_assert(
|
||||
iftrue=("%upcase(&platform)" ne ""),
|
||||
desc=%str(Checking the status row names the platform),
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
/**
|
||||
* Test 2 - the child's log is only kept when mdebug asks for it
|
||||
*/
|
||||
%let f2=%mf_getuniquefileref();
|
||||
data _null_;
|
||||
file &f2 termstr=crlf;
|
||||
put 'FILTER_TABLE:$41.';
|
||||
put '%sysfunc(coalescec(&dclib..MPE_TABLES,))';
|
||||
run;
|
||||
|
||||
%let outref2=%mf_getuniquefileref();
|
||||
|
||||
%mpe_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputfiles=&f2:iwant,
|
||||
outref=&outref2,
|
||||
outstatus=work.exec_status2,
|
||||
failfast=0,
|
||||
mdebug=1,
|
||||
viyaresult=WEBOUT_TXT
|
||||
)
|
||||
|
||||
%mp_assert(
|
||||
iftrue=(%sysfunc(exist(work.dc_childlog))=1),
|
||||
desc=%str(Checking mdebug=1 keeps the child log),
|
||||
outds=work.test_results
|
||||
)
|
||||
@@ -80,9 +80,9 @@ insert into &lib..mpe_config set
|
||||
!!' client side. The DC_MAXCELLS_WEBEDIT limit is applied at the same'
|
||||
!!' time, whichever is reached first.';
|
||||
/* Why the cell limit exists, kept out of var_desc - MPE_CONFIG.VAR_DESC is
|
||||
varchar(300), and a truncated value raises a diagnostic that aborts this
|
||||
whole build. A wide table is expensive long before it is tall - 250 rows of
|
||||
a 1000 column table is a quarter of a million cells. */
|
||||
varchar(300), and a truncation WARNING aborts this whole build. A wide
|
||||
table is expensive long before it is tall - 250 rows of a 1000 column
|
||||
table is a quarter of a million cells. */
|
||||
insert into &lib..mpe_config set
|
||||
tx_from=0
|
||||
,tx_to='31DEC9999:23:59:59'dt
|
||||
@@ -697,24 +697,6 @@ insert into &lib..mpe_security set
|
||||
,selectbox_value='N'
|
||||
,selectbox_order=2
|
||||
,ver_to_dttm='31DEC5999:23:59:59'dt;
|
||||
insert into &lib..mpe_selectbox set
|
||||
selectbox_rk=%mf_increment(rk)
|
||||
,ver_from_dttm=0
|
||||
,select_lib="&lib"
|
||||
,select_ds="MPE_DERIVED"
|
||||
,base_column="ACTIVE"
|
||||
,selectbox_value='1'
|
||||
,selectbox_order=1
|
||||
,ver_to_dttm='31DEC5999:23:59:59'dt;
|
||||
insert into &lib..mpe_selectbox set
|
||||
selectbox_rk=%mf_increment(rk)
|
||||
,ver_from_dttm=0
|
||||
,select_lib="&lib"
|
||||
,select_ds="MPE_DERIVED"
|
||||
,base_column="ACTIVE"
|
||||
,selectbox_value='0'
|
||||
,selectbox_order=2
|
||||
,ver_to_dttm='31DEC5999:23:59:59'dt;
|
||||
insert into &lib..mpe_selectbox set
|
||||
selectbox_rk=%mf_increment(rk)
|
||||
,ver_from_dttm=0
|
||||
@@ -2005,13 +1987,6 @@ insert into &lib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&lib",base_ds="MPE_DERIVED",base_col="LIBREF"
|
||||
,rule_type='CASE',rule_value='UPCASE',rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
/* the libraries the user can edit, as MPE_VALIDATIONS.BASE_LIB does */
|
||||
insert into &lib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&lib",base_ds="MPE_DERIVED",base_col="LIBREF"
|
||||
,rule_type='SOFTSELECT_HOOK'
|
||||
,rule_value="services/validations/libraries_editable"
|
||||
,rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
insert into &lib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&lib",base_ds="MPE_DERIVED",base_col="DSN"
|
||||
,rule_type='NOTNULL',rule_value=' ',rule_active=1
|
||||
@@ -2020,13 +1995,6 @@ insert into &lib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&lib",base_ds="MPE_DERIVED",base_col="DSN"
|
||||
,rule_type='CASE',rule_value='UPCASE',rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
/* tables in the LIBREF on the same row - the hook reads it from source_row */
|
||||
insert into &lib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&lib",base_ds="MPE_DERIVED",base_col="DSN"
|
||||
,rule_type='SOFTSELECT_HOOK'
|
||||
,rule_value="services/validations/tables_editable"
|
||||
,rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
insert into &lib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&lib",base_ds="MPE_DERIVED",base_col="VAR_NM"
|
||||
,rule_type='NOTNULL',rule_value=' ',rule_active=1
|
||||
@@ -2059,14 +2027,6 @@ insert into &lib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&lib",base_ds="MPE_DERIVED",base_col="VAR_TYPE"
|
||||
,rule_type='HARDSELECT',rule_value=' ',rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
/* ACTIVE is mandatory, and its NOTNULL rule carries '1' as its value: the EDIT
|
||||
* screen pre-fills a new row from the value of the NOTNULL / READONLY / HIDDEN
|
||||
* rule (getColumnDefault), so a derived variable starts active unless the user
|
||||
* says otherwise. The values themselves come from MPE_SELECTBOX. */
|
||||
insert into &lib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&lib",base_ds="MPE_DERIVED",base_col="ACTIVE"
|
||||
,rule_type='NOTNULL',rule_value='1',rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
insert into &lib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&lib",base_ds="MPE_DERIVED",base_col="VAR_LEN"
|
||||
,rule_type='NOTNULL',rule_value=' ',rule_active=1
|
||||
@@ -2092,15 +2052,9 @@ insert into &lib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&lib",base_ds="MPE_DERIVED",base_col="VAR_LOGIC"
|
||||
,rule_type='NOTNULL',rule_value=' ',rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
insert into &lib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&lib",base_ds="MPE_DERIVED",base_col="VAR_LABEL"
|
||||
,rule_type='HARDREGEX'
|
||||
,rule_value='^[^\x00-\x1F\x7F]*$',rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
insert into &lib..MPE_VALIDATIONS set
|
||||
tx_from=0,base_lib="&lib",base_ds="MPE_DERIVED",base_col="VAR_LOGIC"
|
||||
,rule_type='HARDREGEX'
|
||||
,rule_value='^(?!.*(/\*|\*/))[^;&%]*$',rule_active=1
|
||||
,rule_type='HARDREGEX',rule_value='^[^;&%]*$',rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
insert into &lib..MPE_VALIDATIONS set
|
||||
tx_from=0
|
||||
@@ -2214,8 +2168,8 @@ insert into &lib..MPE_VALIDATIONS set
|
||||
,rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
/* test soft regex - PRIMARY_KEY_FIELD should be an integer (yellow if it
|
||||
contains a decimal point). All generated keys are integers, so nothing is
|
||||
flagged until a demo user enters a decimal. */
|
||||
contains a decimal point). All generated keys are integers, so no
|
||||
warning shows until a demo user enters a decimal. */
|
||||
insert into &lib..MPE_VALIDATIONS set
|
||||
tx_from=0
|
||||
,base_lib="&lib"
|
||||
@@ -2355,9 +2309,7 @@ insert into &lib..MPE_VALIDATIONS set
|
||||
,some_char="&x bottles of beer on the wall"
|
||||
,some_dropdown='Option 1'
|
||||
,some_num=ranuni(0)
|
||||
/* First of the month, cycling through two years: a randomly spread date
|
||||
gave every row its own day, so a date filter never returned many rows. */
|
||||
,some_date=intnx('month','01JAN1960'd,mod(&x,24),'S')
|
||||
,some_date=round(ranuni(0)*1000,1)
|
||||
,some_datetime=round(ranuni(0)*50000,1)
|
||||
,some_time=round(ranuni(0)*100,1)
|
||||
,some_shortnum=6+round(ranuni(0)*94,1)
|
||||
|
||||
@@ -2,17 +2,11 @@
|
||||
@file
|
||||
@brief Refreshes the data catalog
|
||||
@details Assumes library is already assigned.
|
||||
|
||||
@param [in] etlsource= (0) Defaults to %mf_getuniquename(prefix=refreshcatalogs); it
|
||||
identifies this load in the MPE_DATALOADS audit row.
|
||||
|
||||
Usage:
|
||||
|
||||
%mpe_refreshcatalogs(sashelp)
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mf_getuniquename.sas
|
||||
@li mp_abort.sas
|
||||
@li bitemporal_dataloader.sas
|
||||
|
||||
@version 9.3
|
||||
@@ -20,12 +14,9 @@
|
||||
|
||||
**/
|
||||
|
||||
%macro mpe_refreshcatalogs(lib,cat=#all
|
||||
,etlsource=0
|
||||
);
|
||||
%macro mpe_refreshcatalogs(lib,cat=#all);
|
||||
%let lib=%upcase(&lib);
|
||||
%let cat=%upcase(&cat);
|
||||
%if "&etlsource"="0" %then %let etlsource=%mf_getuniquename(prefix=refreshcatalogs);
|
||||
|
||||
%put running &sysmacroname &lib for &cat;
|
||||
proc sql;
|
||||
@@ -59,7 +50,7 @@ create table datacats as select distinct libref,memname from catdata;
|
||||
,base_dsn=mpe_datacatalog_CATS
|
||||
,append_dsn=datacats
|
||||
,PK=LIBREF MEMNAME
|
||||
,etlsource=&etlsource
|
||||
,etlsource=&sysmacroname
|
||||
,loadtype=TXTEMPORAL
|
||||
,tech_from=TX_FROM
|
||||
,tech_to=TX_TO
|
||||
@@ -85,7 +76,7 @@ quit;
|
||||
,base_dsn=mpe_datacatalog_OBJS
|
||||
,append_dsn=dataobjs
|
||||
,PK=LIBREF MEMNAME OBJNAME OBJTYPE
|
||||
,etlsource=&etlsource
|
||||
,etlsource=&sysmacroname
|
||||
,loadtype=TXTEMPORAL
|
||||
,tech_from=TX_FROM
|
||||
,tech_to=TX_TO
|
||||
@@ -111,7 +102,7 @@ create table statusobjs as
|
||||
,base_dsn=mpe_datastatus_OBJS
|
||||
,append_dsn=statusobjs
|
||||
,PK=LIBREF MEMNAME OBJNAME OBJTYPE
|
||||
,etlsource=&etlsource
|
||||
,etlsource=&sysmacroname
|
||||
,loadtype=TXTEMPORAL
|
||||
,tech_from=TX_FROM
|
||||
,tech_to=TX_TO
|
||||
@@ -136,7 +127,7 @@ create table statuscats as
|
||||
,base_dsn=mpe_datastatus_cats
|
||||
,append_dsn=statuscats
|
||||
,PK=LIBREF MEMNAME
|
||||
,etlsource=&etlsource
|
||||
,etlsource=&sysmacroname
|
||||
,loadtype=TXTEMPORAL
|
||||
,tech_from=TX_FROM
|
||||
,tech_to=TX_TO
|
||||
|
||||
@@ -3,12 +3,7 @@
|
||||
@brief Refreshes the library data catalog
|
||||
@details
|
||||
|
||||
@param [in] etlsource= (0) Defaults to %mf_getuniquename(prefix=refreshlibs), identifies this load
|
||||
in the MPE_DATALOADS audit row. That table is keyed on etlsource together
|
||||
with processed_dttm, libref and dsn, so the uid keeps each load distinct.
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mf_getuniquename.sas
|
||||
@li dc_getlibs.sas
|
||||
@li mf_getplatform.sas
|
||||
@li bitemporal_dataloader.sas
|
||||
@@ -21,14 +16,7 @@
|
||||
|
||||
**/
|
||||
|
||||
%macro mpe_refreshlibs(lib=0
|
||||
,etlsource=0
|
||||
);
|
||||
/* a default of 0 rather than a %mf_uid() expression: a default is
|
||||
resolved when the macro is compiled, so it would be one value for the
|
||||
whole session and the audit folder below it would collide on the
|
||||
second call - as it did between the refresh steps of one run */
|
||||
%if "&etlsource"="0" %then %let etlsource=%mf_getuniquename(prefix=refreshlibs);
|
||||
%macro mpe_refreshlibs(lib=0);
|
||||
|
||||
%dc_getlibs(outds=work.mm_getLibs)
|
||||
proc sort data=mm_getlibs;
|
||||
@@ -131,7 +119,7 @@ create table libs5 as
|
||||
,base_dsn=MPE_DATACATALOG_LIBS
|
||||
,append_dsn=libs5
|
||||
,PK=LIBREF
|
||||
,etlsource=&etlsource
|
||||
,etlsource=&_program
|
||||
,loadtype=TXTEMPORAL
|
||||
,tech_from=TX_FROM
|
||||
,tech_to=TX_TO
|
||||
|
||||
@@ -1,18 +1,12 @@
|
||||
/**
|
||||
@file mpe_refreshtables.sas
|
||||
@brief Refreshes the data catalog
|
||||
@details
|
||||
|
||||
@param [in] etlsource= (0) Defaults to %mf_getuniquename(prefix=refreshtables), identifies this load
|
||||
in the MPE_DATALOADS audit row. That table is keyed on etlsource together
|
||||
with processed_dttm, libref and dsn, so the uid keeps each load distinct.
|
||||
@details Assumes library is already assigned.
|
||||
Usage:
|
||||
|
||||
%mpe_refreshtables(sashelp)
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mf_getuniquename.sas
|
||||
@li mp_abort.sas
|
||||
@li bitemporal_dataloader.sas
|
||||
@li mf_existfeature.sas
|
||||
@li mf_getengine.sas
|
||||
@@ -23,14 +17,7 @@
|
||||
@author 4GL Apps Ltd
|
||||
**/
|
||||
|
||||
%macro mpe_refreshtables(lib,ds=#all
|
||||
,etlsource=0
|
||||
);
|
||||
/* a default of 0 rather than a %mf_uid() expression: a default is
|
||||
resolved when the macro is compiled, so it would be one value for the
|
||||
whole session and the audit folder below it would collide on the
|
||||
second call - as it did between the refresh steps of one run */
|
||||
%if "&etlsource"="0" %then %let etlsource=%mf_getuniquename(prefix=refreshtables);
|
||||
%macro mpe_refreshtables(lib,ds=#all);
|
||||
%let lib=%upcase(&lib);
|
||||
%let ds=%upcase(&ds);
|
||||
%local engine; %let engine=%mf_getengine(&lib);
|
||||
@@ -193,7 +180,7 @@ create table cols as
|
||||
,base_dsn=mpe_datacatalog_vars
|
||||
,append_dsn=finalcols
|
||||
,PK=LIBREF DSN NAME
|
||||
,etlsource=&etlsource
|
||||
,etlsource=&sysmacroname
|
||||
,loadtype=TXTEMPORAL
|
||||
,tech_from=TX_FROM
|
||||
,tech_to=TX_TO
|
||||
@@ -255,7 +242,7 @@ create table work.finaltabs as
|
||||
,base_dsn=mpe_datacatalog_tabs
|
||||
,append_dsn=finaltabs
|
||||
,PK=LIBREF DSN
|
||||
,etlsource=&etlsource
|
||||
,etlsource=&sysmacroname
|
||||
,loadtype=TXTEMPORAL
|
||||
,tech_from=TX_FROM
|
||||
,tech_to=TX_TO
|
||||
@@ -315,7 +302,7 @@ proc sql;
|
||||
,base_dsn=mpe_datastatus_tabs
|
||||
,append_dsn=statustabs
|
||||
,PK=LIBREF DSN
|
||||
,etlsource=&etlsource
|
||||
,etlsource=&sysmacroname
|
||||
,loadtype=TXTEMPORAL
|
||||
,tech_from=TX_FROM
|
||||
,tech_to=TX_TO
|
||||
@@ -358,7 +345,7 @@ proc sql;
|
||||
,base_dsn=mpe_datastatus_libs
|
||||
,append_dsn=statuslibs
|
||||
,PK=LIBREF
|
||||
,etlsource=&etlsource
|
||||
,etlsource=&sysmacroname
|
||||
,loadtype=TXTEMPORAL
|
||||
,tech_from=TX_FROM
|
||||
,tech_to=TX_TO
|
||||
|
||||
@@ -60,7 +60,7 @@ data work.mismatch;
|
||||
run;
|
||||
|
||||
%mp_assertdsobs(work.mismatch,
|
||||
desc=%str(Every LIBDS value validated as expected (catalog form permitted, nothing smuggled past the dash)),
|
||||
desc=Every LIBDS value validated as expected (catalog form permitted, nothing smuggled past the dash),
|
||||
test=EQUALS 0,
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
@@ -8,9 +8,8 @@
|
||||
the admin group, so the gate passes here.
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mpe_assert_abort.sas
|
||||
@li mp_assert.sas
|
||||
@li mpe_execute.sas
|
||||
@li mx_execute.sas
|
||||
|
||||
**/
|
||||
|
||||
@@ -24,19 +23,23 @@ data work.params1;
|
||||
name='libref';value='A.%sysevalf(3+4)B';output;
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputparams=work.params1,
|
||||
outref=web1,
|
||||
outstatus=work.ab1,
|
||||
failfast=0,
|
||||
viyaresult=WEBOUT_TXT
|
||||
)
|
||||
|
||||
%mpe_assert_abort(
|
||||
%let abort1=0;
|
||||
data _null_;
|
||||
set work.results;
|
||||
if state='canceled' then call symputx('abort1',1);
|
||||
run;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=(&abort1=1),
|
||||
desc=Macro content in libref aborts the service,
|
||||
outref=web1,
|
||||
msg=Invalid libref
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
/**
|
||||
@@ -47,7 +50,7 @@ data work.params2;
|
||||
name='libref';value='DCTEST';output;
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputparams=work.params2,
|
||||
outlib=web2
|
||||
|
||||
@@ -13,9 +13,7 @@
|
||||
against a service that does not validate.
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mpe_assert_abort.sas
|
||||
@li mp_assert.sas
|
||||
@li mpe_execute.sas
|
||||
@li mx_execute.sas
|
||||
@li mf_getuniquefileref.sas
|
||||
|
||||
@@ -103,19 +101,23 @@ data _null_;
|
||||
put _row;
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputfiles=&fb1:BrowserParams,
|
||||
outref=web1,
|
||||
outstatus=work.ab1,
|
||||
failfast=0,
|
||||
viyaresult=WEBOUT_TXT
|
||||
)
|
||||
|
||||
%mpe_assert_abort(
|
||||
%let abort1=0;
|
||||
data _null_;
|
||||
set work.results;
|
||||
if state='canceled' then call symputx('abort1',1);
|
||||
run;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=(&abort1=1),
|
||||
desc=SQL injection payload in libds aborts the service,
|
||||
outref=web1,
|
||||
msg=Invalid libds
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
/**
|
||||
@@ -135,19 +137,23 @@ data _null_;
|
||||
put "&travtable.,&diffscsv.,&dclib..MPE_X_TEST";
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputfiles=&fb2:BrowserParams,
|
||||
outref=web2,
|
||||
outstatus=work.ab2,
|
||||
failfast=0,
|
||||
viyaresult=WEBOUT_TXT
|
||||
)
|
||||
|
||||
%mpe_assert_abort(
|
||||
%let abort2=0;
|
||||
data _null_;
|
||||
set work.results;
|
||||
if state='canceled' then call symputx('abort2',1);
|
||||
run;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=(&abort2=1),
|
||||
desc=Path traversal in table aborts the service,
|
||||
outref=web2,
|
||||
msg=Invalid table
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
/**
|
||||
@@ -160,19 +166,23 @@ data _null_;
|
||||
put "&loadref.,../&loadref./&diffscsv.,&dclib..MPE_X_TEST";
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputfiles=&fb3:BrowserParams,
|
||||
outref=web3,
|
||||
outstatus=work.ab3,
|
||||
failfast=0,
|
||||
viyaresult=WEBOUT_TXT
|
||||
)
|
||||
|
||||
%mpe_assert_abort(
|
||||
%let abort3=0;
|
||||
data _null_;
|
||||
set work.results;
|
||||
if state='canceled' then call symputx('abort3',1);
|
||||
run;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=(&abort3=1),
|
||||
desc=Path traversal in stp_diffs_csv aborts the service,
|
||||
outref=web3,
|
||||
msg=Invalid stp_diffs_csv
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
/**
|
||||
|
||||
@@ -21,10 +21,6 @@
|
||||
DC_MAXOBS_WEBEDIT / DC_MAXCELLS_WEBEDIT limit, else 0
|
||||
@li TRUNC_MSG - the reason shown against the table name when TRUNCATED=1,
|
||||
else empty
|
||||
@li DERIVEDVARS - space separated list of the derived (computed) columns
|
||||
configured in MPE_DERIVED, upper case, empty when there are none. They
|
||||
are appended after the physical columns and are read-only; the EDIT grid
|
||||
marks them so a user can see why a column cannot be edited.
|
||||
|
||||
<h5> approvers </h5>
|
||||
<h5> dqrules </h5>
|
||||
@@ -185,23 +181,7 @@ run;
|
||||
|
||||
|
||||
%global loadtype var_txfrom var_txto var_processed filter_text pk coltype
|
||||
sortpk var_busfrom var_busto rk_underlying buskey pre_edit_hook;
|
||||
/* clear first: a global survives the call, so in a reused session it would
|
||||
still hold the previous request's values, and any path that skips an
|
||||
assignment would apply them to this one */
|
||||
%let loadtype=;
|
||||
%let var_txfrom=;
|
||||
%let var_txto=;
|
||||
%let var_processed=;
|
||||
%let filter_text=;
|
||||
%let pk=;
|
||||
%let coltype=;
|
||||
%let sortpk=;
|
||||
%let var_busfrom=;
|
||||
%let var_busto=;
|
||||
%let rk_underlying=;
|
||||
%let buskey=;
|
||||
%let pre_edit_hook=;
|
||||
sortpk;
|
||||
|
||||
%put getting table attributes;
|
||||
proc sql noprint;
|
||||
@@ -360,10 +340,8 @@ select upcase(loadtype)
|
||||
);
|
||||
%put sourcevars=&sourcevars;
|
||||
/* the payload is capped at &maxrows - whichever limit that works out to be
|
||||
- this is what goes to the browser. The cap is a data set option on the
|
||||
SET, not `options obs=`, so no later step in this session can be cut by it:
|
||||
a session option left set would truncate the metadata steps below, and a
|
||||
table wide enough to hit the cell limit would lose columns rather than rows. */
|
||||
- this is what goes to the browser */
|
||||
options obs=&maxrows;
|
||||
data outdata;
|
||||
/* pk fields come first */
|
||||
%if %length(&pk)>0 %then %do;
|
||||
@@ -377,8 +355,9 @@ select upcase(loadtype)
|
||||
/* send empty row if empty table to help with hot rendering */
|
||||
output;
|
||||
%end;
|
||||
set work.out(obs=&maxrows) ;
|
||||
set work.out ;
|
||||
run;
|
||||
options obs=max;
|
||||
|
||||
|
||||
|
||||
@@ -552,7 +531,7 @@ select upcase(loadtype)
|
||||
%mpestp_getdata()
|
||||
|
||||
/* mp_abort cannot exit cleanly from inside a %include, so an abort there
|
||||
(eg in a hook script run via mp_include) only records the failure in
|
||||
(eg in a hook script run via mp_include) only records the error in
|
||||
work.mp_abort_errds. mode=INCLUDE picks that up and aborts properly here. */
|
||||
%mp_abort(mode=INCLUDE)
|
||||
|
||||
@@ -649,19 +628,8 @@ proc sql noprint;
|
||||
select count(*) into: ismap from &mpelib..mpe_xlmap_info
|
||||
where XLMAP_TARGETLIBDS="&orig_libds" and &dc_dttmtfmt. le TX_TO;
|
||||
|
||||
/* the derived columns, named for the EDIT grid so it can mark them as special.
|
||||
They are read-only through the READONLY DQ rule below, but that rule does not
|
||||
say WHY the column cannot be edited - a computed column is a different thing
|
||||
from an ordinary read-only one. work.derived_vars exists on every path, empty
|
||||
when the table has no derived variables. */
|
||||
%let derivedvars=;
|
||||
proc sql noprint;
|
||||
select upcase(name) into: derivedvars separated by ' '
|
||||
from work.derived_vars;
|
||||
quit;
|
||||
|
||||
data sasparams;
|
||||
length colHeaders $20000 filter_text $32767 trunc_msg $200 derivedvars $32767;
|
||||
length colHeaders $20000 filter_text $32767 trunc_msg $200;
|
||||
colHeaders=cats(upcase("%mf_getvarlist(sasdata1,dlm=%str(,))"));
|
||||
pkCnt=countw("&pk");
|
||||
pk="&pk";
|
||||
@@ -672,7 +640,6 @@ data sasparams;
|
||||
if trim(symget('rk_underlying')) ne '' then rk_flag=1;
|
||||
else rk_flag=0;
|
||||
filter_text=symget('filter_text');
|
||||
derivedvars=symget('derivedvars');
|
||||
/* set when the selection was cut down to the row / cell limit, so the EDIT
|
||||
screen can say so against the table name */
|
||||
truncated=&truncated;
|
||||
@@ -694,16 +661,7 @@ proc sort data=&mpelib..mpe_validations
|
||||
by base_col rule_type rule_value;
|
||||
run;
|
||||
|
||||
/* merge with NOTNULL constraints in the physical table, and mark the derived
|
||||
variables read-only: a derived column is computed at read time, so the grid
|
||||
must not accept a value for it. COLS carries the same READ (a derived column
|
||||
has no CLS rule row) but the client only applies COLS.CLS_RULE when CLS_FLAG
|
||||
is set, which is driven by the table's column-level security - so the rule is
|
||||
declared here, where it holds regardless.
|
||||
|
||||
work.derived_vars exists on every path that reaches this step: %mpe_derived
|
||||
writes it before returning, empty when the table has no derived variables,
|
||||
so the union below needs no guard */
|
||||
/* merge with NOTNULL constraints in the physical table */
|
||||
proc sql;
|
||||
create table _data_ as
|
||||
select * from dqrules
|
||||
@@ -716,11 +674,6 @@ union
|
||||
and upcase(memname)="%scan(&orig_libds,2,.)"
|
||||
and upcase(name) in (select name from vars3)
|
||||
and notnull='yes'
|
||||
union
|
||||
select upcase(name) as base_col
|
||||
,'READONLY' as rule_type
|
||||
,'' as rule_value
|
||||
from work.derived_vars
|
||||
order by 1,2,3;
|
||||
data dqrules;
|
||||
set &syslast;
|
||||
|
||||
@@ -14,8 +14,7 @@
|
||||
rows x 1004 columns, created and registered by testsetup) filtered to 200
|
||||
rows: that is under the row limit but 200 x 1004 = 200,800 cells, over the
|
||||
200,000 cell limit, so the cell limit decides the cap -
|
||||
floor(200000 / 1004) = 199 rows. It also checks the column list is whole:
|
||||
the cap applies to the rows the browser is sent, never to the COLS metadata.
|
||||
floor(200000 / 1004) = 199 rows.
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mp_assert.sas
|
||||
@@ -99,7 +98,7 @@ run;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=(&t1_trunc=1),
|
||||
desc=%str(Test 1 - a selection over the row limit is served, flagged as truncated),
|
||||
desc=Test 1 - a selection over the row limit is served, flagged as truncated,
|
||||
outds=work.test_results
|
||||
)
|
||||
%mp_assert(
|
||||
@@ -134,7 +133,7 @@ run;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=(&t2_trunc=1),
|
||||
desc=%str(Test 2 - a selection over the cell limit is served, flagged as truncated),
|
||||
desc=Test 2 - a selection over the cell limit is served, flagged as truncated,
|
||||
outds=work.test_results
|
||||
)
|
||||
%mp_assert(
|
||||
@@ -148,27 +147,6 @@ run;
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
/**
|
||||
* The row cap must not reach the metadata: the COLS payload for this table
|
||||
* describes every column, not the 199 rows the cell limit admits. A row cap
|
||||
* left set as a session option would cut the column list to 199 entries, and
|
||||
* a grid losing 800 columns looks like a missing table rather than a limit.
|
||||
*/
|
||||
%let t2_cols=-1;
|
||||
%let t2_phys=-1;
|
||||
proc sql noprint;
|
||||
select count(*) into: t2_cols from web2.cols;
|
||||
select count(*) into: t2_phys
|
||||
from dictionary.columns
|
||||
where libname='DCTEST' and memname='WIDEBOY';
|
||||
quit;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=(&t2_cols=&t2_phys),
|
||||
desc=%str(Test 2 - every column is described, not just the rows the cap admits),
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
/**
|
||||
* dump results to the log for offline inspection
|
||||
*/
|
||||
|
||||
@@ -65,8 +65,8 @@ select count(*) into: admin_check
|
||||
where groupname="&dc_admin_group";
|
||||
quit;
|
||||
%if &admin_check>0 %then %do;
|
||||
%put %str(WARN)ING: %mf_getuser() is in the &dc_admin_group group;
|
||||
%put %str(WARN)ING: Row Level Security is skipped for admins - test skipped;
|
||||
%put WARNING: %mf_getuser() is in the &dc_admin_group group;
|
||||
%put WARNING: Row Level Security is skipped for admins - test skipped;
|
||||
data work.test_results;
|
||||
length test_result $4 test_description $256 test_comments $256;
|
||||
test_result='PASS';
|
||||
|
||||
@@ -3,14 +3,15 @@
|
||||
@brief testing gethistory service
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mpe_execute.sas
|
||||
@li mp_assert.sas
|
||||
@li mp_assertcols.sas
|
||||
@li mp_assertcolvals.sas
|
||||
@li mp_assertdsobs.sas
|
||||
@li mf_getuniquefileref.sas
|
||||
@li mf_getuniquename.sas
|
||||
@li mp_filterstore.sas
|
||||
@li mpe_derived.sas
|
||||
@li mp_assert.sas
|
||||
@li mp_assertcols.sas
|
||||
@li mx_execute.sas
|
||||
|
||||
**/
|
||||
|
||||
@@ -56,7 +57,7 @@ data _null_;
|
||||
put 'LIBDS:$43. FILTER_RK:8.';
|
||||
put "&dclib..MPE_TABLES,&filter_rk1";
|
||||
run;
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputfiles=&f1:sascontroltable,
|
||||
outlib=web1,
|
||||
@@ -111,8 +112,8 @@ data work.inquery;
|
||||
input GROUP_LOGIC:$3. SUBGROUP_LOGIC:$3. SUBGROUP_ID:8. VARIABLE_NM:$32.
|
||||
OPERATOR_NM:$10. RAW_VALUE:$4000.;
|
||||
datalines4;
|
||||
AND,AND,1,LIBREF,=,"'&dc_libref'"
|
||||
AND,AND,2,DSN,=,"'MPE_LOCK_ANYTABLE'"
|
||||
AND,AND,1,LIBREF,CONTAINS,"'DC'"
|
||||
AND,OR,2,DSN,=,"'MPE_LOCK_ANYTABLE'"
|
||||
;;;;
|
||||
run;
|
||||
%mp_filterstore(
|
||||
@@ -137,7 +138,7 @@ data _null_;
|
||||
put "&dclib..MPE_TABLES,&filter_rk";
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputfiles=&f2:sascontroltable,
|
||||
outlib=web2
|
||||
@@ -217,7 +218,7 @@ data _null_;
|
||||
put "DCTEST.DCFMTS-FC,&filter_rk3";
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputfiles=&f3:sascontroltable,
|
||||
outlib=web3,
|
||||
@@ -287,16 +288,12 @@ quit;
|
||||
)
|
||||
|
||||
data work.inquery4;
|
||||
length GROUP_LOGIC $3 SUBGROUP_LOGIC $3 SUBGROUP_ID 8 VARIABLE_NM $32
|
||||
OPERATOR_NM $10 RAW_VALUE $4000;
|
||||
GROUP_LOGIC='AND'; SUBGROUP_LOGIC='AND'; SUBGROUP_ID=1;
|
||||
VARIABLE_NM='LIBREF'; OPERATOR_NM='=';
|
||||
RAW_VALUE="'"!!trim(symget('dc_libref'))!!"'";
|
||||
output;
|
||||
GROUP_LOGIC='AND'; SUBGROUP_LOGIC='AND'; SUBGROUP_ID=2;
|
||||
VARIABLE_NM='DERIVED_NOTE'; OPERATOR_NM='CONTAINS';
|
||||
RAW_VALUE="'MPE_LOCK'";
|
||||
output;
|
||||
infile datalines4 dsd;
|
||||
input GROUP_LOGIC:$3. SUBGROUP_LOGIC:$3. SUBGROUP_ID:8. VARIABLE_NM:$32.
|
||||
OPERATOR_NM:$10. RAW_VALUE:$4000.;
|
||||
datalines4;
|
||||
AND,AND,1,DERIVED_NOTE,CONTAINS,"'MPE_LOCK'"
|
||||
;;;;
|
||||
run;
|
||||
%mp_filterstore(
|
||||
libds=&dc_libref..MPE_TABLES,
|
||||
@@ -321,38 +318,29 @@ data _null_;
|
||||
put "&dclib..MPE_TABLES,&filter_rk4";
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputfiles=&f4:sascontroltable,
|
||||
outlib=web4,
|
||||
outstatus=work.ab4,
|
||||
failfast=0,
|
||||
mdebug=&sasjs_mdebug
|
||||
)
|
||||
|
||||
%let state4=NONE;
|
||||
/* a cancelled child returns no webout, so the outlib keeps whatever an
|
||||
earlier call left in it - check the job state before reading its members,
|
||||
otherwise the assertion failures below describe the wrong problem */
|
||||
%let abort4=0;
|
||||
data _null_;
|
||||
if not exist('work.ab4') then stop;
|
||||
set work.ab4;
|
||||
call symputx('state4',state);
|
||||
set work.results;
|
||||
if state='canceled' then call symputx('abort4',1);
|
||||
run;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=("&state4"="completed"),
|
||||
iftrue=(&abort4=0),
|
||||
desc=Test4 - the derived variable call completes,
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
/* a cancelled child returns no webout, so the outlib keeps whatever an earlier
|
||||
* call left in it - read its members only when the child completed */
|
||||
%let completed4=0;
|
||||
data _null_;
|
||||
if not exist('work.ab4') then stop;
|
||||
set work.ab4;
|
||||
if state='completed' then call symputx('completed4',1);
|
||||
run;
|
||||
|
||||
%if &completed4=1 %then %do;
|
||||
%if &abort4=0 %then %do;
|
||||
data sasdata4;
|
||||
set web4.sasdata;
|
||||
run;
|
||||
@@ -367,30 +355,10 @@ run;
|
||||
run;
|
||||
%end;
|
||||
|
||||
%if &completed4=1 and "&sasjs_mdebug"="1" %then %do;
|
||||
/* the payload the assertions read, so a missing column or an unapplied filter
|
||||
can be seen without guessing */
|
||||
data _null_;
|
||||
set work.cols4;
|
||||
if _n_=1 then putlog "NOTE: &sysmacroname: Test4 cols payload:";
|
||||
putlog (_all_)(=);
|
||||
run;
|
||||
data _null_;
|
||||
set sasdata4;
|
||||
if _n_=1 then putlog "NOTE: &sysmacroname: Test4 data payload:";
|
||||
putlog (_all_)(=);
|
||||
run;
|
||||
%end;
|
||||
|
||||
proc sql noprint;
|
||||
select count(*) into: derived4
|
||||
from work.cols4
|
||||
where upcase(name)='DERIVED_NOTE';
|
||||
quit;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=(&derived4>0),
|
||||
%mp_assertcols(work.cols4,
|
||||
desc=Test4 - derived column is in the COLS payload,
|
||||
cols=DERIVED_NOTE,
|
||||
test=ALL,
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
@@ -404,19 +372,6 @@ run;
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
/* COLS.CLS_RULE only reaches the grid when CLS_FLAG is set, which a table
|
||||
with no column level security does not do - the read-only promise is
|
||||
carried by this rule instead */
|
||||
data work.chk4e;
|
||||
set web4.dqrules;
|
||||
where upcase(base_col)='DERIVED_NOTE' and rule_type='READONLY';
|
||||
run;
|
||||
%mp_assertdsobs(work.chk4e,
|
||||
desc=Test4 - derived column carries a READONLY rule,
|
||||
test=EQUALS 1,
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
data work.chk4b;
|
||||
set work.cols4;
|
||||
where name='DERIVED_NOTE' and label=:'DERIVED_NOTE=';
|
||||
@@ -435,7 +390,7 @@ run;
|
||||
|
||||
data work.chk4c;
|
||||
set work.sasdata4;
|
||||
if derived_note=cats("%scan(&dclib..MPE_TABLES,1,.)",'.','MPE_LOCKANYTABLE');
|
||||
if derived_note=cats("%scan(&dclib..MPE_TABLES,1,.)",'.','MPE_LOCK_ANYTABLE');
|
||||
run;
|
||||
%mp_assertdsobs(work.chk4c,
|
||||
desc=Test4 - derived value is correct,
|
||||
|
||||
@@ -294,7 +294,7 @@ run;
|
||||
,dc_dttmtfmt=&dc_dttmtfmt
|
||||
)
|
||||
/* mp_abort cannot exit cleanly from inside a %include, so an abort there
|
||||
(eg in a hook script run via mp_include) only records the failure in
|
||||
(eg in a hook script run via mp_include) only records the error in
|
||||
work.mp_abort_errds. mode=INCLUDE picks that up and aborts properly here. */
|
||||
%mp_abort(mode=INCLUDE)
|
||||
|
||||
|
||||
@@ -5,14 +5,13 @@
|
||||
1. loading row level EDIT security against a table with LOADTYPE=REPLACE
|
||||
2. applying LOADTYPE=REPLACE to a table with row level EDIT security
|
||||
|
||||
The staged loads are aborted by the post edit hooks. The assertion fetches
|
||||
the sasjsAbort object that mp_abort writes to the webout before the hook
|
||||
aborts.
|
||||
The staged loads are aborted by the post edit hooks inside the child
|
||||
service. The abort is recorded as a FAILED load in MPE_LOADS (via
|
||||
mpe_loadfail), so the assertions query MPE_LOADS rather than the child
|
||||
response - a canceled child does not always return a webout or log.
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mx_execute.sas
|
||||
@li mpe_execute.sas
|
||||
@li mpe_assert_abort.sas
|
||||
@li mp_assert.sas
|
||||
|
||||
|
||||
@@ -72,6 +71,7 @@ quit;
|
||||
/**
|
||||
* Test 1 - submitting an RLS EDIT rule against a REPLACE table should abort
|
||||
*/
|
||||
%let t1=%sysfunc(datetime());
|
||||
data work.sascontroltable;
|
||||
action='LOAD';
|
||||
message='RLS EDIT rule on REPLACE table should abort';
|
||||
@@ -100,19 +100,36 @@ data work.jsdata;
|
||||
output;
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputdatasets=work.sascontroltable work.jsdata,
|
||||
outlib=web1,
|
||||
outref=wb1,
|
||||
outstatus=work.ab1,
|
||||
failfast=0,
|
||||
mdebug=&sasjs_mdebug
|
||||
)
|
||||
|
||||
%mpe_assert_abort(
|
||||
/* a hook-aborted load is terminated before mpe_loadfail can record a
|
||||
* FAILED status, so the load row stays IN PROGRESS - the abort is
|
||||
* proved by the load NOT completing successfully. Tests in this suite run
|
||||
* in parallel sessions, so match the row for THIS test by its unique
|
||||
* reason_txt (the submitted message) rather than any load in the time
|
||||
* window - an unrelated aborted/IN PROGRESS load would otherwise satisfy
|
||||
* the assertion even if this load completed. The row id (csv_dir/mperef)
|
||||
* is generated inside the aborted child service, which returns no webout,
|
||||
* so reason_txt is the only stable identifier the test controls. */
|
||||
%let abort1=0;
|
||||
proc sql noprint;
|
||||
select count(*) into: abort1
|
||||
from &dc_libref..mpe_loads
|
||||
where processed_dttm>&t1
|
||||
and reason_txt="RLS EDIT rule on REPLACE table should abort"
|
||||
and upcase(status) ne 'SUCCESS';
|
||||
quit;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=(&abort1>0),
|
||||
desc=Checking RLS EDIT rule is rejected for REPLACE loadtype table (#211),
|
||||
outref=wb1
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
/**
|
||||
@@ -120,6 +137,7 @@ run;
|
||||
* time: the rule is evaluated against the submitted data, where a derived
|
||||
* column does not exist
|
||||
*/
|
||||
%let t3=%sysfunc(datetime());
|
||||
|
||||
/* define a derived variable on the target table */
|
||||
proc sql noprint;
|
||||
@@ -167,19 +185,27 @@ data work.jsdata;
|
||||
output;
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputdatasets=work.sascontroltable work.jsdata,
|
||||
outlib=web3,
|
||||
outref=wb3,
|
||||
outstatus=work.ab3,
|
||||
failfast=0,
|
||||
mdebug=&sasjs_mdebug
|
||||
)
|
||||
|
||||
%mpe_assert_abort(
|
||||
%let abort3=0;
|
||||
proc sql noprint;
|
||||
select count(*) into: abort3
|
||||
from &dc_libref..mpe_loads
|
||||
where processed_dttm>&t3
|
||||
and reason_txt="RLS rule on a derived variable should abort"
|
||||
and upcase(status) ne 'SUCCESS';
|
||||
quit;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=(&abort3>0),
|
||||
desc=Checking an RLS rule on a derived variable is rejected,
|
||||
outref=wb3
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
/* tidy up the derived configuration */
|
||||
@@ -193,6 +219,7 @@ quit;
|
||||
* should abort. The active EDIT rule was inserted in the prep section,
|
||||
* long before this child service reads it.
|
||||
*/
|
||||
%let t2=%sysfunc(datetime());
|
||||
|
||||
/* now stage a REPLACE loadtype for DCTEST.WIDEBOY */
|
||||
data work.sascontroltable;
|
||||
@@ -210,19 +237,27 @@ data work.jsdata;
|
||||
_____DELETE__THIS__RECORD_____='No';
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputdatasets=work.sascontroltable work.jsdata,
|
||||
outlib=web2,
|
||||
outref=wb2,
|
||||
outstatus=work.ab2,
|
||||
failfast=0,
|
||||
mdebug=&sasjs_mdebug
|
||||
)
|
||||
|
||||
%mpe_assert_abort(
|
||||
%let abort2=0;
|
||||
proc sql noprint;
|
||||
select count(*) into: abort2
|
||||
from &dc_libref..mpe_loads
|
||||
where processed_dttm>&t2
|
||||
and reason_txt="REPLACE loadtype on RLS-secured table should abort"
|
||||
and upcase(status) ne 'SUCCESS';
|
||||
quit;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=(&abort2>0),
|
||||
desc=Checking REPLACE loadtype is rejected for RLS EDIT-secured table (#211),
|
||||
outref=wb2
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
/**
|
||||
|
||||
@@ -6,27 +6,25 @@
|
||||
table - the engine reads only active rows, so the dependency can never
|
||||
resolve at run time
|
||||
2. a row is staged with active=0 while a surviving active row depends on it
|
||||
3. a staged row breaks a field rule of the shared scan (core's
|
||||
mp_derivedvarcheck), or carries an ACTIVE value the EDIT screen's picker
|
||||
cannot produce
|
||||
|
||||
They should NOT abort when:
|
||||
4. a new active row is added to a table that also holds an inactive row -
|
||||
3. a new active row is added to a table that also holds an inactive row -
|
||||
the inactive row is not part of the dependency graph, so it cannot
|
||||
create a cycle or a stale reference
|
||||
5. a delete-marked RLS rule naming a derived variable is submitted - the
|
||||
4. a delete-marked RLS rule naming a derived variable is submitted - the
|
||||
rule is being removed, so its variables are not validated
|
||||
6. a delete and re-add of the same variable is submitted together - the
|
||||
5. a delete and re-add of the same variable is submitted together - the
|
||||
re-added active row resolves the dependency
|
||||
7. an inactive row is set active again while another active row depends on
|
||||
6. an inactive row is set active again while another active row depends on
|
||||
it - the re-activation is what makes the dependency resolvable
|
||||
|
||||
The staged loads are aborted by the post edit hook. The assertion fetches the
|
||||
sasjsAbort object that mp_abort writes to the webout before the hook aborts.
|
||||
The staged loads are aborted by the post edit hook inside the child service.
|
||||
An aborted load leaves its MPE_LOADS row IN PROGRESS, so each assertion
|
||||
matches on the unique reason_txt (the submitted message) rather than any load
|
||||
in the time window - the suite runs in parallel sessions.
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mpe_execute.sas
|
||||
@li mpe_assert_abort.sas
|
||||
@li mx_execute.sas
|
||||
@li mp_assert.sas
|
||||
|
||||
|
||||
@@ -91,19 +89,27 @@ data work.jsdata;
|
||||
output;
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputdatasets=work.sascontroltable work.jsdata,
|
||||
outlib=web1,
|
||||
outref=wb1,
|
||||
outstatus=work.ab1,
|
||||
failfast=0,
|
||||
mdebug=&sasjs_mdebug
|
||||
)
|
||||
|
||||
%mpe_assert_abort(
|
||||
%let abort1=0;
|
||||
proc sql noprint;
|
||||
select count(*) into: abort1
|
||||
from &dc_libref..mpe_loads
|
||||
where processed_dttm>&t1
|
||||
and reason_txt='Derived var depending on an inactive row should abort'
|
||||
and upcase(status) ne 'SUCCESS';
|
||||
quit;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=(&abort1>0),
|
||||
desc=Checking a dependency on an inactive derived row is rejected,
|
||||
outref=wb1
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
/**
|
||||
@@ -111,6 +117,7 @@ run;
|
||||
* rejected: the control table still shows the row as active, so the check has
|
||||
* to read the staged row too
|
||||
*/
|
||||
%let t2=%sysfunc(datetime());
|
||||
|
||||
proc sql noprint;
|
||||
delete from &dc_libref..mpe_derived
|
||||
@@ -157,19 +164,27 @@ data work.jsdata;
|
||||
_____DELETE__THIS__RECORD_____='No';
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputdatasets=work.sascontroltable work.jsdata,
|
||||
outlib=web2,
|
||||
outref=wb2,
|
||||
outstatus=work.ab2,
|
||||
failfast=0,
|
||||
mdebug=&sasjs_mdebug
|
||||
)
|
||||
|
||||
%mpe_assert_abort(
|
||||
%let abort2=0;
|
||||
proc sql noprint;
|
||||
select count(*) into: abort2
|
||||
from &dc_libref..mpe_loads
|
||||
where processed_dttm>&t2
|
||||
and reason_txt='Deactivating a relied-on derived row should abort'
|
||||
and upcase(status) ne 'SUCCESS';
|
||||
quit;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=(&abort2>0),
|
||||
desc=Checking deactivating a relied-on derived row is rejected,
|
||||
outref=wb2
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
/**
|
||||
@@ -220,7 +235,7 @@ data work.jsdata;
|
||||
output;
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputdatasets=work.sascontroltable work.jsdata,
|
||||
outlib=web3,
|
||||
@@ -298,7 +313,7 @@ data work.jsdata;
|
||||
_____DELETE__THIS__RECORD_____='Yes';
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputdatasets=work.sascontroltable work.jsdata,
|
||||
outlib=web4,
|
||||
@@ -370,19 +385,27 @@ data work.jsdata;
|
||||
_____DELETE__THIS__RECORD_____='Yes';
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputdatasets=work.sascontroltable work.jsdata,
|
||||
outlib=web5,
|
||||
outref=wb5,
|
||||
outstatus=work.ab5,
|
||||
failfast=0,
|
||||
mdebug=&sasjs_mdebug
|
||||
)
|
||||
|
||||
%mpe_assert_abort(
|
||||
%let abort5=0;
|
||||
proc sql noprint;
|
||||
select count(*) into: abort5
|
||||
from &dc_libref..mpe_loads
|
||||
where processed_dttm>&t5
|
||||
and reason_txt='Deleting a relied-on derived row should abort'
|
||||
and upcase(status) ne 'SUCCESS';
|
||||
quit;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=(&abort5>0),
|
||||
desc=Checking deleting a relied-on derived row is rejected,
|
||||
outref=wb5
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
/**
|
||||
@@ -452,7 +475,7 @@ data work.jsdata;
|
||||
output;
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputdatasets=work.sascontroltable work.jsdata,
|
||||
outlib=web6,
|
||||
@@ -534,7 +557,7 @@ data work.jsdata;
|
||||
_____DELETE__THIS__RECORD_____='No';
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputdatasets=work.sascontroltable work.jsdata,
|
||||
outlib=web7,
|
||||
@@ -555,104 +578,6 @@ run;
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
|
||||
/**
|
||||
* Test 8 - a field rule of the shared scan (mp_derivedvarcheck, core) is
|
||||
* applied at submit time: an expression that could submit code of its own is
|
||||
* refused by name, with the reason the scan gives. This is the same scan the
|
||||
* engine runs at read time, so the case pins that the hook calls it rather
|
||||
* than carrying a rule list of its own.
|
||||
*/
|
||||
data work.sascontroltable;
|
||||
action='LOAD';
|
||||
message='A refused function in VAR_LOGIC should abort';
|
||||
libds="&dc_libref..MPE_DERIVED";
|
||||
output;
|
||||
stop;
|
||||
run;
|
||||
|
||||
data work.jsdata;
|
||||
length LIBREF $8 DSN $32 VAR_NM $32 VAR_DEPS $4000 VAR_TYPE $1
|
||||
VAR_LOGIC $4000 VAR_FMT $32
|
||||
_____DELETE__THIS__RECORD_____ $3;
|
||||
LIBREF='DCTEST';
|
||||
DSN='DC_DERIV';
|
||||
VAR_NM='CALLER';
|
||||
VAR_DEPS='';
|
||||
VAR_TYPE='N';
|
||||
VAR_LEN=8;
|
||||
VAR_FMT='';
|
||||
/* no semicolon in the payload: the single-expression rule is tested before
|
||||
the function-name rule, so a quoted semicolon would abort on that first */
|
||||
VAR_LOGIC='dosubl("data _null_")';
|
||||
ACTIVE=1;
|
||||
_____DELETE__THIS__RECORD_____='No';
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputdatasets=work.sascontroltable work.jsdata,
|
||||
outlib=web8,
|
||||
outref=wb8,
|
||||
mdebug=&sasjs_mdebug
|
||||
)
|
||||
|
||||
/* the assertion compares the abort's MSG exactly, so the expected value is
|
||||
assembled on one line: a literal wrapped across two source lines carries the
|
||||
newline and the continuation indent with it */
|
||||
%let exp8=MPE_DERIVED: CALLER: VAR_LOGIC may not call a function that;
|
||||
%let exp8=&exp8 submits code or reaches the file system,;
|
||||
%let exp8=&exp8 the environment or the macro symbol table;
|
||||
|
||||
%mpe_assert_abort(
|
||||
desc=Checking a refused function in VAR_LOGIC is rejected,
|
||||
outref=wb8,
|
||||
msg=&exp8
|
||||
)
|
||||
|
||||
/**
|
||||
* Test 9 - the fields the hook owns rather than the scan: ACTIVE is a picker
|
||||
* in the EDIT screen (HARDSELECT over the MPE_SELECTBOX pair), so a submission
|
||||
* that bypasses the grid cannot land a value the screen would never offer.
|
||||
*/
|
||||
data work.sascontroltable;
|
||||
action='LOAD';
|
||||
message='An ACTIVE value the screen cannot offer should abort';
|
||||
libds="&dc_libref..MPE_DERIVED";
|
||||
output;
|
||||
stop;
|
||||
run;
|
||||
|
||||
data work.jsdata;
|
||||
length LIBREF $8 DSN $32 VAR_NM $32 VAR_DEPS $4000 VAR_TYPE $1
|
||||
VAR_LOGIC $4000 VAR_FMT $32
|
||||
_____DELETE__THIS__RECORD_____ $3;
|
||||
LIBREF='DCTEST';
|
||||
DSN='DC_DERIV';
|
||||
VAR_NM='BADACT';
|
||||
VAR_DEPS='';
|
||||
VAR_TYPE='N';
|
||||
VAR_LEN=8;
|
||||
VAR_FMT='';
|
||||
VAR_LOGIC='1';
|
||||
ACTIVE=2;
|
||||
_____DELETE__THIS__RECORD_____='No';
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputdatasets=work.sascontroltable work.jsdata,
|
||||
outlib=web9,
|
||||
outref=wb9,
|
||||
mdebug=&sasjs_mdebug
|
||||
)
|
||||
|
||||
%mpe_assert_abort(
|
||||
desc=Checking an ACTIVE value outside the picker pair is rejected,
|
||||
outref=wb9,
|
||||
msg=MPE_DERIVED: BADACT: ACTIVE must be 1 or 0
|
||||
)
|
||||
|
||||
/* tidy up so repeated runs start clean */
|
||||
proc sql noprint;
|
||||
delete from &dc_libref..mpe_derived
|
||||
|
||||
@@ -34,7 +34,6 @@
|
||||
<h4> SAS Macros </h4>
|
||||
@li mf_nobs.sas
|
||||
@li mp_abort.sas
|
||||
@li mp_derivedvarcheck.sas
|
||||
|
||||
<h4> Related Macros </h4>
|
||||
@li mpe_derived.sas
|
||||
@@ -100,43 +99,51 @@ run;
|
||||
)
|
||||
|
||||
/**
|
||||
* Field level checks. The rules for the derived variable itself - the name,
|
||||
* the type, length, format, label and the expression - are core's, in
|
||||
* mp_derivedvarcheck, so this hook and the read time engine (%mpe_derived ->
|
||||
* mp_derivedvars) scan a row with the same rules and report the same reasons.
|
||||
* What is added here are the two fields that belong to Data Controller rather
|
||||
* than to the expression: the control table key, which the loader normalises
|
||||
* but does not require, and ACTIVE, which the EDIT screen offers as a picker
|
||||
* over the MPE_SELECTBOX pair - so a submission that bypasses the grid cannot
|
||||
* land a value the screen would never offer.
|
||||
* Field level checks - the same rules %mpe_derived applies at read time
|
||||
*/
|
||||
data work.derived_dcfield(keep=name reason);
|
||||
set work.staging_ds;
|
||||
if upcase(_____DELETE__THIS__RECORD_____)='YES' then delete;
|
||||
length name $32 reason $200;
|
||||
call missing(reason);
|
||||
name=upcase(strip(var_nm));
|
||||
if missing(libref) or missing(dsn) then reason='LIBREF and DSN are required';
|
||||
else if missing(active) or active not in (0,1)
|
||||
then reason='ACTIVE must be 1 or 0';
|
||||
if not missing(reason) then output;
|
||||
run;
|
||||
|
||||
%mp_derivedvarcheck(inds=work.staging_ds
|
||||
,outds=work.derived_field
|
||||
,reserved=%str(_____DELETE__THIS__RECORD_____)
|
||||
)
|
||||
|
||||
/* the control table fields are reported before the expression's own rules, so
|
||||
which of the two passes found a row does not change the message it gets */
|
||||
data work.derived_check;
|
||||
set work.derived_dcfield work.derived_field;
|
||||
set work.staging_ds;
|
||||
length reason $200;
|
||||
if upcase(_____DELETE__THIS__RECORD_____)='YES' then delete;
|
||||
call missing(reason);
|
||||
|
||||
if missing(libref) or missing(dsn) then reason='LIBREF and DSN are required';
|
||||
else if var_nm='' then reason='VAR_NM is required';
|
||||
else if verify(strip(var_nm),'ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_')>0
|
||||
then reason='VAR_NM must be a valid SAS name';
|
||||
else if verify(substr(var_nm,1,1),'0123456789')=0
|
||||
then reason='VAR_NM must not start with a digit';
|
||||
else if var_nm='_____DELETE__THIS__RECORD_____'
|
||||
then reason='VAR_NM is reserved';
|
||||
else if var_type not in ('C','N')
|
||||
then reason='VAR_TYPE must be C or N';
|
||||
else if missing(var_len) or var_len ne int(var_len) or var_len<1
|
||||
then reason='VAR_LEN must be a positive integer';
|
||||
else if var_type='C' and var_len>32767
|
||||
then reason='VAR_LEN exceeds 32767 for a character variable';
|
||||
else if var_type='N' and var_len>8
|
||||
then reason='VAR_LEN exceeds 8 for a numeric variable';
|
||||
else if index(var_fmt,';')>0 or indexc(var_fmt,'&%')>0
|
||||
or index(var_fmt,'/*')>0 or index(var_fmt,'*/')>0
|
||||
then reason='VAR_FMT must be a format name, without ; & % or //*';
|
||||
else if verify(strip(var_fmt),'ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_$.')>0
|
||||
then reason='VAR_FMT must be a format name';
|
||||
else if missing(var_logic)
|
||||
then reason='VAR_LOGIC is required';
|
||||
else if index(var_logic,';')>0
|
||||
then reason='VAR_LOGIC must be a single expression - remove the ;';
|
||||
else if indexc(var_logic,'&%')>0
|
||||
then reason='VAR_LOGIC must not contain macro triggers';
|
||||
else if index(var_logic,'/*')>0 or index(var_logic,'*/')>0
|
||||
then reason='VAR_LOGIC must not contain comments';
|
||||
|
||||
if not missing(reason) then output;
|
||||
run;
|
||||
|
||||
%if %mf_nobs(work.derived_check)>0 %then %do;
|
||||
data _null_;
|
||||
set work.derived_check(obs=1);
|
||||
call symputx('der_errmsg',cats('MPE_DERIVED: ',name,': ',reason));
|
||||
call symputx('der_errmsg',cats('MPE_DERIVED: ',var_nm,': ',reason));
|
||||
call symputx('der_errflag',1);
|
||||
run;
|
||||
%end;
|
||||
@@ -436,7 +443,7 @@ run;
|
||||
|
||||
%mp_abort(iftrue= (&cyc_found=1)
|
||||
,mac=mpe_derived_postedit
|
||||
,msg=%str(MPE_DERIVED: VAR_DEPS contains a cycle: &cyc_path)
|
||||
,msg=%str(VAR_DEPS contains a cycle: &cyc_path)
|
||||
)
|
||||
|
||||
%mend _mpe_derived_postedit;
|
||||
|
||||
@@ -12,9 +12,8 @@
|
||||
assertion cannot pass against a service that does not validate.
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mpe_assert_abort.sas
|
||||
@li mp_assert.sas
|
||||
@li mpe_execute.sas
|
||||
@li mx_execute.sas
|
||||
@li mf_getuniquefileref.sas
|
||||
|
||||
**/
|
||||
@@ -31,19 +30,23 @@ data _null_;
|
||||
put '%sysfunc(coalescec(&dclib..MPE_X_TEST,))';
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputfiles=&f1:iwant,
|
||||
outref=web1,
|
||||
outstatus=work.ab1,
|
||||
failfast=0,
|
||||
viyaresult=WEBOUT_TXT
|
||||
)
|
||||
|
||||
%mpe_assert_abort(
|
||||
%let abort1=0;
|
||||
data _null_;
|
||||
set work.results;
|
||||
if state='canceled' then call symputx('abort1',1);
|
||||
run;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=(&abort1=1),
|
||||
desc=Macro content in libds aborts the service,
|
||||
outref=web1,
|
||||
msg=Invalid libds
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
/**
|
||||
@@ -56,7 +59,7 @@ data _null_;
|
||||
put "&dclib..MPE_X_TEST";
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputfiles=&f2:iwant,
|
||||
outlib=web2
|
||||
|
||||
@@ -36,8 +36,7 @@
|
||||
<h4> SAS Macros </h4>
|
||||
@li mf_existds.sas
|
||||
@li mf_nobs.sas
|
||||
@li mpeinit.sas
|
||||
@li mpeterm.sas
|
||||
@li mf_verifymacvars.sas
|
||||
@li dc_assignlib.sas
|
||||
@li mf_getvarformat.sas
|
||||
@li mp_abort.sas
|
||||
@@ -112,10 +111,7 @@ run;
|
||||
%let startrow=1;
|
||||
%let rows=4000;
|
||||
|
||||
/* a request with no rows never reached the validation below, so this is the
|
||||
only case that is missing inputs - an invalid value is a different abort, and
|
||||
saying "missing" for it would hide the reason */
|
||||
%mp_abort(iftrue= (%mf_nobs(work.iwant)<=0)
|
||||
%mp_abort(iftrue= (%mf_verifymacvars(libds col2)=0)
|
||||
,mac=&_program..sas
|
||||
,msg=%str(Missing inputs from iwant)
|
||||
)
|
||||
|
||||
@@ -13,9 +13,8 @@
|
||||
validate.
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mpe_assert_abort.sas
|
||||
@li mp_assert.sas
|
||||
@li mpe_execute.sas
|
||||
@li mx_execute.sas
|
||||
@li mf_getuniquefileref.sas
|
||||
|
||||
**/
|
||||
@@ -28,23 +27,27 @@
|
||||
%let f1=%mf_getuniquefileref();
|
||||
data _null_;
|
||||
file &f1 termstr=crlf;
|
||||
put 'LIBDS:$100. COL:$100.';
|
||||
put 'LIBDS:$19. COL:$9.';
|
||||
put '%sysfunc(coalescec(&dclib..MPE_X_TEST,)),SOME_TIME';
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputfiles=&f1:iwant,
|
||||
outref=web1,
|
||||
outstatus=work.ab1,
|
||||
failfast=0,
|
||||
viyaresult=WEBOUT_TXT
|
||||
)
|
||||
|
||||
%mpe_assert_abort(
|
||||
%let abort1=0;
|
||||
data _null_;
|
||||
set work.results;
|
||||
if state='canceled' then call symputx('abort1',1);
|
||||
run;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=(&abort1=1),
|
||||
desc=Macro content in libds aborts the service,
|
||||
outref=web1,
|
||||
msg=Invalid libds
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
/**
|
||||
@@ -53,23 +56,27 @@ run;
|
||||
%let f2=%mf_getuniquefileref();
|
||||
data _null_;
|
||||
file &f2 termstr=crlf;
|
||||
put 'LIBDS:$100. COL:$100.';
|
||||
put "&dclib..MPE_X_TEST,%nrstr(%sysfunc(coalescec(SOME_TIME,)))";
|
||||
put 'LIBDS:$19. COL:$9.';
|
||||
put '&dclib..MPE_X_TEST,%sysfunc(coalescec(SOME_TIME,))';
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputfiles=&f2:iwant,
|
||||
outref=web2,
|
||||
outstatus=work.ab2,
|
||||
failfast=0,
|
||||
viyaresult=WEBOUT_TXT
|
||||
)
|
||||
|
||||
%mpe_assert_abort(
|
||||
%let abort2=0;
|
||||
data _null_;
|
||||
set work.results;
|
||||
if state='canceled' then call symputx('abort2',1);
|
||||
run;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=(&abort2=1),
|
||||
desc=Macro content in col aborts the service,
|
||||
outref=web2,
|
||||
msg=Invalid col
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
/**
|
||||
@@ -78,11 +85,11 @@ run;
|
||||
%let f3=%mf_getuniquefileref();
|
||||
data _null_;
|
||||
file &f3 termstr=crlf;
|
||||
put 'LIBDS:$100. COL:$100.';
|
||||
put 'LIBDS:$19. COL:$9.';
|
||||
put "&dclib..MPE_X_TEST,SOME_TIME";
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputfiles=&f3:iwant,
|
||||
outlib=web3
|
||||
|
||||
@@ -15,7 +15,7 @@
|
||||
data work.params;
|
||||
length name $32 value $1000;
|
||||
name='libref';value="&dclib";output;
|
||||
name='ds';value='MPE_TABLES';output;
|
||||
name='ds';value='';output;
|
||||
name='flavour';value='TSQL';output;
|
||||
run;
|
||||
%mx_execute(&_program,
|
||||
@@ -31,7 +31,7 @@ data work.results;
|
||||
input;
|
||||
putlog _infile_;
|
||||
if _n_=2 then do;
|
||||
if _infile_=:'/* TSQL Flavour DDL for' and index(_infile_,'MPE_TABLES')>0 then do;
|
||||
if _infile_=:'/* TSQL Flavour DDL for' then do;
|
||||
putlog 'test passed';
|
||||
output;
|
||||
output;
|
||||
@@ -42,7 +42,7 @@ data work.results;
|
||||
run;
|
||||
|
||||
%mp_assertdsobs(work.results,
|
||||
desc=DDL file is successfully returned for a table,
|
||||
desc=DDL file is successfully returned,
|
||||
test=EQUALS 2,
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
@@ -15,10 +15,10 @@
|
||||
@li mf_existfeature.sas
|
||||
@li dc_assignlib.sas
|
||||
@li mp_abort.sas
|
||||
@li mp_binarycopy.sas
|
||||
@li mp_cntlout.sas
|
||||
@li mp_ds2cards.sas
|
||||
@li mp_ds2csv.sas
|
||||
@li mp_ds2md.sas
|
||||
@li mp_streamfile.sas
|
||||
@li mp_validatecol.sas
|
||||
@li mf_nobs.sas
|
||||
|
||||
@@ -13,9 +13,8 @@
|
||||
validate.
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mpe_assert_abort.sas
|
||||
@li mp_assert.sas
|
||||
@li mpe_execute.sas
|
||||
@li mx_execute.sas
|
||||
|
||||
**/
|
||||
|
||||
@@ -32,19 +31,23 @@ data work.params1;
|
||||
name='filter';value='0';output;
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputparams=work.params1,
|
||||
outref=web1,
|
||||
outstatus=work.ab1,
|
||||
failfast=0,
|
||||
viyaresult=WEBOUT_TXT
|
||||
)
|
||||
|
||||
%mpe_assert_abort(
|
||||
%let abort1=0;
|
||||
data _null_;
|
||||
set work.results;
|
||||
if state='canceled' then call symputx('abort1',1);
|
||||
run;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=(&abort1=1),
|
||||
desc=Macro content in table aborts the service,
|
||||
outref=web1,
|
||||
msg=Invalid table
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
/**
|
||||
@@ -57,19 +60,23 @@ data work.params2;
|
||||
name='filter';value='0 or 1=1';output;
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputparams=work.params2,
|
||||
outref=web2,
|
||||
outstatus=work.ab2,
|
||||
failfast=0,
|
||||
viyaresult=WEBOUT_TXT
|
||||
)
|
||||
|
||||
%mpe_assert_abort(
|
||||
%let abort2=0;
|
||||
data _null_;
|
||||
set work.results;
|
||||
if state='canceled' then call symputx('abort2',1);
|
||||
run;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=(&abort2=1),
|
||||
desc=SQL injection in filter aborts the service,
|
||||
outref=web2,
|
||||
msg=Invalid filter
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
/**
|
||||
@@ -84,19 +91,23 @@ data work.params3;
|
||||
name='filter';value='0';output;
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputparams=work.params3,
|
||||
outref=web3,
|
||||
outstatus=work.ab3,
|
||||
failfast=0,
|
||||
viyaresult=WEBOUT_TXT
|
||||
)
|
||||
|
||||
%mpe_assert_abort(
|
||||
%let abort3=0;
|
||||
data _null_;
|
||||
set work.results;
|
||||
if state='canceled' then call symputx('abort3',1);
|
||||
run;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=(&abort3=1),
|
||||
desc=Path traversal in table aborts the service,
|
||||
outref=web3,
|
||||
msg=Invalid table
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
/**
|
||||
@@ -109,7 +120,7 @@ data work.params4;
|
||||
name='filter';value='0';output;
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputparams=work.params4,
|
||||
outref=web4,
|
||||
|
||||
@@ -78,33 +78,13 @@ run;
|
||||
outview=work.vw_derived,
|
||||
outvars=work.derived_vars
|
||||
)
|
||||
/* A format catalog (LIBREF.CATALOGNAME-FC) must be left for mp_filterstore to
|
||||
rewrite into a cntlout dataset, and a -FC name handed over as targetds is not
|
||||
something it can read - so pass no targetds at all in that case. An empty
|
||||
targetds makes the engine fall back to libds, which is what a plain dataset
|
||||
wants anyway.
|
||||
|
||||
No macro triggers in this comment: the macro processor resolves them even
|
||||
inside a comment, so prose about conditions is itself executed and the parser
|
||||
reports "Expected DO not found" from here. Same reason the values below are
|
||||
plain LET statements with no trailing comma. */
|
||||
%let derived_nobs=%sysfunc(coalesce(%mf_nobs(work.derived_vars),0));
|
||||
%let is_fmtcat=%eval(%length(&ds)>3 and "%upcase(%substr(&ds,%length(&ds)-2,3))"="-FC");
|
||||
%let targetds_val=&ds;
|
||||
%if &derived_nobs>0 %then %do;
|
||||
%let targetds_val=work.vw_derived;
|
||||
%end;
|
||||
/* the derived view keeps precedence: a catalog is only blanked when there is no
|
||||
derived configuration to validate against. Two plain %if blocks rather than
|
||||
%else %if - that construct is what the parser rejected here before. */
|
||||
%if &derived_nobs=0 and &is_fmtcat=1 %then %do;
|
||||
%let targetds_val=;
|
||||
%end;
|
||||
%let derived_nobs=%mf_nobs(work.derived_vars);
|
||||
%let filterds=%sysfunc(ifc(&derived_nobs>0,work.vw_derived,&ds));
|
||||
|
||||
%mp_filterstore(
|
||||
libds=&ds
|
||||
,targetds=&targetds_val
|
||||
,queryds=work.filterquery,
|
||||
libds=&ds,
|
||||
targetds=&filterds,
|
||||
queryds=work.filterquery,
|
||||
filter_summary=&dc_libref..mpe_filteranytable,
|
||||
filter_detail=&dc_libref..mpe_filtersource,
|
||||
lock_table=&dc_libref..mpe_lockanytable,
|
||||
|
||||
@@ -12,9 +12,8 @@
|
||||
assertion cannot pass against a service that does not validate.
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mpe_assert_abort.sas
|
||||
@li mp_assert.sas
|
||||
@li mpe_execute.sas
|
||||
@li mx_execute.sas
|
||||
@li mf_getuniquefileref.sas
|
||||
|
||||
**/
|
||||
@@ -31,19 +30,23 @@ data _null_;
|
||||
put '%sysfunc(coalescec(&dclib..MPE_TABLES,))';
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputfiles=&f1:iwant,
|
||||
outref=web1,
|
||||
outstatus=work.ab1,
|
||||
failfast=0,
|
||||
viyaresult=WEBOUT_TXT
|
||||
)
|
||||
|
||||
%mpe_assert_abort(
|
||||
%let abort1=0;
|
||||
data _null_;
|
||||
set work.results;
|
||||
if state='canceled' then call symputx('abort1',1);
|
||||
run;
|
||||
|
||||
%mp_assert(
|
||||
iftrue=(&abort1=1),
|
||||
desc=Macro content in filter_table aborts the service,
|
||||
outref=web1,
|
||||
msg=Invalid filter_table
|
||||
outds=work.test_results
|
||||
)
|
||||
|
||||
/**
|
||||
@@ -68,7 +71,7 @@ AND,OR,2,DSN,=,"'MPE_LOCK_ANYTABLE'"
|
||||
;;;;
|
||||
run;
|
||||
|
||||
%mpe_execute(&_program,
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputfiles=&f2:iwant &f3:filterquery,
|
||||
outlib=web2
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
<h4> SAS Macros </h4>
|
||||
@li mx_execute.sas
|
||||
@li mp_assertdsobs.sas
|
||||
@li mp_assertcolvals.sas
|
||||
@li mf_getuniquefileref.sas
|
||||
|
||||
**/
|
||||
@@ -33,15 +34,8 @@ run;
|
||||
%mx_execute(&_program,
|
||||
viyacontext=&defaultcontext,
|
||||
inputfiles=&f1:iwant &f2:filterquery,
|
||||
outlib=web1,
|
||||
outref=web1ref,
|
||||
mdebug=&sasjs_mdebug
|
||||
outlib=web1
|
||||
)
|
||||
data _null_;
|
||||
infile web1ref;
|
||||
input;
|
||||
putlog _infile_;
|
||||
run;
|
||||
data result;
|
||||
set web1.result;
|
||||
run;
|
||||
@@ -70,8 +64,7 @@ run;
|
||||
viyacontext=&defaultcontext,
|
||||
inputdatasets=work.iwant work.filterquery,
|
||||
outlib=web2,
|
||||
outref=web2ref,
|
||||
mdebug=&sasjs_mdebug
|
||||
outref=web2ref
|
||||
)
|
||||
data _null_;
|
||||
infile web2ref;
|
||||
@@ -126,8 +119,7 @@ run;
|
||||
viyacontext=&defaultcontext,
|
||||
inputdatasets=work.iwant work.filterquery,
|
||||
outlib=web3,
|
||||
outref=web3ref,
|
||||
mdebug=&sasjs_mdebug
|
||||
outref=web3ref
|
||||
)
|
||||
data _null_;
|
||||
infile web3ref;
|
||||
|
||||
@@ -55,6 +55,7 @@
|
||||
@li mp_cntlout.sas
|
||||
@li mp_getcols.sas
|
||||
@li mp_getpk.sas
|
||||
@li mp_jsonout.sas
|
||||
@li mp_searchdata.sas
|
||||
@li mp_validatecol.sas
|
||||
@li mpe_columnlevelsecurity.sas
|
||||
|
||||
@@ -57,7 +57,7 @@ data _null_;
|
||||
run;
|
||||
%mp_assert(
|
||||
iftrue=(&nobs=&expected),
|
||||
desc=%str(&desc (&searchtype "%superq(searchval)" returned &nobs rows, expected &expected)),
|
||||
desc=&desc (&searchtype "%superq(searchval)" returned &nobs rows, expected &expected),
|
||||
outds=work.test_results
|
||||
)
|
||||
%mend search;
|
||||
@@ -118,6 +118,6 @@ proc sql noprint;
|
||||
quit;
|
||||
%mp_assert(
|
||||
iftrue=(&caprows=&capnobs and &capnobs gt 0 and &capnobs le &capmax),
|
||||
desc=%str(Checking a search beyond the cap returns as many rows as it reports (&capnobs reported, &caprows returned, cap &capmax)),
|
||||
desc=Checking a search beyond the cap returns as many rows as it reports (&capnobs reported, &caprows returned, cap &capmax),
|
||||
outds=work.test_results
|
||||
)
|
||||
@@ -44,12 +44,4 @@ run;
|
||||
|
||||
options metarepository=&repo;
|
||||
|
||||
%if "&sasjs_mdebug"="1" %then %do;
|
||||
data _null_;
|
||||
set &outds;
|
||||
putlog "NOTE: &sysmacroname: &outds:";
|
||||
putlog (_all_)(=);
|
||||
run;
|
||||
%end;
|
||||
|
||||
%mend dc_getlibs;
|
||||
@@ -20,15 +20,7 @@ create table &outds as
|
||||
,engine
|
||||
,'' as libraryid length=17
|
||||
from dictionary.libnames
|
||||
where upcase(libname) not in ("WORK","SASUSER","%upcase(&dc_libref)");
|
||||
where libname not in ('WORK','SASUSER');
|
||||
insert into &syslast values ("&DC_LIBREF", "&DC_LIBNAME",'','V9');
|
||||
|
||||
%if "&sasjs_mdebug"="1" %then %do;
|
||||
data _null_;
|
||||
set &outds;
|
||||
putlog "NOTE: &sysmacroname: &outds:";
|
||||
putlog (_all_)(=);
|
||||
run;
|
||||
%end;
|
||||
|
||||
%mend dc_getlibs;
|
||||
@@ -6,11 +6,6 @@
|
||||
|
||||
The MPELIB should be pre-assigned
|
||||
|
||||
The refresh macros are called from the statements this macro writes into a
|
||||
catalog, so they are not visible as calls to lint.
|
||||
|
||||
@sasjslint {"noUnusedMacros": false}
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mpe_refreshlibs.sas
|
||||
@li dc_assignlib.sas
|
||||
@@ -37,17 +32,12 @@ data libraries;
|
||||
where also libref="&libref";
|
||||
%end;
|
||||
file executor;
|
||||
str=cats('%mpe_refreshtables(',libref,');');
|
||||
str=cats('%mpe_refreshtables(',libref,')');
|
||||
put str;
|
||||
putlog str;
|
||||
str=cats('%mpe_refreshcatalogs(',libref,');');
|
||||
str=cats('%mpe_refreshcatalogs(',libref,')');
|
||||
put str;
|
||||
run;
|
||||
%if "&sasjs_mdebug"="1" %then %do;
|
||||
/* echo the generated statements, so a failure inside one of them can be
|
||||
traced back to the libref that produced it */
|
||||
%inc executor/source2;
|
||||
%end;
|
||||
%else %inc executor;
|
||||
%inc executor/source2;
|
||||
|
||||
%mend dc_refreshcatalog;
|
||||
@@ -6,12 +6,8 @@
|
||||
It also adds the STAGING directory as subdirectory to the BASE
|
||||
library location.
|
||||
|
||||
@warning This STP self destructs! It will delete itself (and the
|
||||
configurator) after a successful run, to avoid being executed twice and
|
||||
overwriting actual data. Pass a KEEPME column in the input data to keep
|
||||
them - test harnesses reseed before every run and cannot redeploy first.
|
||||
|
||||
@sasjslint {"noBannedStrings": false}
|
||||
@warning This STP self destructs! It will delete itself after a successful run
|
||||
to avoid being executed twice (and overwriting actual data)
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mf_getapploc.sas
|
||||
@@ -59,24 +55,6 @@ data _null_;
|
||||
run;
|
||||
%end;
|
||||
|
||||
/* optional KEEPME column - keep this program and the configurator on the drive
|
||||
* after a successful run. A normal install self-destructs so the seeding
|
||||
* cannot be run twice over live data; a test harness reseeds before every run
|
||||
* and wants them to stay put. */
|
||||
%let keepme=0;
|
||||
%if &exist=1 %then %do;
|
||||
data _null_;
|
||||
dsid=open('work.fromjs');
|
||||
/* every row, not just the first: KEEPME may be set on any of them */
|
||||
if varnum(dsid,'KEEPME')>0 then do until(eof);
|
||||
set work.fromjs end=eof;
|
||||
if upcase(cats(keepme)) in ('1','Y','YES','TRUE') then call symputx('keepme',1);
|
||||
end;
|
||||
rc=close(dsid);
|
||||
stop;
|
||||
run;
|
||||
%end;
|
||||
|
||||
%mp_abort(iftrue= (&syscc ne 0)
|
||||
,mac=&_program
|
||||
,msg=%str(Issue on makedata entry)
|
||||
@@ -213,15 +191,7 @@ data _null_;
|
||||
run;
|
||||
%end;
|
||||
|
||||
/* We ran successfully, now remove configurator and makedata STPs, unless the
|
||||
* caller asked to keep them (KEEPME in the input data). Keeping them is for
|
||||
* test harnesses, which reseed before every run and cannot redeploy first. */
|
||||
%if &keepme ne 1 %then %do;
|
||||
%ms_deletefile(&root/services/admin/configurator.sas);
|
||||
%ms_deletefile(&_program..sas);
|
||||
%end;
|
||||
%else %do;
|
||||
%put NOTE: &_program retained, as KEEPME was requested;
|
||||
%end;
|
||||
|
||||
/* We ran successfully, now remove configurator and makedata STPs */
|
||||
%ms_deletefile(&root/services/admin/configurator.sas)
|
||||
%ms_deletefile(&_program..sas)
|
||||
|
||||
@@ -21,15 +21,7 @@ create table &outds as
|
||||
,engine
|
||||
,'' as libraryid length=17
|
||||
from dictionary.libnames
|
||||
where upcase(libname) not in ("WORK","SASUSER","CASUSER","%upcase(&dc_libref)");
|
||||
where libname not in ('WORK','SASUSER','CASUSER');
|
||||
insert into &syslast values ("&DC_LIBREF", "&DC_LIBNAME",'','V9');
|
||||
|
||||
%if "&sasjs_mdebug"="1" %then %do;
|
||||
data _null_;
|
||||
set &outds;
|
||||
putlog "NOTE: &sysmacroname: &outds:";
|
||||
putlog (_all_)(=);
|
||||
run;
|
||||
%end;
|
||||
|
||||
%mend dc_getlibs;
|
||||
@@ -6,11 +6,6 @@
|
||||
|
||||
The MPELIB should be pre-assigned
|
||||
|
||||
The refresh macros are called from the statements this macro writes into a
|
||||
catalog, so they are not visible as calls to lint.
|
||||
|
||||
@sasjslint {"noUnusedMacros": false}
|
||||
|
||||
<h4> SAS Macros </h4>
|
||||
@li mpe_refreshlibs.sas
|
||||
@li dc_assignlib.sas
|
||||
@@ -41,18 +36,13 @@
|
||||
set &mpelib..mpe_datacatalog_libs;
|
||||
where &dc_dttmtfmt. le TX_TO;
|
||||
file executor;
|
||||
str=cats('%mpe_refreshtables(',libref,');');
|
||||
str=cats('%mpe_refreshtables(',libref,')');
|
||||
put str;
|
||||
str=cats('%mpe_refreshcatalogs(',libref,');');
|
||||
str=cats('%mpe_refreshcatalogs(',libref,')');
|
||||
put str;
|
||||
putlog str;
|
||||
run;
|
||||
%if "&sasjs_mdebug"="1" %then %do;
|
||||
/* echo the generated statements, so a failure inside one of them can be
|
||||
traced back to the libref that produced it */
|
||||
%inc executor/source2;
|
||||
%end;
|
||||
%else %inc executor;
|
||||
%inc executor;
|
||||
%end;
|
||||
|
||||
%mend dc_refreshcatalog;
|
||||
Loaded 100 of 102 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user