Compare commits

...
59 Commits
Author SHA1 Message Date
semantic-release-bot 779006b0bf chore(release): 7.18.0 [skip ci]
# [7.18.0](https://git.datacontroller.io/dc/dc/compare/v7.17.0...v7.18.0) (2026-10-19)

### Bug Fixes

* **deps:** bump @sasjs/core to 5.4.0 ([19fb032](19fb032f0d))
* **deps:** pin @sasjs/utils and drop the unused jsrsasign ([05d2d1e](05d2d1e08e))
* **editor:** own the delete-record column client-side ([56025cc](56025cc052))
* **editor:** surface the reserved-column refusal on every getdata path ([b26f557](b26f5571b6))
* **filters:** do not turn a fractional temporal value into a literal ([f710d87](f710d871ff))
* **filters:** fix the dynamic where clause deref and map its temporal values ([ece67d3](ece67d37ba))
* **filters:** keep the range field readable and cover a BETWEEN in the rewrite ([be79f3a](be79f3a1ea))
* **filters:** map an IN entry to its unformatted value before the literal ([8f98ebe](8f98ebe326))
* **filters:** stop the temporal rewrite corrupting a typed value, and bind the range dropdown to the unformatted value ([a5f61d4](a5f61d4a73))
* **filters:** use the unformatted value in every IN branch ([cc01f54](cc01f540c2))
* **makedata:** shorten the DC_MAXCELLS_WEBEDIT description to fit var_desc ([a51d9e5](a51d9e5574))
* **multi-dataset:** surface the reserved-column guard, use the name constant ([2b8a2fa](2b8a2fa103))
* **release:** emit the release body as raw text, not a JSON string ([a05554e](a05554e67a))

### Features

* **filters:** show dates as date literals in the filter query ([662d000](662d00032c))
2026-10-19 14:14:13 +00:00
allan 6f65bb691f Merge pull request 'feat: release 7.18 - sasjs dependency bumps, filter date literals and editor fixes' (#345) from feat/release-7.18 into main
Release / Build-production-and-ng-test (push) Successful in 5m29s
Release / Build-and-test-development (push) Successful in 25m7s
Release / release (push) Successful in 8m12s
Reviewed-on: #345
2026-10-19 13:39:26 +00:00
allan 3d21a0134c Merge pull request 'fix(makedata): shorten the DC_MAXCELLS_WEBEDIT description to fit var_desc' (#346) from fix/makedata-vardesc-length into feat/release-7.18
Build / Build-and-ng-test (pull_request) Successful in 5m54s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m22s
Build / Build-and-test-development (pull_request) Successful in 32m5s
Reviewed-on: #346
2026-10-19 13:39:02 +00:00
dc a51d9e5574 fix(makedata): shorten the DC_MAXCELLS_WEBEDIT description to fit var_desc
Lighthouse Checks / lighthouse (pull_request) Successful in 20m46s
Build / Build-and-ng-test (pull_request) Successful in 5m23s
Build / Build-and-test-development (pull_request) Successful in 30m13s
MPE_CONFIG.VAR_DESC is varchar(300), but the description written for
DC_MAXCELLS_WEBEDIT is 336 characters, so the insert truncates it.  The
truncation raises a WARNING, which sets SYSCC, and makedata aborts on
`&syscc ne 0` immediately after %mpe_makedata():

    WARNING: Character expression will be truncated when assigned to
      character column var_desc.
    ERROR: Execution terminated by an ABORT CANCEL statement

The abort happens before the settings job is written, so a fresh deploy
ends with no control library and no settings.sas at all.  It goes
unnoticed because the SAS test suite reuses whatever library is already
on the estate - the first time it bites is a deploy from scratch.

The description is shortened to 201 characters in both places it is
written - mpe_makedata (new installs) and the 20261003 row/cell limits
migration (existing installs).  The rationale it carried is user-visible
help text, so it moves to a SAS comment above each insert rather than
being dropped.

Verified on a real Viya estate, deploying from scratch:
  - before: makedata aborted, "Err during VIYA1337 build", no settings.sas
  - after:  makedata completed with zero warnings, and the SAS test suite
            then ran 59/59 green
2026-10-19 12:22:13 +00:00
hermes 026930d3e4 Merge pull request 'feat(filters): show dates as date literals in the filter query' (#344) from feat/filter-date-literals into feat/release-7.18
Lighthouse Checks / lighthouse (pull_request) Successful in 20m29s
Build / Build-and-ng-test (pull_request) Successful in 5m6s
Build / Build-and-test-development (pull_request) Successful in 29m19s
2026-10-17 20:10:37 +00:00
dc 05d2d1e08e fix(deps): pin @sasjs/utils and drop the unused jsrsasign
Build / Build-and-ng-test (pull_request) Successful in 5m18s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m50s
Build / Build-and-test-development (pull_request) Successful in 29m57s
@sasjs/utils is imported directly across the client but was left as a caret
range in the 7.18 bump, while every other dependency in the file is exact.
Pin it to 4.0.0, which is the version the lockfile already resolves.

jsrsasign is declared but referenced nowhere in the tracked source - the only
other mentions are the lockfile entry and a historical CHANGELOG line - so
drop it.

Lockfile regenerated with npm install --package-lock-only; the only deltas are
the pinned range and the removed jsrsasign tree. 570/570 unit tests pass.
2026-10-17 19:32:14 +00:00
dc cc01f540c2 fix(filters): use the unformatted value in every IN branch
Build / Build-and-ng-test (pull_request) Successful in 5m12s
Lighthouse Checks / lighthouse (pull_request) Successful in 19m55s
Build / Build-and-test-development (pull_request) Successful in 29m58s
Finding from the review at ece67d3.

dynamicWhereClauseCreator's IN branch mapped the entry through element.values
only when building the literal; the two fallbacks emitted the unmapped value, so
a formatted label could still reach RAW_VALUE.  A temporal entry whose mapped
value is not a whole day / second (sasTemporalLiteral returns null for a
genuinely fractional value) or whose label finds no match in element.values
would fall through to the quoted label on a numeric column, which mp_filtercheck
rejects and which aborts the getcolvals load inside the swallowed catch.

The mapped value is now assigned once and used in all three branches, as
createFilterQueryTable's IN branch does.

One unit case added for the non-literal fallback.  582/582 unit tests pass.
2026-10-16 21:44:08 +00:00
dc ece67d37ba fix(filters): fix the dynamic where clause deref and map its temporal values
Build / Build-and-ng-test (pull_request) Successful in 5m34s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m3s
Build / Build-and-test-development (pull_request) Successful in 29m51s
Two findings from the review at 8f98ebe.

The IN branch read this.clauses.queryObj[groupInd][clauseInd].value, but the
entries sit at queryObj[groupInd].elements[clauseInd].value, so the forEach
dereferenced undefined and threw.  dynamicWhereClauseCreator runs inside
getValues / getValuesLoadMore, where the surrounding try/catch swallowed the
TypeError and the value list silently failed to load.  It now reads
element.value, like the BETWEEN and equality branches.

The BETWEEN branch composed element.value directly, so a temporal BETWEEN
picked from the value list sent the formatted strings into RAW_VALUE, which
mp_filtercheck rejects on a numeric column and which aborts the values load.
Both ends now go through the same formatted-to-unformatted lookup the IN branch
and createFilterQueryTable use, in one unformattedValue helper shared by the
two branches.

query.component gains a spec covering both branches.  581/581 unit tests pass.
2026-10-16 21:03:48 +00:00
dc 8f98ebe326 fix(filters): map an IN entry to its unformatted value before the literal
Build / Build-and-ng-test (pull_request) Successful in 5m18s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m51s
Build / Build-and-test-development (pull_request) Successful in 30m50s
Finding from the review at be79f3a.

createFilterQueryTable's IN / NOT IN branch read element.val directly, so it
produced a temporal literal only because whereClauseCreator had already mutated
query.value[i].val from the formatted label to the unformatted count in place
before submit.  Any path reaching the branch without the preview having run
first would have put the formatted string into RAW_VALUE.

The entry now goes through the same unformattedValue() lookup as the BETWEEN
ends.  An already-unformatted value passes through unchanged, since it no longer
matches a formatted label.

Two unit cases added: an IN entry carrying the formatted label, and one already
carrying the internal count.  578/578 unit tests pass.
2026-10-16 20:21:38 +00:00
dc be79f3a1ea fix(filters): keep the range field readable and cover a BETWEEN in the rewrite
Build / Build-and-ng-test (pull_request) Successful in 5m10s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m27s
Build / Build-and-test-development (pull_request) Successful in 30m5s
Two findings from the review at 72a7963.

whereClauseCreator's guard could not match a BETWEEN: checkOperator composes
it as `<from> AND <to>` and the letters of AND are outside the character class,
so the temporal rewrite was skipped and the filter-panel preview kept the
internal day counts (`BETWEEN 42 AND 43`).  The separator is removed before the
test, so a BETWEEN pair is shown as literals like a single value or an IN list.

Binding the range dropdown to column.unformatted made the field render the
internal value: setOptionValue assigns the clicked option's value to the input's
ngModel, so after choosing 12FEB1960 the field read 42.  The option value is the
formatted label again, as the single-value list and the IN modal already are,
and createFilterQueryTable's BETWEEN branch maps each end back to its
unformatted value before building the literal - the same lookup
whereClauseCreator applies for the preview.

Three new unit cases pin the BETWEEN preview and the BETWEEN RAW_VALUE in both
directions.  576/576 unit tests pass.
2026-10-16 16:29:57 +00:00
dc 72a7963972 test(e2e): cover a temporal BETWEEN chosen from the value list
Build / Build-and-ng-test (pull_request) Successful in 5m56s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m18s
Build / Build-and-test-development (pull_request) Successful in 31m9s
With the pickers off, a temporal BETWEEN takes both of its values from a
soft-select over the column's values, so the clause is composed from whatever
that list binds.  The test chooses 12FEB1960 for both ends of SOME_DATE and
asserts the applied clause carries the literals and still selects the 5 rows
the internal value selected.

Typing into the range fields - which is what the existing helper does - does
not exercise the binding, so both values are picked from the datalist instead.

Also accepts the evaluation agreement in beforeAll when it is presented.  It
is per-origin localStorage, so a fresh browser profile lands on it and the nav
tree never renders; the accept is guarded, so it is a no-op once an earlier
spec in the run has accepted it.
2026-10-16 13:52:19 +00:00
dc a5f61d4a73 fix(filters): stop the temporal rewrite corrupting a typed value, and bind the range dropdown to the unformatted value
Build / Build-and-ng-test (pull_request) Successful in 5m51s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m11s
Build / Build-and-test-development (pull_request) Successful in 31m50s
Two findings from the review at f710d87.

whereClauseCreator's temporal rewrite is a regex pass over the composed value
string, so it rewrote every digit run in it.  A date typed by hand as
12/02/1960 composed to 12/02/1960 and the clause preview rendered
'13JAN1960'd/'03JAN1960'd/'14MAY1965'd.  The pass now runs only when the
composed value is nothing but the numbers the clause is built from - a single
value, a BETWEEN pair or an IN list - so a typed value is left alone while an
internal day count is still shown as a literal.

The range dropdown bound [value]="column.formatted", where the single-value
dropdown binds column.unformatted.  whereClauseCreator's BETWEEN mapping writes
the unformatted value to a local copy rather than back onto the emitted clause,
so the formatted strings reached createFilterQueryTable, sasTemporalLiteral
returned null for them, and RAW_VALUE came out as `12FEB1960 AND 12FEB1960` -
unquoted, which mp_filtercheck rejects for a numeric column and so aborts the
save.  The option value is now the unformatted value, with the formatted text
kept as the label, matching the single-value dropdown.

573/573 unit tests pass, including two new cases pinning the preview: a typed
value is left alone, and an internal day count is still shown as a literal.
2026-10-16 13:18:46 +00:00
dc f710d871ff fix(filters): do not turn a fractional temporal value into a literal
Build / Build-and-ng-test (pull_request) Successful in 5m33s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m47s
Build / Build-and-test-development (pull_request) Successful in 32m9s
sasTemporalLiteral rendered the fraction with String(fraction).slice(1), which
emits raw IEEE-754 noise - 42.3 came out as '00:00:42.29999999999999716't and
1.1 as .10000000000000009.  Only the 0.5 case in the spec was clean.

SAS temporal values are whole days and whole seconds, so the value is rounded to
seconds first, which absorbs the noise a double picks up on the way here
(1.9999999999999 is simply the second 2).  A value that is genuinely fractional
does not land on a second, and rounding it would silently widen the filter, so
it now returns null and the caller sends the raw number instead.

sasTimeFromSeconds has no fraction left to render, so that branch is gone.

Also corrects the doc examples, which paired 34322 with '20APR2042'd - 34322
days after 1960-01-01 is '20DEC2053'd, which is what the spec already asserted.
Same fix in the mock's parser comment.

571/571 unit tests pass.
2026-10-16 13:07:52 +00:00
allan d6e0662b16 Merge pull request 'fix(editor): own the delete-record column client-side' (#343) from fix/getdata-delete-column-client-side into feat/release-7.18
Build / Build-and-ng-test (pull_request) Successful in 5m54s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m16s
Build / Build-and-test-development (pull_request) Successful in 32m5s
Reviewed-on: #343
2026-10-16 13:01:43 +00:00
allan f45b3259c7 Merge pull request 'fix(release): emit the release body as raw text, not a JSON string' (#342) from fix/release-notes-body-encoding into feat/release-7.18
Build / Build-and-ng-test (pull_request) Successful in 6m19s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m33s
Build / Build-and-test-development (pull_request) Successful in 31m11s
Reviewed-on: #342
2026-10-16 13:00:48 +00:00
dc b53f951307 chore: merge the 7.18 release branch into the filter date literals branch
Build / Build-and-ng-test (pull_request) Successful in 6m15s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m40s
Build / Build-and-test-development (pull_request) Successful in 32m53s
2026-10-16 12:42:46 +00:00
dc 9f8c71eb47 chore(deps): bump the sasjs dependencies for the 7.18 release
Build / Build-and-ng-test (pull_request) Successful in 6m11s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m12s
Build / Build-and-test-development (pull_request) Successful in 33m12s
sas/package.json
  @sasjs/cli    4.20.5 -> 4.20.9
  @sasjs/core   5.2.8  -> 5.5.2

client/package.json
  @sasjs/adapter  4.19.1 -> 5.1.0
  @sasjs/utils    ^3.6.0 -> ^4.0.0

Both client bumps are majors, and both are safe here:

- @sasjs/adapter 5.0.0 raised engines.node to >= 22.  The workflows run Node
  24.15.0 (24.5.0 in release.yaml), so the floor is satisfied.  5.1.0 also
  repackaged the adapter as compiled modules - DC imports only the package
  root (SASjsConfig, UploadFile, SASjsRequest), which index.d.ts still
  re-exports through ./types, and the package ships no exports map, so the
  imports DC does make are unaffected.
- @sasjs/utils 4.0.0 stopped re-exporting ./fs from the package root.  DC
  imports @sasjs/utils/input/validators and @sasjs/utils/types/serverType,
  neither of which is fs, and both still ship in 4.0.0.

@sasjs/adapter 5.1.0 depends on @sasjs/utils 4.0.0, so the two move together.

The shelljs override in sas/package.json is dropped: @sasjs/cli 4.20.9 no
longer depends on shelljs, so nothing in the tree resolves to it any more and
the pin was dead config.  The nanoid pin is untouched.

Lockfiles regenerated with npm install --package-lock-only --ignore-scripts,
then validated by enumerating every added, removed and version-changed entry
and tracing each one back to @sasjs/cli, @sasjs/core, @sasjs/adapter or
@sasjs/utils.  No entry was pruned as extraneous.

Verified by reproducing the Build-and-ng-test gates locally:

  npm audit --omit=dev      0 vulnerabilities in root, sas/ and client/
  npm run lint:check        pass
  npm run license-checker   pass
  npm run test:headless     563/563 SUCCESS
  npm run build             pass
2026-10-16 12:41:29 +00:00
dc 19fb032f0d fix(deps): bump @sasjs/core to 5.4.0
Build / Build-and-ng-test (pull_request) Successful in 5m28s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m15s
Build / Build-and-test-development (pull_request) Successful in 31m29s
5.4.0 carries the mp_filtercheck change that accepts a SAS temporal literal
in RAW_VALUE, which is what the filter literal work needs.  Without it the
literal is rejected when the filter is stored, so the clause stays numeric on
a real estate even though the mocks (which do not run the SAS macro) pass.

Verified the build inlines it from the project copy, not the CLI's nested one:
sasjsbuild/server_compileTree.json resolves mp_filtercheck.sas from
node_modules/@sasjs/core, and the compiled server.sas carries the fix.
2026-10-16 11:16:33 +00:00
dc 662d00032c feat(filters): show dates as date literals in the filter query
Build / Build-and-ng-test (pull_request) Successful in 5m41s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m22s
Build / Build-and-test-development (pull_request) Successful in 31m9s
The clause shown above the grid (sasparams.FILTER_TEXT) is the resolved WHERE
clause, so a date filter read SOME_DATE = 34322 rather than a date.  The
client now sends a SAS literal for a DATE, DATETIME or TIME column -
'20APR2042'd, '01JAN1960:00:00:42'dt, '00:00:42't - which SAS accepts in the
same place, so the applied filter is unchanged and the clause is readable.

HelperService.sasTemporalLiteral builds the literal arithmetically in UTC, so
the day cannot shift with the browser timezone, and fractional seconds are
preserved.  It returns null for anything that is not a temporal value, and the
callers then send the raw value exactly as before.

Used by createFilterQueryTable (the applied filter), whereClauseCreator (the
clause preview in the filter panel), dynamicWhereClauseCreator (the filter
getcolvals applies to the value list) and the VA filter service.

Two fixes fall out of the same code paths:

- the dynamic where clause read the second BETWEEN value from
  queryObj[groupInd][clauseInd], which is undefined, so a BETWEEN clause threw;
- an IN over a temporal column sent the *formatted* value, which the backend
  refuses for a numeric column - the entries are now mapped back to the
  unformatted value before the literal is built.

The mock filter parser reads the literals too.  A time literal carries no date
part, so it needs its own pattern - the date pattern alone silently matched
nothing and a time filter returned no rows.

filtering.cy.ts asserts the new clause text and, for every temporal case, the
row count, so a clause that reads well but filters differently cannot pass.
2026-10-16 09:36:36 +00:00
dc b26f5571b6 fix(editor): surface the reserved-column refusal on every getdata path
Build / Build-and-ng-test (pull_request) Successful in 6m24s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m18s
Build / Build-and-test-development (pull_request) Successful in 32m50s
Review on #343:

- initSetup's retry branch called itself from a bare setTimeout, outside the
  callService promise chain, so the reserved-column throw on a retry never
  reached the .catch: getdataError was never set and the user was left on the
  loading spinner. Wrap the retry call and route it through the same failure
  handler the .catch uses (now a private handleGetdataFailure, shared by
  both).
- The multi-dataset catch logged and returned, leaving an empty table
  selection with the file still attached and only a console.warn. Surface it
  with eventService.showInfoModal, the pattern this file already uses for
  other upload problems.
2026-10-16 00:40:29 +00:00
dc 2b8a2fa103 fix(multi-dataset): surface the reserved-column guard, use the name constant
Lighthouse Checks / lighthouse (pull_request) Successful in 21m1s
Build / Build-and-ng-test (pull_request) Successful in 5m14s
Build / Build-and-test-development (pull_request) Successful in 32m10s
Review on #343:

- buildDatasetsObjects() runs inside the Promise.allSettled(...).then(...)
  chain in onStartParsingFile, which had no .catch. prepareDeleteRecordColumn
  throws on a response carrying the reserved delete-record column, so in this
  flow that became an unhandled rejection: the loop aborted, no dataset was
  parsed and no state was shown. Wrap the call and drop the parse, the way
  fetchDataset's own catch handles a failed getdata.
- The rename-to-Delete? block still used the literal name while the same
  block in editor.component.ts uses DELETE_RECORD_COLUMN_NAME. Use the
  constant here too.
2026-10-15 23:46:22 +00:00
dc 56025cc052 fix(editor): own the delete-record column client-side
Build / Build-and-ng-test (pull_request) Successful in 5m36s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m15s
Build / Build-and-test-development (pull_request) Successful in 30m57s
getdata.sas synthesised _____DELETE__THIS__RECORD_____ itself - it added
the column to work.out, hoisted it in front of the PKs, set it to "No" on
every row, gave it a No/Yes dropdown in the metadata and forced MAXLEN=3
in maxvarlengths. In the editor that column is a constant: it is only ever
"No" on load, and its Yes/No rule is already injected client-side
(deleteRecordColumnRule.ts, since #253's vertical-array fix). So the
backend was sending a column the client already owns.

The editor now creates the column, the same way EDIT_STATUS / dc.row_status
is client-only:

- getdata.sas no longer emits it (data steps, the outdata hoist, the
  ctrlOptions dropdown, the maxvarlengths override).
- deleteRecordColumn.ts asserts the response does NOT carry the name, then
  prepends it to COLHEADERS. Everything downstream (DcValidator's rules,
  columnHeader, headerArray) is keyed off COLHEADERS order, so the delete
  column is still index 0 and the grid is unchanged.
- Every row is seeded to 'No' before the validator or HyperFormula sees it.
- A response that does carry the name now throws instead of rendering a
  table whose delete column would shadow a real column of that name. The
  throw lands in the existing getdata .catch, which sets getdataError and
  shows the error state. The name is reserved; the docs note is in the
  sibling docs PR.

The multi-dataset loader (multi-dataset.component.ts) builds its own
DcValidator from the same responses, so it goes through the same helper.

Mocks updated to the new contract (server + sas9 getdata); postdata,
getstagetable and makedata are untouched - they consume or seed the flag
legitimately, and the upload contract in excel.md / files.md is unchanged.

Verified: client unit tests 570/570, npm run build clean, sasjs lint clean
on the changed SAS, sasjs compile -t server succeeds, and against the
mocked estate coltype-delete-record.cy.ts 2/2, editor.cy.ts 55/55,
excel.cy.ts 23/23 and row-cell-limits.cy.ts 5/5.
2026-10-15 22:52:44 +00:00
dc a05554e67a fix(release): emit the release body as raw text, not a JSON string
Build / Build-and-ng-test (pull_request) Successful in 5m31s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m53s
Build / Build-and-test-development (pull_request) Successful in 30m52s
The "Upload assets to release" step builds the new body with
`jq -n --arg body ... '$body + "\n" + ...'` and no `-r`. jq then prints the
result as a JSON string literal - quoted, with `\n` escapes - which the
PATCH that follows encodes a second time, so the stored release body is a
quoted one-liner instead of markdown. On v7.17.0 this rendered the whole
release as a single escaped line in the markdown box.

Add `-r` so the concatenated notes are emitted as raw text. Verified by
running the step's jq invocation against sample notes: without `-r` the
stored body contains literal `\n`, with `-r` it contains real newlines.

release.yaml only runs on a push to main, so this cannot be exercised by
CI on the PR itself.
2026-10-15 20:50:01 +00:00
semantic-release-bot 402ec346b9 chore(release): 7.17.0 [skip ci]
# [7.17.0](https://git.datacontroller.io/dc/dc/compare/v7.16.0...v7.17.0) (2026-10-15)

### Bug Fixes

* **client:** do not publish an empty getcols fallback to the picker ([9f8fc6f](9f8fc6f2eb))
* **client:** map the getcols fallback onto the picker vocabulary ([7cecf79](7cecf79515))
* **client:** pin the dependency entries this release changes ([b034fe5](b034fe5335))
* **deploy:** apply the same identity-call fix to the automatic deploy ([5693b44](5693b4488b))
* **deploy:** disable the manual deploy actions without an admin group ([6ffbdba](6ffbdbadf9))
* **deploy:** disable the manual deploy file inputs while admin groups load ([7094bfd](7094bfdf7c))
* **deploy:** guard the manual deploy form and Viya identity calls ([6522fcb](6522fcb28b))
* **deploy:** never bake an empty admin group into the deploy script ([b0e47fc](b0e47fc4ed))
* **deps:** pin shelljs to clear the braces advisory in the sas audit ([6e50938](6e5093858b))
* **editor:** check the EDIT limits once, and skip switched-off options ([f08c4c2](f08c4c231f))
* **filters:** carry TYPE on the fallback list, and drop the temporal exclusion ([bcb8fde](bcb8fde986))
* **filters:** source the picker variables from public/getcols when the table is not returned ([3a6a216](3a6a216b9d))
* **getcols:** use %global and %mp_getcols, and merge the base branch ([1385741](1385741f2d))
* **getcols:** widen TYPE before assigning the client vocabulary ([bcad05e](bcad05e032))
* **getdata:** apply row level security to selectbox source values ([768d33c](768d33c618)), closes [#340](#340)
* **mocks:** read MPE_CONFIG correctly, and pin the limits to it ([05126ae](05126ae73a))
* repair the SAS suite for the row/cell limits and default the manual deploy admin group ([99ff732](99ff732d67))
* **requests-modal:** widen the dialog so log lines need less scrolling ([3c10d91](3c10d915a5))
* **sas:** keep format and label in the public getcols response ([9c32b1b](9c32b1b1da))
* **sas:** keep the SAS C/N vocabulary in the public getcols response ([e90bcde](e90bcde512))
* **sas:** run the row/cell limit migration as a Viya job ([90af6b8](90af6b8515))
* **startup:** stack the SASjs tag detail so the dump is readable ([1f4c4a4](1f4c4a4d82))

### Features

* **debug:** keep the debug toggle state across a page refresh ([3ce006d](3ce006d6b4))
* **editor:** raise the row limits and add a cell limit to the EDIT screen ([3de6655](3de6655f26))
* **mocks:** read the DC_MAX* options from MPE_CONFIG, and mock the DDL export ([edac9a9](edac9a987a))
* **startup:** call out a missing SASjs tag on the boot screen ([b24eb73](b24eb731ce))
* **startup:** dump the SASjs tag contents in the boot screen checks ([75c1dcc](75c1dccfd2))
* **system:** export the DC library DDL from the system screen ([05f3f49](05f3f49e0c)), closes [#179](#179)
2026-10-15 20:21:53 +00:00
allan e48bf80ab4 Merge pull request 'feat: release 7.17 - DDL export, Handsontable 18.1.1, EDIT row and cell limits' (#333) from feat/export-dc-library-ddl into main
Release / Build-production-and-ng-test (push) Successful in 4m47s
Release / Build-and-test-development (push) Successful in 23m40s
Release / release (push) Successful in 7m49s
Reviewed-on: #333
2026-10-15 19:49:27 +00:00
dc dd958ac4e7 chore(styles): drop the dead manual-deploy CSS block
Build / Build-and-ng-test (pull_request) Successful in 5m47s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m10s
Build / Build-and-test-development (pull_request) Successful in 31m19s
8fdb468 removed the only component matching app-manual-deploy and the
.clear-memory-button rule inside it, leaving the block in styles.scss with no
consumer anywhere in the tree.
2026-10-15 16:48:35 +00:00
dc 8fdb468ad3 refactor(deploy): remove the manual deploy section
Build / Build-and-ng-test (pull_request) Successful in 6m21s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m13s
Build / Build-and-test-development (pull_request) Successful in 33m8s
It is the old Viya deployment form - the one built around uploading a JSON build
file - and it is not coming back. It was reachable only through the manualdeploy
route, which goes with it, so the Viya deploy screen is now the automatic flow
only.

That takes the manualdeploy-only plumbing in deploy.component with it (autodeploy
and its dead commented-out toggle, adminGroups/getAdminGroups, jsonFile) along
with the now-unused Router and LoggerService injections, and the section's two
spec files.
2026-10-15 15:13:00 +00:00
dc af52e4a1bf docs(contributing): drop the tag-based Handsontable key instructions
Build / Build-and-ng-test (pull_request) Successful in 6m30s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m23s
Build / Build-and-test-development (pull_request) Failing after 33m50s
The key now comes from the backend licence (hot_license_key on the decrypted
licence), falling back to DEFAULT_HOT_LICENCE_KEY, so the hotLicenceKey tag
attribute the docs told you to set has no reader. Also reword the missing-tag
comment in parseDcAdapterSettings, which claimed there was no case to handle
while the boot screen reports one.
2026-10-15 15:07:58 +00:00
dc b24eb731ce feat(startup): call out a missing SASjs tag on the boot screen
Build / Build-and-ng-test (pull_request) Successful in 6m34s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m32s
Build / Build-and-test-development (pull_request) Failing after 34m53s
The tag is a mandatory part of index.html - it carries serverUrl, serverType and
the rest of the adapter config - so the missing-tag branch (deactivateApp plus an
abort modal) goes. A missing tag is reported on the tag step itself instead,
which is where the rest of the boot diagnostics already live, and the settings
parse is null-safe so nothing throws on the way there.
2026-10-15 14:54:11 +00:00
dc d81967e6a1 refactor(sasjs-tag): unwire adminGroup and hotLicenceKey from the tag
Build / Build-and-ng-test (pull_request) Successful in 6m29s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m9s
Build / Build-and-test-development (pull_request) Successful in 33m25s
Both were hangovers from the earlier deployment flow.

adminGroup is a DC setting (DC_ADMIN_GROUP, shown on the System screen), not a
tag attribute. Its only readers were the deploy form's default in
deploy.component.ts and manual.component.ts, and both already fall back to
SASAdministrators when it is absent, so the fallback is now the whole logic.

hotLicenceKey was the Handsontable licence key. The customer's own key arrives
in the decrypted licence (hot_license_key), so the client carries the free-tier
default itself and no longer reads it from index.html.

With neither attribute on the tag, the boot-screen dump has nothing to hide:
HIDDEN_TAG_ATTRIBUTES is gone and every attribute is shown, so a wrong or
misspelt one is visible. That closes the review finding on hiding adminGroup.
The missing-tag branch also records why it carries no error state, closing the
other carried finding.
2026-10-15 14:43:42 +00:00
dc c81db17588 chore(index): drop the adminGroup attribute from the SASjs tag
Build / Build-and-ng-test (pull_request) Successful in 5m43s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m48s
Build / Build-and-test-development (pull_request) Successful in 31m50s
It is a hangover from the earlier deployment flow: its only consumers are the
deploy form's default (deploy.component.ts setDeployDefaults and
manual.component.ts), both of which already fall back to SASAdministrators when
it is absent. The attribute stays readable, so an install that still sets it is
unaffected.

hotLicenceKey is left in place - it is not a deployment setting. It is the
Handsontable licence key, seeded into freeTierLicenceData.hot_license_key and
pushed to hotTable.licenseKey in six components, and nothing on the SAS side
supplies it.
2026-10-15 14:19:54 +00:00
dc 3c10d915a5 fix(requests-modal): widen the dialog so log lines need less scrolling
Build / Build-and-ng-test (pull_request) Successful in 5m48s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m50s
Build / Build-and-test-development (pull_request) Successful in 31m8s
Clarity's `xl` modal size is a fixed 57.6rem (1152px), so the SAS log pane -
which is `white-space: pre` and deliberately never wraps - needed left/right
scrolling to read a line. Give the requests modal a 90vw dialog instead; the
log pane itself is untouched, so lines still do not wrap.
2026-10-15 13:43:43 +00:00
dc 3ce006d6b4 feat(debug): keep the debug toggle state across a page refresh
Build / Build-and-ng-test (pull_request) Successful in 6m35s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m55s
Build / Build-and-test-development (pull_request) Successful in 32m38s
The <sasjs> tag in index.html is re-read on every load, so the debug switch fell
back to the tag's value whenever the page was refreshed. Store the state the user
picks from the UI in localStorage, and reinstate it when the adapter is built, so
a refresh keeps the state from before.
2026-10-15 12:38:54 +00:00
dc cc4cbd52e4 refactor(startup): show the SASjs tag as a name/value grid
Build / Build-and-ng-test (pull_request) Successful in 6m30s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m46s
Build / Build-and-test-development (pull_request) Failing after 33m17s
Render the tag attributes as a two-column grid under the label instead of a
single comma-joined line, and leave the admin group and the licence key off the
boot screen.

The missing-tag error state goes too: the tag ships in index.html, so an absent
tag is not a case the boot screen needs to report.
2026-10-15 12:22:27 +00:00
dc 1f4c4a4d82 fix(startup): stack the SASjs tag detail so the dump is readable
Build / Build-and-ng-test (pull_request) Successful in 6m36s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m26s
Build / Build-and-test-development (pull_request) Failing after 34m3s
The check row renders the detail in a narrow right-hand column, which squeezed
the tag dump into a clipped block next to the label. Long details now wrap onto
their own full-width line, left aligned, via a `stacked` flag on the step.
2026-10-15 12:11:57 +00:00
dc 75c1dccfd2 feat(startup): dump the SASjs tag contents in the boot screen checks
Build / Build-and-ng-test (pull_request) Successful in 5m56s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m38s
Build / Build-and-test-development (pull_request) Successful in 32m22s
The boot screen lists the startup checks (app location, Viya deploy, startup
service). Add a fourth step that shows the contents of the <sasjs> tag, so the
effective configuration (serverType, debug, useComputeApi, ...) is visible
without having to read the page source. The step is marked as errored when the
tag is missing, which is the state that triggers the abort modal.

The step is appended after the existing three so the indices the startup flow
drives (sas.service / app.service) are unchanged. HTML lower-cases attribute
names in the DOM, so known attributes are mapped back to their canonical
spelling for display; unknown ones are kept as-is so a misspelt attribute is
still visible.
2026-10-15 11:56:07 +00:00
dc 7094bfdf7c fix(deploy): disable the manual deploy file inputs while admin groups load
Build / Build-and-ng-test (pull_request) Successful in 6m31s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m38s
Build / Build-and-test-development (pull_request) Successful in 33m0s
A file chosen before the identity calls settle runs onSasFileChange ->
addPrecodeLines while selectedAdminGroup is still '', baking '%let admin=;'
into the generated script; the action buttons then enable against that stale
header once the groups load. Disable both file inputs while
adminGroupsLoading, and cover it with a template spec that asserts the
binding (verified red without the guard).
2026-10-14 21:28:48 +00:00
dc fead3d5b26 test(getdata): wrap the RLS skip logic in a macro for sas9 portability
Build / Build-and-ng-test (pull_request) Successful in 5m41s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m46s
Build / Build-and-test-development (pull_request) Successful in 32m11s
The admin-skip conditional used open-code %if/%then/%do, which needs SAS
9.4M5+. The suite also runs on the sas9 target, where an older maintenance
release would fail to compile the test. Wrap the body in
%macro getdata_rls_test() ... %mend and invoke it, so the conditional sits
inside a macro definition (as testsetup.sas's update_settings does).
2026-10-14 21:13:56 +00:00
allan cd5bae7518 Merge pull request 'fix(getdata): apply row level security to selectbox source values' (#341) from fix/rls-selectbox-source into feat/export-dc-library-ddl
Build / Build-and-ng-test (pull_request) Successful in 5m50s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m4s
Build / Build-and-test-development (pull_request) Successful in 30m57s
Reviewed-on: #341
2026-10-14 15:57:47 +00:00
dc 78d436f847 test(getdata): derive RLS value from data and guard open-code return
Build / Build-and-ng-test (pull_request) Successful in 5m46s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m45s
Build / Build-and-test-development (pull_request) Successful in 30m47s
- The startupservice test leaves trailing %let dc_admin_group lines in the
  deployed settings.sas, so the skip guard compares against whatever the
  file resolves to; keep using &dc_admin_group (not &mpeadmins, which is
  never set in a test session).
- Selectbox RLS test now derives the restricted SOME_BESTNUM value from the
  data (testsetup reseeds MPE_X_TEST with random values) and asserts the
  user filter matches at least one value so the grid cannot be empty.
- Replace the open-code %return (invalid outside a macro definition) with
  an %if/%else %do guard around the demote + assert section.
2026-10-14 14:52:47 +00:00
dc 0d3d47500c test(getdata): make selectbox RLS test runnable on admin estates
Build / Build-and-ng-test (pull_request) Successful in 5m45s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m16s
Build / Build-and-test-development (pull_request) Successful in 30m50s
RLS is skipped for admins, and test sessions never resolve &mpeadmins
(testinit.sas runs without mpeinit), so the test checks the runner's
identity groups against &dc_admin_group instead. Admin runners skip
the assertions with an explicit PASS row; everyone else gets the real
RLS selectbox checks. Also documents the filter (row limit) and distinct
value preconditions.
2026-10-14 13:46:55 +00:00
dc 768d33c618 fix(getdata): apply row level security to selectbox source values
Build / Build-and-ng-test (pull_request) Successful in 5m55s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m12s
Build / Build-and-test-development (pull_request) Successful in 31m1s
The cell dropdown (selectbox) values sourced from a library.member.column
reference are read directly from the source table in editors/getdata.sas
(%dq_selects), without applying Row Level Security.  The grid rows are
filtered by %mpe_filtermaster, but the dropdown was not, so values from
restricted rows leaked into the dropdown.

This runs %mpe_filtermaster over the source table and reads the distinct
values through the filtered view.  EDIT mode mirrors the grid on this
(editor) page.  A data step view is used because a fileref cannot be %inc'd
inside a proc sql where clause.  The per-source view, filter table and
fileref are dropped at the end of each loop iteration.

getdata.test.2.sas restricts SOME_BESTNUM in MPE_X_TEST with a Row Level
Security rule (only the value 44 is visible) and asserts the selectbox
values returned by getdata are restricted to 44.  Without the fix the
dropdown returns the other distinct values and the test fails.  It is named
.test.2 because the row/cell limits test added on the target branch already
occupies .test.1.

Closes #340
2026-10-14 11:12:31 +00:00
dc 6ffbdbadf9 fix(deploy): disable the manual deploy actions without an admin group
Build / Build-and-ng-test (pull_request) Successful in 6m38s
Lighthouse Checks / lighthouse (pull_request) Successful in 23m22s
Build / Build-and-test-development (pull_request) Successful in 31m29s
The fallback added in b0e47fc names a group when the identity calls fail, but
the deploy should not proceed on a group the user never saw or chose.  The
three actions that consume it - the precode download, SUBMIT (executeSAS) and
Create Database - are now disabled while the group list is empty, so a failed
or empty identity call cannot produce a deploy script or a makedata run.  A
note next to the group select says why they are disabled.

The template spec asserts both states by rendering the real template.
2026-10-05 22:16:35 +00:00
dc b0e47fc4ed fix(deploy): never bake an empty admin group into the deploy script
Build / Build-and-ng-test (pull_request) Successful in 5m47s
Lighthouse Checks / lighthouse (pull_request) Successful in 23m8s
Build / Build-and-test-development (pull_request) Successful in 32m16s
When the identity calls fail the group lists stay empty and selectedAdminGroup
stayed '' - the manual section, unlike the automatic one, set no default - so
addPrecodeLines wrote `%let admin=;` and createDatabase carried an empty
ADMIN to makedata.

Settle the selection on both paths: applyAdminGroupFilter now also runs when
the calls fail, and falls back the way the automatic section does
(dcAdapterSettings.adminGroup, else SASAdministrators).  The filter previously
sat inside the try, so the failure path skipped it entirely.
2026-10-05 21:50:19 +00:00
dc 5693b4488b fix(deploy): apply the same identity-call fix to the automatic deploy
Build / Build-and-ng-test (pull_request) Successful in 5m54s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m18s
Build / Build-and-test-development (pull_request) Successful in 31m23s
The automatic deploy's getAdminGroups has the identical raw fetch() - no
credentials, no res.ok check, and no catch, so a 401 surfaced as an unhandled
rejection while the list stayed empty.  Same fix as the manual section; this
path predates the release branch.
2026-10-05 19:39:22 +00:00
dc 6522fcb28b fix(deploy): guard the manual deploy form and Viya identity calls
The "Show all groups" toggle was the only ngModel control in the form without
a name or ngModelOptions.standalone, so Angular threw NG01352 on render -
dev-mode only, so a production build hid it, but ng serve and the Cypress job
did not, and the form renders on every visit (needsLogin defaults to false).
Added both, matching the neighbouring controls.

Both Viya identity calls used raw fetch() with no credentials and no res.ok
check.  The deploy app can be served from a different origin to the Viya
server, where the default same-origin credential mode sends no session cookie
and Viya answers 401 - fetch still resolves, the body parses to no items, and
the screen silently rendered an empty group list that the generated deploy
script then carried as an empty admin value.  Passed credentials: 'include'
(matching withCredentials on the app's other Viya calls) and threw on a
non-ok response so the existing error path reports it.

Adds a spec that renders the real template through its module - the other
specs construct the class directly, so nothing ran the template directives,
which is why CI could not catch the ngModel guard.
2026-10-05 19:39:12 +00:00
dc 7cecf79515 fix(client): map the getcols fallback onto the picker vocabulary
Build / Build-and-ng-test (pull_request) Successful in 5m36s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m16s
Build / Build-and-test-development (pull_request) Successful in 31m26s
public/getcols publishes the SAS vocabulary (TYPE C/N, DDTYPE separating
DATE/DATETIME/TIME) but the picker compares TYPE exactly against num/char, so
the fallback list has to be mapped before it is published.  getCols does that
with the rule mergeColsRules uses on the normal path: the date and time types
are numeric.

Adds the missing spec for the store's fallback, covering the mapping, the
pass-through, and the empty-payload guard.
2026-10-05 15:04:15 +00:00
dc e90bcde512 fix(sas): keep the SAS C/N vocabulary in the public getcols response
The fallback rewrite changed the published TYPE from C/N to the client's
num/char, which is a breaking change to a public service.  TYPE goes back to
the SAS vocabulary it has always used, and the client maps it to its own
num/char at the store boundary instead (sas-store.getCols).  ddtype, fmtname,
format and label are unchanged, so the payload is a superset of the old one.

getcols.test now asserts the SAS contract (TYPE is C/N and agrees with ddtype)
rather than the client vocabulary, and the mock mirrors the service.
2026-10-05 15:04:04 +00:00
dc 9c32b1b1da fix(sas): keep format and label in the public getcols response
Build / Build-and-ng-test (pull_request) Successful in 6m20s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m42s
Build / Build-and-test-development (pull_request) Successful in 31m29s
The rewrite dropped format and label from the payload of this public service.
They are restored alongside the new fmtname/ddtype so an existing caller still
finds them, and the mock carries format too so it mirrors the service. TYPE
keeps the picker's num/char vocabulary - the picker compares it exactly, so the
old C/N cannot be kept - and that vocabulary change is now documented in the
service.
2026-10-05 12:59:00 +00:00
dc 9f8fc6f2eb fix(client): do not publish an empty getcols fallback to the picker
setQueryVariablesFromTable fell back to public/getcols, which can return an
empty cols list when the dataset has no metadata. Publishing that pushes []
into the filter picker, whose subscriber reads cols[0].NAME - the throw is
caught, but it leaves the picker half-updated (this.cols assigned, the
filter.cols assignment below the throwing line never run). Skip the store
update when the fallback is empty.
2026-10-05 12:58:47 +00:00
dc b034fe5335 fix(client): pin the dependency entries this release changes
The entries this PR bumps (@angular/animations, common, compiler, core,
forms, platform-browser, platform-browser-dynamic, router at 20.3.33;
@sasjs/adapter 4.19.1; @angular/compiler-cli 20.3.33) carried ^ ranges even
though .npmrc sets save-exact=true, so the manifest drifted on every manual
edit. Pinned them to the versions the lockfile resolves. @angular/cli is
pinned too (to 20.3.37) as the reviewer listed it. npm ci verified against
the lockfile afterwards.
2026-10-05 12:58:41 +00:00
dc 90af6b8515 fix(sas): run the row/cell limit migration as a Viya job
Build / Build-and-ng-test (pull_request) Successful in 6m26s
Lighthouse Checks / lighthouse (pull_request) Successful in 23m33s
Build / Build-and-test-development (pull_request) Successful in 32m38s
Each of the three blocks ended with a single-statement %else %put, which the
Viya compute engine rejects in open code (outside a %macro) with
'Expected %DO not found' - so the migration aborted whenever it was run as a
job. Both branches now use %do/%end. Verified by executing the script as a
real Viya job: all three options updated and inserted, no errors.
2026-10-05 12:25:50 +00:00
dc 99ff732d67 fix: repair the SAS suite for the row/cell limits and default the manual deploy admin group
Build / Build-and-ng-test (pull_request) Successful in 5m50s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m37s
Build / Build-and-test-development (pull_request) Successful in 32m29s
SAS tests:
- getcols.test: the new TYPE-vocabulary assertion had an unquoted comma in
  desc=, so the argument split and the whole test job aborted with ERROR 180.
  Quoted it with %str() - the test now runs (shown RED/GREEN against the
  length type $4 line it guards).
- getdata.test.1 (new): covers the EDIT row and cell limits. dctest.wideboy
  filtered to 300 rows trips DC_MAXOBS_WEBEDIT; filtered to 200 rows (200 x
  1005 = 201,000 cells) trips DC_MAXCELLS_WEBEDIT.
- getdata.test: test 1 fetched MPE_TABLES unfiltered, which now exceeds the
  250 row limit on a populated estate. Filtered it to one libref and moved
  the query assertion from test=NONE to test=ALL.
- exportdb.test: TSQL test now also asserts the export contains no INSERT.

Client:
- manual (Viya) deploy: the SAS Admin group dropdown was never populated
  (ngOnInit was empty). It now fetches the groups and defaults the list to
  the ones the user is a member of, with a 'Show all groups' toggle -
  mirroring the automatic deploy section. Adds three unit tests.
2026-10-05 12:05:34 +00:00
allan d5d28e92c8 Merge pull request 'feat(mocks): read the DC_MAX* options from MPE_CONFIG, and mock the DDL export' (#337) from feat/mock-config-and-ddl into feat/export-dc-library-ddl
Build / Build-and-ng-test (pull_request) Successful in 5m41s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m41s
Build / Build-and-test-development (pull_request) Successful in 31m28s
Reviewed-on: #337
2026-10-05 08:15:31 +00:00
dc 93ef9228aa test(getcols): assert the TYPE vocabulary in the SAS suite
Build / Build-and-ng-test (pull_request) Successful in 5m35s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m10s
Build / Build-and-test-development (pull_request) Successful in 31m32s
Review feedback on this PR.  The cypress cases exercise the mock, which derives
TYPE in JS and never passes through the $1 variable the length statement
widens, so dropping `length type $4;` again would pass everything CI runs.

getcols.test.sas already runs the service end to end, so its second case now
also counts the rows whose TYPE does not match their DDTYPE - 'char' for C,
'num' for N/DATE/DATETIME/TIME.  That covers the vocabulary and the mapping it
is derived from, so a truncated 'n'/'c' and a swapped mapping both fail it.
The suite runs via `sasjs test` against a Viya target, not in CI.

sasjs lint reports no warning on the file.
2026-10-05 01:08:39 +00:00
dc bcad05e032 fix(getcols): widen TYPE before assigning the client vocabulary
Build / Build-and-ng-test (pull_request) Successful in 5m30s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m53s
Build / Build-and-test-development (pull_request) Successful in 31m18s
Review feedback on this PR.  type comes from %mp_getcols as a $1 holding 'C' or
'N' - the macro assigns those with no length statement - and this step sets
cols1, so type inherited that $1.  Assigning 'num' and 'char' to it truncated
them to 'n' and 'c', and the picker compares the vocabulary exactly, so against
the real backend every filter built from the fallback list gave numeric columns
the character operator set and submitted character values unquoted, which
%mp_filtercheck rejects.

length type $4; now precedes the SET, which is the documented way to change an
input variable's length.  ddtype needs nothing: the macro creates it with
'CHARACTER', so it is already $9.

The sibling data steps are not affected - getdata.sas (vars3) and mpe_loader.sas
rename the incoming type to type2 first, so type is a new variable there and its
length comes from the first assignment ('char', 4), which holds both values.

sasjs lint reports no warning on this file.  The cypress case cannot cover this:
the mock sets TYPE in full in JS, and CI has no SAS engine to run the service.
2026-10-04 19:08:31 +00:00
dc dbc3044a0f ci(hooks): scan for secrets in CI, and add the pre-commit framework config
Build / Build-and-ng-test (pull_request) Successful in 5m44s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m18s
Build / Build-and-test-development (pull_request) Successful in 31m18s
Review feedback on this PR: with .npmrc setting ignore-scripts=true the prepare
script never runs, so the native hooks in .git-hooks/ do not activate on a fresh
clone and a commit can go unchecked.

Both standard routes are added, the way sasjs/cli and sasjs/lint carry them:

- .pre-commit-config.yaml pins the gitleaks hook to v8.30.1, so `pre-commit
  install` wires the secret scan without the one-time core.hooksPath command.
- Build-and-ng-test runs the same pinned @nogoo9/gitleaks binary as a step,
  because `pre-commit install` is itself a manual command: CI is the enforcement
  point.  --no-git scans the working tree, so the gate is what the pull request
  adds rather than the repo's history.

The scan runs before the job writes client/.npmrc from the NPMRC secret - that
file carries the private registry token, so scanning after it would flag the
token CI injects itself.

.npmrc is unchanged - ignore-scripts=true stays - and the native hooks stay for
anyone who prefers them.  CONTRIBUTING.md names the framework route next to the
existing one-time command.

Verified: npm ci followed by the scan reports no leaks on a fresh checkout, the
client lint check passes, and pre-commit validate-config plus pre-commit run
--all-files both pass.
2026-10-04 17:47:35 +00:00
allan 1fc7b55e3b Merge pull request 'fix(filters): source the picker variables from public/getcols when the table is not returned' (#338) from feat/filter-vars-fallback into feat/mock-config-and-ddl
Build / Build-and-ng-test (pull_request) Successful in 5m23s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m22s
Build / Build-and-test-development (pull_request) Successful in 31m35s
Reviewed-on: #338
2026-10-04 17:12:17 +00:00
dc bcb8fde986 fix(filters): carry TYPE on the fallback list, and drop the temporal exclusion
Build / Build-and-ng-test (pull_request) Successful in 5m29s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m59s
Build / Build-and-test-development (pull_request) Successful in 31m20s
Review feedback on this PR.

- The fallback list bypassed mergeColsRules, so col.TYPE was never set and every
  column reached the picker as char: char operators for numerics, and unquoted
  character values, which %mp_filtercheck rejects on the value list and on the
  submit alike.  getcols.sas and the mock now emit TYPE in the client's
  vocabulary - char for C, num for N/DATE/DATETIME/TIME - and the cypress case
  no longer stops at selecting the variable: it sets a value, submits, and
  asserts the applied clause reads SITE_NAME = 'Bristol', so the quoting is the
  assertion.
- The temporal exclusion is dropped, matching getdata.sas and viewdata.sas,
  whose cols payloads include those columns - the drop is on the data, not on
  the cols.  The mock's getdata.js cols had the same divergence and is aligned;
  it keeps the exclusion for the rows.  That also removes the SAS/mock
  disagreement on the excluded set, since there is no set.
- client/.npmrc added with ignore-scripts and save-exact (plus legacy-peer-deps
  and fund, matching the root), so a local `cd client && npm i` is guarded the
  way the CI's is.

Verified against the running mock: for MPE_CONFIG (TXTEMPORAL) getcols returns
TX_FROM/TX_TO with TYPE=num and DDTYPE=DATETIME, the char columns as TYPE=char,
and editors/getdata returns cols including both temporal columns while the rows
still exclude them.  row-cell-limits.cy.ts is 5/5.
2026-10-04 16:26:51 +00:00
58 changed files with 2424 additions and 1527 deletions

No files matched your search

+16
View File
@@ -15,6 +15,22 @@ jobs:
with:
node-version: ${{ env.NODE_VERSION }}
- name: Scan for secrets (gitleaks)
# The local pre-commit hook is suppressed entirely when .npmrc sets
# ignore-scripts=true (the prepare script that would activate the
# hooks never runs), so CI is the enforcement point: a leaked
# credential must fail the build, not just a commit.
# Uses the same @nogoo9/gitleaks binary the pre-commit hook pins - the
# gitleaks-action requires a license key for organization repos.
# Runs before client/.npmrc is written below, because that file is
# created from the NPMRC secret and a later scan would flag the
# registry token this job injects itself.
# --no-git scans the working tree, which is what a pull request gate
# is about.
run: |
npm ci
node_modules/.bin/gitleaks detect --no-git --source . --redact --no-banner -v
- name: Install Google Chrome
run: |
apt-get update
+5 -1
View File
@@ -380,8 +380,12 @@ jobs:
# ahead of the installation footer. The section text lives in a
# dedicated script below (.gitea/scripts/verify-section.sh) so this
# YAML scalar stays clean.
# `jq -r` is required: without it jq emits the result as a JSON string
# literal (quoted, with \n escapes), and the PATCH below then encodes
# that literal a second time, so the release body renders as one
# escaped line instead of markdown.
VERIFY_SECTION=$(./.gitea/scripts/verify-section.sh)
NEW_BODY=$(jq -n --arg body "$RELEASE_BODY" --arg verify "$VERIFY_SECTION" \
NEW_BODY=$(jq -nr --arg body "$RELEASE_BODY" --arg verify "$VERIFY_SECTION" \
'$body + "\n" + $verify + "\n\nFor installation instructions, please visit https://docs.datacontroller.io/"')
curl -k --fail-with-body -sS -X PATCH \
-H "$AUTH_HEADER" \
+16
View File
@@ -0,0 +1,16 @@
# Hooks for the pre-commit framework - https://pre-commit.com
#
# The repo also ships native hooks in .git-hooks/, covering the secret scan, a
# 2MB commit size limit, a prettier check and Conventional Commits. But .npmrc
# sets ignore-scripts=true, so the `prepare` script that points git at them
# never runs on `npm i` - a fresh clone commits unchecked until the one-time
# command in CONTRIBUTING.md is run by hand.
#
# This file is the standard route instead: install the framework once with
# `pre-commit install`, and the secret scan runs on every commit. The hook is
# pinned to an exact gitleaks release tag.
repos:
- repo: https://github.com/gitleaks/gitleaks
rev: v8.30.1
hooks:
- id: gitleaks
+62
View File
@@ -1,3 +1,65 @@
# [7.18.0](https://git.datacontroller.io/dc/dc/compare/v7.17.0...v7.18.0) (2026-10-19)
### Bug Fixes
* **deps:** bump @sasjs/core to 5.4.0 ([19fb032](https://git.datacontroller.io/dc/dc/commit/19fb032f0dbdd612651ca4ff3b55d1aefa4850d5))
* **deps:** pin @sasjs/utils and drop the unused jsrsasign ([05d2d1e](https://git.datacontroller.io/dc/dc/commit/05d2d1e08e2c3082f0dd5aa4629d91907c0401fc))
* **editor:** own the delete-record column client-side ([56025cc](https://git.datacontroller.io/dc/dc/commit/56025cc0524d34d3589992e9f1059c13fb5581ed))
* **editor:** surface the reserved-column refusal on every getdata path ([b26f557](https://git.datacontroller.io/dc/dc/commit/b26f5571b66b65166c1158817f2df3027ae45cdc))
* **filters:** do not turn a fractional temporal value into a literal ([f710d87](https://git.datacontroller.io/dc/dc/commit/f710d871fffc0fa451fd8cc4d037a26414725654))
* **filters:** fix the dynamic where clause deref and map its temporal values ([ece67d3](https://git.datacontroller.io/dc/dc/commit/ece67d37ba2c4e2037dea626b9730a692388f025))
* **filters:** keep the range field readable and cover a BETWEEN in the rewrite ([be79f3a](https://git.datacontroller.io/dc/dc/commit/be79f3a1ea5e4c4a157396d276f2c6654f615d39))
* **filters:** map an IN entry to its unformatted value before the literal ([8f98ebe](https://git.datacontroller.io/dc/dc/commit/8f98ebe326978b8da58d8ac895ec946e562b5de4))
* **filters:** stop the temporal rewrite corrupting a typed value, and bind the range dropdown to the unformatted value ([a5f61d4](https://git.datacontroller.io/dc/dc/commit/a5f61d4a73051237b671695f67122c237770c95a))
* **filters:** use the unformatted value in every IN branch ([cc01f54](https://git.datacontroller.io/dc/dc/commit/cc01f540c29d3d5f7dd86943b6eed82b332ce868))
* **makedata:** shorten the DC_MAXCELLS_WEBEDIT description to fit var_desc ([a51d9e5](https://git.datacontroller.io/dc/dc/commit/a51d9e5574383e5d1e1943c95261fbce63e8788d))
* **multi-dataset:** surface the reserved-column guard, use the name constant ([2b8a2fa](https://git.datacontroller.io/dc/dc/commit/2b8a2fa1039da3ab888af8c3ffae7e1b848b250a))
* **release:** emit the release body as raw text, not a JSON string ([a05554e](https://git.datacontroller.io/dc/dc/commit/a05554e67a8d44e32e2d9f5e4a959a9d0592be5e))
### Features
* **filters:** show dates as date literals in the filter query ([662d000](https://git.datacontroller.io/dc/dc/commit/662d00032c27634bbc2d82c392e7875a2e174eff))
# [7.17.0](https://git.datacontroller.io/dc/dc/compare/v7.16.0...v7.17.0) (2026-10-15)
### Bug Fixes
* **client:** do not publish an empty getcols fallback to the picker ([9f8fc6f](https://git.datacontroller.io/dc/dc/commit/9f8fc6f2eb83c78ea9cb220bb74059bab39e66ad))
* **client:** map the getcols fallback onto the picker vocabulary ([7cecf79](https://git.datacontroller.io/dc/dc/commit/7cecf795153138d6f2ad1f87f7bdd12b5d861341))
* **client:** pin the dependency entries this release changes ([b034fe5](https://git.datacontroller.io/dc/dc/commit/b034fe5335c41ec0289e459270839ee7caa8e126))
* **deploy:** apply the same identity-call fix to the automatic deploy ([5693b44](https://git.datacontroller.io/dc/dc/commit/5693b4488b0328022c69dd34f98662d660612578))
* **deploy:** disable the manual deploy actions without an admin group ([6ffbdba](https://git.datacontroller.io/dc/dc/commit/6ffbdbadf9804e0128694ed1c2bdb8f957743ad9))
* **deploy:** disable the manual deploy file inputs while admin groups load ([7094bfd](https://git.datacontroller.io/dc/dc/commit/7094bfdf7cdab4c70e5f1a7763a8a8aeb8cc61a7))
* **deploy:** guard the manual deploy form and Viya identity calls ([6522fcb](https://git.datacontroller.io/dc/dc/commit/6522fcb28b67b910b2784b93251f591bbc79d294))
* **deploy:** never bake an empty admin group into the deploy script ([b0e47fc](https://git.datacontroller.io/dc/dc/commit/b0e47fc4ed4e36878de6505e678667b14d19c5e6))
* **deps:** pin shelljs to clear the braces advisory in the sas audit ([6e50938](https://git.datacontroller.io/dc/dc/commit/6e5093858be106a1c3545a16119d246f69aac3da))
* **editor:** check the EDIT limits once, and skip switched-off options ([f08c4c2](https://git.datacontroller.io/dc/dc/commit/f08c4c231f019979a27fe71fe73798752c015710))
* **filters:** carry TYPE on the fallback list, and drop the temporal exclusion ([bcb8fde](https://git.datacontroller.io/dc/dc/commit/bcb8fde9868b8e1ded1590e204c34054f62d8250))
* **filters:** source the picker variables from public/getcols when the table is not returned ([3a6a216](https://git.datacontroller.io/dc/dc/commit/3a6a216b9d4dfc462e05031039c47002a9156130))
* **getcols:** use %global and %mp_getcols, and merge the base branch ([1385741](https://git.datacontroller.io/dc/dc/commit/1385741f2d303c1432f1f0ba637dd3cb39500a71))
* **getcols:** widen TYPE before assigning the client vocabulary ([bcad05e](https://git.datacontroller.io/dc/dc/commit/bcad05e03291fc3e2b72a4465629e1e330d70c8a))
* **getdata:** apply row level security to selectbox source values ([768d33c](https://git.datacontroller.io/dc/dc/commit/768d33c618a7ea453c3c19723f342a170ec576ea)), closes [#340](https://git.datacontroller.io/dc/dc/issues/340)
* **mocks:** read MPE_CONFIG correctly, and pin the limits to it ([05126ae](https://git.datacontroller.io/dc/dc/commit/05126ae73a1ab2461066eb701b7475674b96f1ec))
* repair the SAS suite for the row/cell limits and default the manual deploy admin group ([99ff732](https://git.datacontroller.io/dc/dc/commit/99ff732d67425b01a8a8f5e3e7c98b3e6d604323))
* **requests-modal:** widen the dialog so log lines need less scrolling ([3c10d91](https://git.datacontroller.io/dc/dc/commit/3c10d915a5134247aff0e987dd4d5c83c197e4c5))
* **sas:** keep format and label in the public getcols response ([9c32b1b](https://git.datacontroller.io/dc/dc/commit/9c32b1b1dae00e4b7bfd4acad94cbed831819d84))
* **sas:** keep the SAS C/N vocabulary in the public getcols response ([e90bcde](https://git.datacontroller.io/dc/dc/commit/e90bcde51218c9e5f562f7aaa65506761b21d6bd))
* **sas:** run the row/cell limit migration as a Viya job ([90af6b8](https://git.datacontroller.io/dc/dc/commit/90af6b851542a97f79172b655070cda559386a5c))
* **startup:** stack the SASjs tag detail so the dump is readable ([1f4c4a4](https://git.datacontroller.io/dc/dc/commit/1f4c4a4d8297568f46bcc16ea208673fedad345b))
### Features
* **debug:** keep the debug toggle state across a page refresh ([3ce006d](https://git.datacontroller.io/dc/dc/commit/3ce006d6b469b4ece2bc769f0a32a1792eb1e774))
* **editor:** raise the row limits and add a cell limit to the EDIT screen ([3de6655](https://git.datacontroller.io/dc/dc/commit/3de6655f26b7e895f60266e9931b46cfc0053975))
* **mocks:** read the DC_MAX* options from MPE_CONFIG, and mock the DDL export ([edac9a9](https://git.datacontroller.io/dc/dc/commit/edac9a987aad3e4217641eed43f697f61c4bf739))
* **startup:** call out a missing SASjs tag on the boot screen ([b24eb73](https://git.datacontroller.io/dc/dc/commit/b24eb731ce089acf496bed1e11a5e34208c1bd11))
* **startup:** dump the SASjs tag contents in the boot screen checks ([75c1dcc](https://git.datacontroller.io/dc/dc/commit/75c1dccfd23557ab321cd7abd1508f20d817c7b8))
* **system:** export the DC library DDL from the system screen ([05f3f49](https://git.datacontroller.io/dc/dc/commit/05f3f49e0c4d7005fa1f93852d73f2368395ab28)), closes [#179](https://git.datacontroller.io/dc/dc/issues/179)
# [7.16.0](https://git.datacontroller.io/dc/dc/compare/v7.15.0...v7.16.0) (2026-09-28)
+7 -8
View File
@@ -19,6 +19,8 @@ git config core.hooksPath ./.git-hooks
The pre-commit hook requires the gitleaks binary provided by the root `@nogoo9/gitleaks` devDependency, so make sure `npm i` has run in the repo root before your first commit.
The standard [pre-commit](https://pre-commit.com) route is available as well, and does not depend on the lifecycle scripts above: install it once with `pre-commit install` and the secret scan in [`.pre-commit-config.yaml`](./.pre-commit-config.yaml) runs on every commit. The same scan also runs in CI, so a leaked credential fails the build regardless of which local hooks are active.
## Dependencies that requires licences
[SheetJS Pro Version](https://www.npmjs.com/package/sheetjs)
@@ -48,15 +50,12 @@ npm i
[Handsontable](https://www.npmjs.com/package/handsontable)
Licence should be inserted in the `client/src/index.html` file:
The licence key comes from the backend licence: the decrypted licence carries `hot_license_key`,
which is published on the `hot_license_key` subject and pushed into every Handsontable instance.
Until a licence is loaded the client falls back to the non-commercial key
(`DEFAULT_HOT_LICENCE_KEY` in `client/src/app/services/licence.service.ts`), so there is nothing to
configure in `index.html` for this.
```
<sasjs
...
hotLicenceKey="LICENCE-GOES-HERE"
>
</sasjs>
```
## Development
Update `client/src/index.html` so that it points to your SAS9, SASVIYA or SASJS backend.
Be aware that VIYA can be configured in such way that it would not work with cross origin frontend.
@@ -25,11 +25,13 @@ const libraryToOpenIncludes = Cypress.env(`libraryToOpenIncludes_${serverType}`)
// 2. Ordinary columns (e.g. SOME_DROPDOWN) still get their rule via the
// new per-row cols[].COLTYPE path — test 2 proves the migration didn't
// silently drop real columns' rules either.
// Column positions below come from COLHEADERS in
// sas/mocks/sasjs/services/editors/getdata.js:
// "_____DELETE__THIS__RECORD_____,PRIMARY_KEY_FIELD,SOME_CHAR,SOME_DROPDOWN,..."
// — childNodes[0] on a body <tr> is the row-header <th>, so childNodes[N+1]
// is the Nth (0-indexed) column in that list.
// Column positions below come from the mock's COLHEADERS
// (sas/mocks/sasjs/services/editors/getdata.js):
// "PRIMARY_KEY_FIELD,SOME_CHAR,SOME_DROPDOWN,..."
// — the client-owned delete column is prepended by the editor itself (see
// deleteRecordColumn.ts), so on screen it is still column 0. childNodes[0] on
// a body <tr> is the row-header <th>, so childNodes[N+1] is the Nth
// (0-indexed) column in that list.
context('coltype vertical-array regression tests: ', function () {
this.beforeAll(() => {
cy.visit(`${hostUrl}/SASLogon/logout`)
+144 -19
View File
@@ -11,6 +11,18 @@ context('filtering tests: ', function () {
this.beforeAll(() => {
cy.visit(`${hostUrl}/SASLogon/logout`, { timeout: longerCommandTimeout })
cy.loginAndUpdateValidKey()
/**
* The evaluation agreement is per-origin localStorage, so a fresh browser
* profile lands on it and the nav tree never renders. Guarded, so it is a
* no-op once an earlier spec in the run has accepted it.
*/
cy.get('body').then(($body) => {
if ($body.find('#TCS input[type="checkbox"]').length) {
cy.get('#TCS input[type="checkbox"]').check({ force: true })
cy.wait(4000)
}
})
})
this.beforeEach(() => {
@@ -51,9 +63,13 @@ context('filtering tests: ', function () {
openFilterPopup(() => {
setFilterWithValue('SOME_TIME', '00:00:42', 'time', () => {
checkInfoBarIncludes(`(( SOME_TIME = 42 ))`, (includes: boolean) => {
if (includes) done()
})
checkInfoBarIncludes(
`(( SOME_TIME = '00:00:42't ))`,
(includes: boolean) => {
if (!includes) return
assertGridRowCount(3, () => done())
}
)
})
})
})
@@ -63,9 +79,13 @@ context('filtering tests: ', function () {
openFilterPopup(() => {
setFilterWithValue('SOME_TIME', '42', 'value', () => {
checkInfoBarIncludes(`(( SOME_TIME = 42 ))`, (includes: boolean) => {
if (includes) done()
})
checkInfoBarIncludes(
`(( SOME_TIME = '00:00:42't ))`,
(includes: boolean) => {
if (!includes) return
assertGridRowCount(3, () => done())
}
)
})
}, false)
})
@@ -75,9 +95,13 @@ context('filtering tests: ', function () {
openFilterPopup(() => {
setFilterWithValue('SOME_DATE', '12/02/1960', 'date', () => {
checkInfoBarIncludes(`(( SOME_DATE = 42 ))`, (includes: boolean) => {
if (includes) done()
})
checkInfoBarIncludes(
`(( SOME_DATE = '12FEB1960'd ))`,
(includes: boolean) => {
if (!includes) return
assertGridRowCount(5, () => done())
}
)
})
})
})
@@ -87,9 +111,13 @@ context('filtering tests: ', function () {
openFilterPopup(() => {
setFilterWithValue('SOME_DATE', '42', 'value', () => {
checkInfoBarIncludes(`(( SOME_DATE = 42 ))`, (includes: boolean) => {
if (includes) done()
})
checkInfoBarIncludes(
`(( SOME_DATE = '12FEB1960'd ))`,
(includes: boolean) => {
if (!includes) return
assertGridRowCount(5, () => done())
}
)
})
}, false)
})
@@ -104,9 +132,10 @@ context('filtering tests: ', function () {
'datetime',
() => {
checkInfoBarIncludes(
`(( SOME_DATETIME = 42 ))`,
`(( SOME_DATETIME = '01JAN1960:00:00:42'dt ))`,
(includes: boolean) => {
if (includes) done()
if (!includes) return
assertGridRowCount(5, () => done())
}
)
}
@@ -120,9 +149,10 @@ context('filtering tests: ', function () {
openFilterPopup(() => {
setFilterWithValue('SOME_DATETIME', '42', 'value', () => {
checkInfoBarIncludes(
`(( SOME_DATETIME = 42 ))`,
`(( SOME_DATETIME = '01JAN1960:00:00:42'dt ))`,
(includes: boolean) => {
if (includes) done()
if (!includes) return
assertGridRowCount(5, () => done())
}
)
})
@@ -134,9 +164,13 @@ context('filtering tests: ', function () {
openFilterPopup(() => {
setFilterWithValue('SOME_DATE', '', 'in', () => {
checkInfoBarIncludes(`(( SOME_DATE IN (42) ))`, (includes: boolean) => {
if (includes) done()
})
checkInfoBarIncludes(
`(( SOME_DATE IN ('12FEB1960'd) ))`,
(includes: boolean) => {
if (!includes) return
assertGridRowCount(5, () => done())
}
)
})
})
})
@@ -156,6 +190,34 @@ context('filtering tests: ', function () {
// })
// })
// })
/**
* With the pickers off, a temporal BETWEEN takes both of its values from a
* soft-select over the column's values, so the value is chosen from that list
* rather than typed. The range dropdown binds the formatted label, as the
* single-value dropdown's list does, so the field reads the date and the
* clause composition maps it back to the internal day / second count before
* the literal is built - a display string that reached sasTemporalLiteral as
* a non-number would come back null and be passed through to RAW_VALUE
* unquoted, which mp_filtercheck rejects on a numeric column and aborts the
* save. The test asserts the applied clause and the row count, so it pins
* the mapping end to end.
*/
it('7 | Non picker - filter date field BETWEEN from the value list', (done) => {
openTableFromTree(libraryToOpenIncludes, 'mpe_x_test')
openFilterPopup(() => {
setTemporalRangeFromValueList('SOME_DATE', '12FEB1960', () => {
checkInfoBarIncludes(
`BETWEEN '12FEB1960'd AND '12FEB1960'd`,
(includes: boolean) => {
if (!includes) return
assertGridRowCount(5, () => done())
}
)
})
}, false)
})
})
const checkInfoBarIncludes = (text: string, callback: any) => {
@@ -166,6 +228,21 @@ const checkInfoBarIncludes = (text: string, callback: any) => {
})
}
/**
* The temporal filter tests assert the applied filter still selects the same
* rows as the internal value did - a readable clause that filtered differently
* would be a worse bug than the one being fixed.
*/
const assertGridRowCount = (expected: number, callback: any) => {
cy.get('#hotTable .ht_master tbody tr', {
timeout: longerCommandTimeout
})
.should('have.length', expected)
.then(() => {
if (callback) callback()
})
}
const openFilterPopup = (
callback?: any,
usePickers: boolean = true,
@@ -331,6 +408,54 @@ const setFilterWithValue = (
if (callback) callback()
}
/**
* Drives a temporal BETWEEN with the pickers off, choosing both values from the
* range soft-select's list rather than typing them - typing bypasses the
* binding this test exists to pin.
*/
const setTemporalRangeFromValueList = (
variableValue: string,
formatted: string,
callback?: any
) => {
cy.wait(600)
cy.focused().type(variableValue)
cy.wait(100)
cy.get('.variable-col .autocomplete-wrapper', { withinSubject: null })
.first()
.trigger('keydown', { key: 'ArrowDown' })
cy.get('.variable-col .autocomplete-wrapper', {
withinSubject: null
}).trigger('keydown', { key: 'Enter' })
cy.focused().tab()
cy.wait(100)
cy.focused().select('BETWEEN').trigger('change')
cy.wait(400)
const chooseFromList = (range: 'start' | 'end') => {
const inputId = `vals-${range}_0_0`
cy.get(`#${inputId}`)
.clear({ force: true })
.type(formatted, { force: true })
cy.get(`#datalist_${inputId} option`)
.contains(formatted)
.click({ force: true })
cy.get(`#${inputId}`).trigger('keyup', { key: 'Escape', force: true })
cy.wait(300)
}
chooseFromList('start')
chooseFromList('end')
cy.contains('.filter-modal button', 'Ok').click()
cy.get('.app-loading', { timeout: longerCommandTimeout }).should('not.exist')
if (callback) callback()
}
const openTableFromTree = (libNameIncludes: string, tablename: string) => {
cy.get('.app-loading', { timeout: longerCommandTimeout })
.should('not.exist')
+25
View File
@@ -157,6 +157,18 @@ context('EDIT row and cell limits, VIEW row limit: ', function () {
// and the suggestion list is usable, not just populated
setFilterVariable('SITE_NAME')
// Submitting is what exercises TYPE. The picker keys its operator set and
// its value quoting on it, and the fallback list never passes through
// mergeColsRules, so without TYPE on this payload a character value would be
// submitted unquoted and the backend's %mp_filtercheck would reject it.
setFilterValue('Bristol')
submitFilter()
cy.get('.infoBar-text', { timeout: longerCommandTimeout }).should(
'contain.text',
"SITE_NAME = 'Bristol'"
)
})
})
@@ -207,6 +219,19 @@ const setFilterVariable = (column: string) => {
cy.get('#vals_var_id0_0').trigger('keyup', { key: 'Escape', force: true })
}
/** The value input of the same soft-select row. */
const setFilterValue = (value: string) => {
cy.get('#vals_0_0').clear({ force: true }).type(value, { force: true })
cy.get('#vals_0_0').trigger('keyup', { key: 'Escape', force: true })
cy.wait(400)
}
const submitFilter = () => {
cy.contains('.filter-modal button', 'Ok').click()
cy.get('.app-loading', { timeout: longerCommandTimeout }).should('not.exist')
cy.wait(2500)
}
/**
* Sets a DC option in MPE_CONFIG through the editor and approves it, then hands
* back to the callback. The same route licensing.cy.ts uses for the licence
+72 -183
View File
@@ -7,22 +7,22 @@
"name": "data_controller-client",
"hasInstallScript": true,
"dependencies": {
"@angular/animations": "^20.3.33",
"@angular/animations": "20.3.33",
"@angular/cdk": "^20.2.14",
"@angular/common": "^20.3.33",
"@angular/compiler": "^20.3.33",
"@angular/core": "^20.3.33",
"@angular/forms": "^20.3.33",
"@angular/platform-browser": "^20.3.33",
"@angular/platform-browser-dynamic": "^20.3.33",
"@angular/router": "^20.3.33",
"@angular/common": "20.3.33",
"@angular/compiler": "20.3.33",
"@angular/core": "20.3.33",
"@angular/forms": "20.3.33",
"@angular/platform-browser": "20.3.33",
"@angular/platform-browser-dynamic": "20.3.33",
"@angular/router": "20.3.33",
"@cds/core": "^6.15.1",
"@clr/angular": "file:libraries/clr-angular-17.9.0.tgz",
"@clr/icons": "^13.0.2",
"@clr/ui": "file:libraries/clr-ui-17.9.0.tgz",
"@handsontable/angular-wrapper": "18.1.1",
"@sasjs/adapter": "^4.19.1",
"@sasjs/utils": "^3.6.0",
"@sasjs/adapter": "5.1.0",
"@sasjs/utils": "4.0.0",
"@sheet/crypto": "file:libraries/sheet-crypto.tgz",
"@types/d3-graphviz": "^2.6.7",
"@types/text-encoding": "0.0.35",
@@ -40,7 +40,6 @@
"hyperformula": "^2.5.0",
"iconv-lite": "^0.5.0",
"jquery-datetimepicker": "^2.5.21",
"jsrsasign": "11.1.1",
"marked": "^5.0.0",
"moment": "^2.30.1",
"ngx-clipboard": "^16.0.0",
@@ -65,8 +64,8 @@
"@angular-eslint/eslint-plugin-template": "19.8.1",
"@angular-eslint/schematics": "19.8.1",
"@angular-eslint/template-parser": "19.8.1",
"@angular/cli": "^20.3.32",
"@angular/compiler-cli": "^20.3.33",
"@angular/cli": "20.3.37",
"@angular/compiler-cli": "20.3.33",
"@babel/plugin-proposal-private-methods": "^7.18.6",
"@compodoc/compodoc": "^2.0.0",
"@cypress/webpack-preprocessor": "^5.17.1",
@@ -4561,19 +4560,6 @@
"node": ">= 14.17.0"
}
},
"node_modules/@cypress/request/node_modules/tough-cookie": {
"version": "5.1.2",
"resolved": "https://registry.npmjs.org/tough-cookie/-/tough-cookie-5.1.2.tgz",
"integrity": "sha512-FVDYdxtnj0G6Qm/DhNPSb8Ju59ULcup3tuJxkFb5K8Bv2pUXILbf0xZWU8PX8Ov19OXljbUyveOFwRMwkXzO+A==",
"dev": true,
"license": "BSD-3-Clause",
"dependencies": {
"tldts": "^6.1.32"
},
"engines": {
"node": ">=16"
}
},
"node_modules/@cypress/webpack-preprocessor": {
"version": "5.17.1",
"resolved": "https://registry.npmjs.org/@cypress/webpack-preprocessor/-/webpack-preprocessor-5.17.1.tgz",
@@ -8866,42 +8852,53 @@
]
},
"node_modules/@sasjs/adapter": {
"version": "4.19.1",
"resolved": "https://registry.npmjs.org/@sasjs/adapter/-/adapter-4.19.1.tgz",
"integrity": "sha512-IpN6UPSdqVb5cAWEBt+jG/i0rhgCubdyp5kxdouuX3ifL313oJW2ZPVUNUvs0+ZGzsXWcGDTNoEzzTrGKiqJlQ==",
"version": "5.1.0",
"resolved": "https://registry.npmjs.org/@sasjs/adapter/-/adapter-5.1.0.tgz",
"integrity": "sha512-l5ETQY2SqxitgVEoQ/LAFPo0wUdoHD5zKazpZ4f63Sr2ws80UE+mmmV85v2KYBO5lkyENOrnVkE61xbjEm7OXg==",
"license": "ISC",
"dependencies": {
"@sasjs/utils": "^3.6.0",
"@sasjs/utils": "4.0.0",
"axios": "1.20.0",
"axios-cookiejar-support": "5.0.5",
"form-data": "4.0.6",
"https": "1.0.0",
"tough-cookie": "4.1.3"
"tough-cookie": "5.1.2"
},
"engines": {
"node": ">=22"
}
},
"node_modules/@sasjs/utils": {
"version": "3.6.2",
"resolved": "https://registry.npmjs.org/@sasjs/utils/-/utils-3.6.2.tgz",
"integrity": "sha512-AfC5OJuJx3NAbntKq2J8E8NNpRXthYfuqOyp2lOmOkTctL3mAfrA54U/cdPQvrlftXNxpK9P2U0RU10qNjlEzQ==",
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/@sasjs/utils/-/utils-4.0.0.tgz",
"integrity": "sha512-lARoCnRM4gSVf8ShoP5elHi7W28K/14MK037dQ0ABgMlP3+r6OuhAZYwK/a70tED47tKxyIoCAKLdP6jrG0ewQ==",
"license": "ISC",
"dependencies": {
"@fast-csv/format": "4.3.5",
"@fast-csv/format": "5.0.7",
"@types/fs-extra": "11.0.4",
"@types/prompts": "2.0.13",
"@types/prompts": "2.4.9",
"chalk": "4.1.1",
"cli-table": "0.3.6",
"consola": "2.15.0",
"find": "0.3.0",
"fs-extra": "11.3.0",
"jwt-decode": "3.1.2",
"prompts": "2.4.1",
"valid-url": "1.0.9"
"consola": "3.4.2",
"fs-extra": "11.4.1",
"jwt-decode": "4.0.0",
"prompts": "2.4.2"
},
"engines": {
"node": ">=22"
}
},
"node_modules/@sasjs/utils/node_modules/@fast-csv/format": {
"version": "5.0.7",
"resolved": "https://registry.npmjs.org/@fast-csv/format/-/format-5.0.7.tgz",
"integrity": "sha512-VdypoRxv7PF+LsyPouTMKdB0d76hync+gLpgdNqfqVK44MsgW4oiCJSdrki2FisWT7v2QGUYDHjp4L7w5oO6gw==",
"license": "MIT",
"dependencies": {
"lodash.escaperegexp": "^4.1.2"
}
},
"node_modules/@sasjs/utils/node_modules/fs-extra": {
"version": "11.3.0",
"resolved": "https://registry.npmjs.org/fs-extra/-/fs-extra-11.3.0.tgz",
"integrity": "sha512-Z4XaCL6dUDHfP/jT25jJKMmtxvuwbkrD1vNSMFlo9lNLY2c5FHYSQgHPRZUjAB26TpDEoW9HCOgplrdbaPV/ew==",
"version": "11.4.1",
"resolved": "https://registry.npmjs.org/fs-extra/-/fs-extra-11.4.1.tgz",
"integrity": "sha512-KYAb4c9BJQI6QqGKthV68OHe0badztdXJWKo0WtBA9IuCFPTKvE5ZdUBglP833aMjhaSPNO4A5j/EkzZtGlKjA==",
"license": "MIT",
"dependencies": {
"graceful-fs": "^4.2.0",
@@ -9647,12 +9644,13 @@
}
},
"node_modules/@types/prompts": {
"version": "2.0.13",
"resolved": "https://registry.npmjs.org/@types/prompts/-/prompts-2.0.13.tgz",
"integrity": "sha512-jwMOIGy49VruR/gYehhJYgpVzB+EVpEE7t7j9m1oTo4HMpOe7KmsyqdBuoxAzA5B4caUgx0cKrWr7wUEqMXJ7Q==",
"version": "2.4.9",
"resolved": "https://registry.npmjs.org/@types/prompts/-/prompts-2.4.9.tgz",
"integrity": "sha512-qTxFi6Buiu8+50/+3DGIWLHM6QuWsEKugJnnP6iv2Mc4ncxE4A/OJkjuVOA+5X0X1S/nq5VJRa8Lu+nwcvbrKA==",
"license": "MIT",
"dependencies": {
"@types/node": "*"
"@types/node": "*",
"kleur": "^3.0.3"
}
},
"node_modules/@types/qs": {
@@ -12187,17 +12185,6 @@
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/cli-table": {
"version": "0.3.6",
"resolved": "https://registry.npmjs.org/cli-table/-/cli-table-0.3.6.tgz",
"integrity": "sha512-ZkNZbnZjKERTY5NwC2SeMeLeifSPq/pubeRoTpdr3WchLlnZg6hEgvHkK5zL7KNFdd9PmHN8lxrENUwI3cE8vQ==",
"dependencies": {
"colors": "1.0.3"
},
"engines": {
"node": ">= 0.2.0"
}
},
"node_modules/cli-table3": {
"version": "0.6.1",
"resolved": "https://registry.npmjs.org/cli-table3/-/cli-table3-0.6.1.tgz",
@@ -12447,15 +12434,6 @@
"dev": true,
"license": "MIT"
},
"node_modules/colors": {
"version": "1.0.3",
"resolved": "https://registry.npmjs.org/colors/-/colors-1.0.3.tgz",
"integrity": "sha512-pFGrxThWcWQ2MsAz6RtgeWe4NK2kUE1WfsrvvlctdII745EW9I0yflqhe7++M5LEc7bV2c/9/5zc8sFcpL0Drw==",
"license": "MIT",
"engines": {
"node": ">=0.1.90"
}
},
"node_modules/combined-stream": {
"version": "1.0.8",
"resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz",
@@ -12737,10 +12715,13 @@
}
},
"node_modules/consola": {
"version": "2.15.0",
"resolved": "https://registry.npmjs.org/consola/-/consola-2.15.0.tgz",
"integrity": "sha512-vlcSGgdYS26mPf7qNi+dCisbhiyDnrN1zaRbw3CSuc2wGOMEGGPsp46PdRG5gqXwgtJfjxDkxRNAgRPr1B77vQ==",
"license": "MIT"
"version": "3.4.2",
"resolved": "https://registry.npmjs.org/consola/-/consola-3.4.2.tgz",
"integrity": "sha512-5IKcdX0nnYavi6G7TtOhwkYzyjfJlatbjMjuLSfE2kYT5pMDOilZ4OvMhi637CcDICTmz3wARPoyhqyX1Y+XvA==",
"license": "MIT",
"engines": {
"node": "^14.18.0 || >=16.10.0"
}
},
"node_modules/content-disposition": {
"version": "0.5.4",
@@ -15371,15 +15352,6 @@
"dev": true,
"license": "MIT"
},
"node_modules/find": {
"version": "0.3.0",
"resolved": "https://registry.npmjs.org/find/-/find-0.3.0.tgz",
"integrity": "sha512-iSd+O4OEYV/I36Zl8MdYJO0xD82wH528SaCieTVHhclgiYNe9y+yPKSwK+A7/WsmHL1EZ+pYUJBXWTL5qofksw==",
"license": "MIT",
"dependencies": {
"traverse-chain": "~0.1.0"
}
},
"node_modules/find-cache-dir": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/find-cache-dir/-/find-cache-dir-4.0.0.tgz",
@@ -16404,12 +16376,6 @@
"node": ">=0.10"
}
},
"node_modules/https": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/https/-/https-1.0.0.tgz",
"integrity": "sha512-4EC57ddXrkaF0x83Oj8sM6SLQHAWXw90Skqu2M4AEWENZ3F02dFJE/GARA8igO79tcgYqGrD7ae4f5L3um2lgg==",
"license": "ISC"
},
"node_modules/https-browserify": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/https-browserify/-/https-browserify-1.0.0.tgz",
@@ -17592,16 +17558,6 @@
"verror": "1.10.0"
}
},
"node_modules/jsrsasign": {
"version": "11.1.1",
"resolved": "https://registry.npmjs.org/jsrsasign/-/jsrsasign-11.1.1.tgz",
"integrity": "sha512-6w95OOXH8DNeGxakqLndBEqqwQ6A70zGaky1oxfg8WVLWOnghTfJsc5Tknx+Z88MHSb1bGLcqQHImOF8Lk22XA==",
"deprecated": "This package is no longer maintained.",
"license": "MIT",
"funding": {
"url": "https://github.com/kjur/jsrsasign#donations"
}
},
"node_modules/jszip": {
"version": "3.10.1",
"resolved": "https://registry.npmjs.org/jszip/-/jszip-3.10.1.tgz",
@@ -17651,10 +17607,13 @@
}
},
"node_modules/jwt-decode": {
"version": "3.1.2",
"resolved": "https://registry.npmjs.org/jwt-decode/-/jwt-decode-3.1.2.tgz",
"integrity": "sha512-UfpWE/VZn0iP50d8cz9NrZLM9lSWhcJ+0Gt/nm4by88UL+J1SiKN8/5dkjMmbEzwL2CAe+67GsegCbIKtbp75A==",
"license": "MIT"
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/jwt-decode/-/jwt-decode-4.0.0.tgz",
"integrity": "sha512-+KJGIyHgkGuIq3IEBNftfhW/LfWhXUIY6OmyVWjliu5KH1y0fw7VQ8YndE2O4qZdMSd9SqbnC8GOcZEy0Om7sA==",
"license": "MIT",
"engines": {
"node": ">=18"
}
},
"node_modules/karma": {
"version": "6.4.4",
@@ -22185,9 +22144,9 @@
"license": "MIT"
},
"node_modules/prompts": {
"version": "2.4.1",
"resolved": "https://registry.npmjs.org/prompts/-/prompts-2.4.1.tgz",
"integrity": "sha512-EQyfIuO2hPDsX1L/blblV+H7I0knhgAd82cVneCwcdND9B8AuCDuRcBH6yIcG4dFzlOUqbazQqwGjx5xmsNLuQ==",
"version": "2.4.2",
"resolved": "https://registry.npmjs.org/prompts/-/prompts-2.4.2.tgz",
"integrity": "sha512-NxNv/kLguCA7p3jE8oL2aEBsrJWgAakBpgmgK6lpPWV+WuOmY6r2/zbAVnP+T8bQlA0nzHXSJSJW0Hq7ylaD2Q==",
"license": "MIT",
"dependencies": {
"kleur": "^3.0.3",
@@ -22291,27 +22250,6 @@
"license": "MIT",
"optional": true
},
"node_modules/psl": {
"version": "1.15.0",
"resolved": "https://registry.npmjs.org/psl/-/psl-1.15.0.tgz",
"integrity": "sha512-JZd3gMVBAVQkSs6HdNZo9Sdo0LNcQeMNP3CozBJb3JYC/QUYZTnKxP+f8oWRX4rHP5EurWxqAHTSwUCjlNKa1w==",
"license": "MIT",
"dependencies": {
"punycode": "^2.3.1"
},
"funding": {
"url": "https://github.com/sponsors/lupomontero"
}
},
"node_modules/psl/node_modules/punycode": {
"version": "2.3.1",
"resolved": "https://registry.npmjs.org/punycode/-/punycode-2.3.1.tgz",
"integrity": "sha512-vYt7UD1U9Wg6138shLtLOvdAu+8DsC/ilFtEVHcH+wydcSpNE20AfSOduf6MkRFahL5FY7X1oU7nKVZFtfq8Fg==",
"license": "MIT",
"engines": {
"node": ">=6"
}
},
"node_modules/pump": {
"version": "3.0.4",
"resolved": "https://registry.npmjs.org/pump/-/pump-3.0.4.tgz",
@@ -22453,12 +22391,6 @@
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/querystringify": {
"version": "2.2.0",
"resolved": "https://registry.npmjs.org/querystringify/-/querystringify-2.2.0.tgz",
"integrity": "sha512-FIqgj2EUvTa7R50u0rGsyTftzjYmv/a3hO345bZNrqabNqjtgiDMgmo4mkUjd+nzU5oF3dClKqFIPUKybUyqoQ==",
"license": "MIT"
},
"node_modules/queue-microtask": {
"version": "1.2.3",
"resolved": "https://registry.npmjs.org/queue-microtask/-/queue-microtask-1.2.3.tgz",
@@ -22893,6 +22825,7 @@
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/requires-port/-/requires-port-1.0.0.tgz",
"integrity": "sha512-KigOCHcocU3XODJxsu8i/j8T9tzT4adHiecwORRQ0ZZFcp7ahwXuRU1m+yuO90C5ZUyGeGfocHDI14M3L3yDAQ==",
"dev": true,
"license": "MIT"
},
"node_modules/resolve": {
@@ -24735,7 +24668,6 @@
"version": "6.1.86",
"resolved": "https://registry.npmjs.org/tldts/-/tldts-6.1.86.tgz",
"integrity": "sha512-WMi/OQ2axVTf/ykqCQgXiIct+mSQDFdH2fkwhPwgEwvJ1kSzZRiinb0zF2Xb8u4+OqPChmyI6MEu4EezNJz+FQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"tldts-core": "^6.1.86"
@@ -24765,7 +24697,6 @@
"version": "6.1.86",
"resolved": "https://registry.npmjs.org/tldts-core/-/tldts-core-6.1.86.tgz",
"integrity": "sha512-Je6p7pkk+KMzMv2XXKmAE3McmolOQFdxkKw0R8EYNr7sELW46JqnNeTX8ybPiQgvg1ymCoF8LXs5fzFaZvJPTA==",
"dev": true,
"license": "MIT"
},
"node_modules/tmp": {
@@ -24843,36 +24774,15 @@
}
},
"node_modules/tough-cookie": {
"version": "4.1.3",
"resolved": "https://registry.npmjs.org/tough-cookie/-/tough-cookie-4.1.3.tgz",
"integrity": "sha512-aX/y5pVRkfRnfmuX+OdbSdXvPe6ieKX/G2s7e98f4poJHnqH3281gDPm/metm6E/WRamfx7WC4HUqkWHfQHprw==",
"version": "5.1.2",
"resolved": "https://registry.npmjs.org/tough-cookie/-/tough-cookie-5.1.2.tgz",
"integrity": "sha512-FVDYdxtnj0G6Qm/DhNPSb8Ju59ULcup3tuJxkFb5K8Bv2pUXILbf0xZWU8PX8Ov19OXljbUyveOFwRMwkXzO+A==",
"license": "BSD-3-Clause",
"dependencies": {
"psl": "^1.1.33",
"punycode": "^2.1.1",
"universalify": "^0.2.0",
"url-parse": "^1.5.3"
"tldts": "^6.1.32"
},
"engines": {
"node": ">=6"
}
},
"node_modules/tough-cookie/node_modules/punycode": {
"version": "2.3.1",
"resolved": "https://registry.npmjs.org/punycode/-/punycode-2.3.1.tgz",
"integrity": "sha512-vYt7UD1U9Wg6138shLtLOvdAu+8DsC/ilFtEVHcH+wydcSpNE20AfSOduf6MkRFahL5FY7X1oU7nKVZFtfq8Fg==",
"license": "MIT",
"engines": {
"node": ">=6"
}
},
"node_modules/tough-cookie/node_modules/universalify": {
"version": "0.2.0",
"resolved": "https://registry.npmjs.org/universalify/-/universalify-0.2.0.tgz",
"integrity": "sha512-CJ1QgKmNg3CwvAv/kOFmtnEN05f0D/cn9QntgNOQlQF9dgvVTHj3t+8JPdjqawCHk7V/KA+fbUqzZ9XWhcqPUg==",
"license": "MIT",
"engines": {
"node": ">= 4.0.0"
"node": ">=16"
}
},
"node_modules/tr46": {
@@ -24882,12 +24792,6 @@
"dev": true,
"license": "MIT"
},
"node_modules/traverse-chain": {
"version": "0.1.0",
"resolved": "https://registry.npmjs.org/traverse-chain/-/traverse-chain-0.1.0.tgz",
"integrity": "sha512-up6Yvai4PYKhpNp5PkYtx50m3KbwQrqDwbuZP/ItyL64YEWHAvH6Md83LFLV/GRSk/BoUVwwgUzX6SOQSbsfAg==",
"license": "MIT"
},
"node_modules/tree-dump": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/tree-dump/-/tree-dump-1.1.0.tgz",
@@ -25519,16 +25423,6 @@
"browserslist": ">= 4.21.0"
}
},
"node_modules/url-parse": {
"version": "1.5.10",
"resolved": "https://registry.npmjs.org/url-parse/-/url-parse-1.5.10.tgz",
"integrity": "sha512-WypcfiRhfeUP9vvF0j6rw0J3hrWrw6iZv3+22h6iRMJ/8z1Tj6XfLP4DsUix5MhMPnXpiHDoKyoZ/bdCkwBCiQ==",
"license": "MIT",
"dependencies": {
"querystringify": "^2.1.1",
"requires-port": "^1.0.0"
}
},
"node_modules/util-deprecate": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz",
@@ -25578,11 +25472,6 @@
"node": ">= 0.10"
}
},
"node_modules/valid-url": {
"version": "1.0.9",
"resolved": "https://registry.npmjs.org/valid-url/-/valid-url-1.0.9.tgz",
"integrity": "sha512-QQDsV8OnSf5Uc30CKSwG9lnhMPe6exHtTXLRYX8uMwKENy640pU+2BgBL0LRbDh/eYRahNCS7aewCx0wf3NYVA=="
},
"node_modules/validate-npm-package-license": {
"version": "3.0.4",
"resolved": "https://registry.npmjs.org/validate-npm-package-license/-/validate-npm-package-license-3.0.4.tgz",
+12 -13
View File
@@ -41,22 +41,22 @@
},
"private": true,
"dependencies": {
"@angular/animations": "^20.3.33",
"@angular/animations": "20.3.33",
"@angular/cdk": "^20.2.14",
"@angular/common": "^20.3.33",
"@angular/compiler": "^20.3.33",
"@angular/core": "^20.3.33",
"@angular/forms": "^20.3.33",
"@angular/platform-browser": "^20.3.33",
"@angular/platform-browser-dynamic": "^20.3.33",
"@angular/router": "^20.3.33",
"@angular/common": "20.3.33",
"@angular/compiler": "20.3.33",
"@angular/core": "20.3.33",
"@angular/forms": "20.3.33",
"@angular/platform-browser": "20.3.33",
"@angular/platform-browser-dynamic": "20.3.33",
"@angular/router": "20.3.33",
"@cds/core": "^6.15.1",
"@clr/angular": "file:libraries/clr-angular-17.9.0.tgz",
"@clr/icons": "^13.0.2",
"@clr/ui": "file:libraries/clr-ui-17.9.0.tgz",
"@handsontable/angular-wrapper": "18.1.1",
"@sasjs/adapter": "^4.19.1",
"@sasjs/utils": "^3.6.0",
"@sasjs/adapter": "5.1.0",
"@sasjs/utils": "4.0.0",
"@sheet/crypto": "file:libraries/sheet-crypto.tgz",
"@types/d3-graphviz": "^2.6.7",
"@types/text-encoding": "0.0.35",
@@ -74,7 +74,6 @@
"hyperformula": "^2.5.0",
"iconv-lite": "^0.5.0",
"jquery-datetimepicker": "^2.5.21",
"jsrsasign": "11.1.1",
"marked": "^5.0.0",
"moment": "^2.30.1",
"ngx-clipboard": "^16.0.0",
@@ -99,8 +98,8 @@
"@angular-eslint/eslint-plugin-template": "19.8.1",
"@angular-eslint/schematics": "19.8.1",
"@angular-eslint/template-parser": "19.8.1",
"@angular/cli": "^20.3.32",
"@angular/compiler-cli": "^20.3.33",
"@angular/cli": "20.3.37",
"@angular/compiler-cli": "20.3.33",
"@babel/plugin-proposal-private-methods": "^7.18.6",
"@compodoc/compodoc": "^2.0.0",
"@cypress/webpack-preprocessor": "^5.17.1",
+13
View File
@@ -318,6 +318,7 @@
class="startup-check"
[class.startup-check--error]="step.status === 'error'"
[class.startup-check--done]="step.status === 'done'"
[class.startup-check--grid]="step.details?.length"
>
@switch (step.status) {
@case ('pending') {
@@ -346,6 +347,18 @@
@if (step.detail) {
<span class="startup-check__detail">{{ step.detail }}</span>
}
@if (step.details?.length) {
<div class="startup-check__grid">
@for (detail of step.details; track detail.name) {
<span class="startup-check__grid-name">{{
detail.name
}}</span>
<span class="startup-check__grid-value">{{
detail.value
}}</span>
}
</div>
}
</div>
}
</div>
+24 -17
View File
@@ -20,6 +20,8 @@ import { DcAdapterSettings } from './models/DcAdapterSettings'
import { AppStoreService } from './services/app-store.service'
import { LicenceService } from './services/licence.service'
import {
getSasjsTagDetails,
SASJS_TAG_STEP,
StartupCheckService,
StartupStep
} from './services/startup-check.service'
@@ -190,40 +192,45 @@ export class AppComponent {
* Parses adapter settings that are found in the <sasjs> tag inside index.html
*/
private parseDcAdapterSettings() {
/* The <sasjs> tag is a mandatory part of index.html - it carries serverUrl,
serverType and the rest of the adapter config - so there is no config to
fall back on when it is absent. A missing one is reported on the boot
screen below rather than handled with an abort modal. */
const sasjsElement = document.querySelector('sasjs')
if (!sasjsElement) {
this.licenceService.deactivateApp()
setTimeout(() => {
this.eventService.showAbortModal(
null,
"Please make sure 'SASJS' tag with config attributes is added to index.html",
null,
'SASjs Config not found'
)
})
return
}
const getAppAttribute = (attribute: string) =>
sasjsElement.getAttribute(attribute) || undefined
sasjsElement?.getAttribute(attribute) || undefined
const dcAdapterSettings = {
serverUrl: getAppAttribute('serverUrl') || '',
appLoc: getAppAttribute('appLoc') || '',
serverType: getAppAttribute('serverType'),
loginMechanism: getAppAttribute('loginMechanism') || '',
adminGroup: getAppAttribute('adminGroup') || '',
dcPath: getAppAttribute('dcPath') || '',
debug: getAppAttribute('debug') === 'true' || false,
useComputeApi: this.parseComputeApi(getAppAttribute('useComputeApi')),
runAsTask: getAppAttribute('runAsTask') === 'true' || false,
contextName: getAppAttribute('contextName') || '',
hotLicenceKey: getAppAttribute('hotLicenceKey') || ''
contextName: getAppAttribute('contextName') || ''
}
this.dcAdapterSettings = dcAdapterSettings as any
this.appStoreService.setDcAdapterSettings(dcAdapterSettings as any)
/* Show the <sasjs> tag contents on the boot screen, so the effective
configuration (serverType, debug, useComputeApi, ...) is visible without
having to read the page source. The tag is mandatory - without it the
adapter has no serverUrl or serverType - so a missing one is called out
plainly rather than silently leaving the step pending. */
if (sasjsElement) {
this.startupCheckService.setTagDetails(getSasjsTagDetails(sasjsElement))
} else {
this.startupCheckService.updateStep(
SASJS_TAG_STEP,
'error',
'SASJS TAG MISSING!!'
)
}
this.appService.sasServiceInit()
}
@@ -2,10 +2,7 @@ import { NgModule } from '@angular/core'
import { RouterModule, Routes } from '@angular/router'
import { DeployComponent } from './deploy.component'
const routes: Routes = [
{ path: '', component: DeployComponent },
{ path: 'manualdeploy', component: DeployComponent }
]
const routes: Routes = [{ path: '', component: DeployComponent }]
@NgModule({
imports: [RouterModule.forChild(routes)],
+11 -37
View File
@@ -29,20 +29,6 @@
></label
>
</clr-checkbox-wrapper>
<!-- <hr />
<div class="clr-checkbox-wrapper">
<input
[(ngModel)]="autodeploy"
type="checkbox"
id="checkbox2"
class="clr-checkbox"
checked
/>
<label for="checkbox2"
>Autodeploy
{{ !jsonFile ? '(json file is not available)' : '' }}</label
>
</div> -->
</div>
</div>
}
@@ -52,29 +38,17 @@
@if (step > 0) {
@switch (true) {
@case (sasJsConfig.serverType === ServerType.SasViya) {
@if (autodeploy) {
<div class="autodeploy-section card">
<app-automatic-deploy
[sasJs]="sasJs"
[sasJsConfig]="sasJsConfig"
[dcAdapterSettings]="dcAdapterSettings"
[appLoc]="appLoc"
[dcPath]="dcPath"
[selectedAdminGroup]="selectedAdminGroup"
(onNavigateToHome)="onNavigateToHome()"
></app-automatic-deploy>
</div>
}
@if (!autodeploy) {
<div id="mainbody" class="card">
<app-manual-deploy
[sasJs]="sasJs"
[sasJsConfig]="sasJsConfig"
[dcAdapterSettings]="dcAdapterSettings"
(onNavigateToHome)="onNavigateToHome()"
></app-manual-deploy>
</div>
}
<div class="autodeploy-section card">
<app-automatic-deploy
[sasJs]="sasJs"
[sasJsConfig]="sasJsConfig"
[dcAdapterSettings]="dcAdapterSettings"
[appLoc]="appLoc"
[dcPath]="dcPath"
[selectedAdminGroup]="selectedAdminGroup"
(onNavigateToHome)="onNavigateToHome()"
></app-automatic-deploy>
</div>
}
@case (sasJsConfig.serverType === ServerType.Sasjs) {
<div class="autodeploy-section card">
+2 -52
View File
@@ -1,8 +1,6 @@
import { Component, OnInit, ViewEncapsulation } from '@angular/core'
import { SasService } from '../services/sas.service'
import { SASjsConfig } from '@sasjs/adapter'
import { Router } from '@angular/router'
import { LoggerService } from '../services/logger.service'
import { ServerType } from '@sasjs/utils/types/serverType'
import { AppStoreService } from '../services/app-store.service'
import { DcAdapterSettings } from '../models/DcAdapterSettings'
@@ -19,15 +17,11 @@ import { DcAdapterSettings } from '../models/DcAdapterSettings'
})
export class DeployComponent implements OnInit {
public step: number = 0
public adminGroups: any = []
public appLoc: string = ''
public dcPath: string = ''
public selectedAdminGroup: string = ''
public autodeploy: boolean = true
public jsonFile: any = null
public sasJs: any
public sasJsConfig: SASjsConfig = new SASjsConfig()
@@ -37,16 +31,10 @@ export class DeployComponent implements OnInit {
constructor(
private appStoreService: AppStoreService,
private sasService: SasService,
private loggerService: LoggerService,
private router: Router
private sasService: SasService
) {
this.dcAdapterSettings = this.appStoreService.getDcAdapterSettings()
if (this.router.url.includes('manualdeploy')) {
this.autodeploy = false
}
this.sasJs = this.sasService.getSasjsInstance()
this.sasJsConfig = this.sasService.getSasjsConfig()
this.appLoc = this.dcAdapterSettings?.appLoc || ''
@@ -61,54 +49,16 @@ export class DeployComponent implements OnInit {
*/
public setDeployDefaults() {
this.dcPath = this.dcAdapterSettings?.dcPath || ''
this.selectedAdminGroup = this.dcAdapterSettings?.adminGroup || ''
if (!this.selectedAdminGroup) {
this.selectedAdminGroup = 'SASAdministrators'
}
this.selectedAdminGroup = 'SASAdministrators'
}
/**
* Accepting terms of service shows next screen
*/
public termsAgreeChange() {
if (!this.autodeploy) {
this.getAdminGroups()
}
this.step++
}
/**
* Fetches admin groups from VIYA to be selected for a backend deploy
*/
public getAdminGroups() {
fetch(
this.sasJsConfig.serverUrl + '/identities/groups?sortBy=name&limit=5000',
{
headers: {
Accept: 'application/json'
}
}
)
.then((res: any) => {
return res.text()
})
.then((res: any) => {
let jsonRes
try {
jsonRes = JSON.parse(res)
} catch (err: any) {
console.error(err)
}
if (jsonRes) {
this.adminGroups = jsonRes.items
this.selectedAdminGroup = this.adminGroups[0].id
}
})
}
public onNavigateToHome() {
window.open(location.href.split('#')[0], '_blank')
}
-2
View File
@@ -2,7 +2,6 @@ import { NgModule } from '@angular/core'
import { CommonModule } from '@angular/common'
import { DeployComponent } from './deploy.component'
import { AutomaticComponent } from './sections/automatic/automatic.component'
import { ManualComponent } from './sections/manual/manual.component'
import { SasjsConfiguratorComponent } from './sections/sasjs-configurator/sasjs-configurator.component'
import { ClarityModule } from '@clr/angular'
import { FormsModule } from '@angular/forms'
@@ -12,7 +11,6 @@ import { DeployRoutingModule } from './deploy-routing.module'
declarations: [
DeployComponent,
AutomaticComponent,
ManualComponent,
SasjsConfiguratorComponent
],
imports: [CommonModule, FormsModule, ClarityModule, DeployRoutingModule]
@@ -1,229 +0,0 @@
<div class="card-header position-relative">
Configurator
<p class="d-inline-block ml-10 mb-10 mt-0">App Location: {{ appLoc }}</p>
<button
(click)="deleteKeys()"
class="btn btn-primary clear-memory-button position-absolute"
>
Clear memory
</button>
</div>
<div class="card-block">
<!-- <div class="card-title">
Client Details
</div> -->
@if (needsLogin) {
<div id="loginForm" class="d-none">
<p class="mb-10">Please log in first</p>
<label for="username" class="clr-control-label">Username</label>
<div class="mb-10 clr-control-container">
<div class="clr-input-wrapper">
<input type="text" id="username" class="clr-input" />
</div>
</div>
<label for="password" class="clr-control-label">Password</label>
<div class="mb-10 clr-control-container">
<div class="clr-input-wrapper">
<input type="password" id="password" class="clr-input" />
</div>
</div>
<button class="btn btn-primary d-none" id="loginBtn">Log in</button>
</div>
}
@if (!needsLogin) {
<form>
<div class="clr-form-control">
<label for="select-full" class="clr-control-label">Admin group</label>
<div class="clr-control-container">
<div class="clr-select-wrapper">
<select
[(ngModel)]="selectedAdminGroup"
[ngModelOptions]="{ standalone: true }"
name="selectedAdminGroup"
id="adminGroupsSelect"
class="clr-select"
>
@for (adminGroup of adminGroups; track adminGroup) {
<option [value]="adminGroup.id">
{{ adminGroup.name }}
</option>
}
</select>
</div>
</div>
</div>
<div class="clr-form-control">
<div [class.hidden]="contextsLoading">
<label for="select-full" class="clr-control-label">Context</label>
<div class="clr-control-container">
<div class="clr-select-wrapper">
<select
[(ngModel)]="selectedContext"
[ngModelOptions]="{ standalone: true }"
name="selectedContext"
class="clr-select"
>
@for (context of allContexts; track context) {
<option [value]="context.name">
{{ context.name }}
@if ((context.attributes | json) != '{}') {
<span>( {{ context.attributes.sysUserId }} )</span>
}
</option>
}
</select>
</div>
</div>
<button
(click)="executableContext()"
type="button"
class="btn btn-icon"
id="contexts-btn"
>
<clr-icon shape="play"></clr-icon>
</button>
</div>
<div [class.hidden]="!contextsLoading" class="d-flex">
<span class="spinner spinner-inline mr-10">
Loading contexts...
</span>
<span> Loading contexts... </span>
</div>
<label for="dcloc" class="mt-20 clr-control-label">DC Loc</label>
<div class="mb-10 clr-control-container">
<div class="clr-input-wrapper">
<input
[(ngModel)]="dcPath"
[ngModelOptions]="{ standalone: true }"
(focusout)="saveDcPath()"
name="dcPath"
type="text"
id="dcloc"
class="clr-input"
/>
</div>
</div>
<p class="mt-10">
Select JSON file to upload (json build file preloaded):
</p>
<div class="d-flex flex-column">
<input
type="file"
(click)="clearUploadInput($event)"
(change)="onJsonFileChange($event)"
/>
<!-- <button *ngIf="downloadFileBtn" (click)="downloadSasPrecodeFile()" style="width: 40px; min-width: 0;" class="btn btn-sm btn-icon">
<clr-icon shape="download"></clr-icon>
</button> -->
</div>
<div class="mt-20 d-flex align-items-center">
<button
(click)="executeJson()"
class="btn btn-primary"
[clrLoading]="isSubmittingJson"
id=""
class="mt-0"
>
SUBMIT JSON
</button>
@if (isJsonSubmitted) {
<span>JSON Submitted Successfully</span>
}
<!-- <span *ngIf="executingScript" class="spinner spinner-inline ml-3">
Loading...
</span> -->
</div>
<p class="mt-10">Select SAS file to upload:</p>
<div class="d-flex flex-column">
<input
type="file"
(click)="clearUploadInput($event)"
(change)="onSasFileChange($event); downloadFileBtn = true"
/>
@if (downloadFileBtn) {
<button
(click)="downloadSasPrecodeFile()"
class="btn btn-sm btn-icon min-w-0 w-40"
>
<clr-icon shape="download"></clr-icon>
</button>
}
</div>
<div class="mt-20 d-flex align-items-center">
<button
(click)="executeSAS()"
class="btn btn-primary"
[clrLoading]="executingScript"
id="deploy"
class="mt-0"
>
SUBMIT
</button>
<!-- <span *ngIf="executingScript" class="spinner spinner-inline ml-3">
Loading...
</span> -->
</div>
</div>
@if (jobLog.length > 0) {
<p class="mb-0 mt-10">File execute completed</p>
<hr />
<div>
<button
(click)="downloadFile(jobLog, 'execute-script-log', 'txt')"
class="btn btn-primary mt-0 mr-20"
>
Download log
</button>
</div>
}
<button
[clrLoading]="createDatabaseLoading"
(click)="createDatabase()"
class="btn btn-primary mt-10"
>
Create Database
</button>
@if (makeDataResponse.length > 0) {
<p class="mb-0 mt-10">Create Database Completed</p>
<hr />
@if (makeDataResponse.length > 0) {
<div class="log-wrapper">
{{ makeDataResponse }}
</div>
}
<button (click)="navigateToHome()" class="btn btn-primary mt-15">
Let's get started
</button>
<button
(click)="downloadFile(makeDataResponse, 'create-database-log', 'txt')"
class="btn btn-primary mt-15"
>
Download log
</button>
<button (click)="validateDeploy()" class="btn btn-primary mt-15">
Validate
</button>
@if (validationState !== 'none' || isValidating) {
<div class="validation-bar">
@if (isValidating) {
<span class="spinner spinner-inline mr-10">
Validating deploy...
</span>
<span> Validating deploy... </span>
}
@if (!isValidating && validationState === 'error') {
<clr-icon shape="exclamation-circle" class="is-error"></clr-icon>
<span> Validation failed </span>
}
@if (!isValidating && validationState === 'success') {
<clr-icon shape="check-circle" class="is-success"></clr-icon>
<span> Validation succeeded </span>
}
</div>
}
}
</form>
}
</div>
Whitespace-only changes.
@@ -1,90 +0,0 @@
import { ManualComponent } from './manual.component'
// validateDeploy() is fire-and-forget (doesn't return its own promise
// chain), so tests need to wait for it to settle some other way. A
// macrotask boundary guarantees every already-queued microtask (regardless
// of how many .then() hops validateDeploy's chain has) has run first.
const flushPromiseChain = () => new Promise((resolve) => setTimeout(resolve, 0))
/**
* validateDeploy() only touches sasService.request, eventService.showInfoModal
* and loggerService.log - these stubs cover exactly that surface, not the
* full real services (no TestBed/DI needed, same plain-instantiation-with-
* stubs precedent as app.service.spec.ts).
*/
const buildDeps = () => {
const sasService: any = {
request: jasmine.createSpy('request')
}
const eventService: any = {
showInfoModal: jasmine.createSpy('showInfoModal')
}
const loggerService: any = {
log: jasmine.createSpy('log')
}
const deployService: any = {}
return { sasService, eventService, loggerService, deployService }
}
const buildManualComponent = (deps: ReturnType<typeof buildDeps>) =>
new ManualComponent(
deps.sasService,
deps.eventService,
deps.loggerService,
deps.deployService
)
describe('ManualComponent - validateDeploy', () => {
it('sets validationState to success when saslibs is present on a normal object response', async () => {
const deps = buildDeps()
deps.sasService.request.and.resolveTo({
adapterResponse: { saslibs: { SOME_LIB: ['SOME_TABLE'] } }
})
const component = buildManualComponent(deps)
component.validateDeploy()
await flushPromiseChain()
expect(component.validationState).toBe('success')
expect(deps.eventService.showInfoModal).not.toHaveBeenCalled()
})
it('sets validationState to error with no modal when saslibs is missing from an otherwise normal object response', async () => {
const deps = buildDeps()
deps.sasService.request.and.resolveTo({
adapterResponse: { SYSSITE: 'SITE1' }
// saslibs deliberately omitted
})
const component = buildManualComponent(deps)
component.validateDeploy()
await flushPromiseChain()
expect(component.validationState).toBe('error')
expect(deps.eventService.showInfoModal).not.toHaveBeenCalled()
})
// Same underlying bug as app.service.spec.ts's equivalent test: a
// misconfigured Viya computeTasks deployment returns a raw "Job error"
// text body, which the adapter resolves as adapterResponse verbatim.
it('sets validationState to error and shows the real response text when adapterResponse is a raw string, not an object', async () => {
const deps = buildDeps()
const rawJobError = [
'Job error',
'The Compute service could not execute the task c74a707a-8b88-46a5-8f28-9a7694ad5e13 because the context 340cd3eb-72ae is not reusable. Please provide a reusable context or supply the ID of an existing session in the task request.',
'path: /compute/tasks',
'correlator: dc1a9fda-8f65-4f32-b4d4-2a70e02179b1;1bcb6a4c-8a16-4053-a790-292544c25a03'
].join('\n')
deps.sasService.request.and.resolveTo({ adapterResponse: rawJobError })
const component = buildManualComponent(deps)
component.validateDeploy()
await flushPromiseChain()
expect(component.validationState).toBe('error')
expect(deps.eventService.showInfoModal).toHaveBeenCalledTimes(1)
const [, message] = deps.eventService.showInfoModal.calls.mostRecent().args
expect(message).toContain(rawJobError)
})
})
@@ -1,346 +0,0 @@
import {
Component,
Input,
OnInit,
Output,
EventEmitter,
ViewEncapsulation
} from '@angular/core'
import SASjs, { SASjsConfig } from '@sasjs/adapter'
import { DcAdapterSettings } from 'src/app/models/DcAdapterSettings'
import { RequestWrapperResponse } from 'src/app/models/request-wrapper/RequestWrapperResponse'
import { DeployService } from 'src/app/services/deploy.service'
import { EventService } from 'src/app/services/event.service'
import { LoggerService } from 'src/app/services/logger.service'
import { SasService } from 'src/app/services/sas.service'
import { getMalformedAdapterResponseMessage } from 'src/app/shared/utils/get-malformed-adapter-response-message'
@Component({
selector: 'app-manual-deploy',
templateUrl: './manual.component.html',
styleUrls: ['./manual.component.scss'],
encapsulation: ViewEncapsulation.None,
standalone: false
})
export class ManualComponent implements OnInit {
@Input() sasJs!: SASjs
@Input() sasJsConfig: SASjsConfig = new SASjsConfig()
@Input() dcAdapterSettings: DcAdapterSettings | undefined
@Output() onNavigateToHome: EventEmitter<any> = new EventEmitter<any>()
public needsLogin: boolean = false
public adminGroups: any = []
public allContexts: any = []
public appLoc: string = ''
public dcPath: string = ''
public selectedAdminGroup: string = ''
public selectedContext: string = ''
public jobLog: string = ''
public makeDataResponse: string = ''
public linesOfCode: any = []
public fileName: string = ''
public preloadedFile: boolean = true
public executeSASEnabled: boolean = false
public contextsLoading: boolean = false
public createDatabaseLoading: boolean = false
public executingScript: boolean = false
public downloadFileBtn: boolean = false
public isValidating: boolean = false
public jsonFile: any = null
public isSubmittingJson: boolean = false
public isJsonSubmitted: boolean = false
public validationState: string = 'none'
constructor(
private sasService: SasService,
private eventService: EventService,
private loggerService: LoggerService,
private deployService: DeployService
) {}
ngOnInit(): void {}
/**
* FIXME: Remove
*/
public async executableContext() {
// getExecutableContexts now need AuthConfig parameter which we don't have on web
// this.contextsLoading = true
// let contexts = await this.sasJs.getExecutableContexts()
// this.allContexts = contexts
// if (contexts.length === 0) {
// this.selectedContext = ''
// } else {
// this.selectedContext = this.allContexts[0].name
// }
// this.contextsLoading = false
}
/**
* Removes sas.json file attached to the input
*/
public clearUploadInput(event: Event) {
this.deployService.clearUploadInput(event)
}
/**
* Reads attached SAS file to be sent to sas for execution (backend deploy)
*/
public onSasFileChange(event: any) {
this.preloadedFile = false
let file = event.target.files[0]
this.fileName = file.name
let fileReader = new FileReader()
fileReader.onload = () => {
if (fileReader.result) {
this.linesOfCode = (fileReader.result as string).split('\n')
this.linesOfCode = this.linesOfCode.filter((el: string) => {
return el !== '' && el !== null
})
this.executeSASEnabled = true
this.addPrecodeLines()
}
}
fileReader.readAsText(file)
}
/**
* Reads attached JSON file to be sent to sas for execution (backend deploy)
*/
public async onJsonFileChange(event: any) {
let file = event.target.files[0]
this.jsonFile = await this.deployService.readFile(file)
}
/**
* Appending precode lines to the attached sas or json file for backend deploy
*/
public addPrecodeLines() {
let headerLines = [
`%let context=${this.selectedContext};`,
`%let appLoc=${this.appLoc};`,
`%let admin=${this.selectedAdminGroup};`,
`%let dcpath=${this.dcPath};`
]
this.linesOfCode.unshift(...headerLines)
}
/**
* Downloadng file with precode included
*/
public downloadSasPrecodeFile() {
let linesAsText = this.linesOfCode.join('\n')
let filename = this.fileName.split('.')[0]
this.downloadFile(linesAsText, filename, 'sas')
}
/**
* Used for downloading log and repsonse as a file
*/
public downloadFile(
content: any,
filename: string,
extension: string = 'txt'
) {
this.deployService.downloadFile(content, filename, extension)
}
/**
* Saving dcpath to localstorage
* FIXME: maybe it'snot necessary
*/
public saveDcPath() {
localStorage.setItem('deploy_dc_loc', this.dcPath)
}
/**
* Send sas.json to be executed (deploying backend)
*/
public async executeJson() {
this.isSubmittingJson = true
try {
let uploadJsonFile = await this.sasJs.deployServicePack(
this.jsonFile,
this.dcAdapterSettings?.appLoc || '',
undefined,
undefined,
true
)
this.isJsonSubmitted = true
} catch (ex: any) {
let textEx = ''
if (typeof ex.message !== 'string') {
textEx = JSON.stringify(ex).replace(/\\/gm, '')
} else {
textEx = ex.message
}
this.eventService.showInfoModal(
'Deploy error',
`Exception: \n ${textEx !== '' ? textEx : ex}`
)
return
}
this.isSubmittingJson = false
}
/**
* Send sas file to be executed (deploying backend)
*/
public async executeSAS() {
this.executingScript = true
this.jobLog = ''
this.makeDataResponse = ''
try {
let executeResponse = await this.sasJs.executeScript({
fileName: this.fileName,
linesOfCode: this.linesOfCode,
contextName: this.selectedContext
})
this.loggerService.log(executeResponse)
if (typeof executeResponse.log === 'string') {
executeResponse.log = JSON.parse(executeResponse.log)
}
if (executeResponse.jobStatus === 'error') {
alert('Error!')
} else {
this.jobLog = executeResponse.log.items
? executeResponse.log.items.map((i: any) => i.line).join('\n')
: JSON.stringify(executeResponse.log)
}
this.executingScript = false
} catch (err) {
this.executingScript = false
}
}
/**
* Running makedata service
* @param newTab open and run in new tab
*/
public createDatabase(newTab: boolean = true) {
if (newTab) {
// Debug is ALWAYS enabled for makedata — it runs only once during
// deployment and the full log is needed to diagnose any issues.
const _debug = '&_debug=131'
let url =
this.sasService.getSasjsConfig().serverUrl +
'/SASJobExecution/?_program=' +
this.dcAdapterSettings?.appLoc ||
'' +
'/admin/makedata' +
'&ADMIN=' +
this.selectedAdminGroup +
'&DCPATH=' +
this.dcPath +
_debug
window.open(url, '_blank')
return
}
this.createDatabaseLoading = true
let data = {
fromjs: [
{
ADMIN: this.selectedAdminGroup,
DCPATH: this.dcPath
}
]
}
/**
* We are overriding default `sasjsConfig` object fields with this object fields.
* Here we want to run this request using original WEB method.
* contextName: null is the MUST field for it.
*/
let overrideConfig = {
useComputeApi: null,
contextName: this.sasJsConfig.contextName,
debug: true
}
this.sasJs
.request(`services/admin/makedata`, data, overrideConfig, () => {
this.sasService.shouldLogin.next(true)
})
.then((res: any) => {
try {
this.makeDataResponse = JSON.stringify(res)
} catch {
this.makeDataResponse = res
}
this.createDatabaseLoading = false
})
.catch((err: any) => {
this.createDatabaseLoading = false
try {
this.makeDataResponse = JSON.stringify(err)
} catch {
this.makeDataResponse = err
}
})
}
public navigateToHome() {
this.onNavigateToHome.emit()
}
public validateDeploy() {
this.isValidating = true
this.sasService
.request('public/startupservice', null)
.then((res: RequestWrapperResponse) => {
this.loggerService.log(res.adapterResponse)
const malformedMessage = getMalformedAdapterResponseMessage(
res.adapterResponse
)
if (malformedMessage) {
this.validationState = 'error'
this.eventService.showInfoModal('Error', malformedMessage)
} else if (res.adapterResponse.saslibs) {
this.validationState = 'success'
} else {
this.validationState = 'error'
}
this.isValidating = false
})
.catch((err: any) => {
this.isValidating = false
this.validationState = 'error'
})
}
public deleteKeys() {
localStorage.removeItem('deploy_dc_loc')
}
}
+42 -8
View File
@@ -89,6 +89,11 @@ import { withoutEditStatus } from './utils/withoutEditStatus'
import { getEditStatusSymbol } from './utils/getEditStatusSymbol'
import { normalizeSortConfig } from './utils/normalizeSortConfig'
import { EDIT_STATUS_COLUMN_NAME } from '../shared/dc-validator/utils/editStatusColumnRule'
import { DELETE_RECORD_COLUMN_NAME } from '../shared/dc-validator/utils/deleteRecordColumnRule'
import {
prepareDeleteRecordColumn,
seedDeleteRecordColumn
} from '../shared/dc-validator/utils/deleteRecordColumn'
import {
errorRenderer,
noSpinnerRenderer,
@@ -3519,14 +3524,24 @@ export class EditorComponent
// A synchronous throw inside initSetup (called from the .then()
// above) lands here too, not just a network failure - log it, or
// the table just renders hidden with no clue why.
// eslint-disable-next-line no-console
console.error('editors/getdata failed:', err)
this.getdataError = true
this.tableTrue = true
this.handleGetdataFailure(err)
})
}
}
/**
* Puts the page into its getdata error state. Shared by the callService
* .catch and initSetup's own retry branch: a throw from the retry runs on a
* bare setTimeout, outside that promise chain, so it would otherwise never
* reach the .catch and the user would be left on the loading spinner.
*/
private handleGetdataFailure(err: any) {
// eslint-disable-next-line no-console
console.error('editors/getdata failed:', err)
this.getdataError = true
this.tableTrue = true
}
/**
* VA mode, initial (unfiltered) route only: briefly wait for VA's buffered
* first message and, if it already carries a filter, save that filter and
@@ -3875,13 +3890,28 @@ export class EditorComponent
this.hotInstance = this.hotTableComponent?.hotInstance!
if (!this.hotInstance) {
// Retry init, don't permanently abort
// Retry init, don't permanently abort. A throw from the retry runs on a
// bare setTimeout rather than in the caller's promise chain, so it would
// never reach that .catch - surface it through the same handler here.
if (attempt < 60) {
setTimeout(() => this.initSetup(response, attempt + 1), 50)
setTimeout(() => {
try {
this.initSetup(response, attempt + 1)
} catch (err) {
this.handleGetdataFailure(err)
}
}, 50)
}
return
}
// The delete column is client-owned (see deleteRecordColumnRule.ts):
// this asserts getdata did not send it, then prepends it to COLHEADERS so
// it is the grid's first column. A throw here lands in the caller's
// .catch - or, on a retry, the branch above - which sets getdataError and
// shows the error state.
prepareDeleteRecordColumn(response)
this.cols = response.data.cols
this.dsmeta = response.data.dsmeta
this.versions = response.data.versions || []
@@ -3931,9 +3961,9 @@ export class EditorComponent
this.checkEmptyRowWhenFilter()
if (this.headerColumns.indexOf('_____DELETE__THIS__RECORD_____') !== -1) {
if (this.headerColumns.indexOf(DELETE_RECORD_COLUMN_NAME) !== -1) {
this.headerColumns[
this.headerColumns.indexOf('_____DELETE__THIS__RECORD_____')
this.headerColumns.indexOf(DELETE_RECORD_COLUMN_NAME)
] = 'Delete?'
}
@@ -3998,6 +4028,10 @@ export class EditorComponent
row[EDIT_STATUS_COLUMN_NAME] = 'U'
}
// The delete flag is client-owned too (see deleteRecordColumnRule.ts) -
// getdata no longer sends the column, so every row starts unmarked.
seedDeleteRecordColumn(this.dataSource)
// Seed HARDFORMULA/SOFTFORMULA columns with their computed formula
// string per row. dataSourceUnchanged isn't populated yet this early in
// a fresh load (only the excel-preview/add-row flows set it) - at this
@@ -1,7 +1,5 @@
import { SASjsConfig } from '@sasjs/adapter'
export interface DcAdapterSettings extends Partial<SASjsConfig> {
adminGroup: string
dcPath: string
hotLicenceKey: string
}
@@ -18,6 +18,11 @@ import {
import { globals } from '../_globals'
import { EditorsGetDataServiceResponse } from '../models/sas/editors-getdata.model'
import { DcValidator } from '../shared/dc-validator/dc-validator'
import {
prepareDeleteRecordColumn,
seedDeleteRecordColumn
} from '../shared/dc-validator/utils/deleteRecordColumn'
import { DELETE_RECORD_COLUMN_NAME } from '../shared/dc-validator/utils/deleteRecordColumnRule'
import { ExcelRule } from '../models/TableData'
import { HotTableInterface } from '../models/HotTable.interface'
import { Col } from '../shared/dc-validator/models/col.model'
@@ -360,7 +365,21 @@ export class MultiDatasetComponent implements OnInit, AfterViewInit {
this.uploadLoading = false
const datasetObjects = this.buildDatasetsObjects(datasets)
// buildDatasetsObjects throws when a getdata response carries the
// reserved delete-record column (see deleteRecordColumn.ts). This chain
// has no .catch, so an uncaught throw here would become an unhandled
// rejection that aborts the loop with nothing shown. Surface it on
// screen, the way the editor flow refuses the same response, rather
// than leaving the user on an empty table selection with the file
// still attached.
let datasetObjects: DatasetsObject[] = []
try {
datasetObjects = this.buildDatasetsObjects(datasets)
} catch (err) {
console.warn('Error building dataset objects', err)
this.eventService.showInfoModal('Upload error', (err as Error).message)
return
}
datasetObjects.forEach((datasetObject) => {
this.parsedDatasets.push({
@@ -1074,6 +1093,12 @@ export class MultiDatasetComponent implements OnInit, AfterViewInit {
for (let response of responses) {
if (response.data) {
// The delete column is client-owned (see deleteRecordColumnRule.ts):
// asserts getdata did not send it, then prepends it to COLHEADERS so
// it is index 0 for everything below.
prepareDeleteRecordColumn(response)
seedDeleteRecordColumn(response.data.sasdata)
const datasetObject: DatasetsObject = {
...response,
hotTable: {
@@ -1097,14 +1122,10 @@ export class MultiDatasetComponent implements OnInit, AfterViewInit {
datasetObject.headerPks = response.data.sasparams[0].PK.split(' ')
if (
datasetObject.headerColumns.indexOf(
'_____DELETE__THIS__RECORD_____'
) !== -1
datasetObject.headerColumns.indexOf(DELETE_RECORD_COLUMN_NAME) !== -1
) {
datasetObject.headerColumns[
datasetObject.headerColumns.indexOf(
'_____DELETE__THIS__RECORD_____'
)
datasetObject.headerColumns.indexOf(DELETE_RECORD_COLUMN_NAME)
] = 'Delete?'
}
@@ -0,0 +1,110 @@
import { QueryComponent } from './query.component'
import { HelperService } from '../services/helper.service'
/**
* dynamicWhereClauseCreator builds the prefilter that getcolvals sends when the
* dynamic where clause is on, and it composes a temporal value into the same
* SAS literal the applied filter carries. The value it prefilters with comes
* from the *other* clauses in the query, so the clause under test sits at index
* 1 and index 0 is skipped.
*/
describe('QueryComponent - dynamic where clause', () => {
const buildComponent = (elements: any[]) => {
const component = new QueryComponent(
{} as any,
{ catchResponseError: () => undefined } as any,
new HelperService(null as any),
{ detectChanges: () => undefined } as any
)
component.clauses = {
queryObj: [{ clauseLogic: 'AND', elements }],
groupLogic: 'AND'
} as any
return component
}
const temporalClause = (over: any = {}) => ({
variable: 'SOME_DATE',
operator: '=',
value: '',
values: [],
type: 'num',
ddtype: 'DATE',
valueVariable: false,
...over
})
it('sends an IN entry as the literal for its unformatted value', () => {
const component = buildComponent([
temporalClause(),
temporalClause({
operator: 'IN',
value: [{ checked: true, val: '12FEB1960' }],
values: [{ formatted: '12FEB1960', unformatted: 42 }]
})
])
const table = component.dynamicWhereClauseCreator(0, 0)
expect(table.length).toBe(1)
expect(table[0].RAW_VALUE).toBe("('12FEB1960'd)")
})
it('maps both ends of a BETWEEN picked from the value list', () => {
const component = buildComponent([
temporalClause(),
temporalClause({
operator: 'BETWEEN',
value: ['12FEB1960', '13FEB1960'],
values: [
{ formatted: '12FEB1960', unformatted: 42 },
{ formatted: '13FEB1960', unformatted: 43 }
]
})
])
const table = component.dynamicWhereClauseCreator(0, 0)
expect(table.length).toBe(1)
expect(table[0].RAW_VALUE).toBe("'12FEB1960'd AND '13FEB1960'd")
})
it('leaves an already-unformatted BETWEEN alone', () => {
const component = buildComponent([
temporalClause(),
temporalClause({
operator: 'BETWEEN',
value: [42, 43],
values: [
{ formatted: '12FEB1960', unformatted: 42 },
{ formatted: '13FEB1960', unformatted: 43 }
]
})
])
const table = component.dynamicWhereClauseCreator(0, 0)
expect(table[0].RAW_VALUE).toBe("'12FEB1960'd AND '13FEB1960'd")
})
it('emits the unformatted value in the non-literal IN fallback', () => {
const component = buildComponent([
temporalClause(),
{
variable: 'SOME_CHAR',
operator: 'IN',
value: [{ checked: true, val: 'ABC ' }],
values: [{ formatted: 'ABC ', unformatted: 'ABC' }],
type: 'char',
ddtype: 'CHAR',
valueVariable: false
}
])
const table = component.dynamicWhereClauseCreator(0, 0)
expect(table[0].RAW_VALUE).toBe("('ABC')")
})
})
+49 -26
View File
@@ -425,35 +425,57 @@ export class QueryComponent
continue
let rawValue = ''
const operator =
this.clauses.queryObj[groupInd].elements[clauseInd].operator
const variable =
this.clauses.queryObj[groupInd].elements[clauseInd].variable
const element = this.clauses.queryObj[groupInd].elements[clauseInd]
const operator = element.operator
const variable = element.variable
/* a temporal value is sent as a SAS literal (eg '20APR2042'd), the
same way the applied filter carries it */
const temporalValue = (value: any) =>
element.valueVariable
? null
: this.helperService.sasTemporalLiteral(value, element.ddtype)
/* the dropdowns and the IN modal hold the *formatted* value (the
soft-select option value, and setINValues), so a picked value is
mapped back to the unformatted one the filter needs */
const unformattedValue = (value: any) =>
(element.values || []).find((v: any) => v.formatted === value)
?.unformatted ?? value
if (operator === 'BETWEEN') {
rawValue = `${this.clauses.queryObj[groupInd].elements[clauseInd].value[0]} AND ${this.clauses.queryObj[groupInd][clauseInd].value[1]}`
const from = unformattedValue(element.value[0])
const to = unformattedValue(element.value[1])
rawValue = `${temporalValue(from) ?? from} AND ${
temporalValue(to) ?? to
}`
} else if (operator === 'IN' || operator === 'NOT IN') {
this.clauses.queryObj[groupInd][clauseInd].value.forEach(
(element: any) => {
if (element.checked) {
let value = element.val
element.value.forEach((item: any) => {
if (item.checked) {
/* map once, and use the unformatted value in every branch - the
literal is built from it, and so are the fallbacks, so a
formatted label can never reach RAW_VALUE */
const value = unformattedValue(item.val)
const literal = temporalValue(value)
if (typeof value === 'string' && !isSpecialMissing(value)) {
if (rawValue.length > 0) {
rawValue += `,'${value}'`
} else {
rawValue = `('${value}'`
}
if (literal) {
rawValue += (rawValue.length > 0 ? ',' : '(') + literal
} else if (
typeof value === 'string' &&
!isSpecialMissing(value)
) {
if (rawValue.length > 0) {
rawValue += `,'${value}'`
} else {
if (rawValue.length > 0) {
rawValue += `,${value}`
} else {
rawValue = `(${value}`
}
rawValue = `('${value}'`
}
} else {
if (rawValue.length > 0) {
rawValue += `,${value}`
} else {
rawValue = `(${value}`
}
}
}
)
})
if (rawValue.length > 0) {
rawValue += ')'
@@ -470,11 +492,12 @@ export class QueryComponent
rawValue = '.'
}
} else {
if (
this.clauses.queryObj[groupInd].elements[clauseInd].type ===
'char'
) {
rawValue = `'${this.clauses.queryObj[groupInd].elements[clauseInd].value}'`
const literal = temporalValue(element.value)
if (literal) {
rawValue = literal
} else if (element.type === 'char') {
rawValue = `'${element.value}'`
}
}
@@ -46,3 +46,72 @@ describe('HelperService - convertArrayValues', () => {
expect(result.length).toBe(3)
})
})
describe('HelperService - sasTemporalLiteral', () => {
let service: HelperService
beforeEach(() => {
service = new HelperService(null as any)
})
it('renders a SAS date value as a date literal', () => {
expect(service.sasTemporalLiteral(42, 'DATE')).toBe("'12FEB1960'd")
expect(service.sasTemporalLiteral(0, 'DATE')).toBe("'01JAN1960'd")
expect(service.sasTemporalLiteral(34322, 'DATE')).toBe("'20DEC2053'd")
})
it('renders a date before 1960', () => {
expect(service.sasTemporalLiteral(-1, 'DATE')).toBe("'31DEC1959'd")
})
it('renders a SAS datetime value as a datetime literal', () => {
expect(service.sasTemporalLiteral(42, 'DATETIME')).toBe(
"'01JAN1960:00:00:42'dt"
)
expect(service.sasTemporalLiteral(86400, 'DATETIME')).toBe(
"'02JAN1960:00:00:00'dt"
)
expect(service.sasTemporalLiteral(31536001, 'DATETIME')).toBe(
"'31DEC1960:00:00:01'dt"
)
})
it('rounds double noise onto the second', () => {
expect(service.sasTemporalLiteral(1.9999999999999, 'TIME')).toBe(
"'00:00:02't"
)
expect(service.sasTemporalLiteral(42.000000000000004, 'DATETIME')).toBe(
"'01JAN1960:00:00:42'dt"
)
})
it('returns null rather than round a genuinely fractional value', () => {
expect(service.sasTemporalLiteral(42.5, 'DATETIME')).toBeNull()
expect(service.sasTemporalLiteral(42.5, 'TIME')).toBeNull()
expect(service.sasTemporalLiteral(42.3, 'TIME')).toBeNull()
})
it('renders a SAS time value as a time literal', () => {
expect(service.sasTemporalLiteral(0, 'TIME')).toBe("'00:00:00't")
expect(service.sasTemporalLiteral(42, 'TIME')).toBe("'00:00:42't")
expect(service.sasTemporalLiteral(3661, 'TIME')).toBe("'01:01:01't")
})
it('accepts a value that arrives as a string', () => {
expect(service.sasTemporalLiteral('42', 'DATE')).toBe("'12FEB1960'd")
})
it('returns null when the value cannot be a literal', () => {
// not temporal
expect(service.sasTemporalLiteral(42, 'N')).toBeNull()
expect(service.sasTemporalLiteral(42, 'C')).toBeNull()
expect(service.sasTemporalLiteral(42, '')).toBeNull()
// not a number
expect(service.sasTemporalLiteral('12FEB1960', 'DATE')).toBeNull()
expect(service.sasTemporalLiteral('', 'DATE')).toBeNull()
expect(service.sasTemporalLiteral(null, 'DATE')).toBeNull()
expect(service.sasTemporalLiteral(undefined, 'DATE')).toBeNull()
// a fractional number of days is not a SAS date
expect(service.sasTemporalLiteral(42.5, 'DATE')).toBeNull()
})
})
+80
View File
@@ -5,6 +5,22 @@ import { SasService } from './sas.service'
const librariesToShow = 50
/** Month names used by a SAS date literal, eg 20APR2042. */
const sasMonths = [
'JAN',
'FEB',
'MAR',
'APR',
'MAY',
'JUN',
'JUL',
'AUG',
'SEP',
'OCT',
'NOV',
'DEC'
]
@Injectable({
providedIn: 'root'
})
@@ -226,6 +242,70 @@ export class HelperService {
})
}
/**
* Formats a SAS internal date / datetime / time value as the SAS literal a
* WHERE clause can carry, eg:
* 34322 (DATE) -> '20DEC2053'd
* 42 (DATETIME) -> '01JAN1960:00:00:42'dt
* 42 (TIME) -> '00:00:42't
*
* The value is days since 1960-01-01 for a date, and seconds since
* 1960-01-01 (datetime) or since midnight (time). The date part is built
* arithmetically in UTC, so the day cannot shift with the browser timezone.
*
* SAS temporal values are whole days and whole seconds, so the value is
* rounded to seconds first - that kills the noise a double picks up on the
* way here, where 1.9999999999999 is simply the second 2. A value that is
* genuinely fractional does not land on a second, and rounding it would
* silently widen the filter, so it returns null instead.
*
* Returns null when the value is not a finite number, is not a whole day or
* second, or the ddtype is not temporal - callers then send the raw value.
*/
public sasTemporalLiteral(value: any, ddtype: string): string | null {
const kind = (ddtype ?? '').toString().toUpperCase()
if (kind !== 'DATE' && kind !== 'DATETIME' && kind !== 'TIME') return null
if (value === null || value === undefined || value === '') return null
const num = typeof value === 'number' ? value : Number(String(value).trim())
if (!isFinite(num)) return null
/* Within the precision of a double, is this a whole second? */
const seconds = Math.round(num * 1e8) / 1e8
if (!Number.isInteger(seconds)) return null
if (kind === 'TIME') return `'${this.sasTimeFromSeconds(seconds)}'t`
const days = kind === 'DATETIME' ? Math.floor(seconds / 86400) : seconds
const datePart = this.sasDateFromDays(days)
if (kind === 'DATE') return `'${datePart}'d`
return `'${datePart}:${this.sasTimeFromSeconds(seconds - days * 86400)}'dt`
}
/** `ddMMMyyyy` for a SAS date value (days since 1960-01-01). */
private sasDateFromDays(days: number): string {
const date = new Date(Date.UTC(1960, 0, 1) + days * 86400000)
return (
this.addLeadingZero(date.getUTCDate()) +
sasMonths[date.getUTCMonth()] +
date.getUTCFullYear()
)
}
/** `hh:mm:ss` for a SAS time value (whole seconds since midnight). */
private sasTimeFromSeconds(seconds: number): string {
return (
this.addLeadingZero(Math.floor(seconds / 3600)) +
':' +
this.addLeadingZero(Math.floor((seconds % 3600) / 60)) +
':' +
this.addLeadingZero(seconds % 60)
)
}
public addLeadingZero(value: string | number): string {
if (typeof value !== 'string') value = value.toString()
+11 -7
View File
@@ -8,13 +8,20 @@ import * as encoding from 'text-encoding'
import { Router } from '@angular/router'
import { Globvar } from '../models/sas/public-startupservice.model'
import { freeTierConfig } from '../free-tier.config'
import { AppStoreService } from './app-store.service'
import { HelperService } from './helper.service'
import { LicenceState } from '../models/LicenceState'
import { decodeLicenceFeatures as decodeFeatures } from './utils/decodeLicenceFeatures'
import { LoggerService } from './logger.service'
import { EventService } from './event.service'
/**
* Handsontable licence key used until a licence is loaded from the backend.
* The decrypted licence carries the customer's own key (`hot_license_key`), so
* this is only the pre-licence / free-tier default - it used to be supplied by
* the `<sasjs>` tag.
*/
export const DEFAULT_HOT_LICENCE_KEY = 'non-commercial-and-evaluation'
@Injectable({
providedIn: 'root'
})
@@ -43,7 +50,7 @@ export class LicenceService {
private freeTierLicenceData: LicenseKeyData = {
demo: true,
hot_license_key: this.appStoreService.getDcAdapterSettings()?.hotLicenceKey,
hot_license_key: DEFAULT_HOT_LICENCE_KEY,
users_allowed: this._licenceState.users_allowed || 1,
valid_until: moment().add(1, 'year').format('YYYY-MM-DD'),
site_id: '',
@@ -63,7 +70,6 @@ export class LicenceService {
constructor(
private loggerService: LoggerService,
private appStoreService: AppStoreService,
private eventService: EventService,
private sasService: SasService,
private helperService: HelperService,
@@ -79,8 +85,7 @@ export class LicenceService {
}
public async activation(startupservice: any) {
this.freeTierLicenceData.hot_license_key =
this.appStoreService.getDcAdapterSettings()?.hotLicenceKey
this.freeTierLicenceData.hot_license_key = DEFAULT_HOT_LICENCE_KEY
await this.setStartupserviceRules(startupservice)
@@ -225,8 +230,7 @@ export class LicenceService {
? licenseData.hot_license_key
: this.hot_license_key.value
if (!hotLicenceKey)
hotLicenceKey = this.appStoreService.getDcAdapterSettings()?.hotLicenceKey
if (!hotLicenceKey) hotLicenceKey = DEFAULT_HOT_LICENCE_KEY
this.hot_license_key.next(hotLicenceKey)
@@ -0,0 +1,313 @@
import { SasStoreService } from './sas-store.service'
import { HelperService } from './helper.service'
/**
* The filter picker keys its operator set, its variable highlight and its value
* quoting on Col.TYPE being exactly 'num' or 'char'. On the normal path
* mergeColsRules derives that from $dataFormats; on the public/getcols fallback
* (used when the EDIT/VIEW limits abort the load) the service speaks the SAS
* vocabulary (TYPE C/N, DDTYPE separating DATE/DATETIME/TIME), so the store has
* to map it. These tests pin both the mapping and the empty-payload guard.
*/
describe('SasStoreService', () => {
const buildStore = (cols: any[]) =>
new SasStoreService(
{
request: () => Promise.resolve({ adapterResponse: { cols } })
} as any,
{} as any,
{} as any
)
const collectColumns = (store: SasStoreService) => {
const seen: any[] = []
store.columns.subscribe((value: any) => seen.push(value))
return seen
}
describe('getCols', () => {
it('maps the SAS C/N TYPE onto the picker vocabulary', async () => {
const store = buildStore([
{ NAME: 'CHAR_COL', TYPE: 'C', DDTYPE: 'C' },
{ NAME: 'NUM_COL', TYPE: 'N', DDTYPE: 'N' },
{ NAME: 'DATE_COL', TYPE: 'N', DDTYPE: 'DATE' },
{ NAME: 'DATETIME_COL', TYPE: 'N', DDTYPE: 'DATETIME' },
{ NAME: 'TIME_COL', TYPE: 'N', DDTYPE: 'TIME' }
])
const cols = await store.getCols('DC.MY_TABLE')
expect(cols.map((col: any) => col.TYPE)).toEqual([
'char',
'num',
'num',
'num',
'num'
])
})
it('leaves the rest of the column payload untouched', async () => {
const store = buildStore([
{
NAME: 'CHAR_COL',
TYPE: 'C',
DDTYPE: 'C',
LENGTH: 8,
LABEL: 'A label'
}
])
const cols = await store.getCols('DC.MY_TABLE')
expect(cols[0].NAME).toBe('CHAR_COL')
expect(cols[0].DDTYPE).toBe('C')
expect(cols[0].LENGTH).toBe(8)
expect(cols[0].LABEL).toBe('A label')
})
it('returns an empty list rather than throwing when the payload is empty', async () => {
const store = buildStore([])
const cols = await store.getCols('DC.MY_TABLE')
expect(cols).toEqual([])
})
})
describe('setQueryVariablesFromTable', () => {
it('publishes the table columns without calling the service', async () => {
let called = false
const store = new SasStoreService(
{
request: () => {
called = true
return Promise.resolve({})
}
} as any,
{} as any,
{} as any
)
const seen = collectColumns(store)
await store.setQueryVariablesFromTable('DC.MY_TABLE', [{ NAME: 'A' }])
expect(called).toBe(false)
expect(seen.length).toBe(1)
expect(seen[0].data.cols.length).toBe(1)
})
it('falls back to getcols and publishes the mapped columns', async () => {
const store = buildStore([{ NAME: 'A', TYPE: 'C', DDTYPE: 'C' }])
const seen = collectColumns(store)
await store.setQueryVariablesFromTable('DC.MY_TABLE', [])
expect(seen.length).toBe(1)
expect(seen[0].libds).toBe('DC.MY_TABLE')
expect(seen[0].data.cols[0].TYPE).toBe('char')
})
it('does not publish an empty fallback, which the picker cannot render', async () => {
const store = buildStore([])
const seen = collectColumns(store)
await store.setQueryVariablesFromTable('DC.MY_TABLE', [])
expect(seen.length).toBe(0)
})
})
})
/**
* The clause shown in the filter panel is composed by whereClauseCreator, and
* its temporal rewrite turns the internal day / second count into a SAS
* literal. It may only do that when the composed value really is nothing but
* those numbers - a value typed by hand is not, and rewriting the digit runs
* inside it produced clauses like '13JAN1960'd/'03JAN1960'd/'14MAY1965'd for
* a date typed as 12/02/1960.
*/
describe('SasStoreService - temporal clause preview', () => {
const buildStore = () =>
new SasStoreService({} as any, new HelperService(null as any), {} as any)
const clauseFor = (
value: any,
operator: string,
ddtype: string,
values: any[] = []
) => ({
queryObj: [
{
clauseLogic: '',
elements: [
{
variable: 'SOME_DATE',
operator,
value,
type: 'num',
ddtype,
valueVariable: false,
values
}
]
}
],
groupLogic: ''
})
it('leaves a typed temporal value alone when it is not a number', () => {
const store = buildStore()
const result = store.whereClauseCreator(
clauseFor('12/02/1960', '=', 'DATE'),
'',
'lib.ds'
)
expect(result.whereClause).toContain('12/02/1960')
expect(result.whereClause).not.toContain("'d")
})
it('still shows an internal day count as a date literal', () => {
const store = buildStore()
const result = store.whereClauseCreator(
clauseFor('42', '=', 'DATE'),
'',
'lib.ds'
)
expect(result.whereClause).toContain("'12FEB1960'd")
})
/**
* A BETWEEN pair is composed as `<from> AND <to>`, so the guard that decides
* whether the rewrite may run has to see past the separator - its letters are
* not part of the numbers the value is built from.
*/
it('shows a BETWEEN pair picked from the value list as literals', () => {
const store = buildStore()
const result = store.whereClauseCreator(
clauseFor(['12FEB1960', '13FEB1960'], 'BETWEEN', 'DATE', [
{ formatted: '12FEB1960', unformatted: '42' },
{ formatted: '13FEB1960', unformatted: '43' }
]),
'',
'lib.ds'
)
expect(result.whereClause).toContain(
"BETWEEN '12FEB1960'd AND '13FEB1960'd"
)
})
})
/**
* createFilterQueryTable builds the RAW_VALUE the backend validates and stores.
* The range dropdown binds the formatted label, so a BETWEEN has to map each
* end back to its unformatted value before the literal is built - otherwise
* sasTemporalLiteral returns null and an unquoted date reaches the backend.
*/
describe('SasStoreService - temporal RAW_VALUE', () => {
const buildStore = () =>
new SasStoreService({} as any, new HelperService(null as any), {} as any)
const betweenQuery = (value: any[], values: any[]) => ({
groupLogic: 'AND',
filterGroups: [
{
clauseLogic: 'AND',
filterClauses: [
{
ddtype: 'DATE',
invalidClause: false,
logic: '',
operator: 'BETWEEN',
operators: [],
type: 'num',
value,
valueVariable: false,
values,
variable: 'SOME_DATE'
}
]
}
]
})
it('maps a BETWEEN picked from the value list back to the internal value', () => {
const store = buildStore()
const clauses = store.createFilterQueryTable(
betweenQuery(
['12FEB1960', '13FEB1960'],
[
{ formatted: '12FEB1960', unformatted: 42 },
{ formatted: '13FEB1960', unformatted: 43 }
]
) as any
)
expect(clauses[0].RAW_VALUE).toBe("'12FEB1960'd AND '13FEB1960'd")
})
it('still builds literals when the values are already internal counts', () => {
const store = buildStore()
const clauses = store.createFilterQueryTable(
betweenQuery(['42', '43'], []) as any
)
expect(clauses[0].RAW_VALUE).toBe("'12FEB1960'd AND '13FEB1960'd")
})
const inQuery = (value: any[], values: any[]) => ({
groupLogic: 'AND',
filterGroups: [
{
clauseLogic: 'AND',
filterClauses: [
{
ddtype: 'DATE',
invalidClause: false,
logic: '',
operator: 'IN',
operators: [],
type: 'num',
value,
valueVariable: false,
values,
variable: 'SOME_DATE'
}
]
}
]
})
it('maps an IN entry picked from the value list back to the internal value', () => {
const store = buildStore()
const clauses = store.createFilterQueryTable(
inQuery(
[{ checked: true, val: '12FEB1960' }],
[{ formatted: '12FEB1960', unformatted: 42 }]
) as any
)
expect(clauses[0].RAW_VALUE).toBe("('12FEB1960'd)")
})
it('passes an already-unformatted IN entry through unchanged', () => {
const store = buildStore()
const clauses = store.createFilterQueryTable(
inQuery(
[{ checked: true, val: 42 }],
[{ formatted: '12FEB1960', unformatted: 42 }]
) as any
)
expect(clauses[0].RAW_VALUE).toBe("('12FEB1960'd)")
})
})
+97 -6
View File
@@ -301,13 +301,29 @@ export class SasStoreService {
* The picker's variable list normally comes from the columns that arrived
* with the table (see setQueryVariablesFromTable). This is the fallback for
* when there is no table - public/getcols reads the metadata only.
*
* public/getcols speaks the SAS vocabulary: TYPE is C/N and DDTYPE separates
* DATE, DATETIME and TIME. The picker keys its operator set (query.component),
* its variable highlight and its value quoting (sas-store) on the client
* vocabulary instead - TYPE is exactly 'num' or 'char' - which
* mergeColsRules derives from $dataFormats on the normal path. That pass
* never runs for this fallback, so map TYPE here, with the same rule: the
* date and time types are numeric.
*/
public async getCols(libds: string) {
const tables = { iwant: [{ libds: libds }] }
const res = await this.sasService.request('public/getcols', tables)
return res.adapterResponse.cols || []
const cols = res.adapterResponse.cols || []
return cols.map((col: any) => ({
...col,
TYPE:
['DATE', 'DATETIME', 'TIME'].includes(col.DDTYPE) || col.TYPE === 'N'
? 'num'
: 'char'
}))
}
/**
@@ -326,7 +342,19 @@ export class SasStoreService {
return
}
this.setQueryVariables(libds, await this.getCols(libds))
const fallback = await this.getCols(libds)
/*
public/getcols emits an empty cols table when the dataset has no
metadata. Publishing that would push an empty array into the filter
picker, whose subscriber reads cols[0].NAME - the throw is caught, but
the picker is left half-updated (this.cols assigned, the filter.cols
assignment below the throwing line never run). There is nothing to
offer, so leave the picker as it was.
*/
if (!fallback || fallback.length === 0) return
this.setQueryVariables(libds, fallback)
}
public async getChangeInfo(tableId: any) {
@@ -568,6 +596,27 @@ export class SasStoreService {
}
oper = this.checkOperator(oper, value, type).operator
/* A temporal value is shown as a SAS literal (eg '20DEC2053'd) rather
than the internal day / second count. Only rewrite when the composed
value is nothing but the numbers the clause is built from - a single
value, a BETWEEN pair or an IN list. The BETWEEN separator is
removed before the test because ' AND ' is not part of that character
set; a typed value that is not a number, such as a date entered by
hand, would otherwise have every digit run inside it rewritten into
an unrelated literal. */
if (
!isValueVariable &&
['DATE', 'DATETIME', 'TIME'].includes(query.ddtype) &&
/^[-0-9.,()\s]+$/.test(String(value).replace(/\s+AND\s+/gi, ' '))
) {
value = String(value).replace(
/-?\d+(?:\.\d+)?/g,
(match) =>
this.helperService.sasTemporalLiteral(match, query.ddtype) ??
match
)
}
if (type === 'char' && oper !== 'IN' && oper !== 'NOT IN') {
if (typeof value === 'undefined') {
value = ''
@@ -634,17 +683,53 @@ export class SasStoreService {
filterQuery.filterGroups.forEach(
(filterGroup: FilterGroup, groupIndex: number) => {
filterGroup.filterClauses.forEach((filterClause: FilterClause) => {
/* A temporal value is sent as a SAS literal (eg '20APR2042'd) rather
than the internal day / second count, so the resolved WHERE clause
- and the FILTER_TEXT shown above the grid - carries a date. */
const temporalValue = (value: any) =>
filterClause.valueVariable
? null
: this.helperService.sasTemporalLiteral(
value,
filterClause.ddtype
)
/* The range dropdown binds the formatted label, as the value list
and the IN modal do, so each end of a BETWEEN is mapped back to
its unformatted value before the literal is built. This is the
same lookup whereClauseCreator applies for the preview - without
it the formatted string reaches sasTemporalLiteral, comes back
null, and RAW_VALUE carries an unquoted date that the backend
rejects on a numeric column. */
const unformattedValue = (value: any) => {
const match = (filterClause.values || []).find(
(entry: any) => entry.formatted === value
)
return match ? match.unformatted : value
}
let rawValue = ''
if (filterClause.operator === 'BETWEEN') {
rawValue = `${filterClause.value[0]} AND ${filterClause.value[1]}`
const from = unformattedValue(filterClause.value[0])
const to = unformattedValue(filterClause.value[1])
rawValue = `${temporalValue(from) ?? from} AND ${
temporalValue(to) ?? to
}`
} else if (
filterClause.operator === 'IN' ||
filterClause.operator === 'NOT IN'
) {
filterClause.value.forEach((element: any) => {
if (element.checked) {
const value = element.val
if (
/* Mapped like the BETWEEN ends - whereClauseCreator normally
rewrites element.val to the unformatted count before submit,
so this is a pass-through for the usual path and a guard for
any path that reaches here without the preview having run. */
const value = unformattedValue(element.val)
const literal = temporalValue(value)
if (literal) {
rawValue += (rawValue.length > 0 ? ',' : '(') + literal
} else if (
typeof value === 'string' &&
(!isSpecialMissing(value) || filterClause.type === 'char')
) {
@@ -672,7 +757,13 @@ export class SasStoreService {
rawValue = '.'
}
} else {
if (filterClause.type === 'char' && !filterClause.valueVariable) {
const literal = temporalValue(filterClause.value)
if (literal) {
rawValue = literal
} else if (
filterClause.type === 'char' &&
!filterClause.valueVariable
) {
rawValue = `'${filterClause.value.replace(/'/g, "''")}'`
}
}
+78 -2
View File
@@ -1,4 +1,4 @@
import { SasService } from './sas.service'
import { DEBUG_STATE_KEY, SasService } from './sas.service'
/**
* SasService's constructor is a plain 8-argument constructor with an empty
@@ -13,7 +13,8 @@ import { SasService } from './sas.service'
const buildService = () => {
const sasjsAdapter: any = {
request: jasmine.createSpy('request').and.resolveTo({}),
getSasRequests: () => []
getSasRequests: () => [],
setDebugState: jasmine.createSpy('setDebugState')
}
const loggerService: any = {
logRequestData: jasmine.createSpy('logRequestData')
@@ -87,3 +88,78 @@ describe('SasService diagnostics payload', () => {
expect(sentBody(sasjsAdapter).browser_info).toBe(supplied)
})
})
/**
* The `<sasjs>` tag is re-read on every load, so the debug toggle has to survive
* a refresh on its own.
*/
describe('SasService debug state', () => {
/** Builds a service whose init can actually run (real adapter, stub deps). */
const initService = (tagDebug = false) => {
const appStoreService: any = {
getDcAdapterSettings: () => ({
serverUrl: '',
appLoc: '/Public/app/dc',
serverType: 'SAS9',
loginMechanism: '',
dcPath: '',
debug: tagDebug,
useComputeApi: null,
runAsTask: true,
contextName: ''
})
}
const service = new SasService(
appStoreService,
{ user: undefined } as any,
{ showInfoModal: jasmine.createSpy('showInfoModal') } as any,
{ setup: jasmine.createSpy('setup') } as any,
{ setup: jasmine.createSpy('setup') } as any,
{ logRequestData: jasmine.createSpy('logRequestData') } as any,
{ updateStep: jasmine.createSpy('updateStep') } as any,
{ navigateByUrl: jasmine.createSpy('navigateByUrl') } as any
)
service.sasServiceInit()
return service
}
beforeEach(() => localStorage.removeItem(DEBUG_STATE_KEY))
afterEach(() => localStorage.removeItem(DEBUG_STATE_KEY))
it('stores the state when the toggle changes', () => {
const { service, sasjsAdapter } = buildService()
service.setDebugState(true)
expect(localStorage.getItem(DEBUG_STATE_KEY)).toBe('true')
expect(sasjsAdapter.setDebugState).toHaveBeenCalledWith(true)
})
it('stores a switch back off, rather than clearing it', () => {
const { service } = buildService()
service.setDebugState(true)
service.setDebugState(false)
expect(localStorage.getItem(DEBUG_STATE_KEY)).toBe('false')
})
it('reinstates the stored state on the next load', () => {
localStorage.setItem(DEBUG_STATE_KEY, 'true')
expect(initService().getSasjsConfig().debug).toBe(true)
})
it('leaves the tag value alone when nothing is stored', () => {
expect(initService(true).getSasjsConfig().debug).toBe(true)
})
it('reinstates an off state over a tag that turns debug on', () => {
localStorage.setItem(DEBUG_STATE_KEY, 'false')
expect(initService(true).getSasjsConfig().debug).toBe(false)
})
})
+15
View File
@@ -41,6 +41,13 @@ const DIAGNOSTICS_SERVICES = [
'services/auditors/postdata'
]
/**
* Where the debug toggle state is kept. The `<sasjs>` tag in index.html is
* re-read on every load, so without this a refresh would drop the state the user
* set from the UI.
*/
export const DEBUG_STATE_KEY = 'dc-debug-state'
@Injectable({
providedIn: 'root'
})
@@ -89,6 +96,13 @@ export class SasService {
this.sasjsAdapter = new SASjs(this.dcAdapterSettings)
/* The tag resets debug on every load, so reinstate the state the user last
chose from the UI. */
const storedDebugState = localStorage.getItem(DEBUG_STATE_KEY)
if (storedDebugState !== null) {
this.sasjsAdapter.setDebugState(storedDebugState === 'true')
}
switch (this.dcAdapterSettings.serverType) {
case ServerType.SasViya: {
this.checkViyaDeploy(this.dcAdapterSettings.appLoc || '')
@@ -803,6 +817,7 @@ export class SasService {
}
public setDebugState(state: boolean) {
localStorage.setItem(DEBUG_STATE_KEY, String(state))
this.sasjsAdapter.setDebugState(state)
}
@@ -0,0 +1,93 @@
import {
getSasjsTagDetails,
SASJS_TAG_STEP,
StartupCheckService
} from './startup-check.service'
describe('StartupCheckService', () => {
let service: StartupCheckService
beforeEach(() => {
service = new StartupCheckService()
})
it('includes a SASjs tag step after the startup checks', () => {
const steps = service.steps$.value
expect(steps.length).toBe(SASJS_TAG_STEP + 1)
expect(steps[SASJS_TAG_STEP].label).toBe('SASjs tag')
expect(steps[SASJS_TAG_STEP].status).toBe('pending')
})
it('shows the tag contents against the SASjs tag step', () => {
service.setTagDetails([{ name: 'serverType', value: 'SASVIYA' }])
expect(service.steps$.value[SASJS_TAG_STEP]).toEqual({
label: 'SASjs tag',
status: 'done',
details: [{ name: 'serverType', value: 'SASVIYA' }]
})
})
it('flags a missing tag on the SASjs tag step', () => {
service.updateStep(SASJS_TAG_STEP, 'error', 'SASJS TAG MISSING!!')
expect(service.steps$.value[SASJS_TAG_STEP]).toEqual({
label: 'SASjs tag',
status: 'error',
detail: 'SASJS TAG MISSING!!'
})
})
})
describe('getSasjsTagDetails', () => {
const tagWith = (attributes: Record<string, string>) => {
const element = document.createElement('sasjs')
Object.entries(attributes).forEach(([name, value]) =>
element.setAttribute(name, value)
)
return element
}
it('reads the tag attributes as name/value pairs', () => {
const details = getSasjsTagDetails(
tagWith({ serverType: 'SASVIYA', debug: 'false', useComputeApi: 'true' })
)
expect(details).toEqual([
{ name: 'serverType', value: 'SASVIYA' },
{ name: 'debug', value: 'false' },
{ name: 'useComputeApi', value: 'true' }
])
})
it('keeps attributes the parser does not know about', () => {
expect(getSasjsTagDetails(tagWith({ serverTyp: 'SASJS' }))).toEqual([
{ name: 'servertyp', value: 'SASJS' }
])
})
it('shows every attribute, including the admin group and the licence key', () => {
const details = getSasjsTagDetails(
tagWith({
serverType: 'SASJS',
adminGroup: 'SASAdministrators',
hotLicenceKey: 'non-commercial-and-evaluation'
})
)
expect(details).toEqual([
{ name: 'serverType', value: 'SASJS' },
{ name: 'adminGroup', value: 'SASAdministrators' },
{ name: 'hotLicenceKey', value: 'non-commercial-and-evaluation' }
])
})
it('flags attributes that carry no value', () => {
expect(getSasjsTagDetails(tagWith({ runAsTask: '' }))).toEqual([
{ name: 'runAsTask', value: '(empty)' }
])
})
})
@@ -3,18 +3,64 @@ import { BehaviorSubject } from 'rxjs'
export type StartupStepStatus = 'pending' | 'in_progress' | 'done' | 'error'
/** A name/value pair shown under a startup step. */
export interface StartupStepDetail {
name: string
value: string
}
export interface StartupStep {
label: string
status: StartupStepStatus
detail?: string
/** Rendered as a grid under the label. */
details?: StartupStepDetail[]
}
/**
* Index of the `<sasjs>` tag dump within the startup steps. It is appended
* after the three startup checks so the indices the startup flow already drives
* (sas.service / app.service) are unchanged.
*/
export const SASJS_TAG_STEP = 3
/**
* Canonical spelling of the `<sasjs>` tag attributes. HTML lower-cases attribute
* names in the DOM, so they are mapped back for display.
*/
const TAG_ATTRIBUTE_NAMES: Record<string, string> = {
serverurl: 'serverUrl',
apploc: 'appLoc',
servertype: 'serverType',
loginmechanism: 'loginMechanism',
admingroup: 'adminGroup',
dcpath: 'dcPath',
debug: 'debug',
usecomputeapi: 'useComputeApi',
runastask: 'runAsTask',
contextname: 'contextName',
hotlicencekey: 'hotLicenceKey'
}
/**
* Reads the attributes of the `<sasjs>` tag for display in the startup checks.
* Every attribute is shown - including ones the parser does not know about, so a
* misspelt one is visible and a wrong value is exactly the misconfiguration this
* dump exists to expose.
*/
export const getSasjsTagDetails = (element: Element): StartupStepDetail[] =>
Array.from(element.attributes).map((attribute) => ({
name: TAG_ATTRIBUTE_NAMES[attribute.name.toLowerCase()] || attribute.name,
value: attribute.value || '(empty)'
}))
@Injectable({ providedIn: 'root' })
export class StartupCheckService {
private steps: StartupStep[] = [
{ label: 'Checking app location', status: 'pending' },
{ label: 'Checking Viya deploy', status: 'pending' },
{ label: 'Loading startup service', status: 'pending' }
{ label: 'Loading startup service', status: 'pending' },
{ label: 'SASjs tag', status: 'pending' }
]
public steps$ = new BehaviorSubject<StartupStep[]>([...this.steps])
@@ -30,6 +76,16 @@ export class StartupCheckService {
}
}
/** Shows the `<sasjs>` tag contents on the boot screen. */
public setTagDetails(details: StartupStepDetail[]) {
this.steps[SASJS_TAG_STEP] = {
...this.steps[SASJS_TAG_STEP],
status: 'done',
details
}
this.steps$.next([...this.steps])
}
public setSteps(steps: StartupStep[]) {
this.steps = steps
this.steps$.next([...this.steps])
@@ -1,27 +1,6 @@
import { VaFilterService } from './va-filter.service'
import { VaMessage } from './va-messaging.service'
/**
* Minimal stand-in for HelperService.convertJsDateToSasDate using the real
* epoch math, so date/datetime assertions use the actual SAS internal values
* without pulling in HelperService's dependency graph (SasService etc.).
*/
class HelperServiceStub {
convertJsDateToSasDate(jsDate: Date, unit = 'days'): number {
const ms = Date.UTC(
jsDate.getFullYear(),
jsDate.getMonth(),
jsDate.getDate(),
jsDate.getHours(),
jsDate.getMinutes(),
jsDate.getSeconds()
)
const sasMs = ms + 315619200000
return unit === 'days'
? Math.floor(Math.abs(sasMs / 86400000))
: sasMs / 1000
}
}
import { HelperService } from './helper.service'
function msg(parameters: any[]): VaMessage {
return {
@@ -63,7 +42,7 @@ describe('VaFilterService', () => {
]
beforeEach(() => {
service = new VaFilterService(new HelperServiceStub() as any)
service = new VaFilterService(new HelperService({} as any))
})
const labelToVar = () =>
@@ -109,28 +88,30 @@ describe('VaFilterService', () => {
expect(build([{ label: 'SOME_NUM', value: 'abc' }])).toEqual([])
})
it('converts temporal values to SAS internal numbers', () => {
it('renders temporal values as SAS literals', () => {
expect(
build([{ label: 'SOME_TIME', value: '0:00:02' }])[0].RAW_VALUE
).toBe('2')
).toBe("'00:00:02't")
expect(
build([{ label: 'SOME_DATE', value: '01JUL1960' }])[0].RAW_VALUE
).toBe('182')
).toBe("'01JUL1960'd")
expect(
build([{ label: 'SOME_DATETIME', value: '01JAN1960:00:06:05' }])[0]
.RAW_VALUE
).toBe('365')
).toBe("'01JAN1960:00:06:05'dt")
})
it('passes an already-internal (bare number) temporal value through raw', () => {
it('renders an already-internal (bare number) temporal value as a literal', () => {
// VA sends the raw SAS internal value when the data item has no format.
expect(build([{ label: 'SOME_DATE', value: '182' }])[0].RAW_VALUE).toBe(
'182'
"'01JUL1960'd"
)
expect(
build([{ label: 'SOME_DATETIME', value: '365' }])[0].RAW_VALUE
).toBe('365')
expect(build([{ label: 'SOME_TIME', value: '2' }])[0].RAW_VALUE).toBe('2')
).toBe("'01JAN1960:00:06:05'dt")
expect(build([{ label: 'SOME_TIME', value: '2' }])[0].RAW_VALUE).toBe(
"'00:00:02't"
)
})
it('parses ISO date shapes identically to DATE9 (VA sends no format)', () => {
@@ -232,9 +213,9 @@ describe('VaFilterService', () => {
expect(clauses.map((c) => [c.VARIABLE_NM, c.RAW_VALUE])).toEqual([
['SOME_CHAR', "'x'"],
['SOME_NUM', '10'],
['SOME_TIME', '2'],
['SOME_DATE', '182'],
['SOME_DATETIME', '365']
['SOME_TIME', "'00:00:02't"],
['SOME_DATE', "'01JUL1960'd"],
['SOME_DATETIME', "'01JAN1960:00:06:05'dt"]
])
})
@@ -247,9 +228,9 @@ describe('VaFilterService', () => {
])
)
expect(clauses.map((c) => [c.VARIABLE_NM, c.RAW_VALUE])).toEqual([
['SOME_DATE', '182'],
['SOME_DATE2', '182'],
['SOME_DTTM', '365']
['SOME_DATE', "'01JUL1960'd"],
['SOME_DATE2', "'01JUL1960'd"],
['SOME_DTTM', "'01JAN1960:00:06:05'dt"]
])
})
})
+33 -21
View File
@@ -223,15 +223,18 @@ export class VaFilterService {
/**
* Formats one VA filter value into a DC RAW_VALUE for the column kind, or null
* when invalid (dropped). Matches DC's native filters, which send the NUMERIC
* INTERNAL value (not SAS literals — those error out in the backend):
* - numeric -> finite number, raw (else null — injection guard)
* - time -> seconds since midnight (VA: "0:00:02" -> 2)
* - date -> days since 1960-01-01 (VA: "01JUL1960" -> 182)
* - datetime -> seconds since 1960-01-01 (VA: "01JAN1960:00:06:05" -> 365)
* when invalid (dropped). Matches DC's native filters, which send a SAS
* literal for a temporal column and the raw value otherwise:
* - numeric -> finite number, raw (else null - injection guard)
* - time -> seconds since midnight, as a SAS time literal
* (VA: "0:00:02" -> '00:00:02't)
* - date -> days since 1960-01-01, as a SAS date literal
* (VA: "01JUL1960" -> '01JUL1960'd)
* - datetime -> seconds since 1960-01-01, as a SAS datetime literal
* (VA: "01JAN1960:00:06:05" -> '01JAN1960:00:06:05'dt)
* - char -> single-quote escaped
* Temporal strings are regex-parsed (digits/letters/colons only), so a value
* either yields a finite number or is dropped — also the injection guard.
* either yields a finite number or is dropped - also the injection guard.
* Date/datetime reuse HelperService.convertJsDateToSasDate so the value is
* byte-identical to what DC's own pickers produce.
*
@@ -239,7 +242,7 @@ export class VaFilterService {
* itself (VA sends it unformatted when the data item carries no format / is
* used as a measure), so it is passed through raw. This is unambiguous because
* the only date formats in use (DATE9, ISO) always contain a letter or a
* separator — a plain integer is never a formatted date.
* separator - a plain integer is never a formatted date.
*/
private formatValueByKind(
value: any,
@@ -254,23 +257,32 @@ export class VaFilterService {
return Number.isFinite(num) ? `${num}` : null
}
case 'time': {
if (this.isBareNumber(s)) return `${Number(s)}`
const secs = this.timeToSeconds(s)
return secs === null ? null : `${secs}`
const secs = this.isBareNumber(s) ? Number(s) : this.timeToSeconds(s)
return secs === null
? null
: this.helperService.sasTemporalLiteral(secs, 'TIME')
}
case 'date': {
if (this.isBareNumber(s)) return `${Number(s)}`
const d = this.parseDate(s, order)
return d
? `${this.helperService.convertJsDateToSasDate(d, 'days')}`
: null
let days: number | null = null
if (this.isBareNumber(s)) days = Number(s)
else {
const d = this.parseDate(s, order)
if (d) days = this.helperService.convertJsDateToSasDate(d, 'days')
}
return days === null
? null
: this.helperService.sasTemporalLiteral(days, 'DATE')
}
case 'datetime': {
if (this.isBareNumber(s)) return `${Number(s)}`
const d = this.parseDatetime(s, order)
return d
? `${this.helperService.convertJsDateToSasDate(d, 'seconds')}`
: null
let secs: number | null = null
if (this.isBareNumber(s)) secs = Number(s)
else {
const d = this.parseDatetime(s, order)
if (d) secs = this.helperService.convertJsDateToSasDate(d, 'seconds')
}
return secs === null
? null
: this.helperService.sasTemporalLiteral(secs, 'DATETIME')
}
default:
return `'${s.replace(/'/g, "''")}'`
@@ -61,13 +61,13 @@ export class DcValidator {
// injected separately — see its own doc comment for why.
//
// COLHEADERS is the authoritative grid order and the only safe key to
// build against: it is the variable list of the *output* dataset, where
// getdata.sas hoists the delete column and the PK to the front ("order
// delete var and pk fields at start of table"), whereas cols[].VARNUM
// describes the *source* table and cols[] itself can arrive sorted by
// NAME. Those orders diverge whenever the PK isn't already the source's
// first column, so rules are keyed by name and emitted in COLHEADERS
// order — never paired positionally against cols[] or VARNUM.
// build against: it is the variable list of the *output* dataset, and the
// editor prepends the client-owned delete column to it (see
// editor.component.ts's initSetup) before this constructor runs, whereas
// cols[].VARNUM describes the *source* table and cols[] itself can arrive
// sorted by NAME. Those orders diverge whenever the PK isn't already the
// source's first column, so rules are keyed by name and emitted in
// COLHEADERS order — never paired positionally against cols[] or VARNUM.
//
// A column with no parseable COLTYPE falls back to a plain text rule
// rather than being dropped: getRules() is consumed positionally against
@@ -0,0 +1,75 @@
import {
prepareDeleteRecordColumn,
seedDeleteRecordColumn
} from './deleteRecordColumn'
import { DELETE_RECORD_COLUMN_NAME } from './deleteRecordColumnRule'
const response = (overrides: any = {}) => ({
data: {
sasparams: [{ COLHEADERS: 'PRIMARY_KEY_FIELD,SOME_CHAR' }],
cols: [],
$sasdata: { vars: {} },
...overrides
}
})
describe('prepareDeleteRecordColumn', () => {
it('prepends the delete column to COLHEADERS', () => {
const res = response()
prepareDeleteRecordColumn(res)
expect(res.data.sasparams[0].COLHEADERS).toEqual(
`${DELETE_RECORD_COLUMN_NAME},PRIMARY_KEY_FIELD,SOME_CHAR`
)
})
it('throws when COLHEADERS already carries the delete column', () => {
const res = response({
sasparams: [{ COLHEADERS: `${DELETE_RECORD_COLUMN_NAME},SOME_CHAR` }]
})
expect(() => prepareDeleteRecordColumn(res)).toThrowError(/reserved column/)
})
it('throws when cols carries the delete column', () => {
const res = response({ cols: [{ NAME: DELETE_RECORD_COLUMN_NAME }] })
expect(() => prepareDeleteRecordColumn(res)).toThrowError(/reserved column/)
})
it('throws when $sasdata vars carries the delete column', () => {
const res = response({
$sasdata: { vars: { [DELETE_RECORD_COLUMN_NAME]: {} } }
})
expect(() => prepareDeleteRecordColumn(res)).toThrowError(/reserved column/)
})
it('leaves COLHEADERS untouched when it throws', () => {
const colHeaders = `${DELETE_RECORD_COLUMN_NAME},SOME_CHAR`
const res = response({ sasparams: [{ COLHEADERS: colHeaders }] })
expect(() => prepareDeleteRecordColumn(res)).toThrowError()
expect(res.data.sasparams[0].COLHEADERS).toEqual(colHeaders)
})
})
describe('seedDeleteRecordColumn', () => {
it("sets every row's delete flag to 'No'", () => {
const rows: any[] = [{ PRIMARY_KEY_FIELD: 1 }, { PRIMARY_KEY_FIELD: 2 }]
seedDeleteRecordColumn(rows)
expect(rows[0][DELETE_RECORD_COLUMN_NAME]).toEqual('No')
expect(rows[1][DELETE_RECORD_COLUMN_NAME]).toEqual('No')
})
it('overwrites a flag a row already carried', () => {
const rows: any[] = [{ [DELETE_RECORD_COLUMN_NAME]: 'Yes' }]
seedDeleteRecordColumn(rows)
expect(rows[0][DELETE_RECORD_COLUMN_NAME]).toEqual('No')
})
})
@@ -0,0 +1,64 @@
import { Col } from '../models/col.model'
import { DELETE_RECORD_COLUMN_NAME } from './deleteRecordColumnRule'
/**
* The parts of a getdata response this module touches. Declared structurally
* so the editor and the multi-dataset loader can both use it without either
* owning the other's response type.
*/
interface DeleteColumnResponse {
data: {
sasparams: { COLHEADERS: string }[]
cols?: Col[]
$sasdata?: { vars?: Record<string, unknown> }
}
}
/**
* Asserts that getdata did NOT send the client-owned delete column, then
* prepends it to COLHEADERS so the grid's first column is the delete flag.
*
* `_____DELETE__THIS__RECORD_____` is created by the editor itself (see
* deleteRecordColumnRule.ts) and getdata must never emit it. Its presence in a
* response can only mean a real table column of that name, or a hook that
* re-added it - either way it would shadow the delete flag, so refuse the load
* rather than render a table whose delete column cannot be trusted. Callers
* run this inside a promise chain whose `.catch` surfaces the error (the
* editor sets `getdataError`).
*
* Call it BEFORE constructing a DcValidator or deriving headerArray: rules,
* `columnHeader` and `headerArray` are all keyed off COLHEADERS order, and the
* delete column is always index 0.
*/
export const prepareDeleteRecordColumn = (response: DeleteColumnResponse) => {
const sasparams = response.data.sasparams[0]
const sentByBackend =
sasparams.COLHEADERS.split(',').includes(DELETE_RECORD_COLUMN_NAME) ||
(response.data.cols || []).some(
(col) => col.NAME === DELETE_RECORD_COLUMN_NAME
) ||
Object.prototype.hasOwnProperty.call(
response.data.$sasdata?.vars || {},
DELETE_RECORD_COLUMN_NAME
)
if (sentByBackend) {
throw new Error(
`getdata returned the reserved column ${DELETE_RECORD_COLUMN_NAME}. ` +
'That name is created by the editor itself and must not exist in the target table.'
)
}
sasparams.COLHEADERS = `${DELETE_RECORD_COLUMN_NAME},${sasparams.COLHEADERS}`
}
/**
* Seeds every row's delete flag to 'No' - getdata no longer sends the column,
* so the editor owns the value too (see deleteRecordColumnRule.ts).
*/
export const seedDeleteRecordColumn = (rows: any[]) => {
for (const row of rows) {
row[DELETE_RECORD_COLUMN_NAME] = 'No'
}
}
@@ -1,16 +1,21 @@
import { DcValidation } from '../models/dc-validation.model'
/**
* `_____DELETE__THIS__RECORD_____` is a client-side UI concept (the delete
* checkbox column) that getdata.sas synthesizes into vars3/vars4 for display
* purposes only — it's never a real column of the underlying dataset, so
* `%mp_getcols` (which builds `cols1`, the left side of the join into
* `cols`) never has a row for it, and its rule can no longer travel via
* `cols[].COLTYPE` (see issue #253's vertical-array fix). The rule is fixed
* and data-independent, so it's injected directly here instead.
* `_____DELETE__THIS__RECORD_____` is a purely client-owned column - the
* delete checkbox column. getdata.sas does NOT emit it: the editor adds the
* name to COLHEADERS itself (see editor.component.ts's initSetup) and seeds
* every row to 'No'. A response that carries it is refused, because the name
* can then only come from a real table column of that name (or a hook that
* re-added it), which would silently shadow the delete flag.
*
* Its rule can never travel via `cols[].COLTYPE` - `%mp_getcols` (which
* builds `cols1`, the left side of the join into `cols`) has no row for it -
* so it is injected directly, here and in DcValidator's constructor.
*/
export const DELETE_RECORD_COLUMN_NAME = '_____DELETE__THIS__RECORD_____'
export const DELETE_RECORD_COLUMN_RULE: DcValidation = {
data: '_____DELETE__THIS__RECORD_____',
data: DELETE_RECORD_COLUMN_NAME,
type: 'dropdown',
source: ['No', 'Yes']
}
+3 -3
View File
@@ -42,11 +42,13 @@
* debug: Whether to enable debug on startup
## Viya only settings ##
* adminGroup: This group will have unrestricted access (Viya Only)
* dcPath: The physical location in which to deploy the staging area (Viya only)
* useComputeApi: Connection method (Viya only)
* contextName: Compute Context (Viya only)
The admin group is not configured here - it is a DC setting (DC_ADMIN_GROUP),
shown on the System screen.
-->
<sasjs
@@ -58,8 +60,6 @@
useComputeApi="null"
runAsTask="true"
contextName="SAS Job Execution compute context"
adminGroup="SASAdministrators"
hotLicenceKey="non-commercial-and-evaluation"
>
</sasjs>
+31 -9
View File
@@ -1664,6 +1664,13 @@ app-requests-modal {
padding: 20px 10px 5px 10px;
}
/* The SAS log pane is `white-space: pre`, so it never wraps. The stock `xl`
width is a fixed 57.6rem, which forced left/right scrolling to read a log
line. */
.requests-modal .modal-dialog {
width: 90vw;
}
.work-tables-dropdown button {
color: var(--clr-nav-link-color, #8c8c8c) !important;
}
@@ -2289,15 +2296,6 @@ app-alerts {
}
// ALERTS.COMPONENT END
// MANUAL-DEPLOY.COMPONENT
app-manual-deploy {
.clear-memory-button {
right: 10px;
top: 2px;
}
}
// MANUAL-DEPLOY.COMPONENT END
// LINEAGE.COMPONENT
app-lineage {
.toggle-switch input[type='checkbox']:checked + label:before {
@@ -4255,6 +4253,30 @@ body[cds-theme='light'] {
color: #c21d00;
}
/* The SASjs tag dump drops onto its own line and reads as a name/value grid. */
&--grid {
flex-wrap: wrap;
align-items: baseline;
}
&__grid {
flex-basis: 100%;
display: grid;
grid-template-columns: auto 1fr;
gap: 1px 12px;
margin-top: 4px;
font-size: 12px;
}
&__grid-name {
color: #999;
}
&__grid-value {
color: #ccc;
word-break: break-word;
}
&__icon--pending {
color: #999;
}
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "dcfrontend",
"version": "7.16.0",
"version": "7.18.0",
"description": "Data Controller",
"devDependencies": {
"@nogoo9/gitleaks": "8.30.1-post.2",
@@ -14,7 +14,6 @@ function makeRows(n) {
const rows = []
for (let i = 0; i < n; i++) {
rows.push({
_____DELETE__THIS__RECORD_____: "No",
PRIMARY_KEY_FIELD: i,
SOME_CHAR: "dummy data row " + i,
SOME_DROPDOWN: dropdowns[i % dropdowns.length],
@@ -241,7 +240,6 @@ const data = {
{ ODS_TABLE: "ENGINEHOST", NAME: "File Size (bytes)", VALUE: "1048576" }
],
maxvarlengths: [
{ NAME: "_____DELETE__THIS__RECORD_____", MAXLEN: 3 },
{ NAME: "PRIMARY_KEY_FIELD", MAXLEN: 4 },
{ NAME: "some_char", MAXLEN: 591 },
{ NAME: "some_dropdown", MAXLEN: 8 },
@@ -261,7 +259,6 @@ const data = {
sasdata: makeRows(100),
$sasdata: {
vars: {
_____DELETE__THIS__RECORD_____: { format: "$3.", label: "_____DELETE__THIS__RECORD_____", length: "3", type: "char" },
PRIMARY_KEY_FIELD: { format: "best.", label: "PRIMARY_KEY_FIELD", length: "8", type: "num" },
SOME_CHAR: { format: "$32767.", label: "SOME_CHAR", length: "32767", type: "char" },
SOME_DROPDOWN: { format: "$128.", label: "SOME_DROPDOWN", length: "128", type: "char" },
@@ -280,14 +277,14 @@ const data = {
},
sasparams: [
{
COLHEADERS: "_____DELETE__THIS__RECORD_____,PRIMARY_KEY_FIELD,SOME_CHAR,SOME_DROPDOWN,SOME_HARDSELECT,SOME_NUM,SOME_DATE,SOME_DATETIME,SOME_TIME,SOME_SHORTNUM,SOME_BESTNUM,READONLY_COL,HIDDEN_COL,ROUND_COL,NUMFMT_COL",
COLHEADERS: "PRIMARY_KEY_FIELD,SOME_CHAR,SOME_DROPDOWN,SOME_HARDSELECT,SOME_NUM,SOME_DATE,SOME_DATETIME,SOME_TIME,SOME_SHORTNUM,SOME_BESTNUM,READONLY_COL,HIDDEN_COL,ROUND_COL,NUMFMT_COL",
FILTER_TEXT: "",
PKCNT: 1,
PK: "PRIMARY_KEY_FIELD",
DTVARS: " SOME_DATE",
DTTMVARS: " SOME_DATETIME",
TMVARS: " SOME_TIME",
COLTYPE: "{\"data\":\"_____DELETE__THIS__RECORD_____\",\"type\":\"dropdown\",\"source\":[\"No\",\"Yes\"]},{\"data\":\"PRIMARY_KEY_FIELD\",\"type\":\"numeric\",\"format\":\"0\"},{\"data\":\"SOME_CHAR\"},{\"data\":\"SOME_DROPDOWN\"},{\"data\":\"SOME_HARDSELECT\"},{\"data\":\"SOME_NUM\",\"type\":\"numeric\",\"format\":\"0\"},{\"data\":\"SOME_DATE\",\"type\":\"date\"},{\"data\":\"SOME_DATETIME\",\"type\":\"datetime\"},{\"data\":\"SOME_TIME\",\"type\":\"time\"},{\"data\":\"SOME_SHORTNUM\",\"type\":\"numeric\",\"format\":\"0\"},{\"data\":\"SOME_BESTNUM\",\"type\":\"numeric\",\"format\":\"0\"},{\"data\":\"READONLY_COL\"},{\"data\":\"HIDDEN_COL\"},{\"data\":\"ROUND_COL\",\"type\":\"numeric\",\"format\":\"0\"},{\"data\":\"NUMFMT_COL\",\"type\":\"numeric\",\"format\":\"0\"}",
COLTYPE: "{\"data\":\"PRIMARY_KEY_FIELD\",\"type\":\"numeric\",\"format\":\"0\"},{\"data\":\"SOME_CHAR\"},{\"data\":\"SOME_DROPDOWN\"},{\"data\":\"SOME_HARDSELECT\"},{\"data\":\"SOME_NUM\",\"type\":\"numeric\",\"format\":\"0\"},{\"data\":\"SOME_DATE\",\"type\":\"date\"},{\"data\":\"SOME_DATETIME\",\"type\":\"datetime\"},{\"data\":\"SOME_TIME\",\"type\":\"time\"},{\"data\":\"SOME_SHORTNUM\",\"type\":\"numeric\",\"format\":\"0\"},{\"data\":\"SOME_BESTNUM\",\"type\":\"numeric\",\"format\":\"0\"},{\"data\":\"READONLY_COL\"},{\"data\":\"HIDDEN_COL\"},{\"data\":\"ROUND_COL\",\"type\":\"numeric\",\"format\":\"0\"},{\"data\":\"NUMFMT_COL\",\"type\":\"numeric\",\"format\":\"0\"}",
LOADTYPE: "UPDATE",
RK_FLAG: 0,
CLS_FLAG: 0
+33
View File
@@ -580,6 +580,7 @@ function makeTableId() {
* ('A','B') -> ["A","B"] (IN list)
* (1,2,3) -> [1,2,3] (IN list, numeric)
* 1 AND 5 -> [1, 5] (BETWEEN)
* '20DEC2053'd-> 34322 (date / datetime / time literal)
*/
function parseFilterRawValue(raw, operator) {
if (raw === undefined || raw === null || raw === '') return null
@@ -598,8 +599,40 @@ function parseFilterRawValue(raw, operator) {
return parseFilterScalar(s)
}
/**
* SAS date / datetime / time literal -> the SAS internal value, or null.
* 34322 <- '20DEC2053'd
* 42 <- '01JAN1960:00:00:42'dt
* 42 <- '00:00:42't
* The client builds these in canonical ddMMMyyyy form (HelperService
* .sasTemporalLiteral), so only that shape needs parsing here.
*/
const TEMPORAL_LITERAL_RE = /^'(\d{2})([A-Z]{3})(\d{4})(?::(\d{2}):(\d{2}):(\d{2})(\.\d+)?)?'(dt|d|t)$/i
/* A time literal carries no date part, so it is matched separately */
const TIME_LITERAL_RE = /^'(\d{2}):(\d{2}):(\d{2})(\.\d+)?'t$/i
function parseTemporalLiteral(s) {
const t = TIME_LITERAL_RE.exec(s)
if (t) {
return Number(t[1]) * 3600 + Number(t[2]) * 60 + Number(t[3]) +
(t[4] ? Number(t[4]) : 0)
}
const m = TEMPORAL_LITERAL_RE.exec(s)
if (!m) return null
const month = MONTHS_ABBR.indexOf(m[2].toUpperCase())
if (month < 0) return null
const days = (Date.UTC(Number(m[3]), month, Number(m[1])) - SAS_EPOCH_MS) / 86400000
const suffix = m[8].toLowerCase()
if (suffix === 'd') return days
const secs = Number(m[4]) * 3600 + Number(m[5]) * 60 + Number(m[6]) +
(m[7] ? Number(m[7]) : 0)
return suffix === 'dt' ? days * 86400 + secs : secs
}
function parseFilterScalar(s) {
s = s.trim()
const temporal = parseTemporalLiteral(s)
if (temporal !== null) return temporal
if (s.startsWith("'") && s.endsWith("'")) {
return s.slice(1, -1).replace(/''/g, "'")
}
+11 -9
View File
@@ -280,16 +280,16 @@ if (tableData && tableData.columns && tableData.rows) {
const targetKey = info ? info.name : key
normalised[targetKey] = info ? formatCellValue(r[key], info.ddtype) : r[key]
}
return { _____DELETE__THIS__RECORD_____: 'No', ...normalised }
return { ...normalised }
})
// $sasdata vars: temporal columns are char ($200.), others keep their SAS type
const vars = { _____DELETE__THIS__RECORD_____: { format: '$3.', label: '_____DELETE__THIS__RECORD_____', length: '3', type: 'char' } }
const vars = {}
for (const col of visibleColumns) {
vars[col.name] = sasVarsEntry(col)
}
const colHeaders = ['_____DELETE__THIS__RECORD_____', ...visibleColumns.map(c => c.name)]
const colHeaders = visibleColumns.map(c => c.name)
// Determine DTVARS, DTTMVARS, TMVARS from column formats
let dtVars = [], dttmVars = [], tmVars = []
@@ -300,11 +300,14 @@ if (tableData && tableData.columns && tableData.rows) {
else if (ddtype === 'TIME') tmVars.push(col.name)
}
// Build cols array. mpGetcols is the JS counterpart of %mp_getcols.sas -
// the macro getdata.sas itself calls to type these columns - so the editor
// sees the same DDTYPE the service would give it.
const cols = mpGetcols(visibleColumns).map((c, i) => {
const raw = visibleColumns[i]
// Build cols array. Every column of the table, as getdata.sas emits them:
// the temporal columns are dropped from the data (the rows below) but not
// from the cols payload, so a filter can still be built on them. mpGetcols
// is the JS counterpart of %mp_getcols.sas - the macro getdata.sas itself
// calls to type these columns - so the editor sees the same DDTYPE the
// service would give it.
const cols = mpGetcols(tableData.columns).map((c, i) => {
const raw = tableData.columns[i]
const ddtype = c.DDTYPE === 'CHARACTER' ? 'C' : c.DDTYPE === 'NUMERIC' ? 'N' : c.DDTYPE
let coltype
if (ddtype === 'DATE') coltype = `{"data":"${raw.name}","type":"date"}`
@@ -327,7 +330,6 @@ if (tableData && tableData.columns && tableData.rows) {
// Build maxvarlengths
const maxvarlengths = [
{ NAME: '_____DELETE__THIS__RECORD_____', MAXLEN: 3 },
...visibleColumns.map(col => {
let maxLen = 0
const lcName = col.name.toLowerCase()
+11 -25
View File
@@ -43,42 +43,28 @@ if (!tableData) {
}
// ─── Build the column list ────────────────────────────────────────────────────
// The picker's variable list, in the same shape the editor gets from getdata:
// NAME / LABEL / DDTYPE / FMTNAME, with the temporal columns excluded exactly as
// the editor excludes them, so the fallback list matches the one the table
// itself would have supplied.
// Every column, as getdata.sas emits them: the temporal columns are dropped from
// the data, not from the cols payload, so a filter can still be built on them -
// which is exactly what "please filter and try again" wants to do.
let cols = []
if (tableData && tableData.columns) {
// MPE_TABLES always lives in the control library (DC_JSLIB / mpelib), not
// in user data libraries - mirroring getdata.js.
const mpeTablesData = makeTableLoader(libDataDir(dcLibref))('MPE_TABLES')
const tableReg = mpeTablesData && mpeTablesData.rows
? mpeTablesData.rows.find(r => r.libref === libref && r.dsn === table)
: null
const excludeCols = new Set()
if (tableReg && (tableReg.loadtype === 'TXTEMPORAL' || tableReg.loadtype === 'BITEMPORAL')) {
if (tableReg.var_txfrom) excludeCols.add(tableReg.var_txfrom.toUpperCase())
if (tableReg.var_txto) excludeCols.add(tableReg.var_txto.toUpperCase())
}
if (tableReg && tableReg.loadtype === 'BITEMPORAL') {
if (tableReg.var_busfrom) excludeCols.add(tableReg.var_busfrom.toUpperCase())
if (tableReg.var_busto) excludeCols.add(tableReg.var_busto.toUpperCase())
}
// mpGetcols, then the same CHARACTER/NUMERIC -> C/N mapping getcols.sas
// applies on top of %mp_getcols, so the picker's fallback list is typed the
// way the editor's own load types it.
cols = mpGetcols(tableData.columns.filter(c => !excludeCols.has(c.name.toUpperCase())))
// mpGetcols, then the mapping getcols.sas applies on top of %mp_getcols:
// DDTYPE in the client's vocabulary, and TYPE as the SAS C/N it has always
// been. The client maps TYPE to its own num/char vocabulary in
// sas-store.getCols, because the picker keys its operator set and its value
// quoting on it and this list does not pass through mergeColsRules.
cols = mpGetcols(tableData.columns)
.map(c => {
const ddtype = c.DDTYPE === 'CHARACTER' ? 'C' : c.DDTYPE === 'NUMERIC' ? 'N' : c.DDTYPE
return {
NAME: c.NAME,
LABEL: c.LABEL,
FORMAT: c.FORMAT || ' ',
FMTNAME: c.FMTNAME || ' ',
DDTYPE: ddtype,
TYPE: ddtype === 'C' ? 'C' : 'N',
CLS_RULE: 'READ',
MEMLABEL: ' ',
DESC: ' ',
+123 -279
View File
@@ -6,8 +6,8 @@
"": {
"name": "dc-sas",
"dependencies": {
"@sasjs/cli": "4.20.5",
"@sasjs/core": "5.2.8"
"@sasjs/cli": "4.20.9",
"@sasjs/core": "5.5.2"
}
},
"node_modules/@asamuzakjp/css-color": {
@@ -154,32 +154,63 @@
}
},
"node_modules/@sasjs/adapter": {
"version": "4.19.1",
"resolved": "https://registry.npmjs.org/@sasjs/adapter/-/adapter-4.19.1.tgz",
"integrity": "sha512-IpN6UPSdqVb5cAWEBt+jG/i0rhgCubdyp5kxdouuX3ifL313oJW2ZPVUNUvs0+ZGzsXWcGDTNoEzzTrGKiqJlQ==",
"version": "4.19.2",
"resolved": "https://registry.npmjs.org/@sasjs/adapter/-/adapter-4.19.2.tgz",
"integrity": "sha512-7uxkpk0kjK4PZ552YK0CsTmEKEurK/1uzloR4rxplbViRkVdaewP5l0hIrGDBVM4z0gmV5pf1jh8uil3/NSAzw==",
"license": "ISC",
"dependencies": {
"@sasjs/utils": "^3.6.0",
"@sasjs/utils": "3.6.4",
"axios": "1.20.0",
"axios-cookiejar-support": "5.0.5",
"form-data": "4.0.6",
"https": "1.0.0",
"tough-cookie": "4.1.3"
}
},
"node_modules/@sasjs/cli": {
"version": "4.20.5",
"resolved": "https://registry.npmjs.org/@sasjs/cli/-/cli-4.20.5.tgz",
"integrity": "sha512-/HSpueCFS+SvXas9FwfP8qoD9skhdTDE5r7dT9Xlrs7HGCBjxDYvSiw6tW0GIjfw3/49Yf8TCVDFBsISang1hg==",
"node_modules/@sasjs/adapter/node_modules/@sasjs/utils": {
"version": "3.6.4",
"resolved": "https://registry.npmjs.org/@sasjs/utils/-/utils-3.6.4.tgz",
"integrity": "sha512-tLbvIFMRb3KrHW5EapvMS/0tSa6W/SYCuR+fcdavayQ4eaUFayeRN3HKu2gXLtWJjHf/HijPVTsNwkIRwDvJMA==",
"license": "ISC",
"dependencies": {
"@sasjs/adapter": "4.19.1",
"@sasjs/core": "5.2.10",
"@sasjs/lint": "4.1.0",
"@sasjs/utils": "3.6.2",
"@fast-csv/format": "4.3.5",
"@types/fs-extra": "11.0.4",
"@types/prompts": "2.4.9",
"chalk": "4.1.1",
"cli-table": "0.3.11",
"consola": "3.4.2",
"fs-extra": "11.4.1",
"jwt-decode": "3.1.2",
"prompts": "2.4.2"
}
},
"node_modules/@sasjs/adapter/node_modules/chalk": {
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/chalk/-/chalk-4.1.1.tgz",
"integrity": "sha512-diHzdDKxcU+bAsUboHLPEDQiw0qEe0qd7SYUn3HgcFlWgbDcfLGswOHYeGrHKzG9z6UYf01d9VFMfZxPM1xZSg==",
"license": "MIT",
"dependencies": {
"ansi-styles": "^4.1.0",
"supports-color": "^7.1.0"
},
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/chalk/chalk?sponsor=1"
}
},
"node_modules/@sasjs/cli": {
"version": "4.20.9",
"resolved": "https://registry.npmjs.org/@sasjs/cli/-/cli-4.20.9.tgz",
"integrity": "sha512-wJCdd6fRI0RVwEDLDsauUbsKQsH69aaYJqDkV9WfSKkRH1VxgU9zFGHdSxrKLOzrjwwybn/DGzh2xcgeeyl6jA==",
"license": "ISC",
"dependencies": {
"@sasjs/adapter": "4.19.2",
"@sasjs/core": "5.3.0",
"@sasjs/lint": "4.1.2",
"@sasjs/utils": "3.6.5",
"chalk": "4.1.2",
"dotenv": "17.4.2",
"find": "0.3.0",
"js-base64": "3.9.3",
"jsdom": "26.0.0",
"jwt-decode": "3.1.2",
@@ -189,7 +220,6 @@
"node-powershell": "5.0.1",
"ora": "5.4.1",
"prompts": "2.4.2",
"shelljs": "0.10.0",
"ssl-root-cas": "1.3.1",
"xml": "1.0.1",
"yargs": "17.6.2",
@@ -198,65 +228,84 @@
},
"bin": {
"sasjs": "build/index.js"
},
"engines": {
"node": "^14.18.0 || >=16.10.0"
}
},
"node_modules/@sasjs/cli/node_modules/@sasjs/core": {
"version": "5.2.10",
"resolved": "https://registry.npmjs.org/@sasjs/core/-/core-5.2.10.tgz",
"integrity": "sha512-Lk/ENHVuXaeAl8M0+Mjh1rcBSdFt4tIM6OPxPPmk78ZD0gYFqAOUZreqIoV9EN+VwPDqYsUPDqXzh0crW6j8OA==",
"version": "5.3.0",
"resolved": "https://registry.npmjs.org/@sasjs/core/-/core-5.3.0.tgz",
"integrity": "sha512-6x1hEPa1pCENyeG0Nro3YcZZ+A8Ns60pO6gDuvG1X6oWdwOKKPpZztQNKE6khFmXXIGl0uUgT9CbappUET+odA==",
"license": "MIT"
},
"node_modules/@sasjs/cli/node_modules/shelljs": {
"version": "0.8.5",
"resolved": "https://registry.npmjs.org/shelljs/-/shelljs-0.8.5.tgz",
"integrity": "sha512-TiwcRcrkhHvbrZbnRcFYMLl30Dfov3HKqzp5tO5b4pt6G/SezKcYhmDg15zXVBswHmctSAQKznqNW2LO5tTDow==",
"license": "BSD-3-Clause",
"dependencies": {
"glob": "^7.0.0",
"interpret": "^1.0.0",
"rechoir": "^0.6.2"
},
"bin": {
"shjs": "bin/shjs"
},
"engines": {
"node": ">=4"
}
},
"node_modules/@sasjs/core": {
"version": "5.2.8",
"resolved": "https://registry.npmjs.org/@sasjs/core/-/core-5.2.8.tgz",
"integrity": "sha512-+K8tXTrsw52NR1y5MFHW0+Jf9euL+GukwqA79FIHHu4Oj+BhHBm1uZiyB5tPrvwl3V8mY1CQhMOT83//bV+jWg==",
"version": "5.5.2",
"resolved": "https://registry.npmjs.org/@sasjs/core/-/core-5.5.2.tgz",
"integrity": "sha512-puojJhAKAKg5o0uHdGSkpQ3cXgzajMY9AXAm3R22l6bZ4GKvC03qRq1ueGjBBF3npNxviwIKCABTsNEtjs1LAw==",
"license": "MIT"
},
"node_modules/@sasjs/lint": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/@sasjs/lint/-/lint-4.1.0.tgz",
"integrity": "sha512-Vpg1s0/zdPMFJ0hRXNXk53HhE3kfRr5gFRS0kvrse+bxgcbqih1qqVW7yZ2krhSCYylp9z13eBXLhXbRrj70VQ==",
"hasInstallScript": true,
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/@sasjs/lint/-/lint-4.1.2.tgz",
"integrity": "sha512-P7QqeMPnhg/AvqH40dmyY2OArlvAYAp81/ijK/PhmS+x/HslpypdLoNy5iN7SwixUWPde5fPwV/ZBmcjVK0uLA==",
"license": "MIT",
"dependencies": {
"@sasjs/utils": "3.6.2",
"@sasjs/utils": "3.6.4",
"ignore": "7.0.11"
},
"engines": {
"node": "^14.18.0 || >=16.10.0"
}
},
"node_modules/@sasjs/utils": {
"version": "3.6.2",
"resolved": "https://registry.npmjs.org/@sasjs/utils/-/utils-3.6.2.tgz",
"integrity": "sha512-AfC5OJuJx3NAbntKq2J8E8NNpRXthYfuqOyp2lOmOkTctL3mAfrA54U/cdPQvrlftXNxpK9P2U0RU10qNjlEzQ==",
"node_modules/@sasjs/lint/node_modules/@sasjs/utils": {
"version": "3.6.4",
"resolved": "https://registry.npmjs.org/@sasjs/utils/-/utils-3.6.4.tgz",
"integrity": "sha512-tLbvIFMRb3KrHW5EapvMS/0tSa6W/SYCuR+fcdavayQ4eaUFayeRN3HKu2gXLtWJjHf/HijPVTsNwkIRwDvJMA==",
"license": "ISC",
"dependencies": {
"@fast-csv/format": "4.3.5",
"@types/fs-extra": "11.0.4",
"@types/prompts": "2.0.13",
"@types/prompts": "2.4.9",
"chalk": "4.1.1",
"cli-table": "0.3.6",
"consola": "2.15.0",
"find": "0.3.0",
"fs-extra": "11.3.0",
"cli-table": "0.3.11",
"consola": "3.4.2",
"fs-extra": "11.4.1",
"jwt-decode": "3.1.2",
"prompts": "2.4.1",
"valid-url": "1.0.9"
"prompts": "2.4.2"
}
},
"node_modules/@sasjs/lint/node_modules/chalk": {
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/chalk/-/chalk-4.1.1.tgz",
"integrity": "sha512-diHzdDKxcU+bAsUboHLPEDQiw0qEe0qd7SYUn3HgcFlWgbDcfLGswOHYeGrHKzG9z6UYf01d9VFMfZxPM1xZSg==",
"license": "MIT",
"dependencies": {
"ansi-styles": "^4.1.0",
"supports-color": "^7.1.0"
},
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/chalk/chalk?sponsor=1"
}
},
"node_modules/@sasjs/utils": {
"version": "3.6.5",
"resolved": "https://registry.npmjs.org/@sasjs/utils/-/utils-3.6.5.tgz",
"integrity": "sha512-2iCy0EmZJCQSG1JsVa1t/ulUb7lfdBnp6wiRPRl9/AaxRu8D30ypqUKn6heos1lxhNxg42JFsry8rlupiajdHg==",
"license": "ISC",
"dependencies": {
"@fast-csv/format": "4.3.5",
"@types/fs-extra": "11.0.4",
"@types/prompts": "2.4.9",
"chalk": "4.1.1",
"cli-table": "0.3.11",
"consola": "3.4.2",
"fs-extra": "11.4.1",
"jwt-decode": "3.1.2",
"prompts": "2.4.2"
}
},
"node_modules/@sasjs/utils/node_modules/chalk": {
@@ -275,19 +324,6 @@
"url": "https://github.com/chalk/chalk?sponsor=1"
}
},
"node_modules/@sasjs/utils/node_modules/prompts": {
"version": "2.4.1",
"resolved": "https://registry.npmjs.org/prompts/-/prompts-2.4.1.tgz",
"integrity": "sha512-EQyfIuO2hPDsX1L/blblV+H7I0knhgAd82cVneCwcdND9B8AuCDuRcBH6yIcG4dFzlOUqbazQqwGjx5xmsNLuQ==",
"license": "MIT",
"dependencies": {
"kleur": "^3.0.3",
"sisteransi": "^1.0.5"
},
"engines": {
"node": ">= 6"
}
},
"node_modules/@types/fs-extra": {
"version": "11.0.4",
"resolved": "https://registry.npmjs.org/@types/fs-extra/-/fs-extra-11.0.4.tgz",
@@ -314,12 +350,13 @@
"license": "MIT"
},
"node_modules/@types/prompts": {
"version": "2.0.13",
"resolved": "https://registry.npmjs.org/@types/prompts/-/prompts-2.0.13.tgz",
"integrity": "sha512-jwMOIGy49VruR/gYehhJYgpVzB+EVpEE7t7j9m1oTo4HMpOe7KmsyqdBuoxAzA5B4caUgx0cKrWr7wUEqMXJ7Q==",
"version": "2.4.9",
"resolved": "https://registry.npmjs.org/@types/prompts/-/prompts-2.4.9.tgz",
"integrity": "sha512-qTxFi6Buiu8+50/+3DGIWLHM6QuWsEKugJnnP6iv2Mc4ncxE4A/OJkjuVOA+5X0X1S/nq5VJRa8Lu+nwcvbrKA==",
"license": "MIT",
"dependencies": {
"@types/node": "*"
"@types/node": "*",
"kleur": "^3.0.3"
}
},
"node_modules/accumulate-stream": {
@@ -402,12 +439,6 @@
"tough-cookie": ">=4.0.0"
}
},
"node_modules/balanced-match": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-1.0.2.tgz",
"integrity": "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw==",
"license": "MIT"
},
"node_modules/base64-js": {
"version": "1.5.1",
"resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz",
@@ -439,16 +470,6 @@
"readable-stream": "^3.4.0"
}
},
"node_modules/brace-expansion": {
"version": "1.1.21",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.21.tgz",
"integrity": "sha512-9zeA+KLZNNzglF2TPKRQEDyx6Yby7daAkuy8MiPzpXPsYDWi/DRM8jmwUDxokQjYqBpv5DgPiwD4h4ZZSy1Ujw==",
"license": "MIT",
"dependencies": {
"balanced-match": "^1.0.0",
"concat-map": "0.0.1"
}
},
"node_modules/buffer": {
"version": "5.7.1",
"resolved": "https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz",
@@ -560,9 +581,9 @@
}
},
"node_modules/cli-table": {
"version": "0.3.6",
"resolved": "https://registry.npmjs.org/cli-table/-/cli-table-0.3.6.tgz",
"integrity": "sha512-ZkNZbnZjKERTY5NwC2SeMeLeifSPq/pubeRoTpdr3WchLlnZg6hEgvHkK5zL7KNFdd9PmHN8lxrENUwI3cE8vQ==",
"version": "0.3.11",
"resolved": "https://registry.npmjs.org/cli-table/-/cli-table-0.3.11.tgz",
"integrity": "sha512-IqLQi4lO0nIB4tcdTpN4LCB9FI3uqrJZK7RC515EnhZ6qBaglkIgICb1wjeAqpdoOabm1+SuQtkXIPdYC93jhQ==",
"dependencies": {
"colors": "1.0.3"
},
@@ -632,17 +653,14 @@
"node": ">= 0.8"
}
},
"node_modules/concat-map": {
"version": "0.0.1",
"resolved": "https://registry.npmjs.org/concat-map/-/concat-map-0.0.1.tgz",
"integrity": "sha512-/Srv4dswyQNBfohGpz9o6Yb3Gz3SrUDqBH5rTuhGR7ahtlbYKnVxw2bCFMRljaA7EXHaXZ8wsHdodFvbkhKmqg==",
"license": "MIT"
},
"node_modules/consola": {
"version": "2.15.0",
"resolved": "https://registry.npmjs.org/consola/-/consola-2.15.0.tgz",
"integrity": "sha512-vlcSGgdYS26mPf7qNi+dCisbhiyDnrN1zaRbw3CSuc2wGOMEGGPsp46PdRG5gqXwgtJfjxDkxRNAgRPr1B77vQ==",
"license": "MIT"
"version": "3.4.2",
"resolved": "https://registry.npmjs.org/consola/-/consola-3.4.2.tgz",
"integrity": "sha512-5IKcdX0nnYavi6G7TtOhwkYzyjfJlatbjMjuLSfE2kYT5pMDOilZ4OvMhi637CcDICTmz3wARPoyhqyX1Y+XvA==",
"license": "MIT",
"engines": {
"node": "^14.18.0 || >=16.10.0"
}
},
"node_modules/cssstyle": {
"version": "4.6.0",
@@ -818,15 +836,6 @@
"integrity": "sha512-8guHBZCwKnFhYdHr2ysuRWErTwhoN2X8XELRlrRwpmfeY2jjuUN4taQMsULKUVo1K4DvZl+0pgfyoysHxvmvEw==",
"license": "MIT"
},
"node_modules/find": {
"version": "0.3.0",
"resolved": "https://registry.npmjs.org/find/-/find-0.3.0.tgz",
"integrity": "sha512-iSd+O4OEYV/I36Zl8MdYJO0xD82wH528SaCieTVHhclgiYNe9y+yPKSwK+A7/WsmHL1EZ+pYUJBXWTL5qofksw==",
"license": "MIT",
"dependencies": {
"traverse-chain": "~0.1.0"
}
},
"node_modules/follow-redirects": {
"version": "1.16.1",
"resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.16.1.tgz",
@@ -864,9 +873,9 @@
}
},
"node_modules/fs-extra": {
"version": "11.3.0",
"resolved": "https://registry.npmjs.org/fs-extra/-/fs-extra-11.3.0.tgz",
"integrity": "sha512-Z4XaCL6dUDHfP/jT25jJKMmtxvuwbkrD1vNSMFlo9lNLY2c5FHYSQgHPRZUjAB26TpDEoW9HCOgplrdbaPV/ew==",
"version": "11.4.1",
"resolved": "https://registry.npmjs.org/fs-extra/-/fs-extra-11.4.1.tgz",
"integrity": "sha512-KYAb4c9BJQI6QqGKthV68OHe0badztdXJWKo0WtBA9IuCFPTKvE5ZdUBglP833aMjhaSPNO4A5j/EkzZtGlKjA==",
"license": "MIT",
"dependencies": {
"graceful-fs": "^4.2.0",
@@ -877,12 +886,6 @@
"node": ">=14.14"
}
},
"node_modules/fs.realpath": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/fs.realpath/-/fs.realpath-1.0.0.tgz",
"integrity": "sha512-OO0pH2lK6a0hZnAdau5ItzHPI6pUlvI7jMVnxUQRtw4owF2wk8lOSabtGDCTP4Ggrg2MbGnWO9X8K1t4+fGMDw==",
"license": "ISC"
},
"node_modules/function-bind": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz",
@@ -938,27 +941,6 @@
"node": ">= 0.4"
}
},
"node_modules/glob": {
"version": "7.2.3",
"resolved": "https://registry.npmjs.org/glob/-/glob-7.2.3.tgz",
"integrity": "sha512-nFR0zLpU2YCaRxwoCJvL6UvCH2JFyFVIvwTLsIf21AuHlMskA1hhTdk+LlYJtOlYt9v6dvszD2BGRqBL+iQK9Q==",
"deprecated": "Old versions of glob are not supported, and contain widely publicized security vulnerabilities, which have been fixed in the current version. Please update. Support for old versions may be purchased (at exorbitant rates) by contacting i@izs.me",
"license": "ISC",
"dependencies": {
"fs.realpath": "^1.0.0",
"inflight": "^1.0.4",
"inherits": "2",
"minimatch": "^3.1.1",
"once": "^1.3.0",
"path-is-absolute": "^1.0.0"
},
"engines": {
"node": "*"
},
"funding": {
"url": "https://github.com/sponsors/isaacs"
}
},
"node_modules/gopd": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/gopd/-/gopd-1.2.0.tgz",
@@ -1074,12 +1056,6 @@
"node": ">= 14"
}
},
"node_modules/https": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/https/-/https-1.0.0.tgz",
"integrity": "sha512-4EC57ddXrkaF0x83Oj8sM6SLQHAWXw90Skqu2M4AEWENZ3F02dFJE/GARA8igO79tcgYqGrD7ae4f5L3um2lgg==",
"license": "ISC"
},
"node_modules/https-proxy-agent": {
"version": "5.0.1",
"resolved": "https://registry.npmjs.org/https-proxy-agent/-/https-proxy-agent-5.0.1.tgz",
@@ -1146,47 +1122,12 @@
"node": ">= 4"
}
},
"node_modules/inflight": {
"version": "1.0.6",
"resolved": "https://registry.npmjs.org/inflight/-/inflight-1.0.6.tgz",
"integrity": "sha512-k92I/b08q4wvFscXCLvqfsHCrjrF7yiXsQuIVvVE7N82W3+aqpzuUdBbfhWcy/FZR3/4IgflMgKLOsvPDrGCJA==",
"deprecated": "This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful.",
"license": "ISC",
"dependencies": {
"once": "^1.3.0",
"wrappy": "1"
}
},
"node_modules/inherits": {
"version": "2.0.4",
"resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz",
"integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==",
"license": "ISC"
},
"node_modules/interpret": {
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/interpret/-/interpret-1.4.0.tgz",
"integrity": "sha512-agE4QfB2Lkp9uICn7BAqoscw4SZP9kTE2hxiFI3jBPmXJfdqiahTbUuKGsMoN2GtqL9AxhYioAcVvgsb1HvRbA==",
"license": "MIT",
"engines": {
"node": ">= 0.10"
}
},
"node_modules/is-core-module": {
"version": "2.17.0",
"resolved": "https://registry.npmjs.org/is-core-module/-/is-core-module-2.17.0.tgz",
"integrity": "sha512-J/vG0zBCbIKOQFfufSwyXdMrsohyJIUNkrnmo6WZGzoM7tr/lsbfW5b2BvisL6zsyMzK9UxV9L6c7AoFbyXHOA==",
"license": "MIT",
"dependencies": {
"hasown": "^2.0.4"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/is-docker": {
"version": "2.2.1",
"resolved": "https://registry.npmjs.org/is-docker/-/is-docker-2.2.1.tgz",
@@ -1461,18 +1402,6 @@
"node": ">=6"
}
},
"node_modules/minimatch": {
"version": "3.1.5",
"resolved": "https://registry.npmjs.org/minimatch/-/minimatch-3.1.5.tgz",
"integrity": "sha512-VgjWUsnnT6n+NUk6eZq77zeFdpW2LWDzP6zFGrCbHXiYNul5Dzqk2HHQ5uFH2DNW5Xbp8+jVzaeNt94ssEEl4w==",
"license": "ISC",
"dependencies": {
"brace-expansion": "^1.1.7"
},
"engines": {
"node": "*"
}
},
"node_modules/ms": {
"version": "2.1.3",
"resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
@@ -1519,15 +1448,6 @@
"integrity": "sha512-gQPNF78qebCQ6tvVFBYrvJdBNOrYZm90ZlXgpIFm06p6qHDHq/XC4TnJftN6OMbxVE0UTBAoRgcsDeJBBooITw==",
"license": "MIT"
},
"node_modules/once": {
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz",
"integrity": "sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==",
"license": "ISC",
"dependencies": {
"wrappy": "1"
}
},
"node_modules/onetime": {
"version": "5.1.2",
"resolved": "https://registry.npmjs.org/onetime/-/onetime-5.1.2.tgz",
@@ -1624,21 +1544,6 @@
"url": "https://github.com/inikulin/parse5?sponsor=1"
}
},
"node_modules/path-is-absolute": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/path-is-absolute/-/path-is-absolute-1.0.1.tgz",
"integrity": "sha512-AVbw3UJ2e9bq64vSaS9Am0fje1Pa8pbGqTTsmXfaIiMpnr5DlDhfJOuLj9Sf95ZPVDAUerDfEk88MPmPe7UCQg==",
"license": "MIT",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/path-parse": {
"version": "1.0.7",
"resolved": "https://registry.npmjs.org/path-parse/-/path-parse-1.0.7.tgz",
"integrity": "sha512-LDJzPVEEEPR+y48z93A0Ed0yXb8pAByGWo/k5YYdYgpY2/2EsOsksJrq7lOHxryrVOn1ejG6oAp8ahvOIQD8sw==",
"license": "MIT"
},
"node_modules/pend": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/pend/-/pend-1.2.0.tgz",
@@ -1708,17 +1613,6 @@
"node": ">= 6"
}
},
"node_modules/rechoir": {
"version": "0.6.2",
"resolved": "https://registry.npmjs.org/rechoir/-/rechoir-0.6.2.tgz",
"integrity": "sha512-HFM8rkZ+i3zrV+4LQjwQ0W+ez98pApMGM3HUrN04j3CqzPOzl9nmP15Y8YXNm8QHGv/eacOVEjqhmWpkRV0NAw==",
"dependencies": {
"resolve": "^1.1.6"
},
"engines": {
"node": ">= 0.10"
}
},
"node_modules/require-directory": {
"version": "2.1.1",
"resolved": "https://registry.npmjs.org/require-directory/-/require-directory-2.1.1.tgz",
@@ -1734,27 +1628,6 @@
"integrity": "sha512-KigOCHcocU3XODJxsu8i/j8T9tzT4adHiecwORRQ0ZZFcp7ahwXuRU1m+yuO90C5ZUyGeGfocHDI14M3L3yDAQ==",
"license": "MIT"
},
"node_modules/resolve": {
"version": "1.22.12",
"resolved": "https://registry.npmjs.org/resolve/-/resolve-1.22.12.tgz",
"integrity": "sha512-TyeJ1zif53BPfHootBGwPRYT1RUt6oGWsaQr8UyZW/eAm9bKoijtvruSDEmZHm92CwS9nj7/fWttqPCgzep8CA==",
"license": "MIT",
"dependencies": {
"es-errors": "^1.3.0",
"is-core-module": "^2.16.1",
"path-parse": "^1.0.7",
"supports-preserve-symlinks-flag": "^1.0.0"
},
"bin": {
"resolve": "bin/resolve"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/restore-cursor": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/restore-cursor/-/restore-cursor-3.1.0.tgz",
@@ -1880,18 +1753,6 @@
"node": ">=8"
}
},
"node_modules/supports-preserve-symlinks-flag": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/supports-preserve-symlinks-flag/-/supports-preserve-symlinks-flag-1.0.0.tgz",
"integrity": "sha512-ot0WnXS9fgdkgIcePe6RHNk1WA8+muPa6cSjeR3V8K27q9BB1rTE3R1p7Hv0z1ZyAc8s6Vvv8DIyWf681MAt0w==",
"license": "MIT",
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/symbol-tree": {
"version": "3.2.4",
"resolved": "https://registry.npmjs.org/symbol-tree/-/symbol-tree-3.2.4.tgz",
@@ -1952,12 +1813,6 @@
"node": ">=18"
}
},
"node_modules/traverse-chain": {
"version": "0.1.0",
"resolved": "https://registry.npmjs.org/traverse-chain/-/traverse-chain-0.1.0.tgz",
"integrity": "sha512-up6Yvai4PYKhpNp5PkYtx50m3KbwQrqDwbuZP/ItyL64YEWHAvH6Md83LFLV/GRSk/BoUVwwgUzX6SOQSbsfAg==",
"license": "MIT"
},
"node_modules/trim-buffer": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/trim-buffer/-/trim-buffer-5.0.0.tgz",
@@ -1989,11 +1844,6 @@
"integrity": "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==",
"license": "MIT"
},
"node_modules/valid-url": {
"version": "1.0.9",
"resolved": "https://registry.npmjs.org/valid-url/-/valid-url-1.0.9.tgz",
"integrity": "sha512-QQDsV8OnSf5Uc30CKSwG9lnhMPe6exHtTXLRYX8uMwKENy640pU+2BgBL0LRbDh/eYRahNCS7aewCx0wf3NYVA=="
},
"node_modules/w3c-xmlserializer": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/w3c-xmlserializer/-/w3c-xmlserializer-5.0.0.tgz",
@@ -2076,12 +1926,6 @@
"url": "https://github.com/chalk/wrap-ansi?sponsor=1"
}
},
"node_modules/wrappy": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
"integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
"license": "ISC"
},
"node_modules/ws": {
"version": "8.21.3",
"resolved": "https://registry.npmjs.org/ws/-/ws-8.21.3.tgz",
+3 -4
View File
@@ -30,11 +30,10 @@
},
"private": true,
"dependencies": {
"@sasjs/cli": "4.20.5",
"@sasjs/core": "5.2.8"
"@sasjs/cli": "4.20.9",
"@sasjs/core": "5.5.2"
},
"overrides": {
"nanoid": "3.3.18",
"shelljs": "0.8.5"
"nanoid": "3.3.18"
}
}
@@ -23,6 +23,10 @@
only for options that are not present at all - an existing install carries
its own rows, so it keeps 500 / 100 until this is run.
Each block below uses %do/%end on BOTH branches. A single-statement
%then/%else in open code (outside a %macro) is rejected by the Viya compute
engine with "Expected %DO not found", so it must not be shortened back.
**/
%let dclib=YOURDCLIB;
@@ -60,7 +64,9 @@ quit;
!!' loaded into the browser in the VIEW screen.';
quit;
%end;
%else %put NOTE: DC_MAXOBS_WEBVIEW is switched off (var_active=0) - not changed;
%else %do;
%put NOTE: DC_MAXOBS_WEBVIEW is switched off (var_active=0) - not changed;
%end;
%if &off_edit=0 %then %do;
proc sql noprint;
@@ -81,7 +87,9 @@ quit;
!!' is reached first.';
quit;
%end;
%else %put NOTE: DC_MAXOBS_WEBEDIT is switched off (var_active=0) - not changed;
%else %do;
%put NOTE: DC_MAXOBS_WEBEDIT is switched off (var_active=0) - not changed;
%end;
%if &off_cells=0 %then %do;
proc sql noprint;
@@ -89,6 +97,10 @@ quit;
set tx_to=&now
where var_name='DC_MAXCELLS_WEBEDIT' and var_active=1 and &now lt tx_to;
/* Why the cell limit exists, kept out of var_desc - MPE_CONFIG.VAR_DESC
is varchar(300), and a truncation WARNING aborts the build. A wide
table is expensive long before it is tall - 250 rows of a 1000 column
table is a quarter of a million cells. */
insert into &dclib..mpe_config
set tx_from=&now
,tx_to='31DEC9999:23:59:59'dt
@@ -98,10 +110,10 @@ quit;
,var_active=1
,var_desc='This sets the maximum number of CELLS (rows multiplied by'
!!' columns) that can be loaded into the browser for editing in the'
!!' EDIT screen. It exists because a wide table is expensive long'
!!' before it is tall - 250 rows of a 1000 column table is a quarter'
!!' of a million cells. Whichever of this and DC_MAXOBS_WEBEDIT is'
!!' reached first applies.';
!!' EDIT screen. Whichever of this and DC_MAXOBS_WEBEDIT is reached'
!!' first applies.';
quit;
%end;
%else %put NOTE: DC_MAXCELLS_WEBEDIT is switched off (var_active=0) - not changed;
%else %do;
%put NOTE: DC_MAXCELLS_WEBEDIT is switched off (var_active=0) - not changed;
%end;
+5 -3
View File
@@ -79,6 +79,10 @@ insert into &lib..mpe_config set
!!' will require a decent browser (ie, not IE) and more memory on the'
!!' client side. The DC_MAXCELLS_WEBEDIT limit is applied at the same'
!!' time, whichever is reached first.';
/* Why the cell limit exists, kept out of var_desc - MPE_CONFIG.VAR_DESC is
varchar(300), and a truncation WARNING aborts this whole build. A wide
table is expensive long before it is tall - 250 rows of a 1000 column
table is a quarter of a million cells. */
insert into &lib..mpe_config set
tx_from=0
,tx_to='31DEC9999:23:59:59'dt
@@ -88,9 +92,7 @@ insert into &lib..mpe_config set
,var_active=1
,var_desc='This sets the maximum number of CELLS (rows multiplied by'
!!' columns) that can be loaded into the browser for editing in the EDIT'
!!' screen. It exists because a wide table is expensive long before it'
!!' is tall - 250 rows of a 1000 column table is a quarter of a million'
!!' cells. Whichever of this and DC_MAXOBS_WEBEDIT is reached first'
!!' screen. Whichever of this and DC_MAXOBS_WEBEDIT is reached first'
!!' applies.';
insert into &lib..mpe_config set
tx_from=0
@@ -98,8 +98,25 @@ data work.results3;
end;
run;
/* scan the whole file for any INSERT - TSQL is DDL only */
data work.results4;
infile web3;
input;
if index(upcase(_infile_),'INSERT INTO') then do;
putlog 'unexpected INSERT found';
output;
stop;
end;
run;
%mp_assertdsobs(work.results3,
desc=TSQL flavour DDL file is successfully returned,
test=EQUALS 1,
outds=work.test_results
)
%mp_assertdsobs(work.results4,
desc=TSQL flavour export contains no INSERTs (DDL only),
test=EQUALS 0,
outds=work.test_results
)
+39 -22
View File
@@ -55,6 +55,8 @@
@li mf_getvarlist.sas
@li mf_existds.sas
@li mf_getquotedstr.sas
@li mf_getuniquefileref.sas
@li mf_getuniquename.sas
@li mf_getuser.sas
@li mf_nobs.sas
@li mf_verifymacvars.sas
@@ -235,7 +237,6 @@ select upcase(loadtype)
%if &loadtype=BITEMPORAL %then %do;
data out (drop=&var_txfrom &var_txto &var_processed &drop_rk );
_____DELETE__THIS__RECORD_____="No";
set &libds;
where %inc filtref;;
run;
@@ -250,7 +251,6 @@ select upcase(loadtype)
%end;
%else %do;
data out (drop=&var_txfrom &var_txto &var_processed &drop_rk);
_____DELETE__THIS__RECORD_____="No";
set &libds;
where %inc filtref;;
run;
@@ -288,23 +288,27 @@ select upcase(loadtype)
)
/* order delete var and pk fields at start of table */
/* order pk fields at start of table. The delete flag is NOT added here:
_____DELETE__THIS__RECORD_____ is a client-owned column, created by the
editor itself (see deleteRecordColumnRule.ts), and getdata must never
emit it - a real table column of that name is refused by the frontend. */
%let sourcevars=%mf_wordsInStr1ButNotStr2(
Str1=%mf_getvarlist(work.out)
,Str2= _____DELETE__THIS__RECORD_____ &pk
,Str2=&pk
);
%put sourcevars=&sourcevars;
/* the payload is capped here, at the row limit - the limits above are
checked against the whole selection, this is what goes to the browser */
options obs=&DC_MAXOBS_WEBEDIT;
data outdata;
/* delete & pk fields come first */
attrib _____DELETE__THIS__RECORD_____ &pk label='';
/* pk fields come first */
%if %length(&pk)>0 %then %do;
attrib &pk label='';
%end;
/* keep remaining variable order */
%if %length(&sourcevars)>0 %then %do;
attrib &sourcevars label='';
%end;
_____DELETE__THIS__RECORD_____="No ";
%if %mf_nobs(work.out)=0 %then %do;
/* send empty row if empty table to help with hot rendering */
output;
@@ -324,7 +328,7 @@ select upcase(loadtype)
by varnum;
run;
data vars3(keep=name type length format label pk varnum ctrloptions formatd);
data vars3(keep=name type length format label pk varnum formatd);
set vars(rename=(format=format2 type=type2));
name=upcase(name);
/* not interested in transaction or processing dates
@@ -345,10 +349,6 @@ select upcase(loadtype)
if name in ('',%upcase(%mf_getQuotedStr(&pk,dlm=%str(,),quote=S)))
then PK='YES';
length ctrlOptions $500;
if name="_____DELETE__THIS__RECORD_____" then ctrlOptions='["No","Yes"]';
else ctrlOptions='';
run;
%mp_abort(iftrue= (&syscc ne 0)
,mac=&_program..sas
@@ -363,9 +363,7 @@ select upcase(loadtype)
length coltype $500.;
format=upcase(format);
coltype=cats('{"data":"',name,'"');
if ctrlOptions ne '' then
colType=cats(coltype,',"type":"dropdown","source":',ctrlOptions,"}");
else if type='num' then do;
if type='num' then do;
if format=:'DATETIME' or format=:'E8601DT' or format=:'NLDATM'
then do;
colType=cats(coltype,',"type":"datetime"}');
@@ -648,7 +646,7 @@ create table dqdata as
/* extract selectbox data */
%macro dq_selects();
%local x source lib ds col;
%local x source lib ds col filtref filterds srcview;
%do x=1 %to &sourcecnt;
%let source=&&source&x;
%let lib=%scan(&source,1,.);
@@ -658,13 +656,31 @@ create table dqdata as
%put &=lib;
%dc_assignlib(READ,&lib)
%dc_casload(&lib..&ds)
/* Selectbox values are read straight from the source table, so the same
Row Level Security that filters the grid must be applied here too -
otherwise values from rows the user is not permitted to see leak into
the cell dropdown. %mpe_filtermaster builds the filter; a data step
view is used because a fileref cannot be %inc'd inside a proc sql where
clause. EDIT mode mirrors the grid on this (editor) page. */
%let filtref=%mf_getuniquefileref();
%let filterds=%mf_getuniquename();
%mpe_filtermaster(EDIT,&lib..&ds,
dclib=&mpelib,
outref=&filtref,
outds=work.&filterds
)
%let srcview=%mf_getuniquename();
data work.&srcview / view=work.&srcview;
set &lib..&ds;
where %inc &filtref;;
run;
proc sql;
create table dqdata&x as
select distinct "&&base_col&x" as base_col length=32
,"&source" as rule_value length=74
,cats(&col) as rule_data length=1000
,&col as tmp_order
from &lib..&ds
from work.&srcview
order by tmp_order;
/* ensure both numerics and char vals are ordered correctly */
data work.dqdata&x (drop=tmp_order);
@@ -677,6 +693,12 @@ create table dqdata as
)
proc append base=dqdata data=dqdata&x;run;
proc sql; drop table dqdata&x;
/* drop the per-source filter artifacts - WORK is session scoped, but the
loop should not leave a view and a filter table behind per source */
proc datasets lib=work nolist;
delete &srcview &filterds;
quit;
filename &filtref clear;
%end;
%mend dq_selects;
%dq_selects()
@@ -688,11 +710,6 @@ run;
%mp_getmaxvarlengths(work.sasdata1,outds=maxvarlengths)
data maxvarlengths;
set maxvarlengths;
if name='_____DELETE__THIS__RECORD_____' then mAXLEN=3;
run;
data xl_rules;
set &mpelib..mpe_excel_config;
where &dc_dttmtfmt. lt tx_to;
@@ -0,0 +1,124 @@
/**
@file
@brief testing getdata row and cell limits
@details The EDIT screen refuses a selection that is too big on either
count - rows (DC_MAXOBS_WEBEDIT) or cells (DC_MAXCELLS_WEBEDIT). The row
check runs first, so a selection over the row limit reports "Table is too
big" and one under the row limit but over the cell limit reports
"Selection is too wide". Both aborts are proven by the child job state
(an aborted service registers no webout).
Fixture: dctest.wideboy (1000 rows x 1004 columns), created and registered
by testsetup. Filtered to 300 rows it is over the 250 row limit; filtered
to 200 rows it is under the row limit but 200 x 1004 = 200,800 cells, over
the 200,000 cell limit.
<h4> SAS Macros </h4>
@li mp_assert.sas
@li mp_filterstore.sas
@li mf_getuniquefileref.sas
@li mx_execute.sas
**/
%let _program=&appLoc/services/editors/getdata;
/**
* build a ROW_ID <= &maxrow filter on dctest.wideboy, leaving the filter_rk
* in &filter_rk
*/
%macro wideboy_filter(maxrow);
data work.inquery;
length GROUP_LOGIC $3 SUBGROUP_LOGIC $3 SUBGROUP_ID 8
VARIABLE_NM $32 OPERATOR_NM $10 RAW_VALUE $4000;
GROUP_LOGIC='AND';
SUBGROUP_LOGIC='AND';
SUBGROUP_ID=1;
VARIABLE_NM='ROW_ID';
OPERATOR_NM='LE';
RAW_VALUE="&maxrow";
output;
run;
%mp_filterstore(
libds=dctest.wideboy,
filter_summary=&dc_libref..mpe_filteranytable,
filter_detail=&dc_libref..mpe_filtersource,
lock_table=&dc_libref..mpe_lockanytable,
maxkeytable=&dc_libref..mpe_maxkeyvalues,
queryds=work.inquery,
outresult=work.result,
outquery=work.query
)
data _null_;
set work.result;
call symputx('filter_rk',filter_rk,'g');
run;
%mend wideboy_filter;
/**
* Test 1 - row limit (300 rows > 250)
*/
%wideboy_filter(300)
%let f1=%mf_getuniquefileref();
data _null_;
file &f1 termstr=crlf;
put 'LIBDS:$43. FILTER_RK:8.';
put "dctest.wideboy,&filter_rk";
run;
%mx_execute(&_program,
viyacontext=&defaultcontext,
inputfiles=&f1:sascontroltable,
outref=web1
)
%let abort1=0;
data _null_;
set work.results;
if state='canceled' then call symputx('abort1',1);
run;
%let syscc=0;
%mp_assert(
iftrue=(&abort1=1),
desc=Test 1 - a selection over the row limit is refused,
outds=work.test_results
)
/**
* Test 2 - cell limit (200 rows x 1004 cols = 200,800 > 200,000)
*/
%wideboy_filter(200)
%let f2=%mf_getuniquefileref();
data _null_;
file &f2 termstr=crlf;
put 'LIBDS:$43. FILTER_RK:8.';
put "dctest.wideboy,&filter_rk";
run;
%mx_execute(&_program,
viyacontext=&defaultcontext,
inputfiles=&f2:sascontroltable,
outref=web2
)
%let abort2=0;
data _null_;
set work.results;
if state='canceled' then call symputx('abort2',1);
run;
%mp_assert(
iftrue=(&abort2=1),
desc=Test 2 - a selection over the cell limit is refused,
outds=work.test_results
)
/**
* dump results to the log for offline inspection
*/
data _null_;
set work.test_results;
putlog 'TEST_RESULT_LINE: ' test_result ' - ' test_description;
run;
@@ -0,0 +1,271 @@
/**
@file
@brief testing that selectbox (cell dropdown) values respect Row Level Security
@details A HARDSELECT / SOFTSELECT rule whose RULE_VALUE is a
library.member.column reference has its distinct values read straight from
that source table by getdata.sas. If the source table carries Row Level
Security rules, the restricted rows must be filtered out of the dropdown too -
otherwise values the user is not permitted to see appear in the cell dropdown
while the grid itself is correctly filtered.
This test makes the runner a 'regular' (non admin) user and restricts
SOME_BESTNUM in MPE_X_TEST to one value selected from the data (the demo
data holds random SOME_BESTNUM values, refreshed by testsetup, so the
value cannot be hardcoded). The demo data must hold more than one
distinct SOME_BESTNUM - true of the SAS-seeded estate, but NOT of the
SASjs server mock, which seeds the seven 44 rows only (see the
'distinct_bestnum' precondition assertion below).
Runners whose SAS identity group is the admin group cannot be demoted by
editing MPE_GROUPS alone - Row Level Security is skipped for them - so the
test reports a skip row instead of running the assertions.
The call passes a filter (PRIMARY_KEY_FIELD < 100) because getdata.sas also
enforces the row / cell limits, and the unfiltered MPE_X_TEST is above the
row limit (642 rows against DC_MAXOBS_WEBEDIT:250) - the limits test
(getdata.test.1.sas) covers that abort separately.
<h4> SAS Macros </h4>
@li mf_getuniquefileref.sas
@li mf_getuniquename.sas
@li mf_getuser.sas
@li mpe_getgroups.sas
@li mp_assert.sas
@li mp_assertcolvals.sas
@li mp_assertdsobs.sas
@li mp_filterstore.sas
@li mx_execute.sas
**/
%let _program=&appLoc/services/editors/getdata;
/**
* The body is wrapped in a macro so the conditional skip below uses
* %if/%then/%do inside a macro definition. Open-code %if needs SAS 9.4M5+,
* and the suite also runs on the sas9 target, so the wrapper keeps the test
* portable across the supported estates (as testsetup.sas's update_settings
* does for its own conditionals).
*/
%macro getdata_rls_test();
/**
* Row Level Security is skipped for admins, so only a non admin runner can
* verify it. The group membership is checked against &dc_admin_group (the
* mpeinit-resolved &mpeadmins is not available in a test session, see
* testinit.sas) and taken from the SAS identity groups, NOT the MPE_GROUPS
* table this test goes on to edit. Admin runners (eg a shared test account
* that is a member of the admin group) simply skip the RLS assertions.
*/
%mpe_getgroups(user=%mf_getuser(),outds=work.testgroups)
proc sql noprint;
select count(*) into: admin_check
from work.testgroups
where groupname="&dc_admin_group";
quit;
%if &admin_check>0 %then %do;
%put WARNING: %mf_getuser() is in the &dc_admin_group group;
%put WARNING: Row Level Security is skipped for admins - test skipped;
data work.test_results;
length test_result $4 test_description $256 test_comments $256;
test_result='PASS';
test_description="GETDATA_TEST_2 skipped - runner is an admin";
test_comments="RLS is skipped for admins so the assertions cannot run";
run;
%end;
%else %do;
/**
* Make the runner a 'regular' user and grant the temp group EDIT access so
* the getdata access check still passes.
*/
data work.groups_backup;
set &dc_libref..MPE_GROUPS;
where upcase(user_name)="%upcase(%mf_getuser())";
run;
%let tempgroup=%mf_getuniquename();
proc sql;
delete from &dc_libref..MPE_GROUPS
where upcase(user_name)="%upcase(%mf_getuser())";
insert into &dc_libref..MPE_GROUPS
set user_name="%mf_getuser()"
,group_name="&tempgroup"
,group_desc="temp group"
,tx_from='01Jan1960:00:00:00'dt
,tx_to='31Dec9999:23:59:59'dt;
insert into &dc_libref..MPE_SECURITY
set tx_from=0
,libref="*ALL*"
,dsn="*ALL*"
,access_level="EDIT"
,sas_group="&tempgroup"
,tx_to='31DEC9999:23:59:59'dt;
quit;
/**
* Row Level Security on MPE_X_TEST - only one SOME_BESTNUM value (selected
* from the data, see header) is visible. The value is taken from the rows
* that match the user filter below, so the filtered grid cannot be empty.
*/
proc sql noprint;
delete from &dc_libref..MPE_ROW_LEVEL_SECURITY
where RLS_TABLE="MPE_X_TEST" and rls_group="&tempgroup";
select distinct some_bestnum into :rls_value trimmed
from &dc_libref..MPE_X_TEST
where some_bestnum is not null and primary_key_field < 100;
select coalesce(max(rls_rk),0) into: max_rk
from &dc_libref..MPE_ROW_LEVEL_SECURITY;
quit;
proc sql;
insert into &dc_libref..mpe_row_level_security set
tx_from=0
,tx_to='31DEC5999:23:59:59'dt
,RLS_RK=&max_rk+1
,RLS_SCOPE='ALL'
,RLS_GROUP="&tempgroup"
,RLS_LIBREF="&dc_libref"
,RLS_TABLE="MPE_X_TEST"
,RLS_GROUP_LOGIC='AND'
,RLS_SUBGROUP_LOGIC='OR'
,RLS_SUBGROUP_ID=0
,RLS_VARIABLE_NM='SOME_BESTNUM'
,RLS_OPERATOR_NM='IN'
,RLS_RAW_VALUE="(&rls_value)"
,RLS_ACTIVE=1;
quit;
/**
* Preconditions - the source column really does hold more than one distinct
* value (so the assertions are meaningful) and the user filter matches at
* least one of them (so the grid cannot be empty).
*/
proc sql noprint;
select count(distinct some_bestnum) into: distinct_bestnum
from &dc_libref..MPE_X_TEST;
quit;
%mp_assert(
iftrue=(&distinct_bestnum>1),
desc=Precondition - MPE_X_TEST.SOME_BESTNUM has more than one distinct value,
outds=work.test_results
)
%mp_assert(
iftrue=(%length(&rls_value)>0),
desc=Precondition - the user filter matches at least one SOME_BESTNUM value,
outds=work.test_results
)
/**
* Store a filter (PRIMARY_KEY_FIELD < 100) so the selection stays under the
* DC_MAXOBS_WEBEDIT row limit.
*/
data work.inquery;
length GROUP_LOGIC $3 SUBGROUP_LOGIC $3 SUBGROUP_ID 8
VARIABLE_NM $32 OPERATOR_NM $10 RAW_VALUE $4000;
GROUP_LOGIC='AND';
SUBGROUP_LOGIC='AND';
SUBGROUP_ID=1;
VARIABLE_NM='PRIMARY_KEY_FIELD';
OPERATOR_NM='<';
RAW_VALUE='100';
run;
%mp_filterstore(
libds=&dclib..MPE_X_TEST,
filter_summary=&dc_libref..mpe_filteranytable,
filter_detail=&dc_libref..mpe_filtersource,
lock_table=&dc_libref..mpe_lockanytable,
maxkeytable=&dc_libref..mpe_maxkeyvalues,
queryds=work.inquery,
outresult=work.filter_result,
outquery=work.filter_query
)
data _null_;
set work.filter_result;
call symputx('f1_rk',filter_rk,'l');
run;
/**
* Call the getdata service - filtered to keep the row count under
* DC_MAXOBS_WEBEDIT (see header).
*/
%let f1=%mf_getuniquefileref();
data _null_;
file &f1 termstr=crlf;
put 'LIBDS:$43. FILTER_RK:8.';
put "&dclib..MPE_X_TEST,&f1_rk";
run;
%mx_execute(&_program,
viyacontext=&defaultcontext,
inputfiles=&f1:sascontroltable,
outlib=web1,
mdebug=&sasjs_mdebug
)
data work.dqdata;
set web1.dqdata;
run;
data work.sasdata;
set web1.sasdata;
run;
/* the selectbox source rows only (exclude MPE_SELECTBOX-derived values) */
data work.dqtest;
set work.dqdata;
where upcase(base_col)='SOME_BESTNUM';
run;
data work.checkchar;
val="&rls_value"; output;
run;
data work.checknum;
val=&rls_value; output;
run;
/**
* Test 1 - control: the grid itself is filtered by RLS. If this fails the
* runner is an admin and the dropdown assertion below cannot be trusted.
*/
%mp_assertcolvals(work.sasdata.SOME_BESTNUM,
checkvals=work.checknum.val,
desc=Test 1 - grid rows respect Row Level Security (control),
test=ALLVALS,
outds=work.test_results
)
/**
* Test 2 - the selectbox values sourced from MPE_X_TEST.SOME_BESTNUM must
* also be restricted to the RLS-visible value (44).
*/
%mp_assertdsobs(work.dqtest,
desc=Test 2 precondition - selectbox values were returned for SOME_BESTNUM,
test=HASOBS,
outds=work.test_results
)
%mp_assertcolvals(work.dqtest.RULE_DATA,
checkvals=work.checkchar.val,
desc=Test 2 - selectbox values respect Row Level Security,
test=ALLVALS,
outds=work.test_results
)
/**
* Cleanup - restore the group / security / RLS rows
*/
proc sql;
delete from &dc_libref..MPE_ROW_LEVEL_SECURITY where rls_group="&tempgroup";
delete from &dc_libref..MPE_SECURITY where sas_group="&tempgroup";
delete from &dc_libref..MPE_GROUPS where group_name="&tempgroup";
quit;
proc append base=&dc_libref..MPE_GROUPS data=work.groups_backup;
run;
%end;
%mend getdata_rls_test;
%getdata_rls_test()
+34 -4
View File
@@ -17,13 +17,43 @@
/**
* Test 1 - basic fetch
* The whole of MPE_TABLES is over the EDIT row limit on a populated estate,
* so fetch a single library's rows - the filter is what keeps the selection
* under the limit. The assertion is simply that data still comes back.
*/
data work.inquery1;
length GROUP_LOGIC $3 SUBGROUP_LOGIC $3 SUBGROUP_ID 8
VARIABLE_NM $32 OPERATOR_NM $10 RAW_VALUE $4000;
GROUP_LOGIC='AND';
SUBGROUP_LOGIC='AND';
SUBGROUP_ID=1;
VARIABLE_NM='LIBREF';
OPERATOR_NM='=';
RAW_VALUE="'&dclib'";
output;
run;
%mp_filterstore(
libds=&dclib..MPE_TABLES,
filter_summary=&dc_libref..mpe_filteranytable,
filter_detail=&dc_libref..mpe_filtersource,
lock_table=&dc_libref..mpe_lockanytable,
maxkeytable=&dc_libref..mpe_maxkeyvalues,
queryds=work.inquery1,
outresult=work.result1,
outquery=work.query1
)
data _null_;
set work.result1;
call symputx('filter_rk1',filter_rk,'g');
run;
%let f1=%mf_getuniquefileref();
data _null_;
file &f1 termstr=crlf;
put 'LIBDS:$43.';
put "&dclib..MPE_TABLES";
put 'LIBDS:$43. FILTER_RK:8.';
put "&dclib..MPE_TABLES,&filter_rk1";
run;
%mx_execute(&_program,
viyacontext=&defaultcontext,
@@ -58,9 +88,9 @@ data query;
run;
%mp_assertcols(work.query,
desc=Query dataset is empty (test cols not obs as table has one row),
desc=Query dataset carries the filter columns,
cols=VARIABLE_NM RAW_VALUE SUBGROUP_ID,
test=NONE,
test=ALL,
outds=work.test_results
)
%mp_assertdsobs(work.sasdata,
+10 -18
View File
@@ -51,25 +51,17 @@ run;
picker as numeric. getdata.sas makes the same call for the same reason. */
%mp_getcols(&ds, outds=cols1)
/* Exclude the system-managed temporal columns, exactly as the editor does, so
the picker's fallback list matches the one the table itself would supply.
%global, not %local: this service is open code, and %local is only valid
inside a macro definition - used here it logs an error on every call and the
variables are never created. */
%global var_txfrom var_txto var_processed;
proc sql noprint;
select var_txfrom, var_txto, var_processed
into: var_txfrom, :var_txto, :var_processed
from &mpelib..mpe_tables
where &dc_dttmtfmt. lt TX_TO
and upcase(libref)=upcase("%scan(&ds,1,.)")
and upcase(dsn)=upcase("%scan(&ds,2,.)");
quit;
data cols(keep=name type length varnum fmtname ddtype);
/* Response contract. `type` keeps the SAS vocabulary this public service has
always used (C/N), and `format` and `label` are carried over from the
previous shape, so an existing caller still finds them unchanged. `fmtname`
(the bare format name) and `ddtype` (which now distinguishes DATE, DATETIME
and TIME) are additions. The client derives its own num/char vocabulary from
ddtype at the store boundary - see sas-store.getCols. */
data cols(keep=name type length varnum fmtname format label ddtype);
set cols1;
if name not in ("&var_txfrom","&var_txto","&var_processed");
/* the picker wants C/N for ordinary columns, as getdata.sas maps it */
/* DDTYPE in the client's vocabulary, as getdata.sas maps it. ddtype needs
no length statement - the macro creates it with 'CHARACTER', so it is
already $9. */
if ddtype='CHARACTER' then ddtype='C';
else if ddtype='NUMERIC' then ddtype='N';
run;
@@ -66,8 +66,20 @@ run;
)
%let nobs=0;
%let nbad=0;
proc sql noprint;
select count(*) into: nobs from web2.cols;
/**
* The service keeps the SAS vocabulary this public contract has always used:
* TYPE is C/N, and DDTYPE separates the date and time types. The client maps
* TYPE to its own num/char vocabulary in sas-store.getCols, so a change here
* that silently switched vocabulary would not be caught by the client suite.
* Every column must be C or N, and the two must agree with ddtype.
*/
select count(*) into: nbad from web2.cols
where type not in ('C','N')
or (ddtype='C' and type ne 'C')
or (ddtype in ('N','DATE','DATETIME','TIME') and type ne 'N');
quit;
%mp_assert(
@@ -76,6 +88,12 @@ quit;
outds=work.test_results
)
%mp_assert(
iftrue=(&nbad=0),
desc=%str(TYPE stays in the SAS C/N vocabulary and agrees with ddtype),
outds=work.test_results
)
/**
* dump results to the log for offline inspection
*/