Commit Graph
43 Commits
Author SHA1 Message Date
hermes 11f56cc54a refactor(deploy): remove vestigial client credential fields
Build / Build-and-ng-test (pull_request) Successful in 5m25s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m57s
Build / Build-and-test-development (pull_request) Successful in 24m44s
The deploy client_id/client_secret fields were only read from localStorage
and never used by the manual or automatic deploy flows. Remove the dead
code so no credential-shaped value is read from browser storage.
2026-09-14 16:45:20 +01:00
hermes 2d31e5a8b9 fix(security): validate libds in stagedata and loadfile
Sanitise the libref.dataset input the same way getdata does, via
mp_validatecol, and abort the service on an invalid value to prevent code
injection through the libds identifier. Format catalog inputs resolve to
work.fmtextract and still pass the check.
2026-09-14 16:45:13 +01:00
hermes cfd8f06435 fix(security): escape cell values in status renderers to prevent DOM XSS
The error/no-spinner/spinner cell renderers wrote the cell value straight
into td.innerHTML. A value containing markup (which can arrive from a dataset
served by the getdata stored program or from a typed edit) was therefore
parsed and executed by the browser. Escape the value so it renders as inert
text, keeping the hardcoded icon markup intact, and add a regression test
that reproduces the injection via a real Handsontable instance.
2026-09-14 16:44:59 +01:00
hermes a6a111db8a test(viya): scope stagedata abort assertions to the submitted load
Build / Build-and-ng-test (pull_request) Successful in 6m4s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m41s
Build / Build-and-test-development (pull_request) Successful in 25m35s
The abort checks matched any non-success load in the time window, so a parallel session's aborted/IN PROGRESS load could satisfy them even if the tested load completed. Key them on this test's unique submitted message (reason_txt), which is preserved because a hook-aborted load finishes before mpe_loadfail can overwrite it.
2026-09-14 13:25:32 +01:00
hermes 8418e0cb1d docs(contributing): document one-time git hook activation
Build / Build-and-ng-test (pull_request) Successful in 5m21s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m1s
Build / Build-and-test-development (pull_request) Successful in 24m44s
2026-09-14 12:44:10 +01:00
hermes d301fc2b4d fix(deps): bump @sasjs/cli to 4.20.3
Clears the moderate adm-zip advisory (GHSA-vwc7-r8mq-g2x9) from the sas npm audit gate - @sasjs/cli 4.20.3 replaces adm-zip with yauzl/yazl.
2026-09-14 12:43:58 +01:00
hermes 82a45b80b4 chore(viya): run tests on the reusable compute context, harden chunk deploy
Build / Build-and-ng-test (pull_request) Successful in 5m20s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m59s
Build / Build-and-test-development (pull_request) Successful in 24m27s
The test jobs on the nextviya estate run as a batch identity that is
not authorised to create sessions in the SAS Job Execution compute
context - child service submissions via mx_execute all failed with
403.  Default the test context to Compute Reusable (matching the
target contextName) in the shared test init programs.

Also give the chunk deploy helper a retry loop for the per-chunk
sasjs run timeouts - estate cold-starts can exceed a single 300s
window.
2026-09-08 20:52:19 +01:00
hermes a8194d73e7 test(viya): align stagedata test payloads with the adapter contract
The web editor submits date/datetime/time cells as formatted strings
and always provides the format-catalog buskey, so the staged CSVs
round-trip through the ANYDT* informats in mpe_loader.  Some tests
built jsdata with raw numeric values or without FMTROW, causing the
staged loads to fail on Viya.

- getstagetable.test: send some_date/some_datetime/some_time as
  formatted strings (as postdata.test.1 already does), and assert on
  flags captured from the first read - the JSON libname can only be
  read once, so re-reading stagetable returns -1 observations
- postdata.test.2: add fmtrow to the staged format catalog rows (the
  TYPE FMTNAME FMTROW buskey)
- stagedata.test.3: assert the #211 hook aborts through MPE_LOADS - a
  hook-aborted child never returns a webout, and the load stays IN
  PROGRESS because mpe_loadfail cannot run after the abort.  The WIDEBOY
  RLS rule is inserted in the prep section so the (possibly pooled)
  child session reads it from the shared filesystem
2026-09-08 20:52:19 +01:00
hermes 1dccf61f62 fix(admin): return refreshcatalog output as webout JSON
The service wrote a raw HTML line straight to the _webout fileref
after the load completed.  On Viya, API-submitted jobs (eg via
mx_execute) do not always have that fileref connected to a job result,
so the output was lost and job-result consumers aborted.  Return a
proper sasparams table through %webout() instead - the same pattern as
the other admin services - and update the test to assert on the
webout sasparams message.
2026-09-08 20:52:19 +01:00
hermes 09f31ce6da fix(deps): bump @sasjs/core to 5.2.7
Brings in the mv_jobflow JSON-escape fix (prevents jobExecution 400s
for job arguments containing quotes or CR/LF) and the graceful
handling of canceled jobs in mv_getjoblog / mv_getjobresult (a
canceled child no longer aborts the calling program when its log or
result is unavailable).  Verified against a live Viya 2026.07 estate:
the full SAS test suite passes with these macros sourced from core.
2026-09-08 20:52:19 +01:00
hermes 1ed31f2a49 chore: docs
Build / Build-and-ng-test (pull_request) Successful in 5m27s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m52s
Build / Build-and-test-development (pull_request) Successful in 24m31s
2026-08-31 08:54:05 +01:00
hermes 490d0137b9 fix(stage): submit data via editor flow, use dynamic date assertion
Build / Build-and-ng-test (pull_request) Successful in 5m14s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m30s
Build / Build-and-test-development (pull_request) Successful in 24m11s
Rewrite stage.cy.ts to follow the same pattern as excel.cy.ts:
open MPE_X_TEST, attach regular_excel.xlsx, submit to create staged
data, then test the formatted/unformatted toggle on the stage page.

The date assertion is now dynamic: reads the raw SOME_DATE value
from the unformatted view and computes the expected SAS date9.
formatted value, rather than hardcoding 12FEB1960.

Remove the fixture stageSubmission code from makedata.js — staged
data is created at runtime by stagedata.js, not by makedata.
2026-08-30 22:29:40 +01:00
hermes f25ecd8828 fix(mocks): limit stage fixture to 1 row, fix column order
Build / Build-and-ng-test (pull_request) Successful in 5m12s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m49s
Build / Build-and-test-development (pull_request) Successful in 24m6s
The stage.cy.ts fixture had 7 rows of MPE_X_TEST data with wide string
values, causing Handsontable to auto-size columns wider so only 4 of
10 fit in the 1280px viewport. The test checks .ht_clone_top headers
which only render visible columns.

Limited to 1 row (matching the original sas9 mock) and moved
_____DELETE__THIS__RECORD_____ to the end of the column order.
2026-08-30 20:20:15 +01:00
hermes c180934167 fix(mocks): SAS datetime format in getchangeinfo, login in stage.cy.ts
getchangeinfo.js: use SAS datetime19. format (DDMMMYYYY:HH:MM:SS)
for SUBMITTED_ON_DTTM and REVIEWED_ON_DTTM, matching the real SAS
output. Use SUBMIT_STATUS_CD field name (not REVIEW_STATUS_ID)
to match the MPE_SUBMIT column name and getchangeinfo.sas output.

stage.cy.ts: add login step in beforeEach (the beforeAll logs out
after uploading the license, so each test needs to re-login).
Use loginAndUpdateValidKey(true) to force license upload.
2026-08-30 20:10:52 +01:00
hermes 6e2548d78c fix(mocks): add fixture staged data and MPE_SUBMIT row for stage.cy.ts
Build / Build-and-ng-test (pull_request) Successful in 5m44s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m25s
Build / Build-and-test-development (pull_request) Failing after 25m9s
stage.cy.ts navigates directly to /stage/DC20221007T122326121_612316_7259
and expects a hot table with MPE_X_TEST data. The mock had no fixture
staged.json or MPE_SUBMIT row for this table ID, so getstagetable
returned 0 rows and getchangeinfo returned empty jsparams.

Added a fixture staged.json with MPE_X_TEST rows (SOME_DATE=42) and
a matching MPE_SUBMIT row (SUBMITTED status) in makedata.js.
2026-08-30 19:25:42 +01:00
hermes 94d2a7970e fix(mocks): look up PK_FIELDS from MPE_TABLES in viewdata, fallback for MPE_AUDIT
Build / Build-and-ng-test (pull_request) Successful in 5m52s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m54s
Build / Build-and-test-development (pull_request) Failing after 25m39s
viewdata.js hardcoded PK_FIELDS as empty, so the viewbox grid treated
all columns as non-PK. removeAllColumns() could delete every column
including the ones the tests expect to be protected.

Now reads MPE_TABLES buskey (mirrors mp_getpk) and falls back to
LOAD_REF LIBREF DSN KEY_HASH TGTVAR_NM for MPE_AUDIT (which has a
known index-based PK in real SAS but isn't registered in MPE_TABLES).

All 12 viewbox.cy.ts tests now pass.
2026-08-30 19:18:36 +01:00
hermes c0af86ef29 fix(mocks): add LIBRARYNAME to startupservice, fix viewbox column order
Build / Build-and-ng-test (pull_request) Successful in 5m19s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m27s
Build / Build-and-test-development (pull_request) Failing after 27m3s
startupservice.js only returned LIBREF in saslibs, but the dc-tree
component renders LIBRARYNAME — without it the nav tree node text was
empty, so libraryToOpenIncludes could not match. Added LIBRARYNAME,
LIBRARYID, and ENGINE to match the viewlibs service format.

Changed libraryToOpenIncludes_SASJS from 'dc' to 'dc_jslib' to avoid
matching multiple libraries (DC_JSLIB and DC996664 both contain 'dc').

viewdata.js: removed alphabetical .sort() on cols (SAS preserves
physical column order via mp_getcols); added empty row with correct
SAS missing values ('.' for numeric, '' for character) when a table
has no data, so the viewbox grid can derive column headers.
2026-08-30 18:52:54 +01:00
hermes 34bd6607c5 fix(mocks): remove SOME_SHORTNUM HARDREGEX from MPE_VALIDATIONS
Build / Build-and-ng-test (pull_request) Successful in 5m51s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m6s
Build / Build-and-test-development (pull_request) Failing after 31m25s
The Excel fixture files (7th_tab, extra_column, duplicate_column,
duplicate_row, regular_xls, regular_with_delete, password, leading
whitespace) contain SOME_SHORTNUM values of 0, 1, and 2. The
HARDREGEX /^(?![0-2](\.\d+)?$).*/ blocks those values, aborting
the upload before it reaches the submit/approve flow. Removing the
rule allows all 23 excel.cy.ts tests and both liveness.cy.ts tests
to pass.
2026-08-30 10:30:32 +01:00
hermes e4b04b1d44 fix(mocks): rename saveTableData in postdata.js to avoid shadowing
Same shadowing issue as rejection.js: postdata.js defined its own
saveTableData(tableName, tableData, tableLibref) which was shadowed
by dcMockUtils.saveTableData(dataDir, tableName, tableData) from the
eval. Renamed to saveTableToLib to eliminate the collision.
2026-08-30 02:39:57 +01:00
hermes fed0396737 fix(mocks): remove shadowed saveTableData in rejection.js
rejection.js defined its own local saveTableData(tableName, tableData)
with 2 args, but dcMockUtils.js (loaded via eval) also defines
saveTableData(dataDir, tableName, tableData) with 3 args. The
dcMockUtils version was shadowing the local one, causing
saveTableData('MPE_SUBMIT', submitData) to pass the table name as
dataDir and the data object as tableName, crashing with
'TypeError: tableName.toLowerCase is not a function'.

Removed the local saveTableData and updated calls to use the
dcMockUtils 3-arg signature: saveTableData(dataDir, tableName, data).
2026-08-30 02:38:13 +01:00
hermes 4f7b845390 fix(mocks): remove MPE_VALIDATIONS and MPE_ALERTS from DC996664 libref
Build / Build-and-ng-test (pull_request) Successful in 5m20s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m59s
Build / Build-and-test-development (pull_request) Failing after 8h28m57s
The multi-load fixture sheets for DC996664.MPE_VALIDATIONS had data
misaligned with the schema, corrupting the shared submit state. The
tests only need MPE_X_TEST and MPE_TABLES, so only register those two.
2026-08-30 01:18:07 +01:00
hermes 13c10be30f fix(cypress): correct mock data, test expectations, and abort modal
Build / Build-and-ng-test (pull_request) Successful in 5m20s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m39s
Build / Build-and-test-development (pull_request) Failing after 34m1s
- editor.cy.ts: assert DC.USER_NAME is non-empty instead of hardcoded
  'sasdemo' (CI runs as a different OS user)
- editor.cy.ts: use =SOME_BESTNUM * 0 + 1936 for a deterministic formula
  result regardless of the column's actual value
- excel.cy.ts: expect computed value '2' instead of formula text '=1+1'
  (no XL_RULE=FORMULA configured on that column)
- excel-multi-load.cy.ts: use DC996664 libref to match the actual fixture
  sheet names
- makedata.js: add labels to DC_JSLIB.MPE_X_TEST schema (SOME_CHAR,
  SOME_DATE, SOME_DATETIME, SOME_TIME) for the viewer-labels tests
- makedata.js: register DC996664 libref with MPE_X_TEST/MPE_TABLES/
  MPE_VALIDATIONS/MPE_ALERTS and copy table files for multi-load fixtures
- makedata.js: change SOME_SHORTNUM HARDREGEX from [1-5] to [0-2]
- loadfile.js: exclude VAR_TXFROM/VAR_TXTO/VAR_PROCESSED columns from the
  missing-cols check for TXTEMPORAL tables (computed at load time)
- InfoModal.ts: add _PROGRAM to AbortDetails
- info-modal.component.html: display _PROGRAM in the abort modal
- 4 call sites: pass abortRes._PROGRAM when building AbortDetails
2026-08-29 23:53:13 +01:00
hermes 8e51fde524 fix(licensing): bridge async proceed() through cy.then and replace Response.arrayBuffer with manual stream reader
Build / Build-and-ng-test (pull_request) Successful in 5m9s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m31s
Build / Build-and-test-development (pull_request) Failing after 33m53s
- licensing.cy.ts: call proceed() unconditionally via cy.then() instead of conditionally chaining .then() after cy.wait(), so the command queue stays open across the async gap and enqueued commands always run
- combinedLicenceKey.ts: replace new Response(stream).arrayBuffer() with an explicit streamToArrayBuffer helper for broader browser support
- remove debug-license.cy.ts, an untracked throwaway repro for the promise-handling question
2026-08-29 20:13:22 +01:00
hermes 5d03164b6a fix: tests and realistic getcolvals.js
Build / Build-and-ng-test (pull_request) Successful in 5m49s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m56s
Build / Build-and-test-development (pull_request) Failing after 33m48s
2026-08-29 13:14:18 +01:00
hermes c0bc87d331 fix(tests): new libref
Build / Build-and-ng-test (pull_request) Successful in 5m27s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m39s
Build / Build-and-test-development (pull_request) Failing after 33m47s
2026-08-29 00:17:13 +01:00
hermes d59ef44226 fix: dynamic getdynamiccolvals.js
Build / Build-and-ng-test (pull_request) Successful in 5m22s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m27s
Build / Build-and-test-development (pull_request) Failing after 33m37s
2026-08-28 23:40:44 +01:00
hermes f213d24f09 fix: failing test
Build / Build-and-ng-test (pull_request) Successful in 5m24s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m25s
Build / Build-and-test-development (pull_request) Failing after 33m33s
2026-08-28 21:29:15 +01:00
hermes a80927cbb6 fix: more mock improvements
Build / Build-and-ng-test (pull_request) Successful in 5m19s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m34s
Build / Build-and-test-development (pull_request) Failing after 33m31s
2026-08-28 16:25:29 +01:00
hermes cc2ff873fa fix: unnecessary getsubmits call removed
Build / Build-and-ng-test (pull_request) Successful in 5m19s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m40s
Build / Build-and-test-development (pull_request) Failing after 33m35s
2026-08-28 14:02:52 +01:00
hermes 3b9344cb60 fix: dynamic js mocks
Build / Build-and-ng-test (pull_request) Successful in 5m16s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m35s
Build / Build-and-test-development (pull_request) Failing after 33m28s
2026-08-28 00:35:31 +01:00
hermes 0082828543 fix: history mock
Build / Build-and-ng-test (pull_request) Successful in 5m16s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m32s
Build / Build-and-test-development (pull_request) Failing after 33m38s
2026-08-27 22:41:47 +01:00
hermes 3595bb9499 fix: using utility macro for webout
Build / Build-and-ng-test (pull_request) Successful in 5m52s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m26s
Build / Build-and-test-development (pull_request) Failing after 33m42s
2026-08-27 17:53:41 +01:00
hermes ee52bb8967 feat: functional JS mocks
Build / Build-and-ng-test (pull_request) Successful in 5m28s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m25s
Build / Build-and-test-development (pull_request) Failing after 33m34s
2026-08-27 17:21:01 +01:00
hermes 18a16fcb06 chore: lint fix
Build / Build-and-ng-test (pull_request) Successful in 5m53s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m3s
Build / Build-and-test-development (pull_request) Failing after 33m36s
2026-08-27 12:42:08 +01:00
hermes 1f4aa6fcea fix: hardening following PR review feedback
Build / Build-and-ng-test (pull_request) Successful in 5m44s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m52s
Build / Build-and-test-development (pull_request) Successful in 26m20s
2026-08-24 15:11:46 +01:00
hermes 69b41559f7 fix: spy for startup
Build / Build-and-ng-test (pull_request) Successful in 5m18s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m27s
Build / Build-and-test-development (pull_request) Successful in 26m24s
2026-08-21 17:31:00 +01:00
hermes 917925f553 fix(deploy): Viya deploy checks, startup diagnostics, and chunked deploy script
Build / Build-and-ng-test (pull_request) Failing after 2m43s
Build / Build-and-test-development (pull_request) Skipped
Lighthouse Checks / lighthouse (pull_request) Successful in 21m6s
- Add StartupCheckService to show deploy check progress on loading screen
  with step-by-step status (appLoc check, Viya deploy, startup service)
- Add console logging throughout checkViyaDeploy and viyaMakedataSuccessfull
  to diagnose deploy flow issues
- Fix missing return after resolve(false) in viyaMakedataSuccessfull when
  folderId is undefined, which caused fall-through to getFolderMembers
  with an undefined ID
- Fix Viya Folders API pagination: getFolderMembers now requests limit=500
  so all members are returned (admin folder has 32 members, default page
  size was hiding makedata)
- Fix error handler in viyaMakedataSuccessfull to resolve(false) instead of
  reject() so the app falls back to setup screen on API errors
- Fix licence key whitespace: SAS makedata initialises keys as a single
  space which is truthy in JS; trim keys before checking so empty keys
  correctly fall back to free tier instead of triggering decryption errors
- Always set _debug=131 on makedata URLs (automatic and manual deploy)
  so full debug output is available for the one-time setup service
- Add makedata completion polling after runMakedataInNewWindow so the app
  auto-reloads when the makedata job self-deletes, instead of hanging
- Fix loading screen slider position from absolute to relative so it does
  not obscure the startup check steps
- Add chunk_deploy.py to sasjs/utils for chunked Viya deploys, splitting
  viya.sas by service and web file boundaries with correct %let path=
  tracking per chunk

Closes #303, #200, #125
2026-08-21 16:00:59 +01:00
hermes 9dab3d50ef chore: sample record in mpe_x_test
Build / Build-and-ng-test (pull_request) Successful in 5m29s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m51s
Build / Build-and-test-development (pull_request) Successful in 25m32s
2026-08-20 12:07:50 +01:00
hermes f7db8719f5 fix: agent skills and nextviya deploys
Build / Build-and-ng-test (pull_request) Successful in 5m8s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m24s
Build / Build-and-test-development (pull_request) Successful in 25m20s
2026-08-19 17:17:30 +01:00
hermes 9023eb2a39 fix: escape regex metacharacters in context name before prxchange
Build / Build-and-ng-test (pull_request) Successful in 5m30s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m5s
Build / Build-and-test-development (pull_request) Successful in 25m28s
symget('_contextname') is concatenated directly into the PRX replacement
string in makedata.sas. A context name containing \ or $ would be
interpreted as a backreference or variable interpolation in the Perl
regex replacement. Escape both characters with a leading backslash in a
data step before embedding the value in the prxchange call.
2026-08-18 14:46:34 +01:00
hermes 72484ae844 fix: clarify debug comment is permanent, extend ComputeContextDetails with Viya response fields
Build / Build-and-ng-test (pull_request) Successful in 5m30s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m2s
Build / Build-and-test-development (pull_request) Successful in 26m15s
- sasjs-configurator: rewrite _debug=131 comment to make explicit that
  debug is ALWAYS enabled for the makedata service (runs once during
  deployment, debug output always wanted for diagnostics). Remove the
  commented-out debug-off line that caused reviewer confusion.
- viya-compute-context-details.model: extend Attributes with
  allowXCMD, sessionInactiveTimeout (number), and make all attribute
  fields optional. Add Environment interface with autoExecLines string
  array. Add environment? to ComputeContextDetails (optional, not all
  contexts expose it).
- automatic.component: rewrite extractRunAs to use
  attributes.runServerAs directly per the actual Viya API response
  shape. Remove all any casts and the non-existent runAsUserId /
  environment.runAsUserId field probes.
2026-08-18 14:45:04 +01:00
hermes 003550b965 chore: reference configurator improvements for Viya deploy
Build / Build-and-ng-test (pull_request) Canceled after 0s
Build / Build-and-test-development (pull_request) Canceled after 0s
Lighthouse Checks / lighthouse (pull_request) Canceled after 0s
fixes #303
2026-08-18 12:44:57 +01:00
hermes 45226e19d6 chore: address PR #306 review feedback
- Add explanatory comments for deliberately enabled debug mode in configurator
- Fix deploy-field-description font-size from 0.55rem to 0.85rem (was unreadably small)
2026-08-18 12:44:54 +01:00