Commit Graph
1562 Commits
Author SHA1 Message Date
dc 00257f9160 ci(client): run the client suite in two more timezones
Lighthouse Checks / lighthouse (pull_request) Successful in 20m42s
Build / Build-and-ng-test (pull_request) Successful in 7m8s
Build / Build-and-test-development (pull_request) Canceled after 0s
2026-10-28 18:04:01 +00:00
dc e51953ca8a fix(deploy): keep the app query when opening the standalone URL 2026-10-28 18:03:53 +00:00
dc aa91daf9f7 fix(embed): record the parameter label only, so a name-keyed va_parmap is reported 2026-10-28 18:03:45 +00:00
allan 00df87f189 Merge pull request 'fix(makedata): pair the %else of the keepme block so it parses in open code' (#389) from fix/makedata-selfdestruct into release/7.20
Sites / deploy (push) Successful in 3m43s
Build / Build-and-ng-test (pull_request) Successful in 5m5s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m48s
Build / Build-and-test-development (pull_request) Canceled after 32m30s
Reviewed-on: #389
2026-10-28 17:09:17 +00:00
allan 65699bdd39 Merge pull request 'fix(deploy): do not offer the deploy wizard inside a report' (#388) from feat/embed-not-configured into release/7.20
Sites / deploy (push) Canceled after 0s
Build / Build-and-ng-test (pull_request) Successful in 5m22s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m6s
Build / Build-and-test-development (pull_request) Successful in 34m34s
Reviewed-on: #388
2026-10-28 17:08:40 +00:00
dc 459ff2da37 fix(deploy): one definition of the standalone URL, and test the guards
Build / Build-and-ng-test (pull_request) Successful in 5m30s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m43s
Build / Build-and-test-development (pull_request) Successful in 33m54s
Review follow-up on the embed guard.

The spec asserted the isEmbedded getter, which mirrors the template rather
than exercising it: dropping !isEmbedded from the @if guards left every test
green. The spec now renders the component in a TestBed and asserts the DOM -
#TCS present standalone and absent for embed=va and embed=true, and the
deploy section withheld after the terms are accepted when embedded. Both
guards were mutation-checked by removing them, which fails the spec.

deployUrl and onNavigateToHome computed the standalone URL two different
ways: origin+pathname in one, location.href.split('#')[0] in the other,
which kept the query string, so a tab opened from ?embed=va#/deploy stayed
embedded. Both now call standaloneUrl() in standalone-url.ts, which parses
the href and drops the query and the fragment. It is tested with a
hash-bearing input, and the navigate test asserts the opened URL carries
neither '?' nor '#', so a regression to split('#') fails.
2026-10-28 17:05:35 +00:00
dc c0ea810766 fix(makedata): pair the %else of the keepme block so it parses in open code
Build / Build-and-ng-test (pull_request) Successful in 4m54s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m1s
Build / Build-and-test-development (pull_request) Successful in 32m10s
The self-destruct block ends with a single-statement %else in open code:

    %if &keepme ne 1 %then %do;
      %mv_deletejes(path=&root/admin, name=makedata);
    %end;
    %else %put NOTE: &_program retained, as KEEPME was requested;

Outside a macro SAS pairs %else only with a %do branch, so this stops with
'ERROR: Expected %DO not found' and 'ERROR: Skipping to next %END statement'
after the deployment has completed. On Viya the job then reports state error
with jobConditionCode 1012, so a successful deployment looks like a failure
exactly when someone is watching a first install.

Both branches now use %do/%end, which is the pattern the repo uses elsewhere
for this reason - see db/migrations/20261003_row_and_cell_limits.sas.

The same block is fixed in the server variant. The two delete calls also gain
their terminating semicolons: the macro call worked without them, but their
absence is what made the block look responsible for the error.

Behaviour is unchanged: the program still deletes itself unless KEEPME was
requested.
2026-10-28 16:38:21 +00:00
allan 62e43034e4 Merge pull request 'feat(embed): resolve va_parmap values through named date anchors' (#386) from feat/va-parmap-transforms into release/7.20
Sites / deploy (push) Successful in 3m46s
Build / Build-and-ng-test (pull_request) Successful in 5m16s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m7s
Build / Build-and-test-development (pull_request) Successful in 34m1s
Reviewed-on: #386
2026-10-28 15:08:45 +00:00
dc 80e8911af8 fix(deploy): do not offer the deploy wizard inside a report
Build / Build-and-ng-test (pull_request) Successful in 5m10s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m36s
Build / Build-and-test-development (pull_request) Successful in 33m18s
Deployment needs an authenticated, standalone session, and the person
looking at a report object is not the person who installs the app. Offering
the wizard there produces a screen whose controls cannot work - the failure
we hit while testing a report: T&C accepted, credentials filled, DEPLOY
clicked, and nothing happened, because the embedded app has no session to
deploy with.

An embedded instance (embed=va or embed=true) now shows what the state is
and where to deploy from, instead of a form that cannot submit. Standalone
behaviour is unchanged.
2026-10-28 15:08:02 +00:00
dc bd3c220af2 fix(embed): never build a clause from a value that did not convert, and say so in the banner
Build / Build-and-ng-test (pull_request) Successful in 5m15s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m25s
Build / Build-and-test-development (pull_request) Successful in 33m8s
Two review findings, both about a filter that looks applied and is not:

- The transform branch of the clause builder interpolated whatever
  convertJsDateToSasDate returned, so an Invalid Date reaching the helper
  would have put a literal NaN in the WHERE clause. A non-finite result is
  now dropped, and unsupportedTransforms reports the parameter, so the
  author sees a banner instead of a broken filter.
- The range count check counted values that survived date parsing, while the
  builder counts resolved bounds. A RANGE sent two values where one did not
  parse therefore built a BETWEEN covering one period with no warning. The
  check now counts what VA sent, the builder refuses a range with more than
  one value rather than picking one, and a value set that is partly
  unparseable gets its own line ('1 of its 2 values was not a date - the
  filter covers only the rest').

The banner also now says what to do: it names the URL parameter to edit and
shows the entry grammar with an example, since the reader is the report
author and has no other way to act on it.
2026-10-28 12:44:05 +00:00
allan 5e8910f238 Merge pull request 'feat(embed): map VA parameters to DC columns from the URL' (#383) from feat/va-param-map into release/7.20
Sites / deploy (push) Successful in 3m44s
Build / Build-and-ng-test (pull_request) Successful in 4m58s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m54s
Build / Build-and-test-development (pull_request) Successful in 33m9s
Reviewed-on: #383
2026-10-28 12:41:09 +00:00
dc 221dd8a15e Merge feat/va-param-map: label-only mapping keys and value-aware same-column warning
Build / Build-and-ng-test (pull_request) Successful in 5m15s
Build / Build-and-test-development (pull_request) Canceled after 0s
Lighthouse Checks / lighthouse (pull_request) Successful in 19m49s
Conflict was valueSignature: the transform branch already builds a clause
value set through buildClauseValueSet, so the helper delegates to it and
takes the parameter's transform, keeping derived dates in the comparison.
2026-10-28 12:34:32 +00:00
dc e9216b0895 fix(embed): match a mapping on the parameter label only, and warn only when the values differ
Build / Build-and-ng-test (pull_request) Successful in 5m33s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m58s
Build / Build-and-test-development (pull_request) Canceled after 0s
Two review findings, plus a scope decision.

A va_parmap key is now matched against the parameter's label only. Matching
the internal name (pr273) as well meant one parameter answered to two keys,
so an entry written against the name and one against the label could never
be told apart: the label entry won and the name entry was silently dead -
no parse problem, no unresolved line, no unmatched line. Dropping the
fallback removes the ambiguity, and an entry keyed on the internal name now
shows up as an unmatched mapping like any other typo.

sameColumnMappings no longer reports two parameters on one column when they
carry the same value: col = 'x' AND col = 'x' filters exactly as one clause
does, yet the banner asserted that different values would show no rows. The
clause value each parameter produces is now compared, and the warning is
raised only when they differ. Parameters that produce no clause at all are
excluded, since they cannot conflict with anything.
2026-10-28 12:32:47 +00:00
dc 37d81c902a test(embed): run the suite in two more timezones
Build / Build-and-ng-test (pull_request) Successful in 5m37s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m37s
Build / Build-and-test-development (pull_request) Canceled after 0s
A transform resolves a report value to dates using local calendar
components, so a test that passes in UTC says nothing about a browser west
of it. test:tz-west and test:tz-east run the whole suite under
America/New_York and Pacific/Auckland; with the UTC construction this test
pair replaces, the bare-number round-trip case fails under the first.

Not wired into CI: it triples the run for a property that is already
pinned by the round-trip assertion, so it stays a local and pre-release
check.
2026-10-28 12:27:42 +00:00
dc 689b5b6709 Merge feat/va-param-map into feat/va-parmap-transforms
Build / Build-and-ng-test (pull_request) Successful in 5m14s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m29s
Build / Build-and-test-development (pull_request) Successful in 33m58s
Brings in the diagnostics fixes from the base branch; conflicts were the
paramMap type widening to VaParmapTarget and the new paramsStable argument,
both of which the transform branch already carries.
2026-10-28 12:24:36 +00:00
dc 10157c1159 fix(embed): resolve transform values as local dates and give ranges whole days
Review findings:

- A bare SAS number was turned into a UTC instant and then read with local
  accessors, so west of UTC the value landed a day early and the anchor was
  computed from the wrong quarter; the round trip to a SAS day number was
  one out. Built with local calendar construction instead, matching
  parseDate and convertJsDateToSasDate.
- RANGE passed keepTime to both bounds, so a DATETIME range started at the
  value's clock time on day one and ended at it on the last day, dropping
  the rest of both boundary days. A range now covers whole days: period
  start at midnight, period end at 23:59:59.
- A RANGE fed more than one parameter value produced no clause and no
  warning, reading as a filter that matched nothing. One BETWEEN cannot
  express several periods, so the count mismatch is now reported.
2026-10-28 12:20:49 +00:00
dc 541a61ba08 fix(embed): make the va_parmap diagnostics say what they mean
Build / Build-and-ng-test (pull_request) Successful in 5m16s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m14s
Build / Build-and-test-development (pull_request) Successful in 33m26s
Review findings, all about a warning that misleads:

- The banner heading said the mapping was ignored, but two of the three
  lists it renders are mappings that are applied (the AND-ed pair) or that
  were never applicable (a parameter VA does not send). Neutral heading.
- unmatchedMappings reported a key as unsent as soon as any parameter had
  been seen. VA sends the parameters a report object's query uses, so a
  name that has not arrived yet looked like a typo. A mapping is now only
  judged once a message introduces no new parameter name, i.e. once the
  object's parameter set has settled.
- sameColumnMappings counted a column the table does not have, so one
  mistake produced the unresolved-mapping line and the AND line. A column
  that is not in the table is now left to unresolvedMappings.
- The Cypress cases asserted exact row counts, which encode the demo
  table's contents. They now read the unfiltered count from the app and
  assert the filtered grid is smaller (or empty), so a fixture change
  cannot fail a behaviour test.
2026-10-28 12:20:37 +00:00
allan ad50acf647 Merge pull request 'fix(ci): publish the typedoc to the webdoc site, not the app default' (#385) from fix/webdoc-named-site into release/7.20
Sites / deploy (push) Canceled after 0s
Build / Build-and-ng-test (pull_request) Canceled after 0s
Build / Build-and-test-development (pull_request) Canceled after 0s
Lighthouse Checks / lighthouse (pull_request) Canceled after 0s
Reviewed-on: #385
Reviewed-by: Allan <allan@4gl.io>
2026-10-28 12:10:16 +00:00
dc cd1ce4a7e2 test(embed): format the transform spec to satisfy prettier --check
Build / Build-and-ng-test (pull_request) Canceled after 0s
Build / Build-and-test-development (pull_request) Canceled after 0s
Lighthouse Checks / lighthouse (pull_request) Canceled after 0s
2026-10-28 12:01:11 +00:00
dc 7e3045a21b fix(embed): build quarter ends from the calendar, not minus 24 hours
Build / Build-and-ng-test (pull_request) Failing after 2m2s
Build / Build-and-test-development (pull_request) Skipped
Lighthouse Checks / lighthouse (pull_request) Canceled after 6m38s
Subtracting a fixed day from the next quarter's start moves by an hour
across a daylight-saving transition, which would place a DATETIME bound
off the intended midnight. Day 0 of the following month is the last day
of the month by construction.
2026-10-28 11:57:23 +00:00
dc 88010c3980 feat(embed): resolve va_parmap values through named date anchors
Build / Build-and-ng-test (pull_request) Failing after 1m55s
Build / Build-and-test-development (pull_request) Skipped
Lighthouse Checks / lighthouse (pull_request) Successful in 20m22s
A VA parameter can now carry a transform on the column side of a mapping:

  va_parmap=SELECTED_DATE-D_DATE:ADD(QUARTER_START,-1)

which filters D_DATE on the report value AND the same point one quarter
earlier, as a single IN clause - so the AND rule, the clause signature,
saveQuery and the reload loop are untouched.

Transforms resolve to one clause with these operations:

- ADD(<ANCHOR>,<offset>)   the value plus one anchor beside it -> IN
- RANGE(<PERIOD>,<offset>) a closed period -> BETWEEN

Anchors: QUARTER_START/END, MONTH_START/END, YEAR_START/END.
Periods: QUARTER, MONTH, YEAR. Offset is a whole number of periods,
defaulting to 0.

The names describe the result rather than mirroring SAS intnx: the
arithmetic runs in the browser, so mirroring intnx would mean
reimplementing its alignment letters and their corners (week alignment,
's' and 'm') with no guarantee of matching SAS. A report author can read
ADD(QUARTER_START,-1) without knowing intnx.

Resolution is anchored on the parameter value, never on today, so a saved
report reproduces the same rows. A transform that cannot run where it is
used - a non-temporal column, a value that is not a date - is skipped and
reported in the existing warning banner rather than silently filtering on
the raw value.
2026-10-28 11:52:53 +00:00
dc c5808a16eb fix(ci): publish the typedoc to the webdoc site, not the app default
Build / Build-and-ng-test (pull_request) Successful in 5m42s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m39s
Build / Build-and-test-development (pull_request) Canceled after 19m50s
`surfer deploy` chooses its target only from `--site-name`; without one it
replaces the app's DEFAULT site. The typedoc job passed `--server
webdoc.datacontroller.io` and no site name, so the 7.19.0 docs were published
over the default site of that app - the marketing site served at
datacontroller.io - instead of to a webdoc site.

Name the site, and give it its alias domain so the first run creates it.
The SAS docs job is left unnamed on purpose: code.datacontroller.io is its own
app whose default site is the docs, and a comment now says so.
2026-10-28 11:49:25 +00:00
dc 0011f7e933 fix(embed): log a parse-time mapping problem once
Build / Build-and-ng-test (pull_request) Successful in 5m0s
Lighthouse Checks / lighthouse (pull_request) Successful in 19m41s
Build / Build-and-test-development (pull_request) Successful in 31m54s
The parse-time problems were logged at app init and then again by the
editor's warning loop, which spreads them into the banner's text. The loop
now logs only what the instance discovers for itself (unresolved column,
unmatched key, AND-ed pair); the parse problems stay in the banner and keep
their single init-time console line.

E2e case 11 pins it: one unparseable entry plus one valid-but-missing
mapping, so the loop certainly runs and logs its own line, and exactly one
line mentions the parse problem.
2026-10-28 11:13:10 +00:00
dc 4ed1e0684d test(embed): prove va_parmap case-insensitivity end to end
Build / Build-and-ng-test (pull_request) Successful in 5m31s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m42s
Build / Build-and-test-development (pull_request) Successful in 33m51s
2026-10-28 09:57:58 +00:00
dc 2623a3fb4b fix(embed): defer the column check until metadata loads, report AND-ed pairs
Build / Build-and-ng-test (pull_request) Successful in 5m50s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m50s
Build / Build-and-test-development (pull_request) Successful in 33m51s
Two review findings, both paths where a filter could look applied while
producing nothing.

- `unresolvedMappings` judged a mapping against an empty column list while the
  grid was still loading, so every entry was briefly reported as naming a
  column the table does not have. An empty or unset `cols` now means "not known
  yet" and returns nothing, so the banner appears only once metadata exists.
- Two parameters mapped to one column are AND-ed, so different values yield no
  rows at all, and nothing said so. `sameColumnMappings()` now reports the
  contended column and both parameters. The clauses are deliberately left as an
  intersection: turning that AND into an OR is a filter-semantics decision, not
  a diagnostic, so it is tracked separately.

Covered by unit tests for both (unloaded column list, one line per contended
column) and e2e case 10, which drives two parameters onto one column and
asserts the banner plus the empty grid.
2026-10-28 09:54:48 +00:00
dc 388f418bf4 test(embed): pin the semantics of two params on one column
Build / Build-and-ng-test (pull_request) Successful in 5m28s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m39s
Build / Build-and-test-development (pull_request) Failing after 35m57s
Records that two parameters mapped to the same DC column are AND-ed, so the
result is the intersection: two different dates match no rows rather than
both dates' rows. Answers the 'filter one column on two dates' case from the
code rather than by assumption, and marks the single-parameter IN (...) shape
as the one that unions.
2026-10-28 09:35:36 +00:00
dc f83dc359e2 fix(embed): one parameter maps one column, and report typos on both sides
Build / Build-and-ng-test (pull_request) Successful in 5m42s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m20s
Build / Build-and-test-development (pull_request) Successful in 33m57s
Three follow-ups from review, all the same class of defect: an author error
that leaves an unfiltered grid looking like a working filter.

- A duplicate key in `va_parmap` silently kept whichever entry came last, so
  `va_parmap=DEPT-REGION&va_parmap=DEPT-OTHER` filtered OTHER with nothing to
  show for it. One VA parameter now maps one column: the first entry wins and
  a conflicting duplicate is reported. The same mapping written twice is not a
  conflict, and two different parameters mapping to one column stay valid -
  that is how two report objects filter the same column from their own
  parameters.
- A parameter-side typo (mapping `DROPDOWN_TAB1` when the report parameter is
  `DROPDOWN_TAB_1`) produced no clause, no banner and no console line.
  VaMessagingService now records every parameter key it has seen, across
  editor reloads, and the editor reports a mapping key that VA has never sent.
  Judged only once parameters are actually arriving, because VA sends just the
  parameters its query uses - an absent key is not on its own an error, and a
  key that turns up later stops being reported.
- The deferred first load cannot check a mapped column against metadata. On
  SAS `mp_filtercheck` aborts the save, so the load falls back to unfiltered;
  the banner now covers that path too, and an e2e case seeds the pre-bootstrap
  message buffer to prove it (and to prove a valid mapping filters the very
  first getdata).
2026-10-28 09:34:02 +00:00
dc 411f6c3354 fix(embed): warn when a va_parmap entry is ignored
Build / Build-and-ng-test (pull_request) Successful in 5m48s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m9s
Build / Build-and-test-development (pull_request) Successful in 33m7s
A `va_parmap` entry that cannot be used was dropped in silence, in both
places it can fail: at parse time (no `-` separator, a missing side, or a
column that is not a plain SAS name) and at apply time (the mapped column is
not in the table being opened). Either way the parameter is skipped, which
degrades to "no filter" - a typo in the report's URL then looks exactly like
a working filter over the whole table.

Both are now reported in terms the report author can act on:

- a warning banner in the editor (VA mode) naming the parameter, the column
  and the table, and pointing at the `va_parmap` entries in the URL
- console.warn for each distinct problem, so it is findable from the report's
  console too

The parser returns `{ map, problems }` rather than a bare Map, VaFilterService
gains `unresolvedMappings()` for the entries the loaded table cannot satisfy,
and the editor merges both sets for the banner (logging once per distinct set
rather than once per VA message).

The banner is covered by two e2e cases, one asserting its painted box inside
the viewport (VA hides the app header, so it has to stand on its own) and one
covering a parse-time failure before any VA message arrives. The alert markup
keeps the message inside the alert's text flow so it wraps, instead of being
laid out beside a title and pushed off the right edge.
2026-10-28 09:08:30 +00:00
allan f042ca82ea Merge pull request 'fix(ci): publish the docs with surfer 7, and rebuild release assets on demand' (#382) from fix/ci-surfer-app-password into release/7.20
Sites / deploy (push) Successful in 3m49s
Reviewed-on: #382
2026-10-28 08:49:33 +00:00
dc 15a1f34d52 feat(embed): map VA parameters to DC columns from the URL
Build / Build-and-ng-test (pull_request) Successful in 5m33s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m20s
Build / Build-and-test-development (pull_request) Successful in 32m46s
Two DC instances over the same table in one VA report could not filter the
same column from different parameters: parameter -> column matching is by
name, and two VA parameters cannot share a label.

The `va_parmap` URL parameter overrides that match per instance, one entry
per mapping (`va_parmap=<VA parameter>-<DC column>`, repeated for more):

  ...?embed=va&va_parmap=REGION_TAB1-REGION

The mapping is additive (unmapped parameters keep the name match), the
column is validated as a plain SAS name at parse time, and an entry only
applies where the named column exists in the table being opened - a miss
skips that parameter rather than falling back to the same-name match the
mapping exists to override. Entries split at the last hyphen, so a label
may itself contain hyphens, and the key matches the parameter label first,
then its internal name (pr###).

Parsed once at app init onto globals (like embed / vaApplyMode) so the
editor reloads a filter apply performs keep the mapping, and both filter
paths (the metadata build and the deferred initial load) honour it.

Also repairs embed-va.cy.ts: a cy.visit differing from the current URL
only in the hash is a same-document navigation, so ?embed=va never took
effect and both existing cases failed; their layout assertions also
described an earlier design (VA now opens read-only with an Edit button
and hides Filter). Cases 3 and 4 drive two mappings of the same column
through a real postMessage and assert the applied filter and row count.
2026-10-28 08:41:15 +00:00
dc 6486c80ed4 fix(ci): rebuild and upload release assets on demand
Build / Build-and-ng-test (pull_request) Successful in 5m38s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m33s
Build / Build-and-test-development (pull_request) Successful in 31m45s
2026-10-28 02:07:31 +00:00
dc dd482c4069 fix(ci): publish docs after the assets, with the webdoc app password 2026-10-28 02:05:10 +00:00
dc f58925b299 fix(ci): install doxygen for the SAS docs job 2026-10-28 02:02:57 +00:00
dc 25ffa71add fix(ci): publish the code documentation with surfer 7 and app passwords 2026-10-28 01:57:23 +00:00
semantic-release-bot 424b111737 chore(release): 7.19.0 [skip ci]
# [7.19.0](https://git.datacontroller.io/dc/dc/compare/v7.18.0...v7.19.0) (2026-10-28)

### Bug Fixes

* address the review findings on the harness and refresh changes ([36e4e88](36e4e88eb1))
* **ci:** dispatch the demo deploy with a tag ref and the actions scope ([698dea6](698dea6d6b))
* **ci:** restore the gitleaks ignore entries for the mock licence pair ([0d58be9](0d58be975d))
* **client:** open the SAS log page with noopener ([072a133](072a133702))
* **client:** refuse a truncated table in the multi-table upload ([1fb4818](1fb4818d21))
* **cypress:** apply the licence key when the estate has none ([be8f445](be8f445ae1)), closes [input#file-upload](https://git.datacontroller.io/input/issues/file-upload)
* **dc_execute:** use = comparisons, not IN, in the fail-fast guard ([fed43cb](fed43cb07e))
* **dc_getlibs:** exclude the DC libref from the dictionary scan ([f5aade5](f5aade5976))
* **dc_refreshcatalog:** terminate the generated statements ([1e22476](1e224763ba))
* **derived:** a fileref with nothing written is a FAIL, not an error ([9b3cf50](9b3cf50b9c))
* **derived:** address five review findings ([ccd468b](ccd468b335))
* **derived:** carry the placement sequence in mpe_derived ([be50cb5](be50cb5d08))
* **derived:** close four gaps in the MPE_DERIVED hook ([9914790](9914790777))
* **derived:** close the RLS and search gaps on derived columns ([9ef813d](9ef813d439))
* **derived:** close the two gaps in the derived/RLS invariant ([422f766](422f766cd4))
* **derived:** compare the outref as text, prove the file is empty ([7830b7b](7830b7bcfa))
* **derived:** do not drop a shadowed column from staging or downloads ([2d5de9a](2d5de9a324))
* **derived:** do not duplicate a shadowed column in the grid payload ([6a1128a](6a1128a463))
* **derived:** eight review findings ([561d7fc](561d7fc3ed))
* **derived:** enforce the ACTIVE picker pair in the extracted check ([1863076](1863076bc9)), closes [#366](#366)
* **derived:** enforce the documented limits, and handle derived filters in VIEW ([d3dec68](d3dec682aa))
* **derived:** filtering a format catalog no longer aborts the service ([469229a](469229aecb))
* **derived:** guard the bitemporal cut-off, coalesce the selectbox seed ([7fb981d](7fb981daa9))
* **derived:** guard the empty physical column list in the dep check ([5f5d868](5f5d8681b9))
* **derived:** hold a derived column read-only whatever the table's security ([5f443b0](5f443b0b16))
* **derived:** ignore a re-added name when collecting inactive rows ([6b5bf3e](6b5bf3e6be))
* **derived:** ignore inactive rows and VIEW rules in the submit guards ([85c89ca](85c89ca2a4))
* **derived:** keep targetds for plain datasets, test the empty file ([f8f6568](f8f65681e1))
* **derived:** make the model enforce the required columns, and leave VAR_FMT optional ([7efbf31](7efbf31ea5))
* **derived:** only the delete marker is a reserved name ([af31c36](af31c36b67))
* **derived:** open the statements file with a long record length ([ee31782](ee31782fbf))
* **derived:** pair the filterds %else with %do/%end ([88a38db](88a38db877))
* **derived:** raise the VAR_ORDER ceiling to 10000 ([27ddcfb](27ddcfb575))
* **derived:** refuse dependencies on inactive rows, and keep the RLS scope guard coherent ([7799a30](7799a300df))
* **derived:** reject a dependency on a row this submission deletes ([8806504](8806504967))
* **derived:** repair the validation rows for MPE_DERIVED ([9a047ad](9a047ad42e))
* **derived:** review feedback - cover the empty response, assert the last row ([e0da9f1](e0da9f1098))
* **derived:** stop the hooks aborting on their own macro variables ([688550a](688550a28d))
* **derived:** truncate the derived label on an even quote boundary ([1b5c4b3](1b5c4b3ecd))
* **derived:** verify on stripped values, and label with the variable name ([4d9ad38](4d9ad38df1))
* **editor:** a blank truncation note is not a warning ([f182179](f182179403))
* **editor:** keep decimal values when an uploaded file is edited manually ([04ce186](04ce1861e7))
* **getcolvals:** report an invalid value, not missing inputs ([4bcee70](4bcee706ff))
* **getdata.test:** pin the derived filter to a libref, and spell the table as it is ([45dce7b](45dce7b11b))
* **getdata:** clear the globals it declares before use ([359b570](359b570b85))
* **getddl.test:** ask for a table, not a whole library ([059cdf1](059cdf176a))
* **getrawdata:** declare mp_ds2md so the Markdown download can run ([661ee19](661ee19cfa))
* **lint:** clear the sasjs lint warnings on the files 7.19 touched ([355c46a](355c46a7b6))
* **makedata:** read KEEPME from every input row, and keep it out of the Viya seed ([d626cb8](d626cb8f2d))
* **mocks:** address the audit-pack review findings ([489819b](489819b051))
* **mocks:** carry the zip flags into the central directory ([c4021a4](c4021a42ff))
* **mocks:** clear the licence for published sites, not for the estate ([62f55db](62f55db170))
* **mocks:** copy the physical column list before deriving ([f411b7d](f411b7d269))
* **mocks:** derive before filtering in the viewer, and drop the columns ([60ec470](60ec470404))
* **mocks:** derive the retained key wherever rows are staged ([74060c0](74060c0585))
* **mocks:** drop derived variables before staging ([73d25b1](73d25b1220))
* **mocks:** key rk_underlying tables on their business key ([347dbb5](347dbb53b0))
* **mocks:** register MPE_DERIVED and seed its field rules ([ee3c878](ee3c8787ef))
* **mocks:** scope the derived hook's graph to the submission ([6cad23f](6cad23fce9))
* **mocks:** search the physical columns only in the viewer ([293022d](293022d46b))
* **mocks:** stop the mock advertising invalid licence keys ([fb36379](fb36379520))
* **mpe_assert_abort:** drop the unused line variable ([9c294c0](9c294c0ab5))
* **mpe_assert_abort:** take the abort's own MSG, tolerate a missing response ([0a52d20](0a52d20c5f))
* **mpe_execute:** fail fast when the child left no status ([a036386](a03638667f))
* **mpe_execute:** no apostrophe in the SAS 9 status note ([2204085](220408538b))
* **mpe_execute:** read an unknown state as unmeasured, not failed ([f2b1428](f2b1428ec3))
* name the startupservice launcher button "Show log" ([3445107](3445107d9a))
* offer the launcher on the globvars failure modal too ([ad2da45](ad2da45389))
* **refresh:** name each catalogue load uniquely in the audit trail ([46bacdd](46bacdd6d9))
* **refresh:** one audit folder per refresh call ([56b6620](56b6620ef4))
* **sites:** round-trip the committed licence before publishing ([4d462d3](4d462d310f))
* **test:** getdata test ([da30266](da30266278))
* **tests:** enable closeouts logging ([fdcd8fa](fdcd8fa4b3))
* **tests:** read the work location from _SASJS_WORK ([033ffd3](033ffd3025))
* **ui:** one error modal at a time ([81ccf75](81ccf758e9))
* **validatefilter:** build targetds without macro triggers in comments ([42355f1](42355f17bc))
* **validatefilter:** drop the dead nobs assignment, keep targetds precedence ([337ea8e](337ea8e5e0))

### Features

* **dc_getlibs:** dump the library list when debug is on ([582523d](582523dc87))
* **derived:** allow a variable to shadow a physical column ([12beeff](12beeff695))
* **derived:** declare VAR_DEPENDENCIES instead of VAR_ORDER ([813bce6](813bce6856))
* **derived:** optional VAR_LABEL on MPE_DERIVED ([7c9ce9a](7c9ce9a712))
* **derived:** pickers, and an ACTIVE default, for MPE_DERIVED ([87e794f](87e794f769))
* **derived:** report the dependency cycle path ([cee1a95](cee1a954ee))
* **derived:** take any text in VAR_LABEL, but not a line break ([78e88a4](78e88a4112))
* **derived:** validate MPE_DERIVED at submit time ([0b010f2](0b010f2c65))
* **derived:** validation rules and a VAR_TYPE dropdown for MPE_DERIVED ([aa6d50e](aa6d50e1ea))
* **derived:** VARIABLE_ORDER, so a derived variable can build on another ([33d043e](33d043ea92))
* **editor:** derived variables in the EDIT screen ([b0fd354](b0fd354ecd))
* **editor:** mark derived variables as computed in the EDIT grid ([6513d09](6513d09f05))
* **editor:** serve a truncated selection with a warning instead of aborting ([34691dc](34691dceec))
* **editor:** show the derived column label in the header tooltip ([cfe6647](cfe6647ca8))
* **mocks:** add getauditfile and the audit-folder artefacts ([752d944](752d944c20))
* **mocks:** implement derived variables in the JS services ([b4fae88](b4fae88ae1))
* **mocks:** validate a submitted MPE_DERIVED row like the hook does ([5d183d1](5d183d1350))
* run startupservice in a new window from its error modal ([40f6133](40f61331aa))
* **sites:** mint a licence for the published sites ([7d9df71](7d9df71fdb))

### Reverts

* **derived:** no shadowing, and no self reference ([5af8e6f](5af8e6fc35))
* **stagedata.test.4:** back to the MPE_LOADS evidence for now ([31aa859](31aa859153))
v7.19.0
2026-10-28 01:42:51 +00:00
allan 849153d4a2 Merge pull request 'ci(release): build the frontend before deploying mocked services' (#381) from fix/ci-release-build-frontend into main
Release / Build-production-and-ng-test (push) Successful in 4m42s
Release / Build-and-test-development (push) Successful in 25m42s
Release / release (push) Failing after 6m56s
Reviewed-on: #381
2026-10-28 01:08:49 +00:00
dc bc9e0996ec ci(release): build the frontend before deploying mocked services
Build / Build-and-ng-test (pull_request) Canceled after 1m32s
Build / Build-and-test-development (pull_request) Canceled after 0s
Lighthouse Checks / lighthouse (pull_request) Successful in 19m52s
2026-10-28 01:07:00 +00:00
allan 66039c91d7 Merge pull request 'feat: release 7.19 - derived variables, startupservice launcher, core v6' (#348) from release/7.19 into main
Release / Build-production-and-ng-test (push) Successful in 4m13s
Release / Build-and-test-development (push) Failing after 1m52s
Release / release (push) Skipped
Reviewed-on: #348
2026-10-28 00:52:14 +00:00
allan abb1ecbcab Merge pull request 'fix(validatefilter): drop the dead nobs assignment, keep targetds branches exclusive' (#380) from fix/validatefilter-branches into release/7.19
Sites / deploy (push) Successful in 4m2s
Build / Build-and-ng-test (pull_request) Successful in 5m37s
Lighthouse Checks / lighthouse (pull_request) Successful in 22m9s
Build / Build-and-test-development (pull_request) Successful in 33m26s
Reviewed-on: #380
2026-10-28 00:07:31 +00:00
dc 337ea8e5e0 fix(validatefilter): drop the dead nobs assignment, keep targetds precedence
Lighthouse Checks / lighthouse (pull_request) Canceled after 0s
Build / Build-and-ng-test (pull_request) Canceled after 0s
Build / Build-and-test-development (pull_request) Canceled after 0s
Review feedback on the merged fix:

- derived_nobs was assigned twice, the first one dead - the coalesced form below
  is the live value.  Removed, so there is one assignment to read.
- making the catalog test a plain %if after the derived-variables test let the
  catalog branch override it; the derived view won before and still should.  The
  catalog branch now requires derived_nobs=0, so the view keeps precedence and
  only a catalog with no derived configuration is blanked.

Two guarded if blocks, not %else %if: that construct is what the parser rejected
in this file before ('Expected DO not found'), and the first attempt at this
review point reproduced it - validatefilter and viewdata both failed to complete
again until it was replaced.

Verified: sasjs test -t new -> 64/64 (100%), TEST=0.
2026-10-28 00:05:19 +00:00
allan 709acfad73 Merge pull request 'fix(validatefilter): build targetds without macro triggers in comments' (#379) from fix/validatefilter-targetds into release/7.19
Sites / deploy (push) Successful in 3m50s
Build / Build-and-ng-test (pull_request) Successful in 5m14s
Build / Build-and-test-development (pull_request) Canceled after 0s
Lighthouse Checks / lighthouse (pull_request) Canceled after 14m43s
Reviewed-on: #379
2026-10-27 23:46:46 +00:00
dc 42355f17bc fix(validatefilter): build targetds without macro triggers in comments
Build / Build-and-ng-test (pull_request) Successful in 5m28s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m37s
Build / Build-and-test-development (pull_request) Successful in 33m19s
The suite failed to complete on the estate with

  ERROR: Expected %DO not found.
  ERROR: Skipping to next %END statement.

and, once the parser had lost sync, 'Apparent symbolic reference IS_FMTCAT not
resolved' - because the %let that defines it never ran.

Two causes, both in the code that decides which dataset the filter is validated
against:

- the comment explaining the catalog case contained the text '%if ...' and the
  macro processor resolves triggers even inside a comment, so the prose was
  executed as code and tripped the parser from there.  The comment now avoids
  macro triggers entirely.
- an empty value compared in a condition is a character operand, and
  %mf_nobs returns nothing when the dataset is missing.  It is wrapped in
  coalesce now, and the catalog flag is computed with %eval instead of a nested
  %if.

targetds is still blanked for a LIBREF.CATALOGNAME-FC name - which is what makes
mp_filterstore rewrite it to a cntlout dataset - and stays the plain libds
otherwise, as an empty targetds falls back to libds in the engine anyway.

The test now passes outref= and mdebug=&sasjs_mdebug rather than a hard-coded
1: the suite already owns that switch, and testinit turns it on by itself when
the run asks for _debug=2477 or 131, which is when the child's response and the
PROC HTTP trace are worth having.

Verified: sasjs test validatefilter -t new -> 3/3 (100%), six assertions, all
PASS, including the catalog and derived-variable cases.
2026-10-27 23:46:19 +00:00
dc 6ef2eec011 ci(sites): start the SASjs server with HTTP_PORT=5000
Sites / deploy (push) Successful in 3m45s
Build / Build-and-ng-test (pull_request) Successful in 5m5s
Build / Build-and-test-development (pull_request) Canceled after 5m11s
Lighthouse Checks / lighthouse (pull_request) Canceled after 7m59s
The Sites workflow was the one place that still started the server without a
listener, so it died with

  [error] Invalid environment variable(s) provided:
   - no listener is enabled
   - set HTTP_PORT and/or HTTPS_PORT (either may be used alone)

and the following step's 120s poll timed out. build.yaml, release.yaml and
lighthouse.yaml already carry the same line.
2026-10-27 23:26:26 +00:00
allan 1ef3e0a97b Merge pull request 'fix(refresh): one audit folder per refresh call' (#378) from fix/refresh-etlsource into release/7.19
Sites / deploy (push) Canceled after 2m14s
Build / Build-and-ng-test (pull_request) Successful in 5m12s
Build / Build-and-test-development (pull_request) Canceled after 0s
Lighthouse Checks / lighthouse (pull_request) Canceled after 13m28s
Reviewed-on: #378
2026-10-27 23:12:33 +00:00
dc 56b6620ef4 fix(refresh): one audit folder per refresh call
Build / Build-and-ng-test (pull_request) Successful in 5m34s
Build / Build-and-test-development (pull_request) Canceled after 0s
Lighthouse Checks / lighthouse (pull_request) Canceled after 18m57s
refreshcatalog.test and viewtables.test failed to complete with

  ERROR: could not create /tmp/DataController/<libref>/dc_staging/refreshcatalog:mpe_refreshcatalogs: + <stamp>

The staging folder a closeout writes its statements to is
&dc_staging_area/&etlsource, and etlsource defaulted to
'refreshcatalog:<step>:%mf_uid()'. A default is resolved when the macro is
compiled, not when it is called, so every refresh in a session reused the same
folder - the first step created it, the next one could not, and the abort took
the whole suite down. Whichever suite reached the staging area first decided
which others failed, which is why the count moved between runs.

etlsource now defaults to 0 and is set per call from
%mf_getuniquename(prefix=<step>): unique, a plain name rather than a path, and
resolved at the point of use.

Verified on the estate: refreshcatalog 4/4 (100%), viewtables 2/2 (100%).
2026-10-27 23:08:47 +00:00
allan fd272eec89 Merge pull request 'test(editor): assert the derived block marker, not a column name' (#377) from fix/derived-block-spec into release/7.19
Sites / deploy (push) Failing after 5m38s
Build / Build-and-ng-test (pull_request) Successful in 5m18s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m43s
Build / Build-and-test-development (pull_request) Canceled after 1m32s
Reviewed-on: #377
2026-10-27 22:25:30 +00:00
dc c463a4e03c test(editor): assert the derived block marker, not a column name
Build / Build-and-ng-test (pull_request) Successful in 5m22s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m58s
Build / Build-and-test-development (pull_request) Canceled after 17m25s
derived-variables.cy.ts:97 expected dc-derived-first on the REGION header,
because the comment assumed REGION opens the computed block. The grid follows
MPE_DERIVED order, and that order is LINE_VALUE, REGION, REGION_LABEL - so the
marker was on LINE_VALUE and correct. The spec asserted the wrong column, and
the development pipeline has been red on it since.

Dumping the header row on a local mock estate (sasjs/server desktop mode,
RUN_TIMES=js) shows:

  6 | LINE_VALUE | dc-derived-header dc-derived-first
  7 | REGION | dc-derived-header
  8 | REGION_LABEL | dc-derived-header beforeHiddenColumn htLastVisibleHeader

and the same dump against the unmodified release/7.19 client, so this was never
a client defect.

The assertion now states the invariant instead: at least one derived header, the
first of them opens the block, and no other one carries the marker. That holds
whatever order the control table returns the variables in.
2026-10-27 22:21:52 +00:00
allan 23015ac41f Merge pull request 'fix(lint): clear the sasjs lint warnings on the files 7.19 touched' (#374) from fix/lint-719-touched-files into release/7.19
Sites / deploy (push) Failing after 5m39s
Build / Build-and-ng-test (pull_request) Successful in 5m26s
Lighthouse Checks / lighthouse (pull_request) Successful in 20m43s
Build / Build-and-test-development (pull_request) Canceled after 20m56s
Reviewed-on: #374
2026-10-27 22:01:09 +00:00
dc a8f62bdfe2 docs(agents): say why webout is in ignoredMacros, and where the macros come from
Build / Build-and-ng-test (pull_request) Successful in 5m44s
Lighthouse Checks / lighthouse (pull_request) Successful in 21m29s
Build / Build-and-test-development (pull_request) Failing after 34m27s
The previous wording named only mpeinit and mpeterm, so the webout entry read as
an unexplained addition. webout is the linter's own default - sasjs compile
injects %webout into every service - and an explicit ignoredMacros list replaces
that default, so it has to be restated. 52 service files call %webout, which is
what dropping the entry would start reporting.
2026-10-27 21:47:19 +00:00
dc 355c46a7b6 fix(lint): clear the sasjs lint warnings on the files 7.19 touched
Build / Build-and-ng-test (pull_request) Successful in 5m30s
Lighthouse Checks / lighthouse (pull_request) Canceled after 14m27s
Build / Build-and-test-development (pull_request) Failing after 34m21s
sasjs lint (@sasjs/cli 6.2.5) reported 24 warnings across the 11 .sas files
this release touches. None of them are in those files now:

- mpeinit and mpeterm are added to ignoredMacros in sas/.sasjslint. Every
  service is compiled with a serviceInit program (sasjs/utils/
  serviceinitsas9|viya|server.sas), and all three declare both macros in
  their own header, which is how a service body's %mpeinit()/%mpeterm()
  resolves without a declaration of its own. An explicit ignoredMacros list
  replaces the default, so webout stays listed.
- Header entries nothing in the file calls are dropped: mp_binarycopy in
  getrawdata (it arrives through mp_streamfile, which calls it - the
  compiled program is unchanged), mp_jsonout in viewdata,
  mp_assertcolvals in validatefilter.test, and mx_execute in testsetup
  (every file that calls it declares it).
- Prose carrying "error"/"warning" is reworded. Where the text is log
  output, %put WARNING: is written %put %str(WARN)ING: so the line the
  log receives is unchanged.
- The server makedata service keeps its doxygen @warning tag - the tag is
  the content - so that file carries @sasjslint {"noBannedStrings": false}.
- Two comment lines in getdata are re-indented to the file's multiple of 2.

AGENTS.md records the ignoredMacros rule and the %str(WARN)ING convention so
the next warning is not silenced by reflex. Compile passes; the 93 warnings
that remain are all in files this release does not touch.
2026-10-27 21:27:54 +00:00