The demo rows had a random date each, so a date filter over the demo table
matched at most a couple of rows - which is not enough to show a date filter
working, in a report or in a test. The generated rows now cycle through 24
first-of-month dates over two years, about 20 rows each.
No test asserts a specific generated date; the fixed rows at the top of the
table are unchanged.
A date parameter whose display format is not DATE9 or a numeric date reached
DC as text like "January 1, 2026" and the value was dropped: the clause
builder produced nothing for that parameter, so with no mapping the editor
filtered on the other parameters only, with no banner to say the date had been
ignored. Reproduced inside a report - a SOME_DATE parameter set to
"January 1, 2026" left the FILTER line reading
(( SOME_DROPDOWN = 'Option 1' )) and the grid at the unfiltered row count.
parseMonthNameDate now accepts a month name spelled out or abbreviated, with
the day before or after it, comma optional and "." or "-" as separators, so
WORDDATE-style values ("January 1, 2026", "Jan 1, 2026", "1 January 2026",
"1-Jan-2026") convert like DATE9 does. Only the first three letters of the
month name are matched, case-insensitively.
A month and year with no day ("January 2026") is still rejected rather than
being read as the first of the month - inventing a day would silently filter
on rows the author did not choose.
Unit tests cover each accepted shape, the case-insensitive match, DATE9 still
parsing as before, and the month-without-day rejection.
The demo rows had a random date each, so a date filter over the demo table
matched at most a couple of rows - which is not enough to show a date filter
working, in a report or in a test. The generated rows now cycle through 24
first-of-month dates over two years, about 20 rows each.
No test asserts a specific generated date; the fixed rows at the top of the
table are unchanged.
A date parameter whose display format is not DATE9 or a numeric date reached
DC as text like "January 1, 2026" and the value was dropped: the clause
builder produced nothing for that parameter, so with no mapping the editor
filtered on the other parameters only, with no banner to say the date had been
ignored. Reproduced inside a report - a SOME_DATE parameter set to
"January 1, 2026" left the FILTER line reading
(( SOME_DROPDOWN = 'Option 1' )) and the grid at the unfiltered row count.
parseMonthNameDate now accepts a month name spelled out or abbreviated, with
the day before or after it, comma optional and "." or "-" as separators, so
WORDDATE-style values ("January 1, 2026", "Jan 1, 2026", "1 January 2026",
"1-Jan-2026") convert like DATE9 does. Only the first three letters of the
month name are matched, case-insensitively.
A month and year with no day ("January 2026") is still rejected rather than
being read as the first of the month - inventing a day would silently filter
on rows the author did not choose.
Unit tests cover each accepted shape, the case-insensitive match, DATE9 still
parsing as before, and the month-without-day rejection.
@sasjs/core 6.1.3 moved the derived variable field rules out of
mp_derivedvars into mp_derivedvarcheck, so the same scan can run before a
definition is used. DC carried its own copy of that list in
mpe_derived_check.sas, which is now deleted: the post edit hook calls the
core macro instead, and the hook keeps only the rules that belong to the
control table rather than the expression - LIBREF/DSN required, and ACTIVE
within the picker pair.
The scan is stricter than the copy it replaces, in three ways:
- VAR_NM may not be _N_, _ERROR_ or _IORC_ (automatic variables of the data
step)
- VAR_LOGIC may not exceed 4000 characters
- VAR_LOGIC may not call a function that submits code or reaches the file
system, the environment or the macro symbol table - a function name cannot
be assembled at run time, so this is a control the expression cannot evade
and it relaxes one: VAR_LABEL may carry a line break, which is flattened to a
space when the label is used, so only the remaining control characters are
refused.
The mock hook mirror is updated to the same rules, and stagedata.test.4
gains the two submit-time cases that pin the new wiring.
# [7.20.0](https://git.datacontroller.io/dc/dc/compare/v7.19.0...v7.20.0) (2026-10-28)
### Bug Fixes
* **ci:** install doxygen for the SAS docs job ([f58925b](f58925b299))
* **ci:** keep foreign release assets, install jq first, pin the CLI ([27686d3](27686d3aad))
* **ci:** publish against the tag this job cuts, not releases/latest ([8888734](888873421c))
* **ci:** publish docs after the assets, with the webdoc app password ([dd482c4](dd482c4069))
* **ci:** publish the code documentation with surfer 7 and app passwords ([25ffa71](25ffa71add))
* **ci:** publish the typedoc to the webdoc site, not the app default ([c5808a1](c5808a16eb))
* **ci:** rebuild and upload release assets on demand ([6486c80](6486c80ed4))
* **ci:** use the lockfile and the pinned CLI in the docs job ([af1813f](af1813f829))
* **ci:** verify the forge certificate on the release API calls ([cf7b21a](cf7b21a8b6))
* **cypress:** hold download out too, and retry once in CI ([c97e886](c97e8866db)), closes [#395](#395) [#398](#398)
* **cypress:** register the deleteFolder task, restore the spec order, hold metanav out ([c803773](c803773d3c)), closes [#395](#395)
* **deploy:** do not offer the deploy wizard inside a report ([80e8911](80e8911af8))
* **deploy:** keep the app query when opening the standalone URL ([e51953c](e51953ca8a))
* **deploy:** one definition of the standalone URL, and test the guards ([459ff2d](459ff2da37)), closes [#TCS](https://git.datacontroller.io/dc/dc/issues/TCS)
* **deploy:** strip the embed parameters by name, and scope the timezone runs to the VA specs ([f7e7538](f7e753863b))
* **embed:** build quarter ends from the calendar, not minus 24 hours ([7e3045a](7e3045a21b))
* **embed:** defer the column check until metadata loads, report AND-ed pairs ([2623a3f](2623a3fb4b))
* **embed:** log a parse-time mapping problem once ([0011f7e](0011f7e933))
* **embed:** make the va_parmap diagnostics say what they mean ([541a61b](541a61ba08))
* **embed:** match a mapping on the parameter label only, and warn only when the values differ ([e9216b0](e9216b0895))
* **embed:** never build a clause from a value that did not convert, and say so in the banner ([bd3c220](bd3c220af2))
* **embed:** one parameter maps one column, and report typos on both sides ([f83dc35](f83dc359e2))
* **embed:** record the parameter label only, so a name-keyed va_parmap is reported ([aa91daf](aa91daf9f7))
* **embed:** resolve transform values as local dates and give ranges whole days ([10157c1](10157c1159))
* **embed:** warn when a va_parmap entry is ignored ([411f6c3](411f6c3354))
* **makedata:** pair the %else of the keepme block so it parses in open code ([c0ea810](c0ea810766))
### Features
* **embed:** map VA parameters to DC columns from the URL ([15a1f34](15a1f34d52))
* **embed:** resolve va_parmap values through named date anchors ([88010c3](88010c3980))
makeDataServer.json seeded ADMIN=AllUsers, a Viya-style group that no account on
the shared estate belongs to. The seed still reports SUCCESS with a group the
caller is not in, and then every getdata/stagedata call aborts with
'<user> is not authorised to edit <libds> in the MPE_SECURITY table', which
reads like a broken deployment. The estate's accounts are in all-users, the
SASjs Server group every logged-in user has, so seed with that and describe the
rule in the estate-testing skill rather than telling the reader to override the
shipped value.
The run on release/7.20 went from four failing specs to two:
csv-limited PASS (order restored)
hook-programs PASS (order restored)
metanav held out, #395
download 7 of 7 fail
download now gets past its hooks - the deleteFolder task is registered - and
fails on the button it clicks:
AssertionError: expected '<button.btn.btn-sm.btn-success...>' to be 'visible'
This element is not visible because its ancestor has 'position: fixed' CSS
property and it is overflowed by other elements.
Something fixed-position sits over it. That needs a scrollIntoView or an overlay
dismissed, and each attempt costs a forty minute CI cycle, so it is held out with
#398 carrying the exact error and the suggested fix. The list reports 22 run, 2
excluded.
editor.cy.ts still fails one test of fifty-five, and it failed the same way in
both runs: 'Expected <div.app-loading> not to exist ... continuously found', a
load timeout on test 28 rather than an assertion. One retry in run mode absorbs
that, none interactively; a real failure still fails twice and the retry's output
is in the log.
The shared-estate cycle is two npm scripts and sasjs test, but the seed in
npm run server is load-bearing: without it testsetup 404s settings.sas and
59 of the 64 test programs fail to complete, which reads like a broken
deployment. Record the targets, the token-pair credentials, the ADMIN group
the estate actually uses, how to read the results, and the frontend
passphrase steps as a repo-local skill, and point the testing doc at it.
Four specs failed on release/7.20 once the list went from 19 to 24. Three of the
four are addressed here; the fourth is a spec that cannot pass and is now
excluded with an issue.
download.cy.ts - it clears the downloads folder in an afterEach through
cy.task('deleteFolder'), and that task was registered only in
cypress/plugins/index.js, which Cypress 10+ never loads unless it is required
from cypress.config.ts. The task is now registered in setupNodeEvents, where
Cypress looks, and the login in its beforeEach only fills the form when the form
is actually on screen - the unconditional version failed with 'the center of this
element is hidden from view' whenever the session was already live.
csv-limited.cy.ts and hook-programs.cy.ts - neither is broken; both depend on
where they sit in the run. csv-limited asserts the free-tier 'Locked Feature'
modal, which an earlier spec removes by planting a valid licence, and
hook-programs asserts an approval row against its target table before other specs
have added rows to the approval tables. The list is now the sequence that was
green for months, with csv-limited first and hook-programs second, and the newer
specs appended rather than inserted. The script says why order matters, because
the next person to alphabetise it will break the same two specs.
metanav.cy.ts - cannot pass, and says so in its own source: '// BLOCKER ...
.clr-accordion-header-button always null although it is present on the page'.
With the duplicate login in its beforeEach removed it now reaches that assertion
and fails there, so it is excluded from the list with #395 as the reason, and the
login fix is kept for whoever picks it up. The coverage check now distinguishes a
deliberate exclusion from a forgotten file, so nothing can go missing silently.
Not verified locally: the specs need an estate seeded by 'sasjs cbd -t server-ci'
plus makedata, which a scratch mock does not reproduce - the app never leaves
app-loading. The CI run is the verification.
semantic-release bumps the working-tree package.json before the release
commit, so the tag is known in the job. Using it for the asset upload and
both dispatches means an overlapping release cannot redirect them.
Only the assets this workflow uploads are removed before a re-run, so a
hand-attached asset on the release survives. jq is installed before the
tag resolve that parses its output, the SASjs CLI is installed at the
version the tag's sas/package.json pins, and the npmrc secret is appended
rather than overwriting the file.
The four shards made the release-branch run red, and the reasons are in the
logs rather than in the theory:
- every failure was a timeout, not an assertion - "cy.type() failed because
the center of this element is hidden from view", "Expected
<div.app-loading> not to exist ... continuously found". Four jobs on one
runner means four SASjs servers, four ng serve and four Chromes competing
for the same CPU, and the app under test starves;
- upload-artifact@v4 does not work on this runner at all:
"GHESNotSupportedError: @actions/artifact v2.0.0+, upload-artifact@v4+ ...
not supported on GHES", so the v3 -> v4 move I made on review advice broke
the artifact step outright.
Measured, from the one sharded run that did complete: 20m34s wall against 33-35
minutes for the single job, with peak concurrency 3 - so a third off, not the
four-fold the split suggests. That is not worth a flaky pipeline while a release
sits on it.
Kept from the sharding work, because those parts were the real gain:
- the spec list has one home, .gitea/scripts/cypress-specs.sh;
- Build-and-ng-test fails when a spec on disk is not listed, so the five that
had never run in CI cannot go missing again - and they do run now;
- videos are zipped and uploaded only on failure;
- one job means one server, which is also what the guards and the balance
arguments were protecting.
The shard and fan-in jobs are gone with the branch that carried them.
curl -k disables verification on calls that carry the release token, so a
man-in-the-middle on the runner's egress could capture a credential that
rewrites releases. The forge serves a publicly trusted certificate (the
runner verifies the same chain on every checkout), so the flag is
unnecessary: drop it from all fifteen calls in the two release workflows.
The standalone URL dropped every query parameter whose name started with
va_, not just va_parmap; the names are now enumerated, and a later
parameter with that prefix survives (the component spec pins the exact
standalone URL instead of a substring, so the query surviving is the
assertion, not an accident of the runner's own query).
The timezone runs are scoped to the VA specs, the only specs with local
calendar arithmetic, so the blocking job no longer runs the whole suite
twice more.
The gate could not fail. With needs: on four shards and no if: always(), a
failed shard skips the fan-in job instead of failing it, and a skipped job is
not a failure on the one check branch protection watches - so the gate was
silent in exactly the case it exists for.
It now runs with if: always() and fails on any shard result that is not
success. The count is checked as well: if the needs context does not resolve on
this runner the step fails saying so, rather than looping over nothing and
reporting green - which is the failure mode a bare loop would have.
Also removes what the last regeneration left behind: the coverage guard step
appeared twice in Build-and-ng-test (so it ran twice) and the shard 1 comment
block appeared three times. The file is regenerated from fix/va-parmap-review-572
rather than from my own previous commit, which is what caused the duplication.
Renaming the shards to cypress-shard-1..4 and adding a job named
Build-and-test-development that does nothing but needs all four.
That name is the status check the branch protection rule requires, so the check
now turns green only when every shard passed - a red shard 3 leaves it
unreported and the PR unmergeable, which is the behaviour the single job had.
No repo setting has to change, and adding a fifth shard later means adding it to
needs rather than remembering to edit branch protection, where forgetting fails
silently.
Review found that five specs ran in no shard - coltype-delete-record,
download, embed-va, metanav, submitted-details - and one of them is the VA
embed suite this release added, 454 lines that had never run in CI. The old
single --spec string omitted them too, so nothing regressed; they were simply
never wired up, and nothing in the repo marked them skipped.
The spec lists now live in .gitea/scripts/cypress-shards.sh, one place, so the
workflow no longer spells them out four times. A guard step in
Build-and-ng-test compares the shards against the specs on disk and fails when a
file belongs to none of them or to more than one - proven by removing a name and
watching it fail with that file listed.
Rebalanced for the five additions, on test count and spec count together, since
each spec pays a few seconds of startup:
shard 1 editor, csv, metanav 57 tests
shard 2 excel, filtering, download, browser-info,
excel-multi-load, hook-programs 49 tests
shard 3 embed-va, viewbox, licensing, filter-panel, liveness 43 tests
shard 4 viewer-labels, row-cell-limits, derived-variables,
ddl-export, coltype-delete-record, submitted-details,
startupservice-launcher, stage, full-table-search,
csv-limited 27 tests
24 specs / 176 tests, each in exactly one shard.
upload-artifact moved from the retired v3 to v4; this change multiplies that
step by four, so it is worth not carrying the dead major forward.
Not in this commit: the three new check names still need adding to branch
protection, which needs repo admin.
Sits on top of fix/va-parmap-review-572 (#391) so the two changes to this
workflow land in one order; the timezone step that PR adds to
Build-and-ng-test is untouched.
The development job ran all nineteen specs in one Cypress invocation on one
runner, so its wall time was the sum of every spec - around half an hour on a
green run, with the whole pipeline waiting on it.
Four jobs now, each with its own runner and its own SASjs server, taking a
subset:
shard 1 editor (55 of the suite's ~148 tests)
shard 2 excel, filtering, csv-limited, csv, stage
shard 3 viewbox, licensing, filter-panel, browser-info, excel-multi-load, liveness
shard 4 viewer-labels, row-cell-limits, derived-variables, ddl-export,
hook-programs, startupservice-launcher, full-table-search
The split is balanced on test count, not file count: editor.cy.ts alone holds 55
tests, so an even spread of files would leave one shard running twice as long as
the others. Its own duration is the floor the pipeline can reach.
The setup steps are identical in every shard, so they are defined once and
aliased with a YAML anchor; each job differs only in SHARD and SPECS. Shard 1
keeps the original job name so an existing required status check still matches.
Video is still recorded - measured at 91s with it and 92s without on the same
spec, so the recording is not what takes the time. What it did cost was the zip
and artifact upload of every run's videos, now guarded by if: failure().
Review follow-up on the embed guard.
The spec asserted the isEmbedded getter, which mirrors the template rather
than exercising it: dropping !isEmbedded from the @if guards left every test
green. The spec now renders the component in a TestBed and asserts the DOM -
#TCS present standalone and absent for embed=va and embed=true, and the
deploy section withheld after the terms are accepted when embedded. Both
guards were mutation-checked by removing them, which fails the spec.
deployUrl and onNavigateToHome computed the standalone URL two different
ways: origin+pathname in one, location.href.split('#')[0] in the other,
which kept the query string, so a tab opened from ?embed=va#/deploy stayed
embedded. Both now call standaloneUrl() in standalone-url.ts, which parses
the href and drops the query and the fragment. It is tested with a
hash-bearing input, and the navigate test asserts the opened URL carries
neither '?' nor '#', so a regression to split('#') fails.
The self-destruct block ends with a single-statement %else in open code:
%if &keepme ne 1 %then %do;
%mv_deletejes(path=&root/admin, name=makedata);
%end;
%else %put NOTE: &_program retained, as KEEPME was requested;
Outside a macro SAS pairs %else only with a %do branch, so this stops with
'ERROR: Expected %DO not found' and 'ERROR: Skipping to next %END statement'
after the deployment has completed. On Viya the job then reports state error
with jobConditionCode 1012, so a successful deployment looks like a failure
exactly when someone is watching a first install.
Both branches now use %do/%end, which is the pattern the repo uses elsewhere
for this reason - see db/migrations/20261003_row_and_cell_limits.sas.
The same block is fixed in the server variant. The two delete calls also gain
their terminating semicolons: the macro call worked without them, but their
absence is what made the block look responsible for the error.
Behaviour is unchanged: the program still deletes itself unless KEEPME was
requested.
Deployment needs an authenticated, standalone session, and the person
looking at a report object is not the person who installs the app. Offering
the wizard there produces a screen whose controls cannot work - the failure
we hit while testing a report: T&C accepted, credentials filled, DEPLOY
clicked, and nothing happened, because the embedded app has no session to
deploy with.
An embedded instance (embed=va or embed=true) now shows what the state is
and where to deploy from, instead of a form that cannot submit. Standalone
behaviour is unchanged.
Two review findings, both about a filter that looks applied and is not:
- The transform branch of the clause builder interpolated whatever
convertJsDateToSasDate returned, so an Invalid Date reaching the helper
would have put a literal NaN in the WHERE clause. A non-finite result is
now dropped, and unsupportedTransforms reports the parameter, so the
author sees a banner instead of a broken filter.
- The range count check counted values that survived date parsing, while the
builder counts resolved bounds. A RANGE sent two values where one did not
parse therefore built a BETWEEN covering one period with no warning. The
check now counts what VA sent, the builder refuses a range with more than
one value rather than picking one, and a value set that is partly
unparseable gets its own line ('1 of its 2 values was not a date - the
filter covers only the rest').
The banner also now says what to do: it names the URL parameter to edit and
shows the entry grammar with an example, since the reader is the report
author and has no other way to act on it.
Conflict was valueSignature: the transform branch already builds a clause
value set through buildClauseValueSet, so the helper delegates to it and
takes the parameter's transform, keeping derived dates in the comparison.
Two review findings, plus a scope decision.
A va_parmap key is now matched against the parameter's label only. Matching
the internal name (pr273) as well meant one parameter answered to two keys,
so an entry written against the name and one against the label could never
be told apart: the label entry won and the name entry was silently dead -
no parse problem, no unresolved line, no unmatched line. Dropping the
fallback removes the ambiguity, and an entry keyed on the internal name now
shows up as an unmatched mapping like any other typo.
sameColumnMappings no longer reports two parameters on one column when they
carry the same value: col = 'x' AND col = 'x' filters exactly as one clause
does, yet the banner asserted that different values would show no rows. The
clause value each parameter produces is now compared, and the warning is
raised only when they differ. Parameters that produce no clause at all are
excluded, since they cannot conflict with anything.
A transform resolves a report value to dates using local calendar
components, so a test that passes in UTC says nothing about a browser west
of it. test:tz-west and test:tz-east run the whole suite under
America/New_York and Pacific/Auckland; with the UTC construction this test
pair replaces, the bare-number round-trip case fails under the first.
Not wired into CI: it triples the run for a property that is already
pinned by the round-trip assertion, so it stays a local and pre-release
check.
Brings in the diagnostics fixes from the base branch; conflicts were the
paramMap type widening to VaParmapTarget and the new paramsStable argument,
both of which the transform branch already carries.
Review findings:
- A bare SAS number was turned into a UTC instant and then read with local
accessors, so west of UTC the value landed a day early and the anchor was
computed from the wrong quarter; the round trip to a SAS day number was
one out. Built with local calendar construction instead, matching
parseDate and convertJsDateToSasDate.
- RANGE passed keepTime to both bounds, so a DATETIME range started at the
value's clock time on day one and ended at it on the last day, dropping
the rest of both boundary days. A range now covers whole days: period
start at midnight, period end at 23:59:59.
- A RANGE fed more than one parameter value produced no clause and no
warning, reading as a filter that matched nothing. One BETWEEN cannot
express several periods, so the count mismatch is now reported.
semantic-release-bot
allan