The viewer's search box does a full table search - every character column by case-sensitive CONTAINS, every numeric column by exact equality (%mp_searchdata). Against the JS mock backend it did nothing: services/public/viewdata.js ignored SEARCHTYPE/SEARCHVAL and always returned the whole table. It also ignored FILTER_RK and hard-coded FILTER_TEXT to blank, so the viewer's filter never filtered anything and its info bar had no clause to show - filtering.cy.ts's info-bar assertions could not be met by the mock.
So this PR fixes the mock, adds a demo table to exercise the feature, and adds the missing E2E coverage.
Applies the stored filter first via mpeFilterMaster, mirroring %mpe_filtermaster(VIEW,...) in viewdata.sas, and returns its WHERE text as sasparams.FILTER_TEXT.
Mirrors %mp_searchdata: CHAR = case-sensitive CONTAINS across every character column, NUM = exact equality across every numeric column, only CHAR/NUM trigger a search, and the search value is stripped of % & ; " exactly as the SAS service does.
Caps rows the way the real service does: MAXROWS 500, NOBS is the uncapped filtered count for a normal view and the capped match count for a search. The if _n_ < &DC_MAXOBS_WEBVIEW quirk of the real service is preserved.
A search with no matches returns no rows, so the client shows its "No data found with given conditions" panel; the single-empty-row fallback is kept for the normal-view-empty-table case only.
MPE_X_SEARCH: 1000 deterministic, obviously fictional deep-sea survey rows in DC_JSLIB, registered in MPE_TABLES. Columns in demo order: PRIMARY_KEY_FIELD, SITE_NAME, VESSEL, SAMPLE_COUNT, DEPTH_M, SPECIES, CRUISE_DATE, EXPEDITION_ID, NOTES.
Seeded so the searches below have exactly the stated results:
search
type
rows
why it is interesting
siphonophore
CHAR
3
only in the middle of a long NOTES value
Halcyon
CHAR
280
matches across two columns (a site and a vessel)
Oceanus Rise
CHAR
100
multi-word site name
Vampire squid
CHAR
100
multi-word species name
EXP-000
CHAR
9
partial match in a short code column
EXP-0001
CHAR
1
unique value
Trench
CHAR
300
3 of the 10 sites
trench
CHAR
0
the search is case sensitive
4210
NUM
2
repeated exact SAMPLE_COUNT, not a PK/depth/date
1000
NUM
12
exact DEPTH_M match, so not only keys match
2000
NUM
1
the last primary key
421
NUM
0
numeric search is exact, not a prefix match
filter VESSEL = 'RV Halcyon'
-
200
then Trench inside it -> 60
New spec - client/cypress/e2e/full-table-search.cy.ts
One test - searches the whole table from the viewer - that opens the table once and makes every variation a search in place. Twelve row-count beats: full view, a value buried mid-NOTES, a value that matches across two columns, multi-word site and species, a partial code, a unique value, case sensitivity, exact numeric matches (a repeated SAMPLE_COUNT, a DEPTH_M, the last primary key), a prefix that must not match, and a search inside a stored filter followed by clearing it. It follows the house pattern (helpers at the bottom, export {}, beforeAll logout + loginAndUpdateValidKey(true), beforeEach home then view/data).
The siphonophore beat scrolls the grid. That result is 3 rows, which is narrow enough for Handsontable to size the columns wider than the grid viewport (scrollWidth 1562 vs clientWidth 1340): a horizontal scrollbar appears and the rightmost column, NOTES - the column the match is actually in - is clipped. The cell text is in the DOM either way, so an assertion passes without scrolling; the scroll is there so a recording of the walkthrough shows why those three rows matched. scrollGridTo sets scrollLeft on #hotTable .wtHolder (Handsontable keeps the header clone in step) and asserts the value landed, and the beat asserts the full NOTES text before scrolling back for the following beats. It is a plain spec step, so CI runs it too.
Three of the beats legitimately match nothing (wrong case, partial number, value not present). They pin real behaviour, so they stay, but they are wrapped in negativeStep() and skipped when the spec is used to record a demo:
A single-row result renders (1 row, 9 cols), not (1 rows, ...), so assertRowCount picks its unit from the count it is given.
No artificial waits - CI runs the file with no env, so there is not a single cy.wait() on that path. Where the soft-selects need their suggestion list closed before the next click, the spec asserts on the closed state (#datalist_* has class hidden) rather than sleeping. A second recording-only flag, demoPause, adds a linger at each row count and is 0 unless the env var is set.
Added to the Cypress spec list in .gitea/workflows/build.yaml.
Skill
Adds .agents/skills/dc-cypress - standing up the mock backend, the spec conventions, the selectors that actually work (including the viewer filter modal - filtering.cy.ts's .btnCtrl .btnView is an editor control that does not exist in the viewer), the clipped-column scroll, and how the demo capture actually behaves: it is the browser window's content area and contains the Cypress runner, so the pane is (W - 450) x (H - 96), Electron is stuck at 1280x720, and a real Chromium browser honours --window-size - which is how a 16:9 pane (1470x827 from a 1920x1010 window) is obtained.
Verification
npx cypress run --browser electron --spec "cypress/e2e/full-table-search.cy.ts,cypress/e2e/viewer-labels.cy.ts,cypress/e2e/filtering.cy.ts,cypress/e2e/viewbox.cy.ts" -> 27 passing, 0 failing (1 / 5 / 9 / 12), re-run after the walkthrough was extended and the grid scroll added.
The same spec run as CI runs it (default 1600x900 viewport, no env, so the negative steps execute too) -> 1 passing, 37s.
Every search and filter count was verified directly against the mock service via POST /SASjsApi/stp/execute before any UI assertion was written.
cd client && npm run lint:check passes.
## Why
The viewer's search box does a full table search - every character column by case-sensitive CONTAINS, every numeric column by exact equality (`%mp_searchdata`). Against the JS mock backend it did nothing: `services/public/viewdata.js` ignored `SEARCHTYPE`/`SEARCHVAL` and always returned the whole table. It also ignored `FILTER_RK` and hard-coded `FILTER_TEXT` to blank, so the viewer's filter never filtered anything and its info bar had no clause to show - `filtering.cy.ts`'s info-bar assertions could not be met by the mock.
So this PR fixes the mock, adds a demo table to exercise the feature, and adds the missing E2E coverage.
## Mock changes - `sas/mocks/sasjs/services/public/viewdata.js`
- Applies the stored filter first via `mpeFilterMaster`, mirroring `%mpe_filtermaster(VIEW,...)` in `viewdata.sas`, and returns its WHERE text as `sasparams.FILTER_TEXT`.
- Mirrors `%mp_searchdata`: `CHAR` = case-sensitive CONTAINS across every character column, `NUM` = exact equality across every numeric column, only `CHAR`/`NUM` trigger a search, and the search value is stripped of `% & ; "` exactly as the SAS service does.
- Caps rows the way the real service does: `MAXROWS` 500, `NOBS` is the uncapped filtered count for a normal view and the capped match count for a search. The `if _n_ < &DC_MAXOBS_WEBVIEW` quirk of the real service is preserved.
- A search with no matches returns no rows, so the client shows its "No data found with given conditions" panel; the single-empty-row fallback is kept for the normal-view-empty-table case only.
- Response shape is otherwise unchanged.
## Demo table - `sas/mocks/sasjs/services/admin/makedata.js`
`MPE_X_SEARCH`: 1000 deterministic, obviously fictional deep-sea survey rows in `DC_JSLIB`, registered in `MPE_TABLES`. Columns in demo order: `PRIMARY_KEY_FIELD`, `SITE_NAME`, `VESSEL`, `SAMPLE_COUNT`, `DEPTH_M`, `SPECIES`, `CRUISE_DATE`, `EXPEDITION_ID`, `NOTES`.
Seeded so the searches below have exactly the stated results:
| search | type | rows | why it is interesting |
|---|---|---|---|
| `siphonophore` | CHAR | 3 | only in the middle of a long `NOTES` value |
| `Halcyon` | CHAR | 280 | matches across two columns (a site and a vessel) |
| `Oceanus Rise` | CHAR | 100 | multi-word site name |
| `Vampire squid` | CHAR | 100 | multi-word species name |
| `EXP-000` | CHAR | 9 | partial match in a short code column |
| `EXP-0001` | CHAR | 1 | unique value |
| `Trench` | CHAR | 300 | 3 of the 10 sites |
| `trench` | CHAR | 0 | the search is case sensitive |
| `4210` | NUM | 2 | repeated exact `SAMPLE_COUNT`, not a PK/depth/date |
| `1000` | NUM | 12 | exact `DEPTH_M` match, so not only keys match |
| `2000` | NUM | 1 | the last primary key |
| `421` | NUM | 0 | numeric search is exact, not a prefix match |
| filter `VESSEL = 'RV Halcyon'` | - | 200 | then `Trench` inside it -> 60 |
## New spec - `client/cypress/e2e/full-table-search.cy.ts`
One test - `searches the whole table from the viewer` - that opens the table once and makes every variation a search in place. Twelve row-count beats: full view, a value buried mid-`NOTES`, a value that matches across two columns, multi-word site and species, a partial code, a unique value, case sensitivity, exact numeric matches (a repeated `SAMPLE_COUNT`, a `DEPTH_M`, the last primary key), a prefix that must not match, and a search inside a stored filter followed by clearing it. It follows the house pattern (helpers at the bottom, `export {}`, `beforeAll` logout + `loginAndUpdateValidKey(true)`, `beforeEach` home then view/data).
The `siphonophore` beat scrolls the grid. That result is 3 rows, which is narrow enough for Handsontable to size the columns wider than the grid viewport (scrollWidth 1562 vs clientWidth 1340): a horizontal scrollbar appears and the rightmost column, `NOTES` - the column the match is actually in - is clipped. The cell text is in the DOM either way, so an assertion passes without scrolling; the scroll is there so a recording of the walkthrough shows *why* those three rows matched. `scrollGridTo` sets `scrollLeft` on `#hotTable .wtHolder` (Handsontable keeps the header clone in step) and asserts the value landed, and the beat asserts the full `NOTES` text before scrolling back for the following beats. It is a plain spec step, so CI runs it too.
Three of the beats legitimately match nothing (wrong case, partial number, value not present). They pin real behaviour, so they stay, but they are wrapped in `negativeStep()` and skipped when the spec is used to record a demo:
const skipNegative = `${Cypress.env('skipNegative')}` === 'true'
A single-row result renders `(1 row, 9 cols)`, not `(1 rows, ...)`, so `assertRowCount` picks its unit from the count it is given.
**No artificial waits** - CI runs the file with no env, so there is not a single `cy.wait()` on that path. Where the soft-selects need their suggestion list closed before the next click, the spec asserts on the closed state (`#datalist_*` has class `hidden`) rather than sleeping. A second recording-only flag, `demoPause`, adds a linger at each row count and is 0 unless the env var is set.
Added to the Cypress spec list in `.gitea/workflows/build.yaml`.
## Skill
Adds `.agents/skills/dc-cypress` - standing up the mock backend, the spec conventions, the selectors that actually work (including the viewer filter modal - `filtering.cy.ts`'s `.btnCtrl .btnView` is an editor control that does not exist in the viewer), the clipped-column scroll, and how the demo capture actually behaves: it is the browser window's content area and contains the Cypress runner, so the pane is `(W - 450) x (H - 96)`, Electron is stuck at 1280x720, and a real Chromium browser honours `--window-size` - which is how a 16:9 pane (1470x827 from a 1920x1010 window) is obtained.
## Verification
- `npx cypress run --browser electron --spec "cypress/e2e/full-table-search.cy.ts,cypress/e2e/viewer-labels.cy.ts,cypress/e2e/filtering.cy.ts,cypress/e2e/viewbox.cy.ts"` -> 27 passing, 0 failing (1 / 5 / 9 / 12), re-run after the walkthrough was extended and the grid scroll added.
- The same spec run as CI runs it (default 1600x900 viewport, no env, so the negative steps execute too) -> 1 passing, 37s.
- Every search and filter count was verified directly against the mock service via `POST /SASjsApi/stp/execute` before any UI assertion was written.
- `cd client && npm run lint:check` passes.
allan
changed target branch from main to fix/viya-test-suite-and-refreshcatalog-json2026-09-16 23:03:37 +00:00
The viewer's search box searches every column of a table - character columns
by case-sensitive CONTAINS and numeric columns by exact equality - but against
the JS mock backend the box did nothing, because the viewdata mock ignored
SEARCHTYPE/SEARCHVAL (and FILTER_RK) entirely.
Mock changes (sas/mocks/sasjs/services/public/viewdata.js):
- apply the stored filter first via mpeFilterMaster, mirroring
%mpe_filtermaster(VIEW,...) in viewdata.sas, and return its WHERE text as
sasparams.FILTER_TEXT (previously hard-coded blank, so the viewer's info bar
never showed a clause and the filter was never applied to the rows)
- mirror %mp_searchdata: CHAR = case-sensitive CONTAINS across every character
column, NUM = exact equality across every numeric column, only CHAR/NUM
trigger a search, search values stripped of % & ; " like the SAS service
- cap the rows the way the real service does (MAXROWS 500; NOBS is the uncapped
filtered count for a normal view, the capped match count for a search)
- a search with no matches now returns no rows, so the client shows its
"No data found with given conditions" panel; the single-empty-row fallback
stays for the normal-view-empty-table case
Mock data (sas/mocks/sasjs/services/admin/makedata.js):
- new MPE_X_SEARCH demo table: 1000 deterministic, obviously fictional
deep-sea survey rows, registered in MPE_TABLES. Seeded so a partial search
hits a value buried in the middle of a long text column, a value spanning two
columns, a repeated exact numeric, and rows that deliberately do not match.
New spec (client/cypress/e2e/full-table-search.cy.ts):
- seven tests covering open/full view, partial character search, case
sensitivity, numeric exact match, no match, search within a filter, and
clearing the search. No artificial waits - state assertions only.
- added to the Cypress spec list in .gitea/workflows/build.yaml
Also adds the dc-cypress skill (.agents/skills/dc-cypress) covering the mock
backend setup, the spec conventions, the selectors that actually work in the
viewer filter modal, and the fixed 1280x720 video capture.
Three of the cases legitimately match nothing - they pin the case sensitivity
of the character search, the exact-match semantics of the numeric search, and
the "no data found" handling. They are worth keeping as regression coverage
but they make a poor demo video.
They now use an `itNegative` alias that resolves to `it.skip` when the spec is
run with `--env skipNegative=true`, so the walkthrough recording only ever
shows searches that return rows:
npx cypress run --browser electron \
--spec cypress/e2e/full-table-search.cy.ts --env skipNegative=true
CI runs the file with no env, so all nine cases still run there. The positive
half of the old case-sensitivity case is split out as its own test (3.1) so the
recording keeps it, and the "no data found" assertion is extracted into
assertNoData().
Opening the table once and making every variation a search in place - rather
than one it() per variation, each paying for the beforeEach navigation again -
matches how a user works, makes the recorded walkthrough one continuous take,
and cuts the spec from 67s to 26s.
The no-match steps (wrong case, partial number, value not present) are now
wrapped in a negativeStep() guard driven by --env skipNegative=true, so the
recording still never shows an empty result while CI (no env) runs every step.
The walkthrough ends inside the filter: clearing the search leaves the filter
applied, which is asserted rather than assumed.
Seven more search beats (Halcyon across two columns, multi-word site and
species, partial expedition code, exact numeric matches on DEPTH_M and the
last primary key), all with counts verified against the mock service first.
The walkthrough now takes 38s in CI against 28s.
A single-row result renders "(1 row, 9 cols)", not "(1 rows, ...)", so
assertRowCount picks its unit from the count it is given.
demoPause adds a recording-only 1.5s linger at each row count (0 unless the
env var is set, so CI still runs the file with no waits), and the dc-cypress
skill now records how the demo capture actually works: a fixed 1280x720
window containing the runner, with the app zoomed to fit the AUT pane -
matching the viewport aspect to that pane (~1.33) fills it, and the crop
rect plus the cut points are measured from the frames.
The three-row siphonophore result is narrower than the grid viewport, so
Handsontable sizes the columns wider than it (scrollWidth 1562 vs
clientWidth 1340): a horizontal scrollbar appears and NOTES - the column the
match is actually in - is clipped. The cell text is in the DOM either way, so
the beat passed without it, but the demo read as three arbitrary rows.
scrollGridTo() sets scrollLeft on #hotTable .wtHolder, asserts the value
landed, and the beat asserts the full NOTES text and scrolls back for the
following beats. It is a plain spec step - CI runs it, no recording flag and
no cy.wait.
Also lets a recording size the browser window: the capture is the window's
content area, so RECORD_WINDOW_SIZE is passed through as --window-size for a
real Chromium browser (Electron ignores it). That is what makes a 16:9 pane
possible: (W-450)/(H-96) = 16/9 at 1920x923 of content, i.e. 1920x1010 of
window, for a 1470x827 pane and the repo's 1600x900 viewport at ~0.90 zoom.
Inert unless the env var is set, so CI is unaffected. dc-cypress updated with
the window-sizing recipe, the clipped-column scroll and the frame
mean/variance scan used to find the cut point.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Why
The viewer's search box does a full table search - every character column by case-sensitive CONTAINS, every numeric column by exact equality (
%mp_searchdata). Against the JS mock backend it did nothing:services/public/viewdata.jsignoredSEARCHTYPE/SEARCHVALand always returned the whole table. It also ignoredFILTER_RKand hard-codedFILTER_TEXTto blank, so the viewer's filter never filtered anything and its info bar had no clause to show -filtering.cy.ts's info-bar assertions could not be met by the mock.So this PR fixes the mock, adds a demo table to exercise the feature, and adds the missing E2E coverage.
Mock changes -
sas/mocks/sasjs/services/public/viewdata.jsmpeFilterMaster, mirroring%mpe_filtermaster(VIEW,...)inviewdata.sas, and returns its WHERE text assasparams.FILTER_TEXT.%mp_searchdata:CHAR= case-sensitive CONTAINS across every character column,NUM= exact equality across every numeric column, onlyCHAR/NUMtrigger a search, and the search value is stripped of% & ; "exactly as the SAS service does.MAXROWS500,NOBSis the uncapped filtered count for a normal view and the capped match count for a search. Theif _n_ < &DC_MAXOBS_WEBVIEWquirk of the real service is preserved.Demo table -
sas/mocks/sasjs/services/admin/makedata.jsMPE_X_SEARCH: 1000 deterministic, obviously fictional deep-sea survey rows inDC_JSLIB, registered inMPE_TABLES. Columns in demo order:PRIMARY_KEY_FIELD,SITE_NAME,VESSEL,SAMPLE_COUNT,DEPTH_M,SPECIES,CRUISE_DATE,EXPEDITION_ID,NOTES.Seeded so the searches below have exactly the stated results:
siphonophoreNOTESvalueHalcyonOceanus RiseVampire squidEXP-000EXP-0001Trenchtrench4210SAMPLE_COUNT, not a PK/depth/date1000DEPTH_Mmatch, so not only keys match2000421VESSEL = 'RV Halcyon'Trenchinside it -> 60New spec -
client/cypress/e2e/full-table-search.cy.tsOne test -
searches the whole table from the viewer- that opens the table once and makes every variation a search in place. Twelve row-count beats: full view, a value buried mid-NOTES, a value that matches across two columns, multi-word site and species, a partial code, a unique value, case sensitivity, exact numeric matches (a repeatedSAMPLE_COUNT, aDEPTH_M, the last primary key), a prefix that must not match, and a search inside a stored filter followed by clearing it. It follows the house pattern (helpers at the bottom,export {},beforeAlllogout +loginAndUpdateValidKey(true),beforeEachhome then view/data).The
siphonophorebeat scrolls the grid. That result is 3 rows, which is narrow enough for Handsontable to size the columns wider than the grid viewport (scrollWidth 1562 vs clientWidth 1340): a horizontal scrollbar appears and the rightmost column,NOTES- the column the match is actually in - is clipped. The cell text is in the DOM either way, so an assertion passes without scrolling; the scroll is there so a recording of the walkthrough shows why those three rows matched.scrollGridTosetsscrollLefton#hotTable .wtHolder(Handsontable keeps the header clone in step) and asserts the value landed, and the beat asserts the fullNOTEStext before scrolling back for the following beats. It is a plain spec step, so CI runs it too.Three of the beats legitimately match nothing (wrong case, partial number, value not present). They pin real behaviour, so they stay, but they are wrapped in
negativeStep()and skipped when the spec is used to record a demo:A single-row result renders
(1 row, 9 cols), not(1 rows, ...), soassertRowCountpicks its unit from the count it is given.No artificial waits - CI runs the file with no env, so there is not a single
cy.wait()on that path. Where the soft-selects need their suggestion list closed before the next click, the spec asserts on the closed state (#datalist_*has classhidden) rather than sleeping. A second recording-only flag,demoPause, adds a linger at each row count and is 0 unless the env var is set.Added to the Cypress spec list in
.gitea/workflows/build.yaml.Skill
Adds
.agents/skills/dc-cypress- standing up the mock backend, the spec conventions, the selectors that actually work (including the viewer filter modal -filtering.cy.ts's.btnCtrl .btnViewis an editor control that does not exist in the viewer), the clipped-column scroll, and how the demo capture actually behaves: it is the browser window's content area and contains the Cypress runner, so the pane is(W - 450) x (H - 96), Electron is stuck at 1280x720, and a real Chromium browser honours--window-size- which is how a 16:9 pane (1470x827 from a 1920x1010 window) is obtained.Verification
npx cypress run --browser electron --spec "cypress/e2e/full-table-search.cy.ts,cypress/e2e/viewer-labels.cy.ts,cypress/e2e/filtering.cy.ts,cypress/e2e/viewbox.cy.ts"-> 27 passing, 0 failing (1 / 5 / 9 / 12), re-run after the walkthrough was extended and the grid scroll added.POST /SASjsApi/stp/executebefore any UI assertion was written.cd client && npm run lint:checkpasses.Three of the cases legitimately match nothing - they pin the case sensitivity of the character search, the exact-match semantics of the numeric search, and the "no data found" handling. They are worth keeping as regression coverage but they make a poor demo video. They now use an `itNegative` alias that resolves to `it.skip` when the spec is run with `--env skipNegative=true`, so the walkthrough recording only ever shows searches that return rows: npx cypress run --browser electron \ --spec cypress/e2e/full-table-search.cy.ts --env skipNegative=true CI runs the file with no env, so all nine cases still run there. The positive half of the old case-sensitivity case is split out as its own test (3.1) so the recording keeps it, and the "no data found" assertion is extracted into assertNoData().