Compare commits
12
Commits
44bc7f7fea
...
v7.12.0
| Author | SHA1 | Date | |
|---|---|---|---|
 semantic-release-bot
|
42e02cdb05 | ||
 Yury
|
347923900f | ||
 allan
|
a4c3989c26 | ||
|
4gl
|
0392a81cbd | ||
|
4gl
|
f9ea53cf78 | ||
|
4gl
|
8fb58eb36e | ||
|
4gl
|
180c2477ed | ||
|
allan
|
f171375899 | ||
|
YuryShkoda
|
57db1179a9 | ||
|
allan
|
d13fab267f | ||
|
allan
|
aaf406b386 | ||
|
YuryShkoda
|
39c8855f37 |
@@ -0,0 +1,86 @@
|
||||
# Regex Validations (HARDREGEX / SOFTREGEX)
|
||||
|
||||
This document describes how the regex validation rules work internally. For user-facing documentation see `docs/dcc-validations.md` in the `docs.datacontroller.io` repo.
|
||||
|
||||
## Overview
|
||||
|
||||
Two validation rule types in `MPE_VALIDATIONS` validate cell values against a regular expression supplied in `RULE_VALUE`:
|
||||
|
||||
- `HARDREGEX` - submission-blocking. A failing value is rejected by the cell validator and painted red (HOT's own `htInvalid` class), so the row cannot be submitted.
|
||||
- `SOFTREGEX` - display-only warning. A failing value is painted yellow (`dc-warning-cell` class) with a tooltip, but submission is not blocked.
|
||||
|
||||
Both rule types are selectable in the MPE_VALIDATIONS RULE_TYPE dropdown; they were added to the selectbox seed data in `sas/sasjs/macros/mpe_makedata.sas` and via the optional migration `sas/sasjs/db/migrations/20260720_v7.12_release.sas`. `RULE_VALUE` is limited to 128 characters, which constrains very long patterns.
|
||||
|
||||
## Config-time validation (SAS side)
|
||||
|
||||
`sas/sasjs/services/hooks/mpe_validations_postedit.sas` runs `prxparse()` on any staged HARDREGEX/SOFTREGEX rule value and aborts the edit with a list of offending `libref.table.column` references if the pattern is invalid. This is a best-effort syntax check to catch typos at config time; an empty pattern is treated as valid (it matches everything in JS). Rows marked for delete are skipped.
|
||||
|
||||
Because patterns must pass `prxparse`, rule values are authored in the SAS PRX delimiter form `/pattern/flags` (although a bare pattern is also accepted for backwards compatibility).
|
||||
|
||||
## Frontend evaluation
|
||||
|
||||
All regex handling lives in the client; there is no server-side re-validation of data values. The per-cell decision flow:
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[Cell value] --> B{Row marked for delete<br/>and not a PK column?}
|
||||
B -- Yes --> Z[No validation / no warning]
|
||||
B -- No --> C{isRegexRuleExempt?<br/>blank, or "." on a numeric column}
|
||||
C -- Yes --> Z
|
||||
C -- No --> D{HARDREGEX rule on column?}
|
||||
D -- Yes --> E{Pattern matches?}
|
||||
E -- No --> F[Invalid: submission blocked,<br/>red htInvalid + REGEX tooltip]
|
||||
E -- Yes --> J[Valid]
|
||||
D -- No --> G{SOFTREGEX rule on column?}
|
||||
G -- Yes --> H{Pattern matches?}
|
||||
H -- No --> I[Warning: yellow dc-warning-cell<br/>+ REGEX tooltip, submission allowed]
|
||||
H -- Yes --> J[Valid]
|
||||
G -- No --> J
|
||||
```
|
||||
|
||||
A malformed pattern never reaches this flow: it is treated as always-valid (HARDREGEX) / never-warn (SOFTREGEX) with a `console.warn`, rather than breaking the editor.
|
||||
|
||||
### `client/src/app/shared/dc-validator/utils/parseRegexRule.ts`
|
||||
|
||||
Converts an authored SAS PRX pattern into a JavaScript `RegExp`:
|
||||
|
||||
1. If the value matches `/^\/(.*)\/([a-z]*)$/s`, the delimiters are stripped and the trailing flags are passed to `new RegExp(body, flags)`. Otherwise the value is used as-is (backwards compatibility with bare patterns).
|
||||
2. Three mechanical Perl→JS translations are applied to the body:
|
||||
- a leading `(?i)` inline modifier is removed and folded into the `i` flag;
|
||||
- `\Q...\E` literal sequences are replaced with escaped literal text;
|
||||
- `\A` → `^` and `\z` → `(?![\s\S])` (end-of-string anchor).
|
||||
3. Perl-only constructs that would need a capture-group-renumbering rewrite (atomic groups `(?>...)`, possessive quantifiers `a++`) are deliberately NOT translated. They throw from `new RegExp`, and every caller treats a throw as "always valid / never warn" (see below) rather than breaking the editor.
|
||||
|
||||
### `client/src/app/shared/dc-validator/utils/isRegexRuleExempt.ts`
|
||||
|
||||
Blank values (`undefined`, `null`, `''`) are exempt from pattern matching on any column type - use a separate NOTNULL rule if populated values must also be enforced. On numeric columns the plain SAS missing (`.`) is also exempt; special missings (`.A`-`.Z`, `._`, bare letters) are NOT exempt anywhere - being deliberately set, they are real values the pattern must match (and on a character column even `.` is real text). `isSpecialMissing` from `@sasjs/utils` is deliberately not used: its optional-dot regex would exempt any single-letter character value ("d", "z") before the regex ever ran. The check takes an `isNumeric` flag, passed by all callers (the dq validator via `dqValidate(rules, value, colType === 'numeric')`, the warning renderer via a `makeRegexWarningRenderer` argument, and `failsSoftRegex` via the column's HOT type).
|
||||
|
||||
### HARDREGEX - blocking validation
|
||||
|
||||
`HARDREGEX` is implemented as a cell validator in `client/src/app/shared/dc-validator/validations/dq-validation.ts`. It returns `true` (valid) for exempt values and for patterns that fail to compile (with a `console.warn`), and otherwise returns `parseRegexRule(ruleValue).test(value.toString())`. Returning `false` makes HOT mark the cell invalid, block submission, and paint it red via its standard `htInvalid` styling.
|
||||
|
||||
### SOFTREGEX - warning renderer
|
||||
|
||||
`client/src/app/editor/utils/regex-warning-renderer.ts` builds a display-only Handsontable renderer (registered per-column by `DcValidator.setupRules` in `client/src/app/shared/dc-validator/dc-validator.ts`). It never returns false; it only:
|
||||
|
||||
- adds a `REGEX: <pattern>` tooltip (`td.title`) when a rule fails;
|
||||
- adds the yellow `dc-warning-cell` class when only SOFTREGEX fails.
|
||||
|
||||
`DcValidator.failsSoftRegex(col, value)` provides the same logic outside the grid (e.g. the edit-record screen).
|
||||
|
||||
### Precedence: HARD and SOFT on the same column
|
||||
|
||||
Only one regex ever runs per column. If a HARDREGEX rule exists, SOFTREGEX is ignored entirely - never compiled, never evaluated - regardless of whether individual cell values pass or fail the hard rule. A value failing HARDREGEX gets the red invalid styling (blocking submission) plus a `REGEX: <pattern>` tooltip; a SOFTREGEX-only column warns in yellow without blocking. This holds in both the renderer and `failsSoftRegex`.
|
||||
|
||||
### Other behaviour
|
||||
|
||||
- Rows marked for delete (`_____DELETE__THIS__RECORD_____ = 'Yes'`) are not warned/validated by the renderer (except primary key columns, which still validate).
|
||||
- A malformed pattern never breaks the editor: the dq validator treats it as always-valid and the renderer disables the warning, logging to the console instead.
|
||||
- The pattern is used as authored - it is NOT auto-anchored. Authors must include `^`/`$` to match the entire cell value.
|
||||
- Column info: `client/src/app/shared/utils/col-info-html.ts` shows the applied pattern in the column-info dropdown - the HARDREGEX pattern if one exists (it is the rule actually applied when both are present), otherwise the SOFTREGEX pattern, otherwise nothing.
|
||||
|
||||
## Testing
|
||||
|
||||
- Unit tests: `parseRegexRule.spec.ts`, `isRegexRuleExempt.spec.ts`, `dq-validation.spec.ts`, `dc-validator.spec.ts` (under `client/src/app/shared/dc-validator/`), `client/src/app/editor/utils/regex-warning-renderer.spec.ts`, `client/src/app/shared/utils/col-info-html.spec.ts`.
|
||||
- E2E: `client/cypress/e2e/editor.cy.ts`.
|
||||
- SAS side: `sas/sasjs/services/editors/stagedata.test.3.sas`, plus seed data in `mpe_makedata.sas`: HARDREGEX "SOME_CHAR must contain 'the' or 'data'" (`/the|data/i`), SOFTREGEX "SOME_CHAR should contain the letter 't'", SOFTREGEX on PRIMARY_KEY_FIELD (`/^\d+$/` - yellow if the key contains a decimal), and HARDREGEX on SOME_SHORTNUM (`/^(??$).*/` - values 1-5 blocked; generated data starts at 6 so demos aren't blocked accidentally).
|
||||
@@ -12,9 +12,12 @@ This is different from `.sas` files, where a maximum line length applies. The no
|
||||
|
||||
Rationale: hard-wrapped prose produces noisy diffs when sentences are edited and reflowed, and Markdown renderers already handle wrapping.
|
||||
|
||||
## SAS files
|
||||
## Linting (required before "done")
|
||||
|
||||
After creating or modifying any `.sas` files, run `sasjs lint` from the `sas/` directory and ensure the files you touched have no lint warnings (the repo currently has pre-existing warnings in other files, which can be ignored).
|
||||
Never consider a change complete until the relevant linters pass on the files you touched — do not rely on the user's pre-commit hooks to catch it:
|
||||
|
||||
- **Client (TypeScript/HTML/etc.)**: run `npm run lint:check` from the `client/` directory (prettier). Fix any failures with `npm run lint:fix`.
|
||||
- **SAS**: after creating or modifying any `.sas` files, run `sasjs lint` from the `sas/` directory and ensure the files you touched have no lint warnings (the repo currently has pre-existing warnings in other files, which can be ignored).
|
||||
|
||||
## No external assets
|
||||
|
||||
@@ -23,3 +26,7 @@ Data Controller must run entirely locally (offline / on-prem, no internet access
|
||||
## The .agent folder
|
||||
|
||||
Agent-related content lives in `.agent/`: technical/agent-facing documentation goes in `.agent/docs/` (not `docs/`), and skills in `.agent/skills/`. When writing explanatory or technical docs about the codebase, put them in `.agent/docs/`.
|
||||
|
||||
## Code comments and test names
|
||||
|
||||
Never reference items that are not active parts of the repository — no "the original bug", "regression from this fix", "this session/PR/commit", or similar ephemeral context. Comments and test names must be self-contained: describe the behaviour being asserted, not the history of how it was discovered. The one exception is a literal link to a ticket/issue tracker.
|
||||
|
||||
@@ -1,3 +1,37 @@
|
||||
# [7.12.0](https://git.datacontroller.io/dc/dc/compare/v7.11.0...v7.12.0) (2026-07-28)
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* adding REGEX validations to mpe_x_test ([bd798b4](https://git.datacontroller.io/dc/dc/commit/bd798b424a7c4b90d613425348d3f99eeb49b025))
|
||||
* default value for label ([a3e46a9](https://git.datacontroller.io/dc/dc/commit/a3e46a968ef08295347b230235f1e79dfa51c5a2))
|
||||
* **deps:** retarget Angular upgrade to 20, not 21 (CI install was broken) ([cac9244](https://git.datacontroller.io/dc/dc/commit/cac9244f92544f432078125c312d05c36ed4b855))
|
||||
* ensure only one REGEX applies at a time ([8fb58eb](https://git.datacontroller.io/dc/dc/commit/8fb58eb36e05fdf7262459b05e53643b3ca672f7))
|
||||
* ensure that no assets (including og links) ever fetch from external sources ([33dcb98](https://git.datacontroller.io/dc/dc/commit/33dcb989d3ae0c5ee0a13fe64195862a945c4348))
|
||||
* include peer dependencies in package-lock for npm ci in pipeline ([b51c770](https://git.datacontroller.io/dc/dc/commit/b51c770782a8b62816e5145496dbea9fb83059e0))
|
||||
* licensecheker ([f60bcef](https://git.datacontroller.io/dc/dc/commit/f60bcef58381ba269415640fa19a27305274939a))
|
||||
* **lint:** remove redundant optional chaining ([d881290](https://git.datacontroller.io/dc/dc/commit/d881290618f25ddb6335125f1a08b5aaf56a8c48))
|
||||
* optimisation, renamed values for DDTYPE to save space ([cfb60e5](https://git.datacontroller.io/dc/dc/commit/cfb60e5e4bfd37dc79fa4e9a6c6649ab1a61f2e4))
|
||||
* patch npm audit vulnerabilities in sas and client dependencies ([e22edf7](https://git.datacontroller.io/dc/dc/commit/e22edf7ed3aa54fc08411cf90d51df9ec30877c3))
|
||||
* **query:** isolate viewbox filter state from the base table's ([7a35cf4](https://git.datacontroller.io/dc/dc/commit/7a35cf4a458791b26ec8141c0c84d97fcb555145))
|
||||
* regenerate client lockfile to resolve Angular peer-dependency drift breaking npm ci ([05fe474](https://git.datacontroller.io/dc/dc/commit/05fe4744d58e7985ecc5ef5f9c1969dbab2d2efb))
|
||||
* **regex:** special missing handling ([180c247](https://git.datacontroller.io/dc/dc/commit/180c2477ed9ae44b4e479e81ec642f40580bcbc8))
|
||||
* removing low severity warning in npm audit ([2a771bb](https://git.datacontroller.io/dc/dc/commit/2a771bb91acaaa16aba44e2d7ddb51b8b7123c10))
|
||||
* removing thousand seperator from plain numerics in EDIT mode ([0392a81](https://git.datacontroller.io/dc/dc/commit/0392a81cbd9a634590ca7a60a316cd12c690a5a5))
|
||||
* **validations:** parse SAS PRX /pattern/flags syntax in HARDREGEX/SOFTREGEX ([7ed3730](https://git.datacontroller.io/dc/dc/commit/7ed3730ae3145dffd639ce3d2f831447d78f4e85))
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* **docs:** adding agents.md and docs for RLS ([359d833](https://git.datacontroller.io/dc/dc/commit/359d833406ace79f96d03c08a5c72fbd3df29442))
|
||||
* **editor:** add HARDREGEX/SOFTREGEX validation rules ([17e4802](https://git.datacontroller.io/dc/dc/commit/17e48028955d7b091e53bad9d7f9e3a089b53af9))
|
||||
* **editor:** evaluate HARDREGEX/SOFTREGEX independently instead of hard-wins precedence ([57db117](https://git.datacontroller.io/dc/dc/commit/57db1179a97973ecf0f711bda6057383de546dd3))
|
||||
* **editor:** show applied HARDREGEX/SOFTREGEX pattern in column info dropdown ([39c8855](https://git.datacontroller.io/dc/dc/commit/39c8855f37477f06d8b8cf5c97e04543070e8385))
|
||||
* **regex:** backend validations on regex strings ([d2c93a4](https://git.datacontroller.io/dc/dc/commit/d2c93a46facb386fd31661c02c0d3d34f38f6e43))
|
||||
* using ALL libraries as validation in MPE_SECURITY. Closes [#279](https://git.datacontroller.io/dc/dc/issues/279) ([62ff0ae](https://git.datacontroller.io/dc/dc/commit/62ff0aee4a976184de65d87ea3c8b8b7cb777938))
|
||||
* validation checks to prevent incompatible RLS rules (eg REPLACE load type). Closes [#211](https://git.datacontroller.io/dc/dc/issues/211) ([ea00c5a](https://git.datacontroller.io/dc/dc/commit/ea00c5afad0daf2a66cdb206f2a7481b992192bf))
|
||||
* validation on RLS for REPLACE, + docs + tests. Closes [#211](https://git.datacontroller.io/dc/dc/issues/211) ([7378f3b](https://git.datacontroller.io/dc/dc/commit/7378f3ba3014141008b4dbde0b89c21a6ce02f69))
|
||||
|
||||
# [7.11.0](https://git.datacontroller.io/dc/dc/compare/v7.10.1...v7.11.0) (2026-07-20)
|
||||
|
||||
|
||||
|
||||
@@ -212,6 +212,12 @@ context('editor tests: ', function () {
|
||||
.clear()
|
||||
.type('not-an-email{enter}')
|
||||
.then(() => {
|
||||
getCellByHeaderAndRow(1, 'REGEX_HARD_COL').should(
|
||||
'have.attr',
|
||||
'title',
|
||||
'REGEX: /[\\w.]+@[\\w]+\\.[a-z]{2,}/'
|
||||
)
|
||||
|
||||
submitTable(() => {
|
||||
cy.get('.modal-body').then((modalBody: any) => {
|
||||
if (
|
||||
@@ -245,10 +251,13 @@ context('editor tests: ', function () {
|
||||
.clear()
|
||||
.type('not a postcode{enter}')
|
||||
.then(() => {
|
||||
getCellByHeaderAndRow(1, 'REGEX_SOFT_COL').should(
|
||||
'have.class',
|
||||
'dc-warning-cell'
|
||||
)
|
||||
getCellByHeaderAndRow(1, 'REGEX_SOFT_COL')
|
||||
.should('have.class', 'dc-warning-cell')
|
||||
.and(
|
||||
'have.attr',
|
||||
'title',
|
||||
'REGEX: /[A-Z]{1,2}\\d{1,2}[A-Z]?\\s?\\d[A-Z]{2}/'
|
||||
)
|
||||
|
||||
submitTable(() => {
|
||||
// Validation passed despite the SOFTREGEX warning: the
|
||||
@@ -264,6 +273,126 @@ context('editor tests: ', function () {
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
it('9 | Info dropdown shows the applied HARDREGEX/SOFTREGEX pattern', () => {
|
||||
openTableFromTree(libraryToOpenIncludes, 'mpe_x_new')
|
||||
|
||||
clickOnEdit(() => {
|
||||
cy.get('.btn.btn-sm.btn-icon.btn-outline-danger', {
|
||||
timeout: longerCommandTimeout
|
||||
}).then(() => {
|
||||
scrollGridRight()
|
||||
|
||||
openColumnDropdown('REGEX_HARD_COL')
|
||||
cy.get('.htDropdownMenu').should(($menu) => {
|
||||
expect($menu.text()).to.include(
|
||||
'HARDREGEX: /[\\w.]+@[\\w]+\\.[a-z]{2,}/'
|
||||
)
|
||||
})
|
||||
cy.get('body').click(0, 0) // close menu
|
||||
|
||||
openColumnDropdown('REGEX_SOFT_COL')
|
||||
cy.get('.htDropdownMenu').should(($menu) => {
|
||||
expect($menu.text()).to.include(
|
||||
'SOFTREGEX: /[A-Z]{1,2}\\d{1,2}[A-Z]?\\s?\\d[A-Z]{2}/'
|
||||
)
|
||||
})
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
it('10 | Info dropdown shows only the applied HARDREGEX pattern when a column has both', () => {
|
||||
openTableFromTree(libraryToOpenIncludes, 'mpe_x_new')
|
||||
|
||||
clickOnEdit(() => {
|
||||
cy.get('.btn.btn-sm.btn-icon.btn-outline-danger', {
|
||||
timeout: longerCommandTimeout
|
||||
}).then(() => {
|
||||
scrollGridRight()
|
||||
|
||||
openColumnDropdown('REGEX_BOTH_COL')
|
||||
cy.get('.htDropdownMenu').should(($menu) => {
|
||||
const text = $menu.text()
|
||||
|
||||
expect(text).to.include('HARDREGEX: /^[A-Z0-9_-]+$/')
|
||||
expect(text).to.not.include('SOFTREGEX: /^.{5,10}$/')
|
||||
})
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
it('11 | REGEX_BOTH_COL: a HARDREGEX failure blocks submission and sets its own tooltip, not yellow', (done) => {
|
||||
openTableFromTree(libraryToOpenIncludes, 'mpe_x_new')
|
||||
|
||||
clickOnEdit(() => {
|
||||
cy.get('.btn.btn-sm.btn-icon.btn-outline-danger', {
|
||||
timeout: longerCommandTimeout
|
||||
}).then(() => {
|
||||
scrollGridRight()
|
||||
|
||||
// 'bad value' fails HARDREGEX (lowercase + space) but is 9 chars,
|
||||
// within SOFTREGEX's 5-10 range - isolates the hard-only failure.
|
||||
getCellByHeaderAndRow(1, 'REGEX_BOTH_COL')
|
||||
.dblclick({ force: true })
|
||||
.then(() => {
|
||||
cy.focused()
|
||||
.clear()
|
||||
.type('bad value{enter}')
|
||||
.then(() => {
|
||||
getCellByHeaderAndRow(1, 'REGEX_BOTH_COL')
|
||||
.should('not.have.class', 'dc-warning-cell')
|
||||
.and('have.attr', 'title', 'REGEX: /^[A-Z0-9_-]+$/')
|
||||
|
||||
submitTable(() => {
|
||||
cy.get('.modal-body').then((modalBody: any) => {
|
||||
if (
|
||||
modalBody[0].innerHTML
|
||||
.toLowerCase()
|
||||
.includes(`invalid values are present`)
|
||||
) {
|
||||
done()
|
||||
}
|
||||
})
|
||||
})
|
||||
})
|
||||
})
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
it('12 | REGEX_BOTH_COL: SOFTREGEX is ignored entirely when HARDREGEX is present', (done) => {
|
||||
openTableFromTree(libraryToOpenIncludes, 'mpe_x_new')
|
||||
|
||||
clickOnEdit(() => {
|
||||
cy.get('.btn.btn-sm.btn-icon.btn-outline-danger', {
|
||||
timeout: longerCommandTimeout
|
||||
}).then(() => {
|
||||
scrollGridRight()
|
||||
|
||||
// 'AB' passes HARDREGEX (uppercase only) but fails SOFTREGEX (too
|
||||
// short) - only one regex runs per column, so no warning is shown.
|
||||
getCellByHeaderAndRow(1, 'REGEX_BOTH_COL')
|
||||
.dblclick({ force: true })
|
||||
.then(() => {
|
||||
cy.focused()
|
||||
.clear()
|
||||
.type('AB{enter}')
|
||||
.then(() => {
|
||||
getCellByHeaderAndRow(1, 'REGEX_BOTH_COL')
|
||||
.should('not.have.class', 'dc-warning-cell')
|
||||
.and('not.have.attr', 'title')
|
||||
|
||||
submitTable(() => {
|
||||
cy.get('#submitBtn', { timeout: longerCommandTimeout })
|
||||
.should('exist')
|
||||
.should('not.be.disabled')
|
||||
.then(() => done())
|
||||
})
|
||||
})
|
||||
})
|
||||
})
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
// Handsontable virtualizes columns — with 17 columns on MPE_X_NEW, only
|
||||
@@ -280,6 +409,24 @@ const scrollGridRight = () => {
|
||||
.scrollTo('right')
|
||||
}
|
||||
|
||||
// Opens a column header's dropdown menu to reach its `info` item, which
|
||||
// has a custom renderer showing NAME/LABEL/TYPE/LENGTH/FORMAT and, when the
|
||||
// column has a HARDREGEX/SOFTREGEX rule, the applied pattern. Clicking the
|
||||
// header text first selects the column - the renderer reads
|
||||
// hot.getSelected() to decide which column to describe.
|
||||
const openColumnDropdown = (headerText: string) => {
|
||||
cy.get('#hotTable .ht_clone_top .htCore thead button.changeType', {
|
||||
timeout: longerCommandTimeout
|
||||
})
|
||||
.parents('th')
|
||||
.filter((_, th) => Cypress.$(th).text().includes(headerText))
|
||||
.last()
|
||||
.as('targetHeader')
|
||||
|
||||
cy.get('@targetHeader').click()
|
||||
cy.get('@targetHeader').find('button.changeType').click({ force: true })
|
||||
}
|
||||
|
||||
// Locates a body cell by its column's header text rather than a hardcoded
|
||||
// childNodes index. Handsontable's hiddenColumns plugin (e.g. HIDDEN_COL,
|
||||
// used by several demo columns ahead of REGEX_HARD_COL/REGEX_SOFT_COL in
|
||||
|
||||
@@ -3254,7 +3254,15 @@ export class EditorComponent implements OnInit, AfterViewInit, OnDestroy {
|
||||
colName = this.hotInstance?.colToProp(selectedCol) as string
|
||||
colInfo = this.$dataFormats?.vars[colName]
|
||||
|
||||
textInfo = buildColInfoHtml(colName, colInfo)
|
||||
const { hardRegexValue, softRegexValue } =
|
||||
this.dcValidator?.getRegexRuleValues(colName) || {}
|
||||
|
||||
textInfo = buildColInfoHtml(
|
||||
colName,
|
||||
colInfo,
|
||||
hardRegexValue,
|
||||
softRegexValue
|
||||
)
|
||||
}
|
||||
|
||||
elem.innerHTML = textInfo
|
||||
|
||||
@@ -19,13 +19,34 @@ describe('makeRegexWarningRenderer', () => {
|
||||
return { hot, container }
|
||||
}
|
||||
|
||||
it('adds dc-warning-cell when the value fails the pattern', () => {
|
||||
const buildHotNumeric = (data: any[]) => {
|
||||
const container = document.createElement('div')
|
||||
document.body.appendChild(container)
|
||||
|
||||
const hot = new Handsontable(container, {
|
||||
data,
|
||||
columns: [
|
||||
{
|
||||
data: 'val',
|
||||
renderer: makeRegexWarningRenderer('^[A-Z]{3}$', undefined, true)
|
||||
},
|
||||
{ data: '_____DELETE__THIS__RECORD_____' }
|
||||
],
|
||||
licenseKey: 'non-commercial-and-evaluation'
|
||||
})
|
||||
hot.render()
|
||||
|
||||
return { hot, container }
|
||||
}
|
||||
|
||||
it('adds dc-warning-cell and a REGEX: title when the value fails the pattern', () => {
|
||||
const { hot, container } = buildHot([
|
||||
{ val: 'abc', _____DELETE__THIS__RECORD_____: 'No' }
|
||||
])
|
||||
|
||||
const td = hot.getCell(0, 0)
|
||||
expect(td?.classList.contains('dc-warning-cell')).toBeTrue()
|
||||
expect(td?.title).toEqual('REGEX: ^[A-Z]{3}$')
|
||||
|
||||
hot.destroy()
|
||||
container.remove()
|
||||
@@ -55,9 +76,9 @@ describe('makeRegexWarningRenderer', () => {
|
||||
container.remove()
|
||||
})
|
||||
|
||||
it('does not add dc-warning-cell for a SAS special missing value', () => {
|
||||
const { hot, container } = buildHot([
|
||||
{ val: '.a', _____DELETE__THIS__RECORD_____: 'No' }
|
||||
it('does not add dc-warning-cell for the plain SAS missing (".") on a numeric column', () => {
|
||||
const { hot, container } = buildHotNumeric([
|
||||
{ val: '.', _____DELETE__THIS__RECORD_____: 'No' }
|
||||
])
|
||||
|
||||
const td = hot.getCell(0, 0)
|
||||
@@ -67,6 +88,18 @@ describe('makeRegexWarningRenderer', () => {
|
||||
container.remove()
|
||||
})
|
||||
|
||||
it('DOES add dc-warning-cell for a special missing on a numeric column (a deliberately-set value)', () => {
|
||||
const { hot, container } = buildHotNumeric([
|
||||
{ val: '.a', _____DELETE__THIS__RECORD_____: 'No' }
|
||||
])
|
||||
|
||||
const td = hot.getCell(0, 0)
|
||||
expect(td?.classList.contains('dc-warning-cell')).toBeTrue()
|
||||
|
||||
hot.destroy()
|
||||
container.remove()
|
||||
})
|
||||
|
||||
it('does not throw and never warns on a malformed pattern', () => {
|
||||
const container = document.createElement('div')
|
||||
document.body.appendChild(container)
|
||||
@@ -128,4 +161,154 @@ describe('makeRegexWarningRenderer', () => {
|
||||
hot.destroy()
|
||||
container.remove()
|
||||
})
|
||||
|
||||
describe('HARDREGEX (second, optional pattern)', () => {
|
||||
// HARDREGEX already blocks submission via dqValidate/HOT's own
|
||||
// htInvalid (unchanged, untouched here) - this renderer only adds the
|
||||
// matching 'REGEX: <pattern>' title on top, so a column with only a
|
||||
// HARDREGEX rule (no SOFTREGEX at all) still tells the user why a cell
|
||||
// is red, not just that it is.
|
||||
const buildHardOnlyHot = (value: string) => {
|
||||
const container = document.createElement('div')
|
||||
document.body.appendChild(container)
|
||||
|
||||
const hot = new Handsontable(container, {
|
||||
data: [{ val: value, _____DELETE__THIS__RECORD_____: 'No' }],
|
||||
columns: [
|
||||
{
|
||||
data: 'val',
|
||||
renderer: makeRegexWarningRenderer(undefined, '^[A-Z]{3}$')
|
||||
},
|
||||
{ data: '_____DELETE__THIS__RECORD_____' }
|
||||
],
|
||||
licenseKey: 'non-commercial-and-evaluation'
|
||||
})
|
||||
hot.render()
|
||||
|
||||
return { hot, container }
|
||||
}
|
||||
|
||||
it('sets a REGEX: title (no dc-warning-cell) when only HARDREGEX fails', () => {
|
||||
const { hot, container } = buildHardOnlyHot('abc')
|
||||
|
||||
const td = hot.getCell(0, 0)
|
||||
expect(td?.classList.contains('dc-warning-cell')).toBeFalse()
|
||||
expect(td?.title).toEqual('REGEX: ^[A-Z]{3}$')
|
||||
|
||||
hot.destroy()
|
||||
container.remove()
|
||||
})
|
||||
|
||||
it('sets no title when the value passes HARDREGEX', () => {
|
||||
const { hot, container } = buildHardOnlyHot('ABC')
|
||||
|
||||
const td = hot.getCell(0, 0)
|
||||
expect(td?.title).toEqual('')
|
||||
|
||||
hot.destroy()
|
||||
container.remove()
|
||||
})
|
||||
|
||||
it('suppresses the HARDREGEX title on a row marked for delete', () => {
|
||||
const container = document.createElement('div')
|
||||
document.body.appendChild(container)
|
||||
|
||||
const hot = new Handsontable(container, {
|
||||
data: [{ val: 'abc', _____DELETE__THIS__RECORD_____: 'Yes' }],
|
||||
columns: [
|
||||
{
|
||||
data: 'val',
|
||||
renderer: makeRegexWarningRenderer(undefined, '^[A-Z]{3}$')
|
||||
},
|
||||
{ data: '_____DELETE__THIS__RECORD_____' }
|
||||
],
|
||||
licenseKey: 'non-commercial-and-evaluation'
|
||||
})
|
||||
hot.render()
|
||||
|
||||
expect(hot.getCell(0, 0)?.title).toEqual('')
|
||||
|
||||
hot.destroy()
|
||||
container.remove()
|
||||
})
|
||||
|
||||
it('prefers the HARDREGEX title over SOFTREGEX when a value fails both', () => {
|
||||
const container = document.createElement('div')
|
||||
document.body.appendChild(container)
|
||||
|
||||
const hot = new Handsontable(container, {
|
||||
data: [{ val: 'ab', _____DELETE__THIS__RECORD_____: 'No' }],
|
||||
columns: [
|
||||
{
|
||||
data: 'val',
|
||||
renderer: makeRegexWarningRenderer('^.{5,10}$', '^[A-Z]{3}$')
|
||||
},
|
||||
{ data: '_____DELETE__THIS__RECORD_____' }
|
||||
],
|
||||
licenseKey: 'non-commercial-and-evaluation'
|
||||
})
|
||||
hot.render()
|
||||
|
||||
const td = hot.getCell(0, 0)
|
||||
// 'ab' fails HARDREGEX (not 3 uppercase letters) AND SOFTREGEX (too
|
||||
// short) - HARDREGEX wins: its title shows, and no yellow class is
|
||||
// added (HOT's own red htInvalid governs this cell instead).
|
||||
expect(td?.classList.contains('dc-warning-cell')).toBeFalse()
|
||||
expect(td?.title).toEqual('REGEX: ^[A-Z]{3}$')
|
||||
|
||||
hot.destroy()
|
||||
container.remove()
|
||||
})
|
||||
|
||||
it('never warns for SOFTREGEX when the column also has HARDREGEX, even for a value that passes HARDREGEX', () => {
|
||||
const container = document.createElement('div')
|
||||
document.body.appendChild(container)
|
||||
|
||||
const hot = new Handsontable(container, {
|
||||
data: [{ val: 'AB', _____DELETE__THIS__RECORD_____: 'No' }],
|
||||
columns: [
|
||||
{
|
||||
data: 'val',
|
||||
renderer: makeRegexWarningRenderer('^.{5,10}$', '^[A-Z0-9]+$')
|
||||
},
|
||||
{ data: '_____DELETE__THIS__RECORD_____' }
|
||||
],
|
||||
licenseKey: 'non-commercial-and-evaluation'
|
||||
})
|
||||
hot.render()
|
||||
|
||||
const td = hot.getCell(0, 0)
|
||||
// 'AB' passes HARDREGEX (uppercase/digits) but fails SOFTREGEX (too
|
||||
// short) - only one regex runs per column, so the soft rule is
|
||||
// ignored entirely: no warning, no title.
|
||||
expect(td?.classList.contains('dc-warning-cell')).toBeFalse()
|
||||
expect(td?.title).toEqual('')
|
||||
|
||||
hot.destroy()
|
||||
container.remove()
|
||||
})
|
||||
|
||||
it('does not throw and never warns on a malformed HARDREGEX pattern', () => {
|
||||
const container = document.createElement('div')
|
||||
document.body.appendChild(container)
|
||||
|
||||
const hot = new Handsontable(container, {
|
||||
data: [{ val: 'abc', _____DELETE__THIS__RECORD_____: 'No' }],
|
||||
columns: [
|
||||
{
|
||||
data: 'val',
|
||||
renderer: makeRegexWarningRenderer(undefined, '[unterminated')
|
||||
},
|
||||
{ data: '_____DELETE__THIS__RECORD_____' }
|
||||
],
|
||||
licenseKey: 'non-commercial-and-evaluation'
|
||||
})
|
||||
|
||||
expect(() => hot.render()).not.toThrow()
|
||||
expect(hot.getCell(0, 0)?.title).toEqual('')
|
||||
|
||||
hot.destroy()
|
||||
container.remove()
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
@@ -2,28 +2,46 @@ import Handsontable from 'handsontable'
|
||||
import { isRegexRuleExempt } from '../../shared/dc-validator/utils/isRegexRuleExempt'
|
||||
import { parseRegexRule } from '../../shared/dc-validator/utils/parseRegexRule'
|
||||
|
||||
const compileRegex = (
|
||||
pattern: string | undefined,
|
||||
ruleType: 'SOFTREGEX' | 'HARDREGEX'
|
||||
): RegExp | null => {
|
||||
try {
|
||||
return pattern ? parseRegexRule(pattern) : null
|
||||
} catch (e) {
|
||||
console.warn(`${ruleType} - invalid pattern, warning disabled: ${pattern}`)
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Builds a display-only HOT renderer for SOFTREGEX: a value that fails the
|
||||
* pattern gets a yellow `dc-warning-cell` class (styles.scss) instead of
|
||||
* blocking submission — SOFTREGEX must never return false from the cell
|
||||
* validator (that would paint htInvalid and block submit), so the warning
|
||||
* lives purely at the render layer, same split as makeNumberFormatRenderer.
|
||||
* Builds a display-only HOT renderer for HARDREGEX/SOFTREGEX: neither ever
|
||||
* returns false from the cell validator here (that's dqValidate's job for
|
||||
* HARDREGEX, which blocks submission and paints HOT's own red htInvalid
|
||||
* independently of this renderer) - this renderer only adds the matching
|
||||
* `REGEX: <pattern>` title, plus a yellow `dc-warning-cell` class when only
|
||||
* SOFTREGEX fails, same split as makeNumberFormatRenderer.
|
||||
*
|
||||
* Only one regex ever runs per column: when a HARDREGEX rule exists,
|
||||
* SOFTREGEX is ignored entirely (never compiled, never evaluated) -
|
||||
* regardless of whether individual cell values pass or fail the hard
|
||||
* rule. A value failing HARDREGEX gets its title (no yellow class - red
|
||||
* htInvalid already covers the color). This mirrors
|
||||
* DcValidator.failsSoftRegex's own precedence.
|
||||
*
|
||||
* Suppressed on rows marked for delete (_____DELETE__THIS__RECORD_____ =
|
||||
* 'Yes') — a warning about data about to be removed is just noise.
|
||||
*
|
||||
* Falls back to no warning ever showing when the pattern itself is
|
||||
* malformed, rather than breaking the cell.
|
||||
* Falls back to no warning ever showing when a pattern itself is malformed,
|
||||
* rather than breaking the cell.
|
||||
*/
|
||||
export const makeRegexWarningRenderer = (pattern?: string) => {
|
||||
let regex: RegExp | null = null
|
||||
|
||||
try {
|
||||
if (pattern) regex = parseRegexRule(pattern)
|
||||
} catch (e) {
|
||||
console.warn(`SOFTREGEX - invalid pattern, warning disabled: ${pattern}`)
|
||||
regex = null
|
||||
}
|
||||
export const makeRegexWarningRenderer = (
|
||||
softPattern?: string,
|
||||
hardPattern?: string,
|
||||
isNumeric: boolean = false
|
||||
) => {
|
||||
const hardRegex = compileRegex(hardPattern, 'HARDREGEX')
|
||||
const softRegex = hardRegex ? null : compileRegex(softPattern, 'SOFTREGEX')
|
||||
|
||||
const baseRenderer = Handsontable.renderers.getRenderer('text')
|
||||
|
||||
@@ -40,13 +58,20 @@ export const makeRegexWarningRenderer = (pattern?: string) => {
|
||||
|
||||
const markedForDelete =
|
||||
instance.getDataAtRowProp(row, '_____DELETE__THIS__RECORD_____') === 'Yes'
|
||||
const exempt = isRegexRuleExempt(value, isNumeric)
|
||||
|
||||
const failsPattern =
|
||||
!!regex && !isRegexRuleExempt(value) && !regex.test(value.toString())
|
||||
const failsHard =
|
||||
!!hardRegex && !exempt && !hardRegex.test(value.toString())
|
||||
const failsSoft =
|
||||
!!softRegex && !exempt && !softRegex.test(value.toString())
|
||||
|
||||
if (failsPattern && !markedForDelete) {
|
||||
if (markedForDelete) return td
|
||||
|
||||
if (failsHard) {
|
||||
td.title = `REGEX: ${hardPattern}`
|
||||
} else if (failsSoft) {
|
||||
td.classList.add('dc-warning-cell')
|
||||
td.title = 'Value does not match the expected pattern'
|
||||
td.title = `REGEX: ${softPattern}`
|
||||
}
|
||||
|
||||
return td
|
||||
|
||||
@@ -204,6 +204,31 @@ export class DcValidator {
|
||||
return isNaN(digits) ? undefined : digits
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns the RULE_VALUEs of a HARDREGEX/SOFTREGEX rule on the given
|
||||
* column, for display in the column-header info dropdown. Both are
|
||||
* returned raw - the caller (buildColInfoHtml) decides which one is
|
||||
* actually applied (HARDREGEX wins when both exist).
|
||||
*
|
||||
* @param col column name
|
||||
*/
|
||||
getRegexRuleValues(col: string): {
|
||||
hardRegexValue: string | undefined
|
||||
softRegexValue: string | undefined
|
||||
} {
|
||||
const hardRegexRule = this.dqrules.find(
|
||||
(rule: DQRule) => rule.BASE_COL === col && rule.RULE_TYPE === 'HARDREGEX'
|
||||
)
|
||||
const softRegexRule = this.dqrules.find(
|
||||
(rule: DQRule) => rule.BASE_COL === col && rule.RULE_TYPE === 'SOFTREGEX'
|
||||
)
|
||||
|
||||
return {
|
||||
hardRegexValue: hardRegexRule?.RULE_VALUE,
|
||||
softRegexValue: softRegexRule?.RULE_VALUE
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Retrieves dropdown source for given dc validation rule
|
||||
* The values comes from MPE_SELECTBOX table
|
||||
@@ -247,14 +272,23 @@ export class DcValidator {
|
||||
* exists. SOFTREGEX never goes through dqValidate/the cell validator (see
|
||||
* setupValidations — it's a display-only grid renderer instead), so the
|
||||
* edit-record modal, which has no grid renderer to hook into, uses this
|
||||
* directly to show the same warning outside the grid. HARDREGEX takes
|
||||
* precedence when both rules apply to the column, matching the grid's own
|
||||
* behaviour (setupValidations skips wiring the SOFTREGEX renderer there
|
||||
* too) — a dual-rule column should render red/blocked, never yellow.
|
||||
* directly to show the same warning outside the grid.
|
||||
*
|
||||
* A column can carry both HARDREGEX and SOFTREGEX at once, but only one
|
||||
* regex ever runs per column: if a HARDREGEX rule exists, SOFTREGEX is
|
||||
* ignored entirely - the cell is already governed by the blocking rule,
|
||||
* so a yellow warning on top would be redundant, even for values that
|
||||
* pass the hard rule. Same precedence as makeRegexWarningRenderer.
|
||||
*/
|
||||
failsSoftRegex(col: string, value: any): boolean {
|
||||
if (this.hasDqRules(col, ['HARDREGEX'])) return false
|
||||
if (isRegexRuleExempt(value)) return false
|
||||
const isNumeric =
|
||||
this.rules.find((rule) => rule.data === col)?.type === 'numeric'
|
||||
if (isRegexRuleExempt(value, isNumeric)) return false
|
||||
|
||||
const hardRegexRule = this.dqrules.find(
|
||||
(rule: DQRule) => rule.BASE_COL === col && rule.RULE_TYPE === 'HARDREGEX'
|
||||
)
|
||||
if (hardRegexRule) return false
|
||||
|
||||
const softRegexRule = this.dqrules.find(
|
||||
(rule: DQRule) => rule.BASE_COL === col && rule.RULE_TYPE === 'SOFTREGEX'
|
||||
@@ -425,25 +459,21 @@ export class DcValidator {
|
||||
this.rules[i].numericFormat = undefined
|
||||
}
|
||||
|
||||
// SOFTREGEX: display-only warning (yellow), never blocks submission —
|
||||
// unlike HARDREGEX (see dq-validation.ts), which goes through the
|
||||
// normal validator/dqValidate path instead. Last-wins against
|
||||
// NUMBER_FORMAT if a column somehow carried both (not expected in
|
||||
// practice — one formats numbers, the other pattern-matches text).
|
||||
//
|
||||
// HARDREGEX takes precedence when both apply to the same column: skip
|
||||
// wiring this renderer entirely, so a failing value stays HOT's own
|
||||
// red htInvalid (from HARDREGEX/dqValidate) rather than being
|
||||
// visually overridden by this renderer's yellow dc-warning-cell.
|
||||
if (
|
||||
this.hasDqRules(ruleColName, ['SOFTREGEX']) &&
|
||||
!this.hasDqRules(ruleColName, ['HARDREGEX'])
|
||||
) {
|
||||
const softRegexRule = this.getDqDetails(ruleColName).find(
|
||||
(rule: DQRule) => rule.RULE_TYPE === 'SOFTREGEX'
|
||||
)
|
||||
// HARDREGEX/SOFTREGEX: submission-blocking for HARDREGEX still goes
|
||||
// through the normal validator/dqValidate path (see dq-validation.ts)
|
||||
// and is unaffected by this renderer. This only wires the display
|
||||
// layer - a 'REGEX: <pattern>' title, plus a yellow dc-warning-cell
|
||||
// when only SOFTREGEX fails (never for HARDREGEX, which relies on
|
||||
// HOT's own red htInvalid instead). Last-wins against NUMBER_FORMAT
|
||||
// if a column somehow carried both (not expected in practice — one
|
||||
// formats numbers, the other pattern-matches text).
|
||||
if (this.hasDqRules(ruleColName, ['HARDREGEX', 'SOFTREGEX'])) {
|
||||
const { hardRegexValue, softRegexValue } =
|
||||
this.getRegexRuleValues(ruleColName)
|
||||
this.rules[i].renderer = makeRegexWarningRenderer(
|
||||
softRegexRule?.RULE_VALUE
|
||||
softRegexValue,
|
||||
hardRegexValue,
|
||||
this.rules[i].type === 'numeric'
|
||||
)
|
||||
}
|
||||
}
|
||||
@@ -532,7 +562,11 @@ export class DcValidator {
|
||||
}
|
||||
|
||||
if (self.isDqCol(col || '')) {
|
||||
const dqValid = dqValidate(self.getDqDetails(col || ''), value)
|
||||
const dqValid = dqValidate(
|
||||
self.getDqDetails(col || ''),
|
||||
value,
|
||||
colType === 'numeric'
|
||||
)
|
||||
|
||||
if (!dqValid) {
|
||||
console.warn(`DQ Validation - invalid (Value: ${value})`)
|
||||
|
||||
@@ -555,7 +555,32 @@ describe('DC Validator', () => {
|
||||
})
|
||||
})
|
||||
|
||||
it('11 | wires a function renderer for a SOFTREGEX rule, without blocking submission', () => {
|
||||
it('11 | wires a function renderer for a HARDREGEX-only rule too (for the REGEX: tooltip)', () => {
|
||||
// HARDREGEX blocking itself is covered by test 10 above - this isolates
|
||||
// the newer addition: even with no SOFTREGEX at all, a renderer must
|
||||
// still be wired so a failing cell gets a 'REGEX: <pattern>' title on
|
||||
// top of HOT's own red htInvalid, not just silence.
|
||||
const dcValidator: DcValidator = new DcValidator(
|
||||
example_sasparams,
|
||||
example_dataformats,
|
||||
example_cols,
|
||||
[
|
||||
...example_dqRules,
|
||||
{
|
||||
BASE_COL: 'SOME_CHAR_ANY',
|
||||
RULE_TYPE: 'HARDREGEX',
|
||||
RULE_VALUE: '^[A-Z]+$',
|
||||
X: 0
|
||||
}
|
||||
],
|
||||
example_dqData
|
||||
)
|
||||
const someCharAnyRule = dcValidator.getRule('SOME_CHAR_ANY')
|
||||
|
||||
expect(typeof someCharAnyRule?.renderer).toEqual('function')
|
||||
})
|
||||
|
||||
it('12 | wires a function renderer for a SOFTREGEX rule, without blocking submission', () => {
|
||||
// SOME_CHAR_ANY carries no other DQ rules in the shared fixture, so this
|
||||
// isolates SOFTREGEX's own wiring. The renderer's own pass/fail/delete-
|
||||
// suppression behaviour is covered by regex-warning-renderer.spec.ts —
|
||||
@@ -589,11 +614,11 @@ describe('DC Validator', () => {
|
||||
)
|
||||
})
|
||||
|
||||
it('12 | HARDREGEX takes precedence over SOFTREGEX on a dual-rule column', () => {
|
||||
// Both rules on the same column: HARDREGEX must win, so the cell
|
||||
// renders red (blocked) via HOT's own htInvalid, not yellow. No
|
||||
// SOFTREGEX renderer is wired at all here, so there's nothing that could
|
||||
// visually compete with htInvalid for that column.
|
||||
it('13 | wires a renderer for a dual-rule column, and HARDREGEX still blocks submission', () => {
|
||||
// Both rules on the same column: submission blocking is still governed
|
||||
// entirely by HARDREGEX/dqValidate (unchanged). A renderer is wired so
|
||||
// the cell gets a 'REGEX: <pattern>' title - its own hard-vs-soft
|
||||
// precedence and coloring are covered by regex-warning-renderer.spec.ts.
|
||||
const dcValidator: DcValidator = new DcValidator(
|
||||
example_sasparams,
|
||||
example_dataformats,
|
||||
@@ -624,10 +649,10 @@ describe('DC Validator', () => {
|
||||
expect(valid).toBeFalse()
|
||||
}
|
||||
)
|
||||
expect(rule?.renderer).toBeUndefined()
|
||||
expect(typeof rule?.renderer).toEqual('function')
|
||||
})
|
||||
|
||||
describe('13 | failsSoftRegex (edit-record modal support for SOFTREGEX)', () => {
|
||||
describe('14 | failsSoftRegex (edit-record modal support for SOFTREGEX)', () => {
|
||||
// SOFTREGEX never goes through dqValidate (see the wiring in
|
||||
// setupValidations), so the edit-record modal — which has no grid
|
||||
// renderer to hook into — calls this directly to show the same warning.
|
||||
@@ -699,7 +724,7 @@ describe('DC Validator', () => {
|
||||
).toBeFalse()
|
||||
})
|
||||
|
||||
it('is false for blank/special-missing values, same exemption as HARDREGEX', () => {
|
||||
it('is false for blank values; special-missing-looking values are real text on a character column', () => {
|
||||
const dcValidator = buildValidator([
|
||||
{
|
||||
BASE_COL: 'SOME_CHAR_ANY',
|
||||
@@ -710,28 +735,141 @@ describe('DC Validator', () => {
|
||||
])
|
||||
|
||||
expect(dcValidator.failsSoftRegex('SOME_CHAR_ANY', '')).toBeFalse()
|
||||
expect(dcValidator.failsSoftRegex('SOME_CHAR_ANY', '.a')).toBeFalse()
|
||||
// ".a" is real text here and fails ^[A-Z]+$
|
||||
expect(dcValidator.failsSoftRegex('SOME_CHAR_ANY', '.a')).toBeTrue()
|
||||
})
|
||||
|
||||
it('is false when the column also has HARDREGEX (precedence — no yellow on a red cell)', () => {
|
||||
it('exempts the plain missing (".") but not special missings on a numeric column', () => {
|
||||
const dcValidator = buildValidator([
|
||||
{
|
||||
BASE_COL: 'SOME_NUM',
|
||||
RULE_TYPE: 'SOFTREGEX',
|
||||
RULE_VALUE: '^[0-9]+$',
|
||||
X: 0
|
||||
}
|
||||
])
|
||||
|
||||
expect(dcValidator.failsSoftRegex('SOME_NUM', '.')).toBeFalse()
|
||||
// Special missings are deliberately-set values and face the pattern.
|
||||
expect(dcValidator.failsSoftRegex('SOME_NUM', '.a')).toBeTrue()
|
||||
})
|
||||
|
||||
it('is false when a value fails both HARDREGEX and SOFTREGEX (precedence — no yellow on a red cell)', () => {
|
||||
const dcValidator = buildValidator([
|
||||
{
|
||||
BASE_COL: 'SOME_CHAR_ANY',
|
||||
RULE_TYPE: 'HARDREGEX',
|
||||
RULE_VALUE: '^[A-Z]{3}$',
|
||||
X: 0
|
||||
},
|
||||
{
|
||||
BASE_COL: 'SOME_CHAR_ANY',
|
||||
RULE_TYPE: 'SOFTREGEX',
|
||||
RULE_VALUE: '^.{5,10}$',
|
||||
X: 0
|
||||
}
|
||||
])
|
||||
|
||||
// 'ab' fails both HARDREGEX (not 3 uppercase letters) and SOFTREGEX
|
||||
// (too short) - HARDREGEX wins, so this must stay false rather than
|
||||
// report a SOFTREGEX warning on a value that's already blocked.
|
||||
expect(dcValidator.failsSoftRegex('SOME_CHAR_ANY', 'ab')).toBeFalse()
|
||||
})
|
||||
|
||||
it('is false for every value when the column has HARDREGEX - SOFTREGEX never runs, even if HARDREGEX passes', () => {
|
||||
const dcValidator = buildValidator([
|
||||
{
|
||||
BASE_COL: 'SOME_CHAR_ANY',
|
||||
RULE_TYPE: 'HARDREGEX',
|
||||
RULE_VALUE: '^[A-Z0-9]+$',
|
||||
X: 0
|
||||
},
|
||||
{
|
||||
BASE_COL: 'SOME_CHAR_ANY',
|
||||
RULE_TYPE: 'SOFTREGEX',
|
||||
RULE_VALUE: '^.{5,10}$',
|
||||
X: 0
|
||||
}
|
||||
])
|
||||
|
||||
// 'AB' passes HARDREGEX (uppercase/digits) but fails SOFTREGEX (too
|
||||
// short) - only one regex runs per column, so the soft rule is
|
||||
// ignored entirely.
|
||||
expect(dcValidator.failsSoftRegex('SOME_CHAR_ANY', 'AB')).toBeFalse()
|
||||
})
|
||||
})
|
||||
|
||||
describe('15 | getRegexRuleValues (column-header info display)', () => {
|
||||
const buildValidator = (dqRules: DQRule[]) =>
|
||||
new DcValidator(
|
||||
example_sasparams,
|
||||
example_dataformats,
|
||||
example_cols,
|
||||
dqRules,
|
||||
example_dqData
|
||||
)
|
||||
|
||||
it('returns only hardRegexValue for a column with just HARDREGEX', () => {
|
||||
const dcValidator = buildValidator([
|
||||
{
|
||||
BASE_COL: 'SOME_CHAR_ANY',
|
||||
RULE_TYPE: 'HARDREGEX',
|
||||
RULE_VALUE: '^[A-Z]+$',
|
||||
X: 0
|
||||
},
|
||||
}
|
||||
])
|
||||
|
||||
expect(dcValidator.getRegexRuleValues('SOME_CHAR_ANY')).toEqual({
|
||||
hardRegexValue: '^[A-Z]+$',
|
||||
softRegexValue: undefined
|
||||
})
|
||||
})
|
||||
|
||||
it('returns only softRegexValue for a column with just SOFTREGEX', () => {
|
||||
const dcValidator = buildValidator([
|
||||
{
|
||||
BASE_COL: 'SOME_CHAR_ANY',
|
||||
RULE_TYPE: 'SOFTREGEX',
|
||||
RULE_VALUE: '^[A-Z]+$',
|
||||
RULE_VALUE: '/\\b(the|data)\\b/i',
|
||||
X: 0
|
||||
}
|
||||
])
|
||||
|
||||
expect(
|
||||
dcValidator.failsSoftRegex('SOME_CHAR_ANY', 'lowercase')
|
||||
).toBeFalse()
|
||||
expect(dcValidator.getRegexRuleValues('SOME_CHAR_ANY')).toEqual({
|
||||
hardRegexValue: undefined,
|
||||
softRegexValue: '/\\b(the|data)\\b/i'
|
||||
})
|
||||
})
|
||||
|
||||
it('returns both values for a column with both HARDREGEX and SOFTREGEX', () => {
|
||||
const dcValidator = buildValidator([
|
||||
{
|
||||
BASE_COL: 'SOME_CHAR_ANY',
|
||||
RULE_TYPE: 'HARDREGEX',
|
||||
RULE_VALUE: '^HARD$',
|
||||
X: 0
|
||||
},
|
||||
{
|
||||
BASE_COL: 'SOME_CHAR_ANY',
|
||||
RULE_TYPE: 'SOFTREGEX',
|
||||
RULE_VALUE: '^SOFT$',
|
||||
X: 0
|
||||
}
|
||||
])
|
||||
|
||||
expect(dcValidator.getRegexRuleValues('SOME_CHAR_ANY')).toEqual({
|
||||
hardRegexValue: '^HARD$',
|
||||
softRegexValue: '^SOFT$'
|
||||
})
|
||||
})
|
||||
|
||||
it('returns both undefined for a column with no regex rule', () => {
|
||||
const dcValidator = buildValidator([])
|
||||
|
||||
expect(dcValidator.getRegexRuleValues('SOME_CHAR_ANY')).toEqual({
|
||||
hardRegexValue: undefined,
|
||||
softRegexValue: undefined
|
||||
})
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
@@ -7,6 +7,10 @@ import { DcValidation } from '../models/dc-validation.model'
|
||||
* Uses Intl.NumberFormat options (HOT 17+) instead of the deprecated numbro
|
||||
* `pattern`/`culture`. `maximumFractionDigits: 20` preserves all natural
|
||||
* decimals (Intl's default of 3 would round); `locale` replaces `culture`.
|
||||
* `useGrouping: false` keeps raw digits (no thousands separator) - a
|
||||
* separator would leak into anything that pattern-matches the displayed
|
||||
* value (eg HARDREGEX/SOFTREGEX), and grouping can be opted into per-column
|
||||
* with the NUMBER_FORMAT rule.
|
||||
*
|
||||
* @param rules Cell Validation rules to be updated
|
||||
* Those rules are passed in the `columns` property Of handsontable settings.
|
||||
@@ -14,7 +18,7 @@ import { DcValidation } from '../models/dc-validation.model'
|
||||
export const applyNumericFormats = (rules: DcValidation[]): DcValidation[] => {
|
||||
for (let rule of rules) {
|
||||
if (rule.type === 'numeric') {
|
||||
rule.numericFormat = { useGrouping: true, maximumFractionDigits: 20 }
|
||||
rule.numericFormat = { useGrouping: false, maximumFractionDigits: 20 }
|
||||
rule.locale = window.navigator.language
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,19 +1,40 @@
|
||||
import { isRegexRuleExempt } from './isRegexRuleExempt'
|
||||
|
||||
describe('isRegexRuleExempt', () => {
|
||||
it('exempts blank/undefined/null', () => {
|
||||
it('exempts blank/undefined/null on any column type', () => {
|
||||
expect(isRegexRuleExempt('')).toBeTrue()
|
||||
expect(isRegexRuleExempt(undefined)).toBeTrue()
|
||||
expect(isRegexRuleExempt(null)).toBeTrue()
|
||||
expect(isRegexRuleExempt('', true)).toBeTrue()
|
||||
})
|
||||
|
||||
it('exempts SAS special missing values', () => {
|
||||
expect(isRegexRuleExempt('.')).toBeTrue()
|
||||
expect(isRegexRuleExempt('.a')).toBeTrue()
|
||||
expect(isRegexRuleExempt('_')).toBeTrue()
|
||||
it('exempts the plain SAS missing (".") on numeric columns', () => {
|
||||
expect(isRegexRuleExempt('.', true)).toBeTrue()
|
||||
})
|
||||
|
||||
it('does not exempt special missings on numeric columns (they are deliberately-set values)', () => {
|
||||
expect(isRegexRuleExempt('.a', true)).toBeFalse()
|
||||
expect(isRegexRuleExempt('._', true)).toBeFalse()
|
||||
expect(isRegexRuleExempt('_', true)).toBeFalse()
|
||||
expect(isRegexRuleExempt('d', true)).toBeFalse()
|
||||
})
|
||||
|
||||
it('exempts nothing but blank on character columns', () => {
|
||||
expect(isRegexRuleExempt('.')).toBeFalse()
|
||||
expect(isRegexRuleExempt('.a')).toBeFalse()
|
||||
expect(isRegexRuleExempt('_')).toBeFalse()
|
||||
})
|
||||
|
||||
it('does not exempt a bare single letter on character columns (real value, not a missing)', () => {
|
||||
// A bare single letter is real character data, not a SAS special
|
||||
// missing - it must reach the pattern, not be exempted.
|
||||
expect(isRegexRuleExempt('d')).toBeFalse()
|
||||
expect(isRegexRuleExempt('z')).toBeFalse()
|
||||
expect(isRegexRuleExempt('A')).toBeFalse()
|
||||
})
|
||||
|
||||
it('does not exempt an ordinary value', () => {
|
||||
expect(isRegexRuleExempt('ABC123')).toBeFalse()
|
||||
expect(isRegexRuleExempt('ABC123', true)).toBeFalse()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -1,13 +1,22 @@
|
||||
import { isSpecialMissing } from '@sasjs/utils/input/validators'
|
||||
|
||||
/**
|
||||
* HARDREGEX/SOFTREGEX both skip pattern-matching for blank and SAS special
|
||||
* missing values (., .a-.z, _) — the same exemption NOTNULL/MINVAL/MAXVAL
|
||||
* already apply elsewhere, since neither convention represents a real
|
||||
* formatted value the pattern is meant to check.
|
||||
* HARDREGEX/SOFTREGEX both skip pattern-matching for:
|
||||
*
|
||||
* - blank values (undefined, null, '') on any column type - enforcing
|
||||
* populated values is NOTNULL's job, not the pattern's; and
|
||||
* - the plain SAS numeric missing (".") on numeric columns - it
|
||||
* represents the absence of a value, same as blank.
|
||||
*
|
||||
* SPECIAL missings (.a-.z, ._, bare letters) are NOT exempt, even on
|
||||
* numeric columns: being deliberately set, they are real values the
|
||||
* pattern is meant to check. This also means isSpecialMissing from
|
||||
* @sasjs/utils (which would match them, with an optional dot) is
|
||||
* deliberately not used here.
|
||||
*/
|
||||
export const isRegexRuleExempt = (value: any): boolean => {
|
||||
export const isRegexRuleExempt = (
|
||||
value: any,
|
||||
isNumeric: boolean = false
|
||||
): boolean => {
|
||||
if (value === undefined || value === null || value === '') return true
|
||||
|
||||
return isSpecialMissing(value)
|
||||
return isNumeric && value === '.'
|
||||
}
|
||||
|
||||
@@ -30,12 +30,28 @@ describe('dqValidate - HARDREGEX', () => {
|
||||
expect(dqValidate(rules, null)).toBeTrue()
|
||||
})
|
||||
|
||||
it('treats SAS special missing values as valid regardless of the pattern', () => {
|
||||
it('treats the plain SAS missing (".") as valid on numeric columns, regardless of the pattern', () => {
|
||||
const rules = [rule({ RULE_VALUE: '^[A-Z]{3}\\d{4}$' })]
|
||||
|
||||
expect(dqValidate(rules, '.')).toBeTrue()
|
||||
expect(dqValidate(rules, '.a')).toBeTrue()
|
||||
expect(dqValidate(rules, '_')).toBeTrue()
|
||||
expect(dqValidate(rules, '.', true)).toBeTrue()
|
||||
})
|
||||
|
||||
it('applies the pattern to special missings on numeric columns (deliberately-set values)', () => {
|
||||
const rules = [rule({ RULE_VALUE: '^[A-Z]{3}\\d{4}$' })]
|
||||
|
||||
expect(dqValidate(rules, '.a', true)).toBeFalse()
|
||||
expect(dqValidate(rules, '_', true)).toBeFalse()
|
||||
expect(dqValidate(rules, 'd', true)).toBeFalse()
|
||||
})
|
||||
|
||||
it('applies the pattern to special-missing-looking values on character columns', () => {
|
||||
const rules = [rule({ RULE_VALUE: '^[A-Z]{3}\\d{4}$' })]
|
||||
|
||||
expect(dqValidate(rules, '.')).toBeFalse()
|
||||
expect(dqValidate(rules, '.a')).toBeFalse()
|
||||
expect(dqValidate(rules, '_')).toBeFalse()
|
||||
// A bare single letter is a real character value and must be matched.
|
||||
expect(dqValidate(rules, 'd')).toBeFalse()
|
||||
})
|
||||
|
||||
it('fails open (treats as valid) when the pattern is malformed', () => {
|
||||
@@ -44,6 +60,19 @@ describe('dqValidate - HARDREGEX', () => {
|
||||
expect(dqValidate(rules, 'anything')).toBeTrue()
|
||||
})
|
||||
|
||||
it('validates a single letter on a character column against the pattern', () => {
|
||||
// A bare single letter ("d", "z") is real character data, not a SAS
|
||||
// special missing - it must reach the pattern, not be exempted.
|
||||
const rules = [rule({ RULE_VALUE: '/the|data/i' })]
|
||||
|
||||
expect(dqValidate(rules, 'd')).toBeFalse()
|
||||
expect(dqValidate(rules, 'z')).toBeFalse()
|
||||
expect(dqValidate(rules, 'the')).toBeTrue()
|
||||
expect(dqValidate(rules, 'DATA')).toBeTrue()
|
||||
expect(dqValidate(rules, 'some data here')).toBeTrue()
|
||||
expect(dqValidate(rules, '')).toBeTrue() // blank stays exempt
|
||||
})
|
||||
|
||||
it('handles a more elaborate pattern (multiple character classes, quantifiers, an escaped literal dot)', () => {
|
||||
// Same email pattern used in the getdata.js mock's REGEX_HARD_COL demo.
|
||||
const rules = [rule({ RULE_VALUE: '[\\w.]+@[\\w]+\\.[a-z]{2,}' })]
|
||||
|
||||
@@ -4,7 +4,11 @@ import { isRegexRuleExempt } from '../utils/isRegexRuleExempt'
|
||||
import { parseRegexRule } from '../utils/parseRegexRule'
|
||||
|
||||
const dqValidation: {
|
||||
[key: string]: (value: any, ruleValue: string | number) => boolean
|
||||
[key: string]: (
|
||||
value: any,
|
||||
ruleValue: string | number,
|
||||
isNumeric?: boolean
|
||||
) => boolean
|
||||
} = {
|
||||
CASE: (value: any, ruleValue: string | number): boolean => {
|
||||
switch (ruleValue) {
|
||||
@@ -51,8 +55,12 @@ const dqValidation: {
|
||||
},
|
||||
// Pattern is used as authored, not auto-anchored — a rule author who
|
||||
// wants a full-value match must write ^...$ themselves.
|
||||
HARDREGEX: (value: any, ruleValue: string | number): boolean => {
|
||||
if (isRegexRuleExempt(value)) return true
|
||||
HARDREGEX: (
|
||||
value: any,
|
||||
ruleValue: string | number,
|
||||
isNumeric: boolean = false
|
||||
): boolean => {
|
||||
if (isRegexRuleExempt(value, isNumeric)) return true
|
||||
|
||||
try {
|
||||
return parseRegexRule(ruleValue.toString()).test(value.toString())
|
||||
@@ -65,10 +73,16 @@ const dqValidation: {
|
||||
}
|
||||
}
|
||||
|
||||
export const dqValidate = (dqRules: DQRule[], value: any): boolean => {
|
||||
export const dqValidate = (
|
||||
dqRules: DQRule[],
|
||||
value: any,
|
||||
isNumeric: boolean = false
|
||||
): boolean => {
|
||||
for (let detail of dqRules) {
|
||||
if (dqValidation[detail.RULE_TYPE]) {
|
||||
if (!dqValidation[detail.RULE_TYPE](value, detail.RULE_VALUE)) {
|
||||
if (
|
||||
!dqValidation[detail.RULE_TYPE](value, detail.RULE_VALUE, isNumeric)
|
||||
) {
|
||||
console.warn(
|
||||
`DQ Invalid Reason: ${
|
||||
detail.RULE_TYPE
|
||||
|
||||
@@ -18,4 +18,60 @@ describe('buildColInfoHtml', () => {
|
||||
'NAME: SOME_CHAR<br>LABEL: Some Character Column<br>TYPE: char<br>LENGTH: 1024<br>FORMAT: $1024.'
|
||||
)
|
||||
})
|
||||
|
||||
it('appends a HARDREGEX line when only a HARDREGEX value is provided', () => {
|
||||
const colInfo: DataFormat = {
|
||||
label: 'Some Character Column',
|
||||
type: 'char',
|
||||
length: '1024',
|
||||
format: '$1024.'
|
||||
}
|
||||
|
||||
expect(
|
||||
buildColInfoHtml('SOME_CHAR', colInfo, '/^[A-Z]+$/i', undefined)
|
||||
).toBe(
|
||||
'NAME: SOME_CHAR<br>LABEL: Some Character Column<br>TYPE: char<br>LENGTH: 1024<br>FORMAT: $1024.<br>HARDREGEX: /^[A-Z]+$/i'
|
||||
)
|
||||
})
|
||||
|
||||
it('appends a SOFTREGEX line when only a SOFTREGEX value is provided', () => {
|
||||
const colInfo: DataFormat = {
|
||||
label: 'Some Character Column',
|
||||
type: 'char',
|
||||
length: '1024',
|
||||
format: '$1024.'
|
||||
}
|
||||
|
||||
expect(
|
||||
buildColInfoHtml('SOME_CHAR', colInfo, undefined, '/^[a-z]+$/')
|
||||
).toBe(
|
||||
'NAME: SOME_CHAR<br>LABEL: Some Character Column<br>TYPE: char<br>LENGTH: 1024<br>FORMAT: $1024.<br>SOFTREGEX: /^[a-z]+$/'
|
||||
)
|
||||
})
|
||||
|
||||
it('appends only the HARDREGEX line when a column has both rules, since HARDREGEX is the one applied', () => {
|
||||
const colInfo: DataFormat = {
|
||||
label: 'Some Character Column',
|
||||
type: 'char',
|
||||
length: '1024',
|
||||
format: '$1024.'
|
||||
}
|
||||
|
||||
expect(
|
||||
buildColInfoHtml('SOME_CHAR', colInfo, '/^[A-Z]+$/i', '/^.{5,10}$/')
|
||||
).toBe(
|
||||
'NAME: SOME_CHAR<br>LABEL: Some Character Column<br>TYPE: char<br>LENGTH: 1024<br>FORMAT: $1024.<br>HARDREGEX: /^[A-Z]+$/i'
|
||||
)
|
||||
})
|
||||
|
||||
it('omits the REGEX line when no regex rule value is provided', () => {
|
||||
const colInfo: DataFormat = {
|
||||
label: 'Some Character Column',
|
||||
type: 'char',
|
||||
length: '1024',
|
||||
format: '$1024.'
|
||||
}
|
||||
|
||||
expect(buildColInfoHtml('SOME_CHAR', colInfo)).not.toContain('REGEX:')
|
||||
})
|
||||
})
|
||||
|
||||
@@ -7,9 +7,23 @@ import { DataFormat } from '../../models/sas/common/DateFormat'
|
||||
*/
|
||||
export function buildColInfoHtml(
|
||||
colName: string,
|
||||
colInfo?: DataFormat
|
||||
colInfo?: DataFormat,
|
||||
hardRegexValue?: string,
|
||||
softRegexValue?: string
|
||||
): string {
|
||||
if (!colInfo) return 'No info found'
|
||||
|
||||
return `NAME: ${colName}<br>LABEL: ${colInfo.label}<br>TYPE: ${colInfo.type}<br>LENGTH: ${colInfo.length}<br>FORMAT: ${colInfo.format}`
|
||||
let html = `NAME: ${colName}<br>LABEL: ${colInfo.label}<br>TYPE: ${colInfo.type}<br>LENGTH: ${colInfo.length}<br>FORMAT: ${colInfo.format}`
|
||||
|
||||
// Only ever one REGEX rule is applied per column: when both HARDREGEX
|
||||
// and SOFTREGEX exist, SOFTREGEX is ignored entirely (same precedence as
|
||||
// makeRegexWarningRenderer / DcValidator.failsSoftRegex). Show only the
|
||||
// rule that is applied.
|
||||
if (hardRegexValue) {
|
||||
html += `<br>HARDREGEX: ${hardRegexValue}`
|
||||
} else if (softRegexValue) {
|
||||
html += `<br>SOFTREGEX: ${softRegexValue}`
|
||||
}
|
||||
|
||||
return html
|
||||
}
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "dcfrontend",
|
||||
"version": "7.11.0",
|
||||
"version": "7.12.0",
|
||||
"description": "Data Controller",
|
||||
"devDependencies": {
|
||||
"@saithodev/semantic-release-gitea": "^2.1.0",
|
||||
|
||||
@@ -52,7 +52,8 @@ function makeRows(n) {
|
||||
ROUND_COL: Number((i + 1 + i / 7).toFixed(5)), // rounds on edit
|
||||
NUMFMT_COL: 1000 + i * 12.5, // shown as EUR
|
||||
REGEX_HARD_COL: "user@example.com", // HARDREGEX: email — starts valid
|
||||
REGEX_SOFT_COL: "SW1A 1AA" // SOFTREGEX: UK postcode — starts valid
|
||||
REGEX_SOFT_COL: "SW1A 1AA", // SOFTREGEX: UK postcode — starts valid
|
||||
REGEX_BOTH_COL: "ABC-123" // HARDREGEX + SOFTREGEX together — starts valid against both
|
||||
})
|
||||
}
|
||||
return rows
|
||||
@@ -241,6 +242,19 @@ let webouts = {
|
||||
DESC: "SOFTREGEX: should be a valid UK postcode, shown as a yellow warning if not, but can still submit",
|
||||
LONGDESC: "",
|
||||
COLTYPE: "{\"data\":\"REGEX_SOFT_COL\"}"
|
||||
},
|
||||
{
|
||||
NAME: "REGEX_BOTH_COL",
|
||||
LABEL: "REGEX_BOTH_COL",
|
||||
FMTNAME: "",
|
||||
DDTYPE: "C",
|
||||
CLS_RULE: "READ",
|
||||
MEMLABEL: "",
|
||||
DESC: "HARDREGEX + SOFTREGEX together: must be uppercase/digits/-/_ (blocking), recommended 5-10 chars long (warning)",
|
||||
LONGDESC: ""
|
||||
// No COLTYPE - DcValidator falls back to a plain { data: name }
|
||||
// rule when COLTYPE is absent, same shape this trivial JSON
|
||||
// would produce anyway (see parseColTypeRow).
|
||||
}
|
||||
],
|
||||
dqdata: [
|
||||
@@ -266,7 +280,9 @@ let webouts = {
|
||||
{ BASE_COL: "ROUND_COL", RULE_TYPE: "ROUND", RULE_VALUE: "2" },
|
||||
{ BASE_COL: "NUMFMT_COL", RULE_TYPE: "NUMBER_FORMAT", RULE_VALUE: '{"style":"currency","currency":"EUR"}' },
|
||||
{ BASE_COL: "REGEX_HARD_COL", RULE_TYPE: "HARDREGEX", RULE_VALUE: "/[\\w.]+@[\\w]+\\.[a-z]{2,}/" },
|
||||
{ BASE_COL: "REGEX_SOFT_COL", RULE_TYPE: "SOFTREGEX", RULE_VALUE: "/[A-Z]{1,2}\\d{1,2}[A-Z]?\\s?\\d[A-Z]{2}/" }
|
||||
{ BASE_COL: "REGEX_SOFT_COL", RULE_TYPE: "SOFTREGEX", RULE_VALUE: "/[A-Z]{1,2}\\d{1,2}[A-Z]?\\s?\\d[A-Z]{2}/" },
|
||||
{ BASE_COL: "REGEX_BOTH_COL", RULE_TYPE: "HARDREGEX", RULE_VALUE: "/^[A-Z0-9_-]+$/" },
|
||||
{ BASE_COL: "REGEX_BOTH_COL", RULE_TYPE: "SOFTREGEX", RULE_VALUE: "/^.{5,10}$/" }
|
||||
],
|
||||
dsmeta: [
|
||||
{ ODS_TABLE: "ATTRIBUTES", NAME: "Data Set Name", VALUE: "DC996664.MPE_X_TEST" },
|
||||
@@ -319,7 +335,8 @@ let webouts = {
|
||||
{ NAME: "round_col", MAXLEN: 8 },
|
||||
{ NAME: "numfmt_col", MAXLEN: 8 },
|
||||
{ NAME: "regex_hard_col", MAXLEN: 128 },
|
||||
{ NAME: "regex_soft_col", MAXLEN: 128 }
|
||||
{ NAME: "regex_soft_col", MAXLEN: 128 },
|
||||
{ NAME: "regex_both_col", MAXLEN: 128 }
|
||||
],
|
||||
query: [],
|
||||
sasdata: makeRows(100),
|
||||
@@ -341,12 +358,13 @@ let webouts = {
|
||||
ROUND_COL: { format: "best.", label: "ROUND_COL", length: "8", type: "num" },
|
||||
NUMFMT_COL: { format: "best.", label: "NUMFMT_COL", length: "8", type: "num" },
|
||||
REGEX_HARD_COL: { format: "$128.", label: "REGEX_HARD_COL", length: "128", type: "char" },
|
||||
REGEX_SOFT_COL: { format: "$128.", label: "REGEX_SOFT_COL", length: "128", type: "char" }
|
||||
REGEX_SOFT_COL: { format: "$128.", label: "REGEX_SOFT_COL", length: "128", type: "char" },
|
||||
REGEX_BOTH_COL: { format: "$128.", label: "REGEX_BOTH_COL", length: "128", type: "char" }
|
||||
}
|
||||
},
|
||||
sasparams: [
|
||||
{
|
||||
COLHEADERS: "_____DELETE__THIS__RECORD_____,PRIMARY_KEY_FIELD,SOME_CHAR,SOME_DROPDOWN,SOME_HARDSELECT,SOME_NUM,SOME_DATE,SOME_DATETIME,SOME_TIME,SOME_SHORTNUM,SOME_BESTNUM,READONLY_COL,HIDDEN_COL,ROUND_COL,NUMFMT_COL,REGEX_HARD_COL,REGEX_SOFT_COL",
|
||||
COLHEADERS: "_____DELETE__THIS__RECORD_____,PRIMARY_KEY_FIELD,SOME_CHAR,SOME_DROPDOWN,SOME_HARDSELECT,SOME_NUM,SOME_DATE,SOME_DATETIME,SOME_TIME,SOME_SHORTNUM,SOME_BESTNUM,READONLY_COL,HIDDEN_COL,ROUND_COL,NUMFMT_COL,REGEX_HARD_COL,REGEX_SOFT_COL,REGEX_BOTH_COL",
|
||||
FILTER_TEXT: FILTER_TEXT,
|
||||
PKCNT: 1,
|
||||
PK: "PRIMARY_KEY_FIELD",
|
||||
@@ -1501,7 +1519,7 @@ let webouts = {
|
||||
// MPE_X_TEST keeps matching the excel upload fixtures (which predate these
|
||||
// columns). The demo tables are built lazily (below) so we don't clone the large
|
||||
// MPE_X_TEST payload unless MPE_X_NEW is actually requested.
|
||||
const RULE_DEMO_COLS = ['SOME_HARDSELECT', 'READONLY_COL', 'HIDDEN_COL', 'ROUND_COL', 'NUMFMT_COL', 'REGEX_HARD_COL', 'REGEX_SOFT_COL']
|
||||
const RULE_DEMO_COLS = ['SOME_HARDSELECT', 'READONLY_COL', 'HIDDEN_COL', 'ROUND_COL', 'NUMFMT_COL', 'REGEX_HARD_COL', 'REGEX_SOFT_COL', 'REGEX_BOTH_COL']
|
||||
|
||||
function stripRuleCols(t) {
|
||||
const uc = new Set(RULE_DEMO_COLS)
|
||||
|
||||
@@ -2012,6 +2012,29 @@ insert into &lib..MPE_VALIDATIONS set
|
||||
,rule_value='/t/'
|
||||
,rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
/* test soft regex - PRIMARY_KEY_FIELD should be an integer (yellow if it
|
||||
contains a decimal point). All generated keys are integers, so no
|
||||
warning shows until a demo user enters a decimal. */
|
||||
insert into &lib..MPE_VALIDATIONS set
|
||||
tx_from=0
|
||||
,base_lib="&lib"
|
||||
,base_ds="MPE_X_TEST"
|
||||
,base_col="PRIMARY_KEY_FIELD"
|
||||
,rule_type='SOFTREGEX'
|
||||
,rule_value='/^\d+$/'
|
||||
,rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
/* test hard regex - SOME_SHORTNUM must not be between 1 and 5. Generated
|
||||
data starts at 6, so nothing is blocked until a demo user enters 1-5. */
|
||||
insert into &lib..MPE_VALIDATIONS set
|
||||
tx_from=0
|
||||
,base_lib="&lib"
|
||||
,base_ds="MPE_X_TEST"
|
||||
,base_col="SOME_SHORTNUM"
|
||||
,rule_type='HARDREGEX'
|
||||
,rule_value='/^(??$).*/'
|
||||
,rule_active=1
|
||||
,tx_to='31DEC5999:23:59:59'dt;
|
||||
insert into &lib..MPE_VALIDATIONS set
|
||||
tx_from=0
|
||||
,base_lib="&lib"
|
||||
@@ -2061,7 +2084,7 @@ insert into &lib..MPE_VALIDATIONS set
|
||||
,some_date=42
|
||||
,some_datetime=42
|
||||
,some_time=42
|
||||
,some_shortnum=3
|
||||
,some_shortnum=8
|
||||
,some_bestnum=44;
|
||||
insert into &lib..mpe_x_test
|
||||
set primary_key_field=1
|
||||
@@ -2071,7 +2094,7 @@ insert into &lib..MPE_VALIDATIONS set
|
||||
,some_date=42
|
||||
,some_datetime=42
|
||||
,some_time=422
|
||||
,some_shortnum=3
|
||||
,some_shortnum=8
|
||||
,some_bestnum=44;
|
||||
insert into &lib..mpe_x_test
|
||||
set primary_key_field=2
|
||||
@@ -2081,7 +2104,7 @@ insert into &lib..MPE_VALIDATIONS set
|
||||
,some_date=42
|
||||
,some_datetime=42
|
||||
,some_time=142
|
||||
,some_shortnum=3
|
||||
,some_shortnum=8
|
||||
,some_bestnum=44;
|
||||
insert into &lib..mpe_x_test
|
||||
set primary_key_field=3
|
||||
@@ -2093,7 +2116,7 @@ insert into &lib..MPE_VALIDATIONS set
|
||||
,some_date=423
|
||||
,some_datetime=423
|
||||
,some_time=44
|
||||
,some_shortnum=3
|
||||
,some_shortnum=8
|
||||
,some_bestnum=44;
|
||||
insert into &lib..mpe_x_test
|
||||
set primary_key_field=4
|
||||
@@ -2103,7 +2126,7 @@ insert into &lib..MPE_VALIDATIONS set
|
||||
,some_date=4231
|
||||
,some_datetime=423123123
|
||||
,some_time=412
|
||||
,some_shortnum=3
|
||||
,some_shortnum=8
|
||||
,some_bestnum=44;
|
||||
%do x=10 %to 500;
|
||||
insert into &lib..mpe_x_test
|
||||
@@ -2114,7 +2137,7 @@ insert into &lib..MPE_VALIDATIONS set
|
||||
,some_date=round(ranuni(0)*1000,1)
|
||||
,some_datetime=round(ranuni(0)*50000,1)
|
||||
,some_time=round(ranuni(0)*100,1)
|
||||
,some_shortnum=round(ranuni(0)*100,1)
|
||||
,some_shortnum=6+round(ranuni(0)*94,1)
|
||||
,some_bestnum=round(ranuni(0)*100,1);
|
||||
%end;
|
||||
|
||||
|
||||
Reference in New Issue
Block a user