blog: add formula, range-value and NOTNULL-constraint caveats to the special missings post
Verified on a real SAS estate: - a physical NOT NULL (or primary key) constraint rejects a special missing, and getdata merges that constraint into a frontend NOTNULL rule - which passes a special missing, so the editor is more lenient than the constraint - a special missing as a MINVAL rule value fails every cell; as a MAXVAL rule value it fails every real number - a HARDFORMULA/SOFTFORMULA reading a special-missing cell returns #VALUE! - PRX and the JS engine agree on the value; SAS pads the numeric-to-character conversion, so an anchored pattern re-used in SAS needs strip()
This commit is contained in:
@@ -50,6 +50,11 @@ These are Data Controller's own rules, configured per column in the `MPE_VALIDAT
|
|||||||
- `MAXVAL` - passes, for the same reason it is below any maximum
|
- `MAXVAL` - passes, for the same reason it is below any maximum
|
||||||
- `HARDREGEX` - checked against the pattern like any other value; unlike blanks and the plain `.`, special missings are **not** exempt, so a numeric column that carries them needs a pattern which allows for a single letter
|
- `HARDREGEX` - checked against the pattern like any other value; unlike blanks and the plain `.`, special missings are **not** exempt, so a numeric column that carries them needs a pattern which allows for a single letter
|
||||||
- `SOFTREGEX` - the same check, but a failure is only a warning rather than a block, and it is ignored entirely if the column also has a `HARDREGEX` rule
|
- `SOFTREGEX` - the same check, but a failure is only a warning rather than a block, and it is ignored entirely if the column also has a `HARDREGEX` rule
|
||||||
|
- `HARDFORMULA` / `SOFTFORMULA` - a formula that reads a special-missing cell does not compute; the grid's spreadsheet engine returns `#VALUE!` for that row, where the plain `.` contributes 0
|
||||||
|
|
||||||
|
One mismatch to be aware of: the `NOTNULL` rule above is the editor's, and it is more forgiving than the SAS constraint it mirrors. A real NOT NULL - or primary key - constraint on the target table rejects a special missing, so on a column carrying one the editor will let the value through and the load will then fail on the constraint.
|
||||||
|
|
||||||
|
Two other sharp edges. A range rule's *value* has to be a number: put a special missing in `MINVAL` and every cell in the column fails, put one in `MAXVAL` and every real number fails. And although the regex pattern is written in SAS PRX syntax, the check itself runs in the browser - SAS only parses the pattern (`PRXPARSE`) when the rule is saved - so the two engines can disagree on exotic patterns, and SAS pads a numeric-to-character conversion, so a pattern re-used in SAS needs `strip()` for an anchored match.
|
||||||
|
|
||||||
`CASE` (`UPCASE` / `LOWCASE`) is a character rule and does not come into it: SAS hands the browser a special missing as an uppercase letter, so there is no case to enforce, and a `CASE` rule on a numeric column would reject the column's numbers rather than the missing.
|
`CASE` (`UPCASE` / `LOWCASE`) is a character rule and does not come into it: SAS hands the browser a special missing as an uppercase letter, so there is no case to enforce, and a `CASE` rule on a numeric column would reject the column's numbers rather than the missing.
|
||||||
|
|
||||||
@@ -86,10 +91,11 @@ We solved that in the open source SASjs Adapter, and it has been in Data Control
|
|||||||
|
|
||||||
And they behave like values in Data Controller's validation rules, which is usually what you want:
|
And they behave like values in Data Controller's validation rules, which is usually what you want:
|
||||||
|
|
||||||
- NOTNULL passes - it is not null
|
- NOTNULL passes - it is not null (though a real SAS NOT NULL constraint is stricter, and does reject a special missing)
|
||||||
- MINVAL fails - it sorts below every number; MAXVAL passes for the same reason
|
- MINVAL fails - it sorts below every number; MAXVAL passes for the same reason
|
||||||
- HARDREGEX checks it against the pattern, unlike blanks and plain .
|
- HARDREGEX checks it against the pattern, unlike blanks and plain .
|
||||||
- SOFTREGEX warns instead of blocking (and is ignored if the column also has a HARDREGEX)
|
- SOFTREGEX warns instead of blocking (and is ignored if the column also has a HARDREGEX)
|
||||||
|
- a HARDFORMULA/SOFTFORMULA that reads a special-missing cell returns #VALUE! rather than a number
|
||||||
- CASE is a character rule, so it has no place on a numeric column
|
- CASE is a character rule, so it has no place on a numeric column
|
||||||
|
|
||||||
So a value recorded as "not collected" cannot quietly satisfy a completeness or range rule.
|
So a value recorded as "not collected" cannot quietly satisfy a completeness or range rule.
|
||||||
|
|||||||
Reference in New Issue
Block a user