dc/docs.datacontroller.io
6
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(validations): document how each rule treats a special missing value #8

Merged
allan merged 7 commits from docs/special-missing-rule-behaviour into main 2026-09-23 22:30:42 +00:00
Conversation 1 Commits 7 Files Changed 1
+33 -5
1 changed files with 33 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/dcc-validations.md
+33 -5
View File
@@ -23,17 +23,17 @@ It is possible to configure a number of other rules by updating the MPE_VALIDATI
## Configurable Checks
Check back frequently as we keep growing this list of checks.
Check back frequently as we keep growing this list of checks. For how each rule treats a special missing value (`.A`-`.Z`, `._`), see [Special Missing Values](#special-missing-values).
|Rule Type|Example Value |Description|
|---|---|---|
|CASE|UPCASE|Will enforce the case of cell values. Valid values: UPCASE, LOWCASE, PROPCASE|
|NOTNULL|(defaultval)|Will prevent submission if null values are present. Optional - provide a default value.|
|MINVAL|1|Defines a minimum value for a numeric cell|
|MAXVAL|1000000|Defines a maximum value for a numeric cell|
|NOTNULL|(defaultval)|Will prevent submission if null values are present - a special missing counts as null. Optional - provide a default value.|
|MINVAL|1|Defines a minimum value for a numeric cell. A special missing value sorts below every number, so it fails a numeric floor, and the floor itself may be a special missing - see [Special Missing Values](#special-missing-values).|
|MAXVAL|1000000|Defines a maximum value for a numeric cell. A special missing value sorts below every number, so it passes a numeric ceiling, and the ceiling itself may be a special missing - see [Special Missing Values](#special-missing-values).|
|READONLY|(defaultval) |Renders the column read-only in the editor. The defaultval is used when rows are added. |
|HIDDEN|(defaultval) |Hides the column from the editor grid while still submitting its data. The defaultval is used when rows are added. |
|ROUND|2 |Rounds numeric input on paste/edit. Positive digits round to number of decimal places (eg 2 rounds to 0.01) Negative digits round to the nearest ten/hundred etc. Half-away-from-zero rounding is applied so `-0.5 → -1` and `2.5 → 3`. |
|ROUND|2 |Rounds numeric input on paste/edit. Positive digits round to number of decimal places (eg 2 rounds to 0.01) Negative digits round to the nearest ten/hundred etc. Half-away-from-zero rounding is applied so `-0.5 -> -1` and `2.5 -> 3`. |
|NUMBER_FORMAT|`{"style":"currency","currency":"GBP"}` |Display-only [`Intl.NumberFormat` renderer](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl/NumberFormat). RULE_VALUE is the JSON options object passed straight to `Intl.NumberFormat`. Does not change the stored value. |
|HARDFORMULA|`= PRICE * VOLUME`|The cell displays a value computed from a formula, using other columns in the same row. The column is read-only - the user cannot override the result. See [Formula Rules](#formula-rules) below.|
|SOFTFORMULA|`= if( DC.ROW_STATUS != 'U', DC.USER_NAME, DC.ORIG_VALUE )`|Like HARDFORMULA, but the user can override the computed value and type their own. See [Formula Rules](#formula-rules) below.|
@@ -45,6 +45,34 @@ Check back frequently as we keep growing this list of checks.
|[SOFTSELECT_HOOK](/dynamic-cell-dropdown)|/physical/path/program.sas|A SAS service (STP or Viya Job) or a path to a SAS program on the filesystem. User-provided values may (or may not) be in this list. Cannot be used alongside a HARDSELECT_HOOK.|
## Special Missing Values
A SAS numeric variable has 28 missing values: the regular `.` and 27 special ones (`.A`-`.Z` and `._`). Data Controller accepts them in a numeric cell as a letter - `a` to `z` - or an underscore, with or without a leading period (`a`, `.a`, `_`, `._`), and stores the matching special missing in SAS. The period is optional, and a plain `.` stores the regular missing.
A numeric column that carries a date, datetime or time format is the exception. Those cells are edited through a date picker rather than the numeric editor, and the picker accepts only a date - so a special missing cannot be typed into one of them.
The rules treat a special missing as a value for some checks and as a missing value for others - and the range rules compare it in the order SAS gives it, below every number but ordered among the other missing values:
|Rule|What happens to a special missing|
|---|---|
|NOTNULL|Fails. A special missing is a missing value, so a NOT NULL (or primary key) constraint rejects it, and a real SAS NOT NULL constraint on the target table rejects it too.|
|MINVAL|Fails if the minimum is a number - a missing sorts below every number, so it is below any numeric floor. A minimum that is itself a special missing is compared in the order the missing values have among themselves, and can accept one.|
|MAXVAL|Passes - a missing sorts below every number, so it is below any numeric ceiling. A ceiling that is itself a special missing is compared in the missing values' own order, and can reject one.|
|CASE|Not applicable - it is a character rule.|
|ROUND|Ignored - ROUND only rounds values that are numbers, so a special missing is left as typed.|
|HARDREGEX / SOFTREGEX|Checked against the pattern like any other value. Unlike blanks and the plain `.`, special missings are **not** exempt. See [Regex Rules](#regex-rules).|
|SOFTSELECT|Supports special missings - a soft dropdown never blocks a value.|
|HARDSELECT|Supports special missings - the value is matched against the dropdown list like any other value.|
|HARDFORMULA / SOFTFORMULA|The formula returns `#VALUE!` for that row instead of a number, and that is what gets submitted.|
In short, a special missing is rejected by NOTNULL, by a primary key column, and by a MINVAL whose floor is a number; a formula returns `#VALUE!` for it. It is fine in a column that carries a MAXVAL, a regex, a dropdown, or ROUND - and it can satisfy a MINVAL when the floor is expressed as a special missing.
A range rule compares in the order SAS itself uses, so a range written in special missings means what SAS would mean by it. That order puts every missing below every non-missing value, and orders the missing values as `._`, then the regular missing, then `.A` through `.Z`. So `MINVAL .A` with `MAXVAL .C` accepts `.B` and rejects `.D`, and a blank - the regular missing - fails a floor of `.A` because it sorts below it. A number sorts above every missing, so it passes a floor of `.A` and fails a ceiling of `.C`.
A primary key column is treated as NOT NULL whether or not the target table carries a physical constraint, and whether or not MPE_VALIDATIONS has a NOTNULL rule for it - a key identifies the row, so a blank and a special missing are both rejected there.
A range rule's own value is read as a number when it looks like one and as a missing value when it is a special missing. A value that is neither - a typo such as `..` or `AB` - satisfies nothing, so every cell in the column fails until the rule is corrected.
## Formula Rules
HARDFORMULA and SOFTFORMULA let you configure a column so that its value is automatically calculated from other columns in the same row - just like a spreadsheet formula. When a user opens the editor, the formula is evaluated live and the result is shown in each cell.