dc
2fe1bc2eea
Verified on a real SAS estate: - a physical NOT NULL (or primary key) constraint rejects a special missing, and getdata merges that constraint into a frontend NOTNULL rule - which passes a special missing, so the editor is more lenient than the constraint - a special missing as a MINVAL rule value fails every cell; as a MAXVAL rule value it fails every real number - a HARDFORMULA/SOFTFORMULA reading a special-missing cell returns #VALUE! - PRX and the JS engine agree on the value; SAS pads the numeric-to-character conversion, so an anchored pattern re-used in SAS needs strip()
11 KiB
title, description, date, author, authorLink, tags, previewImg
| title | description | date | author | authorLink | tags | previewImg | ||
|---|---|---|---|---|---|---|---|---|
| 28 Ways to Be Missing in SAS | A SAS numeric missing is not a lone wolf - there are 28 of them, and Data Controller has handled all of them since v4. How they work, and what the validation rules do with them. | 2026-09-22 09:00:00 | Allan Bowe | https://www.linkedin.com/in/allanbowe/ |
|
./cover.jpeg |
28 Ways to Be Missing in SAS
A numeric missing value in SAS is not a lone wolf. The ordinary missing (.) is one of 28 distinct missing values available for a numeric variable - the other 27 are written with a single character, the letters A to Z or an underscore (._).
They exist because "missing" is usually not the whole story. A survey question that was never reached, a reading that was illegible, a value the respondent refused to give - in a well run process those are different facts, and a lone . throws the difference away. Special missings record why the value is missing.
They are numbers, not text
- Arithmetic on them yields missing -
.A + 1is. proc means,proc summaryand friends exclude them, exactly as they exclude.- They sort below every non-missing number, in the order
._, then., then.Ato.Z - The
NMISS()andCMISS()functions count them as missing - Converting one to text drops the period -
cats(.A)is the stringA
A display gotcha worth knowing: a regular missing prints as ., unless options MISSING= changes that character - set it to blank and a regular missing renders as an empty cell. The option affects only the regular missing; ._ and .A-.Z always print as their own letter. So a blank cell in a SAS listing is still unambiguously a regular missing, and a lone letter is still a special one. Data Controller itself is unaffected either way - it sends a regular missing to the browser as null and a special missing as its letter.
In a SAS dataset they are written with a leading period (.A, .B ... ._). In Data Controller you type just the letter or the underscore - no period - and the letter is not case sensitive. Two letters, or a letter mixed with a number, are refused rather than guessed.
Carrying them between the browser and SAS
The Data Controller frontend and the SAS backend exchange data as JSON, and JSON has no way to express a letter as a numeric value - A is a string. The conversion is handled in the open source SASjs Adapter:
- The adapter infers each column's SAS type from the values it is given. All numeric values means numeric; all strings means character; and a column holding a single character (
a-z,_or.) alongside numeric values is numeric, with the lone characters written to SAS as special missings. nullbecomes.or an empty string, according to the type derived for that column.- Two cases cannot be inferred from the values alone: a numeric column containing only special missings looks like a single character column, and a character column containing only nulls looks numeric. For those, the adapter accepts an explicit format for the column - and Data Controller sends one automatically, because it already knows each column's SAS format from the metadata returned by the backend.
- A value that is neither a number nor a single valid character is refused rather than guessed (for example
aaaa, or!in a numeric column), and a literal.is refused in favour ofnullfor a regular missing.
There is nothing to configure. Special missings are available by default, for numeric cells.
Once in SAS they are ordinary values, so they are what the approval DIFF screen compares, and the DIFF's formatted / unformatted switch shows either the formatted representation or the raw value - useful for confirming exactly which missing was set.
What the Data Controller validation rules do with them
These are Data Controller's own rules, configured per column in the MPE_VALIDATIONS table and applied in the browser as you edit and submit. A special missing is a value somebody deliberately set, so the rules treat it as a value - it is not blank, and it is not exempt:
NOTNULL- passes, because it is not nullMINVAL- fails, because a special missing sorts below every number and so is below any minimumMAXVAL- passes, for the same reason it is below any maximumHARDREGEX- checked against the pattern like any other value; unlike blanks and the plain., special missings are not exempt, so a numeric column that carries them needs a pattern which allows for a single letterSOFTREGEX- the same check, but a failure is only a warning rather than a block, and it is ignored entirely if the column also has aHARDREGEXruleHARDFORMULA/SOFTFORMULA- a formula that reads a special-missing cell does not compute; the grid's spreadsheet engine returns#VALUE!for that row, where the plain.contributes 0
One mismatch to be aware of: the NOTNULL rule above is the editor's, and it is more forgiving than the SAS constraint it mirrors. A real NOT NULL - or primary key - constraint on the target table rejects a special missing, so on a column carrying one the editor will let the value through and the load will then fail on the constraint.
Two other sharp edges. A range rule's value has to be a number: put a special missing in MINVAL and every cell in the column fails, put one in MAXVAL and every real number fails. And although the regex pattern is written in SAS PRX syntax, the check itself runs in the browser - SAS only parses the pattern (PRXPARSE) when the rule is saved - so the two engines can disagree on exotic patterns, and SAS pads a numeric-to-character conversion, so a pattern re-used in SAS needs strip() for an anchored match.
CASE (UPCASE / LOWCASE) is a character rule and does not come into it: SAS hands the browser a special missing as an uppercase letter, so there is no case to enforce, and a CASE rule on a numeric column would reject the column's numbers rather than the missing.
That is usually what you want - a value recorded as "not collected" should not quietly satisfy a completeness or range rule. It does mean a range rule on a column that uses special missings will warn on those rows, which is the signal to either accommodate them in the rule or not use special missings on that column.
video: [Managing Special Missing Values with Data Controller for SAS](https://www.youtube-nocookie.com/embed/ggrcNr23Jzw)
The recording shows the full cycle: typing special missings into numeric cells, the rejections, submitting the changes, approving them, and reviewing the DIFF - including a change from one special missing to another, and the formatted / unformatted switch on date and datetime columns.