From 7d950cbdbee6a2fb5a83f0f07f3ac5b7b63e574a Mon Sep 17 00:00:00 2001 From: dc Date: Wed, 23 Sep 2026 22:41:12 +0000 Subject: [PATCH] feed: copy-edit the special missings post The LinkedIn opener did not parse - "A SAS numeric missing is not a lone wolf. It is 28." - and reading the rest of the post and its LinkedIn version together turned up a further set of style defects: - the article opener made the missing value the wolf, and padded the count - "All numeric values means numeric; all strings means character" was clumsy - procedure and function names were lowercase in places (proc means, cats, options MISSING), and "and friends" was too casual for a list of procedures - "gotcha" and "harmless" undersold the points they introduced - "One thing worth knowing" opened two separate paragraphs - "a value ... and as a value with its own place" repeated itself, in both the article and the LinkedIn version - the adapter bullet list mixed trailing periods with none - the LinkedIn list was split in two by a stray blank line, and "a physical constraint" did not say which constraint - "the most common of them" echoed the "28 of them" in the line above No claims changed - the range-rule behaviour, the period, the primary key and the strict dropdown all read as they did. --- content/feed/sas-special-missings/index.md | 43 +++++++++++----------- 1 file changed, 21 insertions(+), 22 deletions(-) diff --git a/content/feed/sas-special-missings/index.md b/content/feed/sas-special-missings/index.md index 846c980..f900bd1 100644 --- a/content/feed/sas-special-missings/index.md +++ b/content/feed/sas-special-missings/index.md @@ -1,6 +1,6 @@ --- title: '28 Ways to Be Missing in SAS' -description: A SAS numeric missing is not a lone wolf - there are 28 of them, and Data Controller has handled all of them since v4. How they work, and what the validation rules do with them. +description: A SAS numeric missing is not a lone wolf - there are 28 of them, and Data Controller has supported all of them since v4. How they work, and what the validation rules do with them. date: '2026-09-22 09:00:00' author: 'Allan Bowe' authorLink: https://www.linkedin.com/in/allanbowe/ @@ -12,19 +12,19 @@ previewImg: './cover.jpeg' # 28 Ways to Be Missing in SAS -A numeric missing value in SAS is not a lone wolf. The ordinary missing (`.`) is one of **28** distinct missing values available for a numeric variable - the other 27 are written with a single character, the letters `A` to `Z` or an underscore (`._`). +A SAS numeric missing is not a lone wolf: the ordinary missing (`.`) is one of **28** distinct missing values a numeric variable can hold. The other 27 are written with a single character - the letters `A` to `Z`, or an underscore (`._`). -They exist because "missing" is usually not the whole story. A survey question that was never reached, a reading that was illegible, a value the respondent refused to give - in a well run process those are different facts, and a lone `.` throws the difference away. Special missings record *why* the value is missing. +They exist because "missing" is usually not the whole story. A survey question that was never reached, a reading that was illegible, a value the respondent refused to give - in a well-run process those are different facts, and a lone `.` throws the difference away. Special missings record *why* the value is missing. ## They are numbers, not text - Arithmetic on them yields missing - `.A + 1` is `.` -- `proc means`, `proc summary` and friends exclude them, exactly as they exclude `.` +- `PROC MEANS`, `PROC SUMMARY` and the other summarisation procedures exclude them, exactly as they exclude `.` - They sort below every non-missing number, in the order `._`, then `.`, then `.A` to `.Z` -- The `NMISS()` and `CMISS()` functions count them as missing -- Converting one to text drops the period - `cats(.A)` is the string `A` +- `NMISS()` and `CMISS()` count them as missing +- Converting one to text drops the period - `CATS(.A)` is the string `A` -A display gotcha worth knowing: a regular missing prints as `.`, unless `options MISSING=` changes that character - set it to blank and a regular missing renders as an empty cell. The option affects only the regular missing; `._` and `.A`-`.Z` always print as their own letter. So a blank cell in a SAS listing is still unambiguously a regular missing, and a lone letter is still a special one. Data Controller itself is unaffected either way - it sends a regular missing to the browser as `null` and a special missing as its letter. +One display detail is worth knowing: a regular missing prints as `.`, unless `OPTIONS MISSING=` changes that character - set it to blank and a regular missing renders as an empty cell. The option affects only the regular missing; `._` and `.A`-`.Z` always print as their own letter. So a blank cell in a SAS listing is still unambiguously a regular missing, and a lone letter is still a special one. Data Controller itself is unaffected either way - it sends a regular missing to the browser as `null` and a special missing as its letter. In a SAS dataset they are written with a leading period (`.A`, `.B` ... `._`). In Data Controller you type the letter or the underscore, with or without that period - `.a` and `a` are the same missing - and the letter is not case sensitive. Two letters, or a letter mixed with a number, are refused rather than guessed. @@ -34,11 +34,11 @@ There is one cell where the letter cannot be typed at all. A numeric column that The Data Controller frontend and the SAS backend exchange data as JSON, and JSON has no way to express a letter as a numeric value - `A` is a string. The conversion is handled in the open source [SASjs Adapter](https://github.com/sasjs/adapter#variable-types): -- The adapter infers each column's SAS type from the values it is given. All numeric values means numeric; all strings means character; and a column holding a single character (`a`-`z`, `_` or `.`) alongside numeric values is numeric, with the lone characters written to SAS as special missings. +- The adapter infers each column's SAS type from the values it is given. All numeric values mean a numeric column, all strings mean a character column, and a column holding a single character (`a`-`z`, `_` or `.`) alongside numeric values is numeric, with the lone characters written to SAS as special missings. - `null` becomes `.` or an empty string, according to the type derived for that column. - Two cases cannot be inferred from the values alone: a numeric column containing *only* special missings looks like a single character column, and a character column containing only nulls looks numeric. For those, the adapter accepts an explicit format for the column - and Data Controller sends one automatically, because it already knows each column's SAS format from the metadata returned by the backend. -- A value that is neither a number nor a single valid character is refused rather than guessed - `aaaa`, or `!` in a numeric column -- A lone `.` is accepted as another way of typing the regular missing +- A value that is neither a number nor a single valid character is refused rather than guessed - `aaaa`, or `!` in a numeric column. +- A lone `.` is accepted as another way of typing the regular missing. There is nothing to configure. Special missings are available by default, for numeric cells - a date, datetime or time formatted column aside, since those edit through a date picker. @@ -52,9 +52,9 @@ These are Data Controller's own rules, configured per column in the `MPE_VALIDAT - `HARDREGEX` - checked against the pattern like any other value; unlike blanks and the plain `.`, special missings are **not** exempt, so a numeric column that carries them needs a pattern which allows for a single letter - `SOFTREGEX` - the same check, but a failure is only a warning rather than a block, and it is ignored entirely if the column also has a `HARDREGEX` rule - `SOFTSELECT` / `HARDSELECT` - both support them. The dropdown lists a special missing as a bare letter alongside the ordinary values, and a hard rule then accepts it like any other listed value - it still rejects a value that is not in the list -- `ROUND` - harmless. It only rounds values that are numbers, so a special missing is left exactly as it was typed +- `ROUND` - no effect. It only rounds values that are numbers, so a special missing is left as it was typed -The range rules compare in the order SAS itself uses, which is what makes them interesting here. SAS puts every missing below every non-missing value, and orders the missing values among themselves: `._` is the lowest, then the regular missing, then `.A` through `.Z`. So a range can be written in special missings and mean exactly what SAS would mean by it: +The range rules compare in the order SAS itself uses, which is what lets a range be written in special missings. SAS puts every missing below every non-missing value, and orders the missing values among themselves: `._` is the lowest, then the regular missing, then `.A` through `.Z`. A range therefore means exactly what SAS would mean by it: - `MINVAL .A` with `MAXVAL .C` accepts `.B` and rejects `.D`, and a blank - the regular missing - fails that floor because it sorts below `.A` - a number sits above every missing, so it passes a floor of `.A` and fails a ceiling of `.C` @@ -65,11 +65,11 @@ A formula is the one case where a special missing genuinely does not work: - `HARDFORMULA` / `SOFTFORMULA` - a formula that reads a special-missing cell does not compute; the grid's spreadsheet engine returns `#VALUE!` for that row, where the plain `.` contributes 0 -`CASE` (`UPCASE` / `LOWCASE`) is a character rule and does not come into it: SAS hands the browser a special missing as an uppercase letter, so there is no case to enforce, and a `CASE` rule on a numeric column would reject the column's numbers rather than the missing. +`CASE` (`UPCASE` / `LOWCASE`) is a character rule, so it does not apply: SAS hands the browser a special missing as an uppercase letter, so there is no case to enforce, and a `CASE` rule on a numeric column would reject the column's numbers rather than the missing. -One thing worth knowing about the regex rules: although the pattern is written in SAS PRX syntax, the check itself runs in the browser - SAS only parses the pattern (`PRXPARSE`) when the rule is saved - so the two engines can disagree on exotic patterns, and SAS pads a numeric-to-character conversion, so a pattern re-used in SAS needs `strip()` for an anchored match. +Worth knowing about the regex rules: although the pattern is written in SAS PRX syntax, the check itself runs in the browser - SAS only parses the pattern (`PRXPARSE`) when the rule is saved - so the two engines can disagree on exotic patterns, and SAS pads a numeric-to-character conversion, so a pattern re-used in SAS needs `strip()` for an anchored match. -In short, a special missing counts as a value for the pattern and dropdown rules, and as a value with its own place in the order for the range rules: it sits below every number, so a numeric minimum rejects it and a numeric maximum accepts it, while a range written in special missings is decided among the missing values themselves. +In short, a special missing counts as a value for the pattern and dropdown rules, and takes its own place in the order for the range rules: it sits below every number, so a numeric minimum rejects it and a numeric maximum accepts it, while a range written in special missings is decided among the missing values themselves. ## See it in action @@ -84,9 +84,9 @@ LinkedIn version of this post - publish it with the "Special Missings in Data Controller" video attached. Kept in sync with the copy above: same points, same claims, same order. -A SAS numeric missing is not a lone wolf. It is 28. +A SAS numeric missing is not a lone wolf. There are 28 of them. -The ordinary missing (.) is just the most common of them. The other 27 are single characters - the letters A to Z, or an underscore (._) - and they let you record WHY a value is missing: the question was never reached, the reading was illegible, the respondent refused. +The ordinary missing (.) is the best known of the 28. The other 27 are single characters - the letters A to Z, or an underscore (._) - and they let you record WHY a value is missing: the question was never reached, the reading was illegible, the respondent refused. They are real numeric values, not text: @@ -94,7 +94,7 @@ They are real numeric values, not text: - PROC MEANS excludes them, like any other missing - they sort below every number: ._ then . then .A to .Z - NMISS() and CMISS() count them as missing -- a regular missing prints as . unless options MISSING changes it - and that option never touches a special missing +- a regular missing prints as . unless OPTIONS MISSING= changes it - and that option never touches a special missing The hard part is every tool that is not SAS. JSON has no way to say "this number is a letter" - A is just a string. So a value that is perfectly legal in a SAS dataset quietly breaks in the browser, in Excel, in an API. @@ -102,22 +102,21 @@ We solved that in the open source SASjs Adapter, and it has been in Data Control - the adapter infers the column type from the values, so a column of numbers that also holds a lone letter is written to SAS as numeric, with the letter as a special missing - where the type cannot be inferred - a numeric column holding ONLY special missings - Data Controller passes the column format explicitly -- you type the letter or the underscore, with or without the period: a, .a, _, ._, and case does not matter +- you type the letter or the underscore, with or without the period (a, .a, _, ._), and case does not matter - one exception: a date, datetime or time formatted numeric column edits through a date picker, which takes only a date How they behave in Data Controller's validation rules: -- NOTNULL rejects one: a special missing is a missing value, so it fails the rule and a physical constraint. A primary key column is NOT NULL whether or not a rule is configured +- NOTNULL rejects one: a special missing is a missing value, so it fails the rule, and a physical NOT NULL constraint on the target table rejects it too. A primary key column is NOT NULL whether or not a rule is configured - HARDREGEX checks it against the pattern, unlike blanks and plain . - SOFTREGEX warns instead of blocking (and is ignored if the column also has a HARDREGEX) - SOFTSELECT and HARDSELECT both support them - the dropdown lists the missing as a bare letter, and a hard rule accepts it like any other listed value - ROUND leaves it alone, because it only rounds numbers - - a HARDFORMULA/SOFTFORMULA that reads a special-missing cell returns #VALUE! rather than a number The range rules compare in SAS's own order, so a range can be written in special missings: MINVAL .A with MAXVAL .C accepts .B and rejects .D. A missing sorts below every number, so it fails a numeric MINVAL and passes a numeric MAXVAL; a number sits above every missing. -In short: a value for the pattern and dropdown rules, and a value with its own place in the order for the range rules - below every number, so a numeric minimum rejects it and a numeric maximum accepts it. +In short: a value for the pattern and dropdown rules, and its own place in the order for the range rules - below every number, so a numeric minimum rejects it and a numeric maximum accepts it. The video shows the whole cycle - entering them, the rejections, submit, approve, and the DIFF, including a range written in special missings.