dc
4937edd7f4
The special missings material that was added to the v4.0 blog post belongs in its own post: that article is a release announcement for v4.0, not a teaching piece. This reverts the blog change and publishes the content under /feed/. The new post covers what SAS special missings are, how the SASjs Adapter carries them between the browser and SAS, and how Data Controller's validation rules treat them - NOTNULL passes, MINVAL fails, MAXVAL passes, and the regex rules apply as they would to any other value. The LinkedIn version of the copy, and the image prompt for the cover, are recorded in a comment at the foot of the file.
95 lines
7.0 KiB
Markdown
95 lines
7.0 KiB
Markdown
---
|
|
title: '28 Ways to Be Missing in SAS'
|
|
description: A SAS numeric missing is not a lone wolf - there are 28 of them, and Data Controller has handled all of them since v4. How they work, and what the validation rules do with them.
|
|
date: '2026-09-22 09:00:00'
|
|
author: 'Allan Bowe'
|
|
authorLink: https://www.linkedin.com/in/allanbowe/
|
|
tags:
|
|
- Special Missings
|
|
- Data Quality
|
|
---
|
|
|
|
# 28 Ways to Be Missing in SAS
|
|
|
|
A numeric missing value in SAS is not a lone wolf. The ordinary missing (`.`) is one of **28** distinct missing values available for a numeric variable - the other 27 are written with a single character, the letters `A` to `Z` or an underscore (`._`).
|
|
|
|
They exist because "missing" is usually not the whole story. A survey question that was never reached, a reading that was illegible, a value the respondent refused to give - in a well run process those are different facts, and a lone `.` throws the difference away. Special missings record *why* the value is missing.
|
|
|
|
## They are numbers, not text
|
|
|
|
- Arithmetic on them yields missing - `.A + 1` is `.`
|
|
- `proc means`, `proc summary` and friends exclude them, exactly as they exclude `.`
|
|
- They sort below every non-missing number, in the order `._`, then `.`, then `.A` to `.Z`
|
|
- The `NMISS()` and `CMISS()` functions count them as missing
|
|
- Converting one to text drops the period - `cats(.A)` is the string `A`
|
|
|
|
In a SAS dataset they are written with a leading period (`.A`, `.B` ... `._`). In Data Controller you type just the letter or the underscore - no period - and the letter is not case sensitive. Two letters, or a letter mixed with a number, are refused rather than guessed.
|
|
|
|
## Carrying them between the browser and SAS
|
|
|
|
The Data Controller frontend and the SAS backend exchange data as JSON, and JSON has no way to express a letter as a numeric value - `A` is a string. The conversion is handled in the open source [SASjs Adapter](https://github.com/sasjs/adapter#variable-types):
|
|
|
|
- The adapter infers each column's SAS type from the values it is given. All numeric values means numeric; all strings means character; and a column holding a single character (`a`-`z`, `_` or `.`) alongside numeric values is numeric, with the lone characters written to SAS as special missings.
|
|
- `null` becomes `.` or an empty string, according to the type derived for that column.
|
|
- Two cases cannot be inferred from the values alone: a numeric column containing *only* special missings looks like a single character column, and a character column containing only nulls looks numeric. For those, the adapter accepts an explicit format for the column - and Data Controller sends one automatically, because it already knows each column's SAS format from the metadata returned by the backend.
|
|
- A value that is neither a number nor a single valid character is refused rather than guessed (for example `aaaa`, or `!` in a numeric column), and a literal `.` is refused in favour of `null` for a regular missing.
|
|
|
|
There is nothing to configure. Special missings are available by default, for numeric cells.
|
|
|
|
Once in SAS they are ordinary values, so they are what the approval DIFF screen compares, and the DIFF's formatted / unformatted switch shows either the formatted representation or the raw value - useful for confirming exactly which missing was set.
|
|
|
|
## What the validation rules do with them
|
|
|
|
A special missing is a value somebody deliberately set, so the rules treat it as a value - it is not blank, and it is not exempt:
|
|
|
|
- `NOTNULL` - passes, because it is not null
|
|
- `MINVAL` - fails, because a special missing sorts below every number and so is below any minimum
|
|
- `MAXVAL` - passes, for the same reason it is below any maximum
|
|
- `CASE` (`UPCASE` / `LOWCASE`) - compared as text, so `A` passes `UPCASE` but `a` does not
|
|
- `HARDREGEX` / `SOFTREGEX` - checked against the pattern like any other value; unlike blanks and the plain `.`, special missings are **not** exempt, so a numeric column that carries them needs a pattern which allows for a single letter
|
|
|
|
That is usually what you want - a value recorded as "not collected" should not quietly satisfy a completeness or range rule. It does mean a range rule on a column that uses special missings will warn on those rows, which is the signal to either accommodate them in the rule or not use special missings on that column.
|
|
|
|
`video: [Managing Special Missing Values with Data Controller for SAS](https://www.youtube-nocookie.com/embed/ggrcNr23Jzw)`
|
|
|
|
The recording shows the full cycle: typing special missings into numeric cells, the rejections, submitting the changes, approving them, and reviewing the DIFF - including a change from one special missing to another, and the formatted / unformatted switch on date and datetime columns.
|
|
|
|
<!--
|
|
LinkedIn version of this post - publish it with the "Managing Special Missing
|
|
Values with Data Controller for SAS" video attached. Kept in sync with the copy
|
|
above: same points, same claims, same order.
|
|
|
|
A SAS numeric missing is not a lone wolf. It is 28.
|
|
|
|
The ordinary missing (.) is just the most common of them. The other 27 are single characters - the letters A to Z, or an underscore (._) - and they let you record WHY a value is missing: the question was never reached, the reading was illegible, the respondent refused.
|
|
|
|
They are real numeric values, not text:
|
|
|
|
- .A + 1 is .
|
|
- PROC MEANS excludes them, like any other missing
|
|
- they sort below every number: ._ then . then .A to .Z
|
|
- NMISS() and CMISS() count them as missing
|
|
|
|
The hard part is every tool that is not SAS. JSON has no way to say "this number is a letter" - A is just a string. So a value that is perfectly legal in a SAS dataset quietly breaks in the browser, in Excel, in an API.
|
|
|
|
We solved that in the open source SASjs Adapter, and it has been in Data Controller for SAS since v4:
|
|
|
|
- the adapter infers the column type from the values, so a column of numbers that also holds a lone letter is written to SAS as numeric, with the letter as a special missing
|
|
- where the type cannot be inferred - a numeric column holding ONLY special missings - Data Controller passes the column format explicitly
|
|
- you type the letter on its own: a or A, no period, and case does not matter
|
|
|
|
And they behave like values in DC's validation rules, which is usually what you want:
|
|
|
|
- NOTNULL passes - it is not null
|
|
- MINVAL fails - it sorts below every number; MAXVAL passes for the same reason
|
|
- HARDREGEX checks it against the pattern, unlike blanks and plain .
|
|
|
|
So a value recorded as "not collected" cannot quietly satisfy a completeness or range rule.
|
|
|
|
The video shows the whole cycle - entering them, the rejections, submit, approve, and the DIFF.
|
|
|
|
#sas #datacapture #mdm #dataquality
|
|
|
|
Image prompt: a pack of 28 wolves fanning out across a snowy plain at dusk, one wolf standing apart to the left, wide cinematic composition, cool blue-grey palette with a single warm amber accent on the lone wolf, soft depth of field, no text and no logos, 16:9, editorial cover art for a technical blog post about the 28 SAS missing values. Save the generated image as ./cover.jpeg and set previewImg: './cover.jpeg' in the front matter above.
|
|
-->
|