docs.datacontroller.io/docs/dcc-validations.md

layout, title, description, og_image

layout	title	description	og_image
article	Data Validation	Quality in, Quality out! Enforce data quality checks at the point of SAS data entry, both directly via the web interface and also via Excel uploads.	https://i.imgur.com/P64ijBB.png

Data Controller for SAS® - DQ Validations

Overview

Quality in, Quality out! Data Controller lets you enforce quality checks at the point of data entry, both directly via the web interface and also via Excel uploads.

Default Checks

By default, the following frontend rules are always applied:

• Length checking per target table variable lengths
• Type checking per target table datatypes (Character, Numeric, Date, Time, Datetime)
• Not Null check per target table constraints
• Primary Key checking per business key defined in MPE_TABLES

It is possible to configure a number of other rules by updating the MPE_VALIDATIONS table. Simply set the BASE_LIB, BASE_DS and BASE_COL values, and ensure RULE_ACTIVE=1 for it to be applied.

Configurable Checks

Check back frequently as we plan to keep growing this list of checks.

Rule Type	Example Value	Description
CASE	UPCASE	Will enforce the case of cell values. Valid values: UPCASE, LOWCASE, PROPCASE
NOTNULL	(defaultval)	Will prevent submission if null values are present. Optional - provide a default value.
MINVAL	1	Defines a minimum value for a numeric cell
MAXVAL	1000000	Defines a maximum value for a numeric cell
READONLY	(defaultval)	Renders the column read-only in the editor. The defaultval is used when rows are added.
HIDDEN	(defaultval)	Hides the column from the editor grid while still submitting its data. The defaultval is used when rows are added.
ROUND	2	Rounds numeric input on paste/edit. Positive digits round to number of decimal places (eg 2 rounds to 0.01) Negative digits round to the nearest ten/hundred etc. Half-away-from-zero rounding is applied so -0.5 → -1 and 2.5 → 3.
NUMBER_FORMAT	{"style":"currency","currency":"GBP"}	Display-only Intl.NumberFormat renderer. RULE_VALUE is the JSON options object passed straight to Intl.NumberFormat. Does not change the stored value.
HARDREGEX	^[A-Z]{3}$	The cell value must match the regex pattern, otherwise submission is blocked and the cell is highlighted red. See Regex Rules below.
SOFTREGEX	^[A-Z]{3}$	A cell value that does not match the regex pattern is highlighted yellow as a warning, but submission is not blocked. See Regex Rules below.
HARDSELECT	sashelp.class.name	A distinct list of values (max 1000) are taken from this library.member.column reference, and the value must be in this list. This list may be supplemented by entries in the MPE_SELECTBOX table.
SOFTSELECT	dcdemo.mpe_tables.libref	A distinct list of values (max 1000) are taken from this library.member.column reference, and the user-provided value may (or may not) be in this list. This list may be supplemented by entries in the MPE_SELECTBOX table.
HARDSELECT_HOOK	/logical/folder/stpname	A SAS service (STP or Viya Job) or a path to a SAS program on the filesystem. User provided values must be in this list. Cannot be used alongside a SOFTSELECT_HOOK.
SOFTSELECT_HOOK	/physical/path/program.sas	A SAS service (STP or Viya Job) or a path to a SAS program on the filesystem. User-provided values may (or may not) be in this list. Cannot be used alongside a HARDSELECT_HOOK.

Regex Rules

HARDREGEX and SOFTREGEX validate cell values against a SAS (Perl-style) regular expression provided in RULE_VALUE - the same syntax accepted by PRXPARSE.

Things to be aware of:

• Patterns are validated with PRXPARSE when the rule is saved - a post-edit check on MPE_VALIDATIONS itself will reject invalid patterns and list the offending columns.
• The pattern is evaluated in the browser using the JavaScript regex engine, which shares the same core syntax (character classes, quantifiers, groups, alternation, ^/$ anchors, \d \w \s etc). Stick to that common subset: Perl-only constructs such as inline modifiers (?i), \A / \z anchors, possessive quantifiers (a++) and atomic groups ((?>...)) will pass the SAS-side PRXPARSE check but fail (and be silently ignored) in the frontend.
• The pattern is used as authored - it is not auto-anchored. If you want to match the entire cell value, include ^ and $ yourself (eg ^[A-Z]{3}$).
• Blank values are exempt from pattern matching on any column type - use the NOTNULL rule if you also need to enforce populated values. On numeric columns the plain SAS missing (.) is also exempt. Special missings (.A-.Z, ._) are not exempt - being deliberately-set values, they are validated against the pattern like any other value, so on numeric columns make sure your pattern accommodates them (or avoid special missings). On character columns there is no missing-value concept: even . is treated as real text.
• Only one regex rule is ever applied per column. If a column has both a HARDREGEX and a SOFTREGEX rule, the SOFTREGEX rule is ignored entirely - even for values that pass the HARDREGEX - so a dual-rule column behaves exactly like a HARDREGEX-only column. For the same reason, the column-header info dropdown shows only the rule that is applied (the HARDREGEX pattern when both exist).
• Cells in rows that are marked for deletion are not validated / warned (except primary key columns, which still validate).
• In the unlikely event a pattern that fails in the browser slips through (see above), the frontend treats it as always-valid (no blocking, no warning) rather than breaking the editor.
• RULE_VALUE is limited to 128 characters, which constrains very long patterns.

Dropdowns

There are now actually FIVE places where you can configure dropdowns!

1. The MPE_SELECTBOX table
2. The HARDSELECT validation (library.member.column reference)
3. The SOFTSELECT validation (library.member.column reference)
4. The HARDSELECT_HOOK validation (SAS Program)
5. The SOFTSELECT_HOOK validation (SAS Program)

How do these inter-operate?

Well - if you have values in MPE_SELECTBOX and/or HARDSELECT / SOFTSELECT tables, they will be merged together, and served in ADDITION to the values provided by any HOOK program.

Dropdowns are SOFT by default, unless a HARD rule is present.

Data Controller will not let you submit both a HARDSELECT_HOOK and a SOFTSELECT_HOOK on the same variable.