docs.datacontroller.io/docs/dcc-validations.md

---
layout: article
title: Data Validation
description: Quality in, Quality out! Enforce data quality checks at the point of SAS data entry, both directly via the web interface and also via Excel uploads.
og_image: https://i.imgur.com/P64ijBB.png
---


# Data Controller for SAS® - DQ Validations

## Overview
Quality in, Quality out!  Data Controller lets you enforce quality checks at the point of data entry, both directly via the web interface and also via Excel uploads.

## Default Checks
By default, the following frontend rules are always applied:

* Length checking per target table variable lengths
* Type checking per target table datatypes (Character, Numeric, Date, Time, Datetime)
* Not Null check per target table constraints
* Primary Key checking per business key defined in MPE_TABLES

It is possible to configure a number of other rules by updating the MPE_VALIDATIONS table.  Simply set the `BASE_LIB`, `BASE_DS` and `BASE_COL` values, and ensure `RULE_ACTIVE=1` for it to be applied.

## Configurable Checks

Check back frequently as we plan to keep growing this list of checks.

|Rule Type|Example Value |Description|
|---|---|---|
|CASE|UPCASE|Will enforce the case of cell values.  Valid values: UPCASE, LOWCASE, PROPCASE|
|NOTNULL|(defaultval)|Will prevent submission if null values are present.  Optional - provide a default value.|
|MINVAL|1|Defines a minimum value for a numeric cell|
|MAXVAL|1000000|Defines a maximum value for a numeric cell|
|READONLY|(defaultval) |Renders the column read-only in the editor.  The defaultval is used when rows are added. |
|HIDDEN|(defaultval) |Hides the column from the editor grid while still submitting its data. The defaultval is used when rows are added. |
|ROUND|2 |Rounds numeric input on paste/edit. Positive digits round to number of decimal places (eg 2 rounds to 0.01) Negative digits round to the nearest ten/hundred etc.  Half-away-from-zero rounding is applied so `-0.5 → -1` and `2.5 → 3`. |
|NUMBER_FORMAT|`{"style":"currency","currency":"GBP"}` |Display-only [`Intl.NumberFormat` renderer](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl/NumberFormat).  RULE_VALUE is the JSON options object passed straight to `Intl.NumberFormat`.  Does not change the stored value. |
|HARDREGEX|`^[A-Z]{3}$`|The cell value **must** match the regex pattern, otherwise submission is blocked and the cell is highlighted red.  See [Regex Rules](#regex-rules) below.|
|SOFTREGEX|`^[A-Z]{3}$`|A cell value that does not match the regex pattern is highlighted yellow as a warning, but submission is **not** blocked.  See [Regex Rules](#regex-rules) below.|
|HARDSELECT|sashelp.class.name|A distinct list of values (max 1000) are taken from this library.member.column reference, and the value **must** be in this list.  This list may be supplemented by entries in the MPE_SELECTBOX table.|
|SOFTSELECT|dcdemo.mpe_tables.libref|A distinct list of values (max 1000) are taken from this library.member.column reference, and the user-provided value may (or may not) be in this list.  This list may be supplemented by entries in the MPE_SELECTBOX table.|
|[HARDSELECT_HOOK](/dynamic-cell-dropdown)|/logical/folder/stpname|A SAS service (STP or Viya Job) or a path to a SAS program on the filesystem.  User provided values **must** be in this list.  Cannot be used alongside a SOFTSELECT_HOOK.|
|[SOFTSELECT_HOOK](/dynamic-cell-dropdown)|/physical/path/program.sas|A SAS service (STP or Viya Job) or a path to a SAS program on the filesystem.  User-provided values may (or may not) be in this list.  Cannot be used alongside a HARDSELECT_HOOK.|


## Regex Rules

HARDREGEX and SOFTREGEX validate cell values against a SAS (Perl-style) regular expression provided in `RULE_VALUE` - the same syntax accepted by [PRXPARSE](https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/lefunctionsref/p0s9ilagexmjl8n1u7e1t1jfnzlk.htm).

Things to be aware of:

* Patterns are validated with `PRXPARSE` when the rule is saved - a post-edit check on MPE_VALIDATIONS itself will reject invalid patterns and list the offending columns.
* The pattern is evaluated in the browser using the JavaScript regex engine, which shares the same core syntax (character classes, quantifiers, groups, alternation, `^`/`$` anchors, `\d \w \s` etc).  Stick to that common subset: Perl-only constructs such as inline modifiers `(?i)`, `\A` / `\z` anchors, possessive quantifiers (`a++`) and atomic groups (`(?>...)`) will pass the SAS-side PRXPARSE check but fail (and be silently ignored) in the frontend.
* The pattern is used **as authored** - it is not auto-anchored.  If you want to match the entire cell value, include `^` and `$` yourself (eg `^[A-Z]{3}$`).
* Blank values are exempt from pattern matching on any column type - use the NOTNULL rule if you also need to enforce populated values.  On numeric columns the plain SAS missing (`.`) is also exempt.  Special missings (`.A`-`.Z`, `._`) are **not** exempt - being deliberately-set values, they are validated against the pattern like any other value, so on numeric columns make sure your pattern accommodates them (or avoid special missings).  On character columns there is no missing-value concept: even `.` is treated as real text.
* Only one regex rule is ever applied per column.  If a column has both a HARDREGEX and a SOFTREGEX rule, the SOFTREGEX rule is ignored entirely - even for values that pass the HARDREGEX - so a dual-rule column behaves exactly like a HARDREGEX-only column.  For the same reason, the column-header info dropdown shows only the rule that is applied (the HARDREGEX pattern when both exist).
* Cells in rows that are marked for deletion are not validated / warned (except primary key columns, which still validate).
* In the unlikely event a pattern that fails in the browser slips through (see above), the frontend treats it as always-valid (no blocking, no warning) rather than breaking the editor.
* `RULE_VALUE` is limited to 128 characters, which constrains very long patterns.

## Dropdowns

There are now actually FIVE places where you can configure dropdowns!

1.  The [MPE_SELECTBOX](/dcc-selectbox/) table
2.  The HARDSELECT validation (library.member.column reference)
3.  The SOFTSELECT validation (library.member.column reference)
4.  The HARDSELECT_HOOK validation (SAS Program)
5.  The SOFTSELECT_HOOK validation (SAS Program)

How do these inter-operate?

Well - if you have values in MPE_SELECTBOX and/or HARDSELECT / SOFTSELECT tables, they will be merged together, and served in ADDITION to the values provided by any HOOK program.

Dropdowns are SOFT by default, unless a HARD rule is present.

Data Controller will not let you submit both a HARDSELECT_HOOK and a SOFTSELECT_HOOK on the same variable.