dc
42ffc63009
A derived variable can now be built on one with a lower VARIABLE_ORDER, so the pages describe the order column, the ordered rule (replacing the self-contained one) and a chained example.
87 lines
5.4 KiB
Markdown
87 lines
5.4 KiB
Markdown
---
|
|
layout: article
|
|
title: Derived Variables
|
|
description: Derived variables are computed columns that appear in the EDIT screen and can be filtered on, without being stored in the target table.
|
|
---
|
|
|
|
# Derived Variables
|
|
|
|
A derived variable is a column that does not exist in the target table, but is computed when the table is loaded into the [EDIT screen](/dc-userguide/#editor). It is read-only, it appears after the physical columns, and it can be filtered on like any other column.
|
|
|
|
Derived variables are useful when users need to work with a value that is a function of the stored data - a banding, a concatenation, a calculation - without changing the table, the load process, or the ETL.
|
|
|
|
The values are **not stored**. They are computed on every read, and nothing derived is written back when a change is submitted or approved.
|
|
|
|
## Configuration
|
|
|
|
Derived variables are configured per table in the [MPE_DERIVED](/tables/mpe_derived/) control table, which is itself a Data Controller table - so it is maintained through the EDIT screen with the usual review and approve workflow.
|
|
|
|
Each row defines one variable:
|
|
|
|
- `LIBREF` / `DSN` - the target table
|
|
- `VARIABLE_NM` - the name of the new column
|
|
- `VARIABLE_ORDER` - evaluation order; a variable may reference one with a lower order
|
|
- `VARIABLE_TYPE` - `C` for character, `N` for numeric
|
|
- `VARIABLE_LENGTH` - character length, or numeric length (1-8)
|
|
- `VARIABLE_FORMAT` - optional, eg `DATE9.` or `DOLLAR12.2`
|
|
- `VARIABLE_LOGIC` - the SAS expression that produces the value
|
|
- `ACTIVE` - set to 1 to apply the variable
|
|
|
|
A simple example, deriving a region from a state:
|
|
|
|
```
|
|
VARIABLE_NM REGION
|
|
VARIABLE_ORDER 10
|
|
VARIABLE_TYPE C
|
|
VARIABLE_LENGTH 8
|
|
VARIABLE_FORMAT (blank)
|
|
VARIABLE_LOGIC ifc(STATE in ('CA','OR','WA'),'West','East')
|
|
ACTIVE 1
|
|
```
|
|
|
|
A later variable can build on it, provided its order is higher:
|
|
|
|
```
|
|
VARIABLE_NM REGION_LABEL
|
|
VARIABLE_ORDER 20
|
|
VARIABLE_TYPE C
|
|
VARIABLE_LENGTH 14
|
|
VARIABLE_FORMAT (blank)
|
|
VARIABLE_LOGIC cats(REGION,'-',STATE)
|
|
ACTIVE 1
|
|
```
|
|
|
|
For each row, Data Controller generates the following statements when the table is read:
|
|
|
|
```sas
|
|
attrib REGION length=$8 label='DERIVED: ifc(STATE in (''CA'',''OR'',''WA''),''West'',''East'')' format=$8.;
|
|
REGION = ifc(STATE in ('CA','OR','WA'),'West','East');
|
|
```
|
|
|
|
The label defaults to the expression itself, prefixed with `DERIVED: `, so that computed columns are obvious when labels are switched on. The expression is truncated to the maximum label length.
|
|
|
|
The type and length are declared, not inferred, so a value is truncated exactly as you specify. If `VARIABLE_FORMAT` is left blank, character variables default to `$<length>.` and numeric variables to `<length>.`; a date, time or datetime format makes the column render (and filter) as a date, time or datetime, the same way a stored column does.
|
|
|
|
## Filtering
|
|
|
|
The derived variable is created before the filter is applied, so it can be used in the [filter](/filter/) like any other column - including its list of values. A filter on a derived column is stored and re-applied in the normal way, and the value list for a derived column is built from the table under the rest of the filter, exactly as for a stored column.
|
|
|
|
Derived columns themselves are an EDIT screen feature: the VIEW screen and the download services do not show them. A saved filter that references a derived column still applies there, and is evaluated the same way.
|
|
|
|
## Rules and limits
|
|
|
|
- **Expressions only.** `VARIABLE_LOGIC` must be a single SAS expression - no semicolons, macro triggers (`%`, `&`) or comments. Conditional logic is available through functions such as `ifc()`, `choosec()`, `cats()`, `catx()`, `put()`, `input()` and `intnx()`.
|
|
- **Ordered.** Statements are generated in `VARIABLE_ORDER` sequence. An expression may reference the physical columns of the table, and any derived variable with a lower `VARIABLE_ORDER`. A reference to a variable that is not defined earlier is rejected, because SAS would otherwise return a missing value without an error. `VARIABLE_ORDER` is required, and must be a positive integer.
|
|
- **No collisions.** `VARIABLE_NM` must not match an existing column of the target table, or one of the reserved names (`TX_FROM`, `TX_TO`, `PROCESSED_DTTM`, `VAR_TXFROM`, `VAR_TXTO`, `VAR_PROCESSED`, `_____DELETE__THIS__RECORD_____`).
|
|
- **Read-only.** Derived columns cannot be edited in the grid, and any value submitted for them is discarded before the approval package is created.
|
|
- **Recomputed on every read.** The value always reflects the current data and the current definition, and there is no history of previous derived values.
|
|
|
|
A configuration error - a bad expression, an unknown column, a name collision - stops the EDIT screen with a message naming the variable, rather than silently dropping the column.
|
|
|
|
!!! warning "Derived values can expose hidden data"
|
|
An expression can reference a column that is hidden from the user by [Column Level Security](/column-level-security/), and the derived column will show the result. Configure derived variables with the same care as any other data exposure.
|
|
|
|
## Performance
|
|
|
|
The expression is evaluated for each row read, before the filter. The filter already requires a scan of the table, so this adds per-row arithmetic rather than a second pass - but keep expressions cheap (no lookups, no subqueries), and expect the cost to be paid on every load of the EDIT screen, including each request for filter values.
|