docs(config): derived variables in the EDIT screen
New Configuration page explaining derived (computed) variables: what they are, how to configure them in MPE_DERIVED, how filtering works, the rules and limits (expressions only, self-contained, read-only, not stored), and the data-exposure and performance notes. New MPE_DERIVED table page, registered in the Table Guide nav.
This commit is contained in:
dc committed
1 parent
b7794cef47
commit
183037c4d3
3 files changed
+96
No files matched your search
@@ -0,0 +1,70 @@
|
||||
---
|
||||
layout: article
|
||||
title: Derived Variables
|
||||
description: Derived variables are computed columns that appear in the EDIT screen and can be filtered on, without being stored in the target table.
|
||||
---
|
||||
|
||||
# Derived Variables
|
||||
|
||||
A derived variable is a column that does not exist in the target table, but is computed when the table is loaded into the [EDIT screen](/dc-userguide/#edit). It is read-only, it appears after the physical columns, and it can be filtered on like any other column.
|
||||
|
||||
Derived variables are useful when users need to work with a value that is a function of the stored data - a banding, a concatenation, a calculation - without changing the table, the load process, or the ETL.
|
||||
|
||||
The values are **not stored**. They are computed on every read, and nothing derived is written back when a change is submitted or approved.
|
||||
|
||||
## Configuration
|
||||
|
||||
Derived variables are configured per table in the [MPE_DERIVED](/tables/mpe_derived/) control table, which is itself a Data Controller table - so it is maintained through the EDIT screen with the usual review and approve workflow.
|
||||
|
||||
Each row defines one variable:
|
||||
|
||||
- `LIBREF` / `DSN` - the target table
|
||||
- `VARIABLE_NM` - the name of the new column
|
||||
- `VARIABLE_TYPE` - `C` for character, `N` for numeric
|
||||
- `VARIABLE_LENGTH` - character length, or numeric length (1-8)
|
||||
- `VARIABLE_FORMAT` - optional, eg `DATE9.` or `DOLLAR12.2`
|
||||
- `VARIABLE_LOGIC` - the SAS expression that produces the value
|
||||
- `ACTIVE` - set to 1 to apply the variable
|
||||
|
||||
A simple example, deriving a region from a state:
|
||||
|
||||
```
|
||||
VARIABLE_NM REGION
|
||||
VARIABLE_TYPE C
|
||||
VARIABLE_LENGTH 8
|
||||
VARIABLE_FORMAT (blank)
|
||||
VARIABLE_LOGIC ifc(STATE in ('CA','OR','WA'),'West','East')
|
||||
ACTIVE 1
|
||||
```
|
||||
|
||||
For each row, Data Controller generates the following statements when the table is read:
|
||||
|
||||
```sas
|
||||
attrib REGION length=$8 label='DERIVED: ifc(STATE in (''CA'',''OR'',''WA''),''West'',''East'')' format=$8.;
|
||||
REGION = ifc(STATE in ('CA','OR','WA'),'West','East');
|
||||
```
|
||||
|
||||
The label defaults to the expression itself, prefixed with `DERIVED: `, so that computed columns are obvious when labels are switched on. The expression is truncated to the maximum label length.
|
||||
|
||||
The type and length are declared, not inferred, so a value is truncated exactly as you specify. If `VARIABLE_FORMAT` is left blank, character variables default to `$<length>.` and numeric variables to `<length>.`; a date, time or datetime format makes the column render (and filter) as a date, time or datetime, the same way a stored column does.
|
||||
|
||||
## Filtering
|
||||
|
||||
The derived variable is created before the filter is applied, so it can be used in the [filter](/filter/) like any other column - including its list of values. A filter on a derived column is stored and re-applied in the normal way, and the value list for a derived column is built from the table under the rest of the filter, exactly as for a stored column.
|
||||
|
||||
## Rules and limits
|
||||
|
||||
- **Expressions only.** `VARIABLE_LOGIC` must be a single SAS expression - no semicolons, macro triggers (`%`, `&`) or comments. Conditional logic is available through functions such as `ifc()`, `choosec()`, `cats()`, `catx()`, `put()`, `input()` and `intnx()`.
|
||||
- **Self-contained.** An expression may reference the columns of the target table only, not another derived variable. Derived variables are evaluated independently of each other.
|
||||
- **No collisions.** `VARIABLE_NM` must not match an existing column of the target table, or one of the reserved names (`TX_FROM`, `TX_TO`, `PROCESSED_DTTM`, `_____DELETE__THIS__RECORD_____`).
|
||||
- **Read-only.** Derived columns cannot be edited in the grid, and any value submitted for them is discarded before the approval package is created.
|
||||
- **Recomputed on every read.** The value always reflects the current data and the current definition, and there is no history of previous derived values.
|
||||
|
||||
A configuration error - a bad expression, an unknown column, a name collision - stops the EDIT screen with a message naming the variable, rather than silently dropping the column.
|
||||
|
||||
!!! warning "Derived values can expose hidden data"
|
||||
An expression can reference a column that is hidden from the user by [Column Level Security](/column-level-security/), and the derived column will show the result. Configure derived variables with the same care as any other data exposure.
|
||||
|
||||
## Performance
|
||||
|
||||
The expression is evaluated for each row read, before the filter. The filter already requires a scan of the table, so this adds per-row arithmetic rather than a second pass - but keep expressions cheap (no lookups, no subqueries), and expect the cost to be paid on every load of the EDIT screen, including each request for filter values.
|
||||
@@ -0,0 +1,24 @@
|
||||
---
|
||||
layout: article
|
||||
title: MPE_DERIVED
|
||||
description: The MPE_DERIVED table is used to configure computed columns that appear in the EDIT screen and can be filtered on without being stored in the table.
|
||||
---
|
||||
|
||||
# MPE_DERIVED
|
||||
|
||||
The MPE_DERIVED table is used to configure derived (computed) variables - columns that are created when a table is loaded into the EDIT screen, so that they can be displayed and filtered on without being stored. More details are available in the [user guide](/derived-variables/)
|
||||
|
||||
The table is itself registered in [MPE_TABLES](/tables/mpe_tables/), so it is maintained through the EDIT screen with the usual review and approve workflow.
|
||||
|
||||
## Columns
|
||||
|
||||
- `TX_FROM num`: SCD2 open datetime
|
||||
- 🔑 `TX_TO num`: SCD2 close datetime
|
||||
- 🔑 `LIBREF char(8)`: The libref of the target table
|
||||
- 🔑 `DSN char(32)`: The target table
|
||||
- 🔑 `VARIABLE_NM char(32)`: The name of the derived column
|
||||
- `VARIABLE_TYPE char(1)`: `C` for character, `N` for numeric
|
||||
- `VARIABLE_LENGTH num`: Character length, or numeric length (1-8)
|
||||
- `VARIABLE_FORMAT char(32)`: Optional SAS format, eg `DATE9.`
|
||||
- `VARIABLE_LOGIC char(4000)`: The SAS expression that produces the value
|
||||
- `ACTIVE num`: Set to 1 to apply the variable
|
||||
@@ -32,6 +32,7 @@ nav:
|
||||
- MPE_DATASTATUS_TABS: tables/mpe_datastatus_tabs.md
|
||||
- MPE_DATADICTIONARY: tables/mpe_datadictionary.md
|
||||
- MPE_DATALOADS: tables/mpe_dataloads.md
|
||||
- MPE_DERIVED: tables/mpe_derived.md
|
||||
- MPE_EMAILS: tables/mpe_emails.md
|
||||
- MPE_EXCEL_CONFIG: tables/mpe_excel_config.md
|
||||
- MPE_FILTERANYTABLE: tables/mpe_filteranytable.md
|
||||
@@ -58,6 +59,7 @@ nav:
|
||||
- CAS Tables: cas-tables.md
|
||||
- Column Level Security: column-level-security.md
|
||||
- Dates / Datetimes: dcc-dates.md
|
||||
- Derived Variables: derived-variables.md
|
||||
- Dynamic Cell Dropdown: dynamic-cell-dropdown.md
|
||||
- Emails: emails.md
|
||||
- Excel: excel.md
|
||||
|
||||
Reference in new issue
Block a user