LIBREF, DSN, VAR_NM, VAR_TYPE, VAR_LEN, VAR_LOGIC and ACTIVE are NOT NULL in the
table definition, so the requirement is stated once, in the model, instead of
being repeated as NOTNULL rules in MPE_VALIDATIONS for each column.
VAR_FMT stays nullable: it is optional on a derived variable, and its HARDREGEX
already accepts an empty value.
The seven NOTNULL rule inserts are removed from the 7.19 migration, and the
comment on ACTIVE no longer describes a rule that is not there.
- mpe_derived_postedit ran %local at the top level of a file that is included
at execution time, so the hook stopped before validating anything and every
MPE_DERIVED submission failed. The macro language now lives in a macro
definition, errflag and errmsg are declared global, and the file calls it.
- the cycle check aborts after leaving its loop, and the self, undeclared and
field aborts are driven by a count at the point of call rather than by a
wrapping %if, so an abort cannot be outlived. The same shapes are fixed
upstream in sasjs/core#468.
- a case-variant duplicate survived mpe_loader, which compares raw values:
the hook now rejects it after normalisation and names the variable.
- stagedata dropped the derived columns from the output of the security step
rather than from its input, so a crafted submission could still satisfy an
RLS clause through a derived column. work.jsdata itself is rewritten now.
- the VAR_FMT validation rule rejected the bare w.d forms that the hook and
the engine accept.
- the migration carried the same four dangling insert headers as mpe_makedata,
so the v7.19 migration failed for existing installs.
- the filter fixtures in getdata.test and validatefilter.test had been caught
by the variable rename: the filter schema column is VARIABLE_NM.
- the TX_FROM/TX_TO definitions agree across the ddl, the migration and
mpe_makedatamodel.
@sasjs/core bumped to 5.7.1, which carries the where= fix.
A configured label replaces the default NAME=expression; when it is missing or
blank the default is used, so existing rows are unaffected.
- MPE_DERIVED gains VAR_LABEL varchar(256), in the ddl, mpe_makedatamodel and
the v7.19 migration
- mpe_derived uses it, through the same escaping and 256-char truncation as
the default label
- the hook trims it; the mock mirrors the fallback
- test 1 now sets a label on BAND and asserts it
A sequence number has to be maintained by the administrator: inserting a
variable means renumbering everything after it, and a wrong number fails
silently - the derived value comes back missing, not wrong.
VAR_DEPENDENCIES names every input the expression uses, physical columns and
derived variables alike, eg 'REGION STATE'. The statements are emitted in
dependency order, so a variable can build on another one with no sequence to
maintain, and a cycle is rejected.
The declaration is checked against the expression in both directions: a
derived variable referenced by VAR_LOGIC must be declared, and a declared
name that the expression does not reference is an error; a declared name must
resolve to a column of the target table or to a derived variable of the same
table.
Changes:
- MPE_DERIVED: VAR_ORDER (num) becomes VAR_DEPENDENCIES (varchar(4000)), in
the ddl, mpe_makedatamodel and the v7.19 migration
- mpe_derived: dependency validation and a topological sort replace the
VAR_ORDER sort and the forward-reference check
- mpe_derived_postedit: checks self reference, undeclared derived references
and cycles at submit time, across the incoming rows plus the current rows
they do not replace
- MPE_VALIDATIONS: the VAR_ORDER MINVAL/MAXVAL rules are replaced by UPCASE
and a character pattern on VAR_DEPENDENCIES
- mocks: the dependency order is applied in JS, so a chained variable works
even when the rows are loaded child first
- docs and tests updated
MPE_DERIVED columns:
VARIABLE_NM -> VAR_NM
VARIABLE_ORDER -> VAR_ORDER
VARIABLE_TYPE -> VAR_TYPE
VARIABLE_LENGTH -> VAR_LEN
VARIABLE_FORMAT -> VAR_FMT
VARIABLE_LOGIC -> VAR_LOGIC
Updated in the ddl, mpe_makedatamodel, the migration, the MPE_TABLES buskey,
the macro (reads and validation messages), the tests, the mocks and the docs.
A derived variable may need to reference one defined before it (a label that
concatenates a derived region, for example), so the order has to be
configured rather than assumed.
- MPE_DERIVED: new VARIABLE_ORDER column (ddl, makedatamodel, migration).
- mpe_derived: statements are generated in VARIABLE_ORDER then name sequence.
VARIABLE_ORDER is required and must be a positive integer. A reference to a
derived variable with an equal or higher order is rejected, because SAS
would otherwise treat it as uninitialised and return a missing value with no
error; references to lower-order variables are allowed.
- mock: same ordering, each variable compiled against the physical columns
plus those defined earlier, so the frontend can exercise a chained variable.
- makedata: VARIABLE_ORDER on both demo rows plus a chained REGION_LABEL.
- cypress: asserts the chained value in the grid.
- tests: the getdata test seeds a second variable built on the first.
- docs: VARIABLE_ORDER documented, with the ordered rule replacing the
self-contained one.
Adds user-defined computed variables, configured per table in the new
MPE_DERIVED control table. A derived variable is declared with a name,
type, length, optional format and a SAS expression.
The expression is applied in a DATA step view, and the filter is applied
by the step that reads the view, so the variables exist before the
filter is evaluated and can therefore be filtered on. A WHERE clause
cannot see a variable created in the same data step, hence the view.
- getdata: derived columns are added to the COLS payload (read only,
labelled DERIVED: <expression>) and to the data returned to the grid
- getcolvals: values are taken from the derived view, so filter dropdowns
work for derived columns
- validatefilter: the filter is validated against the view, and stored
against the physical table (needs mp_filterstore targetds=)
- stagedata: derived columns are dropped before the approval package
- mpe_makedata / mpe_makedatamodel / ddl / migration: new MPE_DERIVED
table, registered in MPE_TABLES
Values are display-time only - nothing derived is ever written back.
dc