dc/.agent/docs/row-level-security.md

Row Level Security — Technical Deep Dive

This document explains how Row Level Security (RLS) is implemented in the Data Controller backend. For the user-facing configuration guide, see docs.datacontroller.io/row-level-security.

Overview

RLS in Data Controller is implemented as server-side WHERE clause generation. No data leaves SAS without passing through a dynamically generated filter. The filter is built at runtime per request, based on:

1. The requesting user's group memberships (SAS metadata groups + DC groups)
2. The active rules in the MPE_ROW_LEVEL_SECURITY control table
3. The access mode (VIEW / EDIT / download / upload)

Because the filter is expressed as a standard SAS WHERE expression, it works against any engine — Base SAS datasets, database libraries (via implicit SQL pass-through pushdown), SPDE, CAS libnames, etc.

Request Flow

flowchart TD
    A[Client request\nview / edit / download / upload] --> B{Service mode}
    B -->|VIEW / EDIT / DLOAD| C["%mpe_filtermaster(mode, libds)"]
    B -->|ULOAD stagedata.sas| C
    C --> D["%mpe_getgroups()\nmetadata groups + MPE_GROUPS"]
    D --> E{User in &mpeadmins?}
    E -->|Yes| F[No RLS filter\n1=1]
    E -->|No| G[Lookup active rules in\nMPE_ROW_LEVEL_SECURITY\nfor libref.table + user's groups]
    G --> H{Rules found?}
    H -->|No| F
    H -->|Yes| I["%mp_filtergenerate()\nper group, OR'd together"]
    I --> J[WHERE clause written\nto temp fileref]
    F --> J
    J --> K{Read or Write?}
    K -->|Read\nviewdata / getdata / getrawdata| L["where %inc filtref;;\nrows filtered server-side"]
    K -->|Write\nstagedata| M[Inverse filter\nwhere not( filtref )]
    M --> N{badrecords > 0?}
    N -->|Yes| O[Abort submission\nSecurity Problem]
    N -->|No| P[Staging proceeds\nto approval workflow]

Key Components

Component	Location	Role
MPE_ROW_LEVEL_SECURITY	&mpelib (DC control library)	The rule table (scope, group, libref, table, logic, subgroup, variable, operator, raw value, active flag)
%mpe_filtermaster()	sas/sasjs/macros/mpe_filtermaster.sas	Master macro that assembles the full WHERE clause for a request
%mpe_getgroups()	sas/sasjs/macros/mpe_getgroups.sas	Resolves group membership (metadata groups via %dc_getusergroups + MPE_GROUPS DC-internal groups)
%mp_filtergenerate()	SASjs core	Converts a query table (logic/subgroup/variable/operator/value rows) into WHERE clause text
%mp_filtercheck()	SASjs core	Validates rule syntax at edit time (defence against SAS code injection)
mpe_row_level_security_postedit.sas	sas/sasjs/services/hooks/	Post-edit hook that runs %mp_filtercheck whenever the RLS table itself is edited

The Modes

%mpe_filtermaster accepts a mode parameter, and every service that surfaces data calls it with the appropriate mode:

Mode	Caller	Purpose
VIEW	services/public/viewdata.sas, getchangeinfo.sas	Read-only table viewer
EDIT	services/editors/getdata.sas	The EDIT grid (adds "current records only" validity filtering)
DLOAD	services/public/getrawdata.sas	Raw file downloads (RLS scope treated as VIEW)
ULOAD	services/editors/stagedata.sas	Upload validation (RLS scope treated as EDIT)

Scope mapping: DLOAD requests match rules with RLS_SCOPE in ('VIEW','ALL'); ULOAD requests match rules with RLS_SCOPE in ('EDIT','ALL').

Execution Flow of %mpe_filtermaster

The macro writes the final WHERE expression to a temporary fileref (outref), line by line. (A fileref is used because a generated filter may exceed the 64k macro variable limit — and note that %include of a fileref is not allowed directly in a proc sql where clause, hence callers typically use it in a DATA step or data step view.)

1. User-supplied filter (FILTER_RK)

If the request includes a stored filter (filter_rk > 0), its clauses are read from MPE_FILTERANYTABLE / MPE_FILTERSOURCE and generated first via %mp_filtergenerate. RLS clauses are then AND-ed on top — a user filter can only ever narrow results, never widen them beyond RLS.

2. Validity-date filtering (EDIT / DLOAD only)

MPE_TABLES may define VAR_TXFROM / VAR_TXTO (SCD2-style validity variables) for the target table. Unless the user explicitly filtered on those variables, the macro appends:

("<current datetime>"dt < VAR_TXTO)

so that only current records are surfaced in the EDIT grid and downloads.

3. Group resolution and admin bypass

%mpe_getgroups(user=%mf_getuser(), outds=work.groups)

Groups come from two sources, concatenated:

• SAS metadata groups (Viya / EBI / Base-specific logic in %dc_getusergroups)
• The MPE_GROUPS DC table (group assignments managed inside Data Controller)

If the user is a member of the &mpeadmins group, RLS is skipped entirely — admins always see all rows.

4. Rule extraction

Non-admin users trigger a lookup of active, current rules:

create table work.&rlsds as
  select rls_group, rls_group_logic, rls_subgroup_logic, rls_subgroup_id,
         rls_variable_nm, rls_operator_nm, rls_raw_value
  from &mpelib..mpe_row_level_security
  where &dc_dttmtfmt. lt tx_to          /* only current (non-deleted) rules */
    and rls_scope in ("&scopeval",'ALL')
    and upcase(rls_group) in (select upcase(groupname) from work.groups)
    and rls_libref = "<libref>" and rls_table = "<dsname>"
    and rls_active = 1
  order by rls_group, rls_subgroup_id;

5. Clause assembly

If rules exist, they are appended to the fileref as AND ( ... ). Each group the user belongs to contributes one sub-filter, and the group-level sub-filters are joined with OR:

AND ( <group 1 filter> OR <group 2 filter> ... )

Each group's filter is produced by %mp_filtergenerate, which honours:

• RLS_GROUP_LOGIC — how subgroups (identified by RLS_SUBGROUP_ID) are joined
• RLS_SUBGROUP_LOGIC — how individual clauses within a subgroup are joined (AND/OR)

So membership in multiple groups is permissive (OR): the user sees the union of rows permitted by each of their groups.

6. Empty-filter fallback

If nothing was written to the fileref (no stored filter, no validity vars, no matching RLS rules), the macro writes a literal 1=1 so the fileref can always be consumed as a valid WHERE expression. Absence of rules therefore means no restriction — RLS is opt-in per table/group.

How Callers Apply the Filter

Read path (viewdata.sas)

%mpe_filtermaster(VIEW, &libds, dclib=&mpelib, filter_rk=&filter_rk,
  outref=filtref, outds=work.query)

data work.viewdata;
  set &libds;
  where %inc filtref;;
  if _n_ > &DC_MAXOBS_WEBVIEW then stop;
run;

The fileref is %included directly inside the where statement — the filter never passes through client-visible state, so it cannot be tampered with. For database libraries the WHERE expression is pushed down to the database by the SAS engine.

Write path (stagedata.sas) — inverse filter

Uploads cannot be filtered; instead the filter is inverted and any submitted row matching the inverse is rejected:

%mpe_filtermaster(ULOAD, &libds, dclib=&mpelib, outref=filtref, ...)

/* prepare inverse query */
data _null_;
  infile filtref end=eof;
  file &tempref;
  if _n_=1 then put 'where not(';
  input;  put _infile_;
  if eof then put ')';
run;

data work.badrecords;
  set work.jsdata;      /* rows submitted by the user */
  %inc &tempref/source2;;
run;

%mp_abort(iftrue=(%mf_nobs(work.badrecords)>0)
  ,msg=%str(Security Problem - N unauthorised records submitted))

If even one submitted row falls outside the user's permitted row set, the entire staging request is aborted before any approval/apply step.

Incompatibility with REPLACE Load Type

RLS with EDIT scope is incompatible with tables configured with LOAD_TYPE=REPLACE in MPE_TABLES. A REPLACE load wipes and reloads the entire target table, so row-level filtering of submitted records cannot be enforced meaningfully (the rows a user is not allowed to see would also be deleted). Backend validations therefore abort in both directions (see issue #211):

1. mpe_row_level_security_postedit.sas — aborts when activating a rule with RLS_SCOPE in ('EDIT','ALL') against a table whose current MPE_TABLES record has LOADTYPE='REPLACE'.
2. mpe_tables_postedit.sas — aborts when setting LOADTYPE='REPLACE' on a table that already has active, current RLS_SCOPE in ('EDIT','ALL') rules in MPE_ROW_LEVEL_SECURITY.

VIEW-scope rules remain compatible with REPLACE loads, since they only affect read paths.

Edit-Time Validation (Injection Defence)

Because RLS_RAW_VALUE is free text that ends up inside a generated WHERE clause, it is a potential SAS code-injection vector. Mitigations:

1. mpe_row_level_security_postedit.sas — a post-edit hook on the MPE_ROW_LEVEL_SECURITY table itself. Every newly staged rule (with rls_active=1) is grouped by target libref.table and run through %mp_filtercheck(targetds=..., abort=YES), which compiles/tests each clause against the real target table and rejects the whole submission on invalid syntax (see mp_filtercheck).
2. Format rules enforced by validation: character values must be single quoted, IN/NOT IN values must be bracketed, BETWEEN must contain AND.

Additionally, editing MPE_ROW_LEVEL_SECURITY is itself a Data Controller table edit, so it goes through the normal approval workflow, audit trail (MPE_AUDIT), and can be column-restricted via Column Level Security.

Worked Example

Given these active rules (all for MYLIB.MYDS, RLS_ACTIVE=1):

SCOPE	GROUP	GROUP_LOGIC	SUBGRP_LOGIC	SUBGRP_ID	VAR	OP	VALUE
ALL	Group 1	AND	AND	1	VAR_2	IN	('this','or')
ALL	Group 1	AND	AND	1	VAR_3	<	42
ALL	Group 2	AND	AND	1	VAR_4	CONTAINS	'xyz'

A non-admin user in both groups, opening the VIEW page, gets a fileref containing:

AND ( (VAR_2 IN ('this','or') AND VAR_3 < 42) OR (VAR_4 CONTAINS 'xyz') )

resulting in:

data work.viewdata;
  set mylib.myds;
  where (VAR_2 IN ('this','or') AND VAR_3 < 42) OR (VAR_4 CONTAINS 'xyz');
run;

The same user submitting an EDIT upload has the inverse applied to their staged rows; any row not matching the expression above aborts the submission.

Summary of Security Properties

• Enforced server-side in every read service (VIEW/EDIT/DLOAD) and every write service (ULOAD via inverse filter).
• Admin bypass is explicit (&mpeadmins group membership check).
• Multi-group semantics are OR (union of permitted rows); rules within a group/subgroup are AND/OR per configuration.
• Fail-open by design: if no rules match a table, it is unrestricted — RLS must be opted into per table.
• Injection-resistant: values are validated with %mp_filtercheck at the time rules are edited, not at query time.