226 lines
11 KiB
Markdown
226 lines
11 KiB
Markdown
# Row Level Security — Technical Deep Dive
|
|
|
|
This document explains **how** Row Level Security (RLS) is implemented in the Data Controller backend. For the user-facing configuration guide, see [docs.datacontroller.io/row-level-security](https://docs.datacontroller.io/row-level-security/).
|
|
|
|
## Overview
|
|
|
|
RLS in Data Controller is implemented as **server-side WHERE clause generation**. No data leaves SAS without passing through a dynamically generated filter. The filter is built at runtime per request, based on:
|
|
|
|
1. The requesting user's group memberships (SAS metadata groups + DC groups)
|
|
2. The active rules in the `MPE_ROW_LEVEL_SECURITY` control table
|
|
3. The access mode (VIEW / EDIT / download / upload)
|
|
|
|
Because the filter is expressed as a standard SAS `WHERE` expression, it works against **any** engine — Base SAS datasets, database libraries (via implicit SQL pass-through pushdown), SPDE, CAS libnames, etc.
|
|
|
|
## Request Flow
|
|
|
|
```mermaid
|
|
flowchart TD
|
|
A[Client request\nview / edit / download / upload] --> B{Service mode}
|
|
B -->|VIEW / EDIT / DLOAD| C["%mpe_filtermaster(mode, libds)"]
|
|
B -->|ULOAD stagedata.sas| C
|
|
C --> D["%mpe_getgroups()\nmetadata groups + MPE_GROUPS"]
|
|
D --> E{User in &mpeadmins?}
|
|
E -->|Yes| F[No RLS filter\n1=1]
|
|
E -->|No| G[Lookup active rules in\nMPE_ROW_LEVEL_SECURITY\nfor libref.table + user's groups]
|
|
G --> H{Rules found?}
|
|
H -->|No| F
|
|
H -->|Yes| I["%mp_filtergenerate()\nper group, OR'd together"]
|
|
I --> J[WHERE clause written\nto temp fileref]
|
|
F --> J
|
|
J --> K{Read or Write?}
|
|
K -->|Read\nviewdata / getdata / getrawdata| L["where %inc filtref;;\nrows filtered server-side"]
|
|
K -->|Write\nstagedata| M[Inverse filter\nwhere not( filtref )]
|
|
M --> N{badrecords > 0?}
|
|
N -->|Yes| O[Abort submission\nSecurity Problem]
|
|
N -->|No| P[Staging proceeds\nto approval workflow]
|
|
```
|
|
|
|
## Key Components
|
|
|
|
| Component | Location | Role |
|
|
|---|---|---|
|
|
| `MPE_ROW_LEVEL_SECURITY` | `&mpelib` (DC control library) | The rule table (scope, group, libref, table, logic, subgroup, variable, operator, raw value, active flag) |
|
|
| `%mpe_filtermaster()` | `sas/sasjs/macros/mpe_filtermaster.sas` | Master macro that assembles the full WHERE clause for a request |
|
|
| `%mpe_getgroups()` | `sas/sasjs/macros/mpe_getgroups.sas` | Resolves group membership (metadata groups via `%dc_getusergroups` + `MPE_GROUPS` DC-internal groups) |
|
|
| `%mp_filtergenerate()` | SASjs core | Converts a query table (logic/subgroup/variable/operator/value rows) into WHERE clause text |
|
|
| `%mp_filtercheck()` | SASjs core | Validates rule syntax at *edit time* (defence against SAS code injection) |
|
|
| `mpe_row_level_security_postedit.sas` | `sas/sasjs/services/hooks/` | Post-edit hook that runs `%mp_filtercheck` whenever the RLS table itself is edited |
|
|
|
|
## The Modes
|
|
|
|
`%mpe_filtermaster` accepts a `mode` parameter, and every service that surfaces data calls it with the appropriate mode:
|
|
|
|
| Mode | Caller | Purpose |
|
|
|---|---|---|
|
|
| `VIEW` | `services/public/viewdata.sas`, `getchangeinfo.sas` | Read-only table viewer |
|
|
| `EDIT` | `services/editors/getdata.sas` | The EDIT grid (adds "current records only" validity filtering) |
|
|
| `DLOAD` | `services/public/getrawdata.sas` | Raw file downloads (RLS scope treated as `VIEW`) |
|
|
| `ULOAD` | `services/editors/stagedata.sas` | **Upload validation** (RLS scope treated as `EDIT`) |
|
|
|
|
Scope mapping: `DLOAD` requests match rules with `RLS_SCOPE in ('VIEW','ALL')`; `ULOAD` requests match rules with `RLS_SCOPE in ('EDIT','ALL')`.
|
|
|
|
## Execution Flow of `%mpe_filtermaster`
|
|
|
|
The macro writes the final WHERE expression to a temporary **fileref** (`outref`), line by line. (A fileref is used because a generated filter may exceed the 64k macro variable limit — and note that `%include` of a fileref is not allowed directly in a `proc sql` where clause, hence callers typically use it in a DATA step or data step view.)
|
|
|
|
### 1. User-supplied filter (FILTER_RK)
|
|
|
|
If the request includes a stored filter (`filter_rk > 0`), its clauses are read from `MPE_FILTERANYTABLE` / `MPE_FILTERSOURCE` and generated first via `%mp_filtergenerate`. RLS clauses are then **AND-ed on top** — a user filter can only ever narrow results, never widen them beyond RLS.
|
|
|
|
### 2. Validity-date filtering (EDIT / DLOAD only)
|
|
|
|
`MPE_TABLES` may define `VAR_TXFROM` / `VAR_TXTO` (SCD2-style validity variables) for the target table. Unless the user explicitly filtered on those variables, the macro appends:
|
|
|
|
```
|
|
("<current datetime>"dt < VAR_TXTO)
|
|
```
|
|
|
|
so that only current records are surfaced in the EDIT grid and downloads.
|
|
|
|
### 3. Group resolution and admin bypass
|
|
|
|
```sas
|
|
%mpe_getgroups(user=%mf_getuser(), outds=work.groups)
|
|
```
|
|
|
|
Groups come from two sources, concatenated:
|
|
|
|
* SAS metadata groups (Viya / EBI / Base-specific logic in `%dc_getusergroups`)
|
|
* The `MPE_GROUPS` DC table (group assignments managed inside Data Controller)
|
|
|
|
If the user is a member of the `&mpeadmins` group, **RLS is skipped entirely** — admins always see all rows.
|
|
|
|
### 4. Rule extraction
|
|
|
|
Non-admin users trigger a lookup of active, current rules:
|
|
|
|
```sas
|
|
create table work.&rlsds as
|
|
select rls_group, rls_group_logic, rls_subgroup_logic, rls_subgroup_id,
|
|
rls_variable_nm, rls_operator_nm, rls_raw_value
|
|
from &mpelib..mpe_row_level_security
|
|
where &dc_dttmtfmt. lt tx_to /* only current (non-deleted) rules */
|
|
and rls_scope in ("&scopeval",'ALL')
|
|
and upcase(rls_group) in (select upcase(groupname) from work.groups)
|
|
and rls_libref = "<libref>" and rls_table = "<dsname>"
|
|
and rls_active = 1
|
|
order by rls_group, rls_subgroup_id;
|
|
```
|
|
|
|
### 5. Clause assembly
|
|
|
|
If rules exist, they are appended to the fileref as `AND ( ... )`. Each **group** the user belongs to contributes one sub-filter, and the group-level sub-filters are joined with `OR`:
|
|
|
|
```
|
|
AND ( <group 1 filter> OR <group 2 filter> ... )
|
|
```
|
|
|
|
Each group's filter is produced by `%mp_filtergenerate`, which honours:
|
|
|
|
* `RLS_GROUP_LOGIC` — how subgroups (identified by `RLS_SUBGROUP_ID`) are joined
|
|
* `RLS_SUBGROUP_LOGIC` — how individual clauses within a subgroup are joined (AND/OR)
|
|
|
|
So membership in multiple groups is **permissive** (OR): the user sees the union of rows permitted by each of their groups.
|
|
|
|
### 6. Empty-filter fallback
|
|
|
|
If nothing was written to the fileref (no stored filter, no validity vars, no matching RLS rules), the macro writes a literal `1=1` so the fileref can always be consumed as a valid WHERE expression. Absence of rules therefore means **no restriction** — RLS is opt-in per table/group.
|
|
|
|
## How Callers Apply the Filter
|
|
|
|
### Read path (viewdata.sas)
|
|
|
|
```sas
|
|
%mpe_filtermaster(VIEW, &libds, dclib=&mpelib, filter_rk=&filter_rk,
|
|
outref=filtref, outds=work.query)
|
|
|
|
data work.viewdata;
|
|
set &libds;
|
|
where %inc filtref;;
|
|
if _n_ > &DC_MAXOBS_WEBVIEW then stop;
|
|
run;
|
|
```
|
|
|
|
The fileref is `%include`d directly inside the `where` statement — the filter never passes through client-visible state, so it cannot be tampered with. For database libraries the WHERE expression is pushed down to the database by the SAS engine.
|
|
|
|
### Write path (stagedata.sas) — inverse filter
|
|
|
|
Uploads cannot be filtered; instead the filter is **inverted** and any submitted row matching the inverse is rejected:
|
|
|
|
```sas
|
|
%mpe_filtermaster(ULOAD, &libds, dclib=&mpelib, outref=filtref, ...)
|
|
|
|
/* prepare inverse query */
|
|
data _null_;
|
|
infile filtref end=eof;
|
|
file &tempref;
|
|
if _n_=1 then put 'where not(';
|
|
input; put _infile_;
|
|
if eof then put ')';
|
|
run;
|
|
|
|
data work.badrecords;
|
|
set work.jsdata; /* rows submitted by the user */
|
|
%inc &tempref/source2;;
|
|
run;
|
|
|
|
%mp_abort(iftrue=(%mf_nobs(work.badrecords)>0)
|
|
,msg=%str(Security Problem - N unauthorised records submitted))
|
|
```
|
|
|
|
If even one submitted row falls outside the user's permitted row set, the entire staging request is aborted before any approval/apply step.
|
|
|
|
## Incompatibility with REPLACE Load Type
|
|
|
|
RLS with `EDIT` scope is **incompatible** with tables configured with `LOAD_TYPE=REPLACE` in `MPE_TABLES`. A REPLACE load wipes and reloads the entire target table, so row-level filtering of submitted records cannot be enforced meaningfully (the rows a user is *not* allowed to see would also be deleted). Backend validations therefore abort in both directions (see [issue #211](https://git.datacontroller.io/dc/dc/issues/211)):
|
|
|
|
1. **`mpe_row_level_security_postedit.sas`** — aborts when activating a rule with `RLS_SCOPE in ('EDIT','ALL')` against a table whose current `MPE_TABLES` record has `LOADTYPE='REPLACE'`.
|
|
2. **`mpe_tables_postedit.sas`** — aborts when setting `LOADTYPE='REPLACE'` on a table that already has active, current `RLS_SCOPE in ('EDIT','ALL')` rules in `MPE_ROW_LEVEL_SECURITY`.
|
|
|
|
`VIEW`-scope rules remain compatible with REPLACE loads, since they only affect read paths.
|
|
|
|
## Edit-Time Validation (Injection Defence)
|
|
|
|
Because `RLS_RAW_VALUE` is free text that ends up inside a generated WHERE clause, it is a potential SAS code-injection vector. Mitigations:
|
|
|
|
1. **`mpe_row_level_security_postedit.sas`** — a post-edit hook on the `MPE_ROW_LEVEL_SECURITY` table itself. Every newly staged rule (with `rls_active=1`) is grouped by target `libref.table` and run through `%mp_filtercheck(targetds=..., abort=YES)`, which compiles/tests each clause against the real target table and rejects the whole submission on invalid syntax (see [mp_filtercheck](https://core.sasjs.io/mp__filtercheck_8sas.html)).
|
|
2. **Format rules** enforced by validation: character values must be single quoted, `IN`/`NOT IN` values must be bracketed, `BETWEEN` must contain `AND`.
|
|
|
|
Additionally, editing `MPE_ROW_LEVEL_SECURITY` is itself a Data Controller table edit, so it goes through the normal approval workflow, audit trail (`MPE_AUDIT`), and can be column-restricted via [Column Level Security](https://docs.datacontroller.io/column-level-security/).
|
|
|
|
## Worked Example
|
|
|
|
Given these active rules (all for `MYLIB.MYDS`, `RLS_ACTIVE=1`):
|
|
|
|
| SCOPE | GROUP | GROUP_LOGIC | SUBGRP_LOGIC | SUBGRP_ID | VAR | OP | VALUE |
|
|
|---|---|---|---|---|---|---|---|
|
|
| ALL | Group 1 | AND | AND | 1 | VAR_2 | IN | ('this','or') |
|
|
| ALL | Group 1 | AND | AND | 1 | VAR_3 | < | 42 |
|
|
| ALL | Group 2 | AND | AND | 1 | VAR_4 | CONTAINS | 'xyz' |
|
|
|
|
A non-admin user in **both** groups, opening the VIEW page, gets a fileref containing:
|
|
|
|
```
|
|
AND ( (VAR_2 IN ('this','or') AND VAR_3 < 42) OR (VAR_4 CONTAINS 'xyz') )
|
|
```
|
|
|
|
resulting in:
|
|
|
|
```sas
|
|
data work.viewdata;
|
|
set mylib.myds;
|
|
where (VAR_2 IN ('this','or') AND VAR_3 < 42) OR (VAR_4 CONTAINS 'xyz');
|
|
run;
|
|
```
|
|
|
|
The same user submitting an EDIT upload has the inverse applied to their staged rows; any row not matching the expression above aborts the submission.
|
|
|
|
## Summary of Security Properties
|
|
|
|
* **Enforced server-side** in every read service (VIEW/EDIT/DLOAD) and every write service (ULOAD via inverse filter).
|
|
* **Admin bypass** is explicit (`&mpeadmins` group membership check).
|
|
* **Multi-group semantics are OR** (union of permitted rows); rules within a group/subgroup are AND/OR per configuration.
|
|
* **Fail-open by design**: if no rules match a table, it is unrestricted — RLS must be opted into per table.
|
|
* **Injection-resistant**: values are validated with `%mp_filtercheck` at the time rules are edited, not at query time.
|