Merge pull request 'docs(developer-guide): show the pre-edit hook result, fix the re-read drop list' (#17) from docs/pre-edit-hook-images into main

Reviewed-on: #17

.pre-commit-config.yaml

@@ -0,0 +1,11 @@
+# Hooks for the pre-commit framework - https://pre-commit.com
+#
+# This repo carries no native git hooks, so this is the route for a secret scan
+# at commit time: install the framework once with `pre-commit install`, and the
+# scan then runs on every commit. The hook is pinned to an exact gitleaks
+# release tag.
+repos:
+  - repo: https://github.com/gitleaks/gitleaks
+    rev: v8.30.1
+    hooks:
+      - id: gitleaks

docs/developer-guide/filter-pre-edit-hook.md

@@ -100,12 +100,17 @@ The hook below is registered in `MPE_TABLES.PRE_EDIT_HOOK` for a table with a da
     run;
 
     /* re-read the table, so the grid shows both quarters */
-    data work.out (drop=&var_txfrom &var_txto &var_processed);
+    data work.out (drop=&var_txfrom &var_txto &var_processed &drop_rk);
       _____DELETE__THIS__RECORD_____="No";
       set &orig_libds;
       where %inc newfilt;
     run;
 
+    /* keep the order the editor first showed */
+    proc sort data=work.out;
+      by &sortpk;
+    run;
+
     /* hand the widened filter back to the editor */
     data _null_;
       file filtref lrecl=32767;
@@ -125,9 +130,17 @@ The hook below is registered in `MPE_TABLES.PRE_EDIT_HOOK` for a table with a da
 - **The user's clause.** `work.QUERY` is the filter's clauses as structured rows, and `%mp_filtergenerate` turns them back into a WHERE clause. This is the user's filter on its own, with no Row Level Security or validity clause mixed in.
 - **Data Controller's clause.** Calling `%mpe_filtermaster` with `filter_rk=-1` produces the part of the filter that is not the user's: the current-record validity clause and any Row Level Security, or `1=1` when there is neither.
 - **The combined filter.** The two clauses are combined as `( user OR previous-quarter ) AND ( validity + RLS )`, so the previous quarter is added to the user's selection without widening anything the user is not entitled to see.
-- **The rows.** `work.OUT` is rebuilt with the combined filter, mirroring the data step `getdata` uses itself - including dropping the transaction and processing columns it drops.
+- **The rows.** `work.OUT` is rebuilt with the combined filter, using the same drop list as the data step `getdata` runs itself - the transaction and processing columns, plus `&drop_rk`, the surrogate key that `getdata` hides when a table is registered with `RK_UNDERLYING` - and sorted on `&sortpk`, because `getdata` sorts before the hook runs and does not sort again afterwards. On a bitemporal table `getdata` also deduplicates to the latest record per primary key (`by &pk &var_busfrom`); add that step too, or the re-read returns every temporal version rather than the current one.
 - **The returned text.** `filtref` is overwritten with the combined filter, so the clause the editor shows is the one that was actually applied.
 
+### What it looks like
+
+The captures below are the editor for a table with an `ORDER_DATE` column, opened with a filter for the current quarter. In the first, no hook is configured, so the filter is the user's own and the grid holds one quarter of rows. In the second, the hook has run: the filter above the grid carries both quarters, and the rows have grown to match.
+
+![The editor with the user's own filter, one quarter of rows](/img/pre-edit-hook-before.png)
+
+![The same editor after the hook widened the filter to the previous quarter](/img/pre-edit-hook-after.png)
+
 ## Row Level Security
 
 The user's clause and Data Controller's clause have to be kept apart, because Row Level Security lives in the second one. Combining them as `( user OR previous-quarter ) AND ( validity + RLS )` keeps RLS applied to both branches. Wrapping the whole existing filter instead - `( existing OR previous-quarter )` - would let the previous-quarter rows through without their RLS condition, so do not take that shortcut.