docs(rewrite): expand data restore guide with detailed workflow, security, and limitations
Publish to docs.datacontroller.io / Deploy docs (push) Successful in 1m17s
Publish to docs.datacontroller.io / Deploy docs (push) Successful in 1m17s
This commit is contained in:
blog-dev
+55
-13
@@ -7,26 +7,68 @@ og_image: https://docs.datacontroller.io/img/restore.png
|
||||
|
||||
# Data Restore
|
||||
|
||||
For those tables which have [Audit Tracking](/dcc-tables/#audit_libds) enabled, it is possible to restore the data to an earlier state!
|
||||
For tables that have [Audit Tracking](/dcc-tables/#audit_libds) enabled, it is possible to restore the data to an earlier state. This feature is available since Data Controller v6.8.
|
||||
|
||||
Simply open the submit to be reverted (via HISTORY or the table INFO/VERSIONS screen), and click the red **REVERT** button. This will generate a NEW submission, containing the necessary reversal entries. This new submission **must then be approved** in the usual fashion.
|
||||
## How to restore data
|
||||
|
||||
Open the submit to be reverted (via the **History** tab, or the **Info / Versions** screen on the table viewer), and click the red **REVERT** button.
|
||||
|
||||
|
||||
|
||||
This approach means that the audit history remains intact - there is simply a new entry, which reverts all the previous entries.
|
||||
This will generate a **new submission** containing all the reversal entries needed to bring the table back to its previous state. This new submission **must then be approved** in the usual fashion - it goes through the same edit-stage-approve workflow as any other change.
|
||||
|
||||
## Caveats
|
||||
This approach means that the audit history remains intact. There is simply a new entry in the audit trail which reverts all the previous entries.
|
||||
|
||||
Note that there are some caveats to this feature:
|
||||
## How it works
|
||||
|
||||
- User must have EDIT permission
|
||||
- Table must have TXTEMPORAL or UPDATE Load Type
|
||||
- Changes **outside** of Data Controller cannot be reversed
|
||||
- If there are COLUMN or ROW level security rules, the restore will abort
|
||||
- If the model has changed (new / deleted) columns the restore will abort
|
||||
Behind the scenes, rollback is not a silent undo. It is a **first-class approval workflow** just like any other edit. When you choose to restore a previous version, the backend reads the audit table and computes every difference between the current state and the version you want to go back to.
|
||||
|
||||
## Technical Information
|
||||
The process is driven by the open-source [`%mp_stripdiffs`](https://core.sasjs.io/mp__stripdiffs_8sas.html) macro. It extracts all changes recorded in the audit table (the default is [`MPE_AUDIT`](/tables/mpe_audit/), or a custom table configured in `AUDIT_LIBDS`) from the selected version onwards, and left-joins them to the base table to build a staging dataset. Changes are then applied in **reverse chronological order**:
|
||||
|
||||
The restore works by undoing all the changes listed in the [MPE_AUDIT](/tables/mpe_audit/) table. The keys from this table (since and including the version to be restored) are left joined to the base table (to get current values) to create a staging dataset, and then the changes applied in reverse chronological order using [this macro](https://core.sasjs.io/mp__stripdiffs_8sas.html). This staging dataset is then submitted for approval, providing a final sense check before the new / reverted state is applied.
|
||||
- **Deleted rows** are re-inserted with their original values.
|
||||
- **Modified rows** are reverted to their previous values.
|
||||
- **Added rows** are marked for deletion with the `_____DELETE__THIS__RECORD_____` flag.
|
||||
|
||||
Source code for the restore process is available [here](https://git.datacontroller.io/dc/dc/src/branch/main/sas/sasjs/services/editors/restore.sas).
|
||||
The computed differences are written to a new staging package in the approvals directory, complete with a CSV and a `macvars.sas` snapshot of the session context. A new `LOAD_REF` is generated, and the package is submitted via the standard [`mpe_loader`](https://git.datacontroller.io/dc/dc/src/branch/main/sas/sasjs/macros/mpe_loader.sas) service. This means the rollback itself is **reviewable and approvable** - nothing is applied silently.
|
||||
|
||||
Because the rollback creates a new approved changeset rather than silently rewinding history, the full audit trail is maintained: the reversion appears as a new load reference in `MPE_SUBMIT`, `MPE_REVIEW`, `MPE_DATALOADS`, and `MPE_AUDIT`, just like any other submission.
|
||||
|
||||
## Security and access
|
||||
|
||||
Not everyone can restore everything. The [`mpe_checkrestore`](https://git.datacontroller.io/dc/dc/src/branch/main/sas/sasjs/macros/mpe_checkrestore.sas) macro enforces a strict access check before the restore service will run.
|
||||
|
||||
### Who can restore
|
||||
|
||||
- **Admins** - members of the admin group (configured in `MPE_CONFIG`) can restore any table that has audit tracking enabled.
|
||||
- **Editors** - non-admin users must have `EDIT` access to the target table, and must **not** be subject to Column Level Security or Row Level Security restrictions.
|
||||
|
||||
### What is checked
|
||||
|
||||
1. **The load must exist in the audit table.** Loads that were never applied, or tables without an audit table configured, cannot be restored.
|
||||
2. **The user must have `EDIT` access.** This is checked via `MPE_SECURITY`.
|
||||
3. **CLS and RLS restrictions block non-admin users.** If the user belongs to a group with active Column Level Security (`MPE_COLUMN_LEVEL_SECURITY`) or Row Level Security (`MPE_ROW_LEVEL_SECURITY`) rules scoped to `EDIT` or `ALL`, restore is denied. Admins bypass this check.
|
||||
|
||||
!!! note
|
||||
The presence of CLS or RLS rules on a table does **not** prevent the table itself from being restorable. It only prevents non-admin users who are subject to those restrictions from initiating the restore. After a successful restore, the same CLS and RLS filters continue to apply when users view or edit the data.
|
||||
|
||||
If access is denied, the service aborts immediately with a clear reason - no opaque errors.
|
||||
|
||||
## Limitations
|
||||
|
||||
!!! warning
|
||||
Be aware of the following caveats before attempting a restore:
|
||||
|
||||
- **Audit table required.** The table must have `AUDIT_LIBDS` configured in `MPE_TABLES` (it defaults to `MPE_AUDIT`). Without this, there is no change history to roll back from.
|
||||
- **Only Data Controller changes can be reversed.** Changes made directly to the target table outside of Data Controller are not tracked in the audit table and cannot be reverted.
|
||||
- **Supported load types.** Restore is supported for `UPDATE` and `TXTEMPORAL` load types. `REPLACE` loads do not maintain row-level audit history and cannot be restored. `BITEMPORAL` loads maintain audit history but restore behaviour may be constrained by validity windows.
|
||||
- **Model consistency.** If the table structure has changed since the version being restored (for example, columns were added or removed), the restore will abort because the audit diffs no longer match the current schema.
|
||||
- **SCD2 validity windows.** For `TXTEMPORAL` tables, the restore works on the current snapshot only. Historical validity windows are respected by the underlying [`%mp_stripdiffs`](https://core.sasjs.io/mp__stripdiffs_8sas.html) macro, but ensure your approvers understand that the reversion applies to the current state of the data.
|
||||
- **No partial restore.** Rolling back a version reverts **all** changes from that version onwards, not just selected records. There is no way to cherry-pick individual rows from a previous version.
|
||||
|
||||
## Technical background
|
||||
|
||||
The restore process is implemented in the [`restore.sas`](https://git.datacontroller.io/dc/dc/src/branch/main/sas/sasjs/services/editors/restore.sas) service. It delegates access control to [`mpe_checkrestore.sas`](https://git.datacontroller.io/dc/dc/src/branch/main/sas/sasjs/macros/mpe_checkrestore.sas), diff computation to [`%mp_stripdiffs`](https://core.sasjs.io/mp__stripdiffs_8sas.html), and approval workflow submission to [`%mpe_loader`](https://git.datacontroller.io/dc/dc/src/branch/main/sas/sasjs/macros/mpe_loader.sas).
|
||||
|
||||
Audit records are written by the open-source [`%mp_storediffs`](https://core.sasjs.io/mp__storediffs_8sas.html) macro during the normal approval workflow, which compares the pre-load snapshot with the applied changes and appends row-level entries to the audit table. This is why restore is only possible for tables that were loaded through Data Controller's tracked loaders.
|
||||
|
||||
For a lighter overview of this feature, see the [feed post on datacontroller.io](https://datacontroller.io/feed/rollback-data-changes/).
|
||||
|
||||
Reference in New Issue
Block a user