dc
856e57173f
Review 431: dc-overview.md still said "Up to 500 rows can be edited (in the web editor) at one time" - the wrong number (DC_MAXOBS_WEBEDIT ships at 250) and, once dc/dc#352 lands, the wrong behaviour (an over-limit selection is served truncated with a warning, not refused or capped at a count). It now names both limits, whichever binds first, and links to the options that set them. Sweeping for the same stale figure turned up dcu-tableviewer.md, which says 500 twice and uses 500 in its worked example of the row count above the grid. DC_MAXOBS_WEBVIEW has been 2000 since 20261003_row_and_cell_limits.sas (the migration records 500 -> 2000), and mpe_makedata seeds 2000, so those three places were two releases out of date. Fixed here, and flagged in the PR reply as VIEW rather than EDIT in case it belongs in its own change. Checked: no "500 rows" or stale abort wording left in docs/, and the two anchors the overview now links to (dc_maxobs_webedit, dc_maxcells_webedit) exist on the published options page.
4.0 KiB
Data Controller for SAS®: Overview
What does the Data Controller do?
The Data Controller allows users to add, modify, and delete data. All changes are staged and approved before being applied to the target table. The review process, as well as using generic and repeatable code to perform updates, works to ensure data integrity.
What is a Target Table?
A Target Table is a physical table, such as a SAS dataset or a Table in a database. The attributes of this table (eg Primary Key, loadtype, library, SCD variables etc) will have been predefined by your administrator so that you can change the data in that table safely and easily.
How does it work?
From the Editor tab, a user selects a library and table for editing. Data can then be edited directly, or a uploaded from a file. After submitting the change, the data is loaded to a secure staging area, and the approvers are notified. The approver (which may also be the editor, depending on configuration) reviews the changes and accepts / or rejects them. If accepted, the changes are applied to the target table by the system account, and the history of that change is recorded.
Who is it for?
There are 5 roles identified for users of the Data Controller:
- Viewer. A viewer uses the Data Controller as a means to explore data without risk of locking datasets. By using the Data Controller to view data, it also becomes possible to 'link' to data (eg copy the url to share a table with a colleague).
- Editor. An editor makes changes to data in a table (add, modify, delete) and submits those changes to the approver(s) for acceptance.
- Approver. An approver accepts / rejects proposed changes to data under their control. If accepted, the change is applied to the target table.
- Auditor. An auditor has the ability to review the history of changes to a particular table.
- Administrator. An administrator has the ability to add new tables to the Data Controller, and to configure the security settings (at metadata group level) as required.
What is a submission?
The submission is the data that has been staged for approval. Note - submissions are never applied automatically! They must always be approved by 1 or more approvers first. The process of submission varies according to the type of submit.
Web Submission
When using the Web editor, a frontend check is made against the subset of data that was filtered for editing to see which rows are new / modified / marked deleted. Only those changed rows (from the extract) are submitted to the staging area.
Excel Submission
When importing an excel file, all rows are loaded into the web page. You have an option to edit those records. If you edit them, the original excel is discarded, and only changed rows are submitted (it becomes a web submission). If you hit SUBMIT immediately, then ALL rows are staged, and a copy of the excel file is uploaded for audit purposes.
CSV submission
A CSV upload bypasses the part where the records are loaded into the web page, and ALL rows are sent to the staging area directly. This makes it suitable for larger uploads.
Edit Stage Approve Workflow
The EDIT screen loads up to DC_MAXOBS_WEBEDIT rows (250 by default), or as many rows as fit within DC_MAXCELLS_WEBEDIT (200,000 cells) - whichever is reached first, so a wide table loads fewer rows than a narrow one. A larger table is not refused: it loads truncated, with a warning against the table name, and the filter is the way to reach the rest.
These edits are submitted to a staging area. After one or more approvals (acceptances) the changes are applied to the source table.
Use Case Diagram
There are five roles (Viewer, Editor, Approver, Auditor, Administrator) which correspond to 5 primary use cases (View Table, Edit Table, Approve Change, View Change History, Configure Table)
