diff --git a/docs/dci-troubleshooting.md b/docs/dci-troubleshooting.md index 6000176..2300ab1 100644 --- a/docs/dci-troubleshooting.md +++ b/docs/dci-troubleshooting.md @@ -1,3 +1,10 @@ +--- +layout: article +title: Troubleshooting +description: Descriptions of common issues when working with Data Controller, and steps for resolution. +og_image: img/cannotimport.png +--- + # Data Controller for SASĀ® - Troubleshooting ## Overview @@ -73,4 +80,14 @@ After a successful install, your business user may see the following message: The reason is that the context chosen when importing the SPK (perhaps, SASApp) is not available to your business user. It's likely you have multiple contexts. -The SPK must be re-imported with the correct context chosen. This may require regenerating the tables, or adjusting the permissions, if the new context uses a different system account. \ No newline at end of file +The SPK must be re-imported with the correct context chosen. This may require regenerating the tables, or adjusting the permissions, if the new context uses a different system account. + +## Stored Processes Cannot Be Imported Into A Project Repository + +During the SPK import on a SAS 9 instance you may see the following dialog: + +![Stored processes cannot be imported into a project repository](img/cannotimport.png) + +> Stored processes cannot be imported into a project repository + +This can happen when importing with Data Integration Studio and your user profile is making use of a personal project repository. Try re-connecting with the Foundation repository, or import with SAS Management Console (which does not support project repositories). \ No newline at end of file diff --git a/docs/img/cannotimport.png b/docs/img/cannotimport.png new file mode 100644 index 0000000..65bb3f6 Binary files /dev/null and b/docs/img/cannotimport.png differ diff --git a/docs/roadmap.md b/docs/roadmap.md index 3f80491..3fae5e8 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -20,11 +20,12 @@ If you would like to see a new Feature added to Data Controller, then let's have Where features are requested, whether there is budget or not, we will describe the work below and provide estimates. -There are currently three features requested: +There are currently four features requested: * Ability to restore previous versions (estimate - 6 to 9 days). Sponsor needed. * Ability to import complex excel data using Excel Maps (10.5 days) * Ability to make automated submissions using an API (6 days) +* Include change tracking information in a transaction table ### Complex Excel Uploads @@ -108,6 +109,50 @@ The API will be delivered in a container, and will leverage existing data contro Total: **6 days** +#### Change Tracking +Currently, transactional changes made to tables in Data Controller are tracked by means of individual CSV files. A user can navigate to the HISTORY tab, find their change, and download a zip file containing all relevant information such as the original excel that was uploaded, SAS logs, the change records (CSV) and the staging dataset. + +Whilst helpful for investigating individual changes, the mechanism does not curently exist for (easily) tracking changes to particular variables / records across time, nor for (easily) re-instating versions. + +A transaction table would fix this. However, given that the number of changes could be quite large, this should be an optional (configurable) feature. + +The proposal is as follows: + +* Create a switch (new variable) in MPE_TABLES to enable tracking on a table-by-table basis +* Create an MPE_AUDIT_HISTORY table to contain records of all changes to tracked tables + +Model Changes: + +```sas +/* MPE_TABLES updates */ +proc sql; +alter table dc.mpe_tables + add track_changes char(1); + +/* no PK defined as it is a transaction table */ +create table dc.mpe_audit_history( + load_id char(32), + processed_dttm num format=E8601DT26.6, + libref char(8), + dsn char(32), + key_hash char(32), + move_type char(1), + is_pk char(1), + is_diff char(1), + tgtvar_type char(1), + tgtvar_nm char(32), + oldval_num num, + newval_num num, + oldval_char char(32767), + newval_char char(32767) +); +``` + + +- [ ] tidy up & define bitemporal outputs to include "base" records +- [ ] update postdata service to include new outputs +- [ ] create macro to populate new audit tables if flag is set +- [ ] call new macro from ## Delivered Features