chore: automated commit

docs/dci-troubleshooting.md

@@ -1,3 +1,10 @@
+---
+layout: article
+title: Troubleshooting
+description: Descriptions of common issues when working with Data Controller, and steps for resolution.
+og_image:  img/cannotimport.png
+---
+
 # Data Controller for SAS® - Troubleshooting
 
 ## Overview
@@ -74,3 +81,13 @@ After a successful install, your business user may see the following message:
 The reason is that the context chosen when importing the SPK (perhaps, SASApp) is not available to your business user.  It's likely you have multiple contexts.
 
 The SPK must be re-imported with the correct context chosen.  This may require regenerating the tables, or adjusting the permissions, if the new context uses a different system account.
+
+## Stored Processes Cannot Be Imported Into A Project Repository
+
+During the SPK import on a SAS 9 instance you may see the following dialog:
+
+![Stored processes cannot be imported into a project repository](img/cannotimport.png)
+
+> Stored processes cannot be imported into a project repository
+
+This can happen when importing with Data Integration Studio and your user profile is making use of a personal project repository. Try re-connecting with the Foundation repository, or import with SAS Management Console (which does not support project repositories).

docs/roadmap.md

@@ -20,11 +20,12 @@ If you would like to see a new Feature added to Data Controller, then let's have
 
 Where features are requested, whether there is budget or not, we will describe the work below and provide estimates.
 
-There are currently three features requested:
+There are currently four features requested:
 
 * Ability to restore previous versions (estimate - 6 to 9 days).  Sponsor needed.
 * Ability to import complex excel data using Excel Maps (10.5 days)
 * Ability to make automated submissions using an API (6 days)
+* Include change tracking information in a transaction table
 
 ### Complex Excel Uploads
 
@@ -108,6 +109,50 @@ The API will be delivered in a container, and will leverage existing data contro
 
 Total: **6 days**
 
+#### Change Tracking
+Currently, transactional changes made to tables in Data Controller are tracked by means of individual CSV files.  A user can navigate to the HISTORY tab, find their change, and download a zip file containing all relevant information such as the original excel that was uploaded, SAS logs, the change records (CSV) and the staging dataset.
+
+Whilst helpful for investigating individual changes, the mechanism does not curently exist for (easily) tracking changes to particular variables / records across time, nor for (easily) re-instating versions.
+
+A transaction table would fix this.  However, given that the number of changes could be quite large, this should be an optional (configurable) feature.
+
+The proposal is as follows:
+
+* Create a switch (new variable) in MPE_TABLES to enable tracking on a table-by-table basis
+* Create an MPE_AUDIT_HISTORY table to contain records of all changes to tracked tables
+
+Model Changes:
+
+```sas
+/* MPE_TABLES updates */
+proc sql;
+alter table dc.mpe_tables
+  add track_changes char(1);
+
+/* no PK defined as it is a transaction table */
+create table dc.mpe_audit_history(
+  load_id char(32),
+  processed_dttm num format=E8601DT26.6,
+  libref char(8),
+  dsn char(32),
+  key_hash char(32),
+  move_type char(1),
+  is_pk char(1),
+  is_diff char(1),
+  tgtvar_type char(1),
+  tgtvar_nm char(32),
+  oldval_num num,
+  newval_num num,
+  oldval_char char(32767),
+  newval_char char(32767)
+);
+```
+
+
+- [ ] tidy up & define bitemporal outputs to include "base" records
+- [ ] update postdata service to include new outputs
+- [ ] create macro to populate new audit tables if flag is set
+- [ ] call new macro from
 
 ## Delivered Features