chore: automated commit

docs/dci-troubleshooting.md

@@ -1,3 +1,10 @@
+---
+layout: article
+title: Troubleshooting
+description: Descriptions of common issues when working with Data Controller, and steps for resolution.
+og_image:  img/cannotimport.png
+---
+
 # Data Controller for SAS® - Troubleshooting
 
 ## Overview
@@ -73,4 +80,14 @@ After a successful install, your business user may see the following message:
 
 The reason is that the context chosen when importing the SPK (perhaps, SASApp) is not available to your business user.  It's likely you have multiple contexts.
 
-The SPK must be re-imported with the correct context chosen.  This may require regenerating the tables, or adjusting the permissions, if the new context uses a different system account.
+The SPK must be re-imported with the correct context chosen.  This may require regenerating the tables, or adjusting the permissions, if the new context uses a different system account.
+
+## Stored Processes Cannot Be Imported Into A Project Repository
+
+During the SPK import on a SAS 9 instance you may see the following dialog:
+
+![Stored processes cannot be imported into a project repository](img/cannotimport.png)
+
+> Stored processes cannot be imported into a project repository
+
+This can happen when importing with Data Integration Studio and your user profile is making use of a personal project repository. Try re-connecting with the Foundation repository, or import with SAS Management Console (which does not support project repositories).

docs/roadmap.md

@@ -20,11 +20,12 @@ If you would like to see a new Feature added to Data Controller, then let's have
 
 Where features are requested, whether there is budget or not, we will describe the work below and provide estimates.
 
-There are currently three features requested:
+There are currently four features requested:
 
 * Ability to restore previous versions (estimate - 6 to 9 days).  Sponsor needed.
 * Ability to import complex excel data using Excel Maps (10.5 days)
 * Ability to make automated submissions using an API (6 days)
+* Include change tracking information in a transaction table
 
 ### Complex Excel Uploads
 
@@ -108,6 +109,50 @@ The API will be delivered in a container, and will leverage existing data contro
 
 Total: **6 days**
 
+#### Change Tracking
+Currently, transactional changes made to tables in Data Controller are tracked by means of individual CSV files.  A user can navigate to the HISTORY tab, find their change, and download a zip file containing all relevant information such as the original excel that was uploaded, SAS logs, the change records (CSV) and the staging dataset.
+
+Whilst helpful for investigating individual changes, the mechanism does not curently exist for (easily) tracking changes to particular variables / records across time, nor for (easily) re-instating versions.
+
+A transaction table would fix this.  However, given that the number of changes could be quite large, this should be an optional (configurable) feature.
+
+The proposal is as follows:
+
+* Create a switch (new variable) in MPE_TABLES to enable tracking on a table-by-table basis
+* Create an MPE_AUDIT_HISTORY table to contain records of all changes to tracked tables
+
+Model Changes:
+
+```sas
+/* MPE_TABLES updates */
+proc sql;
+alter table dc.mpe_tables
+  add track_changes char(1);
+
+/* no PK defined as it is a transaction table */
+create table dc.mpe_audit_history(
+  load_id char(32),
+  processed_dttm num format=E8601DT26.6,
+  libref char(8),
+  dsn char(32),
+  key_hash char(32),
+  move_type char(1),
+  is_pk char(1),
+  is_diff char(1),
+  tgtvar_type char(1),
+  tgtvar_nm char(32),
+  oldval_num num,
+  newval_num num,
+  oldval_char char(32767),
+  newval_char char(32767)
+);
+```
+
+
+- [ ] tidy up & define bitemporal outputs to include "base" records
+- [ ] update postdata service to include new outputs
+- [ ] create macro to populate new audit tables if flag is set
+- [ ] call new macro from
 
 ## Delivered Features