From 2987333ce9835ea8e89b693b0d049b9193c47849 Mon Sep 17 00:00:00 2001
From: Allan Bowe <>
Date: Thu, 7 May 2020 00:16:01 +0200
Subject: [PATCH] feat: datacatalog info

---
 docs/dcu-datacatalog.md | 33 +++++++++++++++++++++++++++++++++
 docs/licences.md        |  2 +-
 mkdocs.yml              |  1 +
 3 files changed, 35 insertions(+), 1 deletion(-)
 create mode 100644 docs/dcu-datacatalog.md

diff --git a/docs/dcu-datacatalog.md b/docs/dcu-datacatalog.md
new file mode 100644
index 0000000..be13912
--- /dev/null
+++ b/docs/dcu-datacatalog.md
@@ -0,0 +1,33 @@
+# Data Controller for SAS: Data Catalog
+Data Controller collects information about the size and shape of the tables and columns.  The Catalog does not contain information about the data content (values).  
+
+The catalog is based primarily on the existing SAS dictionary tables, augmented with attributes such as primary key fields, filesize / libsize, and number of observations (eg for database tables).
+
+Frequently changing data (such as nobs, size) are stored on the MPE_DATASTATUS_XXX tables.  The rest is stored on the MPE_DATACATALOG_XXX tables.
+
+## Tables
+
+### Libraries
+
+This table contains library level attributes to provide a high level overview of data coverage.  Note that unless you are an administrator, you are unlikely to have the ability to view / open all of these libraries.  To avoid errors when opening invalid libraries, you can add pipe-separated LIBREFs to the DCXXXX.MPE_CONFIG table (var_scope='DC_CATALOG', var_name='DC_IGNORELIBS').
+
+
+### Tables
+
+Table attributes are split between those that change infrequently (eg PK_FIELDS) and those that change often (eg size, modified date, and NOBS).
+
+### Variables
+
+Variable attributes come from dictionary tables with an extra PK indicator.  A PK is identified by the fact the variable is within an index that is both UNIQUE and NOTNULL.  Variable names are always uppercase.
+
+## Assumptions
+
+The following assumptions are made:
+
+* Data _Models_ (eg attributes) are not sensitive.  If so the catalog tables should be disabled.
+* Users can see all tables in the libraries they can access.  The refresh process will close out any tables that are not found, if the user can see at least one table in a library.
+* For a particular site, libraries are unique on LIBREF.  
+
+If you have duplicate librefs, specific table security setups, or sensitive models - contact us.
+
+
diff --git a/docs/licences.md b/docs/licences.md
index 2fc45fc..69ec571 100644
--- a/docs/licences.md
+++ b/docs/licences.md
@@ -1037,7 +1037,7 @@ Data Controller source licences are extracted automatically from our repo using
 │  ├─ url: http://trentm.com
 │  ├─ path: /Users/allan/git/dcfrontend/node_modules/dashdash
 │  └─ licenseFile: node_modules/dashdash/LICENSE.txt
-├─ datacontroller@3.2.0
+├─ datacontroller@3.4.0
 │  ├─ licenses: UNLICENSED
 │  ├─ private: true
 │  ├─ path: ../dcfrontend
diff --git a/mkdocs.yml b/mkdocs.yml
index f749a92..753f6e7 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -6,6 +6,7 @@ pages:
     - DC User Guide: dc-userguide.md
     - File Uploads: dcu-fileupload.md
     - Table Viewer: dcu-tableviewer.md
+    - Data Catalog: dcu-datacatalog.md
 - Configuring Data Controller:
     - Dates / Datetimes: dcc-dates.md
     - Emails: emails.md