fix: doc update
This commit is contained in:
Allan Bowe
parent
454ea7351a
commit
6cc7186f99
@ -5,7 +5,7 @@ description: How to deploy Data Controller in a production SAS Viya environment
|
||||
og_image: https://docs.datacontroller.io/img/dci_deploymentdiagramviya.png
|
||||
---
|
||||
|
||||
# SAS 9 Deployment
|
||||
# SAS Viya Deployment
|
||||
|
||||
## Overview
|
||||
Data Controller for SAS Viya consists of a frontend, a set of Job Execution Services, a staging area, a Compute Context, and a database library. The library can be a SAS Base engine if desired, however this can cause contention (eg table locks) if end users are able to connect to the datasets directly, eg via Enterprise Guide or Base SAS.
|
||||
@ -13,79 +13,141 @@ A database that supports concurrent access is highly recommended.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### System Account
|
||||
|
||||
Data Controller makes use of a system account for performing backend data updates and writing to the staging area. This needs to be provisioned in advance using the Viya admin-cli. The process is well described here: https://communities.sas.com/t5/SAS-Communities-Library/SAS-Viya-3-5-Compute-Server-Service-Accounts/ta-p/620992
|
||||
|
||||
### Database
|
||||
We strongly recommend that the Data Controller configuration tables are stored in a database for concurrency reasons, however it is also possible to use a BASE engine library.
|
||||
|
||||
## Manual Deployment
|
||||
Let us know which database you are using and we will provide the DDL. We have customers in production using Oracle, Postgres, Netezza, SQL Server to name a few.
|
||||
|
||||
### Create Compute Context
|
||||
|
||||
Data Controller makes use of a dedicated compute context,
|
||||
|
||||
|
||||
## Backend
|
||||
|
||||
1 - Import `/sas/import.spk` using SAS Management Console. Make a note of the root location in which this was deployed - as this will be added to the `appLoc` value in the `index.html` file in the [frontend](#frontend) deployment.
|
||||
|
||||
2 - Create a physical staging directory. This folder will contain the logs and CSV files generated by Users. The SAS Spawned Server account (eg `sassrv`) will need write access to this location.
|
||||
|
||||
3 - Update the library to point to a schema in your preferred database
|
||||
|
||||
4 - Deploy the physical tables and register them in metadata. For this, simply compile and run the `mpe_build()` macro using an account with appropriate privileges.
|
||||
Simply run the provided DDL script to create the tables and initial configuration data in your chosen database. Make sure the system account (see prerequisites) has full read / write access.
|
||||
|
||||
!!! note
|
||||
Make sure the SAS Spawned Server account (eg `sassrv`) can access these tables!
|
||||
"Modify schema" privileges are not required.
|
||||
|
||||
## Frontend
|
||||
The Data Controller front end comes pre-built, and ready to deploy to the root of the SAS Web Server (mid-tier), typically `!SASCONFIG/LevX/Web/WebServer/htdocs` in SAS 9.
|
||||
|
||||
Deploy as follows:
|
||||
|
||||
1 - Unzip dcfrontend.zip and upload the entire `datacontroller` directory to the static content server.
|
||||
|
||||
2 - Open the `index.html` file and update the `appLoc` value to the location where the Stored Processes were deployed earlier.
|
||||
|
||||
It should now be possible to use the application - simply navigate to `YOURSASWEBLOC:port/yourRoot/datacontroller` and sign in!
|
||||
|
||||
The next step is to [configure](dcc-tables.md) the tables.
|
||||
### Staging Directory
|
||||
All deployments of Data Controller make use of a staging directory. This is used to store CSV and Excel files as uploaded by end users. This directory should NOT be accessible by end users - only the SAS system account requires access to this directory.
|
||||
A typical small deployment will grow by a 10-20 mb each month. A very large enterprise customer, with 100 or more editors, might generate up to 1 GB or so per month, depending on the size and frequency of the Excel EUCs and CSVs being uploaded. Web modifications are restricted only to modified rows, so are typically just a few kb in size.
|
||||
|
||||
## Deployment Diagram
|
||||
|
||||
The below areas of the SAS platform are modified when deploying Data Controller:
|
||||
The below areas of the SAS Viya platform are modified when deploying Data Controller:
|
||||
|
||||
<img src="/img/dci_deploymentdiagram.svg" height="350" style="border:3px solid black" >
|
||||
<img src="/img/dci_deploymentdiagramviya.svg" height="350" style="border:3px solid black" >
|
||||
|
||||
### Client Device
|
||||
|
||||
Nothing needs to be deployed or modified on the client device. We support a wide range of browsers (the same as SAS). Browsers make requests to the SAS Web Server, and will cache assets such as JS, CSS and images. Some items (such as dropdowns) are kept in local storage to improve responsiveness.
|
||||
## Deployment
|
||||
Data Controller deployment is split between 3 deployment types:
|
||||
|
||||
### SAS Mid Tier
|
||||
* Demo version
|
||||
* Full Version (manual deploy)
|
||||
* Full Version (automated deploy)
|
||||
|
||||
The front end is deployed to the SAS Web Server as described [above](/dci-deploysas9/#frontend). This requires making a dedicated public folder in the htdocs directory.
|
||||
|
||||
### SAS Application Server
|
||||
## Full Version - Manual Deploy
|
||||
|
||||
Given the enhanced permissions needed of the system account, a dedicated / secured STP instance is recommended as described [here](/dci-stpinstance).
|
||||
### Create Compute Context
|
||||
|
||||
All deployments of Data Controller also make use of a staging directory. This is used to store CSV and Excel files as uploaded by end users. This directory should NOT be accessible by end users - only the SAS system account (eg sassrv) requires access to this directory.
|
||||
The Viya Compute context is used to spawn the Job Execution Services - such that those services may run under the specified system account, with a particular autoexec.
|
||||
|
||||
A typical small deployment will grow by a 10-20 mb each month. A very large enterprise customer, with 100 or more editors, might generate up to 1 GB or so per month, depending on the size and frequency of the Excel EUCs and CSVs being uploaded. Web modifications are restricted only to modified rows, so are typically just a few kb in size.
|
||||
We strongly recommend a dedicated compute context for running Data Controller. The setup requires an Administrator account.
|
||||
|
||||
* Log onto SASEnvironment Manager, select Contexts, View Compute Contexts, and click the Create icon.
|
||||
* In the New Compute Context dialog, enter the following attributes:
|
||||
* Context Name
|
||||
* Launcher Context
|
||||
* Attribute pairs:
|
||||
* reuseServerProcesses: true
|
||||
* runServerAs: {{the account set up earlier}}
|
||||
* Now switch to the Advanced tab and enter the following autoexec statements:
|
||||
|
||||
```
|
||||
%let DC_LIBREF=DCDBVIYA;
|
||||
%let DC_ADMIN_GROUP={{YOUR DC ADMIN GROUP}};
|
||||
%let DC_STAGING_AREA={{YOUR DEDICATED FILE SYSTEM DRIVE}};
|
||||
libname &dc_libref {{YOUR DC DATABASE}};
|
||||
```
|
||||
|
||||
To explain each of these lines:
|
||||
|
||||
* `DC_LIBREF` can be any valid 8 character libref.
|
||||
* `DC_ADMIN_GROUP` is the name of a group which will be granted unrestricted access to Data Controller
|
||||
* `DC_STAGING_AREA` should point to a location on the filesystem (this is where the staging files and logs will be stored)
|
||||
* The final libname statement should be configured to point at the database that was figured in [prerequisites](#prerequisites)
|
||||
|
||||
!!! note
|
||||
XCMD is NOT required to use Data Controller.
|
||||
|
||||
### Deploy frontend
|
||||
|
||||
Unzip the frontend into the `/var/www/html` directory of the SAS Web Server. Open the `index.html` and update the `appLoc` attribute - this should point at the root folder in SAS Drive where the Job Execution services will be created.
|
||||
|
||||
### Deploy backend
|
||||
The services can be deployed using a hidden menu in Data Controller. Navigate to the app, and add `/deploy` to the url.
|
||||
|
||||
You are then presented with some options:
|
||||
|
||||
1 - choose your context
|
||||
2 - click "deploy" to deploy the backend services
|
||||
|
||||
You can now launch the application!
|
||||
|
||||
## Full Version - Automated Deploy
|
||||
|
||||
The automated deploy makes use of the SASjs CLI to create the dependent context and job execution services. In addition to the standard prerequisites (a registered viya system account and a prepared database) you will also need:
|
||||
|
||||
* a local copy of the [SASjs CLI](https://sasjs.io/sasjs-cli/#installation)
|
||||
* a Client / Secret - with an administrator group in SCOPE, and an authorization_code GRANT_TYPE. The SASjs [Viya Token Generator](https://github.com/sasjs/viyatoken) may help with this.
|
||||
|
||||
### Prepare the Target and Token
|
||||
|
||||
To configure this part (one time, manual step), we need to run a single command:
|
||||
```
|
||||
sasjs add
|
||||
```
|
||||
A sequence of command line prompts will follow for defining the target. These prompts are described [here](https://sasjs.io/sasjs-cli-add/). Note that `appLoc` is the SAS Drive location in which the Data Controller jobs will be deployed.
|
||||
|
||||
### Prepare the Context JSON
|
||||
This file describes the context that the CI/CD process will generate. Save this file, eg as `myContext.json`.
|
||||
|
||||
```
|
||||
{
|
||||
"name": "DataControllerContext",
|
||||
"attributes": {
|
||||
"reuseServerProcesses": true,
|
||||
"runServerAs": "mycasaccount"
|
||||
},
|
||||
"environment": {
|
||||
"autoExecLines": [
|
||||
"%let DC_LIBREF=DCDBVIYA;",
|
||||
"%let DC_ADMIN_GROUP={{YOUR DC ADMIN GROUP}};",
|
||||
"%let DC_STAGING_AREA={{YOUR DEDICATED FILE SYSTEM DRIVE}};",
|
||||
"libname &dc_libref {{YOUR DC DATABASE}};",
|
||||
],
|
||||
"options": []
|
||||
},
|
||||
"launchContext": {
|
||||
"contextName": "SAS Job Execution launcher context"
|
||||
},
|
||||
"launchType": "service",
|
||||
}
|
||||
```
|
||||
|
||||
### Prepare Deployment Script
|
||||
|
||||
The deployment script will run on a build server (or local desktop) and execute as follows:
|
||||
|
||||
```
|
||||
# Create the SAS Viya Target
|
||||
sasjs context create --source myContext.json --target myTarget
|
||||
|
||||
### SAS Metadata Server
|
||||
|
||||
The items deployed to metadata include:
|
||||
|
||||
* Folder tree
|
||||
* Stored Processes
|
||||
* Library Object & tables
|
||||
|
||||
After the installation process (which updates `settings` and removes the `makedata` STP), there are no write actions performed against metadata.
|
||||
|
||||
### Databases
|
||||
|
||||
We strongly recommend that the Data Controller configuration tables are stored in a database for concurrency reasons, however it is also possible to use a BASE engine library.
|
||||
|
||||
We provide the DDL for creating the tables, we have customers in production using Oracle, Postgres, Netezza, SQL Server to name a few.
|
||||
|
||||
Data Controller does NOT modify schemas! It will not create or drop tables, or add/modify columns or attributes. Only rows can be modified using the tool.
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user