docs.datacontroller.io/docs/dci-deploysasviya.md

158 lines
6.6 KiB
Markdown
Raw Normal View History

2020-10-01 17:22:31 +00:00
---
layout: article
2020-10-02 20:35:55 +00:00
title: DC SAS Viya Deployment
description: How to deploy Data Controller in a production SAS Viya environment
og_image: https://docs.datacontroller.io/img/dci_deploymentdiagramviya.png
2020-10-01 17:22:31 +00:00
---
2020-12-20 10:59:31 +00:00
# SAS Viya Deployment
2020-10-01 17:22:31 +00:00
## Overview
2020-10-02 20:35:55 +00:00
Data Controller for SAS Viya consists of a frontend, a set of Job Execution Services, a staging area, a Compute Context, and a database library. The library can be a SAS Base engine if desired, however this can cause contention (eg table locks) if end users are able to connect to the datasets directly, eg via Enterprise Guide or Base SAS.
A database that supports concurrent access is highly recommended.
## Prerequisites
2020-12-20 10:59:31 +00:00
### System Account
2021-01-05 18:48:13 +00:00
Data Controller makes use of a system account for performing backend data updates and writing to the staging area. This needs to be provisioned in advance using the Viya admin-cli. The process is well described here: [https://communities.sas.com/t5/SAS-Communities-Library/SAS-Viya-3-5-Compute-Server-Service-Accounts/ta-p/620992](https://communities.sas.com/t5/SAS-Communities-Library/SAS-Viya-3-5-Compute-Server-Service-Accounts/ta-p/620992)
2020-10-02 20:35:55 +00:00
2020-12-20 10:59:31 +00:00
### Database
We strongly recommend that the Data Controller configuration tables are stored in a database for concurrency reasons, however it is also possible to use a BASE engine library.
2020-10-02 20:35:55 +00:00
2020-12-20 10:59:31 +00:00
Let us know which database you are using and we will provide the DDL. We have customers in production using Oracle, Postgres, Netezza, SQL Server to name a few.
2020-10-02 20:35:55 +00:00
2021-01-05 18:48:13 +00:00
Simply run the provided DDL script to create the tables and initial configuration data in your chosen database. Make sure the system account (see prerequisites) has full read / write access.
2020-10-02 20:35:55 +00:00
2020-12-20 10:59:31 +00:00
!!! note
"Modify schema" privileges are not required.
2020-10-02 20:35:55 +00:00
2020-12-20 10:59:31 +00:00
### Staging Directory
All deployments of Data Controller make use of a staging directory. This is used to store CSV and Excel files as uploaded by end users. This directory should NOT be accessible by end users - only the SAS system account requires access to this directory.
A typical small deployment will grow by a 10-20 mb each month. A very large enterprise customer, with 100 or more editors, might generate up to 1 GB or so per month, depending on the size and frequency of the Excel EUCs and CSVs being uploaded. Web modifications are restricted only to modified rows, so are typically just a few kb in size.
2020-10-01 17:22:31 +00:00
2020-12-20 10:59:31 +00:00
## Deployment Diagram
2020-10-01 17:22:31 +00:00
2020-12-20 10:59:31 +00:00
The below areas of the SAS Viya platform are modified when deploying Data Controller:
2020-10-01 17:22:31 +00:00
2020-12-20 10:59:31 +00:00
<img src="/img/dci_deploymentdiagramviya.svg" height="350" style="border:3px solid black" >
2020-10-01 17:22:31 +00:00
2020-12-20 10:59:31 +00:00
## Deployment
Data Controller deployment is split between 3 deployment types:
2020-10-01 17:22:31 +00:00
2020-12-20 10:59:31 +00:00
* Demo version
* Full Version (manual deploy)
* Full Version (automated deploy)
2020-10-01 17:22:31 +00:00
2020-12-20 10:59:31 +00:00
## Full Version - Manual Deploy
2020-10-01 17:22:31 +00:00
2020-12-20 10:59:31 +00:00
### Create Compute Context
2020-10-01 17:22:31 +00:00
2020-12-20 10:59:31 +00:00
The Viya Compute context is used to spawn the Job Execution Services - such that those services may run under the specified system account, with a particular autoexec.
2020-10-01 17:22:31 +00:00
2020-12-20 10:59:31 +00:00
We strongly recommend a dedicated compute context for running Data Controller. The setup requires an Administrator account.
2020-10-01 17:22:31 +00:00
2020-12-20 10:59:31 +00:00
* Log onto SASEnvironment Manager, select Contexts, View Compute Contexts, and click the Create icon.
* In the New Compute Context dialog, enter the following attributes:
* Context Name
* Launcher Context
* Attribute pairs:
* reuseServerProcesses: true
* runServerAs: {{the account set up earlier}}
* Now switch to the Advanced tab and enter the following autoexec statements:
2020-10-01 17:22:31 +00:00
2020-12-20 10:59:31 +00:00
```
2021-01-05 18:48:13 +00:00
%let DC_LIBREF=DCDBVIYA;
2020-12-20 10:59:31 +00:00
%let DC_ADMIN_GROUP={{YOUR DC ADMIN GROUP}};
%let DC_STAGING_AREA={{YOUR DEDICATED FILE SYSTEM DRIVE}};
libname &dc_libref {{YOUR DC DATABASE}};
```
2020-10-01 17:22:31 +00:00
2020-12-20 10:59:31 +00:00
To explain each of these lines:
2020-10-01 17:22:31 +00:00
2021-01-05 18:48:13 +00:00
* `DC_LIBREF` can be any valid 8 character libref.
2020-12-20 10:59:31 +00:00
* `DC_ADMIN_GROUP` is the name of a group which will be granted unrestricted access to Data Controller
* `DC_STAGING_AREA` should point to a location on the filesystem (this is where the staging files and logs will be stored)
* The final libname statement should be configured to point at the database that was figured in [prerequisites](#prerequisites)
2020-10-01 17:22:31 +00:00
2020-12-20 10:59:31 +00:00
!!! note
XCMD is NOT required to use Data Controller.
2020-10-01 17:22:31 +00:00
2020-12-20 10:59:31 +00:00
### Deploy frontend
2020-10-01 17:22:31 +00:00
2021-02-17 13:44:05 +00:00
Unzip the frontend into `/var/www/html` directory of the SAS Web Server. Open `index.html` and update the following:
2021-02-17 11:57:30 +00:00
2021-02-17 13:44:05 +00:00
- `appLoc` attribute - this should point to the root folder on SAS Drive where the Job Execution services will be created.
- `contextName` attribute - here you should put the compute context that you created in the earlier step.
2021-05-12 14:53:38 +00:00
- `dcPath` attribute - ensures auto deploy to this physical location on the filesystem.
2020-10-01 17:22:31 +00:00
2020-12-20 10:59:31 +00:00
### Deploy backend
The services can be deployed using a hidden menu in Data Controller. Navigate to the app, and add `/deploy` to the url.
2020-10-01 17:22:31 +00:00
2020-12-20 10:59:31 +00:00
You are then presented with some options:
2020-10-01 17:22:31 +00:00
2020-12-20 10:59:31 +00:00
1 - choose your context
2 - click "deploy" to deploy the backend services
2020-10-01 17:22:31 +00:00
2020-12-20 10:59:31 +00:00
You can now launch the application!
2020-10-01 17:22:31 +00:00
2020-12-20 10:59:31 +00:00
## Full Version - Automated Deploy
The automated deploy makes use of the SASjs CLI to create the dependent context and job execution services. In addition to the standard prerequisites (a registered viya system account and a prepared database) you will also need:
* a local copy of the [SASjs CLI](https://sasjs.io/sasjs-cli/#installation)
2021-01-05 18:48:13 +00:00
* a Client / Secret - with an administrator group in SCOPE, and an authorization_code GRANT_TYPE. The SASjs [Viya Token Generator](https://github.com/sasjs/viyatoken) may help with this.
2020-12-20 10:59:31 +00:00
### Prepare the Target and Token
To configure this part (one time, manual step), we need to run a single command:
```
sasjs add
```
A sequence of command line prompts will follow for defining the target. These prompts are described [here](https://sasjs.io/sasjs-cli-add/). Note that `appLoc` is the SAS Drive location in which the Data Controller jobs will be deployed.
### Prepare the Context JSON
This file describes the context that the CI/CD process will generate. Save this file, eg as `myContext.json`.
```
{
"name": "DataControllerContext",
"attributes": {
"reuseServerProcesses": true,
"runServerAs": "mycasaccount"
},
"environment": {
"autoExecLines": [
"%let DC_LIBREF=DCDBVIYA;",
"%let DC_ADMIN_GROUP={{YOUR DC ADMIN GROUP}};",
"%let DC_STAGING_AREA={{YOUR DEDICATED FILE SYSTEM DRIVE}};",
"libname &dc_libref {{YOUR DC DATABASE}};",
],
"options": []
},
"launchContext": {
"contextName": "SAS Job Execution launcher context"
},
"launchType": "service",
}
```
### Prepare Deployment Script
The deployment script will run on a build server (or local desktop) and execute as follows:
```
# Create the SAS Viya Target
sasjs context create --source myContext.json --target myTarget
2020-10-01 17:22:31 +00:00