Latest Release Announcement
Reveal-Brainspace is improving underlying IT architecture to make it easier to manage operating system (OS) version changes going forward. Starting with the Brainspace 7.0 release, we are introducing a Linux OS that supports Docker, and more specifically, we recommend Ubuntu LTS as the underlying OS supporting Brainspace.
This guide is designed to help our clients quickly and easily deploy or migrate the Brainspace application to the 7.0 Release planned for the first quarter of 2024. Brainspace currently supports Chrome, Edge (Chromium-based versions) or Firefox browsers installed with WebGL enabled.
If you have any questions, please contact Brainspace Support at [email protected] for assistance.
Brainspace 7.0 Server Guidelines
Server Resource recommendations vary depending on client data, document and dataset size, complexity and the Brainspace features being used.
Use these server recommendations as a guideline only: monitor system resource utilization closely and update as required.
If using VM instances, use Resource Allocation Reservations to ensure committed server performance.
For larger volume instances, please consult your Brainspace Solutions Architect or submit a request to [email protected].
Additional Sizing Considerations
The Brainspace Server Guidelines table above will provide full Brainspace functionality and performance for a wide range of document populations. Actual resource needs will vary depending on source documents complexity and the combination of Brainspace analytics being used.
Brainspace processes all extracted text and metadata without a requirement for pre-filtering, however large, complex data types can reduce system performance and require additional resources.
Larger datasets tend to consume more RAM than multiple smaller datasets due to larger brains and more complex relationships within the document population.
Server specific resource considerations.
Application Server – All Active datasets are loaded into RAM for optimal graphical response. Large documents and datasets will increase memory requirements. Memory and Processor utilization is driven by Concept Search, Cluster Wheel and Communications Analysis activity.
Analytics Server – Only one dataset can be built at a time on the Analytics server. Additional Analytics servers can be deployed to allow for multiple, parallel dataset builds to occur. Additional resources (RAM & Processors, always at least a 4:1 ratio) may be required for very large dataset builds or may help improve dataset build times.
On-Demand Analytics (ODA) Server – Only one Focus can be built at a time on the ODA server. Additional ODA servers can be deployed to allow for multiple, concurrent Focus builds. Also, machine learning activity for CMML and PC occur on the ODA server and heavy classifier users should increase system resources accordingly. Additional resources (RAM & Processors, always at least a 4:1 ratio) may be required for very large Focus builds or may help improve CMML, PC and Focus build times.
Increased user activity in Brainspace does not drive system resources quite as much as data and system activity. Plan accordingly for dataset volume (number and size of documents) and active projects concurrently in the system. A good rule of thumb for high user counts is to add 0.5GB per active user (i.e. 20GB for 40 concurrently logged in users) to the Application Server.
Disk usage on the /data volume will expand by up to six times the size of your source text during the Brainspace dataset build process, index creation and optimization. Following the dataset completion and cleanup process the data will typically occupy from 1-2x the source text on disk.
Keep in mind that usage requirements will likely change between minor versions (7.1 to 7.2, 7.2 to 7.3, etc.) as new features are added.
Disk Layout
Brainspace requires 100GB of disk storage for each server, utilizing the following layout. Please note that any deviations from this recommendation may impact upgrades or application performance and should be discussed with the Brainspace Support PM or Engineer prior to upgrade or install. Any file storage for client-managed logging or security applications should be in addition to this recommendation.
The /var partition volume size should be, at minimum, 100GB, and ideally equal to the /localdata size on the Application server in very large environments with tens of millions of documents. The On-Demand Analytics server does its work in a local Docker volume stored in /var/lib/docker.
The /data volume should be a high-performance storage solution. It can be a disk on the Application server that gets shared, using NFS or other supported protocol, to the other servers, or it may be an NFS share from an Enterprise storage solution with minimum 10,000 IOPS and at least 1Gbps actual throughput between servers.
The /localdata volume should be a high-performance storage solution. It can be a disk on the Application server that gets shared, using NFS or other supported protocol, to the other servers, or it may be an NFS share from an Enterprise storage solution with minimum 10,000 IOPS and at least 1Gbps actual throughput between servers.
The NFS share must be configured before install, just like /data.
You will need the IP address of the NFS server and the NFS protocol version handy while performing the install.
This is our recommended disk layout. Alternatives can be configured if desired, so long as the general layout remains as follows:
Mount Point | Volume Size |
|---|---|
Application Server | |
/boot | 250 M |
/var | 100G minimum |
/tmp | 20G |
/ | Remainder |
/localdata (NFS that will be shared) | Variable |
/data (NFS that will be shared) | Variable |
Analytics Server / On Demand Server | |
/boot | 250 M |
/var | 100G minimum |
/tmp | 20G |
/ | Remainder |
/localdata | Variable |
OS Configuration Requirements
Brainspace 7.0 requires a Linux OS that supports Docker. Refer to this link to see a list of supported distributions: https://docs.docker.com/engine/install/#server.
We recommend Ubuntu LTS.
Refer to Docker documentation for specifics of installing Docker on a particular OS: https://docs.docker.com/engine/install/.
If your corporate security policy requires other Linux configuration settings, please notify your Brainspace Project Manager, as alternate settings may prevent Brainspace from installing or functioning as required.
Shared Storage Configuration
Brainspace requires a shared file system for the /data volume available to all Brainspace servers. We recommend using a Network File System (NFS) with nolock option enabled to share the /data storage volume hosted from an enterprise network storage (NAS) or from the Application server to the other servers in the instance.
For 7.0, Brainspace also requires a shared filesystem for /localdata configured the same as /data.
Note
Mount points do not need to be set up on the servers as the NFS client configuration will now be done in the Docker containers. Each server just needs connectivity to the IP address where the shared filesystem is hosted.
Brainspace Network Summary
Brainspace 7.0 uses NGINX running in Docker to host the user interface and that is where the SSL certificate is configured. See Replacing the Default Brainspace Self-Signed SSL Certificate in Brainspace 7.0.
Firewall Access Rules Summary
See Appendix B for table of port allowances.
In Brainspace 7, Docker handles all of the SSL encryption of data between nodes, so configuring TLS is done in Docker. See Docker article Manage swarm security with public key infrastructure (PKI) for technical details.
Security Settings
Brainspace Accounts
A ‘[email protected]’ Super Admin account will be created for your initial Brainspace user interface (UI) access. The Super Admin has the highest level of permissions for completing the installation configuration requirements, as well as for ongoing administrative and troubleshooting tasks across all Groups and Datasets. Brainspace recommends creating individual User, Group Admin and Super Admin accounts for individuals on your team as required to perform their assigned tasks. The original Super Admin account, or other individual account with system-level permissions, will be needed by Brainspace Support personnel should we need to assist your team with future troubleshooting issues. Please do not change the password, or if you do, keep the password in a safe and secure location.
Note
If your corporate security policy conflicts with any of these recommendations, your security policy will take precedence. Please discuss any concerns with your Project Manager to avoid disruption to your application functionality.
SSL Certificates
By default, all Brainspace interfaces (UI and inter-server communications) use self-signed SSL certificates. You may provide your own domain signed certificate for application access and encryption, if required.
Provide certificate in PEM (Base64-encoded ASCII) format, an unencrypted private key, and any intermediate certificates.
Brainspace can also be configured to offload SSL-encryption to a load balancer or proxy server. Please let your Project Manager or Solutions Architect know if you plan to do SSL encryption on another device.
Scheduling Brainspace 7.0 Installation or Upgrade
Planning
Once your servers are built and the environment configured as required, perform an audit of your systems to verify all prerequisites are completed to avoid delays during, or risk rescheduling, the installation.
When you feel your servers meet the configuration requirements send an Install or Upgrade request to [email protected] and you will be given access to the Installation Script which contains the new Preinstall Check Script. We request the output of the "Preinstall Check" be sent back to us so we can determine if there is anything that should be resolved before the installation/upgrade takes place.
Resolve any exceptions noted in the audit results. Brainspace requires a clean system audit before installation resources can be scheduled. If you have questions or are unable to complete any of the configuration requirements on your own, please contact your Project Manager to discuss your questions or to schedule a technical review call with one of our engineers to address any concerns regarding environment build and configuration.
Installation or Upgrade
Brainspace installation resources normally won’t have access to your servers and will rely on a screen sharing application, i.e., WebEx, Zoom, etc. The screenshare presenter must have full access to the server environment where Brainspace will be installed.
Brainspace application installation or upgrade generally takes between 2-4 hours, depending on server access and applicable integration with your third-party applications.
Key Resources Required for Installation
IT (server, storage and network) – Available to assist with server access and to remediate any environment issues that may hinder the Brainspace installation / upgrade process.
Relativity® Admins (as applicable) – Access accounts, plus the ability to access test data within Relativity®, will be required during the installation. Please make sure you have a test workspace in Relativity® to verify functionality after installation.
Brainspace Users/Admins – Following the installation or upgrade, we will perform a brief product orientation, including a demonstration on data ingestion and the key application features, to prepare your team to begin immediately realizing the power and new features in Brainspace 7.0. We will also show how to troubleshoot common issues, download relevant logs and submit questions/problems to Brainspace Support ([email protected]) when you need assistance. All Brainspace Users and Admins are encouraged to participate.
Brainspace Integration Requirements
Relativity® Integration for versions 9.7+ & 10.x+ (including RelativityOne cloud)
The Relativity® Plus connector requires only Relativity® API access via port 443 (https) and only works with Relativity® versions 9.7 and newer. In Brainspace 6.7 you could use the Relativity® Connector on datasets originally built with the Relativity® Connector, as long as the “Use Legacy Names” toggle is on.
Relativity® Integration for versions 9.2 – 9.7 (on-prem deployments only)
Brainspace integrates with older versions of Relativity® using our legacy Relativity® connector. This connector requires access to the Relativity® REST API (https) and direct SQL access (TCP 1433) using Brainspace legacy Relativity® connector.
Brainspace’s legacy Relativity® connector uses the RSAPI service to determine user access permissions to assigned workspaces, and MS-SQL access for high-speed ingestion and sync of Brainspace data back to Relativity®. The Brainspace Relativity® plugin respects the Relativity® ACLs (access control list) for data streaming and overlay back to Relativity® – Workspace access and updates will only occur with proper user authentication credentials.
Integrating with a Single Relativity® SQL Instance
Brainspace utilizes TCP port 1433 (by default; please indicate if you are using a non-standard SQL port) to access the Relativity® MS-SQL database for integration.
Integrating with Multiple/Distributed Relativity® SQL Instances
If using Static ports – Configure each SQL instance with the same port (typically 1433).
If using Dynamic ports – The MS SQL Browser Service must be configured (typically port 1434 UDP) for Brainspace to determine the actual port in use for each SQL instance. The full range of dynamic ports in use must be allowed through the firewall from the Brainspace Application server to each Relativity® SQL instance.
The Relativity® MS-SQL instance must be configured and available prior to Brainspace installation. This account must have EDDSDBO user rights. If unable to use EDDSDBO account, the provided account permissions will need to either be added manually or a script written to automatically update the “brainspace” user to all new workspaces.
Server Role = public
For Workspaces to be used with Brainspace, the database user needs db_datareader, db_datawriter, db_ddladmin and roles
For the main EDDS database, the database user needs db_datareader. For enhanced security, a custom role providing read only access to the Case and ResourceServer tables may be used.
For additional security, set the minimum required column access in Relativity® EDDS db:
Case ArtifactId
Case ServerId
ResourceServer ArtifactId
ResourceServer Name
The MS-SQL account must be configured on all SQL instances in a distributed SQL environment.
Appendix A: Pre-Install Checklist: Server Infrastructure Readiness
Fill out the Pre-Install Checklist to ensure a successful Brainspace installation. Completion is required before installation resources can be scheduled. Contact your Project Manager if you need assistance or have questions with any configuration item. (double-click the checkbox to mark item completed)
Check | Items to Complete | Additional Information |
|---|---|---|
Server Configuration
| Confirm server configuration (OS, Cores, RAM, Disks, Swap, DNS, NTP, patching, etc.) meets Brainspace required specifications. Upload final audit script results to [email protected]. | Run automated system audit script and resolve any identified issues. Full Audit instructions |
SSL Certificate | Which certificate type will you use? | Select all that apply:
|
Install and License Files available | Copy Brainspace installation files (installer + license) to the /data directory on Application server. |
|
Reboot servers | Reboot all servers to ensure services startup properly before beginning the Brainspace installation. Restart the Analytics Server, the Application Server, then the On-Demand Analytics Server to ensure NFS services come online. | [] Servers rebooted (After NFS configuration & Before Brainspace installation) |
Analyst Readiness | Have all of your users (Analysts and Administrators) received training on Brainspace 6 or above? {} Yes, our users are ready! {} Please send me more information on training. | Brainspace offers both self-guided and formalized training opportunities. Self-Paced |
Reveal Academy – Online Admin and Analyst training | ||
Environment Security Standards | FIPS Compliance | {} Compliance restrictions discussed with PM |
Supported Browsers | Confirm your users have Chrome, Edge (Chromium-based versions) or Firefox installed with WebGL enabled | {} Users have a supported browser |
Appendix B: Pre-Install Checklist: Network Access
Source | Destination | Ports | Description |
|---|---|---|---|
Brainspace Common Ports | |||
End Users | Application Server | 443 (HTTPS) | User access to User Interface |
System Admin | Application, Analytics, ODA Servers | 22 (SSH) | Admin access to Secure Shell for System Administrative tasks |
Analytics & ODA Servers | Application Server | 8081 (HTTPS/Tomcat) | Inter-server communication to the Application Server |
Application Server | Analytics & ODA Servers | 1604 (HTTPS) | Inter-server communication from the Application Server |
Application Server | Analytics Server | 111 TCP/UDP (RPC) 2049 TCP/UDP (NFS) 20045 TCP/UDP (STATD) 20047 TCP/UDP (LOCKD) 20048 TCP/UDP (MOUNTD) | Network file sharing for /data |
ODA Server | Application Server | 111 TCP/UDP (RPC) 2049 TCP/UDP (NFS) 20045 TCP/UDP (STATD) 20047 TCP/UDP (LOCKD) 20048 TCP/UDP (MOUNTD) | Network file sharing for Application’s /localdata/brainspace to ODA’s /localdata-share |
Application, Analytics, ODA Servers | Application Server, or PostgreSQL Server if combined | 5432 (PostgreSQL) | Communications to the PostgreSQL database |
Relativity® Integration Ports | |||
Application Server | Relativity® RSAPI Server | 443 (HTTPS) | Relativity® REST API access |
Application Server | Relativity® MS SQL Server(s) | 1433 TCP (SQL) 1434 UDP (SQL Browser Service) | Relativity® SQL Access only, if using Relativity® Classic connector |
Ports to Open
The following ports must be available. On some systems, these ports are open by default.
Port 2377 TCP for communication with and between manager nodes.
Port 7946 TCP/UDP for overlay network node discovery.
Port 4789 UDP (configurable) for overlay network traffic.
Ensure IP Protocol 50 (IPSec ESP) traffic is allowed.
Note:
Port 5000 will need to be open in the closed area to allow images to be transferred to the different hosts in the Swarm.
Amazon ECR Registries
To access Reveal's Amazon ECR registries, traffic to the following URLs must be allowed in the infrastructure:
api.ecr.us-east-1.amazonaws.com
053522176595.dkr.ecr.us-east-1.amazonaws.com
prod-us-east-1-starport-layer-bucket.s3.us-east-1.amazonaws.com
PostgreSQL and RabbitMQ Images
To access the PostgreSQL and RabbitMQ images from Docker Hub, ensure that traffic to the following URLs is permitted in the infrastructure:
Appendix C: Post-Install Checklist (Brainspace Verification)
Complete this checklist after installation to ensure Brainspace is ready for use by your team.
Check | Additional Information | Results |
|---|---|---|
Verify Admin functions | Datasets: Add from DAT file (requires Connector) and view dataset info. | |
Create tags. Download Archive, Process, Status reports; Build Error and Build logs and Import Error Archive. Explain each. Users and Groups: Create ‘[email protected]’ Super Admin account. Create test Group, User and Notebook; grant access to test Notebook. Show how User cannot access other Group Notebooks. | ||
Services: show license status, documents in use, expiration date and disk and memory usage. Download brainspace.log & catalina.out logs from App, analysisServer.out and analysisserver.log files from Analytics servers. Upload to install ticket. | ||
Errors: View latest. Archive. | ||
Create Enron10k dataset from local disk. | Verify build completes to ensure functionality. Performance wise, we expect a full build to complete in 6 to 7 minutes. | |
Build Enron10k from DAT. | Perform build from DAT completes without errors. | |
Concept Search & Cluster Wheel | Search for ‘birthday’ and verify results on Cluster Wheel. Top Terms (time, day, good, work, mom, great, etc.) and Top Concepts (happy birthday, birthday party, celebrate, cake, tortillas, dinosaur, annie, brisket, etc.). Add block of Terms from Cluster Wheel, verify further filtering on Dashboard. | |
Create a Notebook | Create a Notebook from search results. | |
Add to a Notebook | Remove Duplicate documents. Add search results to prior saved Notebook. | |
Create a Focus | Remove block of Terms filter, use results to Create Focus. | |
Communication Analysis | Select 2 people while holding Shift key. Verify people are highlighted and results display for Sent (unique docs, To, cc, bcc, etc.), Received, Sends Most To, Receives Most From, etc. | |
Near Duplicates | Verify ND documents at Cluster Wheel edge. Add to search criteria. | |
License Verification & Usage Reporting | Verify license usage and expiration details (Admin\Services\Application). | |
Download and send to Brainspace | Send to [email protected]. | |
Remove Unused Plug-Ins (Brainspace Only) | Navigate to /var/lib/brains/.brainspace/plugins and remove applicable, unused plug-ins | |
Date Format | Show AUTO (US ISO) and EURO Date Format plus the ability to custom configure any consistent date format. | |
Capture Version number | Access the server details under the Services tab |
Appendix D: Post-Install Checklist (Relativity®)
We will complete this checklist after installation to ensure Brainspace is ready for use by your team.
Relativity® Checks (if applicable) | Description | Additional Information | Results | |
|---|---|---|---|---|
1 | Relativity® applications | Verify install | Confirm the Brainspace Addons Application is installed to at least one workspace. | |
2 | Relativity® Connector Set Up | Add database details (URL, ports, password, etc) and Fields | Confirm after setting up connector. Select ALL overlay fields. | |
3 | Dataset | Create Dataset via Ingest Wizard | Apply customer supplied credentials, create the following Field mappings: *DOCID, *Fulltext (bodytext), Date, DateSent, To, From, CC, BCC. Facet To, From, CC, Date and BCC. (*Minimum required fields) | |
4 | Dataset | Submit Dataset | During submission, tail brainspace.log and info.log to watch for any errors. | |
5 | Sync | Sync Notebook | Confirm functionality via results returned (e.g. has images). | |
6 | Sync | Relativity® Overlay (Test in Relativity®) | Automatic overlay test via executing ‘BD all coding fields’ saved search. Confirm EMT fields populated. | |
7 | Validate | Record Count | # of records and time for build. | |
8 | Sync | Application Library | Push Brainspace application from Test workspace to Relativity® Application Library and unlock (push to library). |
Appendix E: Start-Up Order for Servers and Services
There is no required start-up order for servers and services in Brainspace 7.0 except that the NFS server should be setup prior to installing the software.
For the installation, you should start on the Application server and then add other servers to the cluster. Details are included in the installation guide.