Native File Redaction – Exolution
  • 19 Nov 2024
  • 18 Minutes to read
  • Dark
    Light
  • PDF

Native File Redaction – Exolution

  • Dark
    Light
  • PDF

Article summary

Table of Contents

Overview

Native File Redaction – Exolution is an advanced native file redaction tool that empowers users to redact Excel spreadsheets in their native format efficiently. The application supports point-and-click manual redaction and automated mass redaction based on the provided criteria. The project team will tag the documents identified in Reveal Review into a folder and run them through the Exolution process to flatten the files in preparation for redaction. This process converts all charts and tables to static images to apply and maintain the markups once the redactions are applied in a cell or column.

Accessing Exolution for Clients Instance

Exolution is licensed for each Reveal 11 Review platform. The URL to access exolution is the Reveal 11 URL with exolution- added to the start of the Reveal 11 URL.

Reveal 11 URL: https://consulting.us-east-1.reveal11.cloud/

Exolution Reveal 11 URL: https://exolution-consulting.us-east-1.reveal11.cloud/

Reveal 11 URL

195 - 01 - Reveal 11 URL

Exolution Reveal 11 URL

195 - 02 - Exolution Reveal 11 URL

Windows Application

Exolution supports browser editors and Windows Apps.

Windows app

Browser display

Opens file natively without converting to HTML or any other format.

The browser converts content to HTML to display the content.

Very high-performance.

Performance is not as great as the Windows app.

Supports huge files, up to 2 GB.

Files larger than 20 MB won’t display or have significant performance issues.

Very lightly uses server resources to display documents, run searches, and redact many redactions.

Uses server resources to display documents, run searches, and redact many redactions. Potentially create performance issues when a large number of users are active.

No performance degradation when working with hundreds of thousands of values in single documents.

The view will take a long time to render, even for a few thousand redactions on a single document.

Supports shortcuts for many operations, like deleting all redactions, inverse redact, etc.

Shortcuts are not supported.

Supports only Windows OS.

Supports Windows and Mac OS.

Installation of desktop components is required. See below for installation.

Installation of desktop components is NOT required.

An additional setup is required in the Exolution server to use the Windows app.

The additional setup is not required.

The Window Application functionality requires downloading the Window application.

To Download, go to https://www.litexn.com/. Select Resources, Downloads, and Exolution Client Setup.

195 - 03 - Exolution Download-1

Prepare and Access Documents for Redaction

Note

To open documents in Exolution once they have been prepared for redaction, it is essential that the following fields are included in the field profile checked as 'Is Default' in Reveal.

Table Name

Display Name

ITEMID

Item ID

BEGDOC

Begin Number

FILE_EXTENSION

Extension

DOCUMENT_TYPE_DESC

App Name

EXPORTED_NATIVE_FILE

Exported Native File

MD5_HASH

Duplicate ID

FILESIZE

File Size (bytes)

Without these fields in the 'Is Default' field profile, documents will fail to open in Exolution.

  1. Go to Reveal Review.

  2. Search or filter for Excel documents.

  3. Add Excel documents to Work Folder or Assignment set.

    Note

    Users can only see folders or Assignments if they are on a team with access to them or are given explicit access. For assignments in a pool, the user must have the assignment checked out in Review.

    Recommendation: For reviews with many assignments, the admin will want to have all assigned documents in a single work folder so they only need to run one job to prepare for redaction.

  4. Go to the Exolution web portal (this will be their Reveal Review URL with "exolution-" added in front, e.g., https://exolution-consulting.eu-west-1.reveal11.cloud/).

  5. Login in using the same credentials to access Reveal 11.

  6. Work Folders/Assignments in Reveal 11 will be visible and accessible in Exolution.

    Note

    If logged into Exolution before creating a Work Folder, you'll need to leave the project and go back to view the newly created Work Folder. (Does not appear upon refresh.)


    195 - 04 - Reveal Folders in Exolution

  7. Go to the flyout menu (upper left corner) and select Prepare for Redaction to process and flatten the file for redaction.
     195 - 05 - Exolution Prepare for Redaction selection

  8. Click +Add to create a new job.
     195 - 06 - Add Redaction

    1. Job Name: Enter Job Name (RECOMMENDATION: Enter Name of Work Folder).

    2. Folder List: Select Work Folder.

    3. Sync Redactions: Click the check box to propagate redaction from existing documents to new documents. (Casebook-level propagation in settings must also be On.)
       195 - 07 - Add Prepare Job

    4. Click Create.

  9. Select the new job and click Start. When the job is complete, the "Status" column will change to a green check.
     195 - 08 - Start Prepare Job

  10. Open the fly-out menu and select Redact Documents.

  11. Click on an Assignment or Work Folder to pull up the list of documents.

  12. Select Browser Editor for Exolution's web-based browser editor.
     195 - 09 - Select Browser Editor

  13. Select a document to open Exolution's web-based browser editor.
     195 - 10 - Browser Editor Screen (annotated)

  1. Redaction Type Selection: Allows the user to select a redaction type.

  2. Main Menu: Provides access to most of the redaction-related functions.

  3. Document Navigator: Facilitates browsing the document set.

  4. Casebook Name: Name of the current casebook you are in.

  5. Is Reviewed: Display or edit Reveal field Exolution Review Complete.

  6. Document ID: Document identifier of the current document.

  7. Sheet List: List of the sheets in the current document.

  8. Comment List: List of comments in the current document.

  9. Redaction List: List of the redactions.

  10. Invisible Content List: List of the values with the same background and text content.

  11. Duplicate Sheet List: List of duplicate sheets.

  12. Global Search: Facilitates searching the entire document set.

Applying Redactions

Exolution supports multiple ways to apply redactions; some are document-level redactions, and others are mass redactions across multiple documents. Users must select a redaction type from the Select Redaction menu:

195 - 11 - Select Redaction

Note

All the redactions are propagated to the duplicate sheets or duplicate documents in real-time, depending on the redaction propagation settings for the current workspace, assuming you selected the option to propagate these during Casebook creation. If the redaction propagation selection is set to "None," then the redactions are not propagated to the duplicate sheets or documents.

To Manually Redact a Document:

Users can manually redact a cell range or an object like a comment or a textbox by selecting a redaction label, selecting the cells they want to redact, and then right-clicking the mouse on the selected cells or object:

195 - 12 - Manual Redaction

The redaction will appear as the color of the label chosen:

195 - 13 - Manual Redaction result display

The redactions are shown as highlights, but users can get a preview of the redacted document from the second menu, Display Options, and choosing Redacted Preview:

195 - 14 - Select Redacted Preview

195 - 15 - Redacted Preview

Bulk Redact PII in Multiple Documents Using the PII Explorer

Select the redaction type and choose the PII Explorer from the 1st menu:

195 - 16 - Select PII and Keyword Explorer

The left side of the PII Explorer shows PII types with the number of values found for the documents in the selected saved search. The right table displays the responsive documents with the total number of PII in each document:

195 - 17 - PII Explorer

Select the appropriate PII type and documents you want to redact, then select a redaction type from the Redact drop-down in the right corner. Click Full Value or Partial Value to apply redactions to the documents.

195 - 18 - Redact in PII Explorer

Search and Redact using Document Search.

The Document Search searches the values in cells, comments, sheet names, text boxes, and header/footers.

Select your redaction type and then click the Magnifying Glass to open the Find menu.

195 - 19 - Document Search toolbar

Enter the search criteria in the Find What box:

195 - 20 - Find What

Set your scope in the Within menu:

195 - 21 - Set Find Scope

Select the Options for Regular Expression or a keyword.

195 - 22 - Find Regex Options

Click the Find button.

195 - 23 - Click Find

The results will pop up. Select the ones you want to redact:

195 - 24 - Find Results list

Click on the Redact button to select the desired search:

195 - 25 - Redact selected Find results

Entire Values: Redacts all values in the cells, sheet names, comments, text boxes, and header/footer.

Only Searched Value: Redacts only the matching part of the responsive values.

Except Searched Value: This is opposite to the previous option, Only Searched Value. This option redacts all the content other than the searched value.

Entire Columns: Redacts the entire columns for each responsive value.

Entire Rows: Redacts the entire rows for each responsive value.

Inverse Redactions

Exolution has two options to apply inverse redaction: sheet level and workbook level. The inverse redaction removes all existing redactions and redacts all the previously non-redacted cells and shapes based on the user selection.

Select your redaction type and, go to the 3rd menu, select Invert in either the active sheet or workbook:

195 - 26 - Inverse redaction selection

The options menu pops up when the Invert in Active Sheet or Workbook menu is clicked. Select the options to apply inverse redactions, then click the Apply button.

195 - 27 - Invert Selection Options

Remove Redactions

The following are the ways to remove redactions:

  1. Manually remove a redaction: Right-click on any selected redaction to remove the redaction. This can also be performed from the menu item Redaction Options ➔ Redact | Unredact.
     195 - 28 - Select Redact-Unredact

  2. Remove all redactions from an Active Sheet: Menu item Redaction Options ➔ Clear All In Active Sheet removes all of the redactions in the active worksheet.

  3. Remove all redactions from an entire workbook: Menu item Redaction Options ➔ Clear All In Workbook removes all of the redactions in the whole workbook.
     195 - 29 - Select Unredaction Options

Note

All redactions from the duplicate sheets or files are also removed in real-time, depending upon the Redaction Propagation Option value in the Settings tab.

Burn-in Redactions

  1. Go to the flyout menu (upper left corner), click on Burn-in Redactions and click on +Add.
     195 - 30 - Select Burn-in Redactions

  2. Enter the job details, then click Create.

    1. Job Name: Enter Job Name (RECOMMENDATION: Enter Name of Work Folder).

    2. Folder List: Select Work Folder or Assignment.

    3. TextSet: Enter the name of the text set in Review. (Click +Add to create a new text set.)

    4. Overwrite: If a native/text file exists, overwrite with the existing native and text.

    5. Color Option: Leave as is (Redaction Color), All Black or All White.

    6. Label Options: Determine the type of Label (No Label, Long, Short, or Custom).

    7. Password Protect: Add a password to lockdown ability to edit native.

    8. Apply Stamp: Add a stamp to redaction (Example: Confidential).
       195 - 31 - Add Burn-in Job (annotated)

  3. Select the new job from the list and click Start at the top. Once the job is completed, a green check will appear next to the job.
     195 - 32 - Start Burn-in Job

Once the job is complete, indexing the redaction text set should begin in Review Manager automatically. Once indexing is complete, the text sets will appear in the Document Viewer in Reveal Review.

195 - 33 - Native Redaction Text Sets in Review

Manage Redactions

  1. To Manage Redactions, use the flyout menu and choose Manage Redactions.
     195 - 34 - Select Manage Redactions

  2. Click Add and select a redacted job to add to Manage Redactions.
     195 - 35 - Add job to Manage Redactions

  3. The Lock option locks documents. When a document is locked, the users cannot create a redacted copy or change the file association between the redacted file and the original file. The users, however, can continue to apply redactions in the Exolution viewer.
     195 - 37 - Lock Option

  4. The Freeze option freezes documents. The three freeze options are Freeze Propagation, Freeze All Redactions, and Unfreeze Redactions.
     195 - 38 - Freeze Options

    1. The Freeze Propagation option prevents the document from receiving redactions from duplicate sheets or documents. Direct redaction editing is still allowed, and the users can edit redactions by directly opening them in Exolution. Any redaction applied directly on the documents with the Redaction Propagation Frozen does propagate to the other documents without propagation freeze.

    2. The Freeze All Redactions option prevents all documents from receiving redactions from duplicate sheets or documents, including the redaction propagation and the direct redactions.

    3. The Unfreeze Redactions option unfreezes all the redactions, including redaction propagation and direct redaction change.

Reports

  1. Go to the flyout menu (upper left corner)

  2. Select Reports. There are three reports available in Exolution.
     195 - 39 - Exolution Reports

    1. Reviewers: Total documents redacted by reviewers. A date range and all reviewers or selected reviewers generate the report. The report is available on screen or by downloading an Excel report.
       195 - 40 - Reviewers Report

    2. Redaction Types: Redaction labels are used per document. The report includes DocId (Begin Number), RefID (Item Id), and Redaction Label. The report is available on screen or by downloading an Excel report.
       195 - 41 - Redaction Types Report

    3. Redactions: Complete list of all redactions applied. The report is generated by a date range and all redaction types (color code) or selected redaction types (color code). The report is available on screen or by downloading an Excel report.
       195 - 42 - Redactions Report

Column Name

Description

NativeRedactionID

Unique redaction identifier within the casebook. Auto-increment.

RecordID

Unique Document Identifier within the casebook created by Exolution. Refers to the table tblRecord.

SheetIndex

Zero-based sheet position within the workbook.

RedactionTypeID

Unique redaction type identifier within the casebook. Refers to tblRedactionType.

RedactionCategory

Possible values are 0=Full redaction, 1=Partial redaction, 10=Pseudonymization.

RedactionTextStart

Redaction start position for partial redaction and pseudonymization. -1 for full redactions.

RedactionTextLength

Redaction end position for partial redaction and pseudonymization. -1 for full redactions.

RedactionText

Redaction label (Pseudonym value) for pseudonymization. Redaction co-ordinates for images.

RangeAddress

Address of objects like cell, comment, etc. Name of the things like textbox, chart, etc.

PropagateFromRedactionID

Original redaction ID for the propagated redactions. -1 for the original redaction.

ID

Unique user identifier across Exolution instance. It refers to table tblUserInfo in the casebook SQL Server database and the table ExolutionUserInfo in the instance-level portal database.

TimeStamp

Date time when the redaction gets into the table for the original redactions. TimeStamp value of the original redaction for the propagated redactions.

Settings

  1. Exolution Settings has two primary options: Reviewers Features and Redaction Settings.

  2. Reviewers' Features. Under the Reviewers features, you can select the features reviewers can see. This option affects those with no administrative rights. The options are:
     195 - 43 - Settings - Reviewer Features

    • Browse Document Set – The user can select a batch that shows unique or all documents.

    • File Download - Download the redacted or unredacted document and open it in an Excel file-compatible viewer.

    • Duplicate Sheets – The user can create groups of duplicate sheets in the document set and allows the user to redact the duplicate sheets through this list.

    • Global Search – The user can run a global search. The global search pinpoints the exact location of the responsive values within the documents. The location-aware search returns which cell, comment, textbox, sheet name, or header/footer items have the responsive values. It supports a combination of keywords, a list of values, and Regular Expressions.

    • PII Explorer – The user can create new types of PII or Keywords (including Regular Expressions), perform a global search, and add PII entries to documents.

    • Term Group Explorer - The user can create a group of terms with multiple terms. Exolution identifies each sheet with any of the terms present and displays the sheet with unique matching terms count.

    • Visually Similar Sheet Explorer - It creates groups of sheets based on the look and feel. The data may be the same or different across the sheets in a group, but the look and feel are similar. It also facilitates applying redactions on all sheets in a group in a specific area.

    • Thread Explorer - Exolution supports Excel file threading, similar to email threading. This is when the Excel file is fully contained within another Excel file and listed underneath the larger Excel files.

    • Cluster Explorer - Excel file clustering is based on similar sheets. If two or more documents have at least one duplicate sheet across them, they are placed in a cluster. Then, all other Excel files with at least one duplicate sheet with any of the Excel files in the cluster are also added. The process continues until no more files are left out of the cluster with duplicate sheets with the files in the cluster.

    • Duplicate Sheet Explorer – The user can create groups of duplicate sheets in the document set and allows the user to redact the duplicate sheets through this list.

    • Sheet List Explorer - The users can create lists of the sheets to make the document review more efficient. The Sheet List Explorer facilitates creating and browsing the sheets list.

    • Hidden Content Explorer - This form lists all the documents with hidden and password-protected original sheets. It also lists all the documents with data hidden by making the font and background color the same. The users can open the document in the Exolution Viewer by double-clicking on any document from the list.

    • Pseudonym Explorer - It facilitates redacting each value with a specific redaction label to retain the entity relationships while redacting the values. For Example, redact all instances of [email protected] with the title Email00001 and redact all cases of [email protected] with the label Email00001. It requires a list prepared for each value.

3. Redaction Settings


    1. Redaction Propagation - Real-time Redaction Propagation Redact all duplicates in real time without any extra steps. You can select whether to redact duplicate Sheets, duplicate files, or none.
       195 - 44 - Redaction Propagation Settings
      Selecting Duplicate Sheets, Exolution has a DeepDup feature. DeepDup identifies duplicate sheets across non-duplicate documents and propagates redactions across the duplicates. If you have an Excel file with five sheets and another Excel file with three sheets and two sheets matching between them, then Exolution will propagate redactions for those two duplicate sheets but not the non-duplicate sheets.
      Selecting Duplicate Files propagates to an exact duplicate Excel spreadsheet based on the MD5 Hash values.

    2. Auto Propagation – You can auto-redact several PII, PHI, or customized fields. Auto propagation will automatically redact all types identified after flattening if any value other than the Do Not Auto redact is selected.
       195 - 45 - Auto Propagation Settings

Exolution Permission Settings

Role Groups

There are two permission groups in Exolution: Administrator and Reviewer.

The permission groups for Exolution are defined in the Reveal Review under Project Admin / Permissions / Role Groups.

Exolution Administrator must be a member of a Review Role Group with the word "Administrator" in the group's name.

Exolution Reviewer is any user not qualified as an Exolution Administrator.

Exolution Reviewer Permission

The settings for reviewer permission are in Exolution under settings. The settings are for all Exolution reviewers. See the Reviewers' Features under settings.

Work and Assignment Folders

A user has the same permission to access work and assignment folders as granted in Reveal Review.

Review Permission Considerations

There are no Exolution settings within Permissions in Reveal Review:

195 - 46 - Review Permissions

Additionally, removing permission to Redact from Review does NOT remove permission to redact in Exolution.

195 - 47 - Review Redact Permission

The Permissions setting in Review does not control the permissions for Exolution except for the naming nomenclature of the role group.

Supported File Types

  1. Web-Interface: Excel, CSV, ODS, PDF

  2. Desktop-Interface: Excel, CSV, ODS, PDF, Word Document, ODT, RTF, TXT, MHT, EML

Note

The Reveal license for Exolution supports the Web-Interface file types.

Production Considerations

  1. Create a Production Job just as you would for any production with Excel files that are to be produced natively. If it is an imaged production with slip sheets for Excel, you can set up the slip sheet template just like any other production with slip-sheeted Excel files.

  2. For the Production Export job, you must set the Native File Rules. If it is an all-native production, you can choose All. If it is an imaged production with slip sheets for the Excel files, you can select Custom and set Excluded to Export.
     195 - 48 - Production Export Native File Rules

  3. Once you set the rules, you will see the Native Files option appear. Click on the three dots on the right.
     195 - 49 - Production Export - Native Files

  4. In the popup, make sure "First Native Found" is selected and click on +Add.
     195 - 50 - Production Export - Native Files Selection

  5. Check the box next to the field that contains the path to the redacted native and click Add.
     195 - 51 - Production Export - Add Native File Type

  6. Click and hold on to the six dots next to the redacted path field, drag it to the top of the list, and click Save.
     195 - 52 - Production Export - Set Doc type order

  7. For the Text/OCR Files to Include, select All and repeat the steps above to choose the field for the path to the redacted text files.

Exolution Fields in Review

A set of fields is created in Reveal when the Exolution application is interfaced with a project. All these fields have names starting with "Exo". The fields are intended to track the processes and store document status and errors. These fields should not be used for any purpose other than those associated with Exolution. The following lists the fields with data types, purpose, and update time.

Fields

Type

Field Descriptions / Purpose

When is the field updated?

EXO_RED_NATIVE1

Long Text

Native Text Set Path - Field Created once a text set is created in Exolution and added automatically to "Exolution Field Profile."

The field updates when a text set is created for a document. The field updates when Create Redacted Documents is run. The process overwrites any data existing in this field.

EXO_RED_TEXT1

Long Text

Text Set Path - Field Created once a text set is created in Exolution and added automatically to "Exolution Field Profile."

The field updates when a text set is created for a document. The field updates when Create Redacted Documents is run. The process overwrites any data existing in this field.

Exolution Cluster ID

Integer

Assign identifiers to each document cluster.

The field updates when the users update it from the Exolution interface after creating clusters.

Exolution Error

Long Text

Logs error information when a document fails to add to Exolution or fails to create a redacted file.

The field updates when Add to Exolution or Create Redacted Documents is run.

Exolution File Locked

Boolean (YES/NO)

Indicates whether or not a document is locked.

The field updates when clicking the Lock Documents or Unlock Documents button on the Swap Files tab.

Exolution File Version

Long Text

Stores the redacted file version when the user chooses the Separate Redacted Document option.

The field updates when the user runs a create redacted document function.

Exolution Flattening Status

Long Text

Flattening Status.

The field updates when a file is successfully prepared for redaction.

Exolution Freeze Redaction

Text

Stores the type of freeze applied to the document. Empty if no redaction freeze is applied.

The field updates where time creates or deletes real-time redaction (s). It can also be updated using the button Update Has Redaction on the Review Functions tab and clicked.

Exolution Has Redactions

Boolean (YES/NO)

If this has value Yes when the document has redaction(s). The file is beneficial to QC whether the intended documents have redactions.

The field updates when users create or delete redaction(s) in real time.

Exolution Last Redaction Modified By

Long Text

Stores the name of the user who last created or deleted a redaction.

The field updates in real-time when users create or delete redaction(s).

Exolution Last Redaction Modified Date

Date Time

Stores date-time when last time redaction was added or deleted.

The field updates in real-time when users create or delete redaction(s).

Exolution Original Document

Long Text

Stores the type of freeze applied to the document. Empty if no redaction freeze is applied.

The field updates when the user runs a create redacted document function with the load option Separate Redacted Document.

Exolution Processing Status

Long Text

Indicates the processing status of the document for those added to Exolution. If the field has any value other than Success, then the Add to Exolution has failed for that document.

The field updates when the documents are added to Exolution. The value is changed multiple times when the process of adding is running. If the document is successfully added at the end, then the field's value will be Success.

Exolution Redacted File Created

Boolean (YES/NO)

Indicates whether a redacted file is created for the document. It is helpful to QC the documents before running production to ensure that the redacted files are created for the documents.

The field updates when the Create Redacted Documents function is run.

Exolution Redacted File Created Date

Date Time

Stores date-time when the last time a redacted file was created for the document.

The field updates when the Create Redacted Documents function is run and redaction(s) are added to or removed from a document.

Exolution Redacted File Current

Boolean (YES/NO)

A flag that indicates whether or not any redactions are changed in a document after a redacted file is created last time.

The field updates when the Create Redacted Documents function is run.

Exolution Redacted Group ID

Integer

This relational field stores the artifact ID of the original unredacted document for the unredacted documents and their redacted copies. This value helps to search the original documents and all their redacted copies from the saved search.

The field updates when the Create Redacted Documents function is run.

Exolution Redaction Burn Status

Long Text

Redaction Burn Status.

The field updates when the Burn In Documents function is run.

Exolution Redaction List

Long Text

A list of the redactions applied on the document.

The field updates when the Create.
The redacted Documents function is run. The
Process overwrites any data existing in.
this field.

Exolution Redaction Type

Long Text

Exolution Redaction Type.

The field updates when a redaction is applied to the document.

Exolution Review Complete

Boolean (YES/NO)

Indicates whether or not the document is marked reviewed.

The field updates when a document is marked reviewed.


ESC

Eddy AI, facilitating knowledge discovery through conversational intelligence