Upload Data

Upload jobs will create a corresponding import and export job within Discovery Manager. These can be monitored within the Discovery Manager Environment Module.

Administrators and their designees can upload unprocessed data directly into Reveal by selecting Upload from the Navigation panel. This will quickly load relatively small batches of documents without requiring a visit to the Reveal Review Manager or to Reveal Processing (Discovery Manager). The data uploader can accept virtually any file type, including but not limited to container files such as .zip and .pst files, and forensic images such as EnCase images.

Note

Instance Administrators must have a role with appropriate permissions assigned within a project in order to use Upload.

Creating an Upload Job

1. Within a project, click the Uploads button from the sidebar navigation.

2. This will open the Uploads screen.

   

3. Click the +UPLOAD button to launch the Upload dialogue. This tool is divided into three steps: Setup, Files, and Notifications. These steps are detailed in the following sections.

Upload Job Configuration

Setup

Provide information and make selections for the fields and Setup options detailed below.

1. Job Name: A recognizable name for logging and tracking the upload job; this item is required. The Upload job name in this web interface is also known as the Import job name within Discovery Manager.

2. Description: Additional information about the upload.

3. Files: Choose the type of the files.

   1. Native Files: Original files in their default format, such as Word documents, PDFs, or spreadsheets.

   2. Load Files: Pre-processed files, often used for importing structured data.

   3. Images: Visual files, including JPEGs, PNGs, and other graphic formats.

4. When selecting Native Files.

   

   1. Workflow: Choose the desire processing workflow.

      1. Upload + Process + Promote to Review: Files will be uploaded, processed, and automatically promoted to review.

      2. Upload + Process: Files will be uploaded and processed but not promoted to review. This allows for further actions to be ran against the data prior to promotion.

      3. Upload Only: Files will be uploaded, not processed and not promoted to review.

   2. Duplicates: This checkbox specifies if duplicate files will be included in the upload without any filtering or removal.

5. When selecting Load Files.

   

   1. Method: Choose the method by which the file will be processed.

      1. Add Documents: Add the file as a new document without affecting existing data.

      2. Overlay Data: Overwrite existing data with the new file. The system will replace the current information with the content from the newly uploaded file.

   2. Files to Load: Choose which parts of the file will be loaded.

      1. Metadata: Load only the metadata associated with the file. That includes information such as file name, author, date of creation, and other document-specific details, without including the actual content or rendering.

      2. Metadata + Renderings: Load both the metadata and the renderings (the actual content or visual representation) of the file.

   3. File: Choose the file you want to upload.

   4. Delimiter Profile: Choose the delimiter used in the file to separate data.

      1. Concordance: It refers to mapping the fields in the loaded file to the corresponding fields in the system.

      2. CSV: If the file is in CSV format, ensure it adheres to the standard CSV format for proper upload.

      

   5. Data Language: Choose the language in which the data is written to ensure that the system processes the data correctly, accounting for language-specific formatting, characters, and parsing rules.

   6. Indexing: This checkbox specifies if the data will be indexed after the upload.

6. When selecting Images.

   

7. Image File: Choose the image file you want to upload. The supported formats are Opticon (.OPT) and IPro Load File (.LPF).

8. Image Path: Choose the folder that refers to the location or directory where the image is stored.

9. Unique ID field: Select the unique ID field that will used to assign a distinct identifier to each image, ensuring that every image can be tracked or referenced uniquely in the system.

10. PDF Set: If the image is part of a set of PDF documents or related images, this option links the image to a PDF set for better organization or processing.

    

    1. Existing PDF Set: Select the appropriate PDF set from the list to link the image to an existing set.

    2. Create New PDF Set: If you want to create a new PDF set, click the + button next to the PDF Set field. This will allow you to create a new PDF set and link the image to that newly created set for better organization.

       

11. Overwrite: Choose how existing PDFs are handled during the upload process. It applies only if an existing PDF set is selected in the PDF Set field. These options are not applicable if you are creating a new PDF set.

    1. Do not overwrite if PDF exists: If a PDF file of the same name already exists, it will not be overwritten.

    2. Overwrite existing PDF if no Redactions and Annotations: If the PDF does not contain any redactions or annotations, the system will allow overwriting.

    3. Overwrite existing PDF even with Redactions and Annotations: In the PDF contains redactions or annotations, the system will allow overwriting.

Upload Process Based on File Type

Native Files

After setup, native files go through the following steps: Files and Notifications.

Files

The following instructions guide you on how to select and organize data for processing when Native Files is chosen during the data setup before uploading.

Selecting Folders

1. In the Files step, initially there is a Select Files and Select Folders… button displayed, click it to launch the local file browser.

   
   

2. Choose the local folders to upload. The screenshot below shows a folder named “A to F” being selected in MacOS.
   

3. After making a folder selection, there may be a permission prompt from the web browser to view files on the machine, click View files to grant permission.
   

4. Currently, multiple folders cannot be selected at a time. However, additional files and folders still be added if needed using the + Add Files or + Add Folders button after each selection.

   

Organizing Data

After selecting a folder, subfolders will be populated with respect to the folder hierarchy. There are three fields that can be assigned to the selected data from the dropdown: Custodian, Location, and Source.

1. If there are no existing entries in the dropdown, users have the option of creating them in this step. Choose Add Custodian, Add Location, or Add Source to create an entry.
   

2. Provide the information for each of the fields shown below when creating an entry.

   1. Custodian: The entity in whose possession, custody or control a document was collected.

   2. Location: Indicates from where a document was collected (e.g., HQ, Melbourne Branch, MG Home Office or an indicative code).
      

   3. Source: Records the point of origin of the document as collected at the Location.
      

3. After creating a desired entry, it can be selected from the dropdown.
   

4. The checkbox selection specifies what data will actually be processed before the job is submitted. After making selections, click Next to procede.
   

Notifications

The third and final step is to configure notifications. The Notifications screen is where teams or users are set up to be notified after and Upload job is completed. The user submitting the Upload job is automatically selected.

1. In the Teams and Users tabs, select either the teams, individual users, or both to be notified when the upload job completes.
   

2. The Upload Health Check modal performs three important prior to the upload process starting.
   

   1. S3 Bucket: Confirms that the storage location for this project is ready and accessible.

   2. Project Link: Ensures a proper connection between the Processing and Review databases.

   3. Firewall: Enables firewall access in the network, allowing files to move to storage.

3. After clicking Submit, the file(s) will be added and the upload job will commence. A progress bar is displayed to provide updates. There are also controls to Pause or Cancel a job from this screen.

   

4. After the upload finishes, the Status will as Completed.
   

5. The Uploads page will also list all jobs along with the completion status.
   

Load Files

Load files require additional steps to ensure correct data mapping before analysis. The following steps after setup are: Fields and Analysis.

Fields

1. Field Mapping: In the Fields section of the Fields step, you can select a previously saved or existing field mapping template.

   

   1. Alternatively, you can create a new mapping by clicking the "+" button.

      

2. Preview: Displays a preview of how the data will be mapped before finalizing the process.

   

   1. The mapped data before getting uploaded.

      

3. Auto-Match: Automatically matches fields in the uploaded file with predefined system fields.

   

4. Clear All: Resets all selected field mappings, allowing the user to start fresh the configurations of the fields.

   

5. Download: Exports the current field mapping configuration as a file for future use.

   

6. Project Field: Select a unique identifier for the file that will be uploaded.
   

Once the configuration of the fields is finished, you can click “Next” to the next step.

Analysis

The third and final step is the Analysis step, where the necessary fields are already configured in the system through the Fields step.

There is a notification indicating that the analysis is complete, along with two column below:

• Step: The name of the file where the records will be kept.

• Total: The number of rows.

After clicking Submit, the file(s) will be added and the upload job will commence. A progress bar is displayed to provide updates.

The updated page now displays the overall status of the upload process by organizing the progress into three sections:

• Uploaded Files

• Preparing Documents

• Indexing

Alongside the upload status process, there are two links below:

• Download Log: Allows users to download a file containing the processing logs for the import, which provides an overview of the actions taken during the process.

• Download Errored Records: To download the file that includes records that failed validation or import. This allows you to review and correct the issues before re-uploading the records.

  

Images Files

Since images do not require field mapping, the process is simpler. After setup, there is only one additional step: Analysis

Analysis

The selected images have been analyzed. A notification indicates that the analysis is complete. Clicking Submit, the file(s) will be added and the upload job will commence.

Finalizing the Upload

After completing the necessary steps for each file type and structuring the upload process this way, users can efficiently handle different file types while ensuring accuracy and consistency in data management.