Indexing Holds

Prev Next

Index Data – Web

Indexing locations

In Reveal, there are multiple areas where you can index your data. Different areas provide different indexing functionalities.

  • Indexing in the Review Grid

  • Indexing in the Processing Grid – Processing Index is only metadata and OCR/Loading text sets. No other text sets apply here.


About Text Sets and Indexing

Discovery Manager processing: OCR / Loaded

OCR / Loaded is a default text set typically populated with one of two kinds of document text:

  • Extracted text from within the file.

    • Example: A Word file contains the exact textual contents written in its data, which can be extracted directly.

  • OCRed text

    • Example: A PDF file with no embedded text can be visually scanned by OCR to identify its characters. This may contain errors, depending on the quality of the image and of the OCR program.

Discovery Manager will process files in both ways. If the file has embedded text, it will extract it. If the file can be OCRed, Discovery Manager can run OCR. After Discovery Manager runs its processing engine, your files will either have:

  1. No embedded or OCRed text – This occurs if the file errored during both kinds of processing (e.g. it didn’t have text in it, like an mp3 file).

  2. Extracted text – OCR wasn’t possible (can’t be visually scanned).

  3. OCRed text – There was no embedded text to extract.

  4. OCRed and extracted text – Both processes were successful.

Discovery Manager can only choose to populate the OCR / Loaded text set with one of the two kinds of text. In the case where Discovery Manager successfully processed OCRed and embedded text, by default Discovery Manager will choose OCR over extracted. However, this setting can be changed in Discovery Manager to prioritize embedded text (during Export).

Reveal Web also provides the option to populate the OCR / Loaded text set during an OCR job run through the Review Grid. This will override any processed information done in Discovery Manager.
but it’s best practice to create your own custom OCR text set (see below) to supplement OCR / Loaded so that no critical pieces of data are lost from the original set.

Standard indexing workflows

There are three main ways to process data in Reveal. Each method has its own text sets that auto-index during processing, so further manual indexing is usually needed to generate all of Reveal’s pre-existing text sets for your data – Document_Metadata, Native View (PDF), Spreadsheet View, OCR / Loaded, Extracted, and Transcription.

Important

Rather than trying to generate Transcription or Spreadsheet text sets on your mixed data, it’s faster to isolate A/V files or Spreadsheet Files and perform separate Index jobs for each.

Method 1: Processing in Discovery Manager and Promote to Review

Discovery Manager

  1. Create Import (Processing) job: This processes your data.

  2. Create Review Append Export job: This will push your data from DM to Review, and auto-index the below text sets.

    • Document_Metadata, OCR / Loaded, Native View (PDF)

Reveal Web

  1. Run an Index job: Select the below text sets to index.

    • Extracted

  2. Isolate spreadsheet files: Use advanced search to search for just spreadsheet files.

  3. Run a second Index job: Select the following text set to index for spreadsheets only.

    • Spreadsheet View

  4. Isolate A/V files: Use advanced search to search for just A/V files.

  5. Run a third Index job: Select the following text set for your A/V files only.

    • Transcription

Method 2: 3rd-party import in Review Manager

  1. In Review Manager, process your 3rd-party data and load file.

    • No text sets are generated when processing 3rd-party data through Review Manager.

  2. In Discovery Manager, run an Index job on all documents. This will auto-index the following text set:

    • Document_Metadata

  3. In Reveal Web, run an advanced search that removes spreadsheets and A/V files.

  4. Run an Index job on your remaining data. Select the following text sets to index:

    • OCR / Loaded, Extracted, Native View (PDF)

  5. Isolate spreadsheet files through advanced search.

  6. Run an Index job on your spreadsheet files. Select the following text sets to index:

    • OCR / Loaded, Extracted, Spreadsheet View

  7. Isolate A/V files through advanced search.

  8. Run an Index job on your A/V files. Select the following text set to index:

    • Transcription

Method 3: 3rd-party import in Reveal Web

  1. In Reveal Web, process your 3rd-party data and load file. This will auto-index the following text sets:

    • Document_Metadata, OCR / Loaded, Spreadsheet View, Native View (PDF), Transcription

  2. Run an Index job on your data. Select the following text set to index:

    • Extracted


Indexing Requirements article

Indexing requirements

Certain jobs in Reveal either require, automate, or allow the option to index text sets after the process has occurred.

Required indexing

If you’re performing an import job through Review Manager, metadata text sets require indexing.

Job Type

Required Indexing

Import Job (Review Manager)

Metadata

Automated indexing

Certain jobs will automatically perform indexing on specific text sets:

*OCR can be sent to the OCR / Loaded text set, or new text sets can be created to store the OCR.

Job Type

Auto-Indexing

Import Job (Discovery Manager)

OCR / Loaded*, Metadata

Import Job (Reveal Web)

OCR / Loaded*, Metadata

OCR Job

OCR / Loaded*, Metadata

Transcription Job

Transcription

Translation Job

Translation

Image Labeling Job

Metadata

Note

Import jobs through Review Manager do not automatically index any text sets.

Optional indexing

Certain text sets can be indexed during the following jobs, but indexing is not required:

*OCR can be sent to the OCR / Loaded text set, or new text sets can be created to store the OCR.

Job Type

Optional Indexing

Import Job (Discovery Manager)

Native View (PDF), Spreadsheet View, Extracted

Import Job (Review Manager)

Native View (PDF), Spreadsheet View, Extracted, OCR / Loaded*

Import Job (Reveal Web)

Native View (PDF), Spreadsheet View, Extracted

OCR Job

Native View (PDF), Spreadsheet View

Transcription Job

Native View (PDF), Spreadsheet View

Translation Job

Native View (PDF), Spreadsheet View

Color Detection Job

Native View (PDF), Spreadsheet View, Metadata

Note

For Processing's "Review Append" exports (that is, native and other material ingested through Discovery Manager and not Review Manager), only the Document_Metadata & OCR Loaded will get indexed automatically through the API.

Items not initially indexed (other than Document Metadata, which must be indexed to populate the Grid) may be indexed in the Reveal Web environment. See Index Data – Web for more information.

Footer Design