Run a Full Review

Early Access

This article has been published prior to aji’s GA date. Content may be subject to updates before its official release.

Overview

aji’s Full Review is a complete, end-to-end review of your dataset by GenAI. This is more expensive and takes longer than Hybrid Review, but with the added benefit of receiving citations and reasoning for all of your reviewed documents.

Full Review is performed by following the below steps:

1. Select Documents for Review – Create a target data population to use for review.

2. Start Full Review – Run a Full Review on your dataset.

3. Interpret Full Review Results – View the results of your Full Review and determine next steps.

I. Select Documents for Review

Rather than running a Full Review on all the documents in your project, you can create a target data population that excludes documents you do not want reviewed by GenAI. See our aji Data and Environment Requirements article for some ideas on documents to filter out of your target data population.

II. Start Full Review

Important

Individual Full Review runs are limited to a maximum of 150,000 documents, and reviews at a rate of up to 3,000 aji Units per hour. Multiple Full Review runs can be completed to accommodate larger datasets, but your budget and timeline should be considered beforehand.

1. From the Full AI Run module, click Add. The Review modal will appear.

2. Name This Run: Name your Full Review.

   • Make sure you use a naming convention that makes sense to you and can help with easy reference across all your runs.

3. Review Mode: This selection will be locked on Gen AI Review Only.

4. Documents to Review: Select which documents you want aji to send to GenAI for review.

   • Use existing document selection – use documents from a previous run within your current aji Workflow.

     1. From the dropdown, choose which aji run you want to reuse documents from.

   • Create new document selection – use a new set of documents from a Saved Search or Work Folder (e.g. target data population).

     • All Documents – Review all documents in your project. This option is not recommended.

     • Saved Search – Pull your documents from a pre-existing saved search. The Saved Searches modal will appear.

       

       1. Choose a saved search. You can click the > arrow to expand and view the saved search as it would appear in the Review Grid search bar.

       2. Click SELECT.

     • Work Folder – Pull your documents from a pre-existing work folder.

       1. Use the > arrow to find and then choose a work folder.

5. Click START REVIEW.

III. Interpret Full Review Results

After your Full Review completes, you can view your results by selecting its name from the Full AI Run List table.

Full Review Data Visualizations

Similar to Calibration Run results, your Full Review results are populated in the GenAI Reviewed donut wheel, which is visible in the Full AI Run Results tab.

You can navigate the tabs above the donut wheel for a closer look at ratings by individual document. Notably, the Agreement Report by Document tab lets you navigate to Document Preview, where you can read your document’s reasonings and citations for each Definition in your Definition Set.

For an in-depth explanation of all the above Data Visual tabs, please see our Calibrate aji article.

Perform an Elusion Test

To verify the accuracy of your Full Review results, you can perform an elusion test on the documents reviewed in your run.

Instructions on how to perform an elusion test can be found in our Evaluate Classifier: Elusion Test article.