Run a GenAI Review

Overview

aji’s GenAI Review is a complete, end-to-end review of your dataset by GenAI only. This is more expensive and takes longer than Hybrid GenAI Review, but with the added benefit of receiving reasoning for all of your reviewed documents and citations for documents rated positive (“Yes” or “Strong Yes” GenAI rating).

GenAI Review is performed by following the below steps:

1. Select Documents for Review – Create a target data population to use for review.

2. Start GenAI Review – Perform a GenAI Review on your target data population.

3. Interpret GenAI Review Results – View the results of your GenAI Review and determine next steps.

I. Select Documents for Review

Rather than performing a GenAI Review on all the documents in your project, you can create a target data population that excludes documents you do not want reviewed by GenAI. See our aji Data and Environment Requirements article for some ideas on documents to filter out of your target data population.

II. Start GenAI Review

Important

An individual GenAI Review is limited to a maximum of 150,000 documents, and reviews at a rate of up to 3,000 aji Units per hour. Multiple GenAI Reviews can be completed to accommodate larger datasets, but your budget and timeline should be considered beforehand.

1. From the GenAI Reviews module, click Add. The GenAI Review modal will appear.

2. Name: Name your GenAI Review.

   • Make sure you use a naming convention that makes sense to you and for easy reference across all your reviews.

3. Review Mode: This selection will be locked on GenAI Review Only.

4. Documents to Review: Select which documents you want aji to send to GenAI for review.

   • Use existing document selection – use documents from a previous review.

     1. From the dropdown, choose which aji Review you want to reuse documents from.

   • Create new document selection – use a new set of documents from a Saved Search or Work Folder (e.g. target data population).

     • All Documents – Review all documents in your project. This option is not recommended, since there may be documents you want to exclude from the GenAI Review.

     • Saved Search – Pull your documents from a pre-existing saved search. The Saved Searches modal will appear.

       

       1. Choose a saved search. You can click the > arrow to expand and view the saved search as it would appear in the Review Grid search bar.

       2. Click SELECT.

     • Work Folder – Pull your documents from a pre-existing work folder.

       1. Use the > arrow to find and then choose a work folder.

5. Click START REVIEW.

   • If you don’t have enough aji Units left in your project to start your GenAI Review, it won’t start and you will be notified via a popup. See our aji Unit Allocation – Instance Admin article for more details on assigning aji Units to projects, and our aji Billing article for details on how to estimate your aji Unit usage for a single Calibration, GenAI, or Hybrid GenAI Review.

III. Interpret GenAI Review Results

GenAI Review results can be interpreted through the GenAI Review data visualizations, which are very similar to the Calibration Review data visualizations.

GenAI Review Data Visualizations

Your GenAI Review results are populated in the GenAI Reviewed donut wheel, which is visible in the Results tab.

You can navigate the tabs above the donut wheel for a closer look at your rating metrics. Notably, the Documents tab lets you navigate to Document Preview, where you can read your document’s reasonings and citations for the GenAI’s ratings.

For an in-depth explanation of all the above data visualization tabs (Results, Report, Documents, Definitions, and Context), please see our Calibrate aji article.

Perform an Elusion Test

To verify the accuracy of your GenAI Review results, you can perform an elusion test on the documents in your review.

Instructions on how to perform an elusion test can be found in our Evaluate Classifier: Elusion Test article.