How to 'Ask' Properly

Introduction

Ask is Reveal’s first search feature powered by generative AI (Artificial Intelligence). Joining a robust array of AI features seamlessly woven into Reveal’s state-of-the-art eDiscovery platform, Ask integrates with existing workflows to increase effectiveness, enhance efficiency, and reduce cost.

Designed for discussions. Using Ask is like having a discussion with your data. You start by posing a question – a query, really. Ask responds with a narrative answer accompanied by supporting materials. If the answer does not seem to be what you were looking for or the support is not solid enough, you can refine your question – just as you might when talking with a witness. Similarly, you might start with broader questions to get an overview and as you get insights, tighten your focus.

With Ask, you pose queries about the contents of your documents and receive back narrative answers based on the most semantically similar content. Additional instructions can be included; the system uses these when it prepares its answers.

Built for precision. Ask is built for precision; it is designed to help you quickly get key information you can use to learn more, make decisions, or take actions. Ask is not built for recall; It is not meant to find all of anything for you. Ask also is not built to make decisions for you; what you do with that information is for you to determine.

Meant for the life of a matter. Ask is designed so that you can use it at any point where you want to find useful information in the data available to you. You can use it as soon as you have data available to you in Reveal, to gain a better overall understanding of your matter. You can use it throughout the matter as you develop witness outlines, prepare motions, put together discovery requests or responses, get ready to take or defend a deposition – whenever you want to work with your data.

How it works. With Ask, you pose queries about the contents of your documents and receive back narrative answers based on the most semantically responsive content. Additional instructions can be included that the system uses when it prepares its answers.

When you pose a query to Ask, you are requesting that the platform:

• Perform a semantic search of all or some of the project's documents that is based on your query;

• Order the results of your search based on their semantic similarity to your query;

• From the most semantically similar content, generate a narrative answer to your query; and

• Below the answer, provide information about the most highly ranked documents and snippets of information in those documents, with links to the documents.

A suggested workflow. Although using Ask is simple, for the best results we recommend the following general workflow.

Certain steps mentioned above may be iterative. For instance, the user can refine and resubmit questions repeatedly until Ask returns satisfactory results.

Workflow Steps

Start Using Ask

To start using Ask, open the Ask popup window. To do this, click on the GenAI icon on the right side of the Search Box.

Once you open the Ask popup window, you can close it by clicking on the “X” in the upper right corner of the window. This removes the window from view but does not end your Ask session; click on the wand icon again and the popup window re-opens and you are brought back to where you left off.

If you want a larger Ask popup window, you can make it larger or smaller by dragging the right or bottom sides or the bottom right corner of the window. You also can click on the “Enter Full Screen” icon to the left of the “X”. To exit the full screen view, click on the “Exit Full Screen” icon.

1. Set Scope

Ask lets you choose the scope of documents it searches for information:

• You can have Ask search your entire dataset, or

• You can have Ask limit its search to a subset of those documents.

You can switch between these options as much as you like while you use Ask to interrogate your data.

• Search Entire Dataset

By having Ask work with the contents of your entire dataset, you perform the broadest searches. To do this, make sure you have selected “All Docs” under Search at the bottom of the Ask window.

This increases the chances that you will locate useful information you might not otherwise find. Think of this approach as a broad sweep through your documents.

At the same time, searching a larger dataset does not slow you down; whether you have Ask search 100 documents or one million, it returns results in about 20 seconds.

• Search Subset of Dataset: Current Results

To have Ask search only a subset of your documents, first run a search in Reveal and then select “Current Results” under Search at the bottom of the Ask window.

By narrowing your Ask searches to subsets of documents, you can get more focused answers and more targeted sets of supporting documents. Think of this as your deep dive.

Here are three examples of how you might narrow your dataset before using Ask:

• Limit your Ask queries to the content in email communications that took place during a specific timeframe. Before submitting queries to Ask, do a search in Reveal to get a set of data that meets your criteria. You might, for example, go to the Dashboard, click on “msg” under “Extension” and use your mouse to select the years you care about.

• Limit your Ask queries to data from a limited number of custodians. Before submitting queries to Ask, select the custodian whose data you care about using a Custodian filter, via the Custodian chart, or with a search.

• Limit your Ask queries to documents containing words or phrases from a keyword list. Using Term List Search, add up to 1,000 terms to a list where terms can be individual words, Boolean searches, and searches of data from any field in your database. Preview document and family counts and modify the list, if you want, before running your search. When you are ready, run the list as a search.

With each example, you create a targeted dataset you can use as a starting point for Ask.

2. Craft and Submit Query

Once you have identified the scope of the documents you want Ask to search, craft, and submit to Ask your query and any additional instructions you want Ask to use.

• Craft Question

In principle, there is no limit to the types of queries you can submit to Ask. In practice, the more thoughtfully crafted your queries, the more likely you are to get useful answers.

The following guidelines provide suggested practices for creating more effective queries and avoiding less effective ones.

More effective queries for Ask:

• Queries that pose a single question. It is better to use queries that pose single questions rather than queries that pose compound ones, just as you should during a deposition. If the question you have in mind has two parts, ask each part as a separate query.

• Queries that are explicit and specific. The more specific and explicit you are with the queries you pose to Ask, the more likely you will be to get helpful responses. It helps to avoid ambiguity, such as you might create using pronouns.

• Queries whose scope is aligned with the scope of your interest. If you want an overview, ask a broader query. If you want to drill into details, prepare a narrower query.

Less effective queries for Ask:

• Queries that ask for “all” of something. Ask is designed for precision, not recall. If you use the query, “What were all the steps Enron took to improve profit?”, Ask might reply with an answer stating, “Based on the documents, the key steps Enron took to improve profitability included....” There can be, of course, a substantial difference between the “all” in your query and the “key” in the response.

• Queries that require intuition to answer. Ask returns answers based on the content it finds in the data it searches. It is not designed to make the types of intuitive leaps humans automatically do.

• Queries that require information from metadata. In its current iteration, Ask works only with the content of documents and not the documents’ metadata.

• Queries whose answers require analysis of many documents. Ask prepares narrative information only about the contents of a limited number of documents. If you want a summary of changes in a product’s price across 200 spreadsheets, Ask is not well suited to return a useful answer.

• Craft Any Additional Instructions

With Ask, you can enter additional instructions that let you give Ask specific directions on how to format the response it provides to your query. You might instruct Ask to provide its answer in bullet points or translate it to another language. You also can explore the extent you can make effective use of additional instructions to have Ask provide additional details in its response, organize content chronologically to the extent meaningful dates are present in the content searched, or prepare a list of search terms.

Please remember – the additional instructions only apply to the narrative generated by the top few most semantically similar documents and not all documents found by Ask.

Here are a few examples of additional instructions:

• “Provide answer in bullet points format.”

• “Give answer in French.”

• “After answer, provide a search term list using Boolean operators and put quotation marks around any phrases.”

To add additional instructions, click on More Details and type or paste your additional instructions into the box in the Ask window.

• Submit to Ask

To submit your query to Ask, press the Enter key or click on the up arrow to the right of your query.

If Ask does not find any responsive content, it will let you know that in just a few seconds. Otherwise, Ask should return a narrative response with supporting materials in about 20 seconds.

3. Refine

Ask is designed to be used iteratively. Pose a question and evaluate the response. Refine the question, pose the refined version, and evaluate its response. Continue the refine-evaluate cycle until you have a satisfactory result.

• No Refinements

Sometimes, your first question will return an answer that meets your needs. When that happens, move to the next step: validating the response.

• Minor Refinements

At other times, you might feel that the answer is close to what you are looking for but not there. When that happens, rephrase your query as even a subtle change in wording might yield different results.

• Major Refinements

Yet other times, the answer might seem incorrect, incomplete, or out of line with what you were looking for. There are several reasons why this might happen. They include:

⇒ The answer is not in your dataset.
⇒ The answer is not in the subset of data you are using.
⇒ The answer is in too many places in the data you are using.
⇒ Your query is too narrow.
⇒ Your query is too broad.
⇒ Your query is not designed to return an answer you are trying to get.

When that happens, you can refine your approach. Here are some ways to do that:

• If you have narrowed your scope (you are searching for a subset of dataset), expand or narrow your scope.

  • Note that Ask indexes only unique and pivot documents. Expanding your scope to include duplicates allows it to search more documents.

• Rephrase your query. Even a subtle change in wording might yield different results.

• Use a narrower query to get a deeper, more focused answer.

• Use a broader query to get a broader, high-level answer.

• If you are searching for the entire database, narrow your scope.

• Check your data using other search methods such as keywords, concepts, or date ranges to determine whether your dataset has information Ask can use to prepare a response.

4. Validation

It is important that you validate the answers you get from Ask, just as you should validate the results of any research you perform.

Ask generates each answer based on the contents of References found in Sources. A Source is a document. A Reference is a short passage of information from the text of from a Source where the content of the Reference is semantically similar to the query you submitted. Multiple References can come from a single Source. This approach allows Ask to give you succinct, targeted answers.

As with any approach that summarizes content, an answer you get from Ask might not reflect all information from your documents on a single topic. As a result, it might not be complete, might not be correct, or might not deliver the nuance you need.

For example, where there is a long email discussing employee John Doe’s performance from this year and last year, and John did well last year but poorly this year, if the question is “Did John do well with his annual performance?”, Ask could use only last year’s performance results and return that “John did well” instead of mentioning he did it poorly this year.

Hence the importance of validation.

• Validating with Sources and References

To help you validate results, Ask gives you access to the top 100 Sources and References found in response to your query.

A Source is a document. A Reference is a short passage of text from the Source, where the text of the Reference is semantically like the query you used. Each Source contains at least one Reference.

If you click on Show, you can see the Sources and References. Ask displays the Sources in descending order of the semantic similarity score of the first Reference from each Source. Ask shows an abbreviated version of each Reference; you can click on More to see the full Reference.

Some Sources contain more than one Reference. When that happens, Ask displays the References for the Source in descending order of semantic similarity.

You can use the Sources and References to help you check the accuracy of Ask’s summaries. You can do this several ways:

• Read the References. It might be sufficient to read just the first portion of a Reference, the part initially shown.
   
  You might want to click on More to see the entire Reference.
   

• Read the Sources. You might decide you want to read the document itself, not just the Reference. To go to the document, click on the “ ” next to the Reference score.
   

• Add Sources to Search. You can add any combination of Sources a Reveal search. To add a single Source, right-click on the three dots in the document panel, then select “Add to search”.
   
  To add selected Sources, uncheck the box for all Sources, then check the boxes next to the Sources you want to add to your search and select the ”Add to search” option.
   
  To add all the Sources, make sure the all-sources button is checked and then click the link to add source docs to Search.
   

• Validating with Searches

Reveal provides a wide array of search capabilities other than Ask, ones that use different methodologies to help you locate information. You can use one or a combination of these other search methods to test the results you get from Ask.

• Validating with External Information

It can be useful to turn to outside resources to test the accuracy, reliability, and completeness of answers you get from Ask. You might, for example, talk to individuals, review interview notes, or examine documents not included in your dataset.

5. Review, Tag, Revise, Write and More

Once you have a useful response, it is time to fit your question, the Ask response, and the source information into the workflow for which you undertook this exercise.

As discussed above, while you are in Ask you can use the checkbox next to each document to add that document to a search. From there, you can save that search and use it just as you would any other search in Reveal. You could, for example, use it as part of a review workflow, to designate a set of documents for review, for a specific topic, or by a person with a particular expertise. Or you could, if you wanted, use the results to jump-start Supervised Learning.

You can use the results of searches in Ask throughout the life of your matter. Areas where they can be useful include:

• Preparing and responding to pleadings.

• Preparing witness outlines.

• Preparing initial disclosures.

• Preparing and responding to formal discovery requests such as interrogatories and requests for production of documents as well as informal discovery requests.

• Preparing and responding to motion papers.

• Preparing preliminary and final reports.

• Preparing witnesses for interviews and depositions.

• Preparing deposition outlines.

• And more...