Processing - 2024.11 Release Notes
  • 04 Dec 2024
  • 8 Minutes to read
  • Dark
    Light
  • PDF

Processing - 2024.11 Release Notes

  • Dark
    Light
  • PDF

Article summary

Release Dates: 12/3/2024 - 12/5/2024

*Release notes are subject to change prior to the release deployment

Release Details

Type

Category

Description

New

Export

Users can now export project-level overview and snapshot statistics from the Processing Overview page. By clicking the export button, users can download a single CSV/XLSX file containing detailed information broken down by each import. This enhancement makes it easier to track and analyze project-level processing data in a convenient format.

New

Import

A Reveal processing user now has the ability to name import jobs similar to export jobs so that they can better keep track of their imports. This is an optional field and not required as it is with export jobs.

New

Production & Export

Users can now export processing job-level overview and snapshot statistics directly from the Processing Jobs page. By clicking the new export button, users will be able to download a single CSV/XLSX file that includes detailed information broken down by each import. This feature provides an easy way to analyze and track processing job stats in a portable format.

Update

Field Mapping

Review Promotion jobs now include the "Custodian," "Location," and "Source" fields. These fields are populated based on the values entered during Upload (Web) or Import (Discovery Manager) jobs. The "Custodian" field maps to "Custodian," the "Location" field maps to "Collection Location," and the "Source" field maps to "Collection Source" in Review.

Update

Import

A Reveal Processing user will now be able to export the ‘Import Name’ field populated in the import jobs to allow for better searching and tracking once documents are in Review so that the user can easily identify documents from a particular Import Job.

Update

Reports

Generating reports for cases with extensive data has been optimized in this release. For cases with over one million records and more than 1,000 search terms, users will now see a prompt to speed up the report generation process by converting the scope to a FileID List.

Update

Selective Set

The "Beg FileID" and "End FileID" fields will now be shown in the Selective Set and Export modules.

Update

Workflows

To enhance the user experience during uploads, the progress bar has been updated. Previously, the overall progress and individual file progress were combined into one visual element. Now, the overall progress will now be displayed using a donut progress bar, with percentage information shown in the center. And individual file progress will now show up to three simultaneous file uploads at once. If more than three files are being uploaded, the progress of completed files will be replaced by the progress of new uploads. Once an upload is complete, no progress will be displayed for that file anymore.

Fix

Document Review

An issue when Review Dataloader jobs is stuck in a "Pending" state, no further exports would complete from the processing side. This results in a bottleneck where subsequent jobs can not be processed, causing delays.

Fix

Document Review

Previously, when using the front-end uploader, the customer selected a Custodian to assign to the data being uploaded. However, after the data was processed and promoted to Review, only a subset of the documents had the custodian assigned. The issue has been resolved, and now the selected Custodian is applied to all documents during the upload process. Once the data is processed and promoted to Review, all documents will correctly reflect the selected custodian without the need for manual updates.

Fix

Document Review

Previously, the bulk assignment feature for Custodian, Location, and Source in the new uploader was not functioning as expected when more than one value existed in the dropdown. Specifically, when the first item in the list was selected (which is highlighted by default), the assignment would not apply to the folders. Now, regardless of the order of values in the dropdown, any selected value can be applied to multiple folders.

Fix

Export

The export settings are incorrectly handling single quotes when being serialized and saved to the database. Specifically, single quotes (e.g., '') in combined fields are being improperly escaped, which results in only one quote being stored instead of two. Now, the issue of escaping single quotes when saving data to ensure consistency with the original input has been resolved. Ensuring that FieldSQL and CombinedFields are handling and persisten data correctly without introducing errors.

Fix

Export

Previously, a null value in the avatar byte array could lead to an exception being thrown. This issue has now been addressed, and null values are properly evaluated and handled as expected, preventing any exceptions from occurring. An additional issue was identified where HTM files were not fully generated before an attempt to convert them to text was made. This led to errors during the conversion process. The issue has now been resolved by introducing additional handling to ensure the HTM files are fully generated before conversion.

Fix

Export

The client and support team have encountered repeated export failures when attempting to generate an overlay of documents for duplicate fields (dupepaths, custiwithdupes, etc.) in both Review(Overlay) and Overlay formats. The export errors suggest that the relevant fields are not fully populated in the system, leading to unsuccessful attempts.

Fix

Imaging

A fix was implemented to ensure the PDF imaging status remains as "Success" after completing a subsequent TIFF imaging job on the same dataset. The bar graph now accurately reflects the correct status for both PDF and TIFF imaging jobs.

Fix

Imaging

Users experience an issue after running a TIFF image job in processing, where the resulting images do not appear in the image preview area, even though the job completes successfully. This happens when navigating to the images tab, and no images are shown despite expectations.

Fix

Import

An error was logged stating "Index was outside the bounds of the array," occurring during the metadata extraction process. The job would fail, resulting in no metadata being added back in. After successful testing, it was confirmed that removed metadata and participants are re-added to the tables upon running the QC Batch Reprocessing, indicating that the issue has been resolved.

Fix

Installation & Maintenance

Users are encountering errors when attempting to replace an archive with a repaired version from S3 or local storage during the import QC process. Although the issue has been difficult to replicate consistently, it appears that the system throws an error upon reprocessing, suggesting that this may be a lingering bug related to file replacement processes.

Fix

OCR

An issue was identified in version 92520000 where users selecting "OCR During Import" and "OCR Index Errors" experienced inconsistencies. Although the OCR status showed as "SUCCESS" for all imports, documents still displayed "OCR pending" after initiating searches. This inconsistency originates from the stored procedure, impacting the search process and hindering the client's ability to produce results promptly. Steps will be taken to resolve this stored procedure, and will open a new story for review of all past imports with pending OCR statuses.

Fix

OCR

A bug was identified in Discovery Manager where applying templates or creating new ones sets the OCR timeout to excessively high or negative values instead of the expected 20 minutes. This issue has caused processing times to increase drastically, in some cases by up to 50 times.

Fix

Processing Job Status

A fix was implemented to ensure the job status, percent complete, and modified fields are correctly updated for all copy project settings jobs. Jobs now transition through appropriate statuses (Created, InProcess, and Complete or Error) with Percent_Complete set to 100 upon completion, and the Modified_by_user and Modified_Date fields properly populated.

Fix

Production & Export

RSMF exports were failing due to out-of-range exceptions when participant display data was enclosed solely within brackets. The export logic has been updated to handle these cases properly.

Fix

Production & Export

There was an issue where Teams data exports were failing to maintain sequential order between parent and child documents across load file chunks. This fix ensures that Teams message attachments are now exported sequentially with their parent items, preventing errors during Review Append exports.

Fix

Production & Export

In large exports following Australian Protocol, native and text file consistency has been improved. Previously, some documents, like emails and Word files, might not export fully under Australian settings. This update ensures that both Australian Protocol and Parent/Child exports now generate complete and validated files for native and text formats, improving overall export reliability, aligning with the results seen in Parent/Child exports.

Fix

Production & Export

A bug was found where the delete button for Export Templates was disabled, preventing users from deleting templates. Users had to go through a workaround by navigating to the Project Template section and returning to the Export Table for the delete option to become available.

Fix

Production & Export

An issue where the export process hangs indefinitely, and the PST files continue to grow in size without completing, suggesting repeated data entries into the PSTs. The system fails to provide clear error messages, leaving users without guidance on the issue.

Fix

Production & Export

Users are experiencing an issue where the DOCDISPLAYNAME field remains blank for non-email files during export, even though the necessary information exists in the FILENAME field. The expected mapping to the Review field "SUBJECT_OTHER" is not occurring as intended.

Fix

Project Administration

The 'Allow Duplicates' checkbox in the new uploader is not functioning correctly. When the checkbox is checked, the documents are not properly processed for duplicates. Specifically, while the upload, process, and promote jobs report success, the export job in Discovery Manager shows errors with the message ‘Error Loading Loadfile Into Review’. This error relates to an export containing 0 documents, despite the documents being promoted to Review. Additionally, the export job still shows ‘Promote to Review’ as the Deduplication setting, which enables the removal of duplicate documents, instead of ‘No Deduplication’.

Fix

Project Creation

An issue was identified where silent errors occurred in the Processing API during project creation, causing missing field mappings and subsequent upload failures.

Fix

Reports

A bug was identified where adding export file IDs to a Selective Set resulted in incorrect document and family counts being displayed, creating confusion for clients. Although the underlying data remains accurate, this visual discrepancy has been resolved to ensure that counts correctly reflect the intended items, thereby improving clarity in export operations.


ESC

Eddy AI, facilitating knowledge discovery through conversational intelligence