Skip to content

Commit

Permalink
Merge pull request #18 from relativitydev/develop
Browse files Browse the repository at this point in the history
Trace Release 101
  • Loading branch information
mishakogan committed Oct 24, 2019
2 parents 2df6e65 + 6def939 commit f67d0b2
Show file tree
Hide file tree
Showing 63 changed files with 421 additions and 517 deletions.
6 changes: 3 additions & 3 deletions docs/bist_smoke_tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ It is recommended to run **BIST** on a new workspace or on a test workspace as
1. Navigate to Library Application (Admin), locate Trace and install it
into Test Workspace

2. Ensure that Trace Agent and all dependent Agents are enabled and working
2. Ensure that Trace Agent and all dependent Agents are enabled and working

3. Within the workspace, navigate to Trace → Setup and set Run Option →
Continuous
Expand All @@ -68,7 +68,7 @@ It is recommended to run **BIST** on a new workspace or on a test workspace as

4. Generate test data emails with attachments

5. Extract and Ingest 143 documents with various file types, language
5. Extract and Ingest 144 documents with various file types, language
contents across 2 data batches

4. Create Terms **[AUTOMATED]**
Expand Down Expand Up @@ -132,7 +132,7 @@ It is recommended to run **BIST** on a new workspace or on a test workspace as

- Trace BIST Data Source Generated Batches: `2`

- Total Documents in Trace BIST folder: `142`
- Total Documents in Trace BIST folder: `143`

- Total Deleted Documents: `1`

Expand Down
151 changes: 77 additions & 74 deletions docs/demo_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,35 +87,36 @@ DEMO A: Enabling Office 365 Data Source and auto-tagging documents (alerts)
2. Click into the **Trace:Rules** tab to see sample rules in the workspace.
Rules are how you define the type of content that is relevant to you. The
**Market Abuse** rule has been pre-created for you along with a few others.
![](media/cbcdbe80b994aff6e31fad731db5362e.png)

![](media/demo_guide/Section1.2-1571771958433.png)

3. Click into the **Market Abuse** rule and see the options that make up this
rule:

3. Click into the **Market Abuse** rule and see the options that make up this
rule:
- **Searchable Set:** A Saved Search that the rule runs against

- **Searchable Set:** A Saved Search that the rule runs against
- **Associated Actions:** A set of actions to take on matched documents (this
rule will move the matched documents to a separate folder)

- **Associated Actions:** A set of actions to take on matched documents (this
rule will Tag and Batch out to reviewers)
- **Terms**: Allows you to link Trace Terms to refine your document filter
criteria with an e-discovery grade searching engine (dtSearch)

- **Terms**: Allows you to link Trace Terms to refine your document filter
criteria with an e-discovery grade searching engine (dtSearch)
- The rule is currently **Disabled**
![](media/demo_guide/Section1.3.png)

- The rule is currently **Disabled**
![](media/0c099bc9669ce508b9a1110a66cd15ca.png)
4. Click the **Enable Rule** console button under **Trace Rule Management** section:

4. Click the **Enable Rule** console button under **Trace Rule Management** section:
- The rule is now **Enabled**

- The rule is now **Enabled**

- As data is ingested, this rule will be automatically applied to all incoming
documents
![](media/9121eef9fa793b0003021602cefabbe2.png)
- As data is ingested, this rule will be automatically applied to all incoming
documents
![](media/demo_guide/Section1.4.png)

### Section 2: Importing Data and Running Rules

1. Now let’s test our rules by adding live stream of data into the workspace
with Office 365 Data Source. Go to the **Trace:Data Sources** tab
![](media/eae00fa728b99218930da021e4f58a6c.png)
![](media/demo_guide/Section2.1.png)

2. Click on the **Office 365 Exchange** data source to see how it is configured

Expand All @@ -132,69 +133,68 @@ DEMO A: Enabling Office 365 Data Source and auto-tagging documents (alerts)
- **Trace Monitored Individuals:** lets you select which people to monitor

- The Data Source is currently **Disabled**
![](media/0b80dfed320b157afa02f114b3fbd7c1.png)
![](media/demo_guide/Section2.2.png)

3. Click **Enable Data Source** on the right hand side
![](media/e0b725fd32e467f47cf879fa55a0a7b1.png)
![](media/demo_guide/Section2.2a.png)

4. Wait for Data Batches with data to start showing up – refresh the page
several times, until you start seeing Data Batches on the bottom (it may
take several minutes for first data batch to be fully created). You can see
how much data each Data Batch retrieved and corresponding statuses. Once you
have a few Data Batches in “Completed” state the rest of the Trace workflow
automatically triggers.
![](media/888c38c3f153691242bd9fb200911bd4.png)
4. Data will begin to get pulled from the exchange server for each of the defined Monitored Individuals. Refreshing the page will show data batches as they are pulled from the data source and processed (it may take several minutes for first data batch to be fully created). You can see how much data each Data Batch retrieved and corresponding statuses. Once you have a few Data Batches in “Completed” state the rest of the Trace workflow automatically triggers.

![](media/demo_guide/Section2.4.png)

As time goes on, more batches with data will be created and ingested:
![](media/c3ef4b4a771fbfd3d7b25c7dc5ccf4dc.png)

### Section 3: Viewing Results

1. Go to the **Review Batches** tab. This shows all Review Batches that were
auto-generated from the enabled rules
![](media/38c5bd780c5ada1f3042f8ac7052d37d.png)

2. Click on **Documents** tab:
1. Click on **Documents** tab:

1. Data is proactively being retrieved, extracted and ingested
1. Data is proactively being retrieved, extracted and ingested

2.**Trace Search Index**” is built automatically -\> for your ad-hoc
searching
![](media/69cbd4e87b452afd74276bba27660db4.png)
2. For this demo, all ingested documents are routed to the **Ingestion Folder**

3. Explore Dashboards
![](media/56a2973ce72db92e5bbb474fe8e7c216.png)
![1571775679888](media/demo_guide/1571775679888.png)

1. Once all the data is ingested and analyzed about 7.6% of ingested documents have
matched the rules:
![](media/b263a7470437bb562c35e35c0fc113ed.png)
3. Documents matched to the **Market Abuse** rule will have the **Default Move To Folder** action execute. The matched documents will appear under the **Alerted Documents** folder.

2. All of the Rules that matched:
<br>![](media/87545672cde91318af976d0f7705cc9a.png)
![1571775810589](media/demo_guide/1571775810589.png)

3. File type breakdown of ingested documents:
![](media/2915c8515dde4765959049040c23c5ef.png)
4. Navigate to the **Trace Demo** root folder. You will find the “**Trace Search Index**” is built automatically for your ad-hoc searching.
![](media/69cbd4e87b452afd74276bba27660db4.png)

4. Review matched documents
3. Explore Dashboards
![](media/demo_guide/Section3.3.png)

1. Click on “Market Abuse” on the Rules widget
![](media/7675cfedd1484500512efe27c13ee17f.png)
1. Once all the data is ingested and analyzed about 8.5% of ingested documents have
matched the rules:
![](media/demo_guide/Section3.3i.PNG)

2. Click on “Revenue Sharing” on the Terms widget
![](media/7284f63897ecd8c4c1d154be240a2039.png)
**(blank)** denotes the number of documents where no Rules were matched.

2. All of the Rules that matched:
<br>![](media/demo_guide/Section3.4i.PNG)

3. File type breakdown of ingested documents:
![](media/demo_guide/Section3.3iii.PNG)

3. Review matched documents

3. Click on one of the documents in the list:
![](media/57778d5767b262da54b15eff0d505ff5.png)
1. Click on “Market Abuse” bar on the Rules widget
![1571776936298](media/demo_guide/1571776936298.png)
2. Click on “Revenue Sharing” on the Terms widget
![1571777044365](media/demo_guide/1571777044365.png)
3. Click on the top document in the list:
![1571777162502](media/demo_guide/1571777162502.png)
4. Explore document details:
![1571777552542](media/demo_guide/1571777552542.png)
![1571777717924](media/demo_guide/1571777717924.png)

4. Explore document details:
![](media/42528d221727ff6d48bb9e844bfb0ffb.png)
![](media/c8e3e14b0d67dccba13753f24496b38a.png)
- Matched terms are automatically highlighted in the document

1. Matched terms are automatically highlighted in the document
- Document text and meta-data was automatically extracted

2. Document text and meta-data was automatically extracted
- Related documents are displayed in the “Family” section

3. Related documents are displayed in the “Family” section

### Section 4: Recap

Expand All @@ -203,12 +203,12 @@ Thanks for completing Demo A! Quick recap, here’s what we did:
1. Enabled **Market Abuse** rule for continuous evaluation

2. Viewed rules that are based on Saved Searches and Trace Terms that have
Tagging and Batching as associated actions
an associated action of moving the document to another folder.

3. Enabled ingestion of documents, proactively from live Office 365 instance

4. Validated that the documents were ingested into the workspace, automatically
indexed, searched, tagged and batched out for review
indexed, searched, tagged and moved

5. Reviewed a matched document in the viewer with extracted data, related
documents and highlighted terms
Expand All @@ -225,38 +225,41 @@ DEMO B: Running an Archive Rule
Search** browser in the **Documents** tab and select the “**Non-alerted
Documents**” Saved Search. You’ll notice there are \~5k documents that are
currently in this Saved Search.
![](media/eac385448f2a333626219dbd74f113c8.png)
> **NOTE:** Count on non-alerted documents might vary
![1571779546803](media/demo_guide/1571779546803.png)

> **NOTE:** Count on non-alerted documents might vary
3. Click into the **Trace:Rules** tab to see all rules in the workspace. The
**Delete Rule** has been pre-created for you. Click into the **Delete
Rule**.
![](media/406ada4e092e1e343bce785bebec7e9c.png)
![1571779609579](media/demo_guide/1571779609579.png)

4. You’ll notice that the **Delete Rule** follows a similar format to the prior
demo. The rule runs on the “Non-alerted Documents” Saved Search and has the
“Default Data Archive” action associated with it. This action will delete
all documents in the Saved Search (outside of the set retention policy) once
the rule is enabled. This will enable you to have an **automated retention
policy** in Relativity.

> **NOTE:** For this demo retention has be purposely set to 0 hours, which means that all of the `non-alerted` data will be
delete as soon as the rule runs

delete as soon as the rule runs
5. Now let’s enable the rule so Trace can start enforcing your Data Archive
policy. Go to the **Trace:Rules** tab again and click on the “**Delete
Rule**
![](media/406ada4e092e1e343bce785bebec7e9c.png)
![1571779615711](media/demo_guide/1571779615711.png)

6. Click on **Enable Rule** in console button under **Automation** section
![](media/603645b32fe247cfb851e292c7786beb.png)
6. Click on **Enable Rule** in console button under **Trace Rule Management** section
![1571779666116](media/demo_guide/1571779666116.png)

7. Go back to the Saved Search Browser in the Documents tab and select the
Documents to Delete” Saved Search. Within a few minutes and you’ll start to
Non-alerted Documents” Saved Search. Within a few minutes you’ll start to
see documents getting deleted from this workspace.
![](media/8c040117f5bbed702213148a2aaa0873.png)

![1571779733791](media/demo_guide/1571779733791.png)

8. Go to **Trace:Rules** , navigate to the Delete Rule and Disable it
![1571779820919](media/demo_guide/1571779820919.png)

8. Go to **Trace:Rules** , navigate to the Delete Rule and Disable it
![](media/59aecb6c0a0e49227506ca5d22afac87.png)

Let us know what you think! Reach out to `trace@relativity.com` with any
Let us know what you think! Reach out to `support@relativity.com` with any
feedback. Thanks!
Binary file removed docs/media/0b80dfed320b157afa02f114b3fbd7c1.png
Binary file not shown.
Binary file removed docs/media/0c099bc9669ce508b9a1110a66cd15ca.png
Binary file not shown.
Binary file removed docs/media/2915c8515dde4765959049040c23c5ef.png
Binary file not shown.
Binary file removed docs/media/38c5bd780c5ada1f3042f8ac7052d37d.png
Binary file not shown.
Binary file removed docs/media/406ada4e092e1e343bce785bebec7e9c.png
Binary file not shown.
Binary file removed docs/media/42528d221727ff6d48bb9e844bfb0ffb.png
Binary file not shown.
Binary file removed docs/media/56a2973ce72db92e5bbb474fe8e7c216.png
Binary file not shown.
Binary file removed docs/media/57778d5767b262da54b15eff0d505ff5.png
Binary file not shown.
Binary file removed docs/media/59aecb6c0a0e49227506ca5d22afac87.png
Binary file not shown.
Binary file removed docs/media/603645b32fe247cfb851e292c7786beb.png
Binary file not shown.
Binary file removed docs/media/7284f63897ecd8c4c1d154be240a2039.png
Binary file not shown.
Binary file removed docs/media/7675cfedd1484500512efe27c13ee17f.png
Binary file not shown.
Binary file removed docs/media/87545672cde91318af976d0f7705cc9a.png
Binary file not shown.
Binary file removed docs/media/888c38c3f153691242bd9fb200911bd4.png
Binary file not shown.
Binary file removed docs/media/8c040117f5bbed702213148a2aaa0873.png
Binary file not shown.
Binary file removed docs/media/9121eef9fa793b0003021602cefabbe2.png
Binary file not shown.
Binary file removed docs/media/b263a7470437bb562c35e35c0fc113ed.png
Binary file not shown.
Binary file removed docs/media/c8e3e14b0d67dccba13753f24496b38a.png
Binary file not shown.
Binary file removed docs/media/cbcdbe80b994aff6e31fad731db5362e.png
Binary file not shown.
Binary file added docs/media/demo_guide/1571430373664.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/media/demo_guide/1571775617378.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/media/demo_guide/1571775652697.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/media/demo_guide/1571775679888.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/media/demo_guide/1571775810589.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/media/demo_guide/1571776936298.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/media/demo_guide/1571777044365.png
Binary file added docs/media/demo_guide/1571777162502.png
Binary file added docs/media/demo_guide/1571777552542.png
Binary file added docs/media/demo_guide/1571777717924.png
Binary file added docs/media/demo_guide/1571779546803.png
Binary file added docs/media/demo_guide/1571779609579.png
Binary file added docs/media/demo_guide/1571779615711.png
Binary file added docs/media/demo_guide/1571779666116.png
Binary file added docs/media/demo_guide/1571779733791.png
Binary file added docs/media/demo_guide/1571779820919.png
Binary file added docs/media/demo_guide/Section1.2.PNG
Binary file added docs/media/demo_guide/Section1.3.png
Binary file added docs/media/demo_guide/Section1.4.png
Binary file added docs/media/demo_guide/Section2.1.png
Binary file added docs/media/demo_guide/Section2.2.png
Binary file added docs/media/demo_guide/Section2.2a.png
Binary file added docs/media/demo_guide/Section2.4.png
Binary file added docs/media/demo_guide/Section3.3.png
Binary file added docs/media/demo_guide/Section3.3i.PNG
Binary file added docs/media/demo_guide/Section3.3iii.PNG
Binary file added docs/media/demo_guide/Section3.4i.PNG
Binary file removed docs/media/e0b725fd32e467f47cf879fa55a0a7b1.png
Diff not rendered.
Binary file removed docs/media/eac385448f2a333626219dbd74f113c8.png
Diff not rendered.
Binary file removed docs/media/eae00fa728b99218930da021e4f58a6c.png
Diff not rendered.
Binary file added docs/media/user_documentation/1571073730733.png
Binary file added docs/media/user_documentation/1571073733941.png
Binary file added docs/media/user_documentation/1571079859530.png
Binary file added docs/media/user_documentation/1571081067899.png
Binary file added docs/media/user_documentation/1571081144550.png
Binary file added docs/media/user_documentation/1571087647200.png
Binary file added docs/media/user_documentation/1571088169759.png
Binary file added docs/media/user_documentation/1571088232926.png
54 changes: 50 additions & 4 deletions docs/release_notes.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,58 @@
Relativity Trace Release Notes
================================
- [11.2.11.1 (13 September 2019)](#112111-13-september-2019)
# Relativity Trace Release Notes

- [12.0.5.2 (24 October 2019)](#12052-24-october-2019)
- [11.2.11.1 (13 September 2019) - DEPRECATED](#112111-13-september-2019---deprecated)
- [11.2.10.1 (12 August 2019) - DEPRECATED](#112101-12-august-2019---deprecated)
- [11.2.9.2 (15 July 2019) - DEPRECATED](#11292-15-july-2019---deprecated)
- [11.2.6.1 (17 June 2019) - DEPRECATED](#11261-17-june-2019---deprecated)
- [11.2.4.4 (10 June 2019) - DEPRECATED](#11244-10-june-2019---deprecated)

# 11.2.11.1 (13 September 2019)
# 12.0.5.2 (24 October 2019)

**Relativity Compatibility**

- \> 9.6.202.10

**Features**

- New Data Enrichment Task performs data extraction independently of data retrieval, allows for better throughput and scalability of data ingestion pipeline
- Trace Agents (Manager and Worker) now fully support resource pools which allows to designate multiple agent servers to scale out Trace workflow as needed

**Enhancements**

- Rule Type field can now be specified per rule. This allows to tag different MO field depending on rule configuration. Workflow rule configuration are used to drive workflow within the Relativity workspace
Alert rule configuration is used for standard e-com/a-com identification
- Eliminated redundant query/read audits generated by Trace operations which reduces pressure on SQL server and audit infrastructure
- All actions associated with any rule now operate on net new data that is reflected on dedicated saved search
- RelativityScript (advanced) action now operates on the same saved search, yielding incremental data only
- Conceptual and Classification index automation now takes into account any in-progress activities that lock the index resulting in better robustness
- dtSearch indexes will now perform Full Build automatically (no user interaction) on newly created indexes that are subject to automation
- Data Transformation failures on set of docs (in data batch) now reports detailed per document failures and allows for partial import of successful transformations
- Application installation time is now significantly faster, per workspace upgrade logic now is done by the Trace Manager agent
- BIST testing coverage is now expanded to include:
- Explicit tests of successful Data Retention action execution
- Monitored Individual auto-discovery propagation
- Ingestion flow via Integration Points (as production)
- Monitored Individuals can now take advantage of Secondary Identifier to accommodate variable identifiers across multiple data sources (see documentation)

**Defect Fixes**

- Long text truncation of ingestion state for data sources with large number of Monitored Individuals when updating via UI
- Continuous BIST execution no longer generates orphaned resources and now correctly notifies of errors via Reporting task
- BIST import dependencies were updated to be compatible with RelOne Goatsbeard (10.3+) release
- Monitored Individual Auto-discovery no longer breaks the whole data batch on failures and reports errors on all extracted documents. Additionally, discovered monitored individuals are now properly associated with all documents that were extracted (including sub-attachments)

**Deprecated**

- Tag Action is now explicitly done as part of every rule evaluation and is not available in UI

**Upgrade Considerations**

- This is a **major** version change of Relativity Trace.

# 11.2.11.1 (13 September 2019) - DEPRECATED

> **DEPRECATION NOTE:** This version of Trace is **no longer supported**
**Relativity Compatibility**

Expand Down
4 changes: 2 additions & 2 deletions docs/trace_connectors_user_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ For every Trace Data Source, except for [Microsoft Exchange Data Source](https:/

### Installation Steps

Refer to `Merge1 6.0 User Guide.pdf` guide. Reach out to trace@relativity.com if you don't have access to this guide.
Refer to `Merge1 6.0 User Guide.pdf` guide. Reach out to support@relativity.com if you don't have access to this guide.

### Data Flow Overview

Expand Down Expand Up @@ -122,7 +122,7 @@ Refer to `Merge1 6.0 User Guide.pdf` guide. Reach out to trace@relativity.com if
1. MUST configure Email Alert Settings
2. Send Test Email
3. ![1570209109659](media/trace_connectors_user_guide/1570209109659.png)
8. For data source-specific instructions, Refer to `Merge1 6.0 User Guide.pdf` guide. Reach out to trace@relativity.com if you don't have access to this guide.
8. For data source-specific instructions, Refer to `Merge1 6.0 User Guide.pdf` guide. Reach out to support@relativity.com if you don't have access to this guide.
9. Configure `Importer Schedule` to run at a desirable frequency (daily is the most common frequency)


Expand Down

0 comments on commit f67d0b2

Please sign in to comment.