Open Legal AI Workbench (OLAW)

AI + Legal APIs: A Tool-Based Retrieval Augmented Generation Workbench for Legal AI UX Research.

More info:

"Cracking the justice barrier: announcing the Open Legal AI Workbench". Mar 08 2024 - lil.law.harvard.edu

video.mov

Video: OLAW’s chatbot retrieving court opinions from the CourtListener API to help answer a legal question. Information is interpreted by the AI model, which may make mistakes.

Concept

OLAW is a tool-based Retrieval Augmented Generation (RAG) workbench for legal AI UX research. It consists of a customizable chatbot that can use legal APIs to augment its responses.

The goal of this project is to simplify and streamline experimentation with APIs-based RAG in legal contexts by:

Keeping it simple: The tool should be easy to operate, modify and interpret.
Being highly customizable and modular: Adding a tool to this workbench should be as simple as possible.
Being open and collaborative: A lot of this work generally happens behind the scenes. This project aims at amplifying collaborative research on the uses of AI in legal contexts.

The focus here is on ease of access and experimentation, as opposed to overall performance or production-readiness.

Tool-based RAG?

There are as many "flavors" of RAG as there are implementations of it. This workbench focuses on a tool-based approach, in which the LLM is indirectly given access to APIs as a way to augment its responses.

This process takes place in three steps:

Upon receiving a message from the user, the pipeline asks the LLM to analyze the message to:
- Detect if it contains a legal question
- Use a prompt to determine where to look for additional information (search target)
- Use that same prompt to generate a search statement to use against the search target
Upon identifying a search suggestion.
- The UI presents the search suggestion to the user and ask for confirmation.
Upon confirmation from the user:
- The pipeline performs the suggested search against the search target ...
- ... and uses the results as additional context when asking the LLM to answer the user's question

☝️ Summary

Installation

OLAW requires the following machine-level dependencies to be installed.

Use the following commands to clone the project and instal its dependencies:

# MacOS / Linux / WSL
git clone https://github.com/harvard-lil/olaw.git
cd olaw
poetry install

The workbench itself doesn't have specific hardware requirements. If you would like to use Ollama for local inference with open-source language models, be sure to check their system requirements.

☝️ Summary

Configuring the application

This program uses environment variables to handle settings. Copy .env.example into a new .env file and edit it as needed.

cp .env.example .env

See details for individual settings in .env.example.

A few notes:

OLAW can interact with both the OpenAI API and Ollama for local inference.
- Both can be used at the same time, but at least one is needed.
- By default, the program will try to communicate with Ollama's API at http://localhost:11434.
- It is also possible to use OpenAI's client to interact with compatible providers, such as HuggingFace's Message API or vLLM. To do so, set values for both OPENAI_BASE_URL and OPENAI_COMPATIBLE_MODEL environment variables.
Prompts can be edited directly in the configuration file.

☝️ Summary

Starting the server

The following command will start the OLAW (development) server on port 5000.

poetry run flask run
# Not: Use --port to use a different port

☝️ Summary

Recommended models

While this pipeline can in theory be run against a wide variety of text generation models, there are two key constraints to keep in mind when picking an LLM:

The size of the context window. The target model needs to be able to handle long input, as the pipeline may pull additional context from APIs it has access to.
Ability to reliably return JSON data. This feature is used by the /api/extract-search-statement route.

We have tested this software with the following models:

OpenAI: openai/gpt-4-turbo-preview (128K tokens context)
Ollama: Any version of ollama/mixtral(32K tokens context + sliding window)

We have observed performance with openai/gpt-4-turbo-preview during out initial tests, using the default prompts.

☝️ Summary

Interacting with the WEB UI

Once the server is started, the application's web UI should be available at http://localhost:5000.

The interface automatically handles a basic chat history, allowing for few-shots / chain-of-thoughts prompting.

☝️ Summary

Interacting with the API

OLAW comes with a REST API that can be used to interact programmatically with the workbench.

New to REST APIs? See this tutorial.

[GET] /api/models

Returns a list of available models as JSON.

Sample output

[
    "openai/gpt-4-vision-preview",
    "openai/gpt-4-0613",
    "openai/gpt-4-0125-preview",
    "openai/gpt-4-turbo-preview",
    "openai/gpt-4",
    "openai/gpt-4-1106-preview",
    "ollama/llama2:13b",
    "ollama/llama2:13b-chat-fp16",
    "ollama/llama2:70b",
    "ollama/llama2:70b-chat-fp16",
    "ollama/llama2:7b",
    "ollama/llama2:latest",
    "ollama/mistral:7b",
    "ollama/mistral:7b-instruct-fp16",
    "ollama/mistral:7b-instruct-v0.2-fp16",
    "ollama/mixtral:8x7b-instruct-v0.1-fp16",
    "ollama/mixtral:8x7b-instruct-v0.1-q6_K",
    "ollama/mixtral:latest",
    "ollama/phi:2.7b-chat-v2-fp16"
]

[POST] /api/extract-search-statement

Uses the search statement extraction prompt to:

Detect if the user asked a question that requires pulling information from a legal database
If so, transform said question into a search statement that can be run against a known search target (See SEARCH_TARGETS).

Returns a JSON object containing search_statement and search_target. These properties can be empty.

Sample input

{
    "model": "ollama/mixtral",
    "temperature": 0.0,
    "message": "Tell me everything you know about Miranda v. Arizona (1966)"
}

Notes:

temperature is optional.

Sample output

{
    "search_statement": "caseName:(\"Miranda v. Arizona\") AND dateFiled:[1966-01-01 TO 1966-12-31]",
    "search_target": "courtlistener"
}

[POST] /api/search

Performs search using what /api/extract-search-statement returned.

Returns a JSON object with search results indexed by SEARCH_TARGET.

Sample input

{
    "search_statement": "caseName:(\"Miranda v. Arizona\") AND dateFiled:[1966-01-01 TO 1966-12-31]",
    "search_target": "courtlistener"
}

Sample output

{
    "courtlistener": [
        {
            "absolute_url": "https://www.courtlistener.com/opinion/107252/miranda-v-arizona/",
            "case_name": "Miranda v. Arizona",
            "court": "Supreme Court of the United States",
            "date_filed": "1966-06-13T00:00:00-07:00",
            "id": 107252,
            "ref_tag": 1,
            "status": "Precedential",
            "text": "..."
        },
        {
            "absolute_url": "https://www.courtlistener.com/opinion/8976604/miranda-v-arizona/",
            "case_name": "Miranda v. Arizona",
            "court": "Supreme Court of the United States",
            "date_filed": "1969-10-13T00:00:00-07:00",
            "id": 8968349,
            "ref_tag": 2,
            "status": "Precedential",
            "text": "..."
        },
        {
            "absolute_url": "https://www.courtlistener.com/opinion/8962758/miranda-v-arizona/",
            "case_name": "Miranda v. Arizona",
            "court": "Supreme Court of the United States",
            "date_filed": "1965-11-22T00:00:00-08:00",
            "id": 8953989,
            "ref_tag": 3,
            "status": "Precedential",
            "text": "..."
        }
    ]
}

[POST] /api/complete

Passes messages and context to target LLM and starts streaming text completion. Returns raw text, streamed.

Sample input

{
    "message": "Tell me everything you know about Miranda v. Arizona (1966)",
    "model": "openai/gpt-4-turbo-preview",
    "temperature": 0.0,
    "max_tokens": 4000,
    "search_results": {
        "courtlistener": [...]
    },
    "history": [
        {"role": "user", "content": "Hi there!"},
        {"role": "assistant", "content": "How may I help you?"}
    ]
}

Notes:

temperature is optional.
max_tokens is optional.
history must be an array of objects containing role and content keys. role can be either user or assistant.

☝️ Summary

Adding new tools

This section of the documentation describes the process of making OLAW understand and use additional "search target" beyond the Court Listener API.

1. Declare a new search target

Edit the SEARCH_TARGETS list under olaw/search_results/__init__.py to declare a new search target.

Lets call this new target casedotlaw.

SEARCH_TARGETS = ["courtlistener", "casedotlaw"]

2. Edit search statement extraction prompt

Edit EXTRACT_SEARCH_STATEMENT_PROMPT in your .env file to let the LLM know how to write search statements for this new tool.

This prompt is used by /api/extract-search-statement, which is then able to output objects as follows:

{
    "search_statement": "(Platform-specific search statement based on user question)",
    "search_target": "casedotlaw"
}

The process of designing a performant prompt for that task generally requires a few iterations.

3. Add handling logic

Add a file under the olaw/search_targets/ folder, named after your search target. In that case: casedotlaw.py.

This file must contain a class inheriting from SearchTarget, which defines 1 property and 1 static method:

RESULTS_DATA_FORMAT determining how search results data is structured
search() containing logic for returning search results

You may refer to courtlistener.py as an example.

You will also need to edit olaw/search_results/__init__.py as follows:

Import casedotlaw.py
Edit route_search() to account for that new target

☝️ Summary

Getting Involved

This project is collaborative at its core and we warmly welcome feedback and contributions.

The issues tab is a good place to start to report bugs, suggest features or volunteer to contribute to the codebase on a specific issue.
Don't hesitate to use the discussions tab to ask more general questions about this project.

☝️ Summary

Cite this repository

Cargnelutti, M., & Cushman, J. (2024). Open Legal AI Workbench (OLAW) (Version 0.0.1) [Computer software]

Disclaimer

The Library Innovation Lab is an organization based at the Harvard Law School Library. We are a cross-functional group of software developers, librarians, lawyers, and researchers doing work at the edges of technology and digital information.

Our work is rooted in library principles including longevity, authenticity, reliability, and privacy. Any work that we produce takes these principles as a primary lens. However due to the nature of exploration and a desire to prototype our work with real users, we do not guarantee service or performance at the level of a production-grade software for all of our releases. This includes this project, which is an experimental boilerplate released under MIT License.

Open Legal AI Workbench is an experimental tool for evaluating legal retrieval software and should not be used for legal advice.

☝️ Summary

Name		Name	Last commit message	Last commit date
Latest commit History 13 Commits
.github/screenshots		.github/screenshots
.vscode		.vscode
olaw		olaw
.env.example		.env.example
.gitattributes		.gitattributes
.gitignore		.gitignore
CITATION.cff		CITATION.cff
LICENSE		LICENSE
README.md		README.md
poetry.lock		poetry.lock
pyproject.toml		pyproject.toml
setup.cfg		setup.cfg
wsgi.py		wsgi.py

License

harvard-lil/olaw

Folders and files

Latest commit

History

Repository files navigation

Open Legal AI Workbench (OLAW)

Summary

Concept

Tool-based RAG?

Installation

Configuring the application

Starting the server

Recommended models

Interacting with the WEB UI

Interacting with the API

[GET] /api/models

[POST] /api/extract-search-statement

[POST] /api/search

[POST] /api/complete

Adding new tools

Getting Involved

Cite this repository

Disclaimer

About

Resources

License

Stars

Watchers

Forks

Languages