What Is DeepPavlov Dream
DeepPavlov Dream is an open-source Multiskill AI Assistant Platform built using DeepPavlov Conversational AI Stack. It works on top of DeepPavlov Agent running as container in Docker. It runs on x86_64 machines, and prefers having NVIDIA GPUs on the machine. You can use it to study how multiskill AI assistants can be built, as a base for research projects, or for commercial systems where running open source system on-premises (or in the cloud) is a requirement. It's also a proud descendant of the 3 previous projects also called DREAM:
- Original DREAM Socialbot for Alexa Prize 3 (2019-2020). Read the paper here.
- Dream AI Assistant Platform Demo (2020). Read the post on Medium.
- DREAM 2 Socialbot for Alexa Prize 4 (2020-2021). Read the paper here.
You can think of it as a Dialogue Operating System of sorts, with the emphasize on Operating System. Like in the Linux world, DeepPavlov Dream has multiple distributions:
- DREAM Socialbot is the biggest distribution (~30 Skills, ~30 Annotators) adapted to run w/o Amazon Alexa Prize-specific services. This is almost the same version of the DREAM socialbot as at the end of Alexa Prize Challenge 4. DREAM 2 Socialbot blends together almost 40 different chit-chat and task-oriented skills to engage in open domain conversations. It relies on a selection of modern NLP models and components including ~30 annotators, post-annotators, and knowledge graph integration. Some API services are replaced with trainable models. Some services (e.g., News Annotator, Game Skill, Weather Skill) require private keys for underlying APIs, most of them can be obtained for free. If you want to use these services in local deployments, add your keys to the environmental variables (e.g., ./.env). This version of Dream Socialbot consumes a lot of resources because of its modular architecture and original goals (participation in Alexa Prize Challenge). We provide a demo of Dream Socialbot on our website.
- Dream Mini is a neural generation-based socialbot. Dream Mini can be used a starting version of your custom socialbot because it contains only Dummy Skill (with fallbakcs and topical prompts which help to drive the conersation, and added sometimes to the dialog), DFF-Intent-Responder-Skill to respond to the detected user intents, AIML-based skill as an example, and DialoGPT as a main source of the responses.
- Deepy Base is a base version of Lunar assistant. Deepy Base contains Spelling Preprocessing annotator, template-based Harvesters Maintenance Skill, and AIML-based open-domain Program-y Skill based on Dialog Flow Framework.
- Deepy Advanced is an advanced version of Lunar assistant. Deepy Advanced contains Spelling Preprocessing, Sentence Segmentation, Entity Linking and Intent Catcher annotators, Harvesters Maintenance GoBot Skill for goal-oriented responses, and AIML-based open-domain Program-y Skill based on Dialog Flow Framework.
- Deepy FAQ is a FAQ version of Lunar assistant. Deepy FAQ contains Spelling Preprocessing annotator, template-based Frequently Asked Questions Skill, and AIML-based open-domain Program-y Skill based on Dialog Flow Framework.
- Deepy GoBot is a goal-oriented version of Lunar assistant. Deepy GoBot Base contains Spelling Preprocessing annotator, Harvesters Maintenance GoBot Skill for goal-oriented responses, and AIML-based open-domain Program-y Skill based on Dialog Flow Framework.
For many companies and individuals two key questions about any piece of software are availability of the source code and the license. DeepPavlov Dream AI Assistant Platform including its various distributives, is open source. The entire source code is available for anyone to use and modify as they see fit, for academic, personal, or commercial purposes. In particular, companies may use DeepPavlov Dream in whole or in part in products. Furthermore, it is completely free of charge. Support and consultancy may be available for a fee. Companies should contact us at info@deeppavlov.ai for pricing.
DeepPavlov Dream is available under an Apache 2.0 license, which may be attractive to companies since it does not require them to publish changes they make to the system as the GPL does.
Deepy was originally built in October 2020 as a demo of a simple multiskill AI Assistant to be shown at DeepPavlov's talk at NVIDIA GTC Fall 2020 Conference. However, Deepy is based on DeepPavlov Dream AI Assistant Demo, which in turn is an adaptation of the original DREAM Socialbot created by DeepPavlov's student team for Alexa Prize Grand Socialbot Challenge 3 (2019-2020).
Note: not included in this repository as a distribution.
Note: not included in this repository as a distribution.
Getting DeepPavlov Dream up and running depending on the distrib may be quite simple, but it might require having GPU(s) depending on the number of GPU-heavy components you want to use in your solution. Deepy, the simplest distrib, can be run entirely locally on a dev machine (6GB of RAM), while DREAM and DREAM 2 socialbots will require the use of the proxy services provided by DeepPavlov.ai.
- Clone the repository
- Change directory to it
- Pick the distribution you want to run from
/assistant_dists - Copy
docker-compose.ymlfrom it to the root directory of your repository (system will ask you to confirm rewriting the existing one; confirm it) - Copy
pipeline_conf.jsonfrom it to the/agentof your repository (system will ask you to confirm rewriting the existing one; confirm it) - Write the command
docker-compose -f docker-compose.yml buildto build your distribution but don't press ENTER just yet. - [Optional] If you want to use ASR & TTS modules, add them to the
docker-composecommand file chain; in this case the command below would look like this:docker-compose -f docker-compose.yml -f ass_tts.yml build - For GPU-intensive services, change lines in your
docker-compose.ymlandasr_tts.yml(optional) to specify GPUs you want to run them on. Typical GPU-intensive service that uses BERT model needs ~4GB of GPU RAM so plan accordingly. - Build your system by using the line you've formed above
- Once done, use the same line but also replace
buildin the end withup. - Once the agent is up and running, use your favorite tool (e.g.,
curlor Postman) to talk to Deepy viahttp://localhost:4242/endpoint by providing the following content:
{
"user_id" : "24424252524525",
"payload" : "Hello!"
}
If everything works correctly, you'll get the response like this:
{
"dialog_id": "773dc35a567072c142b9d6d8bdea00fb",
"utt_id": "dfd1c751875439c6d98e3bb1410448f7",
"user_id": "r234242343",
"response": "Hello, I'm a lunar assistant Deepy! How are you?",
"active_skill": "program_y",
"debug_output": [
{
"skill_name": "harvesters_maintenance_skill",
"annotations": {
"emotion_classification": [
{
"anger": 0.46746790409088135,
"fear": 0.3528013229370117,
"joy": 0.3129902184009552,
"love": 0.2804321050643921,
"sadness": 0.35413244366645813,
"surprise": 0.19576209783554077,
"neutral": 0.9979490041732788
}
]
},
"text": "I don't have this information.",
"confidence": 0.5
},
{
"skill_name": "program_y",
"annotations": {
"emotion_classification": [
{
"anger": 0.38495343923568726,
"fear": 0.22263416647911072,
"joy": 0.4415707588195801,
"love": 0.4192220866680145,
"sadness": 0.21526440978050232,
"surprise": 0.19943127036094666,
"neutral": 0.998478353023529
}
]
},
"text": "Hello, I'm a lunar assistant Deepy! How are you?",
"confidence": 0.98,
"ssml_tagged_text": "Hello, I'm a lunar assistant Deepy! How are you?"
}
],
"human_utt_annotations": {
"sentseg": {
"punct_sent": "hello!",
"segments": [
"hello!"
]
},
"spelling_preprocessing": "hello!"
}
}
DeepPavlov Dream is, like all software, under development. However, you can pick any of the distributions (as well call them) from /assistant_dists directory to use it in your own system. We use one of these configs (currently /assistant_dists/deepy_ai_adv/) on our Demo Web Site. Here are some of the features of the current system. Development is ongoing and we hope you will join the community and help out.
Nearly all the documentation is in this wiki. This is a collection of pages with information on many topics relating to DeepPavlov Dream. When you are starting out, you should consult it often. Once you become more experienced, you can edit it, updating pages or adding new ones, just like Wikipedia.
Our team has made some videos about DeepPavlov Dream and its various distribs. You can watch them on YouTube as follow.
- Deepy 3000 Demo: Build Your Own Moonbase A.I. Assistant with DeepPavlov Dream!
- DeepPavlov Community Call #1
- DeepPavlov Community Call #2
Deepy has appeared on the Web in various places. Here are the references:
- Multiskill Conversational AI @ NVIDIA GTC Fall 2020
- Moon base hiring - Deepy 3000 Demo
- Building an AI Assistant for the Moonbase with DeepPavlov.ai
TBD
TBD