DialoGPT - AI Tool
Large-scale pretraining for dialogue
About DialoGPT
DialoGPT adapts pretraining techniques to response generation using hundreds of Gigabytes of colloquial data. Like GPT-2, DialoGPT is formulated as an autoregressive (AR) language model, and uses a multi-layer transformer as model architecture. Unlike GPT-2, which trains on general text data, DialoGPT draws on 147M multi-turn dialogues extracted from Reddit discussion threads. The implementation is based on the huggingface pytorch-transformer and OpenAI GPT-2.
- Commonsense question answering examples
| User | Bot |
|---|---|
| who is the first president of the United States | George Washington |
| what is the boiling point of water? | I think it's about 212 F. |
| which one is bigger, sun or moon? | The sun . |
| which animal has black and white stripes? | A black and white striped cat . |
- Handling context that does not appear in training examples (note that training examples are sampled from 2005 to 2017)
| User | Bot |
|---|---|
| The trading war between China and US is still happening . | I'm sure the Chinese government and the U . S . will be fine with that as long as they keep their arms on the table and the money flowing . |
| who won the world cup in 2018 ? | Germany , I believe . |
| Nvidia's Titan RTX is really good . | I have this card and i love it . No complaints . |
| Can Porsche beat Tesla with its new Taycan EV ? | I don't think so . The Tesla is more powerful than the porsche . |
- Our bot's response to philosophy questions
| User | Bot |
|---|---|
| What is the meaning of a good life ? | I think it's that we're all connected to our past lives and the meaning of life is to live the way you want to and how you want to . |
| Which is more important -- being successful or being happy ? | Being successful is just endless . Being happy is the only thing that matters . |
| Is peace the only way to stop war ? | Not if the world is not willing to work with you . Peace is the ultimate goal , but peace requires more than that . |
| How much freedom should people have ? | The right amount of freedom is the freedom to do as you please , as long as you don't hurt people or property . |
- Finally
| User | Bot |
|---|---|
| What do you think of the DialoGPT repo ? | It's pretty good , but it's very easy to find a bug . |
Please start a issue if you spot any :)
This project is a joint project between MSR AI and Microsoft Dynamics 365 AI Research team
Source: https://www.microsoft.com/en-us/research/project/large-scale-pretraining-for-response-generation/