What is DialogRE
DialogRE is the first human-annotated dialogue-based relation extraction dataset, containing 1,788 dialogues originating from the complete transcripts of a famous American television situation comedy Friends. We annotate all occurrences of 36 possible relation types that exist between an argument pair in a dialogue. DialogRE is available in English and Chinese.
Sample Instances
|
S1: Hey Pheebs. S2: Hey! S1: Any sign of your brother? S2: No, but he's always late. S1: I thought you only met him once? S2: Yeah, I did. I think it sounds y'know big sistery, y'know, `Frank's always late.' S1: Well relax, he'll be here. |
| Argument pair | Trigger | Relation type | |
| (Frank, S2) | brother | per:siblings | |
| (S2, Frank) | brother | per:siblings | |
| (S2. Pheebs) | none | per:alternate_names | |
| (S1, Pheebs) | none | unanswerable |
Relation Schema
| ID | Subject | Relation Type | Object | Inverse Relation |
| 1 | PER | per:positive_impression | NAME | |
| 2 | PER | per:negative_impression | NAME | |
| 3 | PER | per:acquaintance | NAME | per:acquaintance |
| 4 | PER | per:alumni | NAME | per:alumni |
| 5 | PER | per:boss | NAME | per:subordinate |
| 6 | PER | per:subordinate | NAME | per:boss |
| 7 | PER | per:client | NAME | |
| 8 | PER | per:dates | NAME | per:dates |
| 9 | PER | per:friends | NAME | per:friends |
| 10 | PER | per:girl/boyfriend | NAME | per:girl/boyfriend |
| 11 | PER | per:neighbor | NAME | per:neighbor |
| 12 | PER | per:roommate | NAME | per:roommate |
| 13 | PER | per:children | NAME | per:parents |
| 14 | PER | per:other_family | NAME | per:other_family |
| 15 | PER | per:parents | NAME | per:children |
| 16 | PER | per:siblings | NAME | per:siblings |
| 17 | PER | per:spouse | NAME | per:spouse |
| 18 | PER | per:place_of_residence | NAME | gpe:residents_of_place |
| 19 | PER | per:place_of_birth | NAME | gpe:births_in_place |
| 20 | PER | per:visited_place | NAME | gpe:visitors_of_place |
| 21 | PER | per:origin | NAME | |
| 22 | PER | per:employee_or_member_of | NAME | org:employees_or_members |
| 23 | PER | per:schools_attended | NAME | org:students |
| 24 | PER | per:works | NAME | |
| 25 | PER | per:age | VALUE | |
| 26 | PER | per:date_of_birth | VALUE | |
| 27 | PER | per:major | STRING | |
| 28 | PER | per:place_of_work | STRING | |
| 29 | PER | per:title | STRING | |
| 30 | PER | per:alternate_names | NAME/STRING | |
| 31 | PER | per:pet | NAME/STRING | |
| 32 | GPE | gpe:residents_of_place | NAME | per:place_of_residence |
| 33 | GPE | gpe:births_in_place | NAME | per:place_of_birth |
| 34 | GPE | gpe:visitors_of_place | NAME | per:visited_place |
| 35 | ORG | org:employees_or_members | NAME | per:employee_or_member_of |
| 36 | ORG | org:students | NAME | per:schools_attended |
| 37 | NAME | unanswerable | NAME/STRING/VALUE |
Baseline Results
V1: The original English version (used in the paper)
V2-EN: The updated English version with a few annotation errors fixed (updated on August 2020)
V2-CN: The Chinese version (updated on August 2020)
| V1 | V2-EN | V2-CN | ||||||||||
| Dev | Test | Dev | Test | Dev | Test | |||||||
| F1 (σ) | F1c (σ) | F1 (σ) | F1c (σ) | F1 (σ) | F1c (σ) | F1 (σ) | F1c (σ) | F1 (σ) | F1c (σ) | F1 (σ) | F1c (σ) | |
| BERT | 60.6 (1.2) | 55.4 (0.9) | 58.5 (2.0) | 53.2 (1.6) | 59.4 (0.7) | 54.7 (0.8) | 57.9 (1.0) | 53.1 (0.7) | 63.7 (0.9) | 59.5 (0.7) | 63.2 (0.9) | 58.4 (0.8) |
| BERTS | 63.0 (1.5) | 57.3 (1.2) | 61.2 (0.9) | 55.4 (0.9) | 62.2 (1.3) | 57.0 (1.0) | 59.5 (2.1) | 54.2 (1.4) | 65.5 (0.9) | 61.0 (0.4) | 63.5 (0.9) | 58.7 (0.9) |
Questions?
Please contact dialogre@dataset.org.
Sibling Datasets
Check DREAM (https://dataset.org/dream) or C3 (https://dataset.org/c3) if you are interested in dialogue-based multiple-choice machine reading comprehension for English or Chinese.