A 30000+ Chinese MRC dataset - Delta Reading Comprehension Dataset

Related tags

Text Data & NLPDRCD
Overview

Delta Reading Comprehension Dataset

台達閱讀理解資料集 Delta Reading Comprehension Dataset (DRCD) 屬於通用領域繁體中文機器閱讀理解資料集。 本資料集期望成為適用於遷移學習之標準中文閱讀理解資料集。 本資料集從2,108篇維基條目中整理出10,014篇段落,並從段落中標註出30,000多個問題

關於資料集之更詳細資訊請洽詢論文: For more information please refer to Paper https://arxiv.org/abs/1806.00920

Data format 資料格式

  • version : 資料集版本
  • data :
    • title : : 文章標題
    • id : : 文章編號
    • paragraphs :
      • id : : 文章編號_段落編號
      • context : : 段落內容
      • qas :
        • question : : 問題內容
        • id : : 文章編號_段落編號_問題編號
        • answers :
          • answer_start : text在文中位置
          • id : : "1"表示為人工標註的答案,"2"以上為人工答題的答案
          • text : : 答案內容

Example

{
"version": "1.3",
"data": [
  {
    "title": "基督新教",
    "id": "2128",
    "paragraphs": [
      {
        "context": "基督新教與天主教均繼承普世教會歷史上許多傳統教義,如三位一體、聖經作為上帝的啟示、原罪、認罪、最後審判等等,但有別於天主教和東正教,新教在行政上沒有單一組織架構或領導,而且在教義上強調因信稱義、信徒皆祭司, 以聖經作為最高權威,亦因此否定以教宗為首的聖統制、拒絕天主教教條中關於聖傳與聖經具同等地位的教導。新教各宗派間教義不盡相同,但一致認同五個唯獨:唯獨恩典:人的靈魂得拯救唯獨是神的恩典,是上帝送給人的禮物。唯獨信心:人唯獨藉信心接受神的赦罪、拯救。唯獨基督:作為人類的代罪羔羊,耶穌基督是人與上帝之間唯一的調解者。唯獨聖經:唯有聖經是信仰的終極權威。唯獨上帝的榮耀:唯獨上帝配得讚美、榮耀",
        "id": "2128-2",
        "qas": [
          {
            "id": "2128-2-1",
            "question": "新教在教義上強調信徒皆祭司以及什麼樣的理念?",
            "answers": [
              {
                "id": "1",
                "text": "因信稱義",
                "answer_start": 92
              }
            ]
          },
          {
            "id": "2128-2-2",
            "question": "哪本經典為新教的最高權威?",
            "answers": [
              {
                "id": "1",
                "text": "聖經",
                "answer_start": 105
              }
            ]
          },
          {
            "id": "2128-2-3",
            "question": "新教認同幾個唯獨?",
            "answers": [
              {
                "id": "1",
                "text": "五個",
                "answer_start": 171
              }
            ]
          },
          {
            "id": "2128-2-4",
            "question": "文中提及,人唯獨藉信心接受神的赦罪、拯救,此為哪一種唯獨?",
            "answers": [
              {
                "id": "1",
                "text": "唯獨信心",
                "answer_start": 206
              }
            ]
          }
        ]
      },
      {
        "context": "主教制源自天主教的主教制度,幾乎和天主教的主教制度一模一樣,唯一不同的是主教亦可以結婚。天主教的主教制是在使徒們去世後於第二、三世紀興起的主教制度,所以可以說主教制是整個基督宗教中歷史最悠久的神職人員制度。現在行主教制的新教教會已經很少,聖公會就是沿用主教制,從教會制度和禮儀上看來,聖公會基本上屬大公教會傳統。路德宗和衛理公會則由各區會自行選擇使用主教制還是長老制;在香港和澳門,路德會和衛理公會就選用了長老制。然而,在歐洲,例如瑞典、芬蘭、挪威、德國等地,他們則通常採用主教制。長老制,是一個以議會形式管理區會的制度。議會內的成員由各教會選出長老,代表該教會出席會議。顧名思義,長老會就是採用長老制的教會。採用長老制的教會有基督教改革宗長老會、台灣基督長老教會、韓國基督長老教會等。",
        "id": "2128-3",
        "qas": [
          {
            "id": "2128-3-1",
            "question": "新教的主教制度源自於哪一教?",
            "answers": [
              {
                "id": "1",
                "text": "天主教",
                "answer_start": 5
              }
            ]
          },
          {
            "id": "2128-3-2",
            "question": "文中提及,新教的主教可以做什麼?",
            "answers": [
              {
                "id": "1",
                "text": "結婚",
                "answer_start": 41
              }
            ]
          },
          {
            "id": "2128-3-3",
            "question": "哪個會屬於大公教會傳統?",
            "answers": [
              {
                "id": "1",
                "text": "聖公會",
                "answer_start": 142
              }
            ]
          },
          {
            "id": "2128-3-4",
            "question": "以議會形式管理區會的制度,名為?",
            "answers": [
              {
                "id": "1",
                "text": "長老制",
                "answer_start": 241
              }
            ]
          }
        ]
      }
    ]
  }
]
}

Copyright Notice 版權聲明

本資料集整理、改編自維基百科,其內容以CC-BY-SA 3.0條款發布。 台達電子對於本資料集內容之正確性不為任何擔保,且不就因使用或倚賴本資料集而引致的任何損失,承擔任何責任。 CC-BY-SA 3.0相關條款請參考以下連結 http://creativecommons.org/licenses/by-sa/3.0/

DRCD is compiled and adapted from Wikipedia and its content is published under the terms of CC-BY-SA 3.0. Delta Electronics, Inc. makes no representations or warranties of the correctness of the contents of DRCD and will not be liable for any loss or damage arising from the use or reliance on DRCD.

CC-BY-SA 3.0 can be found at http://creativecommons.org/licenses/by-sa/3.0/

Contact us 聯繫我們

You might also like...
DELTA is a deep learning based natural language and speech processing platform.
DELTA is a deep learning based natural language and speech processing platform.

DELTA - A DEep learning Language Technology plAtform What is DELTA? DELTA is a deep learning based end-to-end natural language and speech processing p

CDLA: A Chinese document layout analysis (CDLA) dataset
CDLA: A Chinese document layout analysis (CDLA) dataset

CDLA: A Chinese document layout analysis (CDLA) dataset 介绍 CDLA是一个中文文档版面分析数据集,面向中文文献类(论文)场景。包含以下10个label: 正文 标题 图片 图片标题 表格 表格标题 页眉 页脚 注释 公式 Text Title

A 10000+ hours dataset for Chinese speech recognition
A 10000+ hours dataset for Chinese speech recognition

A 10000+ hours dataset for Chinese speech recognition

Reading Wikipedia to Answer Open-Domain Questions
Reading Wikipedia to Answer Open-Domain Questions

DrQA This is a PyTorch implementation of the DrQA system described in the ACL 2017 paper Reading Wikipedia to Answer Open-Domain Questions. Quick Link

This is my reading list for my PhD in AI, NLP, Deep Learning and more.

This is my reading list for my PhD in AI, NLP, Deep Learning and more.

Code repository for "It's About Time: Analog clock Reading in the Wild"

it's about time Code repository for "It's About Time: Analog clock Reading in the Wild" Packages required: pytorch (used 1.9, any reasonable version s

🐍 A hyper-fast Python module for reading/writing JSON data using Rust's serde-json.
🐍 A hyper-fast Python module for reading/writing JSON data using Rust's serde-json.

A hyper-fast, safe Python module to read and write JSON data. Works as a drop-in replacement for Python's built-in json module. This is alpha software

ANTLR (ANother Tool for Language Recognition) is a powerful parser generator for reading, processing, executing, or translating structured text or binary files.

ANTLR (ANother Tool for Language Recognition) is a powerful parser generator for reading, processing, executing, or translating structured text or binary files.

The model is designed to train a single and large neural network in order to predict correct translation by reading the given sentence.
The model is designed to train a single and large neural network in order to predict correct translation by reading the given sentence.

Neural Machine Translation communication system The model is basically direct to convert one source language to another targeted language using encode

Comments
  • Evaluation problem

    Evaluation problem

    According to the paper:

    F1 score and exact match from Rajpurkar et al. (2016) are used as the evaluation metrics. Both metrics ignore punctuations. In F1 score metric, we consider predictions and ground truth as bag of Chinese character.

    Is the ignored punctuations including fullwidth? Is there a original evaluation script for this dataset?

    opened by penut85420 2
  • Format conversion to SQuAD2.0

    Format conversion to SQuAD2.0

    When you used Bert-Chinese model to do the DRCD tasks like your paper told us, is there anything such as format conversion that we need to do first, and then we can use Bert-Chinese model to do DRCD tasks ?

    p.s. Format conversion means that convert DRCD format to SQuAD2.0 format.

    opened by allenyummy 1
  • Training set problem

    Training set problem

    In the paper, it said that "the training set contains 26,932 questions in 8,014 paragraphs". However, after calculating I found that I got 26936 question's id in the json file.

    opened by Liangtaiwan 1
  • Dev problem

    Dev problem

    The dev set answers are duplicate in the same question. Also, in SQuAD dataset, it has 3 different answers in dev set and test set, so the human performance is much higher than your dataset in EM performance. Are you going to provide more answer?

    opened by Liangtaiwan 1
Releases(v1.0)
Making text a first-class citizen in TensorFlow.

TensorFlow Text - Text processing in Tensorflow IMPORTANT: When installing TF Text with pip install, please note the version of TensorFlow you are run

1k Dec 26, 2022
Persian Bert For Long-Range Sequences

ParsBigBird: Persian Bert For Long-Range Sequences The Bert and ParsBert algorithms can handle texts with token lengths of up to 512, however, many ta

Sajjad Ayoubi 63 Dec 14, 2022
A simple implementation of N-gram language model.

About A simple implementation of N-gram language model. Requirements numpy Data preparation Corpus Training data for the N-gram model, a text file lik

4 Nov 24, 2021
Code for our paper "Mask-Align: Self-Supervised Neural Word Alignment" in ACL 2021

Mask-Align: Self-Supervised Neural Word Alignment This is the implementation of our work Mask-Align: Self-Supervised Neural Word Alignment. @inproceed

THUNLP-MT 46 Dec 15, 2022
Plugin repository for Macast

Macast-plugins Plugin repository for Macast. How to use third-party player plugin Download Macast from GitHub Release. Download the plugin you want fr

109 Jan 04, 2023
A simple Flask site that allows users to create, update, and delete posts in a database, as well as perform basic NLP tasks on the posts.

A simple Flask site that allows users to create, update, and delete posts in a database, as well as perform basic NLP tasks on the posts.

Ian 1 Jan 15, 2022
Utilities for preprocessing text for deep learning with Keras

Note: This utility is really old and is no longer maintained. You should use keras.layers.TextVectorization instead of this. Utilities for pre-process

Hamel Husain 180 Dec 09, 2022
Resources for "Natural Language Processing" Coursera course.

Natural Language Processing course resources This github contains practical assignments for Natural Language Processing course by Higher School of Eco

Advanced Machine Learning specialisation by HSE 1.1k Jan 01, 2023
Facilitating the design, comparison and sharing of deep text matching models.

MatchZoo Facilitating the design, comparison and sharing of deep text matching models. MatchZoo 是一个通用的文本匹配工具包,它旨在方便大家快速的实现、比较、以及分享最新的深度文本匹配模型。 🔥 News

Neural Text Matching Community 3.7k Jan 02, 2023
A CRM department in a local bank works on classify their lost customers with their past datas. So they want predict with these method that average loss balance and passive duration for future.

Rule-Based-Classification-in-a-Banking-Case. A CRM department in a local bank works on classify their lost customers with their past datas. So they wa

ÖMER YILDIZ 4 Mar 20, 2022
A unified tokenization tool for Images, Chinese and English.

ICE Tokenizer Token id [0, 20000) are image tokens. Token id [20000, 20100) are common tokens, mainly punctuations. E.g., icetk[20000] == 'unk', ice

THUDM 42 Dec 27, 2022
HAIS_2GNN: 3D Visual Grounding with Graph and Attention

HAIS_2GNN: 3D Visual Grounding with Graph and Attention This repository is for the HAIS_2GNN research project. Tao Gu, Yue Chen Introduction The motiv

Yue Chen 1 Nov 26, 2022
Contains analysis of trends from Fitbit Dataset (source: Kaggle) to see how the trends can be applied to Bellabeat customers and Bellabeat products

Contains analysis of trends from Fitbit Dataset (source: Kaggle) to see how the trends can be applied to Bellabeat customers and Bellabeat products.

Leah Pathan Khan 2 Jan 12, 2022
In this repository we have tested 3 VQA models on the ImageCLEF-2019 dataset.

Med-VQA In this repository we have tested 3 VQA models on the ImageCLEF-2019 dataset. Two of these are made on top of Facebook AI Reasearch's Multi-Mo

Kshitij Ambilduke 8 Apr 14, 2022
Bnagla hand written document digiiztion

Bnagla hand written document digiiztion This repo addresses the problem of digiizing hand written documents in Bangla. Documents have definite fields

Mushfiqur Rahman 1 Dec 10, 2021
Text editor on python tkinter to convert english text to other languages with the help of ployglot.

Transliterator Text Editor This is a simple transliteration program which is used to convert english word to phonetically matching word in another lan

Merin Rose Tom 1 Jan 16, 2022
ProtFeat is protein feature extraction tool that utilizes POSSUM and iFeature.

Description: ProtFeat is designed to extract the protein features by employing POSSUM and iFeature python-based tools. ProtFeat includes a total of 39

GOKHAN OZSARI 5 Dec 16, 2022
Quantifiers and Negations in RE Documents

Quantifiers-and-Negations-in-RE-Documents This project was part of my work for a

Nicolas Ruscher 1 Feb 01, 2022
Header-only C++ HNSW implementation with python bindings

Hnswlib - fast approximate nearest neighbor search Header-only C++ HNSW implementation with python bindings. NEWS: version 0.6 Thanks to (@dyashuni) h

2.3k Jan 05, 2023
Code for Discovering Topics in Long-tailed Corpora with Causal Intervention.

Code for Discovering Topics in Long-tailed Corpora with Causal Intervention ACL2021 Findings Usage 0. Prepare environment Requirements: python==3.6 te

Xiaobao Wu 8 Dec 16, 2022