MAILIST GRE YIMING ZHANG CODE

We constructed a new dataset by collecting a large set of questions, answers and associated candidate passages in the English exams for middle- and high-school Chinese students within the 12–18 age range.

To address the aforementioned limitations, Making it hard to evaluate the ability of systems in text comprehension over a broader range of topics. Yet another issue in existing datasets is that the topic coverages are often biased due to the specific ways that the data were initially collected, Secondly, answers and questions of most datasets are either crowd-sourced or automatically-generated,īringing a significant amount of noises in the datasets and limits the ceiling performance by domain experts, such as 82% for Children’s Book Test and 84% for Who-did-What. Which leads to the fact that lots of questions can be solved trivially via word-based search and context-matching without deeper reasoning this constrains the types of questions as well. Firstly, in all datasets, the candidate options are directly extracted from the context (as a single entity or a text span), The existing datasets suffer several critical limitations. While having a suitable dataset is crucial for evaluating the system’s true ability in reading comprehension, Which allow researchers to train deep learning systems and obtain results comparable to the human performance. Towards this goal, several large-scale datasets (Rajpurkar et al., 2016 Onishi et al., 2016 Hill et al., 2015 Trischler et al., 2016 Hermann et al., 2015) have been proposed,

MAILIST GRE YIMING ZHANG CODE

The dataset is freely available at and the code is available at. We hope this new dataset can serve as a valuable resource for research and evaluation in machine comprehension. In particular, the proportion of questions that requires reasoning is much larger in RACE than that in other benchmark datasets for reading comprehension,Īnd there is a significant gap between the performance of the state-of-the-art models (43%) and the ceiling human performance (95%). Collected from the English exams for middle and high school Chinese students in the age range between 12 to 18, RACE consists of near 28,000 passages and near 100,000 questions generated by human experts (English instructors), and covers a variety of topics which are carefully designed for evaluating the students’ ability in understanding and reasoning. A new dataset for benchmark evaluation of methods in the reading comprehension task.

Mailist gre yiming zhang