Entrance Exam

The test collections for Fact Validation EN and JA subtasks have t2 texts extracted from real university entrance exams in Japan.

(The t2s for EN subtasks are English translations of Japanese texts in the exams.)

Because the exams we used as text data consist of multiple-choice questions, such t2 texts enable your system to simulate answering multiple-choice questions of real university entrance exams by referring to textual knowledge such as Wikipedia and textbooks.

This is an attempt to emulate human's process to answer entrance exam questions as the RITE task.

This is also an interesting challenge that use real entrance exams for the evaluation of intelligent systems.

We used the National Center Test that National Center for University Entrance Examinations provided.

The National Center Test has tests for several subjects.

We adopted tests for the following subjects for each subtask:

Evaluation Method

Performance of the system is evaluated by correct answer ratio for multiple-choice questions for each subject.

"Y" and "N" labels of the system for Fact Validation subtask are mapped into selection of answers for the original questions.