Skip to content

GTDB dataset for training & evaluation for mathematical OCR systems

Notifications You must be signed in to change notification settings

uchidalab/GTDB-Dataset

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

9 Commits
 
 
 
 
 
 
 
 

Repository files navigation

GTDB-Datasets

CC BY-NC-ND

Introduction

To promote further research into OCR for scientific documents, we releases databases that may be suitable for research outside of InftyProject development. We have carefully scrutinized the data in these releases, and expect that the databases can be relied upon with high confidence. Nevertheless, if you experience any problems with them, please contact us.

The datasets, called GTDB-1 and GTDB-2, consist of document page images collected from scientific journals and textbooks. The GTDB-1 dataset contains 31 English articles on mathematics. The GTDB-2 dataset contains 16 articles. Diverse font faces and mathematical notation styles are included in these articles. A list of the articles in both datasets is provided in the bottom of this page.

The statistics of each dataset are shown in the table. The article pages were originally scanned at 600 dpi. The ground truth annotations for each math symbol and ordinary character were attached manually.

Statistics of the datasets

GTDB-1 GTDB-2
Number of articles 31 16
Number of pages 544 343
Number of math symbols 162,406 115,433
Number of ordinary text characters 646,714 507,412

Here, the ground truth annotation is released. Although we could not include the original document images of articles for copyright reasons, we provide hyperlinks to the web pages of the original documents, where the users can obtain the document images.

Format of annotation CSV files

Annotations for document pages in one article are provided by one CSV file. The format of CSV files are as following:

Infty GT-Data Format Ver.1.1
Sheet,1,AIF_1970_20_493.png,-1
Text,1,116,411,996,559
Line,1,118,411,996,471
Chardata,1,118,416,159,463,0,-1,-1,0141
Chardata,2,164,428,197,462,0,-1,1,016E
Chardata,3,201,428,234,461,0,-1,2,016E
Chardata,4,239,453,249,464,0,-1,3,142E
  • The first line is a file header showing the format version.
  • Records (lines) starting with Sheet denote the beginning of a new page. Each field (column) in Sheet record denotes 1:Sheet, 2:Page ID, 3:Filename of the page image, 4:always -1.
  • Records stating with Text or Image are corresponding to one document layout component. 1:Text or Image, 2:Component ID, 3-6:Coordinates of the bounding box (left, top, right, bottom)
  • Records starting with Line denote the text line. 1:Line, 2:Line ID, 3-6:Coordinates of the bounding box (left, top, right, bottom)
  • Records starting with Chardata denote the characters or symbols. 1:Chardata, 2:Character ID, 3-6:Coordinates of the bounding box (left, top, right, bottom), 7:Text Mode (0:Ordinary text, 1:Math symbol), 8:Link label, 9:ID of parent character, 10:OCR code

The number in Link label field represents the positional relationship to the preceding character, called parent character.

0: horizontal
1: right-superscript
2: right-subscript
3: left-superscript
4: left-subscript
5: upper
6: lower
-1: the first character in expressions or ordinary texts

Conditions of use

GTDB datasets are distributed under CC BY-NC-ND license. Usage for research, development, or testing of OCR systems (not commercial) for scientific documents is permitted, free of charge.

The copyright and license holder of GTDB datasets is
Masakazu Suzuki:
Science Accessibility Net,
Professor emeritus of Kyushu University

Citation:

W. Ohyama, M. Suzuki and S. Uchida, "Detecting Mathematical Expressions in Scientific Document Images Using a U-Net Trained on a Diverse Dataset," in IEEE Access, vol. 7, pp. 144030-144042, 2019, doi: 10.1109/ACCESS.2019.2945825. link

@ARTICLE{8861044,
    author={W. {Ohyama} and M. {Suzuki} and S. {Uchida}},
    journal={IEEE Access}, 
    title={Detecting Mathematical Expressions in Scientific Document Images Using a U-Net Trained on a Diverse Dataset}, 
    year={2019},
    volume={7},
    number={},
    pages={144030-144042},
    doi={10.1109/ACCESS.2019.2945825}}

List of articles

In GTDB-1 dataset

In GTDB-2 dataset

About

GTDB dataset for training & evaluation for mathematical OCR systems

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published