Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add a table_info.csv file in extractedData #38

Open
jeromedockes opened this issue Oct 19, 2023 · 3 comments · May be fixed by #41
Open

add a table_info.csv file in extractedData #38

jeromedockes opened this issue Oct 19, 2023 · 3 comments · May be fixed by #41

Comments

@jeromedockes
Copy link
Member

at the moment pubget outputs some data for each table: a csv file with the content of the table, and a json file with keys like table id, table label, table caption, and the path of the csv file

those are a bit hard to discover & use because they are stored in the article's individual directories, eg in
query_0dbec5035dc626ca916d0317b7b9a76f/articles/0b8/pmcid_7664275/tables/

it would be helpful to create a csv in the extractedData directory grouping the info for all tables in all articles. columns could be
pmcid,table_number,table_id,table_label,table_caption,n_header_rows,table_data_file

table_data_file would be the path to the table's csv, probably relative to the root of the query directory.
the other keys would be those currently provided in the JSON

@jeromedockes
Copy link
Member Author

also maybe extract the content of the table-wrap-foot elements? those are footnotes so not sure if they contain useful information

@adelavega
Copy link
Contributor

This would be great!

@jeromedockes
Copy link
Member Author

also maybe extract the content of the table-wrap-foot elements? those are footnotes so not sure if they contain useful information

this part is done in #40

@jeromedockes jeromedockes linked a pull request Nov 1, 2023 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants