Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generalize Final data.npz output #181

Open
martham93 opened this issue Aug 27, 2020 · 1 comment
Open

Generalize Final data.npz output #181

martham93 opened this issue Aug 27, 2020 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@martham93
Copy link
Contributor

Currently the data.npz output, by default is split into a test and train set with 80% of the data going into training and 20% of the data going into the test set. While the ratio, number of split sets, and names of split sets can be changed, it could be better to return a data.npz and let users handle splitting the data outside of label-maker for increased flexibility.

I think it could be useful if the new version of the data.npz file returned as the x-y-z tile id as the key and the values is a list with 2 elements, the first being the label in numpy array format, and the second being the image in numpy array format.

Another benefit of this change will be that the image label pairs in numpy array format will retrain the x-y-z tile id. The x-y-z tile information is currently only associated with the labels.npz file and not associated with the data.npz file.

thoughts? @drewbo

@martham93 martham93 added the enhancement New feature or request label Aug 27, 2020
@martham93 martham93 self-assigned this Aug 27, 2020
@martham93 martham93 changed the title Generalize Final Data.npz Ouput Generalize Final data.npz output Aug 27, 2020
@martham93
Copy link
Contributor Author

instead of data.npz output will explore the final output using x-array for more flexibility but including a utils script in the repo or example script to go from the final output to tf-records for modeling purposes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant