Make it possible to export word vector embeddings #650
Replies: 3 comments 3 replies
-
|
@cjvanlissa Thank you for posting this request! ping: @PeterLombaers |
Beta Was this translation helpful? Give feedback.
-
|
@cjvanlissa This certainly seems like a nice feature that should be implemented! The feature matrix is already saved in the state file, so it's only a matter taking it out. To get the correspondence between the feature matrix and the tfidf tokens, you'd need to look at the tfidf feature extraction method. You could try to have a go at it yourself if you or your research assistant knows some Python. Otherwise I might look at it because I'm also interested in it, but I can't promise that's before August. |
Beta Was this translation helpful? Give feedback.
-
|
@cjvanlissa I'd like to implement this feature! I am considering a separate extension that does exactly this, as I've implemented something similar during my master's thesis at ASReview. Would you be open for a quick meet up explaining what exactly you need? |
Beta Was this translation helpful? Give feedback.
-
|
What I currently have is an extension that exports the matrix as a pickle and a JSON file with the vocabulary |
Beta Was this translation helpful? Give feedback.
-
|
@cjvanlissa asreview/asreview-extension-vocab-extractor Deze plugin zou beide tf-idf extraction en doc2vec (word2vec) beschikbaar moeten maken! |
Beta Was this translation helpful? Give feedback.
-
|
Erg gaaf @jteijema ! Ik laat mijn onderzoeksassistent er eerst naar kijken, maar hoop er ook zelf mee aan de slag te gaan :) |
Beta Was this translation helpful? Give feedback.
-
As discussed with @Rensvandeschoot, I would like to request adding a feature to export the word vector embeddings for downstream processing. Starting in August, I will have a research assistant who will work with these word vector embeddings for our research. If this is enabled directly in ASReview, we can develop functionality that could be used by ASReview users. From our email communication:
“Die wordt inderdaad opgeslagen, en is makkelijk om uit de state file te halen. Ik kan een python scriptje schrijven als je wil. Als je tfidf als feature extraction gebruikt, dan slaat hij niet op welke tokens er worden gebruikt. Dus dan heb je een feature matrix met daarin de tfidf waardes, maar je weet niet welke waarde bij een specifieke token zoals [machine] of [medicine] hoort. Als je dat wil weten dan vergt het iets meer werk.”
For clarification, it is indeed necessary to know which word corresponds to which feature.
Beta Was this translation helpful? Give feedback.
All reactions