Skip to content

Commit

Permalink
debug encoding in vignette
Browse files Browse the repository at this point in the history
  • Loading branch information
Andreas Blätte authored and Andreas Blätte committed Jul 9, 2023
1 parent 1f24cce commit c79115f
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 4 deletions.
8 changes: 8 additions & 0 deletions README.Rmd
Expand Up @@ -31,6 +31,10 @@ ExplorationWorkbench" presented at LREC 2014
The main function is `detect_duplicates()`.


## Installation

## Using the 'duplicates' package

## Related work

Near duplicate detection is a standard NLP task. There is a wide range of
Expand All @@ -44,3 +48,7 @@ Workbench (CWB). The hashing step which is a selling point for the textreuse
package is performed already, and requirements for tokenizing and hashing the
data are not replicated. The scenario for using the duplicates package is
large, CWB-indexed corpora.


## Citation

8 changes: 4 additions & 4 deletions vignettes/vignette.Rmd
Expand Up @@ -80,7 +80,7 @@ annodata <- duplicates_as_annotation_data(
```{r encode}
library(cwbtools)
regdata <- cwbtools::registry_file_parse(corpus = "REUTERS2", registry = registry())
regdata <- registry_file_parse(corpus = "REUTERS2", registry = registry())
for (s_attr in c("is_duplicate", "duplicates")){
s_attribute_encode(
Expand All @@ -97,8 +97,8 @@ for (s_attr in c("is_duplicate", "duplicates")){
)
}
RcppCWB::cl_load_corpus("REUTERS2", registry = polmineR::registry())
# RcppCWB::cl_load_corpus("REUTERS2", registry = polmineR::registry())
corpus("REUTERS2", registry_dir = registry()) %>%
s_attributes()
# corpus("REUTERS2", registry_dir = registry()) %>%
# s_attributes()
```

0 comments on commit c79115f

Please sign in to comment.