<img src=“https://travis-ci.org/sul-dlss/revs-indexer-service.svg?branch=master” alt=“Build Status” /> <img src=“https://coveralls.io/repos/sul-dlss/revs-indexer-service/badge.svg” alt=“Coverage Status” />
Code to harvest Revs metadata from DOR and index it into Solr for the Revs Digital Library application
-
checkout code
-
rake config
-
edit config/settings.yml file to add in actual solr URLs for each target (for local work only)
-
bundle install
All rake tasks that perform batch indexing will generate log files in the “log” folder within the app itself. You can tail the log file to watch the progress. The log file is also useful since you can pass it to the “reindexer” rake task to retry just the errored out druids. The name of the log file will depend on which rake task you are running, and will be timestamped to be unique.
rake index RAILS_ENV=production target=revs_prod druid=oo000oo0001
rake log_indexer RAILS_ENV=production target=revs_prod log_file=/tmp/mailander_1.yaml log_type=preassembly # preassembly run nohup rake log_indexer RAILS_ENV=production target=revs_prod log_file=/tmp/mailander_1.yaml log_type=preassembly & # for a long running process, which will be most runs that have more than a few dozen druids, nohup it
rake log_indexer RAILS_ENV=production target=revs_prod log_file=/tmp/mailander_1_remediate.yaml log_type=remediate # remediation run
rake log_indexer RAILS_ENV=production target=revs_prod log_file=/tmp/mailander.csv log_type=csv # a simple csv file – it must have a header line, with the header of “druid” definining the items you wish to index
Index an entire collection, including the collection itself, along with all of its members (be sure to check the dor-fetcher-url parameter in the Rails environment you are running under to be sure it is connecting where you expect):¶ ↑
rake collection_indexer RAILS_ENV=production target=revs_prod collection_druid=oo000oo0001 nohup rake collection_indexer RAILS_ENV=production target=revs_prod collection_druid=oo000oo0001 & # for a long running process, e.g. a collection with more than a few dozen druids, nohup it
If you had errors when indexing from a preassembly/remediation log or from indexing an entire collection, you can re-run the errored out druids only with the log file. All log files are kept in the log folder in the revs-indexer-service app.
rake reindexer RAILS_ENV=production target=revs_prod file=log/logfile.log
nohup rake reindexer RAILS_ENV=production target=revs_prod file=log/logfile.log & # probably no need to nohup unless there were alot of errors
rails c
pid='bb895tg4452' indexer = BaseIndexer::MainIndexerEngine.new # to just perform an index of a document: indexer.index(pid,{'revs'=>true}) # do the full indexing (including posting to solr), the 'revs' indicates the configured solr url target in the solr.yml config file indexer.delete pid # delete the druid (from all targets!! so be careful) # to manually test out mapping solr_doc = BaseIndexer.mapper_class_name.constantize.new(pid).convert_to_solr_doc
bundle exec rspec spec
Should be run automatically via Travis-CI
bundle exec cap production deploy # for production bundle exec cap staging deploy # for staging bundle exec cap development deploy # for development