Audit/cleanup the 48 open Solr bugs, some over a decade old

[cclauss]

There are 48 open Solr bugs, some over a decade old.
https://github.com/internetarchive/openlibrary/issues?q=label%3A%22Module%3A+Solr%22+

Is there something that we should be doing to reduce this number and especially where changes would streamline librarian workflows?

Evidence / Screenshot (if possible)

Relevant url?

#3290 (comment)

Stakeholders

@tfmorris @cdrini @mekarpeles @seabelis @BrittanyBunk

[BrittanyBunk]

Thank you for bringing this up @cclauss. I haven't worked on Solr, but I would 'yes' in general to where any Solr issues that should be worked on should be worked on.

I could think of ineffective methods only (like making it more prominent (set the status higher, tag people, etc.), add the Solr wiki page to each issue, create a whole github group dedicated to solr), as I haven't thought about or really worked on Solr before. However, these are going to be useless if only 1-2 people can work on the Solr and they're not available to work on them.

I personally don't know how to get started, so the questions I have are:

1. who has permission to work on Solr?
2. Can anyone do Solr?
3. Where's the database for it if they can (is it on the Wiki page, and if so, where?)?
4. are there going to be overlaying issues if multiple people work on it at the same time (like each person can accidentally undo another person's work if everyone's working on it simultaneously)?

Having this idea I feel is a start for determining what's the next steps in the solution (Note: one of the solutions might be to add instructions on the "enthusiasts and beta testers' page (that I didn't create yet) on how to get started, as all the Wiki instruction pages on Github currently don't explain everything - especially the 'get started' part - only what to do after something's setup. Like the bots page explains what to do with a bot script, but doesn't say how to create one.).

[mekarpeles]

The short answer is 'yes' - any Solr issues that should be worked on should be worked on.

@BrittanyBunk can you please not speak authoritatively as to our roadmap. @cdrini is the lead for Solr. He makes the calls.

[cdrini]

Hey! So I did a full audit of all the "Module: Solr" & "Theme: Search" issues in ~Dec 2019 here: https://coda.io/d/Search-Planning-Notes_dO8sGM90quA/Epics-in-Progress_su1I2#_luBNR . This is linked to on the wiki and project board (although I haven't updated the project board in a while). The two most impactful improvements to solr are (1) create a re-indexable solr (completed this past month); and (2) Update solr to latest version. This is on the milestone for this month. A large chunk of solr issues are blocked on these two things (A lot were blocked on the first). The issue @tfmorris brought up is ~blocked on the second (there are work arounds for our version of solr, but they would require further investment into solr 3.6 specific setup/plugins, which is a waste of time). These 2 issues take a decent amount of work and also require interfacing with the production system, which is why I've taken them up myself. I'll do another audit once these 2 things are complete, since a lot more things should be feasible after that.

I'll close this once that audit is complete 👍

[BrittanyBunk]

@cdrini That's what I was thinking. It needs to switch over to do everything else - I assumed that there's no need to try to make everything work on an outdated Solr - as that won't accomplish much. I mean, you said you'll get to it - and I believe that would be done once the transition is complete - so it made sense to me and I understand the timeline.

Question: are you the only person that can work on the solr bugs? Just so I know to avoid them if no one else can.

[BrittanyBunk]

@BrittanyBunk can you please not speak authoritatively as to our roadmap. @cdrini is the lead for Solr. He makes the calls.

I'm just referring (or 'echoing' if that's a preferable word - not being the decision-maker - I didn't say I would work on them specifically) to what @cdrini and others mentioned before in what would be work on - that yes - some of these do need to be worked on - according to what was brought up in the community meetings. So I'm really confused by this.

Outside of that - it's off topic of what this issue is.

[mekarpeles]

@cclauss point is well taken; This issue itself doesn't prescribe an actionable, closing and leaving to @cdrini to create the appropriate solr project board or epic

[cdrini]

Updated the title to make actionable + added to board 👍

[cdrini]

@BrittanyBunk Nope! The only thing that was explicitly dependent on me was the solr-reindexing flow. Now that we're using docker, anyone should be able to work on anything. I've taken up solr 8 upgrade because I think it's important and requires a decent chunk of time which volunteers might not be able to commit to.

[BrittanyBunk]

@cdrini smart! Thanks. Awesome. I can't run docker, due to not having Windows 10 (I have 8.1). Is there a workaround for it, or is it all docker?

PS - I wish the docker documentation on the OL said both: 1) requirements to install, 2) what it's used for. Mek didn't want too much info that developers don't need it - because they created the software. That's not going to be good for newcomers who may volunteer, now that you did the grunt work. I'm just planning on putting it in the EBT page, but I think I will wait until everything's finished - so the instructions match with what's available. This is just a side note for what I'm doing based on what you said.

[cclauss]

Docker will install on Windows 8.1 as discussed at https://docs.docker.com/toolbox/toolbox_install_windows

[mekarpeles]

Closing this issue as we have

1. Improving Search (No Dead Ends) #2728 for tracking progress in 2021
2. this tracking doc https://coda.io/d/Search-Planning-Notes_dO8sGM90quA/Epics-in-Progress_su1I2#_luBNR
3. new Module: Solr and Theme: Search managed labels added

Module: Solr Issues related to the configuration or use of the Solr subsystem. [managed]
Theme: Search Issues related to search UI and backend. [managed]