173 multithread submission processing

[jcadam14]

Closes #173
Closes #181

• Added the multithread_handler.py to handle creating the event loop, and monitoring the future from the process executor
• Added the ProcessPoolExecutor stuff to the submission POST. I was not able to get this working in its own class/function for some reason, I had to add this stuff for the executor and background task directly to the endpoint. Otherwise, the execution task never fired.
• Added pytests

[github-actions]

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
src/sbl_filing_api
config.py
src/sbl_filing_api/entities/engine
engine.py
src/sbl_filing_api/entities/repos
submission_repo.py					107-110
src/sbl_filing_api/routers
filing.py
src/sbl_filing_api/services
multithread_handler.py					18-19
submission_processor.py					127-135
Project Total

This report was generated by python-coverage-comment-action

[lchen-2101]

why isn't this one in the with context block?

[jcadam14]

Because I was lazy? Updated. Removed the concept of the update function and the get_submission function from having a None session. Calling code will now be expected to handle that piece.

Updated the pytests now for all that, too

[lchen-2101]

shouldn't we use the with context here as well?

[jcadam14]

Yup, updated.

[lchen-2101]

don't you need to pass the session into the argument?

[jcadam14]

Man my brain isn't in it this week apparently. Instead of trying to be fancy, I decided to make the update_submission and get_submission param order just like every other function in the submission_repo, which is first the session, then the things you're getting/updating. I then changed all the places using these calls to just pass in the objects, again like every other call does. Updated all the pytests to follow the same idea.

Three other updates:

• First, I noticed the use of the old validation_monitor function in a pytest and that function was still in the submission_processor even though it was never used. Wiped it all out.
• Second, I updated poetry.lock (with poetry lock) to pull in the latest data-validator. I noticed that it didn't have my performance updates. It seems poetry.lock takes a snapshot of the latest commit on main, but then holds that reference id. It wasn't until I ran poetry lock -> poetry install that it pulled in the changes. It seems with the Dockerfile that because we're just doing 'poetry install', if the poetry.lock file isn't updated when git dependencies have new commits to main, the poetry install will use the older commit head. So we just need to make sure to poetry lock occasionally I guess? Unless I'm completely crazy here.
• Finally (not session pun intended), I changed the ValueError that the ContactInfoDTO was throwing on failed regex validation to a RequestValidationError so that the exception handlers would catch it and return in the standard way we're expecting now. Not sure if this was correct way, but that was raising that exception up and causing it to not get to the client the way we expect exception cases to.

[lchen-2101]

Man my brain isn't in it this week apparently. Instead of trying to be fancy, I decided to make the update_submission and get_submission param order just like every other function in the submission_repo, which is first the session, then the things you're getting/updating. I then changed all the places using these calls to just pass in the objects, again like every other call does. Updated all the pytests to follow the same idea.

Three other updates:

• First, I noticed the use of the old validation_monitor function in a pytest and that function was still in the submission_processor even though it was never used. Wiped it all out.
• Second, I updated poetry.lock (with poetry lock) to pull in the latest data-validator. I noticed that it didn't have my performance updates. It seems poetry.lock takes a snapshot of the latest commit on main, but then holds that reference id. It wasn't until I ran poetry lock -> poetry install that it pulled in the changes. It seems with the Dockerfile that because we're just doing 'poetry install', if the poetry.lock file isn't updated when git dependencies have new commits to main, the poetry install will use the older commit head. So we just need to make sure to poetry lock occasionally I guess? Unless I'm completely crazy here.
• Finally (not session pun intended), I changed the ValueError that the ContactInfoDTO was throwing on failed regex validation to a RequestValidationError so that the exception handlers would catch it and return in the standard way we're expecting now. Not sure if this was correct way, but that was raising that exception up and causing it to not get to the client the way we expect exception cases to.

for the 2nd bullet point, I've always just done poetry update {dependency name}, and it pulls in the latest from the repo (it actually is smart enough to see that repo has had no changes and won't update if so), and updates the lock file for me.

for the 3rd bullet point; keep it ValueError, I just tried it, and it gets thrown correctly as the RequestValidationError, which then gets correctly handled by the request validation error handler.

[lchen-2101]

LGTM