Back out "also test in 3.11 / 3.12"

Summary:
Looks like this was shipped too early, but I didn't notice the 3.11/3.12-specific failures because they were masked by all the other test failures. Now that the other builds are green, I see that we *do* actually have some 3.11+ specific failures that need to be dealt with, so my plan is to revert this to get the build back to green, and then make a second PR to introduce support for 3.11+ in a more official way.

Original commit changeset: f568762544a3

Original Phabricator Diff: D55368002

Reviewed By: deathowl

Differential Revision: D55380013

fbshipit-source-id: cdb4d48178a54fea6ebc41ce9b12bebc823b340a

.github/workflows/build.yml

@@ -8,7 +8,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: ["3.7", "3.8", "3.9", "3.10", "3.11", "3.12"]
+        python-version: ["3.7", "3.8", "3.9", "3.10"]
       # Ensure that all flavours are run to completion even if an other flavor failed
       fail-fast: false