gh-116738: Make _csv module thread-safe #118344
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
corona10
reviewed
Apr 27, 2024
corona10
reviewed
Apr 27, 2024
corona10
reviewed
Apr 28, 2024
Modules/_csv.c
Outdated
| @@ -1582,7 +1585,8 @@ csv_register_dialect(PyObject *module, PyObject *args, PyObject *kwargs) | |||
| dialect = _call_dialect(module_state, dialect_obj, kwargs); | |||
| if (dialect == NULL) | |||
| return NULL; | |||
| if (PyDict_SetItem(module_state->dialects, name_obj, dialect) < 0) { | |||
| res = PyDict_SetItem(module_state->dialects, name_obj, dialect); | |||
There was a problem hiding this comment.
I think that we don't have to use res anymore. :)
corona10
reviewed
Apr 28, 2024
Modules/_csv.c
Outdated
| @@ -34,7 +35,7 @@ typedef struct { | |||
| PyTypeObject *dialect_type; | |||
| PyTypeObject *reader_type; | |||
| PyTypeObject *writer_type; | |||
| long field_limit; /* max parsed field size */ | |||
| int32_t field_limit; /* max parsed field size */ | |||
There was a problem hiding this comment.
Out of curiosity, why is this change needed?
There was a problem hiding this comment.
I saw there is not a _Py_atomic_load_long so I changed it to int32_t, but now I think we can just add it after you mentioned it, thanks for noticing!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This extension module has some module state values which may be read or written in different threads:
field_limit: A simple integer value to store the maximum field size for the parser;dialects: A dict value to store all the dialects by string names.field_limitThere is a public function
field_size_limitthat can read and write it, using a critical section to protect it. For other internal usages,_Py_atomic_load_int32is used to make it thread safe.dialectsThere are three public functions
get_dialect,unregister_dialectandlist_dialectsthat can access this field.For the first two functions, a critical section is used to protect it.And the
list_dialectsfunction just returns the dict's keys withPyDict_Keys. I think its result is readonly andPyDict_Keysitself is thread safe, so we don't need to do anything.Other internal usages of
dialectsinvolve accessing its value withget_dialect_from_registry, which simply callsPyDict_GetItemRefand returns the result. I think we don't need to change it too.Update: I think these APIs is just call
dictAPIs which is already thread-safe, and there are no other state changes inside these APIs, so we don't need add locks on them.