Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add methods to add JSON, CSV and NDJSON document #257

Merged

Conversation

thicolares
Copy link
Contributor

@thicolares thicolares commented Oct 27, 2021

Addresses the following items of #243:

  • addDocumentsJson(string docs, string primaryKey)
  • addDocumentsCsv(string docs, string primaryKey)
  • addDocumentsNdjson(string docs, string primaryKey)

Changes

Tip: in order to ease the code review, you can start reviewing it commit by commit.

  • Expose request options as a parameter of http_post. Motivations:
  • Create methods to add documents as JSON, NDJSON and CSV
    • Add related tests exposing the usage of primaryKeys
    • No "add in batches" included in this PR.

...or not before the request. By exposing the
request options as a Hash param of http_request.

Reason:
- Currently, we transform the HTTP request body into JSON before each request.
- Now, we may have body of different String types (JSON, NDJSON, CSV) that shouldn't be transformed into JSON.
- Thus, we're exposing a flag to tell when a body should be transformed or not into JSON.
- I've decided to use a flag over a complex rules that could check the content along with headers because ideally, the body transformation should be responsibility of a level above that knows about the content.

Other benefit:
- Improve options' organization and its default values, as they were spread around.
- In the future: this could be even extracted into another class, maybe as Value Object

- Note: RuboCop is complaining about "too many optional params" in http_post
- Fix the aux method that merge request options.
- Skip to_json transformation for these methods.
- Add related tests
- Doesn't add in batches.
- Doesn't support key transformations from camelCase to snake_case
...after too many optional params offense in MeiliSearch::HTTPRequest#http_post
@curquiza
Copy link
Member

Hello @thicolares
thanks a lot for your PR and your involvement 😄
I have a lot on my plate right, I'll try to review your PR next week :)
In case you are participating in Hacktoberfest, I'm adding the hacktoberfest-accepted label since your PR is definitely a nice opensource contribution!

@curquiza
Copy link
Member

bors try

meili-bors bot added a commit that referenced this pull request Nov 11, 2021
@meili-bors
Copy link
Contributor

meili-bors bot commented Nov 11, 2021

Copy link
Member

@curquiza curquiza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much @thicolares
And so sorry for the delay, I was really busy

Comment on lines 31 to 35
{ "myId": 123, "title": "Pride and Prejudice", "comment": "A great book" },
{ "myId": 456, "title": "Le Petit Prince", "comment": "A french book" },
{ "myId": 1, "title": "Alice In Wonderland", "comment": "A weird book" },
{ "myId": 1344, "title": "The Hobbit", "comment": "An awesome book" },
{ "myId": 4, "title": "Harry Potter and the Half-Blood Prince", "comment": "The best book" }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you really want to make this test relevant, you might want to use a primary key that does not contain id since MeiliSearch automatically infers it in this case. For example "refNumber"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@curquiza mm.. got it. I'll do that.

@curquiza
Copy link
Member

Hello @thicolares sorry there are now git conlficts with main, could you please remove them? 🙂

@thicolares
Copy link
Contributor Author

@curquiza sure! Sorry about the delay. I believe I'll make the change and push it today :)

@curquiza
Copy link
Member

Take your time, I'm on holiday for two weeks in a few hours ☀️ So I will review when I'll come back

# Conflicts:
#	.rubocop_todo.yml
#	lib/meilisearch/http_request.rb
#	spec/meilisearch/index/documents_spec.rb
- The transform_body method was extract from http_request during c8144ac
- Now, the only possible conversion is to_json
- Then, I rename it to make it not related anyway to the extracted method transform_body
@thicolares thicolares force-pushed the add-add-document-methods-by-its-type branch from 5b95c96 to 9d94183 Compare November 21, 2021 20:59
@thicolares
Copy link
Contributor Author

@curquiza it's done!

Copy link
Member

@curquiza curquiza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @thicolares, for the PR and your patience!

bors merge

@meili-bors
Copy link
Contributor

meili-bors bot commented Dec 7, 2021

Merge conflict.

@curquiza
Copy link
Member

curquiza commented Dec 7, 2021

bors merge

@meili-bors
Copy link
Contributor

meili-bors bot commented Dec 7, 2021

@meili-bors meili-bors bot merged commit 081b8ef into meilisearch:main Dec 7, 2021
@curquiza curquiza added the enhancement New feature or request label Dec 7, 2021
@thicolares thicolares deleted the add-add-document-methods-by-its-type branch December 9, 2021 16:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants