Dev tech blogs aggregator + frontend app implemented with Meilisearch

[snuzi]

Hi,

Recently I implemented an engineering tech blogs aggregator using MeiliSearch.
The project is divided into two parts:

• snuzi/devblogs-aggregator: A PHP project using meilisearch/meilisearch-php for aggregating blogs of tech companies.
• snuzi/devblogs: React JS + meilisearch/meilisearch-react/

Demo: https://devblogs.net/

Any feedback or contribution is appreciated.

[curquiza]

Hello @snuzi!
Thank you so much for sharing this!! 😄

About the settings:

• you don't need to use id as distinctAttribute because id is your unique field. Here are the docs about the distinct attribute. If its usage is still not clear after reading this, please open an issue in the documentation repository, it would be really helpful 🙏
• could you share your blog link so that I can help you improve the relevance? 😄

About the PHP part (this file):

• you should not use getIndex() method this way: $this->getIndex()->updateSettings($settings); (example line 37). Indeed, getIndex() does an HTTP call, and updateSettings() as well. You should use index() instead that just creates an Index object (so the scope) in order to use the other methods.
  Example: $this->index()->updateSettings($settings);
  You want to use getIndex() only if you want to get/fetch the index information.
• You don't need to create an index before pushing documents. Maybe it's what you want, but MeiliSearch will create an index if it does not already exist. Plus, your primaryKey is id and MeiliSearch will infer it (because there is id in this field), you don't have to pass it during document push or index creation.
• In a more general way, you can check out the getting started in the README of meilisearch-php and the getting started of our docs. They explain better what I mentioned in the previous point (index() method + the implicit index creation).

About the React part:

• it seems ok to me, but I'm not the best in front-end, maybe @mdubus or @bidoubiwa can help you with it?
• in any case, sharing your blog could be definitely a good idea to make feedbacks 🙂

[snuzi]

Hello @curquiza ,
Thank you for the prompt response.

• Your suggested improvements about primaryKey, getIndex() make sense, I will consult with your documentation again because I did not spend much time reading it in details.
• I think you did not understand well how this project is working. I am aggregating and indexing official tech blogs from companies such as https://engineering.zalando.com, https://netflixtechblog.com/ etc. There is a dedicated repo for the blogs RSS feed: snuzi/awesome-dev-blogs. Since I am getting the posts from RSS feed, I think your docs-scraper and docs-searchbar is not necessary for this project.
• Please find the demo frontend app here: https://devblogs.net

[curquiza]

Your website is really nice!!!!

Your settings seem perfectly adapted for your usecase (except the distinctAttribute).
I tried your search bar and the relevance is good 👍

[curquiza]

Also, I don't know if it's your usecase, but you might want to take a look at our scraper and its associated search bar for blog/documentation.

• scraper: https://github.com/meilisearch/docs-scraper
• the search bar: https://github.com/meilisearch/docs-searchbar.js
• the article in our docs to use them: https://docs.meilisearch.com/create/how_to/search_bar_for_docs.html#integrate-a-relevant-search-bar-to-your-documentation

[bidoubiwa]

Hello @snuzi your website is really cool!
One thing I have noticed is a weird behavior on the facets:

Is this something you specifically wanted or is it an unexpected behavior?

[snuzi]

Hi @bidoubiwa
Thank you for the kind words and the feedback.

To me this looks like an intended behavior because the blogName (see index-settings.json) facet is a string datatype in the post object. Categories are an `array and seems to work as you are expecting.

This is the first time that I use MeiliSearch, so I think that there is an AND logical operator between Blogs facets.

[bidoubiwa]

Oh I see! one article can only come from one blog. Thus the zeros. It makes sense :)