Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Include search results from trusted sites #91

Open
1 of 2 tasks
AdiChat opened this issue Mar 9, 2018 · 5 comments · May be fixed by #95 or #103
Open
1 of 2 tasks

Include search results from trusted sites #91

AdiChat opened this issue Mar 9, 2018 · 5 comments · May be fixed by #95 or #103
Labels
enhancement feature

Comments

@AdiChat
Copy link
Member

AdiChat commented Mar 9, 2018

I'm submitting a:

  • bug report
  • feature request

Description:

Include results from other trusted sites for a search term in addition to our current results.

To begin with, we may consider the following trusted sites:

  • Wikipedia
  • Tutorialspoint
  • Stackoverflow

To fetch results, we may use create a crawler to crawl trusted sites or scrape the data from a search engine.

You may fetch a few of the top results from a search engine like Google. For a specific site, use search terms like linear search site:wikipedia.org. The number of results should vary for various sites. For instance, we can provide 1 Wikipedia result, 2 Tutorialspoint result and 4 StackOverflow results. This approach will have limitations so, we can cache results to avoid unnecessary scraping.

The ideal cached results may include the url and the metadata to enable further searches. Implementing this will include a scraping code to present results to users during searching.

Caching can be taken care of in a separate feature. For caching, a separate script can be used to collect results for a set of keywords and store it in a database. Additionally, searching should include our cached results as well.

This feature will make Cosmos-Search more usable in general and help us provide a complete learning experience. 👍
@Shweta4321
Copy link
Contributor

@AdiChat I would like to work on this.

@AdiChat
Copy link
Member Author

AdiChat commented Mar 9, 2018

Sure @Shweta4321 Go for it 👍

Keep us updated on your progress.

@sarvanideekshitula sarvanideekshitula linked a pull request Mar 14, 2018 that will close this issue
Open
2 tasks
@sarvanideekshitula
Copy link
Contributor

@AdiChat Please review the Pull Request

@soc221b
Copy link

soc221b commented Mar 17, 2018

@AdiChat In my opinion, we need to implement this with Ajax and lazy loading tech or we could code with client side (i.e., use JS), because:

  1. Efficiency. Engine are always plan that give precision results which users want, so we should just load recommendations from trust sites as top as need.
  2. Slow down a bit. In some scenario, Users aren't always need all links, and he/she might possibly type wrong string for query.

@Shweta4321 Shweta4321 linked a pull request Mar 18, 2018 that will close this issue
Open
@AdiChat
Copy link
Member Author

AdiChat commented Mar 20, 2018

Yes, this will be nice. 👍

It will reduce the load on the server and serve the initial results faster and thus, improving the user experience. 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement feature
Projects
None yet
Milestone
No milestone
4 participants
@AdiChat @sarvanideekshitula @soc221b @Shweta4321