Verify web

[ZoranPandovski]

Description

This PR enhances the Web handler with several updates:

1. Resolves issue #9188.
2. Updates documentation and README to align with our integrations template.
3. Introduces additional unit tests.
4. Refactors numerous helper functions.
5. Improves code quality through linting and the addition of comments.

Type of change

(Please delete options that are not relevant)

• 🐛 Bug fix (non-breaking change which fixes an issue)
• ⚡ New feature (non-breaking change which adds functionality)
• 📢 Breaking change (fix or feature that would cause existing functionality not to work as expected)
• 📄 This change requires a documentation update

Additional Media:

1. Query MindsDB Docs
   
2. Query PDF file from URL
   

Checklist:

• My code follows the style guidelines(PEP 8) of MindsDB.
• I have appropriately commented on my code, especially in complex areas.
• Necessary documentation updates are either made or tracked in issues.
• Relevant unit and integration tests are updated or added.

[MinuraPunchihewa]

Should we also add some tests for CrawlerTable? For it's select() operation to be precise? Such as for correctly raising exceptions when a OR operator is included or when an invalid URL is passed?

[ZoranPandovski]

Yes, I will add them

[MinuraPunchihewa]

@ZoranPandovski Looks like there is a small flake issue to be resolved.

[ZoranPandovski]

@MinuraPunchihewa Fixed, plus few more tests for SELECT