Bedrock cohere fix max length 2048

[kerlion]

Description

It includes 2 changes:

1. For Cohere, count the tokens as text length to avoid 2048 length limitation..
2. For others, count the tokens with tiktoken tokennizer, uses gpt-4 instead of gpt2.

Fixes #3942

Type of Change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update, included: Dify Document
• Improvement, including but not limited to code refactoring, performance optimization, and UI/UX improvement
• Dependency upgrade

How Has This Been Tested?

Test with text file with consecutive punctuation marks or long English words. It will failed with expected maxLength: 2048 before. Now it is fixed by counting the tokens as chars.

• TODO

Suggested Checklist:

• I have performed a self-review of my own code
• have commented my code, particularly in hard-to-understand areas
• My changes generate no new warnings
• I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods
• optional I have made corresponding changes to the documentation
• optional I have added tests that prove my fix is effective or that my feature works
• optional New and existing unit tests pass locally with my changes

[guchenhe]

please define this in the get_num_tokens where its needed, instead of the class level

[crazywoola]

@kerlion Please resolve the conversation above, we will merge it later.