JohnSnowLabs spark-nlp · Discussions

Import Transformers into Spark NLP 🚀
Announcement maziyarpanahi
Spark NLP 5.0.0: Introducing ONNX Support, State-of-the-Art Instructor Embeddings, E5 Text Embeddings, Document Similarity Ranker, and Much More!
Announcement maziyarpanahi
Spark NLP 5.1.0: Introducing state-of-the-art OpenAI Whisper speech-to-text, OpenAI Embeddings and Completion transformers, MPNet text embeddings, ONNX support for E5 text embeddings, new multi-lingual BART Zero-Shot text classification, and much more!
Announcement maziyarpanahi
Spark NLP 5.2.0: Introducing a Zero-Shot Image Classification by CLIP, ONNX support for T5, Marian, and CamemBERT, a new Text Splitter annotator, Over 8000 state-of-the-art Transformer Models in ONNX, bug fixes, and more!
Announcement maziyarpanahi

Sort by: Latest activity

Discussions

You must be logged in to vote

NER - Results Inconsistence Between Hugging Face Model and SparkNLP model converted

danjmp asked Apr 15, 2024 in Q&A · Unanswered

3
You must be logged in to vote

Spark NLP 5.3.3: Patch release

maziyarpanahi announced Apr 5, 2024 in Announcement

0
You must be logged in to vote

Spark NLP 5.3.2: Patch release

maziyarpanahi announced Mar 20, 2024 in Announcement

0
You must be logged in to vote

XlmRoBertaSentenceEmbeddings returns huge amount of embeddings instead of set dimensions

kkwasnioch asked Feb 20, 2024 in Q&A · Unanswered

10
You must be logged in to vote

Spark NLP 5.3.0: Introducing Llama-2 for CasualLM, M2M100 for Multilingual Translation, MPNet & DeBERTa Enhancements, New Document Similarity Features, Expanded ONNX & In-Memory Support, Updated Runtimes, Essential Bug Fixes, and More!

maziyarpanahi announced Feb 27, 2024 in Announcement

0
You must be logged in to vote

Cannot load PipelineModel trained on PySpark in Scala

SidWeng asked Nov 20, 2023 in Q&A · Answered

6
You must be logged in to vote

可以支持中文的识别吗？

seankilly asked Feb 4, 2024 in Q&A · Unanswered

0
You must be logged in to vote

Sindhi Sentence Embedding

ronit450 asked Jan 17, 2024 in Q&A · Unanswered

10
You must be logged in to vote

OCR Functionality

ArijitSinghEDA asked Jan 20, 2024 in Q&A · Answered

1
You must be logged in to vote

Spark NLP 5.2.3: ONNX support for XLM-RoBERTa Token and Sequence Classifications, and Question Answering task, AWS SDK optimizations, New notebooks, Over 400 new state-of-the-art Transformer Models in ONNX, and bug fixes!

maziyarpanahi announced Jan 18, 2024 in Announcement

0
You must be logged in to vote

Wondering why depend on bundle (fat jar) instead of individual libraries

c3-avidmych asked Jan 8, 2024 in Q&A · Unanswered

1
You must be logged in to vote

Spark NLP 5.2.1: Official support for Apache Spark 3.5, Introducing BGE annotator for Text Embeddings, ONNX support for DeBERTa Token and Sequence Classifications, and Question Answering task, new Databricks 14.x runtimes, Over 400 new state-of-the-art Transformer Models in ONNX, and bug fixes!

maziyarpanahi announced Dec 28, 2023 in Announcement

0
You must be logged in to vote

Spark NLP 5.2.2: Patch release

maziyarpanahi announced Jan 1, 2024 in Announcement

0
Import Transformers into Spark NLP 🚀

maziyarpanahi announced Jun 7, 2021 in Announcement

0
You must be logged in to vote

Spark NLP 5.2.0: Introducing a Zero-Shot Image Classification by CLIP, ONNX support for T5, Marian, and CamemBERT, a new Text Splitter annotator, Over 8000 state-of-the-art Transformer Models in ONNX, bug fixes, and more!

maziyarpanahi announced Dec 8, 2023 in Announcement

0
You must be logged in to vote

About Spark core update 3.4.1 and my working code stop

aksh0369-smarten started Nov 16, 2023 in General

1
You must be logged in to vote

Import a SetFit model (MPNetForSequenceClassification) into Spark-NLP

dfustes asked Nov 10, 2023 in Q&A · Unanswered

3
You must be logged in to vote

Spark NLP 5.1.3: New ONNX Configs, ONNX support for BERT Token and Sequence Classifications, DistilBERT token and sequence classifications, BERT and DistilBERT Question Answering, and bug fixes!

maziyarpanahi announced Oct 10, 2023 in Announcement

0
You must be logged in to vote

Spark NLP 5.1.2: Unveiling the First Image-to-Text VisionEncoderDecoder, Over 3,000 ONNX state-of-the-art Transformer Models, Overhaul update in documentation, and bug fixes!

maziyarpanahi announced Sep 26, 2023 in Announcement

0
You must be logged in to vote

Apache Spark utilizes different amount of RAM in different RAM configurations, is this expected?

KhanjanD asked Sep 14, 2023 in Q&A · Closed · Unanswered

5
You must be logged in to vote

Is there a way to get vocabulary (vector representation of all words) of the pre-trained model of word embedding?

valmirosjunior asked Jun 14, 2022 in Q&A · Unanswered

4
You must be logged in to vote

Spark NLP 5.1.1: Introducing ONNX Support for MPNet, AlbertForTokenClassification, AlbertForSequenceClassification, AlbertForQuestionAnswering transformers, access to full vectors in Word2VecModel, Doc2VecModel, WordEmbeddingsModel annotators, 460+ new ONNX models, and bug fixes!

maziyarpanahi announced Sep 11, 2023 in Announcement

0
You must be logged in to vote

how to select gpu

aapst12 asked Sep 9, 2023 in Q&A · Unanswered

1
You must be logged in to vote

BigTetMatcher Error

naydin2 asked Sep 11, 2023 in Q&A · Closed · Unanswered

2
You must be logged in to vote

document_similarity_ranker between 2 dataframes

JLMarin64 started Aug 28, 2023 in General

0

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Import Transformers into Spark NLP 🚀

Spark NLP 5.0.0: Introducing ONNX Support, State-of-the-Art Instructor Embeddings, E5 Text Embeddings, Document Similarity Ranker, and Much More!

Spark NLP 5.1.0: Introducing state-of-the-art OpenAI Whisper speech-to-text, OpenAI Embeddings and Completion transformers, MPNet text embeddings, ONNX support for E5 text embeddings, new multi-lingual BART Zero-Shot text classification, and much more!

Spark NLP 5.2.0: Introducing a Zero-Shot Image Classification by CLIP, ONNX support for T5, Marian, and CamemBERT, a new Text Splitter annotator, Over 8000 state-of-the-art Transformer Models in ONNX, bug fixes, and more!

Categories

Discussions

NER - Results Inconsistence Between Hugging Face Model and SparkNLP model converted

Spark NLP 5.3.3: Patch release

Spark NLP 5.3.2: Patch release

XlmRoBertaSentenceEmbeddings returns huge amount of embeddings instead of set dimensions

Spark NLP 5.3.0: Introducing Llama-2 for CasualLM, M2M100 for Multilingual Translation, MPNet & DeBERTa Enhancements, New Document Similarity Features, Expanded ONNX & In-Memory Support, Updated Runtimes, Essential Bug Fixes, and More!

Cannot load PipelineModel trained on PySpark in Scala

可以支持中文的识别吗？

Sindhi Sentence Embedding

OCR Functionality

Spark NLP 5.2.3: ONNX support for XLM-RoBERTa Token and Sequence Classifications, and Question Answering task, AWS SDK optimizations, New notebooks, Over 400 new state-of-the-art Transformer Models in ONNX, and bug fixes!

Wondering why depend on bundle (fat jar) instead of individual libraries

Spark NLP 5.2.1: Official support for Apache Spark 3.5, Introducing BGE annotator for Text Embeddings, ONNX support for DeBERTa Token and Sequence Classifications, and Question Answering task, new Databricks 14.x runtimes, Over 400 new state-of-the-art Transformer Models in ONNX, and bug fixes!

Spark NLP 5.2.2: Patch release

Import Transformers into Spark NLP 🚀

Spark NLP 5.2.0: Introducing a Zero-Shot Image Classification by CLIP, ONNX support for T5, Marian, and CamemBERT, a new Text Splitter annotator, Over 8000 state-of-the-art Transformer Models in ONNX, bug fixes, and more!

About Spark core update 3.4.1 and my working code stop

Import a SetFit model (MPNetForSequenceClassification) into Spark-NLP

Spark NLP 5.1.3: New ONNX Configs, ONNX support for BERT Token and Sequence Classifications, DistilBERT token and sequence classifications, BERT and DistilBERT Question Answering, and bug fixes!

Spark NLP 5.1.2: Unveiling the First Image-to-Text VisionEncoderDecoder, Over 3,000 ONNX state-of-the-art Transformer Models, Overhaul update in documentation, and bug fixes!

Apache Spark utilizes different amount of RAM in different RAM configurations, is this expected?

Is there a way to get vocabulary (vector representation of all words) of the pre-trained model of word embedding?

Spark NLP 5.1.1: Introducing ONNX Support for MPNet, AlbertForTokenClassification, AlbertForSequenceClassification, AlbertForQuestionAnswering transformers, access to full vectors in Word2VecModel, Doc2VecModel, WordEmbeddingsModel annotators, 460+ new ONNX models, and bug fixes!

how to select gpu

BigTetMatcher Error

document_similarity_ranker between 2 dataframes

JohnSnowLabs spark-nlp Discussions

Pinned Discussions

Categories, most helpful, and community links

Categories

Community links

Discussions