This example shows how to fine-tune a pretrained BERT model for performing text classification.
In this example, you modify a pretrained BERT model for text classification. First, add new layers for classification. Then, retrain the model to fine-tune it, using the original parameters as a starting point. It includes three steps:
- Preprocess text data and initialize BERT model
- Set up and train the network
- Test the model
This example shows the steps for fine-tuning BERT in detail. An alternative approach for document classification using BERT is to use trainBERTDocumentClassifier function.
Clone the repository into a local directory. Open the example script "FineTuning_BERT_for_Classification.mlx".
The example requires data to run. To download the data: :
- Go to https://www.mathworks.com/help/textanalytics/ug/create-simple-text-model-for-classification.html.
- Click on the button "Copy Command" on the top right of the page and paste it in MATLAB Command Window. This will open the example in the directory where the CSV file is stored.
- Copy the CSV file from the example, and paste it in the cloned repository.
- If the file is saved in a different location, change the code to point to the file.
- MATLAB (R2024a or later)
- Text Analytics Toolbox™ (R2024a or later)
- Deep Learning Toolbox™ (R2024a or later)
Sohini Sarkar, ssarkar@mathworks.com
The license is available in license.txt file in this GitHub repository.
Copyright 2024, The MathWorks, Inc.