Register for the 5th NLP Summit, a Free Online Conference on Sep 24-26. Register now.
was successfully added to your cart.

Named Entity Recognition (NER) with BERT in Spark NLP

Training a NER with BERT with a few lines of code in Spark NLP and getting SOTA accuracy.

NER is a subtask of information extraction that seeks to locate and classify named entities mentioned in unstructured text into pre-defined categories such as person names, organizations, locations, medical codes, time expressions, quantities, monetary values, percentages, etc.

BERT for NER

In this post, we will try to show you how to build a state-of-the-art NER model with BERT in Spark NLP library. The model we are going to implement is inspired by a former state of the art model for Named Entity Recognition: Bidirectional LSTM-CNN. This is a novel neural network architecture that automatically detects word- and character-level features using a hybrid bidirectional LSTM and CNN architecture, eliminating the need for most feature engineering.

In Spark NLP, there are three different pre-trained word embeddings models (and their variations) that you can plug and play depending on your use case.

Spark NLP NER

Let’s get started!

We start with downloading the train and test set. We will use the official CoNLL2003 dataset, a benchmark dataset that has been used in nearly all the NER papers. You can download this dataset here.

We import the relevant packages and then start coding.

NER with BERT

Then we convert the CoNLL file to Spark data frame with all the additional fields generated to be used later on.

BERT named entity recognition

The next step is to get the word embeddings through BERT. We will use Spark NLP annotator called BertEmbeddings().

NER with BERT

Then we import the NerDLApproach() annotator, the main module that is responsible for training the NER model.

NER with BERT

Now we can append these two annotators in a pipeline.

NER with BERTFit the pipeline and get predictions.

NER with BERT

In sum, we can train a custom NER in SOTA algorithm with just a few lines of codes in Spark NLP.

You can find the detailed explanations and codes in this notebook.

Read also related articles on the topic:

Named Entity Recognition Python

Try Spark NLP

See in action
preloader