Improving Thai Named Entity Recognition Performance Using BERT Transformer on Deep Networks

Emerging of deep learning and transformer model helps advance many NLP tasks. For Name Entity Recognition (NER), many studies have applied deep transformer architect with Google BERT and ELMO to English language and achieved significant performance improvement compared to traditional embeddings. However, currently there is very little research on applying BERT transformer to Thai NER task, so in this paper we explore two different approaches to apply BERT transform to improve Thai NER which are fine-tuning BERT and using BERT as embedding. We found that Bi-LSTM-CRF network with BERT embedding model achieved a significant performance improvement with averaged F1 Score of 94%, without character embedding feature. This approach yields better performance than using only fine-tuning BERT for Thai NER and better than state of the art on Thai NER from Bi-LSTM with Thai2fit word embedding and character embedding.