AutoMM for Named Entity Recognition in Chinese - Quick Start¶

In this tutorial, we will demonstrate how to use AutoMM for Chinese Named Entity Recognition using an e-commerce dataset extracted from one of the most popular online marketplaces, TaoBao.com. The dataset is collected and labelled by Jie et al. and the text column mainly consists of product descriptions. The following figure shows an example of Taobao product description.

Taobao product description. A rabbit toy for lunar new year decoration.

Load the Data¶

We have preprocessed the dataset to make it ready-to-use with AutoMM.

import autogluon.multimodal
from autogluon.core.utils.loaders import load_pd
from autogluon.multimodal.utils import visualize_ner
train_data = load_pd.load('https://automl-mm-bench.s3.amazonaws.com/ner/taobao-ner/chinese_ner_train.csv')
dev_data = load_pd.load('https://automl-mm-bench.s3.amazonaws.com/ner/taobao-ner/chinese_ner_dev.csv')
train_data.head(5)

	text_snippet	entity_annotations
0	雄争霸点卡/七雄争霸元宝/七雄争霸100元1000元宝直充,自动充值	[{"entity_group": "HCCX", "start": 3, "end": 5...
1	简约韩版粗跟艾熙百思图亲子鞋冬季百搭街头母女圆头翻边绒面厚底	[{"entity_group": "HPPX", "start": 6, "end": 8...
2	羚跑商务背包双肩包男士防盗多功能出差韩版休闲15.6寸电脑包皮潮	[{"entity_group": "HPPX", "start": 0, "end": 2...
3	热水袋防爆充电暖宝卡通毛绒萌萌可爱注水暖宫暖手宝暖水袋	[{"entity_group": "HCCX", "start": 0, "end": 3...
4	童装11周岁13儿童夏装男童套装2017新款10中大童15男孩12秋季5潮7	[{"entity_group": "HCCX", "start": 0, "end": 2...

HPPX, HCCX, XH, and MISC stand for brand, product, pattern, and Miscellaneous information (e.g., product Specification), respectively. Let’s visualize one of the examples, which is about online games top up services.

visualize_ner(train_data["text_snippet"].iloc[0], train_data["entity_annotations"].iloc[0])

雄争霸点卡 HCCX /七雄争霸 MISC 元宝 HCCX /七雄争霸 MISC 100元 MISC 1000 MISC 元宝 HCCX 直充,自动充值

Training¶

With AutoMM, the process of Chinese entity recognition is the same as English entity recognition. All you need to do is to select a suitable foundation model checkpoint that are pretrained on Chinese or multilingual documents. Here we use the 'hfl/chinese-lert-small' backbone for demonstration purpose.

Now, let’s create a predictor for named entity recognition by setting the problem_type to ner and specifying the label column. Afterwards, we call predictor.fit() to train the model for a few minutes.

from autogluon.multimodal import MultiModalPredictor
import uuid

label_col = "entity_annotations"
model_path = f"./tmp/{uuid.uuid4().hex}-automm_ner"  # You can rename it to the model path you like
predictor = MultiModalPredictor(problem_type="ner", label=label_col, path=model_path)
predictor.fit(
    train_data=train_data,
    hyperparameters={'model.ner_text.checkpoint_name':'hfl/chinese-lert-small'},
    time_limit=300, #second
)

=================== System Info ===================
AutoGluon Version:  1.1.1b20240613
Python Version:     3.10.13
Operating System:   Linux
Platform Machine:   x86_64
Platform Version:   #1 SMP Fri May 17 18:07:48 UTC 2024
CPU Count:          8
Pytorch Version:    2.3.1+cu121
CUDA Version:       12.1
Memory Avail:       28.65 GB / 30.95 GB (92.6%)
Disk Space Avail:   190.25 GB / 255.99 GB (74.3%)
===================================================

AutoMM starts to create your model. ✨✨✨

To track the learning progress, you can open a terminal and launch Tensorboard:
    ```shell
    # Assume you have installed tensorboard
    tensorboard --logdir /home/ci/autogluon/docs/tutorials/multimodal/text_prediction/tmp/4ff9a06316a249618b4ca42dd61e24b5-automm_ner
    ```

Seed set to 0
/home/ci/opt/venv/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
GPU Count: 1
GPU Count to be Used: 1
GPU 0 Name: Tesla T4
GPU 0 Memory: 0.42GB/15.0GB (Used/Total)

Using 16bit Automatic Mixed Precision (AMP)
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
HPU available: False, using: 0 HPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]

  | Name              | Type              | Params | Mode 
----------------------------------------------------------------
0 | model             | HFAutoModelForNER | 15.1 M | train
1 | validation_metric | MulticlassF1Score | 0      | train
2 | loss_func         | CrossEntropyLoss  | 0      | train
----------------------------------------------------------------
15.1 M    Trainable params
0         Non-trainable params
15.1 M    Total params
60.345    Total estimated model params size (MB)
Epoch 0, global step 21: 'val_ner_token_f1' reached 0.30200 (best 0.30200), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/text_prediction/tmp/4ff9a06316a249618b4ca42dd61e24b5-automm_ner/epoch=0-step=21.ckpt' as top 3
Epoch 0, global step 42: 'val_ner_token_f1' reached 0.58097 (best 0.58097), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/text_prediction/tmp/4ff9a06316a249618b4ca42dd61e24b5-automm_ner/epoch=0-step=42.ckpt' as top 3
Epoch 1, global step 64: 'val_ner_token_f1' reached 0.69155 (best 0.69155), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/text_prediction/tmp/4ff9a06316a249618b4ca42dd61e24b5-automm_ner/epoch=1-step=64.ckpt' as top 3
Epoch 1, global step 85: 'val_ner_token_f1' reached 0.73002 (best 0.73002), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/text_prediction/tmp/4ff9a06316a249618b4ca42dd61e24b5-automm_ner/epoch=1-step=85.ckpt' as top 3
Epoch 2, global step 107: 'val_ner_token_f1' reached 0.77655 (best 0.77655), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/text_prediction/tmp/4ff9a06316a249618b4ca42dd61e24b5-automm_ner/epoch=2-step=107.ckpt' as top 3
Epoch 2, global step 128: 'val_ner_token_f1' reached 0.83161 (best 0.83161), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/text_prediction/tmp/4ff9a06316a249618b4ca42dd61e24b5-automm_ner/epoch=2-step=128.ckpt' as top 3
Epoch 3, global step 150: 'val_ner_token_f1' reached 0.84013 (best 0.84013), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/text_prediction/tmp/4ff9a06316a249618b4ca42dd61e24b5-automm_ner/epoch=3-step=150.ckpt' as top 3
Epoch 3, global step 171: 'val_ner_token_f1' reached 0.83875 (best 0.84013), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/text_prediction/tmp/4ff9a06316a249618b4ca42dd61e24b5-automm_ner/epoch=3-step=171.ckpt' as top 3
Epoch 4, global step 193: 'val_ner_token_f1' reached 0.84704 (best 0.84704), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/text_prediction/tmp/4ff9a06316a249618b4ca42dd61e24b5-automm_ner/epoch=4-step=193.ckpt' as top 3
Epoch 4, global step 214: 'val_ner_token_f1' reached 0.86800 (best 0.86800), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/text_prediction/tmp/4ff9a06316a249618b4ca42dd61e24b5-automm_ner/epoch=4-step=214.ckpt' as top 3
Epoch 5, global step 236: 'val_ner_token_f1' reached 0.87169 (best 0.87169), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/text_prediction/tmp/4ff9a06316a249618b4ca42dd61e24b5-automm_ner/epoch=5-step=236.ckpt' as top 3
Time limit reached. Elapsed time is 0:05:00. Signaling Trainer to stop.
Epoch 5, global step 244: 'val_ner_token_f1' reached 0.87445 (best 0.87445), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/text_prediction/tmp/4ff9a06316a249618b4ca42dd61e24b5-automm_ner/epoch=5-step=244.ckpt' as top 3
Start to fuse 3 checkpoints via the greedy soup algorithm.
AutoMM has created your model. 🎉🎉🎉

To load the model, use the code below:
    ```python
    from autogluon.multimodal import MultiModalPredictor
    predictor = MultiModalPredictor.load("/home/ci/autogluon/docs/tutorials/multimodal/text_prediction/tmp/4ff9a06316a249618b4ca42dd61e24b5-automm_ner")
    ```

If you are not satisfied with the model, try to increase the training time, 
adjust the hyperparameters (https://auto.gluon.ai/stable/tutorials/multimodal/advanced_topics/customization.html),
or post issues on GitHub (https://github.com/autogluon/autogluon/issues).

<autogluon.multimodal.predictor.MultiModalPredictor at 0x7fe97f4935b0>

Evaluation¶

To check the model performance on the test dataset, all you need to do is to call predictor.evaluate(...).

predictor.evaluate(dev_data)

{'hccx': {'precision': 0.8027910392948954,
  'recall': 0.8569188553508428,
  'f1': 0.8289723170269244,
  'number': 2551},
 'hppx': {'precision': 0.6666666666666666,
  'recall': 0.6258992805755396,
  'f1': 0.6456400742115027,
  'number': 278},
 'misc': {'precision': 0.6637781629116117,
  'recall': 0.7599206349206349,
  'f1': 0.7086031452358926,
  'number': 504},
 'xh': {'precision': 0.6666666666666666,
  'recall': 0.6974789915966386,
  'f1': 0.6817248459958931,
  'number': 238},
 'overall_precision': 0.763517060367454,
 'overall_recall': 0.8146177541304956,
 'overall_f1': 0.7882400758704782,
 'overall_accuracy': 0.8910366650615671}

Prediction and Visualization¶

You can easily obtain the predictions given an input sentence by by calling predictor.predict(...).

output = predictor.predict(dev_data)
visualize_ner(dev_data["text_snippet"].iloc[0], output[0])

家用防尘厨房厨师帽子 HCCX 车间工厂鸭舌工作帽 HCCX 男女食堂餐厅食 HCCX 品卫生帽 HCCX

Now, let’s make predictions on the rabbit toy example.

sentence = "2023年兔年挂件新年装饰品小挂饰乔迁之喜门挂小兔子"
predictions = predictor.predict({'text_snippet': [sentence]})
visualize_ner(sentence, predictions[0])

2023年兔年挂件 HCCX 新年装饰品 HCCX 小挂饰 HCCX 乔迁 MISC 之 HPPX 喜门挂 HCCX 小兔子 HCCX

Other Examples¶

You may go to AutoMM Examples to explore other examples about AutoMM.

Customization¶

To learn how to customize AutoMM, please refer to Customize AutoMM.