AutoGluon Multimodal - Quick Start¶
AutoGluon’s MultiModalPredictor is a deep learning model zoo of model zoos that can automatically build state-of-the-art deep learning models for inputs including images, text, and tabular data. Convert your data into AutoGluon’s multimodal dataframe format, and MultiModalPredictor can predict the values of one column based on the other features.
Begin by making sure AutoGluon is installed, and then import the required modules.
!python -m pip install --upgrade pip
!python -m pip install autogluon
Show code cell output
Requirement already satisfied: pip in /home/ci/opt/venv/lib/python3.10/site-packages (24.1.2)
Requirement already satisfied: autogluon in /home/ci/autogluon/autogluon/src (1.1.1b20240716)
Requirement already satisfied: autogluon.core==1.1.1b20240716 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.core[all]==1.1.1b20240716->autogluon) (1.1.1b20240716)
Requirement already satisfied: autogluon.features==1.1.1b20240716 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon) (1.1.1b20240716)
Requirement already satisfied: autogluon.tabular==1.1.1b20240716 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.tabular[all]==1.1.1b20240716->autogluon) (1.1.1b20240716)
Requirement already satisfied: autogluon.multimodal==1.1.1b20240716 in /home/ci/autogluon/multimodal/src (from autogluon) (1.1.1b20240716)
Requirement already satisfied: autogluon.timeseries==1.1.1b20240716 in /home/ci/autogluon/timeseries/src (from autogluon.timeseries[all]==1.1.1b20240716->autogluon) (1.1.1b20240716)
Requirement already satisfied: numpy<1.29,>=1.21 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (1.26.4)
Requirement already satisfied: scipy<1.13,>=1.5.4 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (1.12.0)
Requirement already satisfied: scikit-learn<1.4.1,>=1.3.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (1.4.0)
Requirement already satisfied: networkx<4,>=3.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (3.3)
Requirement already satisfied: pandas<2.3.0,>=2.0.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (2.2.2)
Requirement already satisfied: tqdm<5,>=4.38 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (4.66.4)
Requirement already satisfied: requests in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (2.32.3)
Requirement already satisfied: matplotlib in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (3.6.3)
Requirement already satisfied: boto3<2,>=1.10 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (1.34.144)
Requirement already satisfied: autogluon.common==1.1.1b20240716 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (1.1.1b20240716)
Requirement already satisfied: ray<2.11,>=2.10.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from ray[default]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (2.10.0)
Requirement already satisfied: hyperopt<0.2.8,>=0.2.7 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.core[all]==1.1.1b20240716->autogluon) (0.2.7)
Requirement already satisfied: Pillow<11,>=10.0.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.multimodal==1.1.1b20240716->autogluon) (10.4.0)
Requirement already satisfied: torch<2.4,>=2.2 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.multimodal==1.1.1b20240716->autogluon) (2.3.1)
Requirement already satisfied: lightning<2.4,>=2.2 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.multimodal==1.1.1b20240716->autogluon) (2.3.3)
Requirement already satisfied: transformers<4.41.0,>=4.38.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from transformers[sentencepiece]<4.41.0,>=4.38.0->autogluon.multimodal==1.1.1b20240716->autogluon) (4.39.3)
Requirement already satisfied: accelerate<0.22.0,>=0.21.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.multimodal==1.1.1b20240716->autogluon) (0.21.0)
Requirement already satisfied: jsonschema<4.22,>=4.18 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.multimodal==1.1.1b20240716->autogluon) (4.21.1)
Requirement already satisfied: seqeval<1.3.0,>=1.2.2 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.multimodal==1.1.1b20240716->autogluon) (1.2.2)
Requirement already satisfied: evaluate<0.5.0,>=0.4.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.multimodal==1.1.1b20240716->autogluon) (0.4.2)
Requirement already satisfied: timm<0.10.0,>=0.9.5 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.multimodal==1.1.1b20240716->autogluon) (0.9.16)
Requirement already satisfied: torchvision<0.19.0,>=0.16.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.multimodal==1.1.1b20240716->autogluon) (0.18.1)
Requirement already satisfied: scikit-image<0.21.0,>=0.19.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.multimodal==1.1.1b20240716->autogluon) (0.20.0)
Requirement already satisfied: text-unidecode<1.4,>=1.3 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.multimodal==1.1.1b20240716->autogluon) (1.3)
Requirement already satisfied: torchmetrics<1.3.0,>=1.2.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.multimodal==1.1.1b20240716->autogluon) (1.2.1)
Requirement already satisfied: nptyping<2.5.0,>=1.4.4 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.multimodal==1.1.1b20240716->autogluon) (2.4.1)
Requirement already satisfied: omegaconf<2.3.0,>=2.1.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.multimodal==1.1.1b20240716->autogluon) (2.2.3)
Requirement already satisfied: pytorch-metric-learning<2.4,>=1.3.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.multimodal==1.1.1b20240716->autogluon) (2.3.0)
Requirement already satisfied: nlpaug<1.2.0,>=1.1.10 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.multimodal==1.1.1b20240716->autogluon) (1.1.11)
Requirement already satisfied: nltk<4.0.0,>=3.4.5 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.multimodal==1.1.1b20240716->autogluon) (3.8.1)
Requirement already satisfied: openmim<0.4.0,>=0.3.7 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.multimodal==1.1.1b20240716->autogluon) (0.3.9)
Requirement already satisfied: defusedxml<0.7.2,>=0.7.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.multimodal==1.1.1b20240716->autogluon) (0.7.1)
Requirement already satisfied: jinja2<3.2,>=3.0.3 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.multimodal==1.1.1b20240716->autogluon) (3.1.4)
Requirement already satisfied: tensorboard<3,>=2.9 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.multimodal==1.1.1b20240716->autogluon) (2.17.0)
Requirement already satisfied: pytesseract<0.3.11,>=0.3.9 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.multimodal==1.1.1b20240716->autogluon) (0.3.10)
Requirement already satisfied: nvidia-ml-py3==7.352.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.multimodal==1.1.1b20240716->autogluon) (7.352.0)
Requirement already satisfied: pdf2image<1.19,>=1.17.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.multimodal==1.1.1b20240716->autogluon) (1.17.0)
Requirement already satisfied: fastai<2.8,>=2.3.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.tabular[all]==1.1.1b20240716->autogluon) (2.7.15)
Requirement already satisfied: xgboost<2.1,>=1.6 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.tabular[all]==1.1.1b20240716->autogluon) (2.0.3)
Requirement already satisfied: lightgbm<4.4,>=3.3 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.tabular[all]==1.1.1b20240716->autogluon) (4.3.0)
Requirement already satisfied: catboost<1.3,>=1.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.tabular[all]==1.1.1b20240716->autogluon) (1.2.5)
Requirement already satisfied: joblib<2,>=1.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.timeseries==1.1.1b20240716->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (1.4.2)
Requirement already satisfied: pytorch_lightning<2.4,>=2.2 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.timeseries==1.1.1b20240716->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (2.3.3)
Requirement already satisfied: gluonts==0.15.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.timeseries==1.1.1b20240716->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (0.15.1)
Requirement already satisfied: statsforecast<1.5,>=1.4.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.timeseries==1.1.1b20240716->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (1.4.0)
Requirement already satisfied: mlforecast<0.10.1,>=0.10.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.timeseries==1.1.1b20240716->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (0.10.0)
Requirement already satisfied: utilsforecast<0.0.11,>=0.0.10 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.timeseries==1.1.1b20240716->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (0.0.10)
Requirement already satisfied: orjson~=3.9 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.timeseries==1.1.1b20240716->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (3.10.6)
Requirement already satisfied: optimum<1.19,>=1.17 in /home/ci/opt/venv/lib/python3.10/site-packages (from optimum[onnxruntime]<1.19,>=1.17->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (1.18.1)
Requirement already satisfied: psutil<6,>=5.7.3 in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.common==1.1.1b20240716->autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (5.9.8)
Requirement already satisfied: setuptools in /home/ci/opt/venv/lib/python3.10/site-packages (from autogluon.common==1.1.1b20240716->autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (65.5.0)
Requirement already satisfied: pydantic<3,>=1.7 in /home/ci/opt/venv/lib/python3.10/site-packages (from gluonts==0.15.1->autogluon.timeseries==1.1.1b20240716->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (2.8.2)
Requirement already satisfied: toolz~=0.10 in /home/ci/opt/venv/lib/python3.10/site-packages (from gluonts==0.15.1->autogluon.timeseries==1.1.1b20240716->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (0.12.1)
Requirement already satisfied: typing-extensions~=4.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from gluonts==0.15.1->autogluon.timeseries==1.1.1b20240716->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (4.12.2)
Requirement already satisfied: packaging>=20.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from accelerate<0.22.0,>=0.21.0->autogluon.multimodal==1.1.1b20240716->autogluon) (24.1)
Requirement already satisfied: pyyaml in /home/ci/opt/venv/lib/python3.10/site-packages (from accelerate<0.22.0,>=0.21.0->autogluon.multimodal==1.1.1b20240716->autogluon) (6.0.1)
Requirement already satisfied: botocore<1.35.0,>=1.34.144 in /home/ci/opt/venv/lib/python3.10/site-packages (from boto3<2,>=1.10->autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (1.34.144)
Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from boto3<2,>=1.10->autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (1.0.1)
Requirement already satisfied: s3transfer<0.11.0,>=0.10.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from boto3<2,>=1.10->autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (0.10.2)
Requirement already satisfied: graphviz in /home/ci/opt/venv/lib/python3.10/site-packages (from catboost<1.3,>=1.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (0.20.3)
Requirement already satisfied: plotly in /home/ci/opt/venv/lib/python3.10/site-packages (from catboost<1.3,>=1.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (5.22.0)
Requirement already satisfied: six in /home/ci/opt/venv/lib/python3.10/site-packages (from catboost<1.3,>=1.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (1.16.0)
Requirement already satisfied: datasets>=2.0.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from evaluate<0.5.0,>=0.4.0->autogluon.multimodal==1.1.1b20240716->autogluon) (2.20.0)
Requirement already satisfied: dill in /home/ci/opt/venv/lib/python3.10/site-packages (from evaluate<0.5.0,>=0.4.0->autogluon.multimodal==1.1.1b20240716->autogluon) (0.3.8)
Requirement already satisfied: xxhash in /home/ci/opt/venv/lib/python3.10/site-packages (from evaluate<0.5.0,>=0.4.0->autogluon.multimodal==1.1.1b20240716->autogluon) (3.4.1)
Requirement already satisfied: multiprocess in /home/ci/opt/venv/lib/python3.10/site-packages (from evaluate<0.5.0,>=0.4.0->autogluon.multimodal==1.1.1b20240716->autogluon) (0.70.16)
Requirement already satisfied: fsspec>=2021.05.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from fsspec[http]>=2021.05.0->evaluate<0.5.0,>=0.4.0->autogluon.multimodal==1.1.1b20240716->autogluon) (2024.5.0)
Requirement already satisfied: huggingface-hub>=0.7.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from evaluate<0.5.0,>=0.4.0->autogluon.multimodal==1.1.1b20240716->autogluon) (0.23.5)
Requirement already satisfied: pip in /home/ci/opt/venv/lib/python3.10/site-packages (from fastai<2.8,>=2.3.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (24.1.2)
Requirement already satisfied: fastdownload<2,>=0.0.5 in /home/ci/opt/venv/lib/python3.10/site-packages (from fastai<2.8,>=2.3.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (0.0.7)
Requirement already satisfied: fastcore<1.6,>=1.5.29 in /home/ci/opt/venv/lib/python3.10/site-packages (from fastai<2.8,>=2.3.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (1.5.54)
Requirement already satisfied: fastprogress>=0.2.4 in /home/ci/opt/venv/lib/python3.10/site-packages (from fastai<2.8,>=2.3.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (1.0.3)
Requirement already satisfied: spacy<4 in /home/ci/opt/venv/lib/python3.10/site-packages (from fastai<2.8,>=2.3.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (3.7.5)
Requirement already satisfied: future in /home/ci/opt/venv/lib/python3.10/site-packages (from hyperopt<0.2.8,>=0.2.7->autogluon.core[all]==1.1.1b20240716->autogluon) (1.0.0)
Requirement already satisfied: cloudpickle in /home/ci/opt/venv/lib/python3.10/site-packages (from hyperopt<0.2.8,>=0.2.7->autogluon.core[all]==1.1.1b20240716->autogluon) (3.0.0)
Requirement already satisfied: py4j in /home/ci/opt/venv/lib/python3.10/site-packages (from hyperopt<0.2.8,>=0.2.7->autogluon.core[all]==1.1.1b20240716->autogluon) (0.10.9.7)
Requirement already satisfied: MarkupSafe>=2.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from jinja2<3.2,>=3.0.3->autogluon.multimodal==1.1.1b20240716->autogluon) (2.1.5)
Requirement already satisfied: attrs>=22.2.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from jsonschema<4.22,>=4.18->autogluon.multimodal==1.1.1b20240716->autogluon) (23.2.0)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /home/ci/opt/venv/lib/python3.10/site-packages (from jsonschema<4.22,>=4.18->autogluon.multimodal==1.1.1b20240716->autogluon) (2023.12.1)
Requirement already satisfied: referencing>=0.28.4 in /home/ci/opt/venv/lib/python3.10/site-packages (from jsonschema<4.22,>=4.18->autogluon.multimodal==1.1.1b20240716->autogluon) (0.35.1)
Requirement already satisfied: rpds-py>=0.7.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from jsonschema<4.22,>=4.18->autogluon.multimodal==1.1.1b20240716->autogluon) (0.19.0)
Requirement already satisfied: lightning-utilities<2.0,>=0.10.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from lightning<2.4,>=2.2->autogluon.multimodal==1.1.1b20240716->autogluon) (0.11.5)
Requirement already satisfied: numba in /home/ci/opt/venv/lib/python3.10/site-packages (from mlforecast<0.10.1,>=0.10.0->autogluon.timeseries==1.1.1b20240716->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (0.60.0)
Requirement already satisfied: window-ops in /home/ci/opt/venv/lib/python3.10/site-packages (from mlforecast<0.10.1,>=0.10.0->autogluon.timeseries==1.1.1b20240716->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (0.0.15)
Requirement already satisfied: gdown>=4.0.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from nlpaug<1.2.0,>=1.1.10->autogluon.multimodal==1.1.1b20240716->autogluon) (5.2.0)
Requirement already satisfied: click in /home/ci/opt/venv/lib/python3.10/site-packages (from nltk<4.0.0,>=3.4.5->autogluon.multimodal==1.1.1b20240716->autogluon) (8.1.7)
Requirement already satisfied: regex>=2021.8.3 in /home/ci/opt/venv/lib/python3.10/site-packages (from nltk<4.0.0,>=3.4.5->autogluon.multimodal==1.1.1b20240716->autogluon) (2024.5.15)
Requirement already satisfied: antlr4-python3-runtime==4.9.* in /home/ci/opt/venv/lib/python3.10/site-packages (from omegaconf<2.3.0,>=2.1.1->autogluon.multimodal==1.1.1b20240716->autogluon) (4.9.3)
Requirement already satisfied: colorama in /home/ci/opt/venv/lib/python3.10/site-packages (from openmim<0.4.0,>=0.3.7->autogluon.multimodal==1.1.1b20240716->autogluon) (0.4.4)
Requirement already satisfied: model-index in /home/ci/opt/venv/lib/python3.10/site-packages (from openmim<0.4.0,>=0.3.7->autogluon.multimodal==1.1.1b20240716->autogluon) (0.1.11)
Requirement already satisfied: opendatalab in /home/ci/opt/venv/lib/python3.10/site-packages (from openmim<0.4.0,>=0.3.7->autogluon.multimodal==1.1.1b20240716->autogluon) (0.0.10)
Requirement already satisfied: rich in /home/ci/opt/venv/lib/python3.10/site-packages (from openmim<0.4.0,>=0.3.7->autogluon.multimodal==1.1.1b20240716->autogluon) (13.7.1)
Requirement already satisfied: tabulate in /home/ci/opt/venv/lib/python3.10/site-packages (from openmim<0.4.0,>=0.3.7->autogluon.multimodal==1.1.1b20240716->autogluon) (0.9.0)
Requirement already satisfied: coloredlogs in /home/ci/opt/venv/lib/python3.10/site-packages (from optimum<1.19,>=1.17->optimum[onnxruntime]<1.19,>=1.17->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (15.0.1)
Requirement already satisfied: sympy in /home/ci/opt/venv/lib/python3.10/site-packages (from optimum<1.19,>=1.17->optimum[onnxruntime]<1.19,>=1.17->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (1.13.0)
Requirement already satisfied: onnx in /home/ci/opt/venv/lib/python3.10/site-packages (from optimum[onnxruntime]<1.19,>=1.17->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (1.16.1)
Requirement already satisfied: onnxruntime>=1.11.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from optimum[onnxruntime]<1.19,>=1.17->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (1.18.1)
Requirement already satisfied: protobuf>=3.20.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from optimum[onnxruntime]<1.19,>=1.17->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (3.20.2)
Requirement already satisfied: python-dateutil>=2.8.2 in /home/ci/opt/venv/lib/python3.10/site-packages (from pandas<2.3.0,>=2.0.0->autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from pandas<2.3.0,>=2.0.0->autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in /home/ci/opt/venv/lib/python3.10/site-packages (from pandas<2.3.0,>=2.0.0->autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (2024.1)
Requirement already satisfied: filelock in /home/ci/opt/venv/lib/python3.10/site-packages (from ray<2.11,>=2.10.0->ray[default]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (3.15.4)
Requirement already satisfied: msgpack<2.0.0,>=1.0.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from ray<2.11,>=2.10.0->ray[default]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (1.0.8)
Requirement already satisfied: aiosignal in /home/ci/opt/venv/lib/python3.10/site-packages (from ray<2.11,>=2.10.0->ray[default]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (1.3.1)
Requirement already satisfied: frozenlist in /home/ci/opt/venv/lib/python3.10/site-packages (from ray<2.11,>=2.10.0->ray[default]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (1.4.1)
Requirement already satisfied: tensorboardX>=1.9 in /home/ci/opt/venv/lib/python3.10/site-packages (from ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (2.6.2.2)
Requirement already satisfied: pyarrow>=6.0.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (17.0.0)
Requirement already satisfied: aiohttp>=3.7 in /home/ci/opt/venv/lib/python3.10/site-packages (from ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (3.9.5)
Requirement already satisfied: aiohttp-cors in /home/ci/opt/venv/lib/python3.10/site-packages (from ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (0.7.0)
Requirement already satisfied: colorful in /home/ci/opt/venv/lib/python3.10/site-packages (from ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (0.5.6)
Requirement already satisfied: py-spy>=0.2.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (0.3.14)
Requirement already satisfied: opencensus in /home/ci/opt/venv/lib/python3.10/site-packages (from ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (0.11.4)
Requirement already satisfied: prometheus-client>=0.7.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (0.20.0)
Requirement already satisfied: smart-open in /home/ci/opt/venv/lib/python3.10/site-packages (from ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (7.0.4)
Requirement already satisfied: virtualenv!=20.21.1,>=20.0.24 in /home/ci/opt/venv/lib/python3.10/site-packages (from ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (20.26.3)
Requirement already satisfied: grpcio>=1.42.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (1.64.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/ci/opt/venv/lib/python3.10/site-packages (from requests->autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /home/ci/opt/venv/lib/python3.10/site-packages (from requests->autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (3.7)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from requests->autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (2.2.2)
Requirement already satisfied: certifi>=2017.4.17 in /home/ci/opt/venv/lib/python3.10/site-packages (from requests->autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (2024.7.4)
Requirement already satisfied: imageio>=2.4.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from scikit-image<0.21.0,>=0.19.1->autogluon.multimodal==1.1.1b20240716->autogluon) (2.34.2)
Requirement already satisfied: tifffile>=2019.7.26 in /home/ci/opt/venv/lib/python3.10/site-packages (from scikit-image<0.21.0,>=0.19.1->autogluon.multimodal==1.1.1b20240716->autogluon) (2024.7.2)
Requirement already satisfied: PyWavelets>=1.1.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from scikit-image<0.21.0,>=0.19.1->autogluon.multimodal==1.1.1b20240716->autogluon) (1.6.0)
Requirement already satisfied: lazy_loader>=0.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from scikit-image<0.21.0,>=0.19.1->autogluon.multimodal==1.1.1b20240716->autogluon) (0.4)
Requirement already satisfied: threadpoolctl>=2.0.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from scikit-learn<1.4.1,>=1.3.0->autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (3.5.0)
Requirement already satisfied: statsmodels>=0.13.2 in /home/ci/opt/venv/lib/python3.10/site-packages (from statsforecast<1.5,>=1.4.0->autogluon.timeseries==1.1.1b20240716->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (0.14.2)
Requirement already satisfied: absl-py>=0.4 in /home/ci/opt/venv/lib/python3.10/site-packages (from tensorboard<3,>=2.9->autogluon.multimodal==1.1.1b20240716->autogluon) (2.1.0)
Requirement already satisfied: markdown>=2.6.8 in /home/ci/opt/venv/lib/python3.10/site-packages (from tensorboard<3,>=2.9->autogluon.multimodal==1.1.1b20240716->autogluon) (3.6)
Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from tensorboard<3,>=2.9->autogluon.multimodal==1.1.1b20240716->autogluon) (0.7.2)
Requirement already satisfied: werkzeug>=1.0.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from tensorboard<3,>=2.9->autogluon.multimodal==1.1.1b20240716->autogluon) (3.0.3)
Requirement already satisfied: safetensors in /home/ci/opt/venv/lib/python3.10/site-packages (from timm<0.10.0,>=0.9.5->autogluon.multimodal==1.1.1b20240716->autogluon) (0.4.3)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /home/ci/opt/venv/lib/python3.10/site-packages (from torch<2.4,>=2.2->autogluon.multimodal==1.1.1b20240716->autogluon) (12.1.105)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /home/ci/opt/venv/lib/python3.10/site-packages (from torch<2.4,>=2.2->autogluon.multimodal==1.1.1b20240716->autogluon) (12.1.105)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /home/ci/opt/venv/lib/python3.10/site-packages (from torch<2.4,>=2.2->autogluon.multimodal==1.1.1b20240716->autogluon) (12.1.105)
Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in /home/ci/opt/venv/lib/python3.10/site-packages (from torch<2.4,>=2.2->autogluon.multimodal==1.1.1b20240716->autogluon) (8.9.2.26)
Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from torch<2.4,>=2.2->autogluon.multimodal==1.1.1b20240716->autogluon) (12.1.3.1)
Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /home/ci/opt/venv/lib/python3.10/site-packages (from torch<2.4,>=2.2->autogluon.multimodal==1.1.1b20240716->autogluon) (11.0.2.54)
Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /home/ci/opt/venv/lib/python3.10/site-packages (from torch<2.4,>=2.2->autogluon.multimodal==1.1.1b20240716->autogluon) (10.3.2.106)
Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /home/ci/opt/venv/lib/python3.10/site-packages (from torch<2.4,>=2.2->autogluon.multimodal==1.1.1b20240716->autogluon) (11.4.5.107)
Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /home/ci/opt/venv/lib/python3.10/site-packages (from torch<2.4,>=2.2->autogluon.multimodal==1.1.1b20240716->autogluon) (12.1.0.106)
Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /home/ci/opt/venv/lib/python3.10/site-packages (from torch<2.4,>=2.2->autogluon.multimodal==1.1.1b20240716->autogluon) (2.20.5)
Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /home/ci/opt/venv/lib/python3.10/site-packages (from torch<2.4,>=2.2->autogluon.multimodal==1.1.1b20240716->autogluon) (12.1.105)
Requirement already satisfied: triton==2.3.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from torch<2.4,>=2.2->autogluon.multimodal==1.1.1b20240716->autogluon) (2.3.1)
Requirement already satisfied: nvidia-nvjitlink-cu12 in /home/ci/opt/venv/lib/python3.10/site-packages (from nvidia-cusolver-cu12==11.4.5.107->torch<2.4,>=2.2->autogluon.multimodal==1.1.1b20240716->autogluon) (12.5.82)
Requirement already satisfied: tokenizers<0.19,>=0.14 in /home/ci/opt/venv/lib/python3.10/site-packages (from transformers<4.41.0,>=4.38.0->transformers[sentencepiece]<4.41.0,>=4.38.0->autogluon.multimodal==1.1.1b20240716->autogluon) (0.15.2)
Requirement already satisfied: sentencepiece!=0.1.92,>=0.1.91 in /home/ci/opt/venv/lib/python3.10/site-packages (from transformers[sentencepiece]<4.41.0,>=4.38.0->autogluon.multimodal==1.1.1b20240716->autogluon) (0.2.0)
Requirement already satisfied: contourpy>=1.0.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from matplotlib->autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (1.2.1)
Requirement already satisfied: cycler>=0.10 in /home/ci/opt/venv/lib/python3.10/site-packages (from matplotlib->autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from matplotlib->autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (4.53.1)
Requirement already satisfied: kiwisolver>=1.0.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from matplotlib->autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (1.4.5)
Requirement already satisfied: pyparsing>=2.2.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from matplotlib->autogluon.core==1.1.1b20240716->autogluon.core[all]==1.1.1b20240716->autogluon) (3.1.2)
Requirement already satisfied: multidict<7.0,>=4.5 in /home/ci/opt/venv/lib/python3.10/site-packages (from aiohttp>=3.7->ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (6.0.5)
Requirement already satisfied: yarl<2.0,>=1.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from aiohttp>=3.7->ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (1.9.4)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from aiohttp>=3.7->ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (4.0.3)
Requirement already satisfied: pyarrow-hotfix in /home/ci/opt/venv/lib/python3.10/site-packages (from datasets>=2.0.0->evaluate<0.5.0,>=0.4.0->autogluon.multimodal==1.1.1b20240716->autogluon) (0.6)
Requirement already satisfied: beautifulsoup4 in /home/ci/opt/venv/lib/python3.10/site-packages (from gdown>=4.0.0->nlpaug<1.2.0,>=1.1.10->autogluon.multimodal==1.1.1b20240716->autogluon) (4.12.3)
Requirement already satisfied: llvmlite<0.44,>=0.43.0dev0 in /home/ci/opt/venv/lib/python3.10/site-packages (from numba->mlforecast<0.10.1,>=0.10.0->autogluon.timeseries==1.1.1b20240716->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (0.43.0)
Requirement already satisfied: flatbuffers in /home/ci/opt/venv/lib/python3.10/site-packages (from onnxruntime>=1.11.0->optimum[onnxruntime]<1.19,>=1.17->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (24.3.25)
Requirement already satisfied: annotated-types>=0.4.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from pydantic<3,>=1.7->gluonts==0.15.1->autogluon.timeseries==1.1.1b20240716->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (0.7.0)
Requirement already satisfied: pydantic-core==2.20.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from pydantic<3,>=1.7->gluonts==0.15.1->autogluon.timeseries==1.1.1b20240716->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (2.20.1)
Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in /home/ci/opt/venv/lib/python3.10/site-packages (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (3.0.12)
Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (1.0.5)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (1.0.10)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /home/ci/opt/venv/lib/python3.10/site-packages (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (2.0.8)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /home/ci/opt/venv/lib/python3.10/site-packages (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (3.0.9)
Requirement already satisfied: thinc<8.3.0,>=8.2.2 in /home/ci/opt/venv/lib/python3.10/site-packages (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (8.2.5)
Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (1.1.3)
Requirement already satisfied: srsly<3.0.0,>=2.4.3 in /home/ci/opt/venv/lib/python3.10/site-packages (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (2.4.8)
Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in /home/ci/opt/venv/lib/python3.10/site-packages (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (2.0.10)
Requirement already satisfied: weasel<0.5.0,>=0.1.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (0.4.1)
Requirement already satisfied: typer<1.0.0,>=0.3.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (0.12.3)
Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (3.4.0)
Requirement already satisfied: patsy>=0.5.6 in /home/ci/opt/venv/lib/python3.10/site-packages (from statsmodels>=0.13.2->statsforecast<1.5,>=1.4.0->autogluon.timeseries==1.1.1b20240716->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (0.5.6)
Requirement already satisfied: distlib<1,>=0.3.7 in /home/ci/opt/venv/lib/python3.10/site-packages (from virtualenv!=20.21.1,>=20.0.24->ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (0.3.8)
Requirement already satisfied: platformdirs<5,>=3.9.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from virtualenv!=20.21.1,>=20.0.24->ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (4.2.2)
Requirement already satisfied: humanfriendly>=9.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from coloredlogs->optimum<1.19,>=1.17->optimum[onnxruntime]<1.19,>=1.17->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (10.0)
Requirement already satisfied: ordered-set in /home/ci/opt/venv/lib/python3.10/site-packages (from model-index->openmim<0.4.0,>=0.3.7->autogluon.multimodal==1.1.1b20240716->autogluon) (4.1.0)
Requirement already satisfied: opencensus-context>=0.1.3 in /home/ci/opt/venv/lib/python3.10/site-packages (from opencensus->ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (0.1.3)
Requirement already satisfied: google-api-core<3.0.0,>=1.0.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from opencensus->ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (2.19.1)
Requirement already satisfied: pycryptodome in /home/ci/opt/venv/lib/python3.10/site-packages (from opendatalab->openmim<0.4.0,>=0.3.7->autogluon.multimodal==1.1.1b20240716->autogluon) (3.20.0)
Requirement already satisfied: openxlab in /home/ci/opt/venv/lib/python3.10/site-packages (from opendatalab->openmim<0.4.0,>=0.3.7->autogluon.multimodal==1.1.1b20240716->autogluon) (0.0.11)
Requirement already satisfied: tenacity>=6.2.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from plotly->catboost<1.3,>=1.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (8.5.0)
Requirement already satisfied: markdown-it-py>=2.2.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from rich->openmim<0.4.0,>=0.3.7->autogluon.multimodal==1.1.1b20240716->autogluon) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from rich->openmim<0.4.0,>=0.3.7->autogluon.multimodal==1.1.1b20240716->autogluon) (2.18.0)
Requirement already satisfied: wrapt in /home/ci/opt/venv/lib/python3.10/site-packages (from smart-open->ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (1.16.0)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from sympy->optimum<1.19,>=1.17->optimum[onnxruntime]<1.19,>=1.17->autogluon.timeseries[all]==1.1.1b20240716->autogluon) (1.3.0)
Requirement already satisfied: googleapis-common-protos<2.0.dev0,>=1.56.2 in /home/ci/opt/venv/lib/python3.10/site-packages (from google-api-core<3.0.0,>=1.0.0->opencensus->ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (1.63.2)
Requirement already satisfied: proto-plus<2.0.0dev,>=1.22.3 in /home/ci/opt/venv/lib/python3.10/site-packages (from google-api-core<3.0.0,>=1.0.0->opencensus->ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (1.24.0)
Requirement already satisfied: google-auth<3.0.dev0,>=2.14.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from google-api-core<3.0.0,>=1.0.0->opencensus->ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (2.32.0)
Requirement already satisfied: language-data>=1.2 in /home/ci/opt/venv/lib/python3.10/site-packages (from langcodes<4.0.0,>=3.2.0->spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (1.2.0)
Requirement already satisfied: mdurl~=0.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from markdown-it-py>=2.2.0->rich->openmim<0.4.0,>=0.3.7->autogluon.multimodal==1.1.1b20240716->autogluon) (0.1.2)
Requirement already satisfied: blis<0.8.0,>=0.7.8 in /home/ci/opt/venv/lib/python3.10/site-packages (from thinc<8.3.0,>=8.2.2->spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (0.7.11)
Requirement already satisfied: confection<1.0.0,>=0.0.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from thinc<8.3.0,>=8.2.2->spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (0.1.5)
Requirement already satisfied: shellingham>=1.3.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from typer<1.0.0,>=0.3.0->spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (1.5.4)
Requirement already satisfied: cloudpathlib<1.0.0,>=0.7.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from weasel<0.5.0,>=0.1.0->spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (0.18.1)
Requirement already satisfied: soupsieve>1.2 in /home/ci/opt/venv/lib/python3.10/site-packages (from beautifulsoup4->gdown>=4.0.0->nlpaug<1.2.0,>=1.1.10->autogluon.multimodal==1.1.1b20240716->autogluon) (2.5)
Requirement already satisfied: PySocks!=1.5.7,>=1.5.6 in /home/ci/opt/venv/lib/python3.10/site-packages (from requests[socks]->gdown>=4.0.0->nlpaug<1.2.0,>=1.1.10->autogluon.multimodal==1.1.1b20240716->autogluon) (1.7.1)
Requirement already satisfied: cachetools<6.0,>=2.0.0 in /home/ci/opt/venv/lib/python3.10/site-packages (from google-auth<3.0.dev0,>=2.14.1->google-api-core<3.0.0,>=1.0.0->opencensus->ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (5.4.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /home/ci/opt/venv/lib/python3.10/site-packages (from google-auth<3.0.dev0,>=2.14.1->google-api-core<3.0.0,>=1.0.0->opencensus->ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (0.4.0)
Requirement already satisfied: rsa<5,>=3.1.4 in /home/ci/opt/venv/lib/python3.10/site-packages (from google-auth<3.0.dev0,>=2.14.1->google-api-core<3.0.0,>=1.0.0->opencensus->ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (4.9)
Requirement already satisfied: marisa-trie>=0.7.7 in /home/ci/opt/venv/lib/python3.10/site-packages (from language-data>=1.2->langcodes<4.0.0,>=3.2.0->spacy<4->fastai<2.8,>=2.3.1->autogluon.tabular[all]==1.1.1b20240716->autogluon) (1.2.0)
Requirement already satisfied: pyasn1<0.7.0,>=0.4.6 in /home/ci/opt/venv/lib/python3.10/site-packages (from pyasn1-modules>=0.2.1->google-auth<3.0.dev0,>=2.14.1->google-api-core<3.0.0,>=1.0.0->opencensus->ray[default,tune]<2.11,>=2.10.0; extra == "all"->autogluon.core[all]==1.1.1b20240716->autogluon) (0.6.0)
import os
import warnings
import numpy as np
warnings.filterwarnings('ignore')
np.random.seed(123)
Example Data¶
For this tutorial we use a simplified and subsampled version of the PetFinder dataset. The goal is to predict pet adoption rates based on their adoption profiles. In this simplified version, the adoption speed is grouped into two categories: 0 (slow) and 1 (fast). We begin by downloading a zip file containing the petfinder datasets and unzipping them in the current working directory.
from autogluon.core.utils.loaders import load_zip
download_dir = './ag_multimodal_tutorial'
zip_file = 'https://automl-mm-bench.s3.amazonaws.com/petfinder_for_tutorial.zip'
load_zip.unzip(zip_file, unzip_dir=download_dir)
Downloading ./ag_multimodal_tutorial/file.zip from https://automl-mm-bench.s3.amazonaws.com/petfinder_for_tutorial.zip...
100%|██████████| 18.8M/18.8M [00:00<00:00, 54.0MiB/s]
Next, we use pandas to read the dataset’s CSV files into DataFrames, noting that the column we are interested in learning to predict is “AdoptionSpeed”.
import pandas as pd
dataset_path = f'{download_dir}/petfinder_for_tutorial'
train_data = pd.read_csv(f'{dataset_path}/train.csv', index_col=0)
test_data = pd.read_csv(f'{dataset_path}/test.csv', index_col=0)
label_col = 'AdoptionSpeed'
The PetFinder dataset comes with a directory of images, and some records in the data have multiple images associated with them. AutoGluon’s multimodal dataframe format requires that image columns contain a string whose value is a path to a single image file. For this example, we will limit the image feature column to only the first image and will need to do some path manipulations to get everything setup correctly for the current directory structure.
image_col = 'Images'
train_data[image_col] = train_data[image_col].apply(lambda ele: ele.split(';')[0])
test_data[image_col] = test_data[image_col].apply(lambda ele: ele.split(';')[0])
def path_expander(path, base_folder):
path_l = path.split(';')
return ';'.join([os.path.abspath(os.path.join(base_folder, path)) for path in path_l])
train_data[image_col] = train_data[image_col].apply(lambda ele: path_expander(ele, base_folder=dataset_path))
test_data[image_col] = test_data[image_col].apply(lambda ele: path_expander(ele, base_folder=dataset_path))
Each animal’s adoption profile includes pictures, a text description, and various tabular features such as age, breed, name, color, and more. Let’s look at a picture and description for an example row of data.
example_row = train_data.iloc[0]
example_image = example_row[image_col]
from IPython.display import Image, display
pil_img = Image(filename=example_image)
display(pil_img)
example_row['Description']
"I rescued Yumi Hamasaki at a food stall far away in Kelantan. At that time i was on my way back to KL, she was suffer from stomach problem and looking very2 sick.. I send her to vet & get the treatment + vaccinated and right now she's very2 healthy.. About yumi : - love to sleep with ppl - she will keep on meowing if she's hugry - very2 active, always seeking for people to accompany her playing - well trained (poo+pee in her own potty) - easy to bathing - I only feed her with these brands : IAMS, Kittenbites, Pro-formance Reason why i need someone to adopt Yumi: I just married and need to move to a new house where no pets are allowed :( As Yumi is very2 special to me, i will only give her to ppl that i think could take care of her just like i did (especially on her foods things).."
Training¶
Now that the data is in a suitable format, we can fit MultiModalPredictor on the training data. Here we set a tight training time budget for this quick demo. More training time will lead to better prediction performance, but we can get surprisingly good performance in a short amount of time.
from autogluon.multimodal import MultiModalPredictor
predictor = MultiModalPredictor(label=label_col).fit(
train_data=train_data,
time_limit=120
)
Show code cell output
No path specified. Models will be saved in: "AutogluonModels/ag-20240716_224107"
=================== System Info ===================
AutoGluon Version: 1.1.1b20240716
Python Version: 3.10.13
Operating System: Linux
Platform Machine: x86_64
Platform Version: #1 SMP Fri May 17 18:07:48 UTC 2024
CPU Count: 8
Pytorch Version: 2.3.1+cu121
CUDA Version: 12.1
Memory Avail: 28.68 GB / 30.95 GB (92.7%)
Disk Space Avail: 189.43 GB / 255.99 GB (74.0%)
===================================================
AutoGluon infers your prediction problem is: 'binary' (because only two unique label-values observed).
2 unique label values: [0, 1]
If 'binary' is not the correct problem_type, please manually specify the problem_type parameter during Predictor init (You may specify problem_type as one of: ['binary', 'multiclass', 'regression', 'quantile'])
AutoMM starts to create your model. ✨✨✨
To track the learning progress, you can open a terminal and launch Tensorboard:
```shell
# Assume you have installed tensorboard
tensorboard --logdir /home/ci/autogluon/docs/tutorials/multimodal/multimodal_prediction/AutogluonModels/ag-20240716_224107
```
INFO: Seed set to 0
GPU Count: 1
GPU Count to be Used: 1
GPU 0 Name: Tesla T4
GPU 0 Memory: 0.42GB/15.0GB (Used/Total)
INFO: Using 16bit Automatic Mixed Precision (AMP)
INFO: GPU available: True (cuda), used: True
INFO: TPU available: False, using: 0 TPU cores
INFO: HPU available: False, using: 0 HPUs
INFO: LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
INFO:
| Name | Type | Params | Mode
------------------------------------------------------------------
0 | model | MultimodalFusionMLP | 207 M | train
1 | validation_metric | BinaryAUROC | 0 | train
2 | loss_func | CrossEntropyLoss | 0 | train
------------------------------------------------------------------
207 M Trainable params
0 Non-trainable params
207 M Total params
828.307 Total estimated model params size (MB)
INFO: Epoch 0, global step 1: 'val_roc_auc' reached 0.56194 (best 0.56194), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/multimodal_prediction/AutogluonModels/ag-20240716_224107/epoch=0-step=1.ckpt' as top 3
INFO: Epoch 0, global step 4: 'val_roc_auc' reached 0.75917 (best 0.75917), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/multimodal_prediction/AutogluonModels/ag-20240716_224107/epoch=0-step=4.ckpt' as top 3
INFO: Epoch 1, global step 5: 'val_roc_auc' reached 0.78417 (best 0.78417), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/multimodal_prediction/AutogluonModels/ag-20240716_224107/epoch=1-step=5.ckpt' as top 3
INFO: Epoch 1, global step 8: 'val_roc_auc' reached 0.77333 (best 0.78417), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/multimodal_prediction/AutogluonModels/ag-20240716_224107/epoch=1-step=8.ckpt' as top 3
INFO: Epoch 2, global step 9: 'val_roc_auc' reached 0.76333 (best 0.78417), saving model to '/home/ci/autogluon/docs/tutorials/multimodal/multimodal_prediction/AutogluonModels/ag-20240716_224107/epoch=2-step=9.ckpt' as top 3
INFO: Time limit reached. Elapsed time is 0:02:01. Signaling Trainer to stop.
Start to fuse 3 checkpoints via the greedy soup algorithm.
AutoMM has created your model. 🎉🎉🎉
To load the model, use the code below:
```python
from autogluon.multimodal import MultiModalPredictor
predictor = MultiModalPredictor.load("/home/ci/autogluon/docs/tutorials/multimodal/multimodal_prediction/AutogluonModels/ag-20240716_224107")
```
If you are not satisfied with the model, try to increase the training time,
adjust the hyperparameters (https://auto.gluon.ai/stable/tutorials/multimodal/advanced_topics/customization.html),
or post issues on GitHub (https://github.com/autogluon/autogluon/issues).
Under the hood MultiModalPredictor automatically infers the problem type (classification or regression), detects feature modalities, selects models from the multimodal model pools, and trains the selected models. If multiple backbones are used, MultiModalPredictor appends a late-fusion model (MLP or transformer) on top of them.
Prediction¶
After fitting the model, we want to use it to predict the labels in the witheld test dataset.
predictions = predictor.predict(test_data.drop(columns=label_col))
predictions[:5]
8 1
70 1
82 1
28 0
63 1
Name: AdoptionSpeed, dtype: int64
For classification tasks, we can just as easily get the prediction probabilities for each output class.
probs = predictor.predict_proba(test_data.drop(columns=label_col))
probs[:5]
| 0 | 1 | |
|---|---|---|
| 8 | 0.482795 | 0.517205 |
| 70 | 0.261425 | 0.738575 |
| 82 | 0.017797 | 0.982203 |
| 28 | 0.838415 | 0.161585 |
| 63 | 0.141667 | 0.858333 |
Evaluation¶
Finally, we can evaluate the predictor on the witheld test dataset on other performance metrics, in this case roc_auc.
scores = predictor.evaluate(test_data, metrics=["roc_auc"])
scores
{'roc_auc': 0.8988}
Conclusion¶
In this quickstart tutorial we saw the basic fit and predict functionality of AutoGluon’s MultiModalPredictor, but we just scratched the surface on its functionality. Check out the in-depth tutorials to learn about other features of AutoGluon’s MultiModalPredictor like embedding extraction, distillation, model fine-tuning, text or image prediction, and semantic matching.