This repository is based on the pytorch implementation of Masked Autoencoders Are Scalable Vision Learners(He et al., 2021).

The system trains MAE model with style classification, genre classification and triplet learning task simultaneously.

For each data point $x^{(a)}$ in a minibatch and the $k$-nearest neighbors $x_1^{(a)}, x_2^{(a)}, \cdots, x_k^{(a)}$, the knn triplet loss is defined as:

where the relevance measure $r_i^{(a)}$ is defined as:

$$r_i^{(a)}= \begin{cases} 1 & \text{ if } s^{(a)} = s_i^{(a)} \text{ and } g^{(a)} = g_i^{(a)} \\ 0 & \text{ if } s^{(a)} \neq s_i^{(a)} \text{ and } g^{(a)} \neq g_i^{(a)} \\ 0.5 & \text{ otherwise} \end{cases} $$

• WikiArt
  • Notice) We used only images that had both style and genre labels.
• MulititasPainting100k