Jan 25, 2024 · In this work, we re-examine inter-patch dependencies in the decoding mechanism of masked autoencoders (MAE).

We introduce Cross-Attention Masked Autoencoders (CrossMAE), which use only cross-attention for decoding in MAE. We show that CrossMAE greatly enhances ...

Nov 5, 2024 · In this work, we examine the impact of inter-patch dependencies in the decoder of masked autoencoders (MAE) on representation learning.

Abstract. Report issue for preceding element. In this work, we re-examine inter-patch dependencies in the decoding mechanism of masked autoencoders (MAE).

This is a PyTorch implementation of the CrossMAE paper Rethinking Patch Dependence for Masked Autoencoders. The code is based on the original MAE repo.

Abstract. In this work, we re-examine inter-patch dependencies in the decoding mechanism of masked autoencoders (MAE). We decompose this decoding mechanism ...

In this work, we re-examine inter-patch dependencies in the decoding mechanism of masked autoencoders (MAE). We decompose this decoding mechanism for masked ...

Jan 25, 2024 · A novel pretraining framework, Cross-Attention Masked Autoencoders (CrossMAE), which leverages only cross-attention between masked and visible tokens.

Jul 16, 2024 · This repo has the models for CrossMAE: Rethinking Patch Dependence for Masked Autoencoders. Please take a look at the GitHub repo to see instructions.

Jan 25, 2024 · The paper introduces CrossMAE, a new framework for masked autoencoders in vision tasks that focuses on decoding efficiency and ...