Convolutional neural networks (CNNs) have recently shown remarkable performance in remote sensing scene image classification. However, long-range dependencies (e.g. non-local similarities) within the scene are often ignored by CNNs. To address this issue, in this paper we develop a new high-order self-attention network (HoSA) for remote sensing scene classification. Specifically, we embed two novel modules, i.e., a self-attention module and a high order pooling module, into off-the-shelf CNN models and then fine-tune the whole network. The advantages of our newly proposed HoSA network are twofold. Firstly, with the self-attention module, the HoSA can capture long-range dependencies within the scenes for high-level semantic feature extraction. Secondly, by means of its high-order pooling mechanism, our newly developed HoSA can further explore high-order information contained in the features. Our experiments with a widely used remote sensing scene data set demonstrate that the proposed HoSA network exhibits better classification performance than the baseline and several well-known methods.