Haze superimposes a veil over remote sensing images, which severely limits the extraction of valuable military information. To this end, we present a novel trinity model to restore realistic surface information by integrating the merits of both prior- and deep learning-based strategies. Concretely, the critical insight of our Trinity-Net is to investigate how to incorporate prior information into convolutional neural networks (CNNs) and Swin Transformer for reasonable estimation of haze parameters. Then, haze-free images are obtained by reconstructing the remote sensing image formation model. Although Swin Transformer has shown tremendous potential in the dehazing task, which typically results in ambiguous details, we devise a gradient guidance module that naturally inherits structure priors of gradient maps, guiding the deep model to generate visually pleasing details. In light of the generality of image formation parameters, we successfully promote Trinity-Net to natural image dehazing and underwater image enhancement tasks. Notably, the acquisition of large-scale remote sensing hazy images and natural hazy images in military scenes is not feasible in practice. To bridge this gap, we construct a remote sensing image dehazing benchmark (RSID) and a natural image dehazing benchmark (NID), including 1000 real-world hazy images with corresponding ground-truth images. To our knowledge, this is the first exploration to develop dehazing benchmarks in the military field, alleviating the dilemma of data scarcity. Extensive experiments on three vision tasks illustrate the superiority of our Trinity-Net against multiple state-of-the-art methods. The datasets and code are available at https://github.com/chi-kaichen/Trinity-Net .