Molecular representation learning is essential to deep learning for chemistry, where the molecule... more Molecular representation learning is essential to deep learning for chemistry, where the molecules are embedded into continuous real-valued vectors as better representations in the large chemical space. Traditional molecular representation learning requires high-quality labels for molecules. However, the precise physicochemical or pharmacological properties of the molecules are expensive to measure and collect. Therefore, self-supervised training of deep learning models on large-scale cheap available chemical data is becoming an increasingly popular choice in research and practice. Masked auto-encoder is one of the most common self-supervised pretext tasks used in molecular representation learning. However, previous masked auto-encoder based methods focus on learning the existence of compounds, which may not be sufficient for chemical understanding. In this paper, we present the Masked Molecule Modeling (MMM), an emerging self-supervised paradigm of molecular representation learning...
Automated data augmentation (ADA) techniques have played an important role in boosting the perfor... more Automated data augmentation (ADA) techniques have played an important role in boosting the performance of deep models. Such techniques mostly aim to optimize a parameterized distribution over a discrete augmentation space. Thus, are restricted by the discretization of the search space which normally is handcrafted. To overcome the limitations, we take the first step to constructing a continuous mapping from R to image transformations (an augmentation space). Using this mapping, we take a novel approach where 1) we pose the ADA as a continuous optimization problem over the parameters of the augmentation distribution; and 2) use Stochastic Gradient Langevin Dynamics to learn and sample augmentations. This allows us to potentially explore the space of infinitely many possible augmentations, which otherwise was not possible due to the discretization of the space. This view of ADA is radically different from the standard discretization based view of ADA , and it opens avenues for utilizi...
In recent years, the analysis of EMI (Electromagnetic interference) signals has become a hot rese... more In recent years, the analysis of EMI (Electromagnetic interference) signals has become a hot research topic in the signal processing field. Particularly, EMI signal classification has attracted more and more attention. Conventional signal classification methods usually select features by experience or implicitly by the shallow artificial neural network, and always results in bad performance on high-dimensional and nonlinear EMI signals. This paper proposed a novel classification method based on wavelet transform and deep belief network, while wavelet transform method is used to reduce the dimension of high-dimensional signals and deep belief network can extract nonlinear features from EMI signals. Then we apply BP neural network as the classifier. Results on the benchmark dataset have shown the superiority of the proposed method compared with the state-of-the-art approaches.
The recent progress on automatically searching augmentation policies has boosted the performance ... more The recent progress on automatically searching augmentation policies has boosted the performance substantially for various tasks. A key component of automatic augmentation search is the evaluation process for a particular augmentation policy, which is utilized to return reward and usually runs thousands of times. A plain evaluation process, which includes full model training and validation, would be time-consuming. To achieve efficiency, many choose to sacrifice evaluation reliability for speed. In this paper, we dive into the dynamics of augmented training of the model. This inspires us to design a powerful and efficient proxy task based on the Augmentation-Wise Weight Sharing (AWS) to form a fast yet accurate evaluation process in an elegant way. Comprehensive analysis verifies the superiority of this approach in terms of effectiveness and efficiency. The augmentation policies found by our method achieve superior accuracies compared with existing auto-augmentation search methods. ...
Molecular representation learning is essential to deep learning for chemistry, where the molecule... more Molecular representation learning is essential to deep learning for chemistry, where the molecules are embedded into continuous real-valued vectors as better representations in the large chemical space. Traditional molecular representation learning requires high-quality labels for molecules. However, the precise physicochemical or pharmacological properties of the molecules are expensive to measure and collect. Therefore, self-supervised training of deep learning models on large-scale cheap available chemical data is becoming an increasingly popular choice in research and practice. Masked auto-encoder is one of the most common self-supervised pretext tasks used in molecular representation learning. However, previous masked auto-encoder based methods focus on learning the existence of compounds, which may not be sufficient for chemical understanding. In this paper, we present the Masked Molecule Modeling (MMM), an emerging self-supervised paradigm of molecular representation learning...
Automated data augmentation (ADA) techniques have played an important role in boosting the perfor... more Automated data augmentation (ADA) techniques have played an important role in boosting the performance of deep models. Such techniques mostly aim to optimize a parameterized distribution over a discrete augmentation space. Thus, are restricted by the discretization of the search space which normally is handcrafted. To overcome the limitations, we take the first step to constructing a continuous mapping from R to image transformations (an augmentation space). Using this mapping, we take a novel approach where 1) we pose the ADA as a continuous optimization problem over the parameters of the augmentation distribution; and 2) use Stochastic Gradient Langevin Dynamics to learn and sample augmentations. This allows us to potentially explore the space of infinitely many possible augmentations, which otherwise was not possible due to the discretization of the space. This view of ADA is radically different from the standard discretization based view of ADA , and it opens avenues for utilizi...
In recent years, the analysis of EMI (Electromagnetic interference) signals has become a hot rese... more In recent years, the analysis of EMI (Electromagnetic interference) signals has become a hot research topic in the signal processing field. Particularly, EMI signal classification has attracted more and more attention. Conventional signal classification methods usually select features by experience or implicitly by the shallow artificial neural network, and always results in bad performance on high-dimensional and nonlinear EMI signals. This paper proposed a novel classification method based on wavelet transform and deep belief network, while wavelet transform method is used to reduce the dimension of high-dimensional signals and deep belief network can extract nonlinear features from EMI signals. Then we apply BP neural network as the classifier. Results on the benchmark dataset have shown the superiority of the proposed method compared with the state-of-the-art approaches.
The recent progress on automatically searching augmentation policies has boosted the performance ... more The recent progress on automatically searching augmentation policies has boosted the performance substantially for various tasks. A key component of automatic augmentation search is the evaluation process for a particular augmentation policy, which is utilized to return reward and usually runs thousands of times. A plain evaluation process, which includes full model training and validation, would be time-consuming. To achieve efficiency, many choose to sacrifice evaluation reliability for speed. In this paper, we dive into the dynamics of augmented training of the model. This inspires us to design a powerful and efficient proxy task based on the Augmentation-Wise Weight Sharing (AWS) to form a fast yet accurate evaluation process in an elegant way. Comprehensive analysis verifies the superiority of this approach in terms of effectiveness and efficiency. The augmentation policies found by our method achieve superior accuracies compared with existing auto-augmentation search methods. ...
Uploads