activation function
Recently Published Documents

Cyberspace has been recognized as a conducive environment for use of various hostile, direct, and indirect behavioural tactics to target individuals or groups. Denigration is one of the most frequently used cyberbullying ploys to actively damage, humiliate, and disparage the online reputation of target by sending, posting, or publishing cruel rumours, gossip, and untrue statements. Previous pertinent studies report detecting profane, vulgar, and offensive words primarily in the English language. This research puts forward a model to detect online denigration bullying in low-resource Hindi language using attention residual networks. The proposed model Hindi Denigrate Comment–Attention Residual Network (HDC-ARN) intends to uncover defamatory posts (denigrate comments) written in Hindi language which stake and vilify a person or an entity in public. Data with 942 denigrate comments and 1499 non-denigrate comments is scraped using certain hashtags from two recent trending events in India: Tablighi Jamaat spiked Covid-19 (April 2020, Event 1) and Sushant Singh Rajput Death (June 2020: Event 2). Only text-based features, that is, the actual content of the post, are considered. The pre-trained word embedding for Hindi language from fastText is used. The model has three ResNet blocks with an attention layer that generates a post vector for a single input, which is passed through a sigmoid activation function to get the final output as either denigrate (positive class) or non-denigrate (negative class). An F-1 score of 0.642 is achieved on the dataset.

To verify safety and robustness of neural networks, researchers have successfully applied abstract interpretation , primarily using the interval abstract domain. In this paper, we study the theoretical power and limits of the interval domain for neural-network verification. First, we introduce the interval universal approximation (IUA) theorem. IUA shows that neural networks not only can approximate any continuous function f (universal approximation) as we have known for decades, but we can find a neural network, using any well-behaved activation function, whose interval bounds are an arbitrarily close approximation of the set semantics of f (the result of applying f to a set of inputs). We call this notion of approximation interval approximation . Our theorem generalizes the recent result of Baader et al. from ReLUs to a rich class of activation functions that we call squashable functions . Additionally, the IUA theorem implies that we can always construct provably robust neural networks under ℓ ∞ -norm using almost any practical activation function. Second, we study the computational complexity of constructing neural networks that are amenable to precise interval analysis. This is a crucial question, as our constructive proof of IUA is exponential in the size of the approximation domain. We boil this question down to the problem of approximating the range of a neural network with squashable activation functions. We show that the range approximation problem (RA) is a Δ 2 -intermediate problem, which is strictly harder than NP -complete problems, assuming coNP ⊄ NP . As a result, IUA is an inherently hard problem : No matter what abstract domain or computational tools we consider to achieve interval approximation, there is no efficient construction of such a universal approximator. This implies that it is hard to construct a provably robust network, even if we have a robust network to start with.

Image classification is critical and significant research problems in computer vision applications such as facial expression classification, satellite image classification, and plant classification based on images. Here in the paper, the image classification model is applied for identifying the display of daunting pictures on the internet. The proposed model uses Convolution neural network to identify these images and filter them through different blocks of the network, so that it can be classified accurately. The model will work as an extension to the web browser and will work on all websites when activated. The extension will be blurring the images and deactivating the links on web pages. This means that it will scan the entire web page and find all the daunting images present on that page. Then we will blur those images before they are loaded and the children could see them. Keywords— Activation Function, CNN, Images Classification , Optimizers, VGG-19

Formal verification of neural networks is critical for their safe adoption in real-world applications. However, designing a precise and scalable verifier which can handle different activation functions, realistic network architectures and relevant specifications remains an open and difficult challenge. In this paper, we take a major step forward in addressing this challenge and present a new verification framework, called PRIMA. PRIMA is both (i) general: it handles any non-linear activation function, and (ii) precise: it computes precise convex abstractions involving multiple neurons via novel convex hull approximation algorithms that leverage concepts from computational geometry. The algorithms have polynomial complexity, yield fewer constraints, and minimize precision loss. We evaluate the effectiveness of PRIMA on a variety of challenging tasks from prior work. Our results show that PRIMA is significantly more precise than the state-of-the-art, verifying robustness to input perturbations for up to 20%, 30%, and 34% more images than existing work on ReLU-, Sigmoid-, and Tanh-based networks, respectively. Further, PRIMA enables, for the first time, the precise verification of a realistic neural network for autonomous driving within a few minutes.

With the improvement of industrial requirements for the quality of cold rolled strips, flatness has become one of the most important indicators for measuring the quality of cold rolled strips. In this paper, the strip production data of a 1250 mm tandem cold mill in a steel plant is modeled by an improved deep neural network (the improved DNN) to improve the accuracy of strip shape prediction. Firstly, the type of activation function is analyzed, and the monotonicity of the activation function is deemed independent of the convexity of the loss function in the deep network. Regardless of whether the activation function is monotonic, the loss function is not strictly convex. Secondly, the non-convex optimization of the loss functionextended from the deep linear network to the deep nonlinear network, is discussed, and the critical point of the deep nonlinear network is identified as the global minimum point. Finally, an improved Swish activation function based on batch normalization is proposed, and its performance is evaluated on the MNIST dataset. The experimental results show that the loss of an improved Swish function is lower than that of other activation functions. The prediction accuracy of a deep neural network (DNN) with an improved Swish function is 0.38% more than that of a deep neural network (DNN) with a regular Swish function. For the DNN with the improved Swish function, the mean square error of the prediction for the flatness of cold rolled strip is reduced to 65% of the regular DNN. The accuracy of the improved DNN is up to and higher than the industrial requirements. The shape prediction of the improved DNN will assist and guide the industrial production process, reducing the scrap yield and industrial cost.

Mangroves, as unique coastal wetlands with numerous benefits, are endangered mainly due to the coupled effects of anthropogenic activities and climate change. Therefore, acquiring reliable and up-to-date information about these ecosystems is vital for their conservation and sustainable blue carbon development. In this regard, the joint use of remote sensing data and machine learning algorithms can assist in producing accurate mangrove ecosystem maps. This study investigated the potential of artificial neural networks (ANNs) with different topologies and specifications for mangrove classification in Iran. To this end, multi-temporal synthetic aperture radar (SAR) and multi-spectral remote sensing data from Sentinel-1 and Sentinel-2 were processed in the Google Earth Engine (GEE) cloud computing platform. Afterward, the ANN topologies and specifications considering the number of layers and neurons, learning algorithm, type of activation function, and learning rate were examined for mangrove ecosystem mapping. The results indicated that an ANN model with four hidden layers, 36 neurons in each layer, adaptive moment estimation (Adam) learning algorithm, rectified linear unit (Relu) activation function, and the learning rate of 0.001 produced the most accurate mangrove ecosystem map (F-score = 0.97). Further analysis revealed that although ANN models were subjected to accuracy decline when a limited number of training samples were used, they still resulted in satisfactory results. Additionally, it was observed that ANN models had a high resistance when training samples included wrong labels, and only the ANN model with the Adam learning algorithm produced an accurate mangrove ecosystem map when no data standardization was performed. Moreover, further investigations showed the higher potential of multi-temporal and multi-source remote sensing data compared to single-source and mono-temporal (e.g., single season) for accurate mangrove ecosystem mapping. Overall, the high potential of the proposed method, along with utilizing open-access satellite images and big-geo data processing platforms (i.e., GEE, Google Colab, and scikit-learn), made the proposed approach efficient and applicable over other study areas for all interested users.

AbstractArtificial neural networks (ANNs) providing sophisticated, power-efficient classification are finding their way into thin-film electronics. Thin-film technologies require robust, layout-efficient devices with facile manufacturability. Here, we show how the multimodal transistor’s (MMT’s) transfer characteristic, with linear dependence in saturation, replicates the rectified linear unit (ReLU) activation function of convolutional ANNs (CNNs). Using MATLAB, we evaluate CNN performance using systematically distorted ReLU functions, then substitute measured and simulated MMT transfer characteristics as proxies for ReLU. High classification accuracy is maintained, despite large variations in geometrical and electrical parameters, as CNNs use the same activation functions for training and classification.

Precipitation intensity estimation is a critical issue in the analysis of weather conditions. Most existing approaches focus on building complex models to extract rain streaks. However, an efficient approach to estimate the precipitation intensity from surveillance cameras is still challenging. This study proposes a convolutional neural network known as the signal filtering convolutional neural network (SF-CNN) to handle precipitation intensity using surveillance-based images. The SF-CNN has two main blocks, the signal filtering block (SF block) and the gradually decreasing dimension block (GDD block), to extract features for the precipitation intensity estimation. The SF block with the filtering operation is constructed in different parts of the SF-CNN to remove the noise from the features containing rain streak information. The GDD block continuously takes the pair of the convolutional operation with the activation function to reduce the dimension of features. Our main contributions are (1) an SF block considering the signal filtering process and effectively removing the useless signals and (2) a procedure of gradually decreasing the dimension of the feature able to learn and reserve the information of features. Experiments on the self-collected dataset, consisting of 9394 raining images with six precipitation intensity levels, demonstrate the proposed approach’s effectiveness against the popular convolutional neural networks. To the best of our knowledge, the self-collected dataset is the largest dataset for monitoring infrared images of precipitation intensity.

In this paper an optimized feedforward neural network model is proposed for detection of IoT based DDoS attacks by network traffic analysis aimed towards a specific target which could be constantly monitored by a tap. The proposed model is applicable for DoS and DDoS attacks which consist of TCP, UDP and HTTP flood and also against keylogging, data exfiltration, OS fingerprint and service scan activities. It simply differentiates such kind of network traffic from normal network flows. The neural network uses Adam optimization as a solver and the hyperbolic tangent activation function in all neurons from a single hidden layer. The number of hidden neurons could be varied, depending on targeted accuracy and processing speed. Testing over the Bot IoT dataset reveals that developed models are applicable using 8 or 10 features and achieved discrimination error of 4.91.10-3%.

AbstractBreast tumors are from the common infections among women around the world. Classifying the various types of breast tumors contribute to treating breast tumors more efficiently. However, this classification task is often hindered by dense tissue patterns captured in mammograms. The present study has been proposed a dense tissue pattern characterization framework using deep neural network. A total of 322 mammograms belonging to the mini-MIAS dataset and 4880 mammograms from DDSM dataset have been taken, and an ROI of fixed size 224 × 224 pixels from each mammogram has been extracted. In this work, tedious experimentation has been executed using different combinations of training and testing sets using different activation function with AlexNet, ResNet-18 model. Data augmentation has been used to create a similar type of virtual image for proper training of the DL model. After that, the testing set is applied on the trained model to validate the proposed model. During experiments, four different activation functions ‘sigmoid’, ‘tanh’, ‘ReLu’, and ‘leakyReLu’ are used, and the outcome for each function has been reported. It has been found that activation function ‘ReLu’ perform always outstanding with respect to others. For each experiment, classification accuracy and kappa coefficient have been computed. The obtained accuracy and kappa value for MIAS dataset using ResNet-18 model is 91.3% and 0.803, respectively. For DDSM dataset, the accuracy of 92.3% and kappa coefficient value of 0.846 are achieved. After the combination of both dataset images, the achieved accuracy is 91.9%, and kappa coefficient value is 0.839 using ResNet-18 model. Finally, it has been concluded that the ResNet-18 model and ReLu activation function yield outstanding performance for the task.