The prominent methods for monitoring structures to date rely on analyzing data measured from contact sensors that are physically attached to a structure. However, these approaches have the high possibility of false alarms due to noises, sensor malfunctions, and complex environmental effects. Under those circumstances, engineers have to conduct on-site investigations to confirm that damage has occurred. To address this challenge, this paper proposes a new vision-based approach for detecting concrete cracks using a convolutional neural network (CNN). Images are firstly taken under uncontrolled situations to collect widely varying crack features. Second, the raw images are divided into 40K images to build training and validation sets. Lastly, the prepared datasets are fed into a deep CNN architecture with eight layers including convolution, pooling, ReLU, and softmax. The trained classifier consequently records 98% of accuracies in both training and validation.