Deterioration of road and pavement surface conditions is an issue which directly affects the majority of the world today. The complex structure and textural similarities of surface cracks, as well as noise and image illumination variation makes automated detection a challenging task. In this paper, we propose a deep fully convolutional neural network to perform pixel-wise classification of surface cracks on road and pavement images. The network consists of an encoder layer which reduces the input image to a bank of lower level feature maps. This is followed by a corresponding decoder layer which maps the encoded features back to the resolution of the input data using the indices of the encoder pooling layers to perform efficient up-sampling. The network is finished with a classification layer to label individual pixels. Training time is minimal due to the small amount of training/validation data (80 training images and 20 validation images). This is important due to the lack of applicable public data available. Despite this lack of data, we are able to perform image segmentation (pixel-level classification) on a number of publicly available road crack datasets. The network was tested extensively and the results obtained indicate performance in direct competition with that of the current state-of-the-art methods.