A multi-class, multi-label deep learning model for the monitoring of road infrastructures is presented in this paper. The employed detection methodology can identify animals, debris, road defects, fire, fog, flooded areas and humans. All these categories are strongly related to the efficient movement of vehicles through a transportation network. Possible detections indicate roadway disruptions of various types. Therefore, they should be detected as fast as possible. Experimental results indicate that the proposed scheme presents high detection results and, thus, can be used in any motorway monitoring process.