The vision ability is fundamentally important for a mobile robot. Many aspects have been investigated during the past few years, but there still remain questions to be answered. This work mainly focuses on the task of road detection, which is considered as the first step for a robot to become moveable. The proposed method combines the depth clue with traditional RGB information and is divided into three steps: depth recovery and superpixel generation, weakly supervised SVM classification and context-aware label transfer. The main contributions made in this paper are (1) Design a novel superpixel based context-aware descriptor by utilizing depth map. (2) Conduct label transfer in an efficient nearest neighbor search and a temporal MRF model. (3) Update the learned model adaptively with the changing scene. Experimental results on a publicly available dataset justify the effectiveness of the proposed method.