In this talk we present a generic higher order graph-based computational model for automatically inferring and learning data interpretations in divers settings. In particular we discuss the interest and theoretical strengths of such representations, propose efficient inference algorithms for low and higher-order rank models, as well as efficient learning methods towards predictive representations that could be learned efficiently from few examples. The interest of such computational solutions is demonstrated in various challenging domains such as computer vision (graph-matching, image-parsing), computer-aided image-based diagnosis (tumor modeling from partial/incomplete annotations, multi-modal fusion, probabilistic digital anatomy) and computational biology (protein prediction).