After many years of decline, hunger in Africa is growing again. This represents a global societal issue that all disciplines concerned with data analysis are facing. The rapid and accurate identification of food insecurity situations is a complex challenge. Although a number of food security alert and monitoring systems exist in food insecure countries, the data and methodologies they are based on do not allow for comprehending food security in all its complexity. In this study, we focus on two key food security indicators: the food consumption score (F C S) and the household dietary diversity score (H D D S). Based on the observation that producing such indicators is expensive in terms of time and resources, we propose the F S P H D (Food Security Prediction based on Heterogeneous Data) framework, based on state-of-the-art machine and deep learning models, to enable the estimation of F C S and H D D S starting from publicly available heterogeneous data. We take into account the indicators estimated using data from the Permanent Agricultural Survey conducted by the Burkina Faso government from 2009 to 2018 as reference data. We produce our estimations starting from heterogeneous data that include rasters (eg, population density, land use, soil quality), GPS points (hospitals, schools, violent events), line vectors (waterways), quantitative variables (maize prices, World Bank variables, meteorological data) and time series (Smoothed Brightness Temperature—SMT, rainfall estimates, maize prices). The experimental results show a promising performance of our framework, which outperforms competing methods, thus paving the way for the development of advanced food security prediction systems based on state-of-the-art data science technologies.