Abstract. While Earth system models (ESMs) are process-based and can be run at high resolutions, they are only limited by computational costs. Reduced complexity models, also called simple climate models or compact models, provide a much cheaper alternative, although at a loss of spatial information. Their structure relies on the sciences of the Earth system, but with a calibration against the most complex models. Therefore it remains important to evaluate and validate reduced complexity models. Here, we diagnose such a model the newest version of OSCAR (v3.1) using observations and results from ESMs from the current Coupled Model Intercomparison Project 6. A total of 99 experiments are selected for simulation with OSCAR v3.1 in a probabilistic framework, reaching a total of 567,700,000 simulated years. A first highlight of this exercise that the ocean carbon cycle of the model may diverge under some parametrizations and for high-warming scenarios. The diverging runs caused by this unstability were discarded in the post-processing. Then, each physical parametrization is weighted based on its performance against a set of observations, providing us with constrained results. Overall, OSCAR v3.1 shows good agreement with observations, ESMs and emerging properties. It qualitively reproduces the responses of complex ESMs, for all aspects of the Earth system. We observe some quantitative differences with these models, most of them being due to the observational constraints. Some specific features of OSCAR also contribute to these differences, such as its fully interactive atmospheric chemistry and endogenous calculations of biomass burning, wetlands CH4 and permafrost CH4 and CO2 emissions. The main points of improvements are a low sensitivity of the land carbon cycle to climate change, an unstability of the ocean carbon cycle, the seemingly too simple climate module, and the too strong climate feedback involving short-lived species. Beyond providing a key diagnosis of the OSCAR model in the context of the reduced-complexity models intercomparison project (RCMIP), this work is also meant to help with the upcoming calibration of OSCAR on CMIP6 results, and to provide a large group of CMIP6 simulations run consistently within a probabilistic framework.