After some investigations, I found out that the cause of the issue is from Azure AD not Grafana.
As can be seen in this warning message, the role assignments won’t be cascaded to nested groups. Only direct users will get access. I tested access on direct and it works as expected.