We studied magnetic states and phase transitions in the van der Waals antiferromagnet $\mathrm{V}{\mathrm{Br}}_3$ experimentally by specific heat and magnetization measurements of single crystals in high magnetic fields and theoretically by the density functional theory calculations focused on exchange interactions. The magnetization behavior mimics Ising antiferromagnets with magnetic moments pointing out-of-plane due to strong uniaxial magnetocrystalline anisotropy. The out-of-plane magnetic field induces a spin-flip metamagnetic transition of first-order type at low temperatures, while at higher temperatures, the transition becomes continuous. The first-order and continuous transition segments in the field-temperature phase diagram meet at a tricritical point. The magnetization response to the in-plane field manifests a continuous spin canting which is completed at the anisotropy field ${\mu }_0{H}_{\mathrm{MA}}\approx 27\mathrm{T}$. At higher fields, the two magnetization curves above saturate at the same value of magnetic moment ${\mu }_{\mathrm{sat}}\approx 1.2{\mu }_{\mathrm{B}}/\mathrm{f}.\mathrm{u}.$, which is much smaller than the spin-only $\left(S=1\right)$ moment of the ${\mathrm{V}}^{3+}$ ion. The reduced moment can be explained by the existence of an unquenched orbital magnetic moment antiparallel to the spin. The orbital moment is a key ingredient of a mechanism responsible for the observed large anisotropy. The exact energy evaluation of possible magnetic structures shows that the intralayer zigzag antiferromagnetic (AFM) order is preferred, which renders the AFM ground state significantly more stable against the spin-flip transition than the other options. The calculations also predict that a minimal distortion of the Br ion sublattice causes a radical change of the orbital occupation in the ground state, connected with the formation of the orbital moment and the stability of magnetic order.

• Received 3 February 2023
• Accepted 29 August 2023

DOI:https://doi.org/10.1103/PhysRevB.108.104416