When two-dimensional crystals are brought into close proximity, their interaction results in reconstruction of electronic spectrum and crystal structure. Such reconstruction strongly depends on the twist angle between the crystals, which has received growing attention due to interesting electronic and optical properties that arise in graphene and transitional metal dichalcogenides. Here we study two insulating crystals of hexagonal boron nitride stacked at small twist angle. Using electrostatic force microscopy, we observe ferroelectric-like domains arranged in triangular superlattices with a large surface potential. The observation is attributed to interfacial elastic deformations that result in out-of-plane dipoles formed by pairs of boron and nitrogen atoms belonging to opposite interfacial surfaces. This creates a bilayer-thick ferroelectric with oppositely polarized (BN and NB) dipoles in neighbouring domains, in agreement with our modeling. These findings open up possibilities for designing van der Waals heterostructures and offer an alternative probe to study moiré-superlattice electrostatic potentials.