To coordinate cellular physiology, eukaryotic cells rely on the rapid exchange of molecules at specialized organelle-organelle contact sites. Endoplasmic reticulum-mitochondrial contact sites (ERMCSs) are particularly vital communication hubs, playing key roles in the exchange of signalling molecules, lipids and metabolites. ERMCSs are maintained by interactions between complementary tethering molecules on the surface of each organelle. However, due to the extreme sensitivity of these membrane interfaces to experimental perturbation, a clear understanding of their nanoscale organization and regulation is still lacking. Here we combine three-dimensional electron microscopy with high-speed molecular tracking of a model organelle tether, Vesicle-associated membrane protein (VAMP)-associated protein B (VAPB), to map the structure and diffusion landscape of ERMCSs. We uncovered dynamic subdomains within VAPB contact sites that correlate with ER membrane curvature and undergo rapid remodelling. We show that VAPB molecules enter and leave ERMCSs within seconds, despite the contact site itself remaining stable over much longer time scales. This metastability allows ERMCSs to remodel with changes in the physiological environment to accommodate metabolic needs of the cell. An amyotrophic lateral sclerosis-associated mutation in VAPB perturbs these subdomains, likely impairing their remodelling capacity and resulting in impaired interorganelle communication. These results establish high-speed single-molecule imaging as a new tool for mapping the structure of contact site interfaces and reveal that the diffusion landscape of VAPB at contact sites is a crucial component of ERMCS homeostasis.