The oceanic magnesium budget is important to our understanding of Earth’s carbon cycle, because similar processes control both (e.g., weathering, volcanism, and carbonate precipitation). However, dolomite sedimentation and low-temperature hydrothermal circulation remain enigmatic oceanic Mg sinks. In recent years, magnesium isotopes (δ26Mg) have provided new constraints on the Mg cycle, but the lack of data for the low-temperature hydrothermal isotope fractionation has hindered this approach. Here we present new δ26Mg data for low-temperature hydrothermal fluids, demonstrating preferential 26Mg incorporation into the oceanic crust, on average by εsolid-fluid ≈ 1.6‰. These new data, along with the constant seawater δ26Mg over the past ~20 Myr, require a significant dolomitic sink (estimated to be 1.5–2.9 Tmol yr−1; 40–60% of the oceanic Mg outputs). This estimate argues strongly against the conventional view that dolomite formation has been negligible in the Neogene and points to the existence of significant hidden dolomite formation.