The marine lithospheric subsurface is one of the largest biospheres on Earth; however, little is known about the identity and ecological function of microorganisms found in low abundance in this habitat, though these organisms impact global-scale biogeochemical cycling. Here, we describe the diversity and metabolic potential of sediment and endolithic (within rock) microbial communities found in ultrasmall amounts (101 to 104 cells cm−3) in the subsurface of the Atlantis Massif, an oceanic core complex on the Mid-Atlantic Ridge that was sampled on International Ocean Discovery Program (IODP) Expedition 357. This study used fluorescence-activated cell sorting (FACS) to enable the first amplicon, metagenomic, and single-cell genomic study of the shallow (<20 m below seafloor) subsurface of an actively serpentinizing marine system. The shallow subsurface biosphere of the Atlantis Massif was found to be distinct from communities observed in the nearby Lost City alkaline hydrothermal fluids and chimneys, yet similar to other low-temperature, aerobic subsurface settings. Genes associated with autotrophy were rare, although heterotrophy and aerobic carbon monoxide and formate cycling metabolisms were identified. Overall, this study reveals that the shallow subsurface of an oceanic core complex hosts a biosphere that is not fueled by active serpentinization reactions and by-products.