For subseafloor microorganisms, defined geological and chemical gradients affect population sizes and community structure. We examined how distinct sediment types influence microbial diversity and community composition and the factors that drive deep-subsurface microbial community structure (e.g., depth, interstitial water chemistry, sample location). During IODP Expedition 349 (South China Sea Tectonics), either coupled ash/clay or turbidite/clay boundaries were sampled, DNA extracted, and the 16S rRNA gene analyzed on an Illumina MiSeq platform. Microbial communities in sediments were distinct from communities in drilling fluid, indicating that drilling-based contamination was unlikely. Illumina sequencing of the 16S rRNA gene yielded 5,453 OTUs (97% identity) representing 44 bacterial phyla and 3 archaeal phyla. Members of the Atribacteria dominated all microbial communities among all sites. Sulfate-reducing bacteria were relatively rare within sulfate-replete sediments in all cores. Ordination of the microbial communities using weighted UniFrac distances revealed significant differences in communities both between sites and between the sulfate reduction zone and methanogenic zone at two of the sites. The number of observed taxa followed an exponential decline with sediment age only in the sulfate reduction zone. Species evenness increased linearly with sediment age regardless of geochemical zonation in the sediment column. Our investigation helps to characterize the factors that drive microbial community structure of the subseafloor and highlights the need to focus on habitat heterogeneity at a scale pertinent to bacteria and archaea in studies of microbial ecology.