Prokaryotes constitute the majority of sedimentary biomass, where they cycle organic carbon and regulate organic matter transformation. The microbes inhabiting sediment are diverse and the factors controlling microbial community composition are not fully understood. Here, we characterized the prokaryotic community using 16S rRNA gene sequencing in 24 stratigraphic layers within a 89 cm (dated to ~1900 years old) sediment core from an anchialine sinkhole in the Bahamas with a stratified water column and anoxic bottom water. The microbial community was dominated by members of the Alphaproteobacteria, Dehalococcoidia, Gammaproteobacteria, Bathyarchaeota, and Campylobacter classes. Most interestingly, subsurface microbial community structure could be correlated to previous evidence for timewise changes in the main source of organic matter that was supplied to the sediment accumulating during the last 2000 years, which itself was caused by regional terrestrial vegetation changes. The C:N ratio was correlated to the relative abundance of the microbial classes, and the microbial communities followed three previously determined time periods based on the source of organic matter, which suggests that the carbon source at time of deposition influences the resultant subsurface microbial community composition. These results show that carbon source is a driver of the microbial community composition inhabiting anoxic sediment, which could have implications for improving understanding of carbon cycling in coastal sedimentary basins.