The Curiosity rover has detected diverse lithologies in float rocks and sedimentary units on the Gale crater floor, interpreted to have been transported from the rim. To understand their provenance, we examine the mineralogy and geology of Gale's rim, walls, and floor, using high-resolution imagery and infrared spectra. While no significant differences in bedrock spectral properties were observed within most Thermal Emission Imaging System and Compact Reconnaissance Imaging Spectrometer for Mars (CRISM) scenes, some CRISM scenes of rim and wall rocks showed olivine-bearing bedrock accompanied by Fe/Mg phyllosilicates. Hydrated materials with 2.48 μm absorptions in Gale's eastern walls are spectrally similar to the sulfate unit in Mount Sharp (Aeolis Mons). Sedimentary strata on the Gale floor southwest of the landing site, likely coeval with the Bradbury units explored by Curiosity, also are hydrated and/or have Fe/Mg phyllosilicates. Spectral properties of these phyllosilicates differ from the Al-substituted nontronite detected by CRISM in Mount Sharp, suggesting formation by fluids of different composition. Geologic mapping of the crater floor shows that the hydrated or hydroxylated materials are typically overlain by spectrally undistinctive, erosionally resistant, cliff-forming units. Additionally, a 4 km impact crater exposes >250 m of the Gale floor, including finely layered units. No basement rocks are exposed, thus indicating sedimentary deposits ≥250 m beneath strata studied by Curiosity. Collectively, the data indicate substantial sedimentary infill of Gale crater, including some materials derived from the crater rim. Lowermost thin layers are consistent with deposition in a lacustrine environment; interbedded hydrated/hydroxylated units may signify changing environmental conditions, perhaps in a drying or episodically dry lake bed.