Volatile molecules containing hydrogen, carbon, and nitrogen are key components of planetary atmospheres. In the pebble accretion model for rocky planet formation, these volatile species are accreted during the main planetary formation phase. For this study, we modelled the partitioning of volatiles within a growing planet and the outgassing to the surface. The core stores more than 90% of the hydrogen and carbon budgets of Earth for realistic values of the partition coefficients of H and C between metal and silicate melts. The magma oceans of Earth and Venus are sufficiently deep to undergo oxidation of ferrous Fe2+ to ferric Fe3+. This increased oxidation state leads to the outgassing of primarily CO2 and H2O from the magma ocean of Earth. In contrast, the oxidation state of Mars' mantle remains low and the main outgassed hydrogen carrier is H2. This hydrogen easily escapes the atmosphere due to the irradiation from the young Sun in XUV wavelengths, dragging with it the majority of the CO, CO2, H2O, and N2 contents of the atmosphere. A small amount of surface water is maintained on Mars, in agreement with proposed ancient ocean shorelines, for moderately low values of the mantle oxidation. Nitrogen partitions relatively evenly between the core and the atmosphere due to its extremely low solubility in magma; the burial of large reservoirs of nitrogen in the core is thus not possible. The overall low N contents of Earth disagree with the high abundance of N in all chondrite classes and favours a volatile delivery by pebble snow. Our model of rapid rocky planet formation by pebble accretion displays broad consistency with the volatile contents of the Sun's terrestrial planets. The diversity of the terrestrial planets can therefore be used as benchmark cases to calibrate models of extrasolar rocky planets and their atmospheres.