Gas-giant planets, like Jupiter and Saturn, acquire massive gaseous envelopes during the approximately 3 Myr-long lifetimes of protoplanetary discs. In the core accretion scenario, the formation of a solid core of around ten Earth masses triggers a phase of rapid gas accretion. Previous 3D grid-based hydrodynamical simulations found that runaway gas accretion rates correspond to approximately 10 to 100 Jupiter masses per Myr. Such high accretion rates would result in all planets with larger than ten Earth-mass cores to form Jupiter-like planets, which is in clear contrast to the ice giants in the Solar System and the observed exoplanet population. In this work, we used 3D hydrodynamical simulations, that include radiative transfer, to model the growth of the envelope on planets with different masses. We find that gas flows rapidly through the outer part of the envelope, but this flow does not drive accretion. Instead, gas accretion is the result of quasi-static contraction of the inner envelope, which can be orders of magnitude smaller than the mass flow through the outer atmosphere. For planets smaller than Saturn, we measured moderate gas accretion rates that are below one Jupiter mass per Myr. Higher mass planets, however, accrete up to ten times faster and do not reveal a self-driven mechanism that can halt gas accretion. Therefore, the reason for the final masses of Saturn and Jupiter remains difficult to understand, unless their completion coincided with the dissipation of the solar nebula.