We explore the growth of planetary embryos by planetesimal accretion up to and beyond the point at which pebble accretion becomes efficient at the so-called Hill-transition mass. Both the transition mass and the characteristic mass of planetesimals that formed by the streaming instability increase with increasing distance from the star. We developed a model for the growth of a large planetesimal (embryo) embedded in a population of smaller planetesimals formed in a filament by the streaming instability. The model includes in a self-consistent way the collisional mass growth of the embryo, the fragmentation of the planetesimals, the velocity evolution of all involved bodies, and the viscous spreading of the filament. We find that the embryo accretes all available material in the filament during the lifetime of the protoplanetary disc only in the inner regions of the disc. In contrast, we find little or no growth in the outer parts of the disc beyond 5-10 AU. Overall, our results demonstrate very long timescales for collisional growth of planetesimals in the regions of the protoplanetary disc in which giant planets form. This means that in order to form giant planets in cold orbits, pebble accretion must act directly on the largest bodies present in the initial mass function of planetesimals with little or no help from mutual collisions.