Pebble accretion is an efficient mechanism that is able to build up the core of the giant planets within the lifetime of the protoplanetary disc gas-phase. The core grows via this process until the protoplanet reaches its pebble isolation mass and starts to accrete gas. During the growth, the protoplanet undergoes a rapid, large-scale, inward migration due to the interactions with the gaseous protoplanetary disc. In this work, we have investigated how this early migration would have affected the minor body populations in our solar system. In particular, we focus on the Jupiter Trojan asteroids (bodies in the coorbital resonance 1:1 with Jupiter, librating around the L4 and L5 Lagrangian points called, respectively, the leading and the trailing swarm) and the Hilda asteroids. We characterised their orbital parameter distributions after the disc dispersal and their formation location and compare them to the same populations produced in a classical in situ growth model. We find that a massive and eccentric Hilda group is captured during the migration from a region between 5 and 8 au and subsequently depleted during the late instability of the giant planets. Our simulations also show that inward migration of the giant planets always produces a Jupiter Trojans' leading swarm more populated than the trailing one, with a ratio comparable to the current observed Trojan asymmetry ratio. The in situ formation of Jupiter, on the other hand, produces symmetric swarms. The reason for the asymmetry is the relative drift between the migrating planet and the particles in the coorbital resonance. The capture happens during the growth of Jupiter's core and Trojan asteroids are afterwards carried along during the giant planet's migration to their final orbits. The asymmetry and eccentricity of the captured Trojans correspond well to observations, but their inclinations are near zero and their total mass is three to four orders of magnitude higher than the current population. Future modelling will be needed to understand whether the dynamical evolution of the Trojans over billions of years will raise the inclinations and deplete the masses to observed values.