We conduct a pebble-driven planet population synthesis study to investigate the formation of planets around very low-mass stars and brown dwarfs in the (sub)stellar mass range between 0.01 M- and 0.1 M-. Based on the extrapolation of numerical simulations of planetesimal formation by the streaming instability, we obtain the characteristic mass of the planetesimals and the initial mass of the protoplanet (largest body from the planetesimal populations), in either the early self-gravitating phase or the later non-self-gravitating phase of the protoplanetary disk evolution. We find that the initial protoplanets form with masses that increase with host mass and orbital distance, and decrease with age. Around late M-dwarfs of 0.1 M-, these protoplanets can grow up to Earth-mass planets by pebble accretion. However, around brown dwarfs of 0.01 M-, planets do not grow to the masses that are greater than Mars when the initial protoplanets are born early in self-gravitating disks, and their growth stalls at around 0.01 Earth-mass when they are born late in non-self-gravitating disks. Around these low-mass stars and brown dwarfs we find no channel for gas giant planet formation because the solid cores remain too small. When the initial protoplanets form only at the water-ice line, the final planets typically have 15% water mass fraction. Alternatively, when the initial protoplanets form log-uniformly distributed over the entire protoplanetary disk, the final planets are either very water rich (water mass fraction 15%) or entirely rocky (water mass fraction 5%).