It is not impossible, though. 4500 renown will give you 200+ party size, with minimal leadership and charisma. Of course, 4500 renown is not trivial to get. Other mods have a lot more ways of increasing party size: - bonuses for being a lord, a marshal or a king. AhUS.

mount and blade increase party size