At that sort of scale, I don’t think you will see any difference in performance.
The only place you need to worry about performance is if you are going to have very big or very small models (1,000,000s or 1/1,000,000s). In which case you might find that floating point imprecision causes your models to render incorrectly.
I think you’ve meant size in terms of file size, not the scale of model.
Bigger file size - takes longer to download, and has more data to deal with. So the goal is to keep it at minimum while still satisfying your quality requirements.
This relates to everything: models, textures, scripts, audio, etc.