99% of the time you want your speakers set to small since subwoofer will be taking care of lower frequencies below the crossover that you have set. And this is a good thing. Subwoofer will be far more efficient at producing lower frequencies than your speakers would, it has an active amplifier which means it take load away from your main amplifier making it run cooler and having more juice to power the main speakers.
One reason for having your speakers set to large would be if you didn’t have subwoofer. “SOME” purists say that stereo music should only be listened with two speakers and no subwoofer and in that case yes you would have speakers set to large BUT I don’t quite believe that to be the case.
tldr: 99% of the time use small, unless you don’t have a subwoofer :)