There is a minimum size allowed under USBC rules, out of the factory or not. Ball manufacturers try to stay close to the large end of the spec so that the balls can be resurfaced a few times.
A good rule of thumb is that as long as the logos and serial numbers are still visible, the ball is still within spec. The balls aren't necessarily designed that way and the rules don't necessarily make that true, but it's a pretty safe guideline.
In practice, aside from the logo and serial number (which can be re-engraved at any point and doesn't have to be the original serial number anyway), you want the ball return to be able to return the ball properly. If the ball is too small, the machinery won't get a good grip on it to pick it up from under the lane bed.
SH