Ok, just as I thought, too much polish. Tried it again, no luck. Then lightly scuffed the surface with a green scotch brite by hand. Now it looks more like OOB.
How do I polish a ball and not have this problem? I barely used a dime size of FF on two sides.