You are correct that there is some type of lever arm at work. But thing to consider is that the 1 oz of weight is not a point mass, more or less it is distributed evenly through out the side of the ball. Further more the 1oz of weight compared to the total weight of the ball is small. Because of this, the amount of difference between being more towards the center of the ball or more towards the cover is negligble.