Saying you averaged 232 in the league last year and now averaging 210 is where the issue is with other bowlers. Technically, you can use any ball you want, whenever you want. But the intent in competition, and league is generally considered competition is to bowl as well as your abilities allow. USBC Rules classifies "establishing an average below the player's ability to gain an unfair advantage in handicap or classified competition" as an unfair tactic and grounds for disciplinary action.
This league in which you are averaging is the the only league you bowl in, thus at the end of the year, an outsider looking at this situation could deduce you are "intentionally" averaging 210 when you are more than likely capable of averaging around 232. If you do not use this average to bowl tournaments in which average is used as a classification or as a basis for handicap, then you are entitled to bowl with whatever ball you want. But if you do decide to bowl using this 210 average as a basis, then I could see there being an issue.
As for the league competition, there is nothing the other participants can do if you are ONLY using the spare ball. The only time I could see it being a problem in the league is the start of next season where you would use last year's average for the first few weeks and you go back to more aggressive equipment and start averaging 230+ again (assuming a 210 average garners you some handicap).
Personally, I wouldn't say anything about it. I think this is one aspect of why many people want to instill a "tournament average". Though your intentions may be honorable, others are not.
I am not a pro-bowler, but I do play one on BallReviews.com