I like a tight thumbhole.  I currently use an ovals in VISE IT inserts.  My thumb pitch is 3/16 forward.  My problem is my thumb goes up and down in size and sometimes I can barely fit it in the ball .... the only remedy seems to be a bigger thumb that rubs less and add/remove tape ... which I HATE as the feel sucks and messing with tape is a pain.

Any ideas?  Could my fit be causing the swelling?

The fit could be if you are squeezing the ball.  My first thought is what do you drink while you are bowling?  If you are dehydrated; that can lead to swelling as well.  I used to have a soda while bowling until I was having swelling issues but then switched to Gatorade or lemonade and the issue went away.  I wasnt always a two hander but even as I am now, my fingers can swell making the finger holes pretty tight.  But lemonade is my go to while I am bowling now and swelling is not an issue anymore and that is even with me living in Florida.

Have your fit checked and rechecked. If it all seems fine, you may just be someone that deals with a lot of fluctuation.

You're using Vise ITs already, so if your fit is fine, you'll just need multiple ITs in different sizes

Ever try Magic Carpet?

http://www.bowl4fun.com/magiccarpet/magiccarpet.htm

Try taking a few ibuprofen about 30 to 45 minutes before you bowl.  I find that when I do my fit stays more consistent throughout the night of bowling.

I  heard about using ibuprofen prior to bowling to prevent finger swelling.
I would add a word of caution for seniors on the use of ibuprofen for this purpose.

My physician warns his older patients over age 63 on the use of ibuprofen due to the increased risk of stomach hemorrhaging in patients in that age group. I'm included in that group. In fact this warning is included on all ibuprofen labels. As a result, I only rarely resort to using ibuprofen unless I'm in pain that Tylenol can't quite handle. (just one more disadvantage of growing old).
My doctor recommends Tylenol for most of his senior patients.

Instead, I control thumb swelling by having my IT thumb slugs made slightly larger. If the thumb is too loose, I use thumb tape. I try to maintain soft grip pressure, and although I too, have several different IT thumb sizes, nothing "fine-tunes" my grip better than good old fashioned thumb tape.