I think the bit is too sharp, and your machine may be too slow. I would not change the speed. I like Merlin's idea. I use an 1 3/8 hole before drilling the 1 1/2 hole for the Switch Grip. After a few holes, your 1 1/2 should get slightly duller and not chatter so much.
Using your maching too fast can be dangerous, especially when cutting plug down. Using your machine fast dulls your bits faster. Too much heat cannot be good for balls or bits. Slower is usually better, keep your bits sharp so you don't have to mash down on the bit to get it to cut, which causes more heat, which dulls your bits, which........
--------------------
Mike Austin's Precision Pro Shop
Houston, TX
Drilling and Tech Advice BirdDogbowling.com
Driller to many "Stars" and Tony Melendez too!
"So Many Balls, only ONE drill press!!"
Chuck Norris is the only person to ever count to infinity..........twice!