What I meant to say is to plug the ball first then use the slugs.
Usually plugging the finger holes will leave a thin wall after you drill the hole. Yes, you can drill out the hole to make it bigger then plug the hole, but the wall will still be a bit thin after you bevel it which could lead to further problems. Plus, I find that plug can crack in the future much easier than the slug.
As far as glueing them in, I never had a problem. Like thumb slugs, I scrape the OD a bit with a rasp file and glue, leaving a bit on top to cut down to make perfectly flush with the top of the ball.
They do come in different sizes and colors, but you are correct, depending on the color of the ball, they may not be a perfect color match.
Now, this solution is not for everyone. You should apply the proper solution based on your customer. Some of my customers get plugged and others slugged.