You can use either sandpaper or acetone to open up a finger insert.
If you use acetone, wear a thin glove to protect your finger from the acetone.The acetone dissolves the vinyl of the insert. Just rub the back of the hole; the cloth with which you rub will become discolored with the color of the insert. That shows you are removing some material.
Sandpaper: fold a 100 grit piece into a long (2.5 - 3") by 1/2" or so, so it will fit into the finger hole. just sand the back of the hole.
Sandpaper is safer for you than acetone. Just be careful how much you remove.
FYI it should be very easy to remove an insert. I understand your not wanting to remove it if you've never done it before. Once you do it once or twice, it's very easy to remove and re-install, if you're careful. It's not rocket science.
Whoever installed it should have glued it in 2 spots. However, some people get carried away and glue the whole circumference; this is over-kill, unnecessary and possibly damaging to the coverstock. Too much Super-Glue interacts with the resin making it overly brittle. Use a 1/8" wide flat blade screwdriver to edge carefully, the outer rim of the insert away from the inside hole of the resin. Use a bright light so you can see the tiny spacing between the two. Go very slowly at first. Tiny tiny fragments of the insert may stay glued. Usually they are so small, they hardly matter.