Hermits won't leave coral alone!

[jray8]

My hermits are constantly harassing my corals (GSP and Zoas). I don't think they are harming them, not that I can tell anyway.
They are also knocking the frags off the rocks. I wake up each morning and have to re-glue 1-3 frags back on the rock work.
Are they picking at the glue? I have been using extra thick CA glue. As of today I added epoxy putty and CA glue both in an attempt to crab proof the frags.
Anyone else had an issue like this?

 

[Gumbies R Us]

My urchin (different scenario than yours) always bulldozed my coral off the rock. Sometimes, he would get on top of the coral, and I thought he was eating it, but he would just eat the algae around it. If your corals are not affected by the crabs, ie. being eaten by them, I would not worry about it.