I added a rock to the aquarium and moved things around and the shrimp are getting along better.
Peppermint shrimp are also naturally going to hide a bit more. The skunk cleaners are also very aggressive at feeding time anyway. I once had a skunk cleaner try to fight a nassarius snail for a pellet. The nassarius snail did not know when to stop eating and ended up killing my shrimp.