It might be bad, but I do it.
I would pet my fish except that I'm a Baptist so those things are probably frowned upon?
Actually, I try to keep my hands out of the tank. I do have a tang that bumps into the feeding cup and has almost knocked it out of my grip a few times. I think I could hand feed and probably pet her but I do worry about where my hands have been and I don't want to spawn a new flu virus.
My new fish generally get hand removed from the acclimation bag and carefully hand lifted into the tank where I slowly release them as I welcome them to our tank.