Healthcare provider here. I post as a hobbyist, this is not professional advice.
I wear gloves as often as possible when handling coral (disposable nitrile) or am working for more than a few seconds inside the tank (shoulder-length pond gloves.) This is 20% to protect me and 80% to protect the critters. They are ancient sea creatures covered in slime and secreting toxins. But I am a giant land animal covered in acidic oil, chemical residues, and sloughing millions of flakes of dead skin.
There are generally three types of risk from your tank:
1) Toxins. Rare for a home hobbyist to get a dangerous dose, usually from a fish or anemone sting, and even then it’s rare for it to be life threatening.
2) Allergic reactions.
A) The vast majority of these are delayed reactions, like one would get from poison ivy. They show up hours or days after contact. These are not life threatening. Benadryl, steroids, time, not much else we do.
B) Less common, and what most folks fear, is anaphylaxis. This happens fast, like within 20 minutes. While it can happen slower, if it’s been more than 12 hours it’s very probably not anaphylaxis. If you have a history, you better get an epi-pen.
C) Infection. Mycobacterium marinum is one to keep on your radar, but it’s rare. Be aware of breaks in your skin barrier (cuts, rashes, hangnails, etc.) and wear gloves when you can.
Personally I have held tons of stuff, including zoa frags, with bare hands. No issues. But I’m observant and a good hand washer. If 24 hours goes by after touching something dicey, I would stop worrying.