I bought a new gorg and then saw this but it looks exactly like the white polyps on my red gorg. Finally, saw it in a different location and is definitely some kind of worm or nudibranch.
I don’t see damage on gorg but the polyps aren’t out all the time either which worries me.
But maybe this...