Some people remove bristle worms in any form because let's face it, they're ugly, but a lot of folks also just leave them in. They're harmless scavengers and can get into the nooks and crannies of your rock better than most anything else. A surefire way to know if it's bristle vs fire is more often than not fire worms are actively hunting, even during the day. Bristle worms go pretty much unseen until dusk or lights out completely then you can spot em with a flashlight.
If you're concerned you can remove it, make sure it doesn't break or you'll still have a bristle worm, best to get it with tongs when it's fully exposed. DO NOT TOUCH THEM. You'll have a bad time, like fiberglass (bristle) or fiberglass + fire sauce (fire worm) in your fingers.
There are also plans for traps you can find around if you want to remove it.