Nature is a place where just about everything dies an early death from violent causes. Unnatural things are not inherently bad, and many organisms are in fact perfectly fine in a smaller space than the wild would provide. Clownfish aren't out in the ocean traveling miles in a day, they pick a spot and stay in it. And anemones don't even have a brain to care with. Plus, corals are largely sessile, they don't care how much space they have.
Xenias don't keep any important organs in their stalks, or anything at all except some support. Anemones are full of more stuff, and are much less resistant to damage. Think of it this way: if I chop a xenia colony into a bunch of little pieces, I get loads of colonies. If I chop an anemone into loads of little pieces, I get a really bad soup.