I'd say it's mostly a question of aesthetics. If you leave them in endoliths and biofilms will colonize the shells pretty quickly doing their stuff to maintain a reef system.
I feed lots of clams to my puffer and I have been lazy about taking the shells out. The shells can potentially contain some phosphates and other trace elements from where they were sourced which can be a good or bad thing. I would say other than shells being a detritus trap there is minimal concern with leaving a few in the aquarium.