Pictures may help, but typically you will want to clean algae build up from the filter chamber (especially if it is slimy). Additionally, work to understand what is causing the algae and address that as well.
Yeah, I'd say you should definitely be removing algae buildup. That build up over time could cause flow issues and it will help you control excess nutrients that come about from that algae dying off and going back into your system.
Yup I always remove algae build up in the back chambers. Helps keep flow moving. I also just like a clean tank. Prefer all my gunk to stay in the refugium chamber.