It's going to grow a lot more. Just clean and don't worry. Remember, never use any type of soap/detergent. It'll wipe right off.
As for the glass, algae is a never ending battle. I would look into a magnetic algae scrubber. Many different types out there. Check with your LFS (Local Fish Store) and ask about one. Magnetic scrubbers allow you to clean the glass in the tank from the outside. You don't even get your hands wet.