Looks like a sponge. Not likely to be the problem with the hammer, it's basically an inanimate object. Unless some is poking into the hammer's flesh, I wouldn't expect it to be a problem. Heck, even then, it shouldn't be doing anything. You can scrape it off if you really want, but it may recur.