Looking at pictures of grounding probes, the other end plugs into a 3 hole socket. That third hole is the ground. In fact, the other two holes (or really, slots) are plastic and only serve to keep the plug in place. If your electrical sockets do not have the third hole, the grounding probe's plug won't plug into your socket.
There may be 2 hole (or actually, 2 slot) variations of these plugs for grounding probes wherein the ground is basically a loop that gets screwed into your wall plate (presumably, the screw goes all the way to the electrical box which in turn would be grounded. But if your house does not have ground to the outlets then basically, the ground for the probe goes absolutely nowhere.
I'm not sure if you can use a GFCI on non-grounded outlets. In any case, I would be incredibly cautious about keeping an aquarium unless you have some safety measures in place in case there is electricity leaked into your tank.
Some are as you described, and some are like the one I had, where they screw to the center screw of the outlet plate. Because your outlet box is grounded also. Unless you have knob and tube wiring as I said before. I am not sure about that.