Here are a few techniques to enhance the sense of depth. You can see how I applied most of them on the right half of my tank. I think I was successful there. On the left, not so much.
- Have an escape point. It's a point where you can see the back glass;
- Create perspective lines that point to that escape point;
- The tank will look deeper if the path that leads to the escape point is not straight and is also in a diagonal and has a few curves;
- Create layers, one behind the other. You can use rock and especially color to create these layers.
- Put larger and less detailed stuff (rocks and corals) in front and smaller more detailed stuff at the back.
- I know this is a difficult one in reef tanks, but raise the ground level at the back.