Honestly, once you price out acrylic, you'll see that most of the parts made with it (like overflow boxes) are not that expensive in comparison to the price of the acrylic itself. I'm toying with the idea of making some containers for an automatic water change system, and while there are some price savings involved in building them myself, I'll probably just buy some commercially made containers.
For example, a 5-gallon acrylic water container sells for about $100. If I were to make a 7-gallon container, it would cost me about $72 in acrylic, to say nothing for my time or the fact that the fumes from making it will make me sick for a day or two. My advice, pay the money for a commercially made overflow box. Your cost savings in making it yourself won't be that great, and to my mind the peace of mind from a professionally made item is worth it for something as critical as an overflow box.
If you do decide to make it yourself, Weld-on
#4, #16 or #40 should all work for your needs. You'll likely need to go to a specialty shop or put aside plenty of time to have it shipped - most of the big-box stores don't carry acrylic glue, and shippers won't ship most acrylic glues except by ground shipping. Make sure you use it in a well-ventilated space, and use gloves. I mostly use #16, which has a very runny consistency - it almost comes out of the tube just by tipping it slightly.
Most places that sell acrylic by the sheet will also cut to size, so you can save yourself some time by planning out your pieces ahead of time and having them cut that way to begin with. I'll typically only cut acrylic myself when precision isn't important - it's not as difficult to cut as people like to pretend it is, but it is very difficult to get a precision cut unless you have specialist (read: expensive) tools. If you must cut acrylic yourself, you have a few options. For large-scale cuts, a table or circular saw (table is better) with a specialty plastic blade is best. You can use a jigsaw or bandsaw to do curves, but you have to be very careful - stressing the material incorrectly can crack it. A hacksaw with a fine-tooth blade can work by hand. Finally, a router with the correct bit can help you do precision work or small jobs.