Camera isn’t the best so hopefully those show well enough.
Ah, these do look solid! Okay, so this is essentially just a cosmetic reseal, plus overflow box!
Would I just put a piece of glass flush against the back of the box and then run silicone up each side of it?
I’ve had my best luck scuffing acrylic up via sandpaper where silicone is going to contact it, giving it more surface area, and irregularities for the silicone to mechanically stick to… there are also specialized silicone primers designed to improve silicone’s adhesion to plastics of various sorts!
In your case, probably a bead of silicone laid vertically up that overflow’s floppy joint on the overflow interior (after scuffing acrylic and cleaning up/surface prepping), then laying your thin glass (or prepped acrylic) strip down into the silicone bead, using the strip itself as a flat surface to extrude the silicone out across it… (silicone traps air very well, you want to use it like a fluid to extrude all the air between parts being bonded from the center-out to get bubble-free seams!)
I’m new to overflows and sumps. Is that what this is?
This is a fairly common “Durso standpipe” design… the inlet is submerged in the overflow box, and air is allowed in via the top, usually metered by a valve… this one has been modified with holes drilled in the side, likely in a futile attempt to get it to run silent!
On that note, this particular single-drain design is relatively outdated, and has been more or less abandoned on new tanks for the inherent noise/lack of redundancy against overflows in favor of dual (Herbie) or triple (BeanAnimal) drain systems!
— because your tank likely has a second hole in that overflow box for water to be pumped back in from the sump, you can actually convert this to a near-silent running dual drain overflow, with the water being returned from the sump plumbed up over the tank rim, if you so-choose!