Like you mentioned, certain species may not have diets that are well-formulated for them. However, in general it is far better for a fish to eat pellets and not frozen than vice versa. The pellets are formulated to meet the nutritional standards for specific species, or varieties of species depending on the diet. They contain vitamin premixes, and typically fish meal to ensure fish get the necessary fatty acids. They are far more complete than non-diverse frozen diets. For instance, if you just feed your fish shrimp they can end up with a vitamin B1 deficiency. There's a lot of thiaminase in shrimp.
Pretty much all farmed fish are fed pelleted diets exclusively, for their entire lives. If you're raising salmon there are a bunch of salmon-specific diets available. It's in the aquarium trade that you see more generalized diets for multiple species because we keep a wider variety.
You'd probably want more of an herbivore diet for your fish than a carnivore diet. I met a well-renowned cichlid breeder once (in FL if that gives it away) and he said that for most of his cichlid species he fed a mix of cichlid pellets and koi pellets to reduce the protein content.
Also, remember that it's generally fine to feed a marine diet to freshwater fishes, but if you do the opposite your fish could end up with fatty acid deficiencies. Freshwater fish are just better at turning one fatty acid into another (desaturation and elongation). Marine fish need more specific fats in their diets, to put it simply.
