A round donut with radius R1 occupies the same space as a square donut with side 2R1. If the center circle of a round donut has a radius R2 and the hole of a square donut has a side 2R2, then the area of a round donut is πR12 - πr22. The area of a square donut would be then 4R12 - 4R22. This doesn’t say much, but in general and throwing numbers, a full box of square donuts has more donut per donut than a full box of round donuts.
The interesting thing is knowing exactly how much more donut per donut we have. Assuming first a small center hole (R2 = R1/4) and replacing in the proper expressions, we have a 27,6% more donut in the square one (Round: 15πR12/16 ≃ 2,94R12, square: 15R12/4 = 3,75R12). Now, assuming a large center hole (R2 = 3R1/4) we have a 27,7% more donut in the square one (Round: 7πR12/16 ≃ 1,37R12, square: 7R12/4 = 1,75R12). This tells us that, approximately, we’ll have a 27% bigger donut if it’s square than if it’s round.
tl;dr: Square donuts have a 27% more donut per donut in the same space as a round one.