It doesn't stop there. Why not 9 instead of 8? Why not 10 instead of 9? There comes a point where designers need to make a call on that sort of thing, and it's usually not an easy one to make. You have to weigh what different players say they want along with what's best for the health of the game you've designed. Sometimes they mesh and it's awesome! Sometimes they don't, and it's frustrating, because you want what's best for your game to also make everyone happy. It's nearly impossible to please everyone all the time, though, so it's a matter of weighing the drawbacks of a system and choosing the approach that is the best overall with the least downsides.
Also, we're not saying 4 player co-op is more fun than having 5 players or 8 players. We're not saying it's less fun, either. We're saying that it's what we believe works best for Diablo III right now. That's due to a number of factors including screen noise, buff management and scaling, player contribution, and making sure certain group compositions didn't feel mandatory. We very carefully deliberated party size during development, and we continue to deliberate party size even now.
