Feel the same way about my own astro shots lol. I was stacking like 30 frames and getting all this weird noise. Now I just do 10-15 good ones and the stars look way sharper.
Yeah, what if the extra frames are just adding more bad data? I mean, I tried stacking a ton of shots once and it made everything look sort of smudgy and soft. Maybe it's just me, but I feel like after a certain point you're not getting more signal, you're just averaging in more little errors from each shot. Now I spend way more time picking the best single frames to even use, and tossing out any with even a little haze or tracking drift.