Your plan will work great I think.

Honestly, hacking in Rare Candies and Max Repels into my games fixed this and made the games far more enjoyable

Insect swarm creates a magical effect, it doesn’t summon literal bugs. And before you call me a buzz kill, it’s kind of lame to try and essentially skip the encounter I spent time building.

To newer DMs: Never admit to your players whether or not you fudge rolls. As the DM, The only thing you need to do to maintain the integrity of your game is to shut your damn mouth when you bend the rules. The players just need the illusion maintained.