What did your vet say ?
Given the operation, I can't see how a bit of swelling could be a worry but it depends how it swells and if it doesn't go down.
I knew of a poor mite who had a slight infection after his gelding but you could see horsey was very unhappy and almost unable to put one hoof in front of the other.
As Poppy said, every case is different and every vet will probably give slightly different advice.
My arab was gelded in my back garden, lying down.
Vet told me to keep him in a week on straw bedding (didn't want shavings or dust getting into the wound). My vet did not put external stitches to allow the wound to drain if need be, just a post-op spray from memory. He had an antibiotic injection the day he was gelded and another inj a few days later.
I think it did swell a bit but then went down rather quickly too - he was fine when he went out into the field a week later.