It depends on the brass. I never deburr Lapua, Norma, or RWS. I will uniform the flash-hole from behind using the Sinclair tools to 0.061" / 0.081". With these makes, that's usually wasted effort, but just occasionally an undersize one appears that needs a lot more effort to ream. One box of Lapua 223 had three really, and I mean really, undersize flash-holes that would have ruined ES/SD values in this little cartridge. The small diameter holes in 220 Russian / BR / 6.5X47L are really consistent though IME.
Some American makes though ........ Deburr? Hell, yes. I find it astonishing that people say they've never seen a burr - a small intense flashlight beam at an angle though the flash-hole from the primer pocket end usually shows some burrs, and often some very large spikes left - they will angle the jet of flame and hot particles from the primer to one side of the powder column.
Primers are explosives and won't be affected by a burr or two as claimed in post #13. You think this pair here getting on for 1/8th inch length and both partially covering the flash-hole would have had no effect?