The 5r, a loose fitting pilot and reamer geometry can all play a part in chatter. Wax paper, IME, works if you catch it in time but if you keep going in, hoping the chatter stops...You're not likely to be happy.
IME, you will have to re chamber to get rid of your chatter. The good thing is, you have options since it's a small bolt face cartridge that has the problem. The bad part is that you already have a chamber and no bushing support. Still, I'd bet on it cleaning up without a problem with a different reamer in a larger cartridge.
Flute spacing, shoulder angle and pilot fit..are just some of the potential causes of chatter.
I've seen 5 groove bbls cut very well while others have what appears to be peaks and valleys in the throat area. That looks ugly but they still shoot like that. Mostly, it just doesn't look like we accept as what it SHOULD look like. Well, who determines that, if they both shoot?
I'd rather chamber a 4 groove and not have to worry about it. What has been said previously is true..that the bushing pushes off to one side as the reamer cuts a land when there is not a land directly opposing it.