I have no idea how experienced a shooter you are, so all of this may not apply. If you're used to bolt gun accuracy, you're going to be a little disappointed with that AR. But some of this should help.
Look in the manual and see if there's a break in period. Some MFG's require a break in period to smooth down some of the manufacturing burrs. Especially true if it's a button rifled barrel. Speaking of.. I looked up the specs on the BRO Spec 15 and couldn't find any information on the barrel. The twist rate will tell you what grain bullets to chase but it looks like you're getting the best groups out of the 55gr Vmax.
Might be worth shooting a group or two without that flash hider. Sometimes armorers will torque the crap out of the FH to time it and that can deform the crown. Keep in mind that you'll have to replace the crush washer and re-time it when you remove the flash hider. Easy job, but you'll need the right tools.
Get rid of the Mil-Spec trigger. Get a Geissele/ALG or a Triggertech. Small investment will give you a much better payoff.
Yes, check clearances between the barrel/gas block and rail. I would go so far as to unbolt the rail and re-torque the bbl nut, but I have everything I need to do that at home.
Also try a different scope. Anything not a Burris or similar. And I would try a different mount. If you're chasing SubMOA then try this.
And if none of this works to your satisfaction, it's barrel time. Other members have had good suggestions for manufacturers. Proof research, bartlein, shilen, krieger, and white oak. I even think Shilen offers a bolt matched to a barrel.