The bullets for sure could cause fliers, but the consistent location of them are hard to explain as bullet caused.I would mostly agree. Most of the time when you see consistent shots out of good groups it is shooter related. BUT sometimes it could be something weird.
I would say several groups have a bug hole with a shot out. When ever I do that, I load 25 of the same load and go shoot five five shot groups and see what I got. was that shot lost to conditions, operator ect.. Now sometimes when you are seeing something like this and it is not the shooter. It is the bullet. Shoot another bullet. Berger 105 HVLD is one I could recommend and if the one shot that is always out goes away its them Hoaky Hornady bullets that have been coming apart. If you print the same groups with the Berger bullet I would look hard at yourself and bench set up.
I once had a freeky accurate 6XC 1-7 twist and I keep loosing shots horizontally at longer ranges and most would say wind ect. but I noticed it was always the same way and the wind did not cause it on some shots for sure. So I talked to a few guys who knew I was serious and it was not the wind and they instructed me that the vertical grip on my rifle and thumb hole stocks are known to cause unwanted horizontal so I changed how I was shooting the rifle and the problem went away. Now if it was out I call the wind did it.
If changing bullets I would say, stay away from Unforgiving VLD’s and go with Berger 105 Hybrids or 107 Sierras.