I was concerned about bullet break-up too - we have some big mulies around here, and black bear are commonly found during deer season as well - so for my .25-06 I worked up a fast, accurate load with the 100 gr Barnes TSX.
Starting w/new Win brass, I prepped the brass, primed w/Federal 210 match primers, and started working up with H4350. Barnes recommended H4350, and I decided to give it a try since it's worked well for me in the past in other cartridges. Interestingly Barnes doesn't list specific recommended powder charges for their TSX bullets, but instead said to start w/Barnes X data and start working up.
I worked up to some pretty high powder charges - with no pressure signs. Best accuracy though was at a comfortable 53 grains of H4350 @ 3340 fps. Accuracy was sub MOA out to 300 yards, the farthest I tested it. Only shot one deer, and the bullet exited, so I can't say what the recovered pill looked like. I understand that's common for Barnes bullets since they have no lead core, there's not much to come apart.
When I commented about my good results from this load to Barnes, they noted that the same .25-06 bullet had been used succesfully on elk... Not sure I'm willing to push it to that level, but it made me feel pretty good about carrying it for both deer and black bear. The rifle is a Rem 700 CDL, 24" factory sporter barrel. No bedding, no tuning up at all, all factory. The Sierra has proven a bit more accurate, but like you mention, it does break up pretty easily. I don't think that's all bad given chest shots on deer though.
Regards, Guy