It depends if I am shooting off hand standing or prone or from a bench very different beasts!
It also matters if I am shooting 3 rounds or am I putting 40-100 rounds down range?
You hit a particular wall at some point. Like with hand guns I can put 1000+ rounds of 9mm or 22lr down range and never know it. 1 box of 44 Rem Mag and the next day even in college my thumb and wrist on my right hand would be numb or tingling for a day after wards.
I can put 100 rounds of 300WM down range even with a muzzle break and never feel it but have known guys suffer from what they described as tell tale symptoms of a concussion from shooting a 2 day match with 30-06 or even 308. My father in-law and I could shot hundreds of rounds of magnums from a bench and be fine but my brother in-law was in serious pain and black and blue from just 5 rounds of 300WM.
You see the same thing in contact sports, martial arts, soccer some people can just take more shock with no obvious effects. It is not about being tough so much as ones natrual constitution. Just like some people can handle the cold or the heat better than other's.
Now this assumes you not nutrient deffecient, healthy, active and not just getting older. If you are chronicly low on Vitamin C, D, Zink, Sodium Chloride, Potaasium, Magnessium, CalciumBoron and the B-Vitamins you will be intollerant to heat, cold, impacts, stress etc....You will bruise easy and your joints will hurt a lot and you tendons and ligaments will be chronicly inflamed. Oh and water!!!! The more dehydrated you get the more likely you are to sustain a concussion and the higher the grade of concussion is likely to be!
I am 51 now and I would not want to shoot more than 20-30 rounds of anything 338LM or larger if I can help it just because I am older and do have some health issues in my joints. I can shoot more standing off hand than I can prone since I can use my entire body to soak up the recoil. Big boys prone would wear me down faster. This assumes hunting level of accuracy or at most US Army Infrantry standards not DMR or better and deffinetly not F Open level of accuracy.
If I had to shoot F-Open levels of accuracy a 30-06 level of recoil would be my limit. If shooting a Silhouttee Match I would not want more than 308Win level of recoil at the max especially if it is a 2 day event!
If I am hunting dangerious game I could handle a 416 Rigby since you are likely only talking about 2-3 shots max per day for a week or less and that almost never happens right you will spend far more time hunting thin skinned non-dangerious game with a something like 338 Win Mag.
For some people how the recoil develops matters more than total recoil. My wife in her youth had no problems with my 300WM but my super light weight 12ga slug gun that I radicaly ligthened gave her issues and she grew up shooting trap, skeet and hunting with a 870 Wingmaster her entire life. Keep in mind in college I was 6 foot1 242lbs. and had given up a football scholarship to University of Wisconsin because my team of Internests and several surgeons told me that if I accepted the Scholarship I would likely need a cane or a walker by the time I graduated. So I am a very large muscular guy. My then girfriend now wife is 5'2 and 120lbs. I built that rifled slug gun so I could dissapear into the woods for days or weeks at a time. So the barrel was made lighter, the stock lighter and the 12" by 1" steel rod in it's butt was removed. My point is that just the recoil numbers alone do not always translate how the recoil happens as in the speed with which it happens, the weight of the rifle, and the muzzle blast all matter. A 30-06 with a 16" barrel and radial break on it will kill most people faster than a 300WM in a 26" with no break! We see this with 223 even you will learn to hate the guy next to you at some tactical club shoot next to you with a 11.5" AR with break on it real fast likewise shoot that then shoot a 20" M16A2 and experince the difference. This is with the tiny 5.56 NATO scale that up.
A lot of people think the 9.3x62 is a beast even though on paper it does not look like much. That said it is well known to split stocks and punish the shooter. It is how quickly the recoil develops it is like the difference between a big guy giving you a firm but powerful shove versus vs an Olympic sprinter giving you a very fast kick to the gut. The shove might have more actual force behind it but is better tolerated than the much faster but less power kick from the sprinter due to the difference in the time it takes for the full force to build and dispate through you.
So for me I am weakest at handguns! A box of 44 Rem Mag is my limit. I have known plenty of guys who could out do me any day of the week and twice on Sunday when it comes to magnums and big bore handguns. On top of that small light weight handguns that snap sharply also beat me up. Shortly out of college so well in my prime I shot a Khar 10MM I think it was and that gun beat me up fast. I do not recall if it was a fullsize or compact I want to say it was rather compact. I loved the gun but would have gotten it in 9mm for sure if I bought one not 10mm!
I definetly think that lifting weights and doing some modest cardio can help you shoot better. Even a comprehensive body weight routine with some mild cardio or HIIT can drasticly improve your ability to shoot if not shooting from a bench and can improve your ability to endure any sort of stress compared to doing nothing. Youth is an advantage but not a must since experince, reading the wind and reloading skill usually come with age!
For me the weight of the rifle matters more. Holding a heavy rifle or having to cary a heavy rifle wears me down more than the recoil. I can see the day comming though where my love for heavy magums will come to an end!
P.S. Never neglect the mental game. Rehersal, self hypnosis, professional hypnosis, affirmations, visualization, calling your shots, and confidence all play a role from Olympic athlete to plinking in the back yard!