I am also recoil-sensitive, mainly due to a couple of sternum reconstructions. Not the same thing, but somewhere in the same neighborhood.
Like you, I find he 20ga OK, but my 120lb Granddaughter does not. Thinking upon that, I have concluded that for folks like me, it may pay well to chose my chamberings as if I was picking them for her. Since we often share rifles, I pretty much am.
Although I can handle the .30-'06, I find the .280 Rem, built on the same parent case, to be as or more accurate, lighter on the shoulder, and longer in the reach department. For hunting, a 270 Win makes sense. I have already aged out of the 1Kyd competition scene. I can still shoot the distance, but the travel exceeds my abilities.
I have expanded my selection of .223 rifles, and would go as far as using a 22-250 if they prove ineffective. For now, my range opportunities all tend to all into 250yd or shorter; if they exceed that, then I will need a 22-250.
Swap the 308's barrel for a 260, and the distances become more reachable with the bonus of lighter recoil. Shooting a 142gr from that case capacity is a lot easier on the shoulder than shooting a 175gr (or larger)
I have never been a big fan of magnums or AI's, seeing them as excessive and maybe too expensive relative to barrel wear. They may be popular for reasons I can evade, and what I need to get done can be done without them. My sole exception is a Win '94AE Trapper Carbine chambered in 44 Mag. Hell of a brush gun for deer.
My bigger bores had value, and I converted that value into smaller chamberings.
I still shoot service rifles, and do it with a Garand and a few 91/30's. Yes, they kick more. The 91/30's employ Limbsaver slip-ons and stock packs (cheekbones can get beat up too), and I have to limit the loads in the Garand anyway.
With some economical minor mods, the 91/30's can become a lot more fun to shoot.
The simplest recoil reduction mod is done with a sandbag/shot bag. Place it between the rifle butt and your shoulder. This increases the amount of mass the recoil must act on before it reaches your shoulder (but does not add to he overall rifle weight). This is a pretty good justification for an adjustable LOP Butt plate Modification.
For whatever reason your Doctor chooses to warn you off shooting, the warning makes good sense to a certain degree. My wife has had hip replacements, and a friend has had operations to reattach torn ligaments. My Wife follows directions and is healing well (this takes years to fully evolve), while my friend disregards his surgeon's warnings and is on his third (and last) attempt to restore his biceps function. I doubt he will ever take the warnings seriously enough, and that his injury will likely prove permanent.
Don't be that guy.
Greg