Yes the big bore vs smaller bore debate could go (and I'm sure will go) on forever. I guess I am partial to the .264 bore size as I have owned a 264WM for over 40 years now. I know it can easily handle moose. Have not had one go more than 10-20 yards after taking a 140 grain Nosler Partition.
I think that sometimes when we look for accuracy we focus on the cartridge and barrel maker. However as we all know accuracy is about a whole bunch of little things. One aspect often overlooked, is what I would call cartridge support. Can you get quality brass, quality reloading dies, and bullets? I don't think there is anything magic about the 308 as a cartridge. It is a somewhat underpowered 30 caliber. However, the cartridge support is phenomenal because of the history and popularity of use in long range target shooting.
An example at the other end of the scale is the 264WM. It has zero cartridge support other than a good choice of bullets. It would be very difficult to get it operating as a target gun. The 30-06 is kind of between those two extremes for cartridge support. Not real popular as a target gun, so support is lacking. The other aspect is recoil in a 6.5 lb gun. I don't like the recoil in my 264, and the 30-06 which I have shot, is worse.
In an earlier post, I mentioned the 6.5x55 Swede. Forgot that it has a larger head dimension and would require a bolt replacement.
I think a good argument could be made however for using the 6.5-284 as a long range elk cartridge in a lightweight gun. Cartridge support is right up there with the 308, and long range accuracy potential is undisputed. With careful load development, I would expect you could get velocity in a 140 grain up around 3,000 fps, and with the high BC bullets available that should deliver a punch out a long distance. It is also nice to not have to deal with huge drops in the trajectory. And while recoil will not exactly be light in a 6.5 lb gun, it will be more reasonable than that of a 30-06.
A couple of links to look at if you have not seen them before.
http://www.6mmbr.com/sixfive284.html
http://www.norma.cc/content.asp?Typ=59&Lang=2&DocumentID=263&Submeny=4&Rubrik=Calibers&Title=6,5x284 Norma