It doesn't really work that way. Harder, cold worked brass is more stiff and brittle so it obturates (forms around) the mandrel more and doesn't spring back as much as softer, annealed brass. Think of metal fatigue, if you bend it enough times without heat it will eventually stretch out and break.
Yes, harder, cold-worked brass is stiffer but that makes it MORE "springy", not less. In fact the dictionary definition of spring brass is: "common brass stiffened by cold working or heat treatment". Spring brass is used where corrosion resistance and low cost is important.
Consider a non annealed case neck which has been work hardened by firing and sizing vs a fully annealed one. If you size both necks to the same diameter, let's say .002" smaller than the diameter of the bullet, the harder case will grip the bullet more tenaciously. It will resist the expansion associated with the seating process because it's harder, stronger, and more "springy".
The soft neck, on the other hand, will comply and expand more easily when the bullet is seated and not provide as much bullet grip.
To visualize the difference in grip, think about a pin or bearing inserted into a piece of steel with an "interference fit" vs sticking that pin or bearing into a ball of pizza dough. Which one is harder to extract?
It's important to realize that if you use the same neck sizing bushing on two different cases, one hard and one soft, the softer annealed one will have a smaller finished diameter because it has less "spring back" than the work-hardened one. So to reach the same finished diameter, you gotta' use two different sized bushings. It follows that you can't predict that a hard case will have more bullet grip than a soft one unless you specify the diameter. That diameter depends on the neck bushing size and the amount of spring back associated with the hardness of the brass. It isn't rocket science, but you must consider the important factors.
Remember, the differences we're talking about between annealed brass and brass which has been fired a couple of times is no where near the difference between steel and pizza dough. Therefore, quantifying any effects when the bullet hits the target is tricky. Brian Litz tested brass fired many times against brass annealed each cycle. He found no difference in performance. Still, I anneal each cycle.
It's not that I don't believe the results of Mr. Litz's tests. Rather I'm convinced there is a genuine performance gain by annealing every time, but it's like much of what we do in that trying to isolate and measure a tiny difference in actual performance is difficult when our testing procedure involves a gun held by a human along with bullets flying through air. That is to say, the noise associated with gathering data like this obscures the tiny improvement we're looking for, but that doesn't mean that the effect isn't real.