Why is the recommended amount of exercise per week so low?It's actually more like 5.3% of your waking hours.
If the average person works 40 hours a week (and most work more than that), then it's actually more like 8.3% of your available time.
Add to that an average commute of an hour each day, you're at 9%. That's not to mention all the monotonous time you already spend cooking, cleaning, travelling, urinating, defecating, grooming, looking after dependents (children), etc...etc...
It's easily 10% of a person's day or more when you give the situation context.
Relatively speaking it's hardly an insignificant amount of time as you suggest. Especially if you consider that our ancestors moved mostly out of necessity and not for 'health reasons' as we might today. Nor would they have employed the same strategies that we can today.
Quantity (time spent) exercising also completely ignores the quality of that time. There is a difference in the guidelines between vigorous activity and moderate or light activity. A person who does 20 minutes of vigorous activity daily is probably just as well off, or potentially better off than a person that stands or walks for hours a day. You can't treat gardening the same as weight training. Deliberate exercise is very different from indirect movement for hunting, gathering or manual labour.
So the quantity of time is irrelevant in the absence of discussing quality.
Athletes are stronger today than ever before, more powerful, faster, bigger, etc...etc... Not because of significant changes in our genetics, but because we understand more about training/technology and have manipulated the body to do better than before.
With our current understanding of human movement we can create a far greater physical strain and adaptation in a significantly shorter amount of time spent. Interval training is a good example. Recovery research another good example. We now understand training and how the body works far better than we used to. This means we can increase quality and shorten duration if we desire. Sometimes this leads to a better overall effect, depending on objectives.
A person who does 20 minutes of high quality, specific, and high intensity training, will probably need significantly more recovery (rest) than someone who does four hours of brisk walking anyway. Though additional low-intensity exercise does aid in recovery, 4 hours of it, certainly isn't needed either.
And so rest and recovery from exercise is just as important for 'health' and 'well-being' as the exercise itself is. We understand this a lot better than we used to as well, and most athletes are using some form of deliberate recovery strategy as a part of their training, everything from HRV, to program tracking, to sleep tracking, etc...etc...
Science has theoretically made physical preparation far better, but also shown the value of rest and relaxation to health and performance as well.
Lastly, I think we have a misheld belief about just what our ancestors did. Take the paleo diet for instance -- How to Eat Like a Caveman (the Real Kind).
Just like we easily assume what our ancestors ate, we assume a lot about how they moved. We assume that our ancestors moved a ton because it was a harsh environment and you had to hunt and/or gather your food, which presumably must have taken a lot of time and movement. At certain times, during certain conditions, that probably happened.
However, conservation of energy was equally important if you didn't have access to a lot of it, so we had to be choosy about what we pursued and/or gathered. There are strong indications that we have always spent a lot of time socializing. Lower intensity exercise can also be done for longer, which again ignores quality. It is unlikely that high intensity movement would have been done for any more than a few minutes per day in the past, and that the majority of movement would have been walking, carrying/moving things or light running. This would probably put it on par with the average manual labourer today. A person who exercises deliberately is significantly different.