Why does it take so long to gain results from working out when it only takes a short amount of time to lose them?
Because evolutionary speaking it is not efficient to carry all that muscle if you don't need it. Muscle burns energy, fat stores it(simplistic explanation, I know). Anyway, your body was designed to survive in the worst situations, starvation. As soon as you tell your body that you don't "need" the muscle, i.e. stop exercising, your body very quickly cannibalizes that muscle down to your minimum requirements in order to conserve energy. Trust me, you wouldn't want it any other way, just in case you find yourself stranded somewhere without food(energy).
Want to be bigger and stronger and stay bigger and stronger then never tell your body enough is enough.