What is Internet like in Nepal or Madagascar hostels/cafes?
Nepal: Let's start with some obvious facts. Speed of just 256 Kbps (mind it!) is defined as broadband in Nepal. The cheapest and most affordable Internet is state-owned telco's ADSL which often sucks at speed and service (except for those living near telecom exchange offices).
According to the government records, ADSL is quite popular among households and services (161,133 subscribers) that require just always-on Internet - and as mentioned in your question 5 to 10 USD hoteliers fall under those okay hotels. Except YouTube cache and CDN providers including Akamai's services you should expect sluggish Internet with almost omnipresent ADSL.
Is the internet full access internet though?, is it very spotty or slow?
You should not be expecting high speed Internet in Nepal's (5-10USD) hotel. And now with as much as 14 hours of power cut, you never know Internet router will have power back up.
Or is it like NZ where you have to pay for every 500mb or gig in the hostels.
Not at all. Most of those (5-10USD) hotels provide free Internet. However, you can purchase data package ranging from MB to GB volume with decent cellular 3G speed if you really need fast and portable (limited to within major cities) Internet. Probably, cellular data is much less than what you pay for in NZ.
In some medium level hotel and cafe of major cities of Nepal ( like Kathmandu or Pokhara) you can expect net speed of about 2mbps.
In low level hotel you may not have internet facility, if you have it will be too poor.
For star hotel and all, I don't have much idea. You can always consult their information board online before you book the room.
Kathmandu, Birgunj , Biratnagar or Pokhara and other towns have mostly internet access internet facility with good speed in hotel and cafes.