Can cement dust/drywall dust burn my hands?

It isn't burned, it is abrasive action. The little fine particles, especially in the gypsum drywall, act like tiny little pieces of gl. You'll be fine. It may be a little sore and it may callous.