Hi knittenkitten, Welcome!
I think that the body only takes what it needs. When you work all week you may require extra hours of sleep. It depends on the person and what type of work you do. I find mental work more tiring than a little physical work.

Sitting still, writing my checks, and doing the bills, is more exhausting than scrubbing the bathroom. But that is me. Too much physical activity and I am overdoing it. I ration my day.

If you not sleeping in bed those extra hours because you are depressed, than consider yourself lucky, some people have trouble sleeping at night, working or not.

I sleep well too. I think your DH is right.