Not OP, but for me, it's the isolation. I wouldn't quite call it miserable unless I can't go out. I lasted only a few days into WFH for COVID before I went to live with my parents until I was forced back to the office. Just a few weeks ago I was snowed in for a week and I was about to crawl up the walls.
If I were single, I can see how that would make a difference. When I was forced to both work from home and not go to the gym and hang out with friends after work and I was single a decade ago during bad weather, that was miserable.
I'm single and I love WFH and I avoid the office as much as possible, coming less than required. Especially now with the stupid hotdesking. I actually have more time to socialize now <3