Me living in a 4 BR house with only our 4 family members instead of us taking in a stranger is also a bit selfish.

Everyone does some level of selfish things; trying to shape work so that you find it enjoyable (and therefore likely something that others would also find enjoyable) is an acceptable form of selfishness to most.