a friend of my wife's asked her husband, 'how do you feel about having another?' and said 'you should have seen his face!'
I told my wife that my answer would have been 'how do you feel about bringing them up on your own?' which - luckily - she thought was funny.
I guess the answer is, who's doing most of the work? If it's split equally - or as equally as possible - then you should have a fair say. the kind of dad who's always working and never home can't really complain. I think from what you've said that you're pretty firmly in the former camp. I'm afraid it seems (to me, anyway) a little extreme to get a broken heart from not having four kids. They are often a joy, but incredibly hard work, too. Plus, there are already a hell of a lot of people in the world, and reources are finite (whether you believe in global warming or not).
http://househusbandconfessions.wordpress.com