The New York Times has an article entitled Single and Paying for It (via Steve). It's framed in terms of the gay marriage debate, but much of it is more generally applicable.
Though most people would agree that these distinctions are arbitrary and unfair, as a society we tend not to notice that breaks for people who are married translate into penalties for those of us who are not.
Whenever you're arguing for benefits on one side, remember that it's often essentially the same as arguing for pentalties on the other. I made a similar point regarding affirmative action.
Of course, I could never support gay marriage. But I don't think there should be a constitutional amendment to define marriage as the union between a man and a woman. Our constitution is far more concerned with process than it is about social issues, and such an amendment would violate that spirit.
Rather, I think the government should get out of the marriage business altogether. Penalties or bonuses for singles or couples will always be arbitrary. Plus, the fact that you need a license from the state to get married seems strange to me. I don't know why the state needs to be concerned with whether its citizens are married or not. It seems it should only know about our names, where we live, and who our assets should go to in the event of death.
As for other things that depend on the definition of marriage, such as insurance benefits, it seems that in the absence of any government mandate one way or the other, insurance companies would be free to decide what to do. They'd be subject to market forces, which may result in insurance companies generally deciding in a way I wouldn't like, but I'd much prefer that to the government mandating it. Similar arguments can be made for how to handle things like splitting assets in the case of divorce. I'd be interested to hear if anyone could present issues I haven't thought of.
On a more philosophical note, I question people's motivations regarding marriage. As a Christian, I know why marriage has meaning, and who it's limited to. When I put on my atheist hat, I question why people consider marriage significant at all, and any definition of "marriage" simply becomes arbitrary. In the absence of God, the ephemeral certainty of an "oath" and the uncertain nature of "love" would make me question what marriage even is. Greg Bahsnen used to say (paraphrasing) "I don't know why atheists get married", or similarly, have funerals. If you take offense to that, you're missing his philosophical point.
++