My argument was not that the Golden Rule was useless but rather that it assumes a whole lot of cultural background. It is only in a particular kind of society with particular ideas about fairness and equity that the Golden Rule has the specific content that we take to be obvious. We cite it and think, well that's clear but it's only clear when we say this to people who already share most of these cultural assumptions.
But try and imagine how the Golden Rule would apply in the society of Attila the Hun. It's not that it wouldn't make sense to them, it would make a significantly different sense. It would make a sense that would not be acceptable to us.
That is why the appeal to what reasonable people think doesn't get us anywhere. For reasonable people can only mean here the people who already accept the notions of fairness and equity that we take to apply to the golden rule.
The cheap version of this argument, but not invalid, is to remind ourselves that there was a time (not that long ago) when most reasonable people thought segregation was just fine. It didn't clash with their ideas of the Golden Rule.
To apply the point a little less snarkily, think of why literally applying an eye for an eye etc. is not in accord with the Golden Rule. For we must believe that the Golden Rule would include compensation of some sort for damage caused by negligence. And if I believe you should pay for your negligence I also must believe that I should.
But—and see how quickly this gets really complicated—the application of the rule also includes a clause forbidding certain inhumane compensation so that even if I act negligently so as to put your eye out the compensation cannot actually be for you to do the same to me.
And if we make this allowance for the Golden Rule, then we can also do it for the pursuit of happiness.