It's a nonsense expression, more commonly "heavens to Betsy" or "heavens to Murgatroyd". This version gets only one hit, which is the example you're quoting. They've chosen "handkerchiefs" for the rhythm and alliteration.

So you're right in assuming it's like "my goodness".