They say you never forget your first time.

Mine was during my Queensland high school years way back in the 1980s, when the local newspaper published a letter in which I pointed out that the author of a previous letter denouncing the commercialisation of Christmas by our culture’s “traditional enemies” was a well-known supporter of the anti-Semitic pseudo-historian David Irving. The following week, some anonymous person posted me a newsletter from the notorious far-right Australian League of Rights.

I try to remind myself of the sense of shock at receiving that hate-filled envelope when listening to shaken young Muslim women describe their own first experience of being targeted by racists for daring to participate in public discourse. This type of harassment has become so normalised that the ugly scenes of Sam Dastyari surrounded by mocking thugs from the United Patriots Front (or as they pedantically corrected Dastyari when he referred to them by that name, “Patriot Blue”) would hardly come as a surprise, had the so-called patriots not been so proud of their work that they filmed it and posted it on social media. Muslims from the beer-drinking Dastyari to the headscarf-wearing Yassmin Abdel-Magied have been forced to run the same gauntlet of harassment and abuse. My own book launch last year was interrupted by the ranting racist at the back of the room who (in Helen Razer’s words) “began with ‘How can Mrs Hussein possibly justify …’ and ended, about twenty years later, with … ‘the religion of Satan!!!'”

Dastyari responded to the bullies in the pub by calling them racist rednecks. I told the Ranting Racist in Readings that I am not Mrs Hussein to anyone, that my friends call me Shakira but since he clearly didn’t want to be my friend, he could call me Dr Hussein. But coming up with the perfect comeback line on the spot is less important than the responses that come after the event.

As you would hope and expect, both Malcolm Turnbull and Bill Shorten have denounced the attempt to bully Dastyari. As you would also expect, Pauline Hanson declined to follow their lead, accurately describing Dastyari as “a smartarse” and less-accurately implying that the whole incident was probably just a publicity stunt by Dastyari himself. So far, so completely to script.

It’s very easy for anyone-who-isn’t-Pauline-Hanson to denounce bullies like Patriot Blue.  It is more complicated to work out how to respond to their stunts without amplifying their voices. One of the most commonplace responses is to explain exactly how the racists are mistaken — as though, in Ghassan Hage’s words “the racists’ greatest sin is that they are bad thinkers” — thinking that anti-racists then scramble to correct. As if in anticipation of the Patriots’ demand to know “what race is Islam?”, on Tuesday New Matilda published an article by Michael Brull which was described as “destroying” the “but Islam’s not a race” comeback. Victorian MP Tim Watts’ straight-into-the-camera “What race is dickhead?” response to Dastyari’s bullies was far more to the point than was Brull’s point-by-point unpicking.

I’ve done more than my share of fact-checking racism myself, on everything from female genital mutilation to headscarves to halal-certified food to yes, that familiar favourite, “Islam is not a race”. Michael Brull has not “destroyed” that particular axiom any more than I did. It is the cockroach of political memes, destined to survive the apocalypse.

Let’s not respond to stunts from the so-called patriots by fact-checking their racism, then. And more importantly, let’s not allow them to distract us from the much more powerful racist bullies who are responsible for the closure of remote Aboriginal communities and the internment of asylum-seekers in offshore detention centres in violation of international law although not (it is becoming increasingly clear) in violation of the legendary and much-lauded “Australian values”. “Ordinary Australians” might be shaken by the patriots’ ugly stunts, but they also use it to console themselves that this is what real racism looks like. It doesn’t look like their local MP (although it may look a bit like their One Nation Senator), it doesn’t look like the mainstream media that they consume every day and above all, it doesn’t look like them.

Except it does.