The journalist Zsolt Bayer was one of several Hungarian citizens handed an Order of Merit of the Knight's Cross by President Janos Ader in a ceremony in Budapest on Saturday, one of Hungary's national days.
By late Monday, 40 previous recipients of the decoration including scientists, artists, and academics had posted messages on social media saying they were returning their own awards in protest.
Bayer, who is often photographed in Orban's company, is a founder of a civil group that organises massive pro-government street demonstrations. He also writes an occasional column in the right-wing Magyar Hirlap daily.
He has in the past compared the Roma people, Hungary's 600,000- to 700,000-strong largest minority group, to "animals", and written remarks seen as anti-Semitic in a country with a Jewish community estimated at over 100,000.
The head of Hungary's largest Jewish organisation Maszihisz, Andras Heisler, said he was also handing back his award given him in 2011 as he did not want to belong to the same "group of people" as Bayer.
The journalist "is a racist, an anti-Semite, who pollutes Hungary with his incandescent Gypsy-hatred and nation-destroying ideas," Heisler wrote in a Facebook message.
In 2013 the Magyar Hirlap newspaper was fined around 250,000 forints (around 800 euros, $1,000) by Hungary's media regulator over anti-Roma remarks written by Bayer.
"They are not suitable for being among people. Most are animals, and behave like animals," wrote the journalist.
Bayer's award was given for his work in setting up a body representing victims of Communism and not his column-writing, Orban's office said in a weekend statement, adding that there were no plans to withdraw the honour.