TV host Maher likened Islam to the mafia, while author Harris said it was the "mother lode of bad ideas".
Hollywood actor Affleck couldn't hide his anger at their argument, describing it as "gross, it's racist, it's like saying 'you shifty Jew'".
O'Reilly, on his The O'Reilly Factor show, found himself in the unusual position of agreeing with "über liberal" Maher.
"Mr Maher is correct on the overall effect Islam is having on the world right now. The truth is militant Islam continues to drive worldwide terrorism and have plenty of sanctuaries from which to commit their evil," he said.
Then, in one of the most bizarre sign offs in the history of fair and balanced news coverage, he added: "Ben Affleck should well understand that he himself would be beheaded in a heartbeat by these Isis animals."