The actor's comments appeared to reference the fact that Hollywood is known for being predominantly left-leaning politically.
Allen plays a conservative husband and father on his show "Last Man Standing," a role believed to be based on his own life.
Show host Kimmel also asked Allen about his attendance at Trump's inauguration.
"I was invited, we did a VIP thing for the vets," Allen said, "and went to a veterans ball, so I went to go see Democrats and Republicans.
"Yeah. I went to the inauguration."
Twitter users have come out on both sides, in support of and outraged over Allen's comments.
— Lori Hendry (@Lrihendry) March 19, 2017
The Los Angeles Times recently reported that the film and television industry in Hollywood has become increasingly hostile to conservatives, with reports of secret underground organizations and "clandestine support groups" for the minority of Hollywood personalities with right-leaning views.