Yunus Bek Yevkurov, deputy head of the Volga-Ural Military District headquarters, has been appointed as acting president.
Zyazikov, 51, a former KGB officer and a close ally of Prime Minister Vladimir Putin, has served as president of the troubled region, which neighbors with Chechnya, since 2002.
He has come under pressure in recent months over his failure to curb the surge in militant activity in the province. Human rights groups have also accused federal authorities of torture and abductions in the region.
In March this year, Zyazikov dissolved his government and called for major economic and social reforms.
In August, Ingushetia gained international media attention over the death of Magomed Yevloyev, a prominent journalist who had been involved in an argument during a plane flight with Zyazikov. Yevloyev, who ran a banned website that had called for protests against the local government, was shot in the head while in a police vehicle, and died in hospital on August 31.
Commenting on Zyazikov's dismissal, a member of Russia's upper house of parliament, Issa Kostoyev, said it is unlikely that a new Ingush president could solve the "huge range of problems" in the republic.
"The current problems can be only solved by the federal center, and until this is done no one in Ingushetia will have quiet, normal jobs and stability... The republic's leadership can be changed ad infinitum, but the new leader will not inherit individuals, but existing problems."
The Ingush opposition has voiced hope that Yevkurov can improve the situation, and pledged support for the acting president.
