Rogozin began his work at the organization's HQ in Brussels on Monday. He gave Scheffer an official letter and during a meeting an inlaid war tomahawk, especially made in Russia.
Rogozin said he hoped that the NATO secretary general would bury the hatchet as a sign of "putting an end to all conflicts between Russia and NATO."
"He is holding the tomahawk and now we have to find a spade to bury this hatchet as deep as possible in the ground," Rogozin said.
Russia and the Western military bloc have scaled down military cooperation, but still conduct anti-terrorism patrols in the Mediterranean, exchange intelligence data and information on each other's air movements, and cooperate in the missile defense sphere and in fighting drug trafficking from Afghanistan.
The Russian official also said that his meeting with Scheffer lasted an hour instead of the planned 30 minutes.
Last Sunday he reiterated that, "Russia and NATO should not exchange mutually unacceptable conditions, but focus on progress in areas that ensure positive results in military and non-military cooperation."
He added that NATO had issues that could not be solved without Russia's participation, in particular the operation against Taliban forces in Afghanistan.
"Our main goal is to respect each other's interests and make the world more predictable and secure," the Russian diplomat said.
Rogozin, 44, who was appointed to the post by President Vladimir Putin on January 9, will attend a session of the Russia-NATO Council on January 30.