The Vatican said in a statement that "fragments of human bone" were discovered during restoration work on property near the diplomatic office of the Holy See in Rome and that an investigation is already under way.
The statement added that the probe is being led by chief prosecutor Giuseppe Pignatone, who ordered forensic specialists to determine the "age, sex and date of death" of the body.
Also, the investigators will reportedly try to find out whether the remains are a DNA match for teenagers Emanuela Orlandi or Mirella Gregori, who went missing separately in Rome in 1983.
The Orlandi case is believed to be one of the biggest mysteries in modern Italian history, which investigators believe could be connected to Gregori's disappearance.
Speculation is rife that 15-year-old Orlandi was kidnapped to prod the authorities to release Mehmet Ali Agca, the Turk who served a life sentence in an Italian jail for attempting to assassinate Pope John Paul II in 1981. Orlandi was last seen on June 22, 1983, when she was leaving a music class.
As far as Gregori is concerned, the 15-year-old disappeared shortly after she told her mother that she was going out to speak with her school friend but never returned. The incident took place exactly 40 days before Orlandi's disappearance.