The man, a Bosnian national, was detained in Tulln, some 25km (15 miles) west of the Austrian capital.
His arrest comes after a 42-year-old Bosnian national was arrested on Monday after having tried to enter the U.S. embassy with a rucksack containing two grenades and nails. The would-be bomber was also reported to have been in possession of a book with Islam in the title, as well as a compact disc, the contents of which are as yet unknown.
The plot failed after the bag set off an embassy metal detector.
The Bosnian subsequently attempted to flee the scene, throwing the bag away in a nearby street, but was apprehended by police after a short chase.
Police said the motive for the attempted bombing was not immediately clear, as the man spoke very poor German and made confused statements.
Explosives experts have yet to determine whether or not the explosive device could have been successfully detonated.
The recent incident came almost a month after three men were taken into custody in neighboring Germany on suspicion of preparing terrorist acts on a range of targets, including Frankfurt airport and a U.S. Air Force base in Ramstein.
Also last month, Austrian police arrested three Austrian citizens of Arabic origin in connection with the posting of an online video that threatened Austria and Germany with attacks over the countries' involvement in Afghanistan. One of the men was later released due to a lack of evidence.