A map of voting in the Swiss cantons showed wide discrepancies in opinions in different parts of the country.
In French-speaking Vaud, in the west of Switzerland, 73 percent of voters supported dumping nuclear power, but in German-speaking Glarus and Schwyz around 56 percent of the electorate wanted to keep it.
Switzerland has five ageing nuclear power stations, which provide 38 percent of the country's energy needs.
Two are at Beznau and the others are at Muhleberg, Gosgen and Leibstadt, all in the north of the country.
The tsunami in Japan in 2011, which led to a major emergency at the Fukushima nuclear power station, signaled the beginning of the end for nuclear power in Switzerland.
In June 2011, the Swiss Parliament resolved to start phasing out nuclear power in 2034. But pressure, led by the Green Party, has grown for an earlier switch to renewables.
As a result of the May 21 referendum, Switzerland will begin phasing out nuclear power in 2019.
With more than a third of the country's power currently coming from nuclear, it will represent a major challenge for the electricity companies to find new sources in such a short time frame.
But the president of Switzerland's Green Party, Regula Rytz, said the vote was a "moment of historic change."
"The Swiss population has said no to the construction of new nuclear power plants and yes to the development of renewable energy. The conditions have also been set whereby the economy and households will need to take responsibility for the future. It's absolutely magnificent," said Rytz.
Energy Minister Doris Leuthard had been urging voters to back the switch to renewables despite the huge cost, which opponents had been highlighting.
"After six years of debate in parliament and at committee level, a new chapter in Switzerland's energy policy can begin," said Leuthard at a news conference. "But there is still a lot of work to do."
The World Nuclear Association says radioactive waste is mostly handled by Zwilag, a company owned by the four Swiss nuclear companies.
Zwilag began using a central interim dry cask storage facility for high-level wastes at Wurenlingen in 2001.
Its Alpine topography and high levels of rainfall makes Switzerland ideal for hydroelectric power generation and it saw a boom between 1945 and 1970, but they remain loss-making enterprises.
Dr. Michael Bluck, director of the Center for Nuclear Engineering at Imperial College in London, said there was a degree of "horses for courses" when it came to individual countries' decisions whether to carry on with a nuclear industry.
"In Switzerland's case 60 percent of its energy comes from hydroelectric power, which is a great thing to have. Their geography already lends itself to that," Dr. Bluck told Sputnik.
He said Britain, for example, had negligible hydroelectric power — Dinorwig in Wales is the only pumped storage hydroelectric scheme in the UK — and therefore had decided to build a new nuclear power station at Hinkley C.
Dr. Bluck said Switzerland's decision "did not help the narrative" of the nuclear industry but it would be wrong to start writing its obituary.
"Nuclear is expensive and when governments make a decision it is often on the basis of cost. If nuclear came down in price — for example with the construction of small modular reactors — that would make a difference," he told Sputnik.
Dr. Bluck said it was ironic that Switzerland and neighboring Germany, which has also decided to phase out nuclear, would probably end up importing electricity from France, which has the largest nuclear power industry in Europe.
Felix Muri, a Swiss politician who had campaigned against shutting down nuclear power prematurely, said he believed the real cost of the switch had been kept from voters and would end up hitting consumers in their pockets.
Switzerland will join Austria, Italy, Portugal, Ireland, Poland and Greece, none of whom produce nuclear power.
Last year, the British government controversially gave the go-ahead for a new nuclear power station at Hinkley in Somerset despite security concerns and fundamental design problems.