Smith, 49, was arrested in Charlotte, North Carolina, the same day authorities also picked up Bryon and Carl Rye, two brothers based in South Carolina, as part of the investigation.
A criminal complaint states that all three men were charged with distributing at least 1,540 pounds of marijuana since 2014, a crime that could land them in prison for five to 40 years.
A storage unit belonging to Carl Rye, 41, contained marijuana that was seized, and though the pot was legally grown in California for medical purposes, the criminal complaint said it was marked as being "unlawful to distribute."
The brothers and Smith appeared for an initial arraignment in a Columbia federal courtroom on Monday and are being held without bond.
Bryon Rye, 43, supplied the money to buy marijuana from Smith, according to the complaint, which was then given to Carl Rye to sell throughout South Carolina, whose customers included a Beaufort County drug distributor.
The arrests concluded a three-year federal Drug Enforcement Agency investigation that zeroed in on the culprits using stake-outs, video surveillance cameras, text-message records, drug sniffing dogs, vehicle electronic tracking devices, airplane records, safe deposit box records, text messages and other tactics.
More than a dozen heavily armed agents picked up the Rye brothers in a daytime raid Saturday, after forcing their way into a house in Columbia, South Carolina’s Rosewood neighborhood.
"SWAT guys were in their full gear on the edge [of the wall in front of the house]. I think they were trying to protect the scene," said Pam Moore, 49, who witnessed some of the raid as she arrived home Saturday afternoon.
She also noticed boxes outside the house that read "DEA Evidence," according to The State.
Smith was recruited by the CIA as "a special agent for the Office of Security" and spent 16 years as Navy SEAL, according to the website of a security and crisis management company he worked for previously. He is listed as a principal associate on the site, but on Monday the company clarified that Smith hasn’t worked there for more than five years.
He also reportedly received a Special Operations Medic of the Year Award along with a bronze star during a tour in Iraq.
Legal papers say that Smith landed his private airplane, a twin-engine Cessna with a 1,000-mile range, at Columbia’s L.B. Owen airport on multiple occasions. There, he would pick up cash from the Ryes.
Records show that Smith’s plane was purchased under the business name of "The Praetorian," and was supposed to be used to furnish medical aid to parts of Central America.
