The artwork, which is close to the port town's ferry terminal in Kent, shows a workman on a ladder removing one of the bright yellow stars.
The guerrilla artist from Bristol is known for his controversial pieces, one of which had been painted outside a youth club in the UK and sold for US$405,000.
Representatives for the elusive artist confirmed to the Press Association that the piece had been done by him. Banksy also posted images of the work on his verified Instagram account.
The artwork which has been painted in a town that acts as one of Britain's main gateways to the European continent comes as the UK approaches a general election in June, as well as on-going Brexit negotiations.
The piece is also just a short crossing from Calais, where Banksy sent most of his exhibitions from the "Dismaland" art project in Weston-super-Mare, Somerset, to be used as shelter for migrants in the notorious Jungle camp.
Sources say that Banksy has left an enigmatic comment on Brexit with this new piece.
Banksy is well known for artwork which takes on current affairs issues. One of his more recent works protested the use of tear gas against refugees in Calais while others have addressed surveillance and policing in the UK.