As fighting continued, rebels raised their flag inside the town and could be seen controlling a large chunk of it.
Libyan rebel fighters fired their rifles and automatic weapons into the air to celebrate their capture of large areas of of Zawiya, about 50 kilometers west of the capital, Tripoli, AFP reports.
Rebels could be seen on video raising two pre-Qaddafi-era flags over a mosque inside the city to mark their conquest.
An al-Jazeera TV correspondent inside Zawiya said forces loyal to embattled leader Moammar Gadhafi control the northern half of the city and are sporadically lobbing mortar shells at rebel-held portions of the city.
A group of rebel fighters cheered their advance into Zawiya and told journalists that a recent string of victories over government forces should help propel them to the capital Tripoli in the near future.
"Thanks to God we have now liberated Niyad in the Western (Nafousa) Mountains, as well as Sabha, and now Zawiya, and God willing tomorrow we will be in the capital, Tripoli," one of the rebels told a BBC camera crew.
The rebels are also claiming to have made advances inside the eastern oil port city of Brega, taking several neighborhoods. Arab satellite channels showed rebels controlling parts of Brega, but it could not be independently confirmed how much of the town they hold.
A rebel victory in Zawiya would represent a significant blow for the government as Zawiya sits on the strategic coastal highway between the Tripoli and the Tunisian border.
That highway has been a major supply line for Tripoli in recent weeks, providing food and fuel for the capital.
Zawiya was one of the first towns to rebel against Qaddafi in the early weeks of the rebellion, which began in February. Qaddafi forces retook Zawiya in early March, after weeks of a bloody siege that resulted in hundreds of casualties.
At the same time, unconfirmed reports have said that Qaddafi is negotiating with the rebels .