The city of Malacca was conquered by the Portuguese in 1511. To suppress the attacks of the Malacca Sultanate, they immediately started the constructions of temporary camps, as well as the main stronghold called the Fortaleza de Malaca, now better known as A Famosa. The Portuguese took around five months to complete the buildings. Many of the laborers died from the scorching heat and shortage of food.

For a little more than a century afterwards, the Portuguese controlled the port city and its spice trade from A Famosa. The Dutch East India Company, desiring power in the East, sought to destroy Portuguese colonies in the early 17th century. Following a series of small incursions and skirmishes, the Dutch conquered Malacca in 1641 with the help of local allies. By this time, the invasion had caused more than 7,000 casualties.

The Dutch rule lasted for 183 years, during which the city saw relative peace thanks to the treaty between them and the Sultanate of Johor. When Napoleon's army invaded Holland in 1795, the Dutch allowed the British to take control of Malacca without any battle. The British had little interest in keeping it, however, and they destroyed the A Famosa fortress in 1807 along with most of the city.

It is partially preserved today, including the gate, sewage system and defensive walls, as well as a reconstruction of its watchtower. Its name, which means "famous" in Portuguese, is often mispronounced even by local Malaysians. The misinterpretation is that it is pronounced as /ei/ Famosa, as if the Portuguese word a is the same as the letter A in English, but its more accurate pronunciation is /a/ Famosa, almost like, "uh!"