Abdul Ali Mazari was born 1946 in the village of Charkint, south of the city of Mizar-i-Shrief, in northern Afghanistan, into a Hazara family. That is the reason he used surname Mazari.

He studied theology in Qom, Iran. There he got involved in the mujahideen resistance movement against the Russians in Afghanistan. He was one of the leading figures in unifying the Hazara resistance parties into one unified party, Hizb-i-wahdat (Unity Party). This party played a leading and positive role in voicing and fighting for the oppressed Hazara people of Afghanistan. During the civil war it held all the Hazara-populated areas in Afghanistan. He was highly respected figure among the Hazaras, for he had given them a voice and pride, he was affectionately called Baba (father) Mazari and Ustad (teacher) Mazari.

During the siege of Kabul, the Talibans lured him for negotiations and then brutally killed him. His funeral procession march on foot from Kabul to Bamiyan and then Mazar-i-Sharif, where he was buried.