Mourners at the ceremony in a large mosque on Friday just north of the capital Doha included Khaled Meshaal, who is tipped to be the new Hamas leader.
Other senior Hamas officials and Qatar's Emir Sheikh Tamim bin Hamad al-Thani also attended.
He will be buried in a cemetery in the city of Lusail, north of Doha.
Haniyeh's coffin, draped in the Palestinian flag, was carried across the mosque past hundreds of people along with the coffin of his bodyguard, who was killed in the same attack in Tehran on Wednesday.
Iran and Hamas accuse Israel of carrying out the killing of Ismail Haniyeh in Tehran. (AP PHOTO)
Senior Hamas official Sami Abu Zuhri to Reuters by phone as he attended the funeral: "Our message to the occupation (Israel) today is that you are sinking deep in the mud and your end is getting closer than ever. The blood of Haniyeh will change all equations."
Haniyeh was killed by a missile that hit him directly in a state guesthouse in Tehran where he was staying, senior Hamas official Khalil Al-Hayya told a news conference, quoting witnesses who were with him.
Iran and Hamas accuse Israel of carrying out the killing and have pledged to retaliate against their foe.
Israel has not claimed responsibility for the death nor denied it.
The strike was one of several that have killed senior figures in Hamas or the Lebanese movement Hezbollah, fuelling concern that the war in Gaza between Israel and Palestinian militants is turning into a regional conflict stretching from the Red Sea to the Lebanon-Israel border and beyond.
In the United States, US President Joe Biden said Haniyeh's killing was not helpful to international efforts to secure a ceasefire in the war in Gaza, now in its 10th month.
"It doesn't help," Biden told reporters on Thursday when asked if the action ruined the chances of a truce.
Elusive military leader Mohammed Deif is among the Hamas and Hezbollah figures killed in strikes. (AP PHOTO)
Qatar has been leading the peace effort along with Egypt and the United States, Israel's main ally.
Haniyeh had been the face of Hamas's international diplomacy as war raged back in Gaza and had taken part in the indirect ceasefire talks.
He was seen by many diplomats as a moderate compared to the more hardline members of the Iran-backed group inside Gaza, although some Israeli commentators have said he was considered by some on the Israeli side as an obstacle to a deal.
Appointed to the Hamas top job in 2017, he moved between Turkey and Doha, escaping the travel curbs of the blockaded Gaza Strip.
In May, the International Criminal Court prosecutor's office requested arrest warrants for three Hamas leaders, including Haniyeh, as well as Israeli Prime Minister Benjamin Netanyahu for alleged war crimes. Israel and Palestinian leaders have dismissed the allegations.
Iran held its own funeral ceremony for Haniyeh on Thursday which was attended by his widow Amal.
"Say hello to all the martyrs of Gaza, say hello to the leaders, to all Gaza's martyrs, all the Muslims," Amal Haniyeh said as she mourned beside his coffin.
The killing of a senior Hezbollah commander crossed red lines, chief Sayyed Hassan Nasrallah says. (EPA PHOTO)
Hamas politburo member Izzat al-Rishq called on people to pray for his soul in all mosques around the world.
"Let today, Friday, be a day of overwhelming anger denouncing the assassination crime and rejecting the genocide in the Gaza Strip," he said in a statement.
While Israel has not said it carried out the killing, it has announced that an airstrike it mounted last month killed the elusive Hamas military leader Mohammed Deif in Gaza.
Hamas has not confirmed nor denied the death of Deif.
Hezbollah confirmed on Wednesday that its senior military commander Fuad Shukr had been killed in an Israeli strike on a building in Beirut.
Hezbollah vowed on Thursday a "definite" response to Shukr's killing, saying it had crossed red lines and that the decades-old rivalry between foes had entered a new phase.
"We are looking for a real response, not a performative response, and for real opportunities. A studied response," said Hezbollah chief Sayyed Hassan Nasrallah, speaking in a televised address to mark the funeral of the slain commander.
