"The Rwanda scheme was dead and buried before it started," Starmer said in his first press conference on Saturday.
"It's never acted as a deterrent. Almost the opposite."
No asylum seeker has been deported under the scheme, described by critics as an "Alice in Wonderland adventure that was both absurd and inhumane".
Scores of people, reportedly as many as 200, who were due to be deported have recently been released on bail by immigration judges because there was no longer a realistic prospect of removal within a reasonable timescale.
Sir Keir has said he will curb Channel crossings by hiring specialist investigators and using counter-terror powers to "smash the criminal gangs" behind the flow of migrants into the UK, but how this will work in practice remains largely unknown.
The financial implications of walking away from the deal and the total bill to the taxpayer are not yet known.
Earlier this year Rwanda's President Paul Kagame suggested British taxpayers' money could be repaid if the deal failed, though his view of the change in UK leadership is uncertain.
Yolande Makolo, a spokeswoman for his administration, later said the country had "no obligation" to return any of the funds but if the UK requested a refund "we will consider this".
Starmer told reporters in a wood-panelled room at 10 Downing St that he was "restless for change," but would not commit to how soon Britons would feel improvements in their standards of living or public services.
His Labour Party delivered the biggest blow to the Conservatives in their two-century history on Friday in a landslide victory on a platform of change.
His new government takes on the massive challenge of fixing a heap of domestic woes and winning over a public weary from years of austerity, political chaos and a battered economy.
Prime Minister — UK Prime Minister (@10DowningStreet) @Keir_Starmer chaired his first Cabinet this morning.He outlined the government's mission to restore Britain to the service of working people. pic.twitter.com/M7d8e9k8DzJuly 6, 2024
"We have a huge amount of work to do, so now we get on with our work," Starmer said.
He said it had been the honour of his life to be asked by King Charles III to form a government in a ceremony that officially elevated him to prime minister.
Among a raft of problems they face are boosting a sluggish economy, fixing a broken health care system, and restoring trust in government.
"Just because Labour won a big landslide doesn't mean all the problems that the Conservative government has faced has gone away," said Tim Bale, professor of politics at Queen Mary University of London.
In his first remarks as prime minister Friday after the "kissing of hands" ceremony with Charles at Buckingham Palace, Starmer said he would get to work immediately, though he cautioned it would take some time to show results.,
"Changing a country is not like flicking a switch," he said as enthusiastic supporters cheered him outside his new official residence at 10 Downing.
"This will take a while. But have no doubt that the work of change begins — immediately."
He will have a busy schedule following the six-week campaign, heading out Sunday to visit each of the four nations of the UK — England, Scotland, Wales and Northern Ireland -- that he said had each voted in favour of Labour.
He will then travel to Washington for a NATO meeting Tuesday and will host the European Political Community summit July 18, the day after the state opening of Parliament and the King's Speech, which sets out the new government's agenda.
with PA
