The 37-year-old never clicked into gear after arriving from personal leave for the second match of the series against Australia, averaging just 6.2 from five knocks with a high score of 10.
Rohit's fate appeared sealed when coach Gautam Gambhir refused to guarantee his place in the XI on Thursday.
At the toss, it was confirmed he would become the first Indian captain to be dropped midway through an international series.
Earlier, Rohit was seen kicking a soccer ball around before play rather than warming up.
India's vice-captain and star player Jasprit Bumrah will assume the captaincy, having led the team to victory in the series opener in Perth - the tourists' only win of the series so far.
India insisted Rohit had chosen to rest from the pivotal match, which will decide whether they are still a chance to qualify for the World Test Championship final against South Africa.
"Our captain has shown his leadership and opted to rest in this game," said Bumrah.
"There is a lot of unity in this team, there is no selfishness.
"Whatever is in the team's best interest, we are looking to do that."
Shubman Gill (pic) will replace dropped skipper Rohith Sharma in the fifth Test at the SCG. (Jono Searle/AAP PHOTOS)
Shubman Gill will replace Rohit in the XI as India bat first in the fight to level the series and retain the Border-Gavaskar Trophy.Â
Gill, who has played 31 Tests, will come in at first drop as KL Rahul joins Yashasvi Jaiswal at the top of the order in Rohit's place.
Rohit faces an uphill battle to play Test cricket again given he turns 38 in April, and India will not play again until June either on tour of England or in the World Test Championship final.
After six years around India's white-ball set-up, Rohit burst onto the Test cricket scene during Sachin Tendulkar's 2013 farewell tour in the West Indies.
He produced a player-of-the match performance on debut at Eden Gardens, where he belted a stunning 177 in his first innings with the bat - the second-highest score for an Indian Test player on debut.
His recent form concerns had pre-dated the tour of Australia; he made only one score above 50 in India's memorable 3-0 home series loss to New Zealand during October and November.
Rohit managed only one double-digit score in four innings against Bangladesh during the series before that.
He has averaged 40.57 across his 67 Tests with 12 centuries to his name, but none on any of his four tours of Australia.
The axing continued a turbulent tour at the selection table for the visitors, who were forced to contend with the mid-series retirement of Ravichandran Ashwin after the drawn Gabba Test.
Rohit has not been the only big-name Indian batter to struggle for runs this series, with Virat Kohli averaging 12.4 across five innings since an unbeaten century in the series opener.
Blockbuster wicketkeeper-batter Rishabh Pant is also yet to pass 50 this series.
37°C