Michael Daley has conceded his presence could damage Labor's chances at the federal election and has announced he will not re-contest the party's NSW leadership a day after vowing to run for the job.
Mr Daley said he didn't want to be a distraction to the federal Labor campaign and wanted the NSW party to have clear air to select a new leader following Saturday's election loss.
"I have always put the interests of the party first and in making this decision, continue to do so," Mr Daley said in a statement on Tuesday.
Mr Daley on Monday stood aside as leader following the party's poor showing in the election and his disastrous final week of campaigning, but said he would contest a leadership ballot after the federal poll due in May.
A video from September emerged last week of him telling a pub forum that Asian migrants were taking local jobs and then he stumbled on key numbers behind his education policies during a live television debate.
Federal Labor MP and shadow treasurer Chris Bowen has denied claims he wanted Mr Daley removed to protect opposition leader Bill Shorten.
AAP understands Mr Bowen lent on party powerbrokers to push NSW General Secretary Kaila Murnain to oust Mr Daley.
Mr Daley was Ms Murnain's preferred candidate for state Labor leader and she wanted him to stay on.
Mr Shorten denied he pressured Mr Daley into stepping down but says it was the right thing to do.
"He's the one that made the final decision. I did make it clear, though, that those comments (about Asian migrant) shouldn't have been said," he said earlier on Tuesday.
Mr Daley defended his record on Monday, saying he had "picked up the pieces that the party was in" when he took over in November following the resignation of Luke Foley.
The Labor leader insisted he had support and that he'd had positive conversations with colleagues, leading him to recontest the leadership.
Frontbenchers Chris Minns and Jodi McKay will likely challenge for the leadership. Interim leader Penny Sharpe said she would not run.
Australian Associated Press