A statement from the South African presidency said Mr Mandela, 94, had been admitted just before midnight.
Mr Mandela spent 18 days in hospital in December undergoing treatment for a lung infection and gallstones.
He is widely regarded as the father of the nation for leading the struggle against apartheid.
Mr Mandela served as South Africa's first black president from 1994 to 1999. However, his health has caused concern for some time.
A presidential spokesman told the BBC that Mr Mandela was conscious and was receiving the best possible medical treatment.
"I think we need to be clear that the doctors are attending to Madiba [Mr Mandela] on a continuous basis," spokesman Mac Maharaj said.
"They prefer to act on the side of caution, and the moment they felt there was a recurrence of the lung infection, they felt that it warranted immediate hospitalisation given his age and given his history."
However, the BBC's Andrew Harding in South Africa says the abrupt nature of Mr Mandela's late-night admission is likely to raise concerns.
Appeal for prayers
The government statement said President Jacob Zuma wished Mr Mandela a speedy recovery.
"We appeal to the people of South Africa and the world to pray for our beloved Madiba and his family and to keep them in their thoughts. We have full confidence in the medical team and know that they will do everything possible to ensure recovery," President Zuma said.
The former president is often fondly referred to by his clan name, Madiba.
It is the fourth time Mr Mandela has been admitted to hospital in just over two years.
He first contracted tuberculosis in the 1980s while detained on the windswept Robben Island where he served 18 of the 27 years he was imprisoned for sabotage.
His lungs are said to have been damaged when he worked in a prison quarry.
Despite his long imprisonment, Mr Mandela forgave his former enemies and as president urged South Africans of all races to work together and seek reconciliation.
In 1993 he was awarded the Nobel Peace Prize.
The treatment he received in December 2012 was his longest spell in hospital since leaving prison in 1990.
Earlier this month he spent a night in hospital following a check-up.
Mr Mandela retired from public life in 2004 and has been rarely seen in public since.
He lives in Qunu, a small rural village in Eastern Cape province, where he says he spent the happiest days of his childhood.