Anum Parvaiz, 54, and her 20-year-old daughter Shabina Begum are also believed to have died in the blaze that broke out in Wake Road, Sharrow, Sheffield.
The cousin of the 54-year-old, Ishfaq Hussain Kayani, said: “It’s just a huge tragedy. Words cannot describe how sad this is.
“It’s a tragedy beyond description.”
He said a nine-week-old baby girl was among those who had died, as well as nine-year-old Adhyan Nazim and seven-year-old Amaan Nazim.
“As soon as the news broke just after midnight, we came round. I’ve been with the family.
“It’s so sad. A couple of hours ago the whole family was leading their normal life – quite prosperous, happy.
“Just in a matter of minutes the whole family has been destroyed. The whole family has gone.
“We are a close-knit family, there are enough people around to support the family.”
He added: “They were such a lovely family, highly respected. Quite an industrious family who had everything going. This tragedy has struck and we have to try our level best to pull through this.”
South Yorkshire Fire and Rescue said the fire began shortly after midnight on the residential street, which is lined with cars on both sides.
A fire brigade spokesman said: “An investigation has commenced, involving police and fire officers, to establish the cause of the fire.”
Four fire engines were called to the scene and an aerial ladder was used to tackle the blaze.
Police and ambulance services also went to the house after receiving emergency calls.
Police, who cordoned off the road, said the damaged building was a terrace house.
A South Yorkshire Police spokesman said: “At this stage we are not able to confirm where in the house the fire broke out or if anyone else was injured.”
Police officers and firefighters remained at the scene, he said.
A spokesman for South Yorkshire Fire and Rescue said details about the fire would become clearer later today.
He said: “Our investigators are arriving here at 9am, so we will know more later on this morning.”
Local councillor Qubran Hussain, who knew the family, said he believed smoke alarms alerted people in the street who broke windows to try to get into the property.
Mr Hussain said he came down to the scene in the night.
He added: “The lady came out – the grandmother came out. But then she found all the children were inside so she went back in and she’s also died.”
Mr Hussain said he employed Mrs Begum’s husband for many years as a taxi driver in nearby Dronfield. He said he thought the children’s father, who is also a taxi driver, was out working at the time.
The whole of Wake Street and parts of neighbouring streets were condoned off today and guarded by a large number of police and community support officers. Used ambulance equipment was strewn on the pavement near the terraced house.
Source: Asian Image