London Mayor Sadiq Khan has said he told Prime Minister Boris Johnson “forcibly” that construction workers should not be going into work.
Mr Khan said he advised Mr Johnson at a Cobra meeting on Monday that quite a lot of construction is not “critical or essential”.
The PM placed the UK on lockdown on Monday evening.
But Housing, Communities and Local Government Secretary Robert Jenrick said later that work on building sites can continue, though workers should practise social distancing.
He told ITV’s Good Morning Britain: “My view is no, and I made that point quite forcibly at yesterday’s Cobra, I made that point quite clearly to the Prime Minister.
“According to the Government’s advice the answer is yes.”
British Safety Council chairman Lawrence Waterman said on Tuesday that all non-essential construction work should be stopped so that workers can stay safe.
He added: “The construction sector needs clarity from the Government – on most sites social distancing will be impossible or simply unsafe.
“All non-essential construction should end now so that construction workers can go home and stay home like everyone else.”
While some companies said they were closing sites down and suspending work, others said they would remain open with “strict precautions” in place.
Transport for London (TfL) announced on Tuesday that work on its Crossrail sites was being temporarily suspended but that essential maintenance of the transport network will continue.
Aylesbury Tory MP Rob Buter said he had told ministers that work on HS2 should be stopped after contractors in his area were failing to comply with social distancing.
He tweeted: “Following shocking reports of @HS2ltd contractors failing to comply with social distancing & even coughing over local people, I’ve told the #HS2 minister that work must stop now, to protect the health of my constituents. Yesterday I wrote to the CEO calling for an immediate halt.”
HS2 said its “joint venture partners” are responsible for their sites and the health, safety and well-being of their workers.
A spokesman added: “We are working closely with our joint venture partners, who are responsible for their sites, the health, safety and well-being of their staff and the impact of their work on the adjacent community.
“Over the course of this week and into next, we will be reviewing the majority of works on our construction sites in-line with government advice on dealing with Covid-19.”
House-builder Taylor Wimpey also said it has closed its construction sites, show homes and sale sites.
Construction workers told the PA news agency that, although they are scared of catching the virus and taking it home to their families, they are reporting for work because otherwise they will not get paid and risk losing their jobs.
A crane operator at a London construction site with 400 to 500 workers said: “Everyone on site at the minute feels angry and unprotected.”
He said staff have been given no guidance on what was happening other than “just carry on until told otherwise”.
The man, who asked not to be named, said if he opted not to go in he “wouldn’t get paid or even could lose my job”.
“(It’s) scary really – I’ve got a family and kids at home. I’m in London around all these people, bringing it all back into an isolated home.”
He added: “It’s not going to change until support is offered to workers who are registered as self-employed.
“They have no incentive to stay at home, they have bills to pay.
“Everyone on site at the minute feels angry and unprotected.
“But they still have bills to pay, they still have rent to pay, they still have to buy food.
“Construction sites will stay open until the Government issues an order to close.
“But the Government cannot issue the order to close until it offers support to the one million-plus construction workers – it has to go hand in hand.”