They will not be paid for their work. Instead the men, who come from the persecuted Rohingya ethnic group, have been coerced into erecting the 230km long fence by the threat of violence against their families.
The Rohingyas are a distinct ethnic group from Myanmar's Rakhine State. The authorities in Yangon have refused to recognise them as citizens and they have been persecuted for their cultural difference and practice of Islam.
For many, life in Myanmar has become so difficult that they have fled across the border to Bangladesh. Over the past year 12,000 Rohingyas have been caught crossing the border illegally.
Now they are being forced to build a fence to prevent such escapes.
"The Myanmar army have forced all of the men living in the villages on the border to work on the fence," a worker involved in the construction says. "Most of them are Rohingyas. If we don't do as they say they beat us and our families."
So far they have fenced off 70km of border in what experts believe is an attempt by Yangon to increase control of the lucrative smuggling trade that flourishes in the area.
"Illegal trade between Myanmar and Bangladesh has formerly been in favour of Bangladesh, but this will change now,"explains Professor Imtiaz Ahmed, from Dhaka University. "The country that controls the barriers between borders can also assert greater control over the illegal trade."
Bangladesh and Myanmar have never agreed on their borders, and an ongoing dispute over where their maritime frontiers lie has seen tension rise along the Naf river.
The contested maritime border involves a patch of sea believed to contain valuable oil and gas. Control of these waters could make either country very rich, and experts say that diplomatic relations between the two countries has deteriorated as a result of the dispute.
"The tension was heightened last November when the Myanmar Navy came in to put a rig in what Bangladesh claims, rightly, to be our own territorial water," says Retired Major General ANM Muniruzzaman, from the Bangladesh Institute of Peace and Security Studies.
"Eventually the Bangladeshi diplomatic efforts diffused the situation, and the Myanmar navy rig went back, but the Myanmar government has consistently told Bangladesh that this is their water, and that they will come back. When that happens, perhaps the Myanmar government wants to put a dual pressure on Bangladesh, not only from the sea but also from the land border."
That process may have already started. Myanmar has deployed 50,000 men to the border with Bangladesh, and in the past month alone, Dhaka has responded by sending an additional 3000 troops to the area in a manoeuvre codenamed "Operation Fortress."
Officially, the Bangladeshi government denies there is tension along the border. The troops say they are there to monitor and stop the illegal trafficking of goods and people.
But the soldiers know that relations between the two countries are strained.
"We have a border through which we can observe the other side of the river. Our troops morale is very high, under any circumstances we are ready to protect the integrity and sovereignty of our country," says Lieutenant Colonel Mozammel, commanding officer of Border Guards Bangladesh in Teknaf.
|Many Rohingyas have fled to Bangladesh|
Malika is one of those who crossed the Naf river illegally. Her feet are swollen from the three-day walk to escape Yangon's soldiers.
She says she suffered horrific abuse there and had no choice but to leave.
"I couldn't stay there, the soldiers raped me over and over again," she says. "The Myanmar army do not consider us as humans."
But once in Bangladesh, the refugees face new problems. Of more than 400,000 Rohingyas believed to have slipped across the border into Bangladesh, just 26,000 have been offically recognised as refugees by the Bangladeshi government and the United Nations.
The authorities refuse to feed and house the rest.
Even the handful of NGOs working here are not allowed to provide food or medical aid or education facilities to unregistered Rohingyas because the government fears that this would spark tensions between poor local villagers and the new arrivals.
Fadlullah Wilmot, the director of Muslim Aid in Bangladesh, explains: "More than 44 per cent of the population in this area are ultra poor, that means that their daily income only provides their basic food needs. The literacy rate is about 10 per cent. The wage rate is low, so of course there are tensions."
In 1992, the Bangladeshi government, under the supervision of UNHCR, organised the forced repatration of 250,000 Rohingyas on the basis that the refugees would be given citizenship by the Myanmar authorities. That promise was never kept.
Professor Ahmad believes the refugees are trapped between a rock and a hard place.
"Myanmar's position is they do not recognise them as citizens, they are stateless within Myanmar, and they are also stateless when they come to Bangladesh," he says.
"No-one wants us. This is humiliating."
Rohingya refugee in Bangladesh
Trapped in limbo between two countries that don't want them, the Rohingyas have become a bargaining chip for both Bangladesh and Myanmar as they try to settle their border dispute.
In Bangladesh's refugee camps, frustration and anger are rife amongst the beleagured minority.
"We cannot work. Our children can't go to school. Our wives aren't allowed to see doctors," one man says. "We cannot receive any food aid. No one wants us. This is humiliating, we have no arms, but we are ready to fight and to blow ourselves up. People need to know that we exist."