The House of Lords has rejected a Labour proposal to save the Agricultural Wages Board in a decision which could mean a loss of millions in pay across the sector.
The Agricultural Wages Board sets minimum rates of pay and conditions for farm workers across England and Wales.
Peers voted by 192 to 163 against the Labour plan on Wednesday night.
The government claims abolishing the board will simplify employment laws for around 40,000 farm businesses but opponents of the plan argue that it would have a devastating affect.
Labour spokesman Lord Whitty said the government had estimated that by getting rid of it wages in the agriculture sector would be cut by a total of £240 million.
He said the plan was "dangerous."
Mr Whitty added: "The minister claims this is a great removal of burdens on small businesses, but actually the operation of the board has in many cases been of great benefit to small farmers," he said.
He added: "A wage cut for workers will almost certainly end up being a benefit for the supermarkets.
"The supermarket buyers once they hear that the wages board and the minimum rates have been abolished will go back to their farmers and their suppliers and say 'we want a cut in the prices that we are giving'."
The Bishop of Hereford, the Right Reverend Anthony Priddis, said that when other wages councils were abolished in the 1980s, the government chose to keep the Agriculture Wages Board to prevent wages being slashed.
He said: "In order to consolidate and build upon the progress achieved in terms and conditions during the last 30 years we need to retain and further develop and update the Agriculture Wages Board, not abolish it."