Gunmen from a royalist Filipino sect ambushed and killed five Malaysian policeman on Saturday night, officials said today.
Malaysian officials fear that armed insurgents who have 19th-century claims on the land have now infiltrated three coastal districts on Borneo.
Two attackers were also shot, escalating tensions in Sabah state, where about 200 members of a clan led by a brother of Sultan Jamalul Kiram III of Philippine province Sulu occupied a village last month.
Security forces clashed with the clan members in coastal Lahad Datu on Friday, leaving 12 Filipinos and two Malaysian police dead.
The remaining clan members have refused to budge, while concerns have grown that other groups from the Philippines might enter Sabah.
The police team was attacked while inspecting a settlement in Semporna, more than 90 miles from Lahad Datu.
Police are also investigating sightings of armed foreigners in a third seaside district.
Foreign Minister Alistair Burt's admission that the Cameron government has "supported" a survey of attitudes to US drone strikes in Pakistan's tribal areas amounts to a tacit admission of British involvement.