Sabah police chief Hamza Taib said the group, armed with guns and machetes, was arrested about 12 nautical miles from the state’s east coast, less than an hour away by boat from the southern Philippines.
The group’s wooden motor boat was towed to Sandakan on the east coast of Sabah on Borneo island but no details were given of where the 32 are being held.
More than 200 armed Islamist followers of the self-styled “Sultan of Sulu” landed in February to claim Sabah for their leader, reviving a centuries-old territorial row.
The incursion and a Malaysian counter-assault in early March left at least 68 militants dead along with 10 security personnel, according to authorities, and strained relations with Manila.
But authorities appear to have failed to catch most of the militants, leading to a prolonged security crisis which has forced Malaysia to relocate villages on the coast considered vulnerable to infiltration.
Philippine media earlier Monday had cited Muslim rebels as saying that 1,000 gunmen were trying to break through a naval blockade to support the fight against Malaysian security forces.