I see this happen often when a happy hour ends. Also, say on
pokerstars, where there are occasionally bonus
hands being played and a lot of people join a lot of tables as the time approaches then leave when the hand happens. As other have said, some people just like to play at full tables and when 2 or 3 people leave, they don't want to keep playing there. These all happen in the normal course, and are not suspicious.
That said, I have also seen teams of point colluders on PS leave at once.
If you come across something like this, check to see whether any of them have moved to another table or if they all left the site. If they are all suddenly gone, send the info off to the site and they can see if there's anything to your suspicions. The site can check whether the players are always on together and playing together. This could suggest a multiple-accounter as well.
There are innocent reasons this could happen and not-so-innocent.