We just updated the servers so that matches start if no more players join after 3,5 minutes of waiting. This will sometimes mean smaller games but we hope this change is for the better.
Please leave feedback on this change on the forums as we cannot answer here.