Labour voters are more likely to stick with their party