Labor suffered bigger swings in safer seats