Leaving the single market at the turn of the year has damaged our trade