Presumably Starmer, and others, consider that this decision can be seen as antisemitic since it is the fans of the Israeli club who are being restricted when, possibly, if there was any trouble, these fans would be on the receiving end and not be the perpetrators - though they may be if provoked. Then there is the possibility of protests separate from those attending the match, and disturbance to the local residents.
Rocks and hard places come to mind.