Mix cherries, brandy and half of the caster sugar in a bowl; let soak for 1 hour.
Preheat the oven to 230 C / Gas 8. Lightly grease a 23cm pie dish.
Remove cherries from brandy mixture using a slotted spoon and transfer to the prepared dish. Pour brandy mixture into a blender; add remaining sugar, milk, flour, eggs, vanilla extract, lemon zest, salt and allspice to the blender. Pulse until mixture is smooth; pour over cherries.
Bake in the preheated oven for 5 minutes. Reduce heat to 180 C / Gas 4 and continue baking until golden and puffy, 45 to 50 minutes. Cool slightly and dust with icing sugar to serve.