Put the sponge into the bottom of a large glass bowl. Spread the raspberry jam over the sponges.
Mix the sherry and other alcohol in a glass and then sprinkle over the sponge base. Once the liquid has soaked into the sponge, add a layer of raspberries and sliced bananas.
To make the fresh custard: whisk the egg yolks, sugar and the vanilla extract until it becomes pale and creamy.
Pour 250ml milk and 150ml cream into a saucepan and heat the mixture until it starts to boil. Gradually, a little at a time; stir the hot milk into the egg yolk mixture.
Put this mixture into a new saucepan and stir continuously over a low heat. The mixture should now start to thicken. Once thick pour into a separate bowl and set aside to cool.
Once cooled pour the mixture over the layered fruit and sponge. Spread out evenly. Whip the double cream and sweeten with sugar, if you like. Spoon this mixture on top of the custard. Decorate with toasted almonds.