Preheat the oven to 180 C / Gas 4. Grease and flour a cupcake tin, or line with paper cases.
Stir together the cocoa and water to form a paste. Set aside.
In a medium bowl, cream together the sugar and butter. Beat in the eggs, one at a time, then stir in the vanilla.
Combine flour and baking powder, add to the creamed mixture and mix well. Gradually stir in the milk until the mixture is smooth.
Spoon half of the cupcake mixture into a second bowl. Stir in the cocoa mixture till well combined. Spoon mixture into the prepared tin, filling cases 3/4 full. Repeat with the vanilla cupcake mixture.
Bake cupcakes for 20 to 25 minutes. Cupcakes are done when they spring back to the touch. Let cool completely before topping with your favourite icing.