Stir together the chicken livers and milk in a bowl. Set aside to soak for 2 hours, then drain and discard the milk.
Melt the butter in a large frying pan over medium heat. Stir in the onion and garlic; cook and stir until the onions have softened and turned translucent, about 10 minutes. Stir in the rosemary, thyme, bay leaves and drained chicken livers. Cook and stir until the livers have firmed on the outside but are still pink on the inside, about 5 minutes. Transfer the liver mixture to a bowl and pour the wine into the frying pan. Simmer the wine until it has reduced to 1/4 of its original volume, then return the livers to the frying pan and continue cooking until no longer pink in the centre, about 5 minutes more.
Remove and discard the bay leaves. Scrape the livers into a liquidiser or food processor and add the capers, salt and pepper. Puree until smooth, then scrape the mixture into a serving dish. Cover with cling film and refrigerate at least 4 hours until cold.