Put the potatoes and garlic in a baking dish or tin, add boiling water to cover and boil for 5 minutes. Drain well and turn into a shallow ovenproof dish. Sprinkle with the rosemary, season with salt and pepper and set aside.
Meanwhile, pare 2 of the oranges thickly, and cut the rind into strips, and place the strips into a small roasting tin. Prick the skin on the duck breasts all over and put into the tin, skin side up. Scrunch the bay leaves to release their flavour, then rub them over the duck skin. Put two leaves under each duck breast. Season with salt and pepper.
Roast in the preheated oven for 5 minutes. Remove, and spoon a little duck fat from the roasting tin over the potatoes and roll them around in the dish to coat them.
Roast the duck and the potatoes until both are well browned and cooked, about 20 minutes. Turn and baste the duck twice more. Add a little more duck fat to the potatoes, if necessary, and pour off any excess from the roasting tin.
Cut the remaining orange into wedges. Discard the bay leaves and orange peel before serving and garnish with fresh leaves. Serve with orange wedges so that the juice can be squeezed over the duck breasts.