Add warmed milk, sugar and yeast to the bowl of a stand mixer. Let stand until yeast starts to foam, about 10 minutes. Attach paddle hook to stand mixer while yeast is activating.

In a separate medium bowl, whisk together the flour and salt. Add flour mixture, butter and egg yolks to the yeast mixture and beat on medium speed until just combined, then switch to the dough hook and continue to mix until dough forms a ball and cleanly pulls away from the bowl, about 7-10 minutes. Transfer dough to a greased bowl, cover and allow the dough to rise in a warm place for 1 hour.

Spray a baking sheet with non stick cooking spray and set aside. Transfer the dough to a lightly floured work surface and roll out to a 1/2 inch thickness. Cut donuts using a 3 inch donut cutter and gently transfer the donuts (and holes) to the prepared baking sheet, spacing them about 1 inch apart. Spray the tops of the donuts and holes with nonstick cooking spray and loosely cover with plastic wrap. Let stand in a warm place until they have almost doubled in size, about 30 minutes.

Using a deep fryer or dutch oven, heat oil to 350º. Line a baking sheet with a wire rack and set aside. Working in batches, fry the donuts and donut holes until lightly golden brown, about 1 minute per side. Transfer to the wire rack and let cool for 10 minutes before glazing.

To make the glaze, whisk together root beer extract, root beer and powdered sugar in a medium bowl. Add more root beer if needed to thin out the glaze.

Dip the donuts, one at a time, into the warm glaze, covering the top half of each donut with glaze. Allow glaze to set for a few minutes, then top with a large scoop of ice cream in the center of each donut and serve immediately.