In a medium bowl, whisk together 3 cups of flour, 1 tsp baking powder, and 1/2 tsp salt until evenly combined. Set aside.
Beat 1 cup of cool room-temperature butter with 1 cup of granulated sugar on medium-high speed for 3 to 4 minutes, scraping down the bowl halfway through, until pale and fluffy.
With the mixer on low, add the egg, 1 tsp vanilla extract, and 2 tbsp milk, then mix on medium speed for about 1 minute until smooth. Scrape down the sides of the bowl.
With the mixer on its lowest speed, gradually add the flour mixture in 2 to 3 additions, mixing just until no streaks of flour remain. Do not overmix.
Divide the dough into 2 equal disks, wrap each tightly in plastic wrap, and refrigerate for at least 1 hour or up to 3 days.
Preheat your oven to 375°F (190°C) and line two baking sheets with parchment paper or silicone baking mats.
On a lightly floured surface, roll one disk of chilled dough to an even 1/4-inch (6mm) thickness, working from the center outward and rotating the dough as you go.
Press cookie cutters straight down into the dough without twisting, then use an offset spatula to transfer the cut shapes to the prepared baking sheet, spacing them about 1 inch apart.
Bake on the center rack for 8 to 11 minutes, until the edges are set and just barely golden. The centers should still look slightly underdone; they will firm up as they cool.
Let cookies rest on the pan for 5 minutes, then transfer to a wire rack and cool completely (about 30 to 60 minutes) before decorating.
Whisk together 2 cups powdered sugar, 3 tbsp milk, and 1/2 tsp vanilla until smooth, adjusting consistency with more milk or powdered sugar as needed, then divide into small bowls and stir in gel food coloring.
Spoon, brush, or pipe the glaze onto each cooled cookie, then add sprinkles or sanding sugar immediately while the glaze is still wet. Allow to set for 30 to 60 minutes before stacking.