Move the boxes of puff pastry from the freezer to the refrigerator the night before, or leave them on the counter for 30 to 40 minutes. The pastry is ready when it unfolds easily without cracking along the fold lines.
Place an oven rack in the center position and preheat the oven to 400°F. Line two large rimmed baking sheets with parchment paper.
In a medium bowl, add the 2/3 cup pumpkin purée, 1/4 cup brown sugar, 2 tablespoons granulated sugar, and 1 teaspoon of the pumpkin pie spice. Stir until smooth, thick, and evenly colored. Divide the filling into two equal portions of about 1/3 cup each.
Lightly dust your work surface with a pinch of the all-purpose flour. Unfold one sheet of puff pastry and lay it flat. Gently roll over it once or twice with a rolling pin to smooth out any creases. Transfer the sheet to one of the lined baking sheets.
Scoop one portion (about 1/3 cup) of the filling into the center of the pastry sheet. Use the back of a spoon or a small offset spatula to spread it into a thin, even layer, leaving a 1/4-inch border of bare pastry around all four edges.
Unfold and smooth a second sheet of puff pastry, then lay it directly on top of the filling. Line up the edges and press around the border with your fingertips to seal. Roll the pin over the top one time, using very light pressure. Refrigerate for 10 minutes if the pastry feels warm or soft.
Use a pizza cutter or sharp knife to trim off any uneven edges. Cut the pastry sandwich into 9 strips, each about 1 inch wide. Press straight down and slice in one smooth motion.
Pick up one strip at each end, holding it gently so the filling stays inside. Twist the ends in opposite directions three to four times to create a spiral. Place the twist on a lined baking sheet, pressing both ends down lightly. Space the twists about 1 1/2 inches apart.
Repeat steps 4 through 8 with the remaining two sheets of puff pastry and the second portion of filling. You should end up with 18 twists total, about 9 on each baking sheet.
In a small bowl, whisk together the egg and 1 tablespoon of the milk until completely smooth. In another small bowl, stir together the 3 tablespoons sanding sugar and 1/2 teaspoon of the pumpkin pie spice. Brush the egg wash lightly over the tops and sides of each twist, then sprinkle the spiced sugar generously over the twists.
Bake one pan at a time on the center rack for 15 to 18 minutes, rotating the pan halfway through. The twists are done when they are puffed, deep golden brown, and crisp on the outside. If they brown too fast, tent them loosely with foil for the last few minutes.
Let the twists rest on the baking sheet for 2 minutes so they firm up. Move them with a spatula to a wire cooling rack set over a sheet of parchment or a baking sheet. Cool for about 10 minutes before glazing.
In a small bowl, whisk together the 1 cup powdered sugar, the remaining 2 tablespoons milk, the 1 teaspoon vanilla extract, and the remaining 1/2 teaspoon pumpkin pie spice. Whisk until smooth. Drizzle the glaze over the warm twists in zigzag lines, or serve it in a small bowl for dipping. Let the glaze set for 5 minutes, then serve.