Take the pie crusts out of the fridge and let them sit at room temperature for about 10 minutes so they unroll without cracking. Measure out all filling ingredients so everything is ready to go, and lightly flour your counter.
Preheat your oven to 375°F (190°C) with a rack in the center. Line two large baking sheets with parchment paper and set them nearby.
In a medium bowl, mix the pumpkin puree, brown sugar, pumpkin pie spice, cinnamon, salt, and vanilla. Sprinkle the cornstarch over the top and stir until smooth with no brown sugar clumps.
Unroll one pie crust onto your floured counter, dust the top with flour, and gently roll into a rough 9x12-inch rectangle. Trim the edges, cut into 6 rectangles about 4x4 1/2 inches, and repeat with remaining crusts for 24 rectangles total.
Place 12 rectangles on the lined baking sheets, spacing them about 1 inch apart. Spoon a heaping tablespoon of filling into the center of each and spread gently, leaving a 1/2-inch border. Avoid overfilling.
Whisk the egg and 1 tablespoon milk until smooth. Brush the bare border of each filled rectangle with egg wash, lay a second rectangle on top, press out trapped air, and crimp the edges firmly with a fork.
Brush the tops with a thin layer of egg wash and poke 3 or 4 small holes in each with a fork so steam can escape. If the dough feels soft or warm, chill the pans for 10 minutes.
Bake for 16 to 20 minutes, rotating the pans halfway through, until the crust is deep golden brown. Bake one pan at a time on the center rack if your oven runs uneven, and watch the last few minutes closely.
Let the pop tarts rest on the pan for 5 minutes, then transfer to a wire cooling rack and cool for at least 20 minutes. Do not glaze warm pastries.
Whisk together the powdered sugar, vanilla, pumpkin pie spice, and salt. Add 2 tablespoons milk and whisk until smooth, then add the last tablespoon a teaspoon at a time until thick but pourable.
Spoon or drizzle the glaze over each cooled pop tart and spread to the edges. Add sprinkles or extra pumpkin pie spice while wet, then let the glaze set for about 15 minutes before serving.