Preheat oven to 350°F (175°C). Line an 18x13-inch half sheet pan with parchment paper extending over the edges, then lightly spray with non-stick cooking spray.
Beat melted butter, granulated sugar, and brown sugar on medium-high for 2 minutes until light and fluffy, then add eggs one at a time and mix in vanilla extract.
In a separate bowl, whisk together flour, Dutch-process cocoa powder, baking soda, and salt until no lumps remain.
Add dry ingredients to wet in two additions, mixing on low and adding the milk between additions, until you have a thick, smooth, spreadable batter.
Spread batter evenly across the prepared pan to about 1/4-inch thickness, poke holes in a 1-inch grid with a chopstick, then bake 15–18 minutes until set but still slightly soft.
Cool in pan 5 minutes, then lift out using the parchment overhang and cool completely on a wire rack before transferring to the freezer for 15 minutes to firm up.
Soften ice cream 15–20 minutes, spread into an even 1-inch layer in a parchment-lined 9x13-inch dish, and freeze 1–2 hours until firm.
Cut the chilled cookie slab in half crosswise, unmold the ice cream layer onto one half, top with the second cookie half pressing lightly, then freeze at least 3–4 hours until completely firm.
Using a sharp knife dipped in hot water and wiped dry between cuts, slice the frozen slab into 12 rectangles (approximately 3x4 inches each), returning to the freezer if it softens too much.
If desired, roll the exposed ice cream edges of each sandwich in mini chocolate chips, sprinkles, chopped nuts, or cookie crumbs, pressing gently to adhere.
Wrap each sandwich individually in parchment paper or plastic wrap, place in an airtight freezer container, and freeze at least 2 hours (ideally overnight) before serving.