Whisk the cornstarch and water together in a small pot over medium-low heat.
Add in the jam and bring to a simmer for a few minutes, stirring frequently.
Remove from heat and let cool.
Unwrap the puff pastry and lay out one sheet on a large cutting board.
Fold the sheet in half evenly to create three, even seam lines and six rectangular quadrants.
Cut along the lines to make six rectangles.
Whisk egg and milk together in a small bowl and brush all the rectangles fully with the mixture.
Add a dollop (~1 tbs) of cooled jam to the center of THREE rectangles.
Lay the other three rectangles carefully on top and use your fingers to press down on the edges.
Use a fork to press into the seams around all the edges for a firmer seal and then prick the top several time with the fork.
Lay out these three pop tarts onto a large parchment covered baking sheet and repeat this process with the second sheet of puff pastry to create six total pop tarts.
Let pop tarts cool before icing.
Whisk powdered sugar, milk, and vanilla extract together until icing forms.
Spoon out over pop tarts and sprinkle immediately with sprinkles (once icing cools and sets, sprinkles will not adhere).
Icing should set in about half an hour, but they can be enjoyed immediately for a messy treat!