In a bowl, stir together the yogurt, fruit jam, milk, and vanilla. Add additional milk, if needed to thin the mix if it feels too creamy.
Evenly divide the mix between 6-8 Dixie cups or popsicles molds. I recommend Dixie cups over molds as the pops can be hard to remove from molds. Insert popsicle sticks. Freeze until firm, 4 hours or overnight. To make layered popsicles, pour 2-3 flavors into each mold. Keep in the freezer until ready to eat!