Preheat oven to 350°F and prepare the bundt pan with non-stick spray or butter.
In a medium sized bowl, whisk flour, baking and salt together and set aside.
In a separate bowl, massage sugar and lemon zest together with fingers. This will help with infuse the sugar with the lemon flavor.
Next, whisk the yogurt and lemon juice into the sugar. Then stir in the eggs one at a time until incorporated. This will help the sugar and proteins bind properly. Add the dry ingredients and combine. Fold in the olive oil.
Transfer batter to the pan and bake for 45-55 minutes until baked through. Let cool slightly. Then turn out from pan.
To make Lavender Drizzle, warm whole milk in a small pan or microwave until slightly bubbling around the edges. Stir in dried lavender and let steep for 15 minutes. Strain lavender from milk then whisk milk with powdered sugar until dissolved. Drizzle over cake.