Preheat the oven to 350℉.
Microwave ½ cup milk for 40 seconds. (Add about another 1 tsp of milk before heating since some of the milk will be absorbed into the tea bags. You need ½ cup of steeped milk for the batter.) Put the 4 chamomile tea bags in the warmed milk to steep while you prepare the rest of the batter.
Put the 8 tbsp unsalted butterr, 1 cup granulated sugar, and ½ tsp salt in the bowl of a stand mixer. (If the butter was in the refrigerator, it needs to sit at room temperature for about 30 minutes to soften slightly. Or, microwave it at 20% power for 30 seconds to soften.) Use the paddle attachment to beat on low for about 30 seconds to combine, and then medium for 1½ to 2 minutes, until light and fluffy.
Add 1 egg to the bowl and beat on low speed for 20 seconds. Then add the 2nd egg and beat on low for another 20 seconds. Scrape down the sides and bottom of the bowl with a rubber spatula to evenly incorporate the ingredients into the batter.
Spoon 1 ½ cups all-purpose flour into measuring cups and add to a medium bowl. Add 1 tsp baking powder and whisk together.
Pour ⅓ of the flour mixture into the batter and mix on low for 10 seconds. Then remove the tea bags from the milk. Discard the tea bags. Add half the milk to the batter and mix on low for 10 seconds. Then add another ⅓ of the flour mixture and mix for 10 seconds, followed by the remainder of the milk, and mix for 10 seconds. Add the rest of the flour mixture and mix for 10 seconds on low.
Add the ¾ tsp lemon oil OR 3 tsp of finely grated lemon zest to the batter, and mix on low until evenly mixed (10-15 seconds).
Spray a 6-cup loaf pan with baking spray that contains oil and flour. Make sure to coat the corners and sides of the pan.
Spoon the batter into the prepared loaf pan. Use a rubber spatula to spread the batter evenly. Shake the pan to make sure the batter gets into the crevices of the design of the pan if using a shaped pan.
Bake at 350℉ for 50-55 minutes. Use a cake thermometer to test for doneness. It will turn bright red when done, with no wet batter. Or, use a toothpick inserted into the center.
Put the pan on a cooling rack for 10 minutes. If the cake has risen above the top edge of the pan, use a serrated knife held horizontally along the top edge of the pan to cut off the dome of the cake. Then put the serving plate upside down on top of the pan, and flip both together. Gently shake the pan to release the cake. Put the plate on top of the cooling rack while you prepare the glaze.