Preheat the oven to 325°F and grease and flour three 6 inch cake pans or two 8 or 9 inch cake pans. Set aside.
Combine the sugar and the lemon zest in a small bowl and rub the lemon zest into the sugar with your fingers to infuse the sugar. Set aside.
In a stand mixer fitted with a whisk attachment, beat the softened butter and oil together until they become homogenous, about 3 minutes. This will not work if your butter is not completely room temperature.
Slowly add the granulated sugar to the oil and butter, beating on high until the mixture is light and fluffy and turns a very pale yellow, about 3 minutes.
Add the eggs in one at time, scraping down the bowl in between each addition. Add the lemon extract and the vanilla extract. Beat the mixture on high for 3 minutes more to incorporate as much air as possible.
In separate bowl sift together the flour, salt, and baking powder. Alternate mixing in the dry ingredients and the buttermilk in 4 batches, starting with the buttermilk and ending with the dry ingredients. It’s okay if the batter still has a few lumps in it–you don’t want to over mix it. Last, fold in the poppy seeds.
Divide the batter evenly into the prepared pans and bake for 30 minutes or until the cake is a lightly golden brown and springs back when pressed. This will depend on your oven. Take out of the oven and transfer to a cooling rack. Let the cakes cool completely.
While the cakes cool, make the frosting. Beat the butter and cream cheese together until combined.
Scrape down the bowl and add the powdered sugar, 1 Tablespoon of the milk, vanilla extract, lemon extract and salt. Beat until light and fluffy, about two minutes. If the frosting is too thick, add in a Tablespoon of milk at a time until desired consistency.
Place first cake on a cake stand, then top with about 1 cups of frosting, using an offset to spatula to spread the frosting evenly to the edges of the cake, bringing it up at the edges so it is a little taller than the frosting in the center of the cake. This will help keep your lemon curd contained. Spread about 1/3 cup lemon curd in the middle of the cake, leaving about a 1 inch border. Layer the second layer and repeat the process. Place the third layer on top and frost the top and sides of cake with remaining frosting. If making a naked cake, use an offset spatula to scrape off any excess frosting. If desired, sprinkle the outside of the cake with more poppy seeds. Refrigerate until ready to serve!