Step 1

Preheat the oven to 350ºF and prepare three 6-inch cake pans by spraying the sides with cooking spray and placing a wax or parchment paper circle into the bottom of each one. Measure the cake flour by spooning into your measuring cup and leveling it. Place it into a medium sized bowl and add the baking powder, baking soda, salt, ground cinnamon, allspice, nutmeg, and ground ginger. Whisk together and set aside.

Step 2

In the bowl of your stand mixer (a handheld mixer works fine too!), cream the butter on high for two minutes until it's light and fluffy. Add in white and brown sugars and continue to mix on high for another two minutes, scraping down the bowl and paddle as needed. Add the eggs one at a time, mixing thoroughly after each addition. Add pumpkin puree and vanilla and mix for one minute on medium-high, scraping down the bowl and paddle once more.

Step 3

With the mixer on low speed, add in the dry ingredients and mix until just combined. Add the buttermilk in a steady stream and mix until just incorporated. Scrape down the sides and bottom of the bowl and give it a few stirs to make sure there are no lumps. Batter will be slightly thick, but pourable.

Step 4

Pour batter evenly between the cake pans (fill them no more than ⅔ full) and bake for 35-40 minutes, until a toothpick inserted comes out clean. Cool the cakes in the pan for ten minutes before removing and continuing to cool on a wire rack or flat surface.

Step 5

With a hand mixer or paddle attachment on your stand mixer, cream the butter and brown sugar on medium-high until it’s creamy, light, and homogenous. About 4 minutes.

Step 6

With the mixer on low, add the powdered sugar a few cups at a time, scraping down after each addition and making sure each cup is fully incorporated before adding the next one.

Step 7

Add vanilla, milk, and salt and mix on low for another minute until fully incorporated.

Step 8

Chill a metal mixing bowl or the bowl of your stand mixer in the freezer for 15 minutes. Once the bowl is chilled, add to it the heavy whipping cream and powdered sugar. Beat with a whisk attachment (a stand mixer or hand mixer will work for this) on high until peaks form and the mixture is thick and whipped, about 5 minutes. Remove the whipped cream mixture from the mixing bowl and set aside in a separate bowl.

Step 9

Add the softened cream cheese, canned pumpkin, vanilla, cinnamon, and nutmeg into the mixing bowl and mix on medium speed until it’s fully combined and there are no lumps, about 5 minutes.

Step 10

Using a rubber spatula, fold the whipped cream mixture into the pumpkin mixture until combined. Store in the refrigerator in an airtight container if not using immediately.

Step 11

Combine the whole milk and vanilla. Set aside if using right away or refrigerate until ready to use.