Step 1

Arrange a rack in the middle of the oven and heat the oven to 350°F. Meanwhile, cut 1/2 stick unsalted butter into a few pieces and place in a small microwave-safe bowl. Microwave until melted, about 20 seconds. Transfer 2 teaspoons to a 9x13-inch baking dish and brush to evenly coat the bottom and sides. Set aside the remaining melted butter.

Step 2

Place 1 cup packed dark brown sugar then add 2 of the large bananas in a large bowl. Mash the bananas into the sugar with a fork until mostly smooth with no large chunks. Add the reserved melted butter, 1 teaspoon ground cinnamon, 1/2 teaspoon kosher salt, 1/4 to 1/2 teaspoon ground nutmeg, and 1/4 teaspoon ground ginger, if using. Mix with the fork until you have a thick paste.

Step 3

Add 3 cups whole milk, 2 large eggs, 2 large egg yolks, 1/4 cup dark rum, and 1 teaspoon vanilla extract. Whisk until well-combined, about 1 minute.

Step 4

Cut 1 loaf stale French bread into 1-inch cubes. Add to the banana mixture and gently fold together with a large kitchen spoon, making sure each cube is coated. Let sit until the bread has soaked up most of the custard, 5 to 7 minutes. Meanwhile, cut the remaining banana crosswise into 1/2-inch-thick rounds.

Step 5

Pour the bread mixture into the prepared baking dish and spread into an even layer. Nestle the sliced bananas cut-side up into the mixture. Bake until the bread is puffed and golden-brown, and the bananas are slightly browned, about 1 hour. (If your baking dish is less than 4-inches deep, the butter may bubble up and over the sides of the baking dish. Place the baking dish on top of a baking sheet before baking.) Meanwhile, make the rum toffee sauce.

Step 6

Melt 1 stick unsalted butter in a large, wide saucepan over medium heat. Add 1/2 cup packed dark brown sugar and 1/4 teaspoon kosher salt and whisk to combine. Bring to a gentle simmer, about 2 minutes. Continue cooking, whisking constantly, until it resembles caramel, 1 to 2 minutes.

Step 7

Turn off the heat and very carefully pour in 3 tablespoons dark rum. (Be careful: The mixture will steam and bubble up.) Whisk the rum into the sugar. Bring back to a gentle simmer over medium heat. Cook, whisking frequently, until slightly darkened in color, about 1 minute.

Step 8

Turn off the heat. Add 1/4 cup heavy cream and whisk to combine. Let sit at room temperature until ready to serve.

Step 9

When the bread pudding is ready, let cool for 10 to 15 minutes (it will deflate slightly). Serve with the rum toffee sauce poured over each serving.