Preheat oven to 350°F (175°C). Line a 9x5-inch loaf pan with parchment paper, leaving overhang on sides, and lightly grease.
Peel and mash the banana in a small bowl with a fork until mostly smooth (about 1/2 cup).
In a large bowl, whisk together almond flour, coconut flour, baking powder, cinnamon, xanthan gum, and salt until well combined.
In another large bowl, beat softened butter and sweetener with an electric mixer for 2-3 minutes until light and fluffy.
Add eggs one at a time to butter mixture, beating well after each addition until fully incorporated and creamy.
Add mashed banana, banana extract, vanilla extract, and almond milk to wet ingredients. Beat until smooth and well combined.
Pour wet ingredients into dry ingredients and fold gently with a rubber spatula until just combined. Do not overmix.
If using, gently fold in chopped nuts or chocolate chips. Reserve a small handful for topping if desired.
Pour batter into prepared loaf pan, spread evenly into corners, and smooth the top. Sprinkle with reserved toppings if using.
Bake for 45-55 minutes until golden brown and toothpick inserted in center comes out with moist crumbs. Tent with foil if browning too quickly.
Cool in pan for 15-20 minutes, then lift out using parchment overhang. Transfer to cooling rack and cool completely before slicing (about 1 hour).