Preheat your oven to 350 degrees, and generously grease an 8-inch square or round cake pan. Optionally, line the cake pan with parchment paper that has over-hanging strips to help with removing the cake from the pan. In a large bowl, stir together the flour, almond flour, sugar, cinnamon, baking powder, ginger, cloves, and salt. In a medium bowl, stir together the egg, buttermilk, oil, and almond extract. Pour the wet ingredients into the dry, and stir together with a spatula until well combined. Pour the cake batter into your prepared cake pan and spread evenly. Then spread your filling mixture on top of the batter- it's ok if it swirls together a little.

For the crumb topping: In a medium bowl, stir together the flour, almond flour, brown sugar, melted butter, oil, and almond extract with a fork until large crumbs form. Sprinkle the crumbs evenly all over the top of the cake, using all of the crumb mixture. Bake the cake for 38-40 minutes, or until a cake tester comes out fairly clean, with just some apple mixture sticking to it. Cool completely in the pan before slicing and serving.