Heat the oven to 350 degrees. Coat an 8 by 8-inch baking pan with butter or nonstick spray.
TOPPING: Mix the flour, brown sugar, cinnamon, allspice and salt in large bowl. With a pastry blender or fork, cut the butter into the mixture until it forms pea-size lumps. Stir in the pecans.
CAKE: In a medium-sized bowl, whisk together the flour, baking powder, and salt. In another bowl, add the butter and sugar and beat with a hand mixer until the mixture is light and fluffy, about 3 to 4 minutes. Add the eggs, 1 at a time. Stir in the buttermilk and the vanilla. Slowly add the dry ingredients and stir until completely incorporated. Fold in the berries with a rubber spatula, then add the batter to the baking pan. Sprinkle the streusel topping over the batter. Bake for 45 to 50 minutes. Remove from oven and let cool for before serving.