Preheat oven to 400 degrees Fahrenheit. Line a baking sheet with a silicone baking mat.
In a large bowl, whisk together all-purpose flour, white whole wheat flour, brown sugar, baking powder, cinnamon, nutmeg, allspice, and salt. Cut in butter until mixture resembles coarse crumbs.
In a small bowl, whisk together milk, egg, and vanilla extract. Add to flour mixture and stir until just combined. Gently stir in chopped apple.
Drop by ¼ Cup-full onto prepared baking sheet.
Bake for 11-13 minutes, or until bottoms are golden brown and center is set. Remove to a wire rack and turn oven to broil.
After turning oven to broil, prepare glaze. Whisk together confectioner's sugar, milk, and vanilla extract until smooth. Brush a thin layer of glaze over fritters. Place under broiler.
Broil for 2-4 minutes, or until tops start to turn golden and the glaze bubbles.
Carefully remove baking sheet to a wire rack. Brush remaining glaze over the warm fritters.
Allow glaze to harden and serve.
Best eaten same day.