Make the vanilla bean brioche dough: In the bowl of a stand mixer or in a large mixing bowl, mix together the yeast with the milk. Add the eggs, butter, flour, salt, sugar, and vanilla bean paste and mix until the ingredients are just barely combined. Let the mixture stand for 5 minutes so the flour has time to hydrate.
Using the hook attachment, beat on medium speed for about 5 minutes until a soft dough forms. Increase speed to medium-high and beat until the dough is soft and supple, about 6 minutes longer. *If you do not have a stand-mixer with a hook attachment, knead the dough by hand in this step.*
Transfer the dough to a lightly floured surface and knead it with your hands for 1 minute. Form the dough into a ball and transfer it to a lightly greased bowl. Cover the dough loosely with plastic wrap, a paper towel, or aluminum foil and let sit in a warm place until doubled in size, about 30 to 45 minutes.
Make the chai spiced filling: While the rolls are rising, make the filling. Add the softened butter, dark brown sugar, cinnamon, ginger, cardamom, allspice, and pepper to a medium bowl and combine using a fork until a thick paste forms. Set aside.
Generously grease the bottom and sides of a muffin pan with butter. Turn the risen dough out onto a lightly floured work surface, punch it down and, using a rolling pin, roll into a 14×6 inch rectangle. Make sure the dough is smooth and evenly thick.
Spread the chai spiced filling all over the dough. Starting with the shorter end, tightly roll up the dough to form a 6-inch-long log and cut into 6 even rolls. Arrange them in the prepared muffin cups. Cover the pan and allow the rolls to rise for 20 minutes or until nearly double.
Preheat oven to 375°F (190°C).
Once the rolls have risen, bake the rolls for 15-17 minutes, until the rolls are lightly golden brown and the center of the rolls are cooked through. Remove pan from the oven and allow to cool on a wire rack for about 10 minutes as you make the icing.
Make the bourbon vanilla bean cream cheese icing: Add the softened cream cheese and the butter to a small bowl and beat to combine. Add the remaining icing ingredients to the bowl and whisk until smooth and combined. If the icing is too thin, add a little more powdered sugar; if it's too thick, add more milk (or bourbon). Drizzle icing over warm rolls. Cover leftover frosted or unfrosted rolls tightly and store at room temperature for up to 3 days.