Preheat oven to 375 degrees F.
Remove the stems from the mushrooms and chop the stems finely.
In a nonstick frying pan over medium-high heat, bring the chicken broth to a boil.
Add the shallot and mushrooms stems; cook, stirring occasionally, until the vegetables are tender and most of the liquid has evaporated, about 2 minutes.
In a medium-size bowl, combine the vegetable mixture, rice, mayonnaise, tarragon, and half the Parmesan cheese.
Using a paper towel, wipe out the frying pan and place over medium heat.
Add the olive oil and mushroom caps; saute until coated with oil, 20 to 30 seconds.
Arrange the caps, stem end up, on a baking sheet.
Mound about 2 teaspoons of the rice mixture in each cap.
Sprinkle with the remaining Parmesan cheese.
Bake until the cheese is lightly browned and the mushrooms are tender but still hold their shape, about 10 minutes.
To serve, dust each cap lightly with paprika.