Great question — when doing user research you always have to be very careful about how you reach out to your users. You don’t want to say “Hey, we see you used our site 6 times, then stopped for 32 days, then came back 17 days ago and have been active since, can we call you?”.
We first reached out to our targeted user list with a simple message. Something like: “We see that you registered recently for our site. We are doing research to learn how to improve our experience. Would you be open to a 10 minute phone call to help us?” Because these were users who had become recently active, they were bigger fans of the service and were more likely to say yes.
Also the goal was to get to 10 conversations, so it wasn’t like we barraged tons of people. I find that 10 different conversations should give you enough different perspectives to derive anecdotal patterns. Further conversations become diminishing returns.