Willie Tran of Calendly: Growth Product Management
Be ready to be wrong often as a growth PM, shared Willie Tran, Principal Growth PM at Calendly, for my interview series Austin Voice Of Product. Our interview has been edited for clarity.
Subscribe to the Austin VOP newsletter
Austin VOP #111
What was your path to product management?
The honest truth is that like every product manager, it was complete randomness and quite a bit of luck that got me here. I’m super grateful every morning I wake up that I have this career. I mean, I studied Political Science in college, and I was a research fellow focused on quantitative research methodology. I was frequently ridiculed by my friends about what I was going to do for a career. To be fair, I genuinely had no idea…I ended up working some random jobs I was really miserable at for a few years and then at some point I said, “Alright I have to try this tech thing that’s coming up.” Mind you, I grew up in Houston where it was dominated by oil & gas and at this time people outside of the Bay Area were highly skeptical about tech.
I ended up connecting with the cofounders of a startup that was a part of the first batch of Techstars Austin where they hired me as their first marketer. Heads up — I was terrible. They eventually moved me into an Enterprise PM role andwas terrible at that too. I didn’t even know what product management was. I read like ten pages of the Intercom book on product and thought I was ready to go. But of course, that didn’t help. It doesn’t tell you how to respond to your engineers when they identify an edge case your spec doesn’t cover. They eventually made me Head of Product and I led a small team, but in hindsight, it was premature. I definitely wasn’t ready to be a leader that early in my career, but I’m eternally grateful to them for believing in me.
I became obsessed with this idea of A/B testing and experimentation because that was kind of what I did as a research fellow and that was the last time I was really happy with what I was doing. I thought maybe there’s something here and I started looking into this new “growth” thing.
I actually wasn’t sure if I still wanted to be a PM. I just knew I wanted to do more of this new growth and experimentation thing. Dropbox was starting their Product Growth team at the time, and I was able to convince them to bring me on board as one of their first Growth PMs. Since then I’ve run hundreds of experiments, built, scaled, and led Product Growth teams, and implemented experimentation programs from scratch. I think I can confidently say that I’m pretty good at this Product Management thing now!
What advice do you have about Growth Product Management?
If you want to get into Product Growth, really learn your fundamentals with statistics. A lot of people like to talk about very rudimentary concepts like “statistically significant” or “minimum detectable effect” but I’ve found a lot of people don’t actually understand them to a fundamental level. I see a lot of people run an experiment for a week or two and as soon as they see it was positive, they turn it off. Is this wrong? Maybe. Running an experiment for a week isn’t inherently bad, but not knowing the variables which decide how long an experiment should run for is. If you’re not sure, you could be creating a false positive by instilling human error into the experiment (peeking problem, sampling error, etc). But maybe running it for a week is okay and when you looked at it, it just happened to be stat sig positive.
Also, don’t get attached to any experiment ideas. You will have a ~20% win rate. So expect to throw away 80% of your experiments. Just be ready to be wrong often.
If you work for a company with high traffic like Dropbox, you can have high experimentation velocity and a low experimentation win rate of 10% and still have a very positive net effect. But if you don’t have the luxury of high traffic, which is pretty common, then you need to think of a strategy that has a high experimentation win rate. You have to take time to really understand your user and solve their problems. Don’t just throw best practices at them.
A lot of people think Product Growth is — ”move the number, move the number, move the number.” Moving the number is the ultimate goal and that is the part the business side will resonate with. But you need to approach it with the mindset of a behavioral scientist — you are trying to learn what makes users tick. What are they thinking at this point? How are they feeling? You’re running experiments to essentially (in)validate that emotion, that hypothesis. You take these insights and either build on them or pass them on to another team to help the company be successful.
The beauty of Product Growth is I don’t really have the time to become attached to the ideas because we can have three or four experiments running at the same time. And with my current win rate of ~20%, I’m wrong 80% of the time. Could you imagine if I had any attachment to these experiments ? I would be a psychopath after being hurt so much.
There’s this concept of “winning and losing experiments” but that’s the wrong way to think about it. We often think about the positives but we need to care about negative results equally as much. The biggest killer for a product is apathy — that the user does not care. So if you move a metric in a stat sig negative direction, what that tells you is that the user actually cares about that element, you just executed on it incorrectly.
What has inspired you lately?
This is probably a weird answer… but it’s just having my dog. Dogs don’t think about their career or about the future. They just love every second you spend with them and are happy that you are there. That makes me think — how can I be more present like my dog?
I feel very grateful that I can spend my life on a career I genuinely love and make a comfortable wage as well. I just happened to have these unique skills that happened to line up into this weird role that did not exist eight years ago.
The fact is that I still study statistics in my free time and I still think about ways to teach others about what I do. When I remind myself how grateful and lucky I am, it makes me a little bit hungrier to get better. And my dog does that for me.
What is exciting about the product you are working on now?
I’m sure many folks you talk to are excited about the product and features they’re working on. But truthfully, I don’t get excited over features or the product I’m working on. What gets me excited is the behavioral psychology of the users of the product and running experiments to improve their user experience. Like running an experiment on the Calendly paywall: changing some things and being able to identify the emotional state and intent of the user when they reach this point. We are trying to answer what users want when they hit a paywall. Do they actually care about the different pricing plans or are they trying to get back to where they were? That is a really simple question but that is really important if you’re trying to build this experience. If we learn what they’re feeling, then we should always design the experience around that intent. But intent is hard to identify and experimentation is a fantastic way to get to that, assuming you design the experiment correctly.
A lot of people use the brute force method of applying some best practices they have read about or seen in some book or blog article. But it’s so much better to experiment and have a hypothesis around how to answer those questions. You end up with so many good insights that you get to build an experience that moves the number without hitting a local maximum.
It’s a long term view, but a problem I have seen with Product Growth teams is they stop being successful after six months because they picked up all the low hanging fruit and their win rate starts to severely decline. If you don’t have the muscle of driving insights and testing well-designed hypotheses around user problems, your Product Growth org is going to severely struggle after you’ve picked up all the low hanging fruit.
How might we build a stronger product community in Austin?
How do we define “strong ?” By trying to build stronger bonds between existing product managers or by trying to build our strength by getting more people into product management? I feel like the biggest thing is trying to help people get into product. It is infamously hard. If we are trying to optimize for value creation, getting people from zero to one, into product management will have a larger net value.
So how do you make people like you and me more approachable? I love mentoring others. My favorite thing to do is to help other people get into product management. I tell folks to reach out and I will give them time. So I will say here too — if you are interested in product, reach out to me — honestly.
A lot of product management content makes it seem like it’s easy to get into product management. But I like to give the real answer, which is that I got lucky. I found people who trusted me for some reason. It’s not a smooth path. It’s a journey and an individual journey for everyone. So the question I ask is are you willing to take that journey knowing that there is a good chance you might not end up in Product?
Product Management has become very popular. PMs make a lot of money and it looks like they just come up with and implement a lot of ideas. The money part is true but not the ideas part. It isn’t as fun as you think. And your ideas are shot down all the time. Definitely take the time to learn the realities of this job. It’s 80% politics and 20% execution.
Last question, what is your favorite product?
I have to split this up into two. My favorite software product is 1Password. I think it’s cool that they sell peace of mind which no one talks about how important that is. There are a lot of details in the 1Password product that makes you feel like it’s secure. Little things like when you type in your password, it has a vault opening animation. It doesn’t do anything to make the product more secure but it creates a feeling. It’s such a simple product and these details make a big difference.
My favorite non-software product is watches. Not smartwatches but regular watches. I am wearing my G-Shock John Mayer limited edition watch and the reason I like this so much is that watches are one of the few products that give you something without expecting anything in return. Smartwatches give you something but expect your attention back — this or that notification. A watch is the only thing that is there for you, your servant and they don’t expect anything in return. It’s one of the very few things in our life that has high daily active usage and is purely in service for you.
Thank you, Willie!
Austin VOP is an interview series with current and future product leaders to inspire the next generation of product leaders.
Subscribe to the Austin VOP newsletter
