Product Led Growth: Building Growth Teams at Pinterest

Episode Transcription

Anna Eaglin:

Well, product-led growth is a rising strategy in terms of awareness and implementation. Many of us in product are still wondering how to make decisions, how to just structure and what other companies are working on in general, as it relates to product-led growth?

Christian Beck:

Yeah, on this episode, we're going to be talking with Prianka Rayamajhi and Lindsay Mindler both part of the growth design team at Pinterest. Before we dig into how they work, we want to highlight why they joined the company.

Prianka Rayamajhi:

I was one of the early adopters and so I'd always been a really big fan and so when that opportunity arrived, I fell in love with the design team, just seeing the amount of talent that there was. And then with growth in particular, I was really interested in just pushing the boundaries of my own design and seeing how I could become a growth designer that I've been hearing so much about.

Lindsay Mindler:

I wanted to join a company that was a bit larger, so I could really grow in my leadership skills.

Anna Eaglin:

What I love about this episode is that we get a chance to go behind the scenes of Pinterest, understanding how their product supports both the consumer and the business. One thing they do to serve their wide variety of customers is run experiments. These experiments allow them to determine where to focus their resources most effectively.

Prianka Rayamajhi:

So the goal of that is for us to really prove out that small user problem and expand and iterate as we prove it out. And if we're not necessarily solving a problem through that, through experimentation, then we're looking at what are the causes there, what are the behaviors there, what's happening to maybe we can pivot and try something different.

Lindsay Mindler:

Our job at the higher levels to be looking two years out at market trends, but the six to 12 months, we're trying to create a framework for the teams that isn't super directive, but is at least guiding, a guiding way of where we should be moving based off of the research and analytics that we've done.

Christian Beck:

When thinking about growth teams, my concern is that they make it too focused on solving business problems and lose sight of the users and the actual product.

Lindsay Mindler:

On the growth team, we have very rigorous metricals, then typically like to balance it out with a more qualitative goal, to make sure that we're not just letting the metrics guide us, we should be metrics and data informed and not data-driven. Even for a growth team, that's really important because you can game any sort of metric you want, but you'll start to see the longterm growth drop-off if you're not listening to your users.

Anna Eaglin:

So striking the balance between moving the metrics or goals forward without losing sight of the user experience is critical.

Lindsay Mindler:

I think one of the biggest mistakes I've seen growth teams make is they're producing everything in a business problem. And so basing everything in a user problem, and making sure that whatever solution you're doing is actually solving that human problem. And then ideally validating that with research.

Christian Beck:

So let's get back to this striking the balance issue Anna just mentioned, how was it actually done? Prianka starts off the conversation, answering how she leads her team to create the right approach.

Prianka Rayamajhi:

I think for my team in particular, when we're thinking about this S and B's in particular, small to medium sized businesses, I like to think about sustainability and design. So like, when I'm thinking about building something new, even if it's, let's say like a marketing upsell, I want to understand the user journey of when is the right moment for this upsell and what state is the user in. For example, if I'm a new user and I don't even know what a pen is on Pinterest, will a marketing offer enticed me to create right, like a pen and spend. I'm not sure because I really want to make sure that they have that context before. So really understanding when the right time is, let's say maybe they created a few pins, maybe they've created an ad, maybe that's the right time. So really understanding a user journey there and understanding when is the right time and using data to back that up with that qualitative, right? When a user journey will help to inform that rather than just putting those red buttons everywhere.

Anna Eaglin:

You mentioned this idea of you always have this qualitative research coming in all the time, and you're always learning this, this always on research model. I don't think I've ever really heard to it referred like that. I'd love to learn a little bit about that. And how do you keep those lines of communication open with customers? And are you researching certain initiatives or are there just common things that you're always looking for from customers and other users?

Lindsay Mindler:

So when we say always on, basically you think about it as if research is looking a year out. And so we work really closely with research to start to drive them in the right direction. So for example, if we're starting to get a sense of, okay, these key markets might be ones that we want to go into. So there's five of them, we need to narrow it down to one or two that we're really going to focus on. Research will go away, take the insights that they've already learned over the past couple of years, and we'll work with data analysts to figure out, okay, which of these key markets should we go into? Typically, what they'll do is then they'll go and run research over a period of time and be bringing that to us constantly. So by the time we get to that next half or that next quarter, they already have that research and they've already moved on to something else. And so they're really looking pretty far ahead, and they're just constantly looking at that really longterm light of where we should be going.

Lindsay Mindler:

And of course, we're giving a little bit of direction there, but we really rely on the research team because they are the ones on the ground talking to our customers. And a lot of times we have the opportunity to go on the ground with them, but they're really huge experts on where we should be going. So that's one thing we do as well as, as we're designing, we are testing out product as much as possible. So for all our different audiences, we have dedicated researchers who will do this always on approach, looking at these really big themes. And so even if we're seeing a gap in data around let's say a certain segment of our audiences, lower on activation in certain parts of the product and we're like, we have no clue why, right? Like metrics are not telling us that story, they'll go and basically put together a plan of how we're going to understand that. And so they're looking really far ahead, as well as in the short term of when we need to validate the things that we're designing, they're helping to validate that on the go as well.

Christian Beck:

I want to hear a little more about how the research team works. Sounds like you have a fairly large research team, and if they're, are they split between the researching the product and doing market research, or is it the same team that owns or the same person that?

Lindsay Mindler:

We actually have a pretty small scrappy research team, they're just like powerhouse research team though. We have researchers, qualitative researchers dedicated to audiences and then within the research team, we also have quantitative researchers and market researchers. And the market researchers will typically go across to all different audiences because there's far fewer of them. And so they're doing everything at the same time, they're dedicated, more qualitative researchers per audiences are doing a bit of the always on research, but then also very hands on with what we're designing every day. They come to design crits, and really involved in decision making, which is a really important thing to have on your team.

Christian Beck:

I'm hearing these bits and pieces, but it might be helpful to start with something that was like the initial target. We need to go figure out this, or we need to go after this market and maybe walk through how that goes through the stages of your team. Would it be possible to focus on a story like that?

Lindsay Mindler:

Yeah. A big initiative that we had over the past year and a half at Pinterest was focusing on small and medium sized businesses S and Bs. And how we decided to do this was, it actually started with our consumer problem, which was, we wanted to provide even more diversity in content to our consumers, so that they were having an even better experience on Pinterest finding new and unique ideas that would help them do real things in their lives. And so we have very large advertisers and large businesses on Pinterest, but we also want to just better support the small and medium sized businesses who were using the product in very different ways. A lot of them are actually consumers themselves. And so we were really treating those small and medium sized businesses as consumers. So an example of this is, they would sign up as a pinner, potentially they would convert to a business, but they were still landing on the consumer page every time they came back to Pinterest, which we call the home feed.

Lindsay Mindler:

And they weren't getting any sort of customized experience, they really had to dig to understand how they became successful on Pinterest. And we were just not giving them the tools where larger, advertisers in larger businesses, they're in it every single day, they are typically very sophisticated in their knowledge of how to build into these digital products and the small and medium sized businesses weren't. So we took a look at where it was breaking down, both in the out of product experience, all the way through onboarding and then that continual retention of those users. And what we ended up doing was actually doing a two year out vision of where we wanted to go before we started iterating at all, because you need to have a guiding light of where you want to go to start to make some decisions. And what ended up happening was we ended up doing a bunch of different work streams, we revamped our onboarding experience for small and medium sized businesses or we created entire mobile experience for them within the product.

Lindsay Mindler:

We redid business profiles, and we also created much better content creation tools as well as created a landing experience.

Christian Beck:

Backing up one sec. How did you, you started with the problem. How do you uncover that these problems?

Lindsay Mindler:

This one I think was hitting us in the face a little bit, where in research, we would see people onboarding onto Pinterest, so new users onboarding onto Pinterest and they're landing on their first home feed and the content within there is just not, it wasn't diverse enough. And we were doing a lot of efforts to help diversify that, but we wanted to go even deeper. And I think even for our core users who are using the product every day, every week, every month, they start to find the same things over and over again. And so we want to provide really fresh content continually. So qualitatively, we were seeing that firsthand. So in lab sessions where we were bringing pinners in, we're doing that a lot, especially for onboarding to understand how can we help them comprehend the product faster so they can get the most out of it. We are seeing that happen as well as with core users. So sometimes it would even pop up when we were intentionally looking for it and I think those are problems that you shouldn't ignore, even if you're looking for something else.

Lindsay Mindler:

And then we've just validated that with the data of, okay, how often is X amount of business being shown to these users? And it was just above where we wanted it to be. And so we started with a consumer problem first, and we have a deep belief at Pinterest that in order to help businesses be successful, we need to make sure we're solving consumer problems and connecting their business to help solve that. And so this was like a perfect example of that.

Prianka Rayamajhi:

So going off of what Lind mentioned earlier around how these small to medium sized businesses were getting that same experience as the pinner. They're going to their home feed, which is really just for inspirational content, and then we have your business profile, which is more this place to curate your content. There wasn't a place where you could see your organic content and your advertising content in one area. So there needed to be this cohesive landing experience for businesses to basically manage their businesses and take the most important actions that they can to be successful. And so we designed this landing page called business hub. And in this area, we were able to provide these modules. The first module consists of the most important actions that you could take to get started, complete your account, things like that and so we have these cards that we call activation cards to get you up and running.

Prianka Rayamajhi:

And it's built around this experience framework that says, okay, we think you're here as this type of business, we have the signal that you're this type of business, and you've maybe haven't done this action, here's something that you can do to further gain proficiency, et cetera. And then we have your organic module, so all of your fresh content that you're posting, CC the most recent content, you can even see your content that's also doing really well. And seeing some quick stats that you can dig and say, hey, this is doing well or I need to take action on this, the impressions are going down, right? And then we have your an ads module. So in that area, you can see how your ads are performing and also create more ads, et cetera. And what we saw from this linear experience is that we saw pin creation go up. So a lot of people were creating a ton of fresh content, we were seeing more people promoting their content as well, and just engaging in more business actions, further down that funnel.

Anna Eaglin:

So was there any part of this project where, it sounds like you were, I would assume testing these hypotheses as you went. Was there any part of this project where you got something wrong or you had this idea and the data was like, no, we're not doing it that way.

Prianka Rayamajhi:

Oh yeah. So when I mentioned earlier around having your recent organic content and your trending content, initially, we just had your recent content. Well, when you throw in analytics into it and you see a bunch of zeros, you're just like, what the heck? I just posted this two seconds ago and I'm seeing zero, right? So that's not a good sign. And so what we wanted to do was also provide what we tested was a toggle where you have your top pins versus your recent pins. And we turned on the top pins first, so everyone could see that. And what we saw were people promoting their content and engaging with that module more.

Anna Eaglin:

So one thing that you guys have mentioned is that ability to trust the data, but also trust your gut at the same time. So how, when it comes to things like this, because as a designer, you put something in the world and sometimes you just have a feeling that this is really going to work and maybe the initial reactions aren't great, but you're like, I just, I feel like I know this market, I know they're going to love this. How do you balance your gut and the data?

Prianka Rayamajhi:

For me personally, I tend to lead with research. So I always have that, I think this is right, because I've seen pain points. I've seen people complain about this online, or I've seen, like I have an inkling and for me, I always start with a user problem because it really helps me as a designer feel grounded and able to make that data driven decision. And also it helps me to say, okay, if this is wrong, it's wrong because there's something that this data is telling me. And so I need to now understand that and go look at why that is not correct. So I think for me, it's like, Oh, that gut feeling always is still rooted in previous research and in previous understandings of where it came from.

Lindsay Mindler:

What I would say is, I think about this in three ways. The first is I think just heavy debate with the triad, I think it's really important to have product engine design because we're all bringing very different perspectives. And so through debate sometimes where you can just come to a conclusion and resolution. The second is just through doing variant testing. And so there's been cases where we can't come to an agreement and where maybe design and engineering aren't seeing eye to eye and don't agree with it, what's going to actually work and research isn't telling us anything to help create that clarity. And so sometimes we'll just test variants depending on how much of an effort it is. And then the third thing I would say is, I think it's extremely important to really second guess data sometimes. And so we've had situations where we've run experiments and one recent one was, design the experience, we thought it was so much better than what was previously there. The data was showing that numbers were dropping though, as far as our activation rates and we just didn't believe it.

Lindsay Mindler:

And we just kept pushing and partnered really closely with data analysts to understand, dig into a little bit of the, okay, which part of our audience is actually dropping and what step in the funnel is it? To understand why this might be happening. And we actually found that the reason it was dropping is because it was a bunch of, it was like, I didn't think it was like 6,000 spam accounts or something. So we were like, this is actually a win. We got rid of all these spam accounts and the users that we want to help comprehend their product better, are actually doing that, and those ones are actually going up once you remove this small cohort or bad users. And so that was actually a huge win. And so I think even if you see the data going in the wrong direction, it's really important for you to dig into the why. And second guess it some time.

Christian Beck:

Yeah. I was just thinking, as you're telling that story, you have that judgment and you think you really believe strongly. And I was going to ask, is data like the final decision maker when that happens? It sounds like sometimes you have a gut instinct and maybe you go research it to validate or invalidate it. And in this case you probably uncovered something that was richer in the data than maybe you even expected.

Lindsay Mindler:

Yeah. I think for growth teams, you'll have periods in time where you are making decisions just based off of metrics, but obviously validating with qualitative data and the metrics should be going up. You will reach a point where you hit a local Maxima and you can't iterate your way through it anymore and you need to rethink the product. And part of that is really relying on research to guide you in what that should be, because when you completely redo an experience, and you don't do small iterations, you'll see drops in certain areas. But then you use those same tactics to continually iterate and help conversion of those experiences just in the same way you did before with the previous baseline. And so in those cases where you're knowingly going into it with, hey, we're in a local Maxima, we're not seeing numbers going up very much like we used to in these spaces, as we're iterating, that's a good indication that you're reaching that. And you need to do some deep digging with research to understand why, and then do a larger step change that might drop metrics.

Lindsay Mindler:

You need to have that buffer of how much are you willing to drop it though, to make sure you're not going too far off, ideally it remains neutral, but you need to bucket for some negative loss and then apply the same conversion tactics that you used previously. And eventually, I mean, we've always seen it go past where it was previously, luckily we've been in that spot. So having something like a holdout group in those situations is really important to help validate that you're doing the right thing when it comes to getting to a new global Maxima.

Christian Beck:

Real quick, just on me, what is a holdout group?

Lindsay Mindler:

Hold up group is, let's say you're redesigning an entire onboarding experience, you do this one really big step change, and you really set to 20% of the users to see how it's doing. You continue to iterate and basically use 95% of your users are seeing the new experience, but maybe 5% aren't and you're holding it for a year plus. So you get to see the longterm impacts of what you've been doing, which is really crucial to make sure that you're not just optimizing for the short term. And it was really helpful in learnings for the next thing you were going to do that you believe is going to have a longterm impact versus short term.

Christian Beck:

So it's fair to say, you guys are creating parallel timelines in people's lives. And I think I just figured out how we're going to end up experiencing multiple timelines in life, is what it sounds like, for a full year.

Lindsay Mindler:

I guess a little bit.

Christian Beck:

Yeah. Interesting. I have another question and I don't know if this was covered Prianka, but when we kicked off this story of the problem, and then you jumped in with the business hub. I was curious at that moment, how is what you're doing at that stage of say designing a new feature, something large. Sounds like business hub was large, it has its own name versus what you're doing in other phases where it's changes smaller tweaks to things. What does the design process look like? And how does growth play a part in that when you're actually spinning up a whole new feature?

Prianka Rayamajhi:

I'm in a couple of teams and one of my teams focuses more on these long term kind of growth, growthy things where we're building up these big features so that we can do experiments, those smaller experiments on those surfaces and really learn from them. So we have that and then we have more of like, okay, here are these three metrics that our team is really focused on in this half that we want to go alone and here are some potential work streams. So we really break it up into various work streams. Let's say that one of them is just activating, growing the number of businesses on our platform, right? So let's say we want to go on that, we would come up with a huge list of ideas and problems that we want to solve for, that could potentially help hit that metric. And so in that case we're coming up with very small numbers, let's say like in our onboarding experience, let's say in that we want to understand if people want to trying to, let me think of a good one Lind.

Lindsay Mindler:

For the small stuff. I think. Tiny things like how we ask about you as a business. What type of business are you? We even do small iterations in what does that list look like? So to understand what's resonating with people. So if we're seeing 30% of people are choosing the other option, we need to iterate it and a little bit more. So run some variants to understand what is the right list and bucketing for businesses so that they can identify who they are as a business. And then we can help support them as that type of business down the line. So tiny little things like that of just changing a list out.

Prianka Rayamajhi:

Yeah, I think going off of that, like for example, let's bring back marketing offers, for example, or incentives more so, like if we were able to incentivize a small group, really thinking through that subset and then trying to grow that number of subset to get people to sign up, for example. So that could be one way of doing a smaller project and then thinking through longterm, what does that look like? So once we've proven it out for a particular user group, you can look at other business types or you can look at other spaces of businesses. Like let's say very small businesses like your XCs or smaller businesses than that to medium sized businesses. Yeah. You can take a look at how you can design that incentivizing feature for each of those buckets.

Anna Eaglin:

As a final question, I would love a perspective from both of you on this one. So with the rise of growth teams and the way, it's almost disrupting traditional design and product teams and how we've all learned to work together, how do you think that design and product are changing as an outcome of that?

Lindsay Mindler:

Actually, I see it working out differently at different companies to be honest with you. At Pinterest, we have product managers who, I think a third to half of them on our team are actually coming from data analysts backgrounds, which is so helpful because then I can match designers who are going to push for the longterm vision. I think the way that design is changing and growth, when I'm looking to hire growth design leads, managers, product designers, I'm really looking for designers who are able to paint that longterm vision of where we should be going, but also can get really deep into data. I think designers naturally really leverage research and so that one is a little easier to find, but them being able to dive into data, ask the right questions or, Oh, well, have we looked at the segment and why it's dropping off here? Here's a way that maybe we can look at it. As well as when we're running an experiment, being able to read an experiment dashboard and start to make some hypotheses of why certain metrics are going in certain directions is really important for a growth designer to have.

Lindsay Mindler:

And then I think on the product side, I think growth is actually, although new, it's changing really rapidly. I think we went through a phase with growth teams I mean, at my last job as well, where we were just like growth means you're just focused on metrics. You don't look at research, you're just going and go and going at metrics and I think that's changing pretty rapidly because I think it gets people in the butt a lot. So it's like, you'll start to see that longterm growth start to drop off and if you're doing the wrong things for your user, because you're just trying to chase a metric, it will never work out. And so I think growth teams have shifted to be a lot more diligent about solving real user problems. And I think product management, obviously I can't speak, I'm not a product manager, but so I can't speak too much to that, but I see our product managers even evolving on the team to really focus on the user problems first.

Lindsay Mindler:

But they're always bringing that expertise of making sure that all of the user problems we're focusing on, we're prioritizing them the right ways to help with the business as well, and trying to make the most impact as possible. And they're always two steps ahead of what data segments we should be looking into, which is just an amazing pairing between design and product management.

Prianka Rayamajhi:

For me, in particular, one thing that I've learned the most from being a growth designer is how much you have to understand how other teams function within your organization is critical to the success of your team. For example, just understanding where you can experiment is key and when you can experiment, understanding when things are shipping in other teams to leverage, that is very critical. So in terms of growth, I feel like you're kind of a diplomat in your role because you just have to really stay on top of what other teams are doing. And also what other people, designers, companies, are doing across the industry. I think that's really critical for just ideation and just seeing what longterm success looks like.

Christian Beck:

This is maybe a super specific question or not. I'm just curious, do you think and either of you can answer, but I'm thinking of you Prianka, we went to the same program. But do you think like the field of UX design is changing or is it just evolving to handle growth? Like you look at yourself as a designer, a growth designer today, do you feel like the things you're doing are vastly different than what you learned or how you started your career or have they just evolved to include this data intelligence? I can't remember, we actually had a word for just having the understanding of how to utilize data.

Prianka Rayamajhi:

Yeah. I think it goes back to what Lind was saying about that gut feeling, the abductive logic that designers are naturally wired to do. I think it evolved for me personally, going from just building flash websites back in the day to where I am now, it's really still using that abductive logic, but just having more tools in my toolbox to just work with. And it's made me think about design in a completely different way from where I was at four years ago, when I was leaning more heavily on just qual. I'm like, Oh man, now I can think about so many more lovers to work with because of quantitative data at scale.

Anna Eaglin:

Prianka and Lindsay provided interesting insights into how their roles enabled Pinterest to embrace the product led growth strategy, while sharing specifically how they make decisions. Next week, we'll connect with Jim Cantu, digging into how to build growth teams. Thanks so much for listening to the show this week, if you haven't yet be sure to subscribe, rate and review this podcast. Until then, visit innovatemap.com/podcast and subscribe to learn how you can take your product to the next level. As always we're curious, what does Better Product mean to you? It is up on Twitter @innovatemap, or shoot us an email at podcast@innovatemap.com.

Christian Beck:

I'm Christian.

Anna Eaglin:

And I'm Anna, and you've been listening to Better Product.

Christian Beck:

Better Product.

Anna Eaglin:

Drop mic.