The fight for the soul of research, with Learner’s CEO Alec Levin

By Ian Floyd, Angie Peng|10 min read|Updated Aug 24, 2023

It’s been a hard year for UX researchers. Layoffs have affected every function in tech, but researchers especially.

Alec Levin, CEO of Learners, a professional growth network for UX researchers, thinks this lean period could be a time to think seriously about how researchers can prove their business value. Specifically, by:

Bearing in mind that researchers represent the interests of the business – not the user.
Considering how research relates to revenue, profit margins, cost of goods sold, average revenue per user, and LTV to CAC ratios.
Expanding knowledge of business concepts to successfully execute strategic or generative research.

“Ultimately, research is about generating knowledge,” said Alec. “You figure out what information you need to get, you synthesize that into knowledge – but it has to be the stuff that matters. And the only way you can know if it matters is if you understand what matters to not only the business as an entity, but also the individuals who are leading functions there, too.”

In this week’s episode of “What's in it for them?,” Alec discusses:

How UX researchers can definitively prove their value to the business
The research teams that are exceptional at aligning research with go-to-market strategies
Why incentives are so important, even when budgets are being cut

The following is an abridged version of our discussion.

Angie Peng: What are some trends that you see in research as a space? Are there any patterns in terms of how the space is going, or any speculation as to where the industry is going?

Alec Levin: Yeah, sure. I think it's shrinking right now, and I think it's going to continue to stay smaller than it was a year and a half ago or a year ago for a while. I think other disciplines where there's been layoffs are going to recover relatively quickly, but I don't think research will.

There's a bit of a fight, I think, for this soul of research right now in terms of ‘what are we, what is our job?’What do we do here?’

I love the research community. I spend all my time in this. But I think there's also been a fair bit of a lack of self-reflection. A lot after layoffs. Of course, it's painful. And of course there are things that are beyond our control. But I've seen very few people talk about the things that were within our control. Things that we could have done better, things that we could have done differently.

One of the common myths I hear over and over again that I am doing my best to squash is this idea that research represents the user. That's false. Categorically false. We represent the business. Users don't elect us. Customers don't pick us, right? Businesses do. And they do so with an expectation of value that will come back in the form of greater growth or greater profits.

That's just the reality of it. Unless you work for a nonprofit or a union or a government agency, then it's different. But the vast majority of us don't work in those situations. And there's been a real aversion for researchers to talk about things like revenue, and things like profit margins, and things like cost of goods sold, and things like average revenue per user, and LTV to CAC ratios and conversion funnels.

And one of the big things that everyone's been really excited about is doing what we call ‘strategic research’, or ‘generative research’. You cannot do those things successfully unless you understand the business concepts that your partners in product or your partners in the leadership team care about the most.

And so I think until we do a little bit of growing up in that area, I think it's going to be very hard to see how this space grows until we see more leadership there.

Angie Peng: Have you seen this done well? What does it look like when research and go to market are very aligned?

Alec Levin: My favorite research team in the world, I think, is Slack’s.

There's some amazing people there. I think I know about two thirds of them now. And they're a really special team because I think, I'm guessing Slack is about 4,000 to 5,000 employees. I might be wrong. You want to take a stab at how big their research team is?

Ian Floyd: I don’t know. Four or five people.

Angie Peng: I was gonna say 20 to 30.

Alec Levin: I think it's eight, including management. And one of the things that's really interesting about them is first of all, the way that they work with the rest of the company is so interesting. And every time I talk with the leader of that team, Christina, I've just learned so much more.

And this is coming from somebody who spends a lot of time talking to people in the space. And so her team is entirely – the actual IC researchers are oriented around core business questions. And a few years ago they gave a talk at our conference, pre-pandemic, and I reference it all the time, but they talk about how their team works, and one of the examples that they give is when Slack was expanding to Japan.

And they tell this story about how the research team was deeply involved in that expansion. Now, of course, there's a lot of differences between American business culture where they're based in, and Japanese business culture. There's differences in how the language works, right? And the syntax.

There's differences around conventions on who needs to be informed about what, there's differences around how it's appropriate to market things, and what are the appropriate things to do.

They thought, ‘in order for us to be successful there, these are all things that we need to understand and we currently don't understand them.’ So the research team is tasked with going out and figuring that out. And they helped with what kind of product modifications needed to be made. What kind of conventions and changes to how notifications work, but also things like writing the speech for the CEO when they open their office there.

And so these are all the things where – ultimately, research is about generating knowledge, right? You figure out what information you need to get, you synthesize that into knowledge, but it has to be stuff that matters, right? And the only way you can know if it matters is if you understand what matters to not only the business as an entity, but also the individuals who are leading the functions there, too.

Ian Floyd: And so let's say that I'm a researcher and I have found myself at a company that isn't doing that, right? How do I go about building that buy-in and really setting that new strategy to, realign everything in that direction?

Alec Levin: The thing that you definitely don't do is try and go to everybody and tell theem that they're doing it wrong and fight them.

But that seems to be our instinct. What we really need to do is start with an acknowledgement that, in order to be successful at our jobs, it needs to start with the foundation of trust. What's interesting – there's all this conversation going on around disinformation and misinformation and all sorts of stuff like that, right?

And what's the difference between like true information and disinformation or misinformation? On the surface, they look exactly the same, right? The difference is, or one of the big differences is: do you trust the source? Does it align with things that you already understand?

This foundation of trust is pretty sorely missing for a lot of teams and researchers, and that's where you need to start. So that doesn't start with things like, ‘Hey, I'm going to give you a seminar on how to do ethnographic research.’ It probably starts with stuff like, ‘Hey, are you watching the game later today?’

‘Do you wanna come over and we can watch?’, or whatever, or, ‘Do you want to go catch it at a bar?’, or whatever. ‘Hey, are you interested in coffee?’

it starts with relationships, right? And you go person by person, right? Being honest about what your goals are and fitting those goals into other people's goals too, right?

I really want to make sure that this team has everything that they need to know in order to hit our core quarterly goals for Q whatever, right? And so I'd really like to know more about you, more about the challenges that you're facing. Start from an area of curiosity and inquiry. Then once you've done that, and people feel heard, then you can start talking about methods.

Then you can start talking about ways that you might be able to help. But our instincts are to just prescribe stuff when we don't know very much about what's going on behind the scenes. Because we haven't even asked those questions.

And so that’s step zero, is, do you have trusted relationships in terms of your peers, in terms of the folks that are managing you?In terms of the leadership team? You don't have that. That's where you have to start.

Ian Floyd: And whenever there are budget cuts one of the top things is obviously labor, and that comes in the form of layoffs. The other is incentive budgets. How should researchers go about justifying research incentives? And what role do research incentives play in the whole process?

Alec Levin: That's a good question. Incentives, sometimes, I think, are misunderstood. For example, if you're building a product in, let's say, the B2B space, right? And your ideal customer is – I don't know, maybe they're a sales director, let's say.

They're probably doing pretty well financially, odds are. And you might think that an incentive doesn't matter. But the token does matter a lot, right?

It's a sign of appreciation. It's a sign of respect, that I value the time that you gave me. And so that's one of the things that I think is counterintuitive. Of course, there's some ethical considerations around incentives that if you're extracting value from them, it's probably a good idea, especially when you're dealing with lower income communities, marginalized communities – especially important.

Point of view of you trying to figure out what's important and what you need to know. Making sure that the people on the other end of these conversations feel valued is important. And I think there's just three things that predict whether or not people are willing to participate in your research, and you can adjust them.

One of them is, do you have mission alignment? So for example, you're building a widget that saves somebody two hours a day. And if you just need to figure out how to build the orient or configure the widget correctly, people are going to be very interested in participating in your study, because they don't want to do that task for two hours a day anymore.

So, do you have mission alignment? That can also be values alignment. Second one is, are you a decision maker, or do you have influence, right? So, people don't want to spend time with folks where, I can give you all the most valuable information in the world, but you can't do anything with it.

You can't actually impress change. Then, there's not going to be a lot of interest there. And then the last one is the incentives. And that matters both from a compensation point of view, but also, I'm showing you that I value you. And so depending on what you're building and what stage your company is at, adjusting those.

The incentive is obviously the one that's most adjustable, but those are the things that you're gonna want to consider in terms of how you think about your recruitment and incentive strategy.

Angie Peng: What are some different types of incentives you've seen work, depending on the situation?Like monetary, non-monetary. How do you actually express thank you? Or, ‘I value your time’ to a user?

Alec Levin: I mean it definitely depends on context. So for example, if you're doing research with gig work, gig economy workers, definitely give them cash. Cash or cash equivalent.

I used to do this when I was broke doing my first startup. I was bicycling food around downtown Toronto in the middle of the winter. It was not fun, and $100 was a lot of money. Or 80 bucks is a lot of money. Obviously, that's one thing if you're dealing with sometimes higher income folks, again, if you're in B2B research, then there are things you can think about, stuff like, ‘hey, we're gonna be doing a leadership dinner. I'd love to invite you out to that.’

That's probably going to be more meaningful – situations, connections. Peers, and people they can rely on, is extremely valuable. So those are some things to think about. You can do swag and stuff like that. That's exclusive, but you have to have the right kind of brand for that to be meaningful at all.

So I think that's probably not recommended, but those are the types of things. I think it's great to get creative with that. And they're not mutually exclusive. You can do more than one.

Angie Peng: You're building, obviously, a tech platform. You work in the tech industry. What's a piece of technology that you think is overrated in the UX space, and then what’s underrated? And you can just answer one of them if you want, but I'm curious if you have any thoughts or hot takes.

Alec Levin: Overrated and underrated.I'm gonna reframe the adjective a little bit. I think there's a lot in the research space. There's too much time being talked about research repositories. There are places where they're done well and there are places where they're needed. And I don't mean to suggest that there's not.

However, I think doing a repository properly takes a lot of work. It requires some very thoughtful curation taxonomy. It's just a ton more than people think from the get go. I had a conversation with somebody who was setting up or who set up a successful repository at a bank.

It took nine months. And a lot of companies like the freshness, if you will, of the research insights, it gets stale very quickly, right? And so I think research teams need to think about what kind of work that they do, right? There are places where creating these foundational knowledge and study documents make a lot of sense, but I think for the most part they don't.

I think for the most part, you want to be able to be right there in front of the decisions that are important that people need to make with, ‘here's some information that's gonna help you make the right choice’, and then just it's gone.

Alec: I think on the other side, there's a lot of hand wringing about AI and the large language models and stuff. I think the right take on this in the research space is if you're doing work that can be plausibly replaced by an AI model, you should think about that.

Because the reality is what researchers really bring to the table is critical thinking skills around how you analyze information and synthesize that into knowledge and transmit that to other people. And this large language model stuff – these are new tools for people who are thoughtful about what information matters, and how to synthesize that into knowledge.

These are new tools that are going to make you much more productive, right? And allow you to do a lot more. And if all the work that you've been doing is replaceable by these language models, then how much of that critical thinking stuff were you doing ahead of time, right? So to me, if you can imagine that, let's say someone invents a new tool that basically can do all usability testing, right?

If you're a researcher that was bringing a lot of critical thinking to the table, and that's why your skills are valuable. All of a sudden you can say, great, I'm going to set up all these usability stuff for all the products that we're building, all the features that we're building, and I'm going to let them run.

And then I can just double check the results, make sure they make sense, right? But then I can focus on something else, right? So free up your time to do different things. I think more researchers should learn about disruption theory and how that works because what we're doing is we're freeing up our labor to work on more highly leveraged bits of work when we have new technology that allows us to offload parts of our job.

Even though there's a lot of hype around the space, I think it's not quite fully understood. And I think that it's going to do a lot more for research than a lot of other fields.

Ian Floyd: There's a concept in finance of, when there's blood in the streets is when you invest – and, forgive the violent expression. But to compare that to research, you said that you see the research industry shrinking right now. As the economy is, let's say volatile or unpredictable, is that the wrong move? Should we be investing in research or at least becoming better at being more lean and efficient?

Alec Levin: Whenever there's change is a great time to invest in research, right?

So, for example, probably the most ideal time in the world to do a ton of research was at the beginning of the pandemic. Everything was changing. Everything that we have taken for granted. One of the things that was funny about the 2010s when I was doing this stuff is most people I would talk to would say, I don't need research. I already know what my users want, or I don't need to investigate, this is already understood.

And luckily, that's changed. It's gotten better. But one of the things that people forgot is like, the things that we take for granted is how we analyze the world. Like for example, a conversion funnel.

Somebody invented a conversion funnel in the SaaS context, right? The pirate metrics one and this one, and then that one, whatever, right? Somebody actually invented that. Where did that come from? That came from an understanding of the world informed by conversations and by observation about this framework.

A helpful quantitative way for us to measure what's going on in the world, right? And what happens is over time these frameworks become brittle, right? They're not easy to change and the world changes. And so all of a sudden they stop making as much sense, right? But they've been very useful for a long time, so we don't really want to throw them out and change them.

So when the beginning of the pandemic happened, everything was thrown out the window, right? Everything changed. And for every single business, you need to understand, ‘how are my consumer's lives changing?’ How are their purchasing behaviors changing? How are their lifestyles changing? How is everything changing?’

And so some companies adapted really well, and I think research had something to do with that. So if you think about Airbnb, international Travel went to zero, but local domestic travel, especially to rural areas, blew up. That was awesome. So how did they make sure that they were prepared for that?

When you have a lot of changes, it’s a great time to invest in research. However, that doesn't necessarily mean it's a great time to invest in hiring a ton of researchers, because I'm sure you've seen the thing where like the more people you add, like the number of lines of communications increases squared.

That's a real dynamic. And on top of that, are the people that you're hiring. They understand what the team needs to be successful, right? I think it has to do with some of research’s academic roots and some of the people who were early in this space 15, 20 years ago.

But we have to be comfortable talking about money. We have to be comfortable talking about margins. We have to be comfortable talking about numbers, right? Of course, we're very people-oriented, and that's super important. That's why we're great at our jobs. But to just ignore that because it, at times, makes us uncomfortable, means that we're not partners.

And that's one of the things that I think is one of the reasons why research has been hit so hard in these layoffs, which is, again: How many people in our community know what LTV stands for? I don't think it's 50%. And that is a travesty. That is one of the base numbers that you need to understand about a business.

What is the lifetime value of a customer. So that's where I think, great time to invest in research, for sure, when there's a lot of change. Definitely. But that doesn't necessarily mean that just adding more researchers to your team, especially if they don't understand the context of what you're trying to accomplish and how to be helpful, that might not make sense necessarily.

So I'm hoping a lot of the stuff that we're going to be doing over the next year, for example, will be making a lot of these concepts easier to understand, bringing in lots of examples, bringing in lots of stories so that people can get a better understanding of what is the mindset, context, goals, motivations of the people they work with so that they can be much more successful partners.

Angie Peng: I'm gonna reframe Ian's original question a little bit because he's talking about investing in research from the point of view of a company, but reframing that to, if I'm a person, who is in research – I'm a research manager. I'm a UX researcher. I'm looking to get into UX research. I think there's a couple things you've talked about where it seems like you would recommend investing in, right? Like understanding AI, understanding automation, how to free up higher level work for yourself. How to better understand the business, right? How to be a good partner, understand business metrics.

Is there anything else that you would recommend investing in if you were a researcher, or if you were a research manager, or if you're looking to break into UX research?

Alec Levin: Yeah – two things. So one is, I think, for sure, make sure you have the right tooling.

So obviously you guys make a great incentive product for getting people what they need and allowing you to work with a global audience. So, especially if you're working in that kind of space, for sure, whether it's the right tools to capture, original clips of the research sessions that you're running so that people can see what's actually going on – I think that's really valuable.

So there's lots of tooling like that. But then also, don't forget to invest in relationships, right? I think that's another thing that they don't teach you in school, but it's actually one of the most important things. It's, do you know people? Do they trust you?

And that comes from whether you watch the ball game with them or get coffee with them, or you watch the same sitcom as they do, and that's a really important thing to do as well.

Continue reading

Here are a few things for UX researchers and product teams to watch out for around planning, recruiting participants, and effectively moderating sessions.

September 11, 20245 min. read

Ux research