To get all episodes of ‘What The Research’ podcast directly in your inbox, join the behaviorXproduct newsletter by clicking here.
A couple of months ago, I came across an amazing nudge database of research papers made by Mark Egan at the Stirling Behavioural Science Centre. I spent some time converting it into a searchable Notion database to help other researchers benefit from it. However, one thing that struck me was that almost all research papers only had successful nudging interventions. The researchers aimed to create a behavior change and were able to achieve it using a specific combination of nudges.
And that got me thinking – Do nudges always work and when they don’t work, is the research still valuable?
Hello and welcome to another episode of What The Research podcast. I am your host Rohit Kaul. I am a marketer and behavior science enthusiast.
In this episode, I will focus on a research paper where the researchers ran five large-scale field experiments to change the commuting behavior of a company’s employees. They used behavioral interventions to nudge them to environmentally sustainable behavior. And it did not work.
The paper is titled “What we can learn from five naturalistic field experiments that failed to shift commuter behavior” and was published by Ariella S. Kristal and Ashley V. Whillans, from the Harvard Business School. I must thank the researchers here for documenting this field experiment where behavior change nudges did not work, as on many occasions we learn a lot more from stuff that doesn’t work compared to the stuff that works.
So here’s what the field experiment was all about.
The company was an airport in Europe with over 75,000 employees. The researchers aimed to examine if they can use nudges to successfully reduce single occupancy commutes to and from the office. In a previous survey, 49% of employees had reported that they drive to the office alone in a car and 61% of these commuters had mentioned that they can consider carpooling for the commute. The company already had a carpool matching process that the researchers leveraged in the study.
The researchers undertook 5 different field experiments. Of course, the employees did not know that they were part of an experiment.
In the first experiment, they aimed to increase the number of registrations for the carpool service. They worked with two hypotheses here:
- Reducing friction encourages follow-through and use of a service
- Social proof improves compliance with sustainable behavior
To achieve this they sent letters to more than 54000 employees. The employees in the control condition did not get a letter. In treatment 1, they got a standard letter with information about carpooling service, in treatment 2, they reduced the friction by prominently featuring a link to the form where employees can register for the carpool service. In treatment 3, they added testimonials from two other employees with their images.
However, between those who got a standard letter and those who got behaviorally informed letters, there wasn’t any difference in signing up for the carpool service.
In the 2nd study, they tested two other behaviorally informed interventions on employees who were already registered for carpool service but were not active – personalized recommendations and opportunity cost reminders.
This is because in the survey, that I had mentioned earlier, a majority of employees had mentioned that they will undertake carpool if they found the right match for it with people in their shift and commute route.
Also, judgement and decision-making research shows that people do not spontaneously recognize opportunity costs and it has to be made salient to them to make the right decision.
The researchers sent out emails to 871 employees with the control group getting the standard mail, treatment 1 getting a mail with featured personalized carpool matches and treatment 2 getting a mail with these matches and money they will save when carpooling. But only one employee from this study became an active carpooler.
In the next study, they attempted to increase the use of public transportation by offering financial incentive of using one-week free bus trial to 7564 employees who did not use bus for commuting but lived near the bus routes. In the control condition, the employees received a letter that told them about bus routes near them, as well as how to purchase discounted transit cards through their employer.
Employees in the treatment group received the same letter, along with vouchers for a 7 days free bus trial. But the researchers did not see any statistically significant difference between the people who did not get a discount card and those who got one.
The 4th study was a follow-up to the previous one aimed at those recipients who did not use any of the free vouchers during the trial week. The control group received no follow-up letter and the treatment group received a letter that highlighted the cost of the free trial that they had missed out on, along with information about how they could still take advantage of discounted travel leveraging the loss aversion bias. But the effect size was negligible, indicating that not using the free trials did not amount to a loss for the employees.
The fifth and last research focused on the nudge of creating a plan. They emailed employees in the treatment group with a personalized travel plan that included options to carpool, discounted transit passes, public transport routes, etc. in one single mail. The control group did not get any email. The researchers found no impact of personalized travel plan on commuting behavior.
So the authors tried nudges based on improving salience, social norms, reducing friction, personalization, making opportunity cost salient, financial incentives, reminders, and creating a plan. And none of these made any significant impact in the employees’ commuting behavior.
After going through this research paper, I had three observations.
Firstly, one of the core premises of nudging the employees to environmentally sustained commuting behavior was based on a survey where many of them said that they could consider carpooling, finding a carpool match is a key barrier and they prefer to use a sustainable commuting alternative. This is a classic case of stated preferences vs true preferences that many consumer surveys suffer from, especially when the survey is about the environment.
People say things that will make them look good to others or even to themselves without sharing their true intentions. Market researchers are acutely aware of this issue and hence tend to make surveys double-blind where the respondent and the surveyor don’t know what is the real intention behind the survey.
Secondly, the research is based on identifying specific nudges from literature and applying these in this experiment. The stage of deep-diving into the minds of the employees hasn’t been covered in the research paper.
In a very interesting research paper, titled Budging beliefs, nudging behaviour, authors argue that the success of an intervention depends on understanding people’s current behaviour and beliefs to ensure that any nudge will actually “budge” them from their current beliefs. They propose a BBC model for this, where BBC stands for Belief – Barriers – Context. In fact the authors of the commuting research paper also mention future research should target other barriers that prevent employees from carpooling and which they might be more reluctant to admit on self-report surveys.
The third thought is that while the nudges struggled to make employees adopt a carpooling commuting behavior, ridesharing companies like Uber and Lyft have successfully created carpool products called Uberpool and Lyftline which are big revenue earners for them. So I was curious to know how Uber and Lyft are able to get 3 unknown people in a car when in this research accurately targeted nudges failed to do so for a group of people who all work at the same place and belong to the same company.
The answer may lie in the BBC model. I looked at a bunch of different research papers and passed these through the BBC lens and here’s what I think. Of course, these are my views and not of the researchers and havent been validated by any sort of experimentation.
So the first B is Belief. One of the research studies done in the US at the Michigan State University on carpooling highlights the fact that people who were aware about carpooling service and wanted to use the service were worried that they will end up with someone very awkward in the carpool and then will have to face them in the campus daily. Thus, they wanted to have more information about their carpool partners before signing up for carpool. This belief doesn’t impact how Uber or Lyft work because people rarely end up with same set of people again on carpool and commuters dont belong to one company or one university, making it easier to get through the ride. This research found that once carpooling mixers were organized, more people started using carpool.
Second B is Barrier. One of the biggest barriers regarding using any sort of public transport is uncertainty – What route to plan, how much time it will take, will I reach my destination on time. In fact Uber did a research that showed that for Uberpool, the top three important features for consumers are ETA, On-demand availability, and on-trip duration. All of this point towards the barrier of time uncertainty. Uber is able to overcome this barrier by showing you the ETA, the trip duration, and the cost right when you are making the booking.
C is for Context. I found this particularly interesting and something Uber has really cracked. When you open the Uber app, it shows you the amount of money that you will save if you use Uberpool vs Uberx. This is one of the biggest last-minute nudges to the users to book Uberpool. In the commuting study, the researchers did make the opportunity cost of not using carpool salient but it did not work for them.
This is because of the context in which this information is consumed. When Uber shows you the opportunity cost of booking Uberpool, you are ready to commute and looking for the best way to reach your destination. However, when the employees in this research read the opportunity cost messages, it was in emailers and most likely they read it sitting comfortably in their office chair. I wonder what would have been the impact of this message when it was delivered to them early in the morning as they were planning to start their commute, repeatedly for a few days showing them that with every passing day when they did not use carpool how much money they have foregone cumulatively.
So, in summary, I strongly believe that while nudges do work but it’s important to first have a strong handle on the BBC Beliefs – Barriers – Context and leverage these to identify the right nudges and the right way to deliver these at scale for behavior change.
Hope you liked this episode of What The Research. If you haven’t yet subscribed, please consider joining the mailing list that will get you not only the new episodes from the podcast but also my behaviorXproduct newsletter directly in your inbox.
Till next time. Take care. Stay safe, stay healthy.
Links to the research papers:
What we can learn from five naturalistic field experiments that failed to shift commuter behavior: https://www.researchgate.net/publication/338128319_What_we_can_learn_from_five_naturalistic_field_experiments_that_failed_to_shift_commuter_behaviour
Budging beliefs, nudging behaviour: https://link.springer.com/article/10.1007/s11299-019-00200-9
The Perfect uberPOOL: A Case Study on Trade-Offs: https://www.epicpeople.org/wp-content/uploads/2018/12/Paper-5-2-Lo-Morseman.pdf
Carpooling study at the Michigan State University: https://www.liebertpub.com/doi/full/10.1089/sus.2017.0020