“calibrating for expectations changes the interpretation”
My family decided to plan a get-together with extended family this summer, and as inevitably happens, we entered a familiar round of the “hot potato” game (where everyone tries to pass off an undesirable task to someone else) regarding who will be responsible for handling the logistics of finding and reserving a place. I found myself holding the hot potato when the buzzer sounded, and so started on the process of finding a rental that would meet everyone’s desires and specifications. We were looking for a small cabin-esque atmosphere, only complicated by the fact that it needed to house 12 people (including a large handful of hyperactive children.)
After spending some time on the frustrating websites of a handful of New Jersey campgrounds, I eventually gave up and made my way back to AirBnb. I was pleasantly surprised by the offerings that came up, managing to find not one, but two rentals that would fit our criteria and were within a reasonable budget.
Option 1 was for approximately $500/night.
Option 2 was for approximately $400/night.
The price difference wasn’t a deal breaker of any sort, but it did come with the interesting caveat that Option 2 was at a reduced cost – we could see in its history that it was typically priced at nearly double, closer to $800. For whatever reason, the dates we selected were at a steep discount.
Now any good economist would tell you that the discount doesn’t inherently matter – you look at the facts in front of you. Given the value and cost of Option 1, and the value and cost of Option 2, which do you find more attractive?
In order to determine the value, one proxy that makes sense to use is the AirBnb rating, left by previous renters. Both spaces had near-identical ratings – 4.93 and 4.91, respectively. Not enough to show any type of meaningful, measurable difference.
Taking these ratings at face value, it would seem that the quality of the AirBnbs can be assumed to be similar.
But I realized that the ratings could not be taken at face value, for the simple reason that people are too clever for their own good. Or at the very least, we know to place things in context.
All of the renters who left reviews on Option 1 did so under the context of how much they were paying. They were asking themselves, “At a $500 price point, how do we rate this AirBnb?”
Option 2 was at a steep discount, and was typically priced at $800. This means that renters for Option 2 were asking themselves a different question: “At an $800 price point, how do we rate this AirBnb?”
Obviously, these are different questions. The more you pay, the higher the expectations.
Not all 5-star ratings are equal.
At an extreme, you can imagine that booking a $40 room in a basement (which I did once in Colorado) does not come with the same expectations as booking a $5000 villa (which I admittedly have yet to do). Even if both have the same 5-star rating, nobody expects the quality to be the same.
And so, when we take the ‘normal’ price points into consideration, it becomes clear that the 4.9 rating of Option 2 is almost certainly higher in real terms than the 4.9 rating of Option 1 – calibrating for expectations changes the interpretation.
This principle applies in many other situations beyond vacation rentals. Where else might we see this?
Asking a rich friend if they thought a restaurant was expensive compared to what they’re used to – it might not be to them, but if you asked a friend who’s less well-off, you might receive a different answer.
In business, if you ask somebody how satisfied they were with the quality of the service they received, this will be intrinsically tied to the price point as well. If I go to a really fancy event, I might expect valet service when I drop off the car. The same will not be true when I go to McDonald’s.
Just like with the AirBnb ratings, realize that almost every piece of information in the form of an opinion that we receive is grounded in some kind of context, and being able to recognize that context and work backwards can help us come up with our most optimal answers.
Leave a Reply