Consumers rely on online ratings to guide everything from restaurant choices to furniture purchases. One review might not be trustworthy, but when websites average their ratings together, the thinking goes, its score should reflect the “wisdom of the crowd” and accurately capture its quality.
But research by Balázs Kovács, an assistant professor of organizational behavior at Yale SOM, and his collaborators suggests that this system contains a flaw. According to their analyses, products or businesses that initially receive poor reviews are likely to get fewer ratings in the future. Since the rest of the “crowd” doesn’t weigh in, the item may be stuck with an unfairly low score. It “might never be corrected,” Kovács says.
“If there are few reviews, the score is not only inaccurate but is biased in a systematic way,” adds Gaël Le Mens, a professor of behavioral science at the Universitat Pompeu Fabra, who collaborated with Kovács, along with Judith Avrahami and Yaakov Kareev of the Federmann Center for the Study of Rationality at the Hebrew University of Jerusalem. “There is a systematic tendency for such scores to be underestimations.”
To understand the team’s reasoning, consider the following scenario. Two new restaurants of equal quality open. An accurate rating for both would be, say, three stars. The average of the first few reviews will likely not be exactly three stars since the sample size is small. By chance, restaurant A might get lucky and average four stars after the first week, while restaurant B gets unlucky and averages two stars.
What happens next? Diners who read online ratings will likely choose restaurant A instead of B. Restaurant A accumulates more reviews, and the rating eventually converges to an accurate average of three stars. But since few people try restaurant B, it gets stuck at two stars—even though it’s as good as A.
The rating isn’t the only problem, Kovács says. Review sites often order options from highest to lowest scores. If your business is poorly reviewed, “you go to the bottom,” he says. “People are not going to choose and rate you.”
To find evidence for this pattern, the researchers analyzed 78 million Amazon ratings and 2.2 million Yelp ratings. Highly rated products and businesses accumulated reviews more quickly than poorly rated ones did, they found. If an item’s score increased by one star, it received reviews an average of 16% faster on Amazon and 14% faster on Yelp.
To demonstrate that this leads items with few reviews to be underestimated, the team created a computer model. The model simulated the process of users selecting items to rate, based on their current scores, and submitting more ratings. As expected, an item that started out poorly was likely to remain stuck with a score that was lower than its true quality.
There’s no need for businesses to plead for high ratings. “If you just tell them, ‘Write an honest review,’ it will help.”
Next, the researchers conducted an experiment in which participants rated pictures presented on their computers. First, the participants were shown 50 buttons, each corresponding to a picture, and had to choose 10 to view and rate. For some participants, the buttons were ordered so that those corresponding to pictures with the highest ratings were at the top of the page, while lower-rated ones were near the bottom. Previous research suggests that in this situation, people are more likely to pick options near the top of the page, and thus pictures with the highest ratings would receive the largest quantity of additional ratings. This is what happened in the experiment. And just as the model predicted, pictures with few ratings tended to earn average scores that were lower than their “actual” quality, as determined by other participants rating pictures presented at random.
The team also demonstrated they could reverse the bias by displaying the buttons in the opposite order. For some participants, the buttons were ordered from lowest- to highest-rated. In this case, pictures with few ratings got average scores that exceeded their actual quality.
Finally, the team studied ratings of iPad cases on the French and German Amazon sites. The products were exactly the same, but ratings differed depending on the site. By comparing the data, the researchers could isolate the effect of the number of reviews. They found that having 10 times more reviews was linked to an average score 0.35 stars higher.
The study suggests that customers shouldn’t dismiss low-rated items with few reviews. “You’re likely to be positively surprised,” Le Mens says.
The results also suggest that businesses that start off with bad reviews can counteract the effect by asking more customers to review their product or service. There’s no need to plead for high ratings, Le Mens says: “Even if you just tell them, ‘Write an honest review,’ it will help.”
Online review sites also could take steps to counteract this bias, the researchers say. For instance, the sites could avoid displaying average scores for products with only a few reviews. And instead of ordering options based strictly on ratings, they could randomize the order a bit. Or the sites could identify items with few reviews and encourage users to rate them. “That’s one of the ways to fight this,” Kovács says.
Is a $15 minimum wage enough for Amazon employees to be financially secure?
No. For a full-time employee, $15 an hour works out to $31,200 annually, which definitely does not provide financial security, particularly in the urban labor markets in which Amazon tends to operate.
What is the likely effect of this change on Amazon and its workforce?
Amazon is likely to have an easier time recruiting and retaining entry-level people, but it is not clear what will happen to those who are a bit above the new minimum wage, who have seen their relative standing eroded somewhat by the bottom being raised. In research my colleague Amy Wrzesniewski and I have done on a firm that did something similar, they saw increased turnover among employees who were exempt from the pay change, who were presumably disgruntled about their loss of relative status within the firm.
Are other employers likely to follow Amazon’s lead?
Not all firms have been subjected to the bad press about labor issues that Amazon has, and so may not be so inclined to jump on this bandwagon. On the other hand, Amazon is presumably making this change in part because they are finding it difficult to recruit and retain people in a tight labor market, and their competitors (a number of whom have already raised minimum pay) may be forced by the labor market to comply. And having now decided voluntarily to go to $15 an hour, Amazon now has a strong interest in using its lobbying power to try and get the minimum wage raised so as to impose this burden on their competitors.
The UN Intergovernmental Panel on Climate Change’s October 2018 report offered a stark new warning. Many of the catastrophic impacts of climate change previously thought to be distant and theoretical may arrive by 2040, absent radical change. Myles Allen, an Oxford climate scientist and one of the report’s authors, told the New York Times, “It’s telling us we need to reverse emissions trends and turn the world economy on a dime.” A few weeks later, the Trump administration released the congressionally mandated National Climate Assessment, which concludes that the effects of climate change could reduce U.S. GNP by 10% by the end of the century.
To avoid impacts that would include inundated coastlines and intensified droughts and wildfires, as well as expanding poverty and food shortages—damages forecast to cost $54 trillion—the shift to a low-carbon economy must happen quickly, something the UN report describes as technically possible if politically improbable. One path to reducing carbon output: grow renewable energy sources from 20% of electricity generation today to 67% by 2050 while dropping coal from nearly 40% to under 7%.
The cost of renewable energy continues to fall. The technologies and tools for a smarter, more resilient power infrastructure shimmer tantalizingly close, yet we aren’t seeing industrial-scale implementations funded by private-sector investors moving promising pilots to ubiquity. In the past, this type of funding gap has prompted public-sector intervention to help innovation over the hump. In some instances, the intervention was through policy, in others through subsidy or even direct investment.
In this case, one intervention is already underway, at least on the state level: the creation of green banks to facilitate financing of low-carbon infrastructure.
Connecticut created the first such bank in the United States in 2011. Formed as a quasi-public agency, it was tasked with making clean energy more affordable and accessible. The state’s budget crisis led to a drastic cut in the green bank’s budget in 2018, which led the organization to spin out a nonprofit affiliate that is able to tap different sources of funding. The aim is to continue as much of the previous work as possible, though it will be split between the two entities.
In 2014, New York created a green bank using a different model—a state-sponsored specialized financial entity with a $1 billion capitalization that provides loans that accelerate clean energy deployment. As Richard Kauffman, chairman of energy and finance for New York state, explains, “The objective of the New York Green Bank is to operate one standard deviation away from where a private-sector financing market exists.” Charging market rate for loans, it targets projects that have viable financials but aren’t able to secure loans. The aim is to demonstrate the viability of the market, which in turn will encourage the private sector to bring its much deeper resources to bear.
Yale Insights talked with him about the goals, challenges, and tradeoffs in the various models of green banks.
Q: What need does NY Green Bank meet?
NY Green Bank is a signature policy of Governor Andrew Cuomo. It’s not really a bank per se, but a specialized finance company. Because of bank capital rules and other regulatory issues, it’s very expensive for traditional financial institutions to offer loans to many types of clean energy projects. That’s where NY Green Bank comes in. It offers market-rate loans where the problem is availability of financing.
NY Green Bank has been operating for nearly four years and has extended over $522 million in credit. We believe those funds have been leveraged more than three times over. So, there are now over $1.7 billion in clean energy projects in New York that would not have otherwise happened.
Q: What type of projects get funding through NY Green Bank?
Most are smaller solar or energy efficiency projects. To offer a specific example, there’s a very robust energy efficiency market focused on educational institutions and hospitals. For no money down, service providers do energy efficiency upgrades for these institutions. Over a seven-year period, the service provider recoups its costs and makes a profit while the institution gets a share of the resulting energy efficiency savings.
Why seven years? Banks are reluctant to lend for longer than that. In some cases, seven years is fine. In other cases, a longer time period would make deeper retrofits cost effective, so NY Green Bank offers loans of 8 to 15 years.
There’s a market for this type of longer-dated paper among institutional investors. Once we reach the scale to package these loans, that will let us relend the money to other projects.
Q: What are the policy goals NY Green Bank aims to support?
Governor Cuomo saw that the existing energy system is not financially sustainable. A million New Yorkers had difficulty paying their power bills. Because of shale gas, the cost of producing power is down, but the cost of distributing it to customers continues to grow, so power bills continue to increase.
At the same time, the cost of alternatives, such as solar panels, batteries, and fuel cells, are declining. Those with the wherewithal to have a distributed solution will take advantage of those opportunities, leaving the cost of the existing infrastructure to be spread over fewer and fewer customers and creating inequity.
Beyond those cost and equity issues, the governor sees economic opportunity in the transition to a clean energy economy. The shift to a lower-carbon energy system is happening. That’s not to say that the energy system will be entirely distributed solutions, but we can certainly expect a mix of large-scale generation and distributed solutions which will give communities, companies, and individuals more choice.
We’re using state authorities to advance our policies; we’re not looking to the federal government. Much of our strategy was implemented before the Obama administration announced its clean energy policy. Additionally, we’re not changing our policy because of the Trump administration.
Q: What are the political dynamics around green banks generally?
Green banks can appeal to both the left and the right because they offer a way to support the clean energy economy and enlist the private sector. However, green banks can run the risk of falling between political constituents. Some conservatives ask if green banks really do something that the private sector wouldn’t otherwise be doing. They may object in the same way some conservatives have objected to the U.S. Export–Import Bank.
“We say the objective of NY Green Bank is to operate one standard deviation away from where a private sector financing market exists.”
On the other hand, some environmental groups have seen the funding of green banks as taking money away from one-time grants for renewable energy or energy efficiency projects. Many people on the left get nervous about relying upon market-based solutions. They like that grants are typically linked to particular mandates—this is how much solar power we’re going to deploy. Environmental groups like the mandate and the funding for grants to achieve the mandate.
There is a robust debate about financing the transition to clean energy. Where would subsidies be useful? Where can private sector investors move things forward?
My own perspective is that the key challenge is around scale. Institutional investors are looking to allocate hundreds of millions of dollars. It’s very difficult to bring together hundreds of millions of dollars of clean energy projects, especially when they need to be structured similarly enough to be packaged for sale to institutional investors.
To me this means we need policies that encourage more projects and entities like a green bank that can help aggregate smaller projects. If the green bank can prove that there is an attractive opportunity, the private sector will step in to scale it. Then we solve both parts of the problem.
We say the objective of NY Green Bank is to operate one standard deviation away from where a private sector financing market exists. Most private sector companies, given a choice, do not want to be in partnership with government. If the government is involved in financing and offers any subsidy, there will always be questions of adverse selection. Maybe they can’t get financing because there’s something fundamentally flawed in the project. But since NY Green Bank is charging market rates, it’s hard to argue that the green bank is not creating loans that wouldn’t have happened otherwise.
Charging a market rate of return, rather than subsidizing projects, is also important in the sense that as long as we’re paid back on our capital, we have a self-sustaining institution.
Q: The Connecticut Green Bank has been credited with sparking the green bank movement, but it also ran into trouble. What lessons can you draw from that experience?
Connecticut’s green bank enjoyed bipartisan supported when it was created. The state’s budget crisis led the legislature to raid funding that had been allocated to the green bank to solve the state’s general budget problems. At a critical moment, the green bank didn’t have legislators on either side of the aisle who were willing to fight for it.
It’s possible that part of why it became a political orphan is that it was part of a set of policies aimed at enabling clean energy markets, as opposed to being the market. The Connecticut Green Bank was helping private sector banks to reduce the cost of financing. The end customers getting the loans for renewable energy or energy efficiency projects didn’t necessarily know that they were actually getting a benefit from the green bank.
Another issue is metrics. The Connecticut Green Bank was measuring its effectiveness in terms of greenhouse gas reductions relative to traditional grants. By that measure it was much more effective than traditional grants. But since it wasn’t set up around financial returns, it was dependent on continued annual funding. When the annual funding was reduced, the overhead of the bank meant it couldn’t continue to execute on its strategy and had to radically restructure.
When you add these things together, we don’t really know what the legislators would have done without the overall budget crisis. One takeaway is, even with very good policy, don’t forget the politics.