Summary of post

Problem: We have a list of items each with a rating by users. We want to put the highest-rated items at the top and lowest-rated at the bottom. We need some “score” to sort by.

correct solution

#### Wrong solution 1

score = (positive ratings) - (negative ratings)

Magnitude (# of ratings) outweighs percentage of ratings.

#### Wrong solution 2

score = average rating = (positive ratings) / (total ratings)

Percentage of ratings outweighs magnitude of ratings.

#### Correct solution

score = lower bound of Wilson score confidence interval for a Bernoulli parameter

Balances the proportion of positive ratings with the uncertainty of having a small number of observations.

Given the ratings I have, there is a 95% chance that the “real” fraction of positive ratings is at least what?

insert complicated equation here

A bunch of implementations

## Other applications

Useful to know with confidence what percentage of people took some sort of action. Examples:

• detect spam/abuse: what percentage of people who see this item will mark it as span?
• “best of” list: what percentage of people who see this item will mark it as “best of”?
• “most emailed” list: what percentage of people who see this page will click “email”?

In top rated lists, it can be useful to sort on positive ratings per page view, download, or purchase, rather than per rating, since many people will not rate if satisfied with an item.