Wednesday, September 29, 2010

New Pricing Algorithms

Summary


After several months of effort, we've changed the way calculate we prices. For most products, the prices are close to what they were. For rare games and very popular games, the new prices better reflect the market. Here's a summary of the changes
  • a dramatically reduced weight for JJGames prices
  • more accurate prices for low-volume games
  • statistically confident prices for high-volume games
  • infrastructure changes to support more products (so we can add all PC Games eventually)
The full, gritty details are below.

Background

We collected our first price point in December 2006. At that time, VGPC wasn't even a glimmer in our eye. Collecting prices was an internal JJGames project to try and get a better idea what used games were worth. After using the data internally, we realized that others might want to see it too. When we launched VGPC in November 2007, we used the same pricing algorithm we had used from the start: an average of Half, Amazon and JJGames prices.
In February 2008, we started collecting eBay prices and added them to the average.

The following month, we recognized that using an average was inaccurate in many cases. We asked an economist friend for an easy way to improve the system. He suggested a two-tiered algorithm using the median price for products with 3 or more price points and the minimum price for products with 2 or fewer. We've used that algorithm for the last 30+ months, although the minimum rule rarely triggered.

In October 2009, we started discussing ideas for a more flexible pricing algorithm that adjusted itself to market conditions common with used video games. For example, there are lots of games that only sell a few copies each year. Other games sell dozens every day. Most games are somewhere in the middle. We wanted an algorithm that did "the right thing" for each of those classes of games to give the most accurate prices we could come up with.

Here's an explanation of how we achieved each of the main goals.

Lower Weight for JJGames Prices

One of our first goals was to give JJGames prices a much lower weight than eBay, Amazon and Half. JJGames does a much smaller sales volume than those three. It's also not a consumer-to-consumer marketplace like the others. The new algorithm only considers raw sales data from JJGames (never listing prices) and mixes those sales directly with eBay and private sales. We plan to add true sales data from other sources in the future.

The net effect is that a single sale on JJGames contributes substantially less than 1% to the final price of a game on VGPC, on average. This change has caused the VGPC price of some games to decrease which we think puts them closer to market value.

Collectible and Rare Games

Our second main goal was to price rare games more accurately. We're big fans of the rare game market and were disappointed that our site sometimes messed up when pricing rare, collectible games. Communities interested in artwork, baseball cards, Stradivarius violins and rare video games tend to price rare collectibles based on the most recent sale.

For example, there's only one copy of Nintendo Campus Challenge 1991. The last time it sold was for $20,100. Our old algorithm priced it at $14,000 which is obviously wrong. The new algorithm prices it correctly.

The trick was deciding when a game was rare enough to qualify for this algorithm. Some games have a low sales volume, but are readily available on Half or Amazon. We decided that a game averaging less than 1 sale per month with no available retail listings would be eligible. Currently about 4% of used games and 7% of brand new games are priced this way.

High Volume Games

Games that sell dozens of copies in a single day offered an exciting opportunity during the redesign: statistically significant market prices. In internal discussions, we had often wondered whether it were possible to have objective confidence about the prices we calculate. For most games it's not currently feasible, but for some high volume games it is.

This technique is only used for about 1% of prices (both used and brand new), so we won't belabor the details. The end result is that we're 80% certain that the price we calculate for these games is within 10% of the "true" market price. We also dynamically adjust between one, three, seven and fourteen days worth of sales to balance accuracy and recency. We choose the most recent sales data for which we can have the necessary confidence.

Typical Games

Most games are neither rare nor traded in high volumes. For these games, our old algorithm worked quite well. About 87% of used games and 71% of brand new games use the following algorithm now:
  • calculate the median sales price during the last 2 weeks
    • if there are no sales during that time, find the most recent sale
  • find the lowest listing price on Amazon
  • find the lowest listing price on Half
  • calculate the median of these three values.
It's worth noting that we do not include GameStop and JJGames listing prices in this algorithm. Those sites are retail venues with prices set by a single decision maker. Amazon and Half prices are established by a market interaction between buyers and sellers.

This algorithm uses the median heavily because the median is a robust way to measure central tendency. That robustness is very handy when dealing with sparse game data where outliers are likely. (Incidentally, the median is as robust as theoretically possible since it has a breakdown point of 0.5)

Although considering listing prices is not ideal, they're included because many games have lots of listings and the lowest list price is close to the sales price. Specifically, the list price is typically within 6% of the sales price.

No Price at All

Unfortunately, there's not always enough data to estimate a game's price at all. About 7% of used games and 20% of brand new games are in this category. As we continue to collect more sales and listing data, these percentages should decrease.

Scalability

While reworking the pricing algorithms, we also made some substantial architectural changes to our site so that we can handle higher sales and product volumes. Many of you have asked us to add all Mac and PC games to our list. These architectural changes were the last major hurdle holding us back.

Community Input

We've spent a lot of time thinking about, implementing, testing and adjusting these algorithms to produce accurate prices. Our goal has been to make prices that are useful for gamers and small game stores. We're very much interested in adding new pricing algorithms based on your feedback.

What do you think about the new algorithms?

9 comments :

GamesOgre said...

It's quite interesting to read about your methods for calculating the values of games.

What I'm also curious to know is whether or not you try to sort out NTSC vs PAL versions of games like Stadium Events or Little Samson. Since the NTSC version of each game is rarer, knowing the difference will strongly influence the price. However, with many sneaky eBayers trying to list games like these without stating that they're PAL (in order to get higher bids), I'm guessing it isn't easy to know off hand whether a piece of data you've gathered is realistic to the NTSC value. Have you come up with a way to overcome that challenge?

As always, loving your price guide, and I appreciate any improvements to it.

mndrix said...

@GamesOgre: For Stadium Events and Little Samson, we try to manually review the sales and classify them according to NTSC or PAL. We also rely on members of the community to click the "thumbs down" icon next to completed eBay sales listed on our site if a sale is incorrectly classified. Between the two, we try to correct as many problems as we can.

Anonymous said...

Thanks JJ!

This is a good change.

This site is awesome and I can't thank you enough for running it!

Keep up the good work!

Anonymous said...

how much can i get PSP new edition for my son?

Anonymous said...

These prices are somewhat right. Nov. 2010 you listed Snatcher at 155. I saw that at the time, there were four listings of this game on ebay. Around 200-220 and one listing at a buy it now price of 500. I paid 219 on an auction, how accurate is that Nov price? I noticed a month prior it was valued at 202.50, so that was a steep depreciatation.

JJ Hendricks said...

@anonymous - In November there have been 2 sales at less than $140. Someone on Amazon is listing a copy for $99. So all of these make us think the price as decreased.

You can see the completed auctions by clicking the link next to the "Volume" on this page:
Snatcher for Sega CD

Mike said...

This is a great site and a wonderful resource, thank you!

Have you ever considered adding a "Collection valuation" feature? As a collector I would love to be able to use your algorithm to estimate the value of my collection.

Granted it wouldn't be super accurate because there is no tracking done on with or without boxes/instructions on the older games but it would still be valuable information.

Thanks again!

JJ Hendricks said...

@mike - Great idea. We have considered that "collection valuation" idea and actually have it on our list of features to add to the site.

Hopefully we can implement it sometime in the future and will be sure to post about it on the blog when we do.

Unknown said...

I can't wait!

I was also wondering if you had ever done some statistical analysis on boxed copies versus non-boxed? I'd be curious how much a box and instruction manual contribute to the overall value of a game. Is it 25% over the cost of the cart itself or as much as double?!

I realize it's very different game to game and the auction titles can be misleading but it seems like you have the raw data to figure out a rough average, at least on a per system basis.

Post a Comment

ShareThis

 

Login | Create Account