Actually, the prices are not downloaded in that sense. They are based on the search results that we receive for the Buy Parts/Multi-Buy pages during the last 24 hours. If a part has not been queried via that method it will not have any BL price data included in it's average calculation. At least that's for BL parts, the averages also include BO and ToyPro data which is available in full every day but there are far fewer stores for them.
Thanks for that. It makes sense that the 6 months average prices would be just as slow for RB to pull as it is for me. But if the RB "total value" figure could be calculated on the *used* for sale prices instead of new (or if that was an option the user could set), I think it would make that figure a more interesting for most part collectors.
Doesn't make sense to run the price check for every part every day. Maybe for "new" part/color combos just added otherwise spread it out over a week for older parts/sets which are unlikely to change drastically fast. Personally I could run my RB site completely without price info, because I don't care about it. I am sure there is people who do. but if I could customize my settings to not load that info and get a faster experience that would be great, I could always turn it back on the few times I might want to see it.
The problem is what you already encountered. It's a heavy load on BL to go by all parts. The for-sale prices are easy available in a download.
Some people uses their store as their collection database. Or they are hoping for someone who doesn't know how the site works.
It would be great if the total value was based on used prices instead of new (or if that were an option the user could control). I have a script that will go fetch all the 6-months average used prices for my parts lists and calculate a more realistic value, but it's slow to run and i'd love for RB to do that for me.
Also, well done on excluding the outliers. I really wonder about the business model of some of those stores . . .