HangryDave

Not receiving full JSON set inventories from API?

Recommended Posts

Hello, I use the Rebrickable API for an app that downloads a set inventory(using/api/v3/lego/sets/{set_num}/parts/) and it seems that recently it either has been reading the JSON wrongly(unlikely, I tested in several ways) or hasn't been receiving the full JSON file. I tested it on the API documentation where it lets you try it out, and it didn't work there either. To make sure it wasn't new code that was breaking, I tested an old version that I knew worked, and it had the same problem.

Here's the file that I receive for set 7965-1: https://pastebin.com/xVwwJWcQ

(I would have included a linted version, but that takes up too much room, sorry.)

I checked against the same inventory downloaded a few months ago to test, and the old one was indeed longer: https://pastebin.com/zvDxWrVr

I'm not sure what the problem is. Does anyone have any ideas or is anyone able to confirm the issue?

Thanks!

Share this post


Link to post
Share on other sites

HangryDave,

I had the same problem (also remarked in your mentioned thread from feanferr) and I can only recommend to ALWAYS inspect the value of the "next"-field in the JSON-result:

{ "count": 263, "next": "https://rebrickable.com/api/v3/lego/sets/7965-1/parts/?page=2", "previous": null, "results": [ {...

It is possible, even if you use page-size of 1000, that the API doesn't return you 1000 but for example only 100 (it's per configuration and depends on performance and can even be changing dynamically).

So to receive the complete result you've got to call the url given by next (if it isn't null) and collect the results until you have next == null. This is the nearly the only way to determine that you've got the full result. And because this is valid for nearly all GET-API-methods, you should try to handle it in general on all your API-GET-calls. The other way to determine is to count your number of results received (the count of elements in result-array) and compare that to the number given in the "count"-field at the beginning. And then you can use next-URL to receive the next page.

 

 

Share this post


Link to post
Share on other sites

yep,  just keep on pulling till the next param gives you a null value.  So similar to this (it is essentially how I do it)

while (Next != null) {	
	url = Next
	Result = API_Call(url)
	...Process the result
	Next = Result.Next
}

Basic psudo code, but you should get the idea,  and this will work no matter how small the page size gets.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now