ChubbyArse

Invalid Token when requesting next page

Recommended Posts

Hi all, I'm having some trouble calling the API and some strange behaviour which I can't explain - so I hope someone can help.

I am retrieving parts for a set, and limiting each request to 10, and then whilst there is a `next` value returned - then waiting a second and calling again with the `next` url. This only works for a couple of iterations, when (and this is where it gets interesting) I start to get back a `401 { detail: "Invalid token." }`. So for example, the app process goes:

 

https://rebrickable.com/api/v3/lego/sets/76025-1/parts/?page=1&page_size=10 - 200 with results
https://rebrickable.com/api/v3/lego/sets/76025-1/parts/?page=2&page_size=10 - 401 Invalid token

However, I can then call the following:

curl -X GET --header 'Accept: application/json' --header 'Authorization: key XXXXXXXX' 'https://rebrickable.com/api/v3/lego/sets/76025-1/parts/?page=2&page_size=10

Then I can run my app process again and I get the result:

https://rebrickable.com/api/v3/lego/sets/76025-1/parts/?page=1&page_size=10 - 200 with results
https://rebrickable.com/api/v3/lego/sets/76025-1/parts/?page=2&page_size=10 - 200 with results
https://rebrickable.com/api/v3/lego/sets/76025-1/parts/?page=3&page_size=10 - 401 Invalid token

Which I can then curl for page 3 and then my app will work for the first three pages and then so on. 

So, please bear with me on this paragraph, it gets confusing.... Even more interesting is that my app process may get to page 5 (with the help of running curl commands to advance it) and then barfs on page 6 - then if I curl for page 7 (skipping 6) - the process still barfs at getting page 6 - then if I curl page 6 - the process will then barf on page 8 - as page 7 was curled prior to curling page 6.

It like the curl method works and then the result is cached for the url, and then when the app process runs, it is allowed any cached results, but then barfs on any new ones.

My app process is NodeJS using the request npm module and setting the Accept and Authorization headers - I've copied in the code below (getAsync is a wrapped request.get method)

Any ideas on this - it's a bit odd! 

Thanks

async function getSetPartsAsync(setNum){

    let ordinal = 0;
    let options = {
        url: `https://rebrickable.com/api/v3/lego/sets/${setNum}/parts/?page=1&page_size=10`,
        json: true,
        headers: {
            'Accept': 'application/json',
            'Authorization': 'key ' + process.env.rebrickableApiKey
        }
    }

    do {
        
        let result = await getAsync(options);
        if (result.statusCode !== 200){
            console.error(result.statusCode);
            console.error(result.body);
            break;
        }

        options.url = result.body.next;

        result.body.results.forEach(result => {
            console.log(`${++ordinal} - ${result.part.name} | ${result.color.name}`);
        });

        await new Promise((resolve) => setTimeout(resolve, 1000));

    } while (options.url);

}

 

Share this post


Link to post
Share on other sites

Not sure because I do it in a complete different env (C#).

But I think (based on other examples I've seen for npm request) that your Authorization-header must be like this:

let options = {
        url: `https://rebrickable.com/api/v3/lego/sets/${setNum}/parts/?page=1&page_size=10`,
        json: true,
        headers: {
            'Accept': 'application/json',
            Authorization: 'key ' + process.env.rebrickableApiKey
        }
    }

So it seems that Authorization is treated as a special kind of header in NPM request (se for example here https://stackoverflow.com/questions/30137231/npm-request-with-headers-and-auth/30137343)

Maybe that helps...

Share this post


Link to post
Share on other sites

So, I need to hold my hands up in shame - there was nothing wrong with my code - the rebrickableApiKey process env variable tucked away in my launch.json file was wrong - a rogue 's' had found itself on the end of the actual key (I blame switching between macs and windows keyboards... ;-) )

Anyway, it has highlighted a potential security flaw - it would seem that should a url be retrieved and presumeable cached - then another request for the same url, but an invalid api key - is returned without checking the key. It's easily done by running the code in the original post with a valid key and then running it again with an invalid key.

Hope it helps someone.

Share this post


Link to post
Share on other sites
2 hours ago, ChubbyArse said:

So, I need to hold my hands up in shame

Something I never have had to do... ;-) 
Try debugging a code in a file while you do the test in a different file, and wonder why the fixes won't help.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now