djm

Rebrickable part without LDraw equivalent part?

Recommended Posts

A question related to the same work identified in this post.

I'm looking at the Rebrickable external ids for LDraw parts. I was wonder the following;

  1. If a given Rebrickable part does not explicitly list an different external id for its equivalent LDraw part, does this always imply that the LDraw part id is the same as the Rebrickable part id?
  2. Are there any Rebrickable parts which do not actually have a corresponding LDraw part (for example, this)? If so, is there anything in the API result for the part which means there is no LDraw part (as opposed to it having the same part id, and hence not listed in the external ids)?

Regards,

David

Share this post


Link to post
Share on other sites

Hi, David;

> If a given Rebrickable part does not explicitly list an different external id for its equivalent LDraw part, does this always imply that the LDraw part id is the same as the Rebrickable part id?

That's what the system assumes. There will be no translation for import or export for that part. As to the LDraw image, I have rendered some 14,363 images in a default color (Unknown color: Dark Blue) based upon the official and unofficial LDraw libraries, with some third party additions. If we have an LDraw image in the right color, that image will be displayed. If not, the system reverts back to the Unknown color image. If that doesn't exist, No Image Available is used.

> Are there any Rebrickable parts which do not actually have a corresponding LDraw part

Oh yeah, way too many! -smile- Again, I am not sure if you can get that info using the API, but I can get it, if you need it. Besides parts that are not yet defined in LDraw, there's also a whole bunch of LDraw parts not used in Rebrickable, as LDraw sometimes uses different part names then we do. Whenever we run into those, we can add a mapping to make the connection, but I am sure there are hundreds of part currently not used.

Take care,
Simon

Share this post


Link to post
Share on other sites

Thanks Simon,

that tells me that I will need to do some kind of comparison to the LDraw part ids to confirm whether or not the LDraw part actually exists. Off I go to the LDraw forum.

Regards,

David

Share this post


Link to post
Share on other sites
1 hour ago, djm said:

Off I go to the LDraw forum.

If you find any missing/incorrect mappings please let us know through here or a Change Request on the part page. Then we will update the part, and come a step closer to a perfect database.

Share this post


Link to post
Share on other sites

David;

I ran a search on all 14,361 LDraw definitions currently used by Rebrickable:

    searching for 'KEYWORD': found 5,053 occurrences in 4,233 files
    searching for 'KEYWORDS bricklink': found 149 occurrences in 149 files
    seraching for 'KEYWORDS set': found 788 occurrences in 780 files

Presuming our Bricklink mappings are correct, those 149 LDraw definitions with Bricklink part number can be automatically checked and added if needed. But that's only 1% of the entire LDraw database. Why doesn't LDraw include such information by default?

I am working on two lists: LDraw parts not used by Rebrickable, and Rebrickable parts without LDraw part...

 

Share this post


Link to post
Share on other sites

David:

I wrote some scripts to analyze this stuff:

    Rebrickable currently has 36,332 parts
    Our LDraw library contains 14,361 parts
    We have 1,655 LDraw maps in place
    
    6,931 Rebrickable parts have an LDraw image
        29,401 have NOT (missing_ldraw.lst)

    7,765 LDraw parts are used by Rebrickable
        6,596 are NOT (not_used_ldraw.lst)

I can get you both lists, tell me what format would work for you.

Take care,
Simon

PS I will try to follow the discussion on the LDraw forum, but I will only respond here, to allow other admins (and Nathan) to step in if needed.

Share this post


Link to post
Share on other sites

Hi Simon,

thanks for that. If possible, a simple text file with one part per line for the two files (missing_ldraw.lst and not_used_ldraw.lst). Perhaps pop them onto BrickSafe and send me a message with the link?

18 hours ago, Simon said:

David;

I ran a search on all 14,361 LDraw definitions currently used by Rebrickable:

    searching for 'KEYWORD': found 5,053 occurrences in 4,233 files
    searching for 'KEYWORDS bricklink': found 149 occurrences in 149 files
    seraching for 'KEYWORDS set': found 788 occurrences in 780 files

Presuming our Bricklink mappings are correct, those 149 LDraw definitions with Bricklink part number can be automatically checked and added if needed. But that's only 1% of the entire LDraw database. Why doesn't LDraw include such information by default?

I am working on two lists: LDraw parts not used by Rebrickable, and Rebrickable parts without LDraw part...

 

There's an aversion to defining the corresponding BrickLink number within an LDraw file. If this approach is used, it means that a renumbering of the part within BrickLink would require the LDraw part file be amended and then resubmitted for approval. I'm currently scratching the itch to see if there is a way to define the relationship between the LDraw part number and other systems (Rebrickable!, BrickLink, LEGO, ...) with a separate automatically generated mapping file.

David

Share this post


Link to post
Share on other sites

I will upload those two files later today.

A question came up in the team: our system automatically creates a link to ldraw.org, like this:

     part 3010 - link http://www.ldraw.org/cgi-bin/ptscan.cgi?q=3010&utm_source=rebrickable

This leads to a Part Tracker File Scan page, that lists all the results. Our problem is, that only for parts in the unofficial parts library, the page contains a "details" link that shows a rendered image of the part. For parts in the official parts library, there is only a "DAT" link, that gives the definition itself. When we are looking for a LDraw part number for a particular parts, the absence of images for official parts is a big problem. If we can't easily verify the correct part by looking at an image, it's almost impossible to find the right LDraw part, and so we leave it.

In other words, we have everything in place to create the kind of mapping file you are looking for; showing the relationship between LDraw, Rebrickable AND Bricklink part numbers and I am pretty sure we could actually create such a file on a monthly basis, if only we could easily see images of available LDraw definitions.

Do you know if LDraw has pages where those images are displayed? If not, would you be so kind as to bring this up on the Ldraw forum? (quote me if needed)

If we had a better way to find the LDraw parts, or if LDraw designers could be persuaded to post a CR on Rebrickable to add LDraw mapping to the parts they created, I am pretty sure Rebrickable can solve this problem, and it would be a win-win situation for both LDraw.org and Rebrickable.

Take care,
Simon

Share this post


Link to post
Share on other sites

David;

I downloaded complete.zip and ldrawunf.zip half an hour ago, extracted both and merged ldrawunf/parts into ldraw/parts. Then I ran a dir /b *.dat > ld-current.lst and removed ".dat" from all filenames. The resulting ld-current.lst is compared with the Rebrickable parts database and the draw maps to create missing_l draw.lst (Rebrickable parts not using an LDraw image) and not_used_ldraw.lst (Ldraw parts not used by Rebrickable). All three files placed in folder missing and zipper to missing.zip.

Please note:

    missing_ldraw.lst also contains RB part categories that are NOT recognized by Ldraw, such as sticker sheets, znap parts and books and such in our Other category. If needed, I can filter those out.

    not_used_ldraw.lst also contains obsolete parts, aliases and physical colour parts. Again, if needed, I can filter them out.

Now to create a mapping file (just as a test)

Suppose I generate a csv file with the following layout (all partnumbers):

ldraw,rebrickable,bricklink,brickset,brickowl,peeron
        
for each part in the above mentioned ld-current.lst. We'd have a 100% correct LDraw/Rebrickable mapping, and I could write some scripts to see if the BL, BS, BO and PE parts actually do exist. That doesn't give a 100% certainty, but it's better then nothing. Considering Rebrickable's user base I'd presume most of those other mapping are also correct. Would that be a good basis to create the kind of xml mapping file you have in mind?

 

 

Share this post


Link to post
Share on other sites
On 4/18/2019 at 12:22 AM, Simon said:

<snip>

Do you know if LDraw has pages where those images are displayed? If not, would you be so kind as to bring this up on the Ldraw forum? (quote me if needed)

If we had a better way to find the LDraw parts, or if LDraw designers could be persuaded to post a CR on Rebrickable to add LDraw mapping to the parts they created, I am pretty sure Rebrickable can solve this problem, and it would be a win-win situation for both LDraw.org and Rebrickable.

Hi Simon,

I've posted a question in the LDraw forum regarding images for official parts.

I'm away for the Easter break, so I won't have a chance to have a look at the lists you created until some time next week.

Regards,

David

Share this post


Link to post
Share on other sites
Posted (edited)

LDraw Admin here.

Yes there is a way to get official part images. The url is www.ldraw.org/library/official/images/parts/<part number>.png

You can do the same for unofficial (I.e. Parts Tracker parts) by replacing official with unofficial in the above url. 

I ask that if you use the images you download a local copy and not link to our server

Edited by billthefish

Share this post


Link to post
Share on other sites

Thank, Orion, good to meet and welcome to the forum!

That's good to know.

We're not using LDraw images from your server: we created our own images for each part/color combination, and I am currently re-rendering them at a higher quality (example1/example2).

Currently, Rebrickable uses 7,765 LDraw parts from a base library of 14,361 parts (official/unofficial and some third party). 6,110 of those are found by part number match, another 1,655 by manually added LDraw map. For each part in our database we have an automatically created link to the Part Tracker.

Our main problem is the following. Consider a relatively new part like this:

 3010pr0041 Brick 1 x 4 with I Incredibles Logo print

That part number does not exist in LDraw, so our automated link results in No Matches Found. A Parts Tracker search for 3010 gives 136 results, and I have to read through all those descriptions to find out that Ldraw probably doesn't have that pattern yet. But there is no way to be sure, and a part that might look right based upon the description, like:

    parts/3010pz1.dat
    Brick 1 x 4 with Yellow Batgirl Logo with Black Border Pattern

doesn't have a link to an image to verify that. If the Part Tracker would display these images, or have a link to an image, just like the details link for unofficial parts, it would be much easier for us to add that LDraw map, and make our own LDraw image available for our users.

And if that map is present, Rebrickable can provide the mapping from LDraw to Bricklink, Brickowl, Birckset and Peeron, and the list of LEGO element ids connected to that part.

So I am hoping for some kind of cooperation between LDraw and Rebrickable, firstly to make sure that Rebrickable accurately uses everything the LDraw organization has generated so far (some  6, 596 LDraw dat files are still not used by RB), and maybe, in the future, to provide LDraw with an automated list of newly released LEGO parts that might be included in the LDraw Parts Request forum.

I welcome all your thoughts about this subject... -smile-

Take care,
Simon

Share this post


Link to post
Share on other sites

If I understand your question correctly, you want a URL for a person to quickly look at all the patterned parts for a given element to match the Bricklink/Brickowl designation to the LDraw equivilent. If that's the case I can point you to the patterned parts summary portion of the Parts Tracker:

http://www.ldraw.org/cgi-bin/ptpatterns.cgi

The default is minifig torsos (973.dat) but if you append ?p=<basr part number> to the url, you'll get a listing of all the patterned parts (official and unofficial) for the given base part.

Share this post


Link to post
Share on other sites

Man, that is fantastic - that is going to save me hundreds of hours or work!

And I shamefully admit that by looking at that page, I finally start to understand the numbering system LDraw uses for printed parts. -smile-

As to the minifig torsos - I want to create a whole series of unofficial shortcuts to render images of torsos with precolored arms and hands, which is the way LEGO delivers them in sets, and the way Rebrickable stores them in the parts database. With that overview page that project also becomes much easier.

In other words, thank you so much for your information, and if there is anything I can do for you or LDraw, please let me know.

Take care,
Simon

Share this post


Link to post
Share on other sites

Hi, i'm new to Lego, and databases confused me a lot.

Can someone explain why LDraw have ~11000 and Rebrickable ~32000 (with just ~5000 common) and (if i understand it right) Bricklink has ~50000 parts in list.

Bricks codes are realy over 50 000 or there is some trick(scheme) that i do not understand?

Share this post


Link to post
Share on other sites

Firstly Bricklink list stickered parts which add thousands. 

Secondly A lot of parts on RB probably don’t have the correct Ldraw mapping which causes the discrepancies between databases. 

Also the difference will be from older parts and assemblies which LDraw probably don’t have or inventory differently to us. 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now