Simon

Updating LDraw Images

Recommended Posts

Greetings, Folks

During the last three month we have been working on a set of updated LDraw part images, and with the new hardware that Nathan added recently, we can now slowly start implementing the new images. In this post I want to outline the problems with our current set of LDraw part images, describe the scope of the update project, present a schedule of the implementation, and look forward to more improvements during the coming year. In the next post, I will show examples of current and updated LDraw part images.

Backgrounds

Our current set of LDraw part images is based upon 11,000 LDraw part definitions and it was generated in 2016 with some small updates in the following years. As we need an image for each part in every used color, and we had about 135 different colors back then, our current set of LDraw part images totals about 73,000 items, varying in size from 100x75px to 250x250 px.

There are some problems with our current LDraw part images:

  • some images seem to have a wrong color (black and red most obviously)
  • the quality of the some images (transparent parts) is not so good
  • white and transparent parts are difficult to recognize
  • the viewing angle of LDraw images is the opposite to the angle of element images
  • some parts are better displayed full front
  • photos can be zoomed out to 500x500px, LDraw images can't
  • we current have 178 colors and many new colors don't have the right LDraw images
  • most important: there are many new LDraw definitions that we currently don't use

Scope

November last year we started researching these problems. Our new set is based upon 14,364 LDraw definitions, consisting of full LDraw official and unofficial libraries, and several third-party libraries. The rendering of the bulk of the new images took two month, December and January, with ten quad-cores running simultaneously for 16 hours per day, approximating some 10,000 hours computer time. All bulk rendering was done with PovRay with image generation by L3P.

Our new LDraw part images are four time as large (500x500px), rendered at the highest quality. Viewing angle and colors problems of all solid colors are all corrected. Most importantly, we will be adding about 4,000 new LDraw images, and our total image set is expected to grow to about 100,000 items.

For special colors, such as transparent, metallic, chrome, speckle, glitter, glow-in-dark, rubber, pearl, PovRay is not delivering the most perfect results. For these colors, we are going to use Studio's Eyesight renderer instead of PovRay. For the most common parts, these Eyesight images have already been rendered, and they will be part of the initial update. The remainder will follow afterwards.

Schedule

Knowing that a large update like this will inevitably produce new errors, we are going to implement the new part images in small steps. I am still working on those Eyesight special color renders, and it will take at least four month to finish those, so the entire update will take at least that period.

We will start by updating the Unknown images, which are used when we do not have an image in the right color. An example of this can be seen here. The color Vintage Green was introduced for parts from the period 1945 to 1958, but the thumbnails are shown in blue, because we do not have rendered those parts in the right color yet. This means that for many parts for which we now haven't got an ldraw image, an new part image will occur, but with the default color for Unknown, which is a Medium Dark Blue. We already have a lot of these unknown colored ldraw images, but if you notice some new ones, there is an easy way to check: if all the colors of the part show the same image with the same color, then the image is newly added in the unknown color. When we add all the other colors during then next few month, they will replace the unknown color images automatically, and eventually, all parts for which we have an ldraw image should show all available colors correctly.

When the new Unknown images are available, we are going to take a couple of weeks to check for errors and correct them. Then all corrected parts need to be re-rendered in all the right colors. When that is done, we will slowly, color for color, update the existing set of images.

More Improvements

For the second half of this year, we hope to implement even more improvements of the LDraw images, including Eyesight renderings for all transparent, metallic, chrome, speckle, glitter, glow-in-dark, pearl colors, and using rubber black for all black tires and such. The latter will NOT change the color code for black tires, but it will make the images look more natural.

Conclusion

In a perfect world a quality improvement such as this might even go unnoticed; but in the real world many will immediately see some differences, and wonder what we're up to now. I hope by announcing this update in as much detail as I can, I have taken away some of your worries.

I will be updating this thread to keep you informed of new developments, and if you have any questions, or you think you see an error or a problem, please let me know.

Take care,
Simon

Share this post


Link to post
Share on other sites

Here are some examples of the improvements:

Camera angle and color corrected (left: old/right: new):

black-old.jpg   black-new.jpg

blue-old.jpg   blue-new.jpg

CLICK TO ENLARGER

Transparent:

studio-trans.jpg   studio-trans-dark-blue.jpg   studio-trans-red.jpg

Metallic:

studio-flat-silver.jpg   studio-flat-gold.jpg   studio-copper.jpg

Chrome:

studio-chrome-silver.jpg   studio-chrome-gold.jpg

Speckle:

studio-speckle-black-gold.jpg   studio-speckle-black-silver.jpg

Glitter:

studio-glitter-trans-purple.jpg   studio-glitter-trans-yellow.jpg

Pearl:

studio-milky-white.jpg   studio-pearl-gray.jpg   studio-pearl-white.jpg

Share this post


Link to post
Share on other sites

(I had to lock the thread to get all the images in, and that took a while... topic is now unlocked...)

Share this post


Link to post
Share on other sites
Posted (edited)

Looks great but I still think the transparent needs some work, they look like trans-clear with color only on the edges. sure the edges shall have deeper color but they aren't that transparent in real life especially not after scratching them up by building with them. not sure what you can do with those render tools thought. it need to measure thickness while ray tracing, not just number of surfaces and make them loose alpha the thicker they get.

in fact they look funky where is the wall thickness on those bricks, you should be able to see the end of the walls on the "floor"

never mind took out some brick and the thickness seem to be reflected correctly might need some tweaking with the colors still especially if any of those is one of the darker colors, the trans-red is hard to see through. 

Edited by biodreamer

Share this post


Link to post
Share on other sites

Remember that the example images are 800x640; the final images will be 500x500 when zoomed out, 250x250 regular in the parts details page, and 85x85 thumbnails in search pages and lists. When the size of an image is reduced, the perceived color intensity increases.

To be sure, the decision to use Eyesight for transparent parts is not merely based on quality. A solid color 500x500 high quality render in PovRay takes between 1 and 3 minutes on average. However, transparent renders take much longer. My rule of thumb, which usually works well, is half an hour for each transparent stud. So a transparent 2x4 takes 4 hours, and a transparent 6x8 plate takes about 24 hours. For a single part, that is not a problem, but I have to renader at least 5,000 transparent parts. That's a problem!

In Studio every high quality 800x600 render takes 10 minutes, no matter how complex the part or whether it is solid or transparent.

In our old set of LDraw images we solved the PovRay problem with transparent parts by using a much lower quality and smaller image size, but I want to have that zoom feature available for all ldraw images, and so I need to create 500x500 images, even for transparent parts. So if those colors are a little too light, I fear we have to live with it, at least for now.

Share this post


Link to post
Share on other sites

14,363 newly rendered LDraw images for color -1 Unknown are now active.

RBG: 0033B2

0033b2.png

Share this post


Link to post
Share on other sites
13 hours ago, Simon said:

14,363 newly rendered LDraw images for color -1 Unknown are now active.

RBG: 0033B2

Ooo exciting! All your work coming to fruition. Has to be satisfying. 

Share this post


Link to post
Share on other sites
Posted (edited)

I hope this isn't too off-topic, but I found nowhere else to post it.

 

If there is no official LDraw shortcut for a specific assembly, for example 973p47c01 or any torso + arms + hands for that matter, would it be possible to store unofficial shortcuts locally and auto-generate LDraw images for Rebrickable? As I understand it, currently Rebrickable only checks the existence of of an LDraw file at Ldraw-org. (I made this shortcut in Notepad, but there must be some way to make it more automized I hope.)

0 973p47c01 Torso Castle Classic Shield Red/Gray Print / Light Gray Arms / Yellow Hands
0 Name: 973p47c01.dat
0 Author: Tore Eriksson
0 Unofficial LDraw Shortcut

0 // torso
 1 4 0 -72 0 1 0 0 0 1 0 0 0 1 973p47.dat

0 // arms
 1 7 -15.552 -63 0 0.9855 -0.1699 0 0.1699 0.9855 0 0 0 1 3818.dat
 1 7 15.552 -63 0 0.9855 0.1699 0 -0.1699 0.9855 0 0 0 1 3819.dat

0 // hands
 1 14 -23.552 -46 -10 0.942 0.335 0.0072 -0.2404 0.6906 -0.6821 -0.2336 0.6409 0.7312 3820.dat
 1 14 23.552 -46 -10 0.942 -0.335 0.0072 0.2404 0.6906 -0.6821 0.2336 0.6409 0.7312 3820.dat

0
 

 

Edited by SimLego
Minor typo in the LDraw file rendered one hand invisible.

Share this post


Link to post
Share on other sites

Hi, SimLego;

First off: I saw your CR's with new LDraw mappings, thanks for those, it's been a great help. I have still have about 7,500 LDraw definitions that we're not using, and even though many of those are either sub-parts, obsolete or alias definitions, I am pretty sure many still need to be added. If you're interested, I can share a list of defs not used.

As to creating unofficial shortcuts for torso's, that is a great idea. I already added several third-party definition sets (digital-bricks and modulex), and I have no problem adding another set of shortcuts to get good renderings of torsos. Just not sure how to organize it. -smile-

In any case, I need to keep a separate list for these torso shortcuts, because eventually, I want to render them full front instead of at a 30 degrees vertical angle, to make the printing better visible. Will need to do the same for minifig heads, certain panels, and several others. Right now, my scripts render everything at the same angle, and I still got some 5,000 transparent part/color combinations to render, before I can add support for multiple angles.

Let me test the 973p47c01.def you created first, see how it looks, and then we'll discuss how to organize things, okay?

Take care,
Simon

Share this post


Link to post
Share on other sites

Here are some test renders:

first two with default viewing angle (first L3P colors, second LDraw colors.), last two full front.

I like L3P colors better, and I must admit that the angled version looks more natural then the full front version. Could be that I am just more used to it, and obviously it's a matter of personal preference, but in any case, I think the printing is fine, and the end result is good enough for me.

What do you think?

default / L3P

973p47c01-1.png

default / LDraw

973p47c01-2.png

full front / L3P

973p47c01-3.png

full front / LDraw

973p47c01-4.png

Share this post


Link to post
Share on other sites

looks a bit to reflective to me, seeing the hands in form of shadows would be fine but now it more look like it's a mirror, the rest of the imperfection is due to low polycounts and the fact that the print isn't a texture but regular polygons. (flaw in ldraw) so neither of those can be expected to change.

what is that weird shadow on the neck in all renders?

Share this post


Link to post
Share on other sites

Here's an even better one, using LGEO for 3818, 3819 and 3820.

(Just realized that there about 250 newer LGEO defs that haven't been used because L3P depends on a lg_element.lst that needs to be update manually. Which means I have compare my current LGEO set, which is three years old, with the most modern version, add all new defs to lg_elements.lst, then find out which parts were involved and re-render all of those. Got something to do for the next few weeks... -smile-)

973p47c01.png

Share this post


Link to post
Share on other sites

That looks much better, now it's just the pattern that is low a resolution.

Share this post


Link to post
Share on other sites

I agree, but realize you're looking at a 500x500 image: the main images on the part pages are only 250x250, and for the search results and lists we use this:

973p47c01.jpg

no matter how low the resolution, this is always better then no image at all. And the advantage over photos is that all thumbnails have the same viewing angle, and, at least, consistent colors.

 

Share this post


Link to post
Share on other sites

Just checked three randomly selected figs, and they all have that black rectangle on the 'neck' - no idea what it's for, but it is there.

Share this post


Link to post
Share on other sites

yeah getting the same angle when taking photos is hard even if you do all of them yourself. and then getting the right lightning and without having your setup cast shadows on the scene.

Share this post


Link to post
Share on other sites

I was not talking about the black square it's for the pattern machine in the Lego factory. I just don't get the "shadow" thingy shape on the neck. what cause that curvy form.

Share this post


Link to post
Share on other sites

Ahh, I think that's a reflexion of the torso, just like the neck is reflected on the top surface of the torso.

Share this post


Link to post
Share on other sites

@SimLego

I'd like to change the color of the torso to 16:

0 // torso
1 16 0 -72 0 1 0 0 0 1 0 0 0 1 973p47.dat

thus my scripts will render the part in Red, and the system doesn't have to fall back to the Unknown Color version.

Is that OK with you?

Share this post


Link to post
Share on other sites
27 minutes ago, Simon said:

@SimLego

I'd like to change the color of the torso to 16:

0 // torso
1 16 0 -72 0 1 0 0 0 1 0 0 0 1 973p47.dat

thus my scripts will render the part in Red, and the system doesn't have to fall back to the Unknown Color version.

Is that OK with you?

Of course it is OK. I thought about doing that in the first place, but realized it could cause some other problems when viewed directly in an LDraw viewer.

Share this post


Link to post
Share on other sites

@SimLego-

Using your example I am confident I can create more of those Unofficial LDraw Shortcuts, but I got a lot going on right now, so it might take a few weeks. Obviously, if you create some others in the mean time, that would be perfect. If needed, you can mail them to simonvanmeygaarden at gmail.com, and I'll take care of it.
If you have any other examples of shortcuts we can use, please let me know.

Take care,
Simon

Share this post


Link to post
Share on other sites
On 3/17/2019 at 4:39 PM, Simon said:

@SimLego-

Using your example I am confident I can create more of those Unofficial LDraw Shortcuts, but I got a lot going on right now, so it might take a few weeks. Obviously, if you create some others in the mean time, that would be perfect. If needed, you can mail them to simonvanmeygaarden at gmail.com, and I'll take care of it.
If you have any other examples of shortcuts we can use, please let me know.

Take care,
Simon

Like I said, I have to get the making of those shortcuts automized somehow, and right now I have no programming enviroment installed in any of my computers, so it might take a while for me to even get started on this huge project. But at least it's in my pipline.

 

/SimLego

Share this post


Link to post
Share on other sites

Well, I have python on all my systems, and selecting 450 ldraw definitions that start with 973p and using the filename and description to generate a shortcut like the one you posted can be done in an hour or so. The main problem is getting the color of hands and arms, but if I can relate the ldraw torso to the Rebnrickable torso+arms+hands definition, I might be able to grab those colors from the RB part name, and translate them back into LDraw color codes. Let me think about that some more...

If I start generating more unofficial ldraw shortscut, what numbering should I use? Original torso number followed by "c01"? Can I put my name in there as author?

Share this post


Link to post
Share on other sites
10 minutes ago, Simon said:

 

 8<----

If I start generating more unofficial ldraw shortscut, what numbering should I use? Original torso number followed by "c01"? Can I put my name in there as author?

As long as the file is stored within the virtual Empire of Rebrickable, there is no reason to name it anything else than the Rebrickable part number. Plus ".dat", of course.

It depends on how many "humans" will have access to your LDraw files. Any rendering software will not care if you even write a header or not. Basically, the renderer just cares for the five type 1 lines (for one torso, two arms, and two hands.) The original work of my shortcut is actually made by Leonardo Zide, and who knows where he's copied it from, or if he used some Minifig Generator. So you are free to put your name there - or omit that line if it's just going to be used by rendering software.

So, for 973pr1244c01, where LDraw has one number for the torso, BrickLink another, and Rebrickable yet a third one, you are free to use any file name you like. But I believe it is most convenient to stick with the Rebrickable number. And the lines of the header are all optional, the only purpose for them is to make it easier for those who have access to the actual LDraw files you create/generate.

 

0 Torso Mechanic Blue Overalls, Tools in Pocket Print / Medium Blue Arms / Yellow Hands
0 Name: 973pr1244c01.dat

0 // torso
 1 16  0 -72 0  1 0 0  0 1 0  0 0 1  973p8z.dat
 0 // or use color code 73 directly

0 // arms
 1 73 -15.552 -63 0 0.9855 -0.1699 0 0.1699 0.9855 0 0 0 1 3818.dat
 1 73 15.552 -63 0 0.9855 0.1699 0 -0.1699 0.9855 0 0 0 1 3819.dat

0 // hands
 1 14 -23.552 -46 -10 0.942 0.335 0.0072 -0.2404 0.6906 -0.6821 -0.2336 0.6409 0.7312 3820.dat
 1 14 23.552 -46 -10 0.942 -0.335 0.0072 0.2404 0.6906 -0.6821 0.2336 0.6409 0.7312 3820.dat

0
 

Share this post


Link to post
Share on other sites

We have just uploaded and activated 50,000 (50,382 to be precise) new LDraw images in the first 50 solid colors. Here's an overview (color-code, number of images, color name):

-1    14362    [Unknown]
0    4722    Black
1    3372    Blue
2    1905    Green
3    1028    Dark Turquoise
4    3240    Red
5    1254    Dark Pink
6    1577    Brown
7    2838    Light Gray
8    2023    Dark Gray
9    915    Light Blue
10    925    Bright Green
12    497    Salmon
13    454    Pink
14    2975    Yellow
15    3333    White
17    248    Light Green
18    321    Light Yellow
19    1519    Tan
23    159    Dark Blue-Violet
25    1106    Orange
26    622    Magenta
27    796    Lime
28    706    Dark Tan
29    395    Bright Pink
30    311    Medium Lavender
31    189    Lavender
69    187    Light Purple
70    1183    Reddish Brown
71    2578    Light Bluish Gray
72    2180    Dark Bluish Gray
73    553    Medium Blue
74    126    Medium Green
77    130    Light Pink
78    309    Light Flesh
84    421    Medium Dark Flesh
85    796    Dark Purple
86    616    Dark Flesh
89    97    Royal Blue
92    292    Flesh
100    28    Light Salmon
110    148    Violet
112    85    Blue-Violet
115    71    Medium Lime
118    1090    Aqua
120    103    Light Lime
125    167    Light Orange
191    741    Bright Light Orange
212    337    Bright Light Blue
216    102    Rust
226    563    Bright Light Yellow

End of next week I will start uploading the second batch of solid colors, and post an update on the progress of special colors.

Some parts might not look as good as others. Usually this means that the part is still using low-poly meshes, and I can probably make them look much better by updating some LGEO definitions.

If you spot such images, please post them in this thread (preferably with a link to the Parts Details page).

Take care,
Simon

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now