Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Utils: Deprecate Exporters (Blender, 3DS Max and Maya) #12903

Closed
Mugen87 opened this issue Dec 18, 2017 · 68 comments
Closed

Utils: Deprecate Exporters (Blender, 3DS Max and Maya) #12903

Mugen87 opened this issue Dec 18, 2017 · 68 comments

Comments

@Mugen87
Copy link
Collaborator

Mugen87 commented Dec 18, 2017

I'd like to suggest the deprecation (removal) of the Blender, 3DS Max and Maya JSON exporters for two reasons:

  • The exporters are not really actively maintained. There is a long list of non-trivial bugs, feature request and suggestions no one cares about (see the Blender issue list). Because all exporters are based on different APIs (Blender, 3DS Max...) and written in different programming languages (not JavaScript), it's very hard to maintain this stuff.
  • The more important point is that the quality and maturity of certain loaders (e.g. FBXLoader and GLTFLoader) were greatly improved over time. In other words, it is much more likely to load FBX or glTF files with proper results. Besides, these loaders are actively maintained by project collaborators.

So instead of exporting to the JSON format, users should focus on other formats like FBX or glTF. And in context of asset delivery, especially glTF is a much better format than (uncompressed) JSON.

@looeee
Copy link
Collaborator

looeee commented Dec 19, 2017

+1 for this - the existence of poorly maintained and out of date exporters in the repo is a unnecessary point of confusion for newcomers, especially since as @Mugen87 points out there are now much better options.

@Usnul
Copy link
Contributor

Usnul commented Dec 19, 2017

Note: official GLTF Blender exporter (https://github.com/KhronosGroup/glTF-Blender-Exporter) doesn't currently allow proper animation export (only 1 animation per object is supported currently).
KhronosGroup/glTF-Blender-Exporter#112

@looeee
Copy link
Collaborator

looeee commented Dec 19, 2017

@Usnul how does that compare to the Blender JSON exporter in this repo?

@Usnul
Copy link
Contributor

Usnul commented Dec 19, 2017

@looeee
Blender JSON exporter exports animations just fine. There are some weird glitches with UVs or vertex normals in certain conditions which are not present in GLTF exporter, but that's another story.

@looeee
Copy link
Collaborator

looeee commented Dec 19, 2017

OK, I don't use Blender so I won't comment further on that.

However I can say from experience that trying to use the 3DS Max exporter, which hasn't been updated in several years, is an endless headache and I would strongly support removing it from the repo in favour of the FBX format.
Nearly everything exported by the 3DS Max FBX exporter is now supported by the FBXLoader and since that exporter is maintained by AutoDesk we can count on it staying up to date.

Similarly with the Maya FBX exporter, although the FBXLoader still needs to be updated to properly support pivots points there.

@Mugen87
Copy link
Collaborator Author

Mugen87 commented Dec 19, 2017

When i remember correctly, Blender does not export animations when you select BufferGeometry. There is definitely a problem with morph targets, see #10932. Besides, the exporter uses the "old" animation hierarchy format and not the one of the current animation system.

@looeee
Copy link
Collaborator

looeee commented Dec 19, 2017

I think we should remove the Revit exporter README too - it just links to a separate repo which looks like it's barely been touched in last couple years.

There are probably hundreds of three.js tools we could link to like this, but unless we are willing to make sure that they are actively maintained and working correctly, I don't think we need to include links to them here.

Googling for "three.js revit exporter ' finds the repo just fine anyway.

@Mugen87
Copy link
Collaborator Author

Mugen87 commented Dec 19, 2017

I think we should remove the Revit exporter README too - it just links to a separate repo which looks like it's barely been touched in last couple years.

👍

@Mugen87
Copy link
Collaborator Author

Mugen87 commented Dec 19, 2017

the existence of poorly maintained and out of date exporters in the repo is a unnecessary point of confusion for newcomers

I just want to highlight this statement because i think it's very important for three.js. When users encounter these exporters, they expect tools that just work. But in most cases they get confused and maybe a bad impression of the entire project. That's no good. 😢

@wolfgangvonludwigsburg
Copy link
Contributor

As an innocent user, may I dare to give a comment ...

Concerning Blender:
If three.js would have a native .blend format importer,
there would obviously be no need for an exporter.

No installation effort on Blender side,
no compilation from blender API/structs to javascript through python code ...

There were tremendious efforts to write the JSON exporter in python,
I wonder why you collaborators did'nt choose this way ... ?

@Usnul
Copy link
Contributor

Usnul commented Dec 19, 2017

@wolfgangvonludwigsburg
It's a fairly simple question to answer. Blend format is not very close to three.js internal representation, so some conversion is necessary. JSON format of three.js is very close to internal representation, so job of conversion from blender representation to three.js is there either way, whether you choose to call it Exporter or Loader. That being said, .blend is not a common transfer format, nothing except for blender really supports it, so having a Loader for it would satisfy fairly small audience, since even people who use blender prolifically tend to use other software also, and .blend is not a format of choice as an exchange format. Usually it's obj, fbx or some open standard like gltf or collada.

@looeee
Copy link
Collaborator

looeee commented Dec 19, 2017

@wolfgangvonludwigsburg if you were to look into building a .blend file loader, this would probably be a good place to start:

https://raginggazebo.com/parsing-blender-3d-files-blend-1-of-3/

@wolfgangvonludwigsburg
Copy link
Contributor

Many THX for your comments!

But why is the Blender JSON exporter such error prone ...
=> It depends on the Blender API, is written in Python, shall understand the Blender data structs,
and compiles to an JSON format, that three.js reconverts into its internals ...

A very lot of tasks, that all can (and really do!) fail, mainly on Blender version changes.
Well, I would prefer the "direct data compilation" way ...

@Mugen87
Copy link
Collaborator Author

Mugen87 commented Dec 19, 2017

Well, I would prefer the "direct data compilation" way.

That's no good approach. Loading blend files directly into apps is some kind of misuse. Instead, you should utilize a file format that is intended for data transfer and asset delivery. glTF is the first standardized format that focuses on this aspect from the application's point of view. That's one reason why i promote glTF so many times 😇 . It should be the standard for all upcoming 3D apps.

@donmccurdy
Copy link
Collaborator

donmccurdy commented Dec 19, 2017

Well, I would prefer the "direct data compilation" way ...

Unfortunately there are deeper problems with using .blend this way. It stores things like edit history, current state of the Blender UI, blender Add-on data. The format changes as new Blender releases come out. And the file sizes can be very large, so it would never be a good choice for high-quality 3D applications. The best way to get data directly from Blender is to use Blender's APIs (as the FBX and glTF exporters do), rather than trying to reverse-engineer Blender's internal .blend format.

@donmccurdy
Copy link
Collaborator

To the original question, I'm going to skip adding a yes/no vote as I don't have much familiarity 3D authoring workflows.

But as a maintainer and primarily a JS dev, being able to focus on loading and support for formats like FBX and glTF — which have actively maintained third-party authoring tools — does feel more likely to give reliable results.

Aside, if there are any problems with current glTF ecosystem that prevent it from being a good enough workflow to solve this type of need, that's helpful feedback as well. 🙂

@wolfgangvonludwigsburg
Copy link
Contributor

I did'nt argue to use the Blender .blend format as a general exchange format,
but it may simplify the collaboration between Blender and three.js ...

I cannot estimate the efforts, maintaining a Python exporter + GUI under the Blender framework,
but I guess, that this can easily become a horror ... ;-)

@donmccurdy
BTW, Wavefront .obj, 3D Studio .3ds are native formats too, that are in widely spreaded use ...

@donmccurdy
Copy link
Collaborator

BTW, Wavefront .obj, 3D Studio .3ds are native formats too, that are in widely spreaded use ...

Yeah I think OBJ isn't mentioned here because the format doesn't support some popular features like animation, but certainly three.js will keep supporting OBJ for a very long time — it's a classic and reliable format.

3DS Max's .3ds is in the same category as Blender's .blend or Maya's .mlt or Cinema4D's .c4d. Each editor has its own internal format, and those formats change as the software does. Maintaining loaders for every modeling tool's private internal format is far harder and bug-prone than using the export formats that the editor's own developers have provided. Worth noting that both 3DS and Blender come with FBX export built in — maintained by their authors — and that Blender will have built-in glTF export in a future version, too.

@wolfgangvonludwigsburg
Copy link
Contributor

wolfgangvonludwigsburg commented Dec 19, 2017

One last throw in ...

We already do have one modeling tool independant ex/importer: Collada,
but for any reasons these seems not so much accepted.

Preferably I did have this implementation kind in mind:
a loading solution, based on Google Protocol Buffers

https://developers.google.com/protocol-buffers/

It's a

... language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler.

Since the 2D/3D vector data we have to deal with are not so overly complex in nature, developing one Blender data file schema should be simply feasable ...

@mrdoob
Copy link
Owner

mrdoob commented Dec 19, 2017

There is actually a blend file parser in javascript 😁
https://github.com/Galactrax/js.blend

@wolfgangvonludwigsburg
Copy link
Contributor

uhh, I get support ... - thank you mrdoob ;-))

@wolfgangvonludwigsburg
Copy link
Contributor

THE biggest (in terms of Terrabyte) 3D Model Application (Google Maps 3D) uses this efficient kind (protobufs) of implementation ...

@mrdoob
Copy link
Owner

mrdoob commented Dec 19, 2017

Going down the path of loading .blend may not be very sane. But is not really that different from loading .dae and .fbx...

@mrdoob
Copy link
Owner

mrdoob commented Dec 19, 2017

Anyway, I agree with the idea of deprecating the exporters.
However, I would wait until gltf is a tiny bit more mature and tested. Summer 2018?

@hccampos
Copy link

I agree with removing those exporters as well. You could even move them to some other repo where they can RIP :)

@donmccurdy
Copy link
Collaborator

donmccurdy commented Dec 20, 2017

However, I would wait until gltf is a tiny bit more mature and tested. Summer 2018?

Waiting seems reasonable to me. In particular for glTF it would be good to have a few things in place first:

  • Blender shipping with glTF export built in, and multiple animation support.
  • Improved workflow from Maya and 3DS Max to glTF, or just more testing of FBX2GLTF.
  • Updates to the official COLLADA2GLTF converter for glTF 2.0.

Revisiting the question in Summer 2018 sounds about right.

@looeee
Copy link
Collaborator

looeee commented Dec 20, 2017

However, I would wait until gltf is a tiny bit more mature and tested. Summer 2018?

For the Blender exporter, I tend to agree. However I recommend removing at least the 3DS Max exporter immediately since there is already a mature and much better alternative in FBX.

@mrdoob
Copy link
Owner

mrdoob commented Dec 20, 2017

However I recommend removing at least the 3DS Max exporter immediately since there is already a mature and much better alternative in FBX.

Sounds good to me 👌

@dherben
Copy link

dherben commented Jan 19, 2018

Chiming in as a user of the Blender exporter here: one advantage of having the JSON is that it is extremely simple to modify after the export. Blender is only a part of our automated toolchain, and we do a lot of processing on the exported JSON, before it is ready to go to our web viewer. Having a transfer format that is so close to the internal threejs format, is a major bonus for us.

Also, we've done some tests with other export formats, and have found that the JSON format is not so bad in terms of transfer size, once decently compressed, much better than Collada or FBX, for example. We did a quick protobuffers-based trial, and could go a bit smaller that way, but nothing that was worth the hassle to us.

For large scenes and slow clients, the speed of parsing and converting to the internal ThreeJS model also becomes important. Since JSON parsing is heavily optimized in the browsers, and since the model structure is so similar, we've assumed that the JSON format would do quite well in this regard. Haven't actually tested this.

@alexkowel
Copy link

Soft8Soft just released glTF-based exporter for 3ds Max. So it might be a good alternative to the removed JSON exporter.

@donmccurdy
Copy link
Collaborator

donmccurdy commented Feb 8, 2018

Thanks @alexkowel, great to see that (and congrats!). If you could add a link on this thread we'll list it with other glTF resources. 🙂

@dherben good points, thanks —

About speed of parsing, I expect you'll find glTF to be even faster. The difference essentially is that metadata is still "just JSON" whereas the big parts of the payload (vertex positions, animation data) are in an ArrayBuffer chunk that can used to create a typed array for THREE.BufferAttribute constructors. In a fully-optimized loader (I don't know if THREE.GLTFLoader is quite there yet) you don't ever have to read or copy that data in JS.

But for modifications to the data as part of a pipeline, I agree that plain JSON is easier to work with, as you said. There are libraries in various languages for working with glTF, but the tooling is not yet very mature.

@donmccurdy
Copy link
Collaborator

donmccurdy commented Feb 8, 2018

Current status of this issue:

Exporters:

  • Maya exporter
  • 3DS Max exporter
  • Blender exporter
    • not going anywhere right now, may revisit in the future.
    • Removed May 2018.

Converters:

EDIT: Updated May 2018.

@pjoe
Copy link

pjoe commented Feb 8, 2018

@donmccurdy Just wanted to get back after experimenting more with glTF Blender exporter and Three.js loader. Turns out it is actually working quite well as scene format for our use case so far. What I'm now doing is to load the exported glTF file to a Three.js scene and then manipulate the Three.js scene hierarchy (swapping out placeholders with dynamic loaded stuff) before doing first render.

There are probably still some features I'd like to see in glTF exporter, will try to comment or do PRs there :)

@donmccurdy
Copy link
Collaborator

There are probably still some features I'd like to see in glTF exporter, will try to comment or do PRs there :)

Awesome, please do! 🙂

@Mugen87 Mugen87 mentioned this issue Feb 20, 2018
11 tasks
@WestLangley
Copy link
Collaborator

Blender exporter not going anywhere right now, may revisit in the future.

So is anyone going to be working on the current Blender exporter bugs? I expect the answer is no. In that case, we should say so.

@mrdoob
Copy link
Owner

mrdoob commented Mar 1, 2018

If no one else does before me... I would like to, at least, try to solve the rotation issues. #13130

@kroko
Copy link

kroko commented Apr 21, 2018

So is anyone going to be working on the current Blender exporter bugs? I expect the answer is no.

I did not read the whole discussion, but my two cents.

Last week I was asked to help a company that has a solution which uses Three.js in content delivery within permanent exhibition. Signages with 3D models that users can explore and interact with. The original devs that are long gone have used Three.js JSON Format. To prepare and get new models in (with a condition that changing runtime code is nogo) is a real nightmare.

IMHO Three.js should focus on supporting established and as well as fast growing formats such as FBX and glTF. Priority on those formats that can hold multiple UV data bins (thus beloved OBJ should NOT be encouraged). Then animation. I know that Blender export addon supposedly supports both, but so does FBX.

Imagine workflow where Blender'ed stuff goes out to

  1. WebGL
    Three.js obviously 😃

  2. OpenGL 3.3+
    raw / Cinder

  3. OpenGL ES 2.0 on RISC
    From smartphone to RaspberryPi

  4. Game engines

  5. Real-time graphics apps that are integrated with media servers (Hippo, D3 disguise)
    The ones stage VFX guys use.

Instead of having to use many different exporters for different outputs the goal would be to use 1, max 2 formats. When writing OGL in the first 3 cases... the same model format should be used, that's it. For the last two points FBX is the king (different implementations for COLLADA across packages make it difficult to work with) and there actually the model is not "exposed".
I am not bashing Three.js JSON Format or Blender addon (@mrdoob 🙇 🙏 ), it has (had?) it's use and probably was invented to solve issues that other formats could not at the time in the first place (and I do also have NIHS syndrome).
I just want to share that from the perspective where one has to constantly deliver to different outputs within the industry Three.js JSON Format does not fit in the landscape, it is superfluous.

@mrdoob
Copy link
Owner

mrdoob commented Apr 24, 2018

@kroko definitely agree 👍

I think the formats landscape is starting to become more clear. The three.js json format was done because there was no json format at the time. Defining a file format was the last thing I wanted to do when I was already doing a rendering engine and a API 😩

@marquizzo
Copy link
Contributor

marquizzo commented May 2, 2018

As a newbie, Three.js JSON exporter was very educational because it allowed me to see the raw data and underlying structure of the output geometries. Other exporters, as efficient as they are, don't let you see the data because they're mostly in a binary format.

I agree that removing it from this repo would be the best option today, but as @hccampos said, maybe it could be placed in its own repo to remain for educational purposes.

@mrdoob
Copy link
Owner

mrdoob commented May 4, 2018

I agree that removing it from this repo would be the best option today, but as @hccampos said, maybe it could be placed in its own repo to remain for educational purposes.

There will always be the option of exporting as JSON from http://threejs.org/editor/

@donmccurdy
Copy link
Collaborator

Closed in #14117.

@Mugen87
Copy link
Collaborator Author

Mugen87 commented May 30, 2018

I suggest we close now all open issues which are related to the Blender exporter. Agreed?

@stmaccarelli
Copy link

I think someone may write some "official" documentation / pipeline for 3D models export/import (for basic and special features), common troubleshoot, and prune out or update all the outdated documents and examples. EG the knight example is very confusing, given that you don't have a blender json exporter anymore. Maybe JSONLoader for 3d models should be just kept for retrocompatibility, but we have to read that, etc.

@donmccurdy
Copy link
Collaborator

@stmaccarelli there is some new documentation here: https://threejs.org/docs/#manual/introduction/Loading-3D-models, but yes please let us know what else would be helpful!

@stmaccarelli
Copy link

@donmccurdy I think that paper is poor.
Right now the entire 3d import / animation system, given the mix of documentations, examples and things found on the internet from different eras, is confusing.

The master Animation System doc would be ok if the refs of the single objects would be correct.
Let's take AnimationClip reference. I'm not sure I'm exporting morphTargets the right way, but here I read:

.CreateClipsFromMorphTargetSequences ( name : String, morphTargetSequence : Array, fps : Number, noLoop : Boolean ) : Array
Returns an array of new AnimationClips created from the morph target sequences of a geometry, trying to sort morph target names into animation-group-based patterns like "Walk_001, Walk_002, Run_001, Run_002 ..."

problem is that if I import a glTF file, there's no morphTargetSequence array... there are some morphTargetSomething objects stored here and there, but I don't know what and how to use.

I think we should have some docs describing 3D Models management / workflows with very simple examples.
AND the 3D loaders references should all respect the same template.
Let's say
1- simple 3D model import
2- material(s) import (with all the bells and whistles like the different maps, maps params, multimaterial management, etc)
3- entire scene import ( like how to traverse / manage imported scenes like the ones from glTF)
4- skeletal animation(s) import and management
5- morph animation(s) import and management

We should also check that all the examples that feature 3D model loading, respect the same pattern.

We should update examples, and while not possible we should write clearly that some parts are deprecated and what (like in the knight example... )

EG- if we decide that the JSON file format for 3D model should be deprecated, in favour of - say - glTF, it makes no sense that the only animation example that features both skeletal animation and morphtargets is the old knight one, that uses a JSON model from 10 versions ago, that stores a data structure that is not used anymore.

@donmccurdy
Copy link
Collaborator

@stmaccarelli There is no single recommended end-to-end workflow; I think we'd be hard pressed to provide that given different skill levels, preferences for free vs paid modeling tools, and particular needs.

I don't think you'd commonly need that CreateClipsFromMorphTargetSequences method. The document above does not go into the details of using any particular loader (as you mention, they're inconsistent), but the GLTFLoader docs do —

loader.load('foo.glb', function(gltf) {
  const clips = gltf.clips;  // Array<THREE.AnimationClip>
  const model = gltf.scene; // THREE.Scene
  ...
});

In this case it doesn't matter whether the clips are TRS, skinned, or morph targets — you can play the animations all alike. I've written one possible workflow using Mixamo and Blender. Here is another using Maya.

We should also check that all the examples that feature 3D model loading, respect the same pattern.

AND the 3D loaders references should all respect the same template.

This is a fair point, and we have some room to improve here.

it makes no sense that the only animation example that features both skeletal animation and morphtargets is the old knight one, that uses a JSON model from 10 versions ago, that stores a data structure that is not used anymore.

Strictly speaking the JSON format and data structure are not deprecated, and it's still a totally reasonable way to [de]serialize a scene, but point taken. @mrdoob what do you think about our replacing some of the animation examples? Such as:

animation / keyframes / json
animation / scene
animation / skinning / blending
animation / skinning / morph

@looeee
Copy link
Collaborator

looeee commented Jul 15, 2018

what do you think about our replacing some of the animation examples?

+1 for this. I vote for replacing at least the soldier from animation / skinning / blending a with a more modern model from Mixamo, such as this one:

screenshot-www mixamo com-2018 07 15-10-04-00

We could blend between idle / walk / run, and it would probably be possible to convert the model to glTF if that is preferred.

Model size would be around 9mb, while the current model along with all the associated files is 71mb!!

For animation / skinning / morph we could use the model I've been testing FBX morph targets with:

im

It's about the same size as the current knight model, but since morph targets are more commonly used for facial expressions I'd say this one makes more sense. Again, it's currently in FBX format but could be converted to glTF if that is preferred.

@donmccurdy
Copy link
Collaborator

donmccurdy commented Aug 3, 2018

Here are a few examples to consider:

screenshot link size      
iondrive link 6 MB
vacation link 3 MB
lain link 5 MB
handpainted link 12 MB

They're all animated, work correctly in three.js, and could probably be compressed or optimized a bit more than the Sketchfab download version. I don't have a rigged character with nice run/walk cycles, but the Mixamo->glTF workflow isn't too bad.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests