feature: Add potree 2.0 parser

[ketourneau]

Description

We add Potree2Source, Potree2Layer, Potree2Node and example (potree2_25d_map.html) for loading potree 2.0 file format in iTowns.
Pull request for sample data must be accepted before (iTowns/iTowns2-sample-data#15).

Motivation and Context

We need to load potree 2.0 file format in iTowns to meet our project requirements.

Do I need to duplicate the potree tests to run them on potree2 ?

[Desplandis]

Hi @ketourneau and thanks for your contribution!
Is this PR still in draft or ready to review? If ready, I'll start reviewing it by the end of the week.

Do I need to duplicate the potree tests to run them on potree2 ?

I advise to add some tests (duplicates of potree 1.x parser tests and/or your own).

[ketourneau]

@Desplandis thanks for your reply, I will add potree 2 tests.

Actually we got some trouble to use use itowns package with potree2 loader (web worker) in a vite project.
I try to do worker import like here unsuccessfuly.
potree-core package work's fine in vite project (it use same web worker for potree 2).

If we change in itowns .babelrc
"presets": [["@babel/preset-env", { "modules": false }]],
Worker works fine in vite project but test failed in itowns.

So if someone can help to fix the problem (I think it's a babel config problem) ?
You can find vite sample project here

[Makaronelle]

@Desplandis thanks for your reply, I will add potree 2 tests.

Actually we got some trouble to use use itowns package with potree2 loader (web worker) in a vite project. I try to do worker import like here unsuccessfuly. potree-core package work's fine in vite project (it use same web worker for potree 2).

If we change in itowns .babelrc "presets": [["@babel/preset-env", { "modules": false }]], Worker works fine in vite project but test failed in itowns.

So if someone can help to fix the problem (I think it's a babel config problem) ? You can find vite sample project here

@Desplandis FYI those very same web workers cannot work on our end without some kind of tinkering with Babel, which breaks all iTowns' tests. At least, that's what we got so far since our resources around web workers are currently busy and we cannot allocate any of their time.

What we do know: since we call those web workers from Potree2BinParser, they do get recognized by Webpack and Babel, they do get outputted, but keep their URL as it is. Since they are located in the "Worker" subfolder, they get called as "Worker/worker_name". Once we use them in our project, these files can't be resolved unless we add aliases left and right.

We could add more logic to this worker loading process (something in line with what pdf-js is doing) but we still wanted to ask any of this project's contributor if there will be some plan on the way for either outputing or if it should be done in a certain way.

Our endgoal would be to put iTowns' library as it is, without cloning or adding on top of it any building logic.

[Desplandis]

@ketourneau @Makaronelle If I understand your issue correctly, it seems the problem is that rollup does not bundle/alias the two workers in your vite project ?

I am not familiar with vite but looking at the web workers documentation and the output of babel on our side (after a npm run transpile), it seems that babel does not transform the absolute path to the worker to a relative path. rollup expecting a relative path, it does not bundle/alias your workers.

After setting a relative path to the workers file, it seems to fix this issue:

✓ 624 modules transformed.
dist/index.html                                            0.52 kB │ gzip:   0.31 kB
dist/assets/potree2-decoder.worker-ffeff9b8.js            10.37 kB
dist/assets/potree2-brotli-decoder.worker-7fc1bb98.js    162.99 kB
dist/assets/index-323ff7d8.css                             0.07 kB │ gzip:   0.08 kB
dist/assets/index-3d1b9e4c.js                          2,241.98 kB │ gzip: 614.49 kB

[ketourneau]

@Desplandis Thanks for your reply.

I try with relative path but we got the same error in vite (maybe we could resolve it later).
For now, we have an error when we try to execute test on Potree2BinParser.
I push potree2BinParser test (Work In Progress) :

npm run base-test-unit test/unit/potree2BinParser.js

Potree2BinParser.js:27
new URL('../Workers/potree2-brotli-decoder.worker.js', import.meta.url)
SyntaxError: Cannot use 'import.meta' outside a module

I try to add babel-plugin-transform-import-meta but I got this error when I try to execute potree2BinParser test :

Potree2BinParser
ReferenceError: Worker is not defined

[Desplandis]

This a nice start. =)
I left some comments that should be addressed before getting this merged.
I did not however reviewed the two workers because eslint generates two many errors, I'll do it in the following review.

[Desplandis]

@ketourneau I also see that you have added brotli decompression library as as local dependency?
I would prefer that such dependency be resolved by npm or added at runtime à la three.js GLTFLoader.

[ketourneau]

I modified code according to your comments.
I still have 2-3 remarks to deal with and I will fix and add tests on potree2.

[ketourneau]

I add more description about metadata.json and improve Promise in Potree2BinParser.

I'm still working on tests.

[ketourneau]

@Desplandis I've completed the tests. Is everything good except the bigint issue ?

[gauthier-th]

Hi, any update on this?

[Desplandis]

@gauthier-th Following #2195, I expect this PR to be merged upstream by the end of February.
@ketourneau @Makaronelle Since this is open for quite some time, I will rebase your PR on the master branch as well as doing some adjustments work.
We decided to cut support of old browsers and distributing as ESM (see #2256), I will :

• Remove the transpile BigInt to JSBI step
• Use a common WorkerPool implementation that I intend to share with the COPC one.