odoc generates exponentially huge docs

[panglesd]

odoc's inlining is responsible for generating docs that are exponentially huge compared to their input.

A simple example demonstrating this is:

module type S0 = sig end

module type S1 = sig
  module A : S0
  module B : S0
end

module type S2 = sig
  module A : S1
  module B : S1
end

module type S3 = sig
  module A : S2
  module B : S2
end

module type S4 = sig
  module A : S3
  module B : S3
end

module type S5 = sig
  module A : S4
  module B : S4
end

module type S6 = sig
  module A : S5
  module B : S5
end

module type S7 = sig
  module A : S6
  module B : S6
end

module type S8 = sig
  module A : S7
  module B : S7
end

module type S9 = sig
  module A : S8
  module B : S8
end

module type S10 = sig
  module A : S9
  module B : S9
end

module type S11 = sig
  module A : S10
  module B : S10
end

module type S12 = sig
  module A : S11
  module B : S11
end

module type S13 = sig
  module A : S12
  module B : S12
end

module type S14 = sig
  module A : S13
  module B : S13
end

module type S15 = sig
  module A : S14
  module B : S14
end

module type S16 = sig
  module A : S15
  module B : S15
end

which generates more than 1Gb of almost empty HTML files. For instance:

4.0K	_build/default/_doc/_html/explode/Explode/module-type-S13/B/B/B/B/B/B/A/B/B/B/A/A/B
12K	_build/default/_doc/_html/explode/Explode/module-type-S13/B/B/B/B/B/B/A/B/B/B/A/A
4.0K	_build/default/_doc/_html/explode/Explode/module-type-S13/B/B/B/B/B/B/A/B/B/B/A/B/A
4.0K	_build/default/_doc/_html/explode/Explode/module-type-S13/B/B/B/B/B/B/A/B/B/B/A/B/B
12K	_build/default/_doc/_html/explode/Explode/module-type-S13/B/B/B/B/B/B/A/B/B/B/A/B
28K	_build/default/_doc/_html/explode/Explode/module-type-S13/B/B/B/B/B/B/A/B/B/B/A
4.0K	_build/default/_doc/_html/explode/Explode/module-type-S13/B/B/B/B/B/B/A/B/B/B/B/A/A
4.0K	_build/default/_doc/_html/explode/Explode/module-type-S13/B/B/B/B/B/B/A/B/B/B/B/A/B
12K	_build/default/_doc/_html/explode/Explode/module-type-S13/B/B/B/B/B/B/A/B/B/B/B/A
4.0K	_build/default/_doc/_html/explode/Explode/module-type-S13/B/B/B/B/B/B/A/B/B/B/B/B/A
4.0K	_build/default/_doc/_html/explode/Explode/module-type-S13/B/B/B/B/B/B/A/B/B/B/B/B/B
12K	_build/default/_doc/_html/explode/Explode/module-type-S13/B/B/B/B/B/B/A/B/B/B/B/B

Although so many levels are not common, the explosion can happen much faster when the base of the exponent is bigger.

A rough idea of a fix would be to allow the driver to generate the pages lazily "on-demand":

$ odoc html-generate -o html/ --sub-path A/A/B/B/A/index.html a.odocl

The driver or server would be responsible for generating and garbage-collecting the files.

[panglesd]

I forgot to mention that the rendering phase is taking most of the time in such example. Hence the solution of generating the page on-demand is for performance reasons, not only for size reasons.

More precisely, most of the time is spent in this line. So what takes so much time is outputing the html to the files.

On the example above:

• time spent compiling: 1.132021s
• time spent linking: 1.147978s
• time spent turning into a value of Document type: 8.585029s
• time spent turning into a value of Tyxml type: 17.133151s
• time spent (pretty-)printing the html: 60.294026s
• Total: 88.292205s

So printing is responsible for 2 third of the time...

[Julow]

Is the module really that big or are they implemented as a chain of aliases ?

I wonder if we could skip generating a page for A if it's implemented as a non-hidden alias (I'm not sure we can know that) or if specified with a new tag:

module type S16 = sig
  module A : S15
  (** @alias M15.A *)
end

(I assumed that there were only one module of this signature and that the module types could be removed and inlined in the module declarations)