I like the idea :-)

I think this is a horrendous idea. I mean, this word doesn't begin to express how awful of an idea I think this is.

Quoting myself on this (#745 (comment)):

[...] documentation is bound to the release cycle of the project. That mess isn't caused by having multiple repos, but by having docs in the wrong repo. The site is a different product, hence it's absolutely normal for it to be a different repo.

Why in the world would you want to inflate the lektor repository with the showcase? I know you did check out that repo previously, you must've noticed the loooong download, and the problem doesn't lie in assets/static:

$ cd ~/Work/lektor-website/
$ du -sh .
96M	.
$ du -sh assets/static
2.1M	assets/static

Do you really want to do another release every time someone submits something for the showcase? Or a plugin? Or when you write a blog post? These things don't belong with the code. Documentation belongs with the code (and the install script, I can't think of anything else).

I think what we should do is:

• extract the docs, their models and their assets with git filter-branch. It's gonna be a bit of work, but nothing to get scared about.
• import them here into a docs folder.
• eat our own dog-food by building this when docs changes, sure.
• figure out how this all fits into the website. Do we want to jump through hoops and figure out how to import an external content tree? Or keep things simple and publish on, say, docs.getlektor.com?
• (we could also jump through another set of hoops and figure out how to publish for reathedocs maybe..)

(Refs lektor/lektor-website#226)

Later edit: the point of keeping the docs with the code is also being able to publish different docs for different releases. It's something we need to figure out as well.

I think this is a horrendous idea. I mean, this word doesn't begin to express how awful of an idea I think this is.

Hey man, tone down that attitude. Let your arguments make your point instead of belittling others' suggestions. It's completely unnecessary because the points you raise and the arguments you make are completely fine on their own.

know you did check out that repo previously, you must've noticed the loooong download, and the problem doesn't lie in assets/static

I have not. I just did a fresh download in 5.829 s, which is something I very rarely. This is however of course not reason to dismiss the fact that the repository size may affect people.

The main reason why I wanted to remove the assets/static/ folder is that it's so easy to generate it automatically, it makes the diffs less noisy and I see no real reason not to.

Do you really want to do another release every time someone submits something for the showcase? Or a plugin?

A release of what? It's not like we'd do a version bump every time the website is updated. The website would automatically be rebuilt and redeployed every time any of it's files are updated regardless of current release just like we currently do it. Basically nothing would change apart from there being one repository instead of two.

I honestly didn't think about the showcase or blog since they are so under-utilized. I'd argue that it makes some sense to have the blog at hand since it'll contain release notes and one could argue that the posts are documentation. However, the size of the showcase folder is a bit of a bummer.

It seems like there is some possibility to reduce the current size by forcing a preview image size, but that doesn't solve the problem if the showcase grows. We should also use LFS for the images probably. (A mad idea would be to run something like Firefox headless and visit the sites to generate the preview rather than keeping a binary image in tree)

Ideally I also think we mainly would do this for the docs, but I don't like the idea of splitting the lektor website into two parts unless we can come up with a really compelling solution for it.

I have no qualms with having multiple things in the same repository as long as they are somewhat related and it makes maintenance easier. I don't think the argument that "These things don't belong with the code" is good enough on its own.

Ok, so I forgot the website and the docs were they same thing. I suggest we split the docs and the website and move the docs inside the repo.

What is the docs build/hosting process is right now? Who has access to the server and domain? Might need to consider that if you change the layout.

The current deployment is supposed to be automated through Travis (https://github.com/lektor/lektor-website/blob/master/.travis.yml#L20). We probably need to extract the SSH key from Travis' secrets, which I guess anyone in the Website "Team" should be able to do. Could you check @goanpeca?

How do we do the split though? That's something we really need to consider before doing anything I'd say. Will they be two separate sites that don't share any assets? Will they look like one site but built like two? If so where do we manage the shared assets and how does that impact the contribution process?

I don't have the answers to those questions which is why I proposed moving the entire site.

Hey man, tone down that attitude. Let your arguments make your point instead of belittling others' suggestions. It's completely unnecessary because the points you raise and the arguments you make are completely fine on their own.

Sorry if I came across as .. belittling (really? sorry...). It was not my intention to be an ass. I just find the idea really, really awful, I guess I emphasized that until I overdid it.

I have not. I just did a fresh download in 5.829 s, which is something I very rarely. This is however of course not reason to dismiss the fact that the repository size may affect people.

You submitted a patch to the docs, I assumed you cloned the repo. That's how its amazing size got my attention in the first place. I don't know what you mean by 5 seconds, I just did a git clone, takes about a minute. That's awful.

The main reason why I wanted to remove the assets/static/ folder is that it's so easy to generate it automatically, it makes the diffs less noisy and I see no real reason not to.

Ok, we should make that happen in the website repo.

A release of what? It's not like we'd do a version bump every time the website is updated. The website would automatically be rebuilt and redeployed every time any of it's files are updated regardless of current release just like we currently do it.

Well, if you want to publish the site from master with a hook, fine, but the fact that we currently do it doesn't mean it's a good thing. Publishing docs from master is a bad thing, unless it's done automatically in /docs/master or somesuch.

(I don't know how publishing is done, but currently it's not automatically. I approve of doing it for the website, but for the docs, things are different.)

Basically nothing would change apart from there being one repository instead of two.

... and the fact that said repo would become chock-full of stuff has no reason to be there.

It seems like there is some possibility to reduce the current size by forcing a preview image size, but that doesn't solve the problem if the showcase grows. We should also use LFS for the images probably. (A mad idea would be to run something like Firefox headless and visit the sites to generate the preview rather than keeping a binary image in tree)

So, you keep coming with workarounds to justify your idea. (I'll say it again, I personally find this a really, really bad idea, so much that I'm surprised you'd even think of it. I really just can't get myself to understand your point of view, that's how large the divide is. I guess that's why my tone was so off... (sorry, again..))

LFS would maybe be a good idea for the devs, but not for the one-time committer, and that's precisely the people that would want to push something in the showcase.

Ideally I also think we mainly would do this for the docs, but I don't like the idea of splitting the lektor website into two parts unless we can come up with a really compelling solution for it.

My view is that the lektor website containing the docs (structurally speaking) was a mistake from the very start, that was perpetuated. I can only guess @mitsuhiko wanted to showcase lektor's capabilities and got carried away. But every other project out there has docs in the same repo with the code, for a good reason. And I don't know of any project that has the marketing material in the same repo with the code.

I have no qualms with having multiple things in the same repository as long as they are somewhat related and it makes maintenance easier. I don't think the argument that "These things don't belong with the code" is good enough on its own.

Well, I do, quite vehemently as you could see. Code is code. Docs are tied to the code. Website is website. Website integrates docs. Code does not integrate website. Single repo = mistake. That's the way I see it.

I am not saying this is a good example of content, but on Spyder we have a website (with lektor)

https://www.spyder-ide.org/ which serves a purpose and has docs https://docs.spyder-ide.org/

Which "looks"similar, but is actually a sphinx thing. I suggest we do something like that. Even if lektor was originally used for the docs, I don't think that was a great idea in terms of workflow, but more of a test of what lektor could be used for.

So the sites could look similar website and docs, but be generated and serve different purposes.

How do we do the split though? That's something we really need to consider before doing anything I'd say. Will they be two separate sites that don't share any assets? Will they look like one site but built like two? If so where do we manage the shared assets and how does that impact the contribution process?

The simplest way about this would be publishing on a different subdomain.

If we come up with a way to publish to separate folders per release tag (which we should), then we will have solved the problem that makes it difficult momentarily to publish into /docs.

I started separating the assets back when I submitted lektor/lektor-website#226 but gave up on the project. IIRC there's not much overlap, I think duplicating the few common things wouldn't really hurt, and it's a good idea to have the docs buildable as something really stand-alone. (If we really must share, then we could set up some overengineered git subtree workflow).

Which "looks"similar, but is actually a sphinx thing. I suggest we do something like that. Even if lektor was originally used for the docs, I don't think that was a great idea in terms of workflow, but more of a test of what lektor could be used for.

Well, fine, but that's a large project on its own. The docs have models and templates with custom stuff (all translatable into sphinx I guess, but lots of work).

I do think that having the docs lektor-generated is some sort of a "tour de force". Great marketing. Unless it's a gimmick, which I don't think it is. Is the lektor workflow so awful compared to the sphinx workflow?

Great marketing. Unless it's a gimmick, which I don't think it is. Is the lektor workflow so awful compared to the sphinx workflow?

It is not standard, which opens a bunch of cans of worms and raises the barrier of entry and possible contributors.

Well, fine, but that's a large project on its own. The docs have models and templates with custom stuff (all translatable into sphinx I guess, but lots of work).

I am willing to work on this front in order to keep docs that relate to code and how to use lektor close to the code and leave the website as a showcase and marketing site, which does not need to be updated so frequently.

I don't think that argument stands while the product's content format is
.lr. You'd expect a lektor contributor to know how to produce lektor
content. :) ..

Contributors are more than just people who write code or use the tool themselves. That is a very narrow (and harmful) vision of open source.

You submitted a patch to the docs, I assumed you cloned the repo. That's how its amazing size got my attention in the first place. I don't know what you mean by 5 seconds, I just did a git clone, takes about a minute. That's awful.

I meant that I just did a fresh clone in 5 seconds as an anecdotal way to say that I didn't notice the repository size, not that repository size isn't a problem. I do agree that keeping the size down is desirable.

To be perfectly clear:

$ time git clone git@github.com:lektor/lektor-website.git
Cloning into 'lektor-website'...
remote: Enumerating objects: 58, done.
remote: Counting objects: 100% (58/58), done.
remote: Compressing objects: 100% (46/46), done.
remote: Total 4694 (delta 23), reused 18 (delta 5), pack-reused 4636
Receiving objects: 100% (4694/4694), 50.94 MiB | 15.22 MiB/s, done.
Resolving deltas: 100% (2427/2427), done.

real	0m5,000s
user	0m0,784s
sys	0m0,234s

It seems like there is some possibility to reduce the current size by forcing a preview image size, but that doesn't solve the problem if the showcase grows.

So, you keep coming with workarounds to justify your idea.

I don't see how my reply justifies my idea, if anything it's against it. I'm basically just writing ideas that I've had that are dead ends. The headless Firefox idea is obviously silly and over engineered (and intended as a joke).

(I don't know how publishing is done, but currently it's not automatically. I approve of doing it for the website, but for the docs, things are different.)

I think it's supposed to be automatic, but the build has been broken due to the werkzeug.contrib issue. (https://github.com/lektor/lektor-website/blob/master/.travis.yml#L20). It did deploy successfully in October I think: https://travis-ci.org/github/lektor/lektor-website/builds/646114497

I like the idea of migrating the docs to their own /docs/<master/stable> (I'm equally fine with subdomain but then we need DNS access). I don't think it makes that much sense to maintain more than stable and master. Would we want to serve documentation for releases that we don't want anyone to run anyway? If we only have two alternatives we can just use different deploy targets which would point to separate which simplifies the process a bit.

If someone have access to flow.srv.pocoo.org I assume it's a simple change to nginx to serve the docs subdirectory from a different tree. That way we don't have to deal with serving one lektor tree within another.

Great marketing. Unless it's a gimmick, which I don't think it is. Is the lektor workflow so awful compared to the sphinx workflow?

It is not standard, which opens a bunch of cans of worms and raises the barrier of entry and possible contributors.

I really think we should use Lektor for the docs. It does a pretty good job. I think the structure of the documentation needs some work but switching to Sphinx will not inherently solve that. I agree with @xlotlu that Lektor contributors hopefully know how to use Lektor itself 😄. Maybe we're narrow minded here and if so I'm curious what scenario you're thinking of.

Seems like we all agree on moving the docs at least. I'm happy with that as long as we find a satisfying solution for how it fits together with the rest of the website.

Would anyone mind if we made the docs appear like their own separate thing, as in not directly part of the website? If we do want them to look like one entity, maybe a theme (I've never used them so I don't know their full potential) would be the best way to share the base code between them?

I meant that I just did a fresh clone in 5 seconds

$ time git clone git@github.com:lektor/lektor-website.git
Receiving objects: 100% (4694/4694), 50.94 MiB | 15.22 MiB/s, done.
                                                 ^ ah, well

There's the explanation. I get some 1MB/s here, I always presumed that's capped on the github side.

The headless Firefox idea is obviously silly and over engineered (and intended as a joke).

Well, whoosh me.

I don't think it makes that much sense to maintain more than stable and master. Would we want to serve documentation for releases that we don't want anyone to run anyway?

I don't have a strong opinion on this. It's more like.. what kind of project do we want to be? If we do backwards-incompatible changes will we want to maintain some old branch for a while?

I really think we should use Lektor for the docs. [...] Maybe we're narrow minded here and if so I'm curious what scenario you're thinking of.

In the beginning we need to go the Lektor way anyway. If anybody wants to do a Sphinx port let's do it after we have this in place?

Would anyone mind if we made the docs appear like their own separate thing, as in not directly part of the website? If we do want them to look like one entity, maybe a theme (I've never used them so I don't know their full potential) would be the best way to share the base code between them?

I have no position on this, until I see what comes out of the build process when it's a separate thing.

I offer to do the surgical filter-branch and stick-it-in procedure, unless someone else wants to do it. I did this a while ago so I probably won't take that long to re-rtfm. Once this is done we can deal with some real data.

Contributors are more than just people who write code or use the tool themselves.

They are? So what is the crowd we're catering to?

That is a very narrow (and harmful) vision of open source.

Ah, ok.

Hosting them on a subdomain and putting the docs in the same repo as the code is pretty common. That's attractive to me. The rest of getlektor.com can stay on it's own and update less frequently. Short term, the moved docs can look as similar as possible, and then once that's up we can modify it as we like.

Sphinx could potentially be used with Lektor. We could access it with a plugin or as a build step. "Both" seems like a decent option to me at some point, but it can evolve. I don't think we need to sort through that quite yet.

Back to hosting: I do not have access to flow.srv.pocoo.org. I think I asked Armin once if he's fine moving off of the server we're using. I can raise that question again. I think it would showcase Lektor even more to move both the website and docs site to something like AWS Cloudfront. That would further dogfood a rather prominent plugin, and would make the site faster. Maybe it's just me, but I definitely notice a slower-than-I'd-prefer load time on getlektor.com. That's not a good look for Lektor in particular. I'm more than willing to sponsor that cost.

If we're discussing hosting doesn't it make the most sense to just use the GitHub provided one? You can have custom domain names for it.

That seems fine to me. That's still dogfooding and moving away from the server to something that's about as fast as possible. I'd love to get this thing snappy.

I agree with @runfalk , AWS is a rock solid option for web hosting but it would be nice to know if the Lektor site sees enough traffic and has long load times to justify expanding Lektor project infrastructure vs simplicity of GitHub pages. As an example, if most of our website visitors are repeat visitors coming back for documentation, we don't risk of losing them as users if the site takes an extra second to load. Github pages still would likely be faster than what it is now.

If you don't mind being more and more tied up with micros.. err, github.