Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Speeding up generation #25

Open
pauldambra opened this issue Mar 22, 2017 · 5 comments
Open

Speeding up generation #25

pauldambra opened this issue Mar 22, 2017 · 5 comments
Labels
enhancement

Comments

@pauldambra
Copy link

pauldambra commented Mar 22, 2017 •
edited

Thanks for this plugin!

I found builds were taking 18 seconds for me.

require 'thread'
require 'thwait'

  # Generates a new AMP post for each existing post
  class AmpGenerator < Generator
    priority :low
    def generate(site)
      dir = site.config['ampdir'] || 'amp'
      threads = site.posts.docs.map do |post|
        Thread.new do
          index = AmpPost.new(site, site.source, File.join(dir, post.id), post)
          index.render(site.layouts, site.site_payload)
          index.write(site.dest)
          site.pages << index
        end
      end
      ThreadsWait.all_waits(*threads)
    end
  end
end

I amended the main loop to start a new thread for each AmpPost to be generated and then wait for them all to finish.

That took my build from 18 seconds down to 7 seconds.

Happy to open a PR if you like?
@juusaw
Copy link
Owner

juusaw commented Mar 28, 2017

Sounds good! I'd be happy to test the parallelization and accept the PR.

@pauldambra
Copy link
Author

pauldambra commented Mar 28, 2017 via email

@pauldambra
Copy link
Author

Just looking at adding this and having to include changes made here since I copied the files for my blog...

One of the changes added a layout:amp to the top of the AMP layout

---
layout: amp
---
<!doctype html>
<html amp lang="en">

but that just causes warnings and doesn't render anything for me...

Build Warning: Layout 'amp' requested in amp/2016/yarn/index.html does not exist.

I think I must be misunderstanding something..?

@juusaw
Copy link
Owner

juusaw commented Apr 1, 2017

Hi, did you update all the changed files before applying your updates? I managed to get the threaded version working from the latest version of master.

class AmpGenerator < Generator
    priority :low
    def generate(site)
      dir = site.config['ampdir'] || 'amp'
      threads = site.posts.docs.map do |post|
        next if post.data['skip_amp'] == true
        Thread.new do
          site.pages << AmpPost.new(site, site.source, File.join(dir, post.id), post)
        end
      end
      ThreadsWait.all_waits(*threads)
    end
  end

@pauldambra pauldambra mentioned this issue Apr 1, 2017
Closed
@pauldambra
Copy link
Author

:) It's worked for me without the skip for many builds now. If it's working for you too then was doing too many things at once this morning... I must have done something silly

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@pauldambra @juusaw