Greener coding - Making a 'gold' reference configuration with the Wagtail demo site

[mrchrisadams]

Hi there.

I hope this isn't too left-field for the discussion here. I'm a fan of wagtail, I've used it in production on a few projects, and personally it's my favourite CMS. After doing a few talks about django and climate, I wanted to see if there was interest in the community in demonstrating some of the ideas using Wagtail as as real world reference project.

I've recently been working with the nice folks at Green Coding Berlin, who have set up a test rig for running code, for a few given users journeys then tracking the energy usage for different deployment configurations of the software you might use.

We did it recently to make some comparisons between running a user journey for a Wordpress site that was hosted by a more common dynamically hosted LAMP stack, then comparing the same to using a 'baked' version of the static site.

The system works by taking a few processes like a browser, an app server, and providing a degree of isolation (shown here using docker, but only for convenience - you can use other more robust forms of isolation too), running them as a system under test, then carrying out the same user journey for different 'systems'.

The diagram, extremely simplified looks like this:

The actual running rig right now looks a bit more like this

Anyway, you can see an example of the charted data for the difference processes in this busy diagram below - the blue is the resource usage for a browser visiting a hosted site, the yellow is a Maria DB server, and the green is the Apache and PHP server:

There's more data if you fancy wading through it for this test run, but I need to stress, this is all in development, and I can't make any guarantees about the data being online forever.

By comparison, you can check the resource usage there for a common lamp stack style setup, versus a 'baked' version where once content has been created, you only need the static server.

You have basically the same user journeys, and you can see similar spikes in resource usage client side, but server side it's pretty much flat. If you look at the entire system data for the second run, you'll see that the total energy use for the whole system, including the browser is about a third of the energy.

It shouldn't come as a surprise that serving static files is less energy intensive, but I hadn't come across numbers on a per-run basis like this before, and I think you can use this a basis for some interesting ideas in django world.

Adapting this to Django and Wagtail

For the demo above, we used real content from the non profit I help run, The Green Web Foundation, but I think that the Wagtail Bakery Demo is a pretty good reference application, that you could run in a number of different deployment configurations to better understand the resource usage trade-offs associated with different stacks used.

I'm thinking you might be able to use it to answer questions like:

Is there a similar saving from using wagtail with the other bakery app, the wagtail bakery django app as part of a deployment process?

Is it possible to run the entire common wagtail editing user journeys using an entirely scale-to-zero stack?

_ (ie. some provider of Cloud Run-style services for app servers, using something like neon.tech for OSS scale-to-zero Postgres, and object storage for static files)?

Are the savings worth the extra hassle compared to running it on a VPS?

I'm not sure you can answer that with charts, but you can at least get some idea of the difference in resource footprint for the different deployment configurations.

Related - other ways to get metrics for high level comparisons

I totally appreciate that the project I've linked to above is only testing stuff you can containerise and run in the rig pictured, but there are other ways of getting numbers for other parts of the system too, for making some initial comparisons.

Below is an example for the environmental footprint from running code in a number of configurations from a "functions as a service" based way to comparing it to common setups for serving the same code, like running the same work in a VPS, or VPS with warm standby:

https://www.linkedin.com/pulse/quantifying-greenness-faas-lukasz-mastalerz/

There are also similar examples where other OSS tooling like cloud carbon footprint have been used to model out the likely environmental footprint of consuming other 3rd party services from larger cloud providers, like this post from Chris Ward.

This post demonstrates using cloud carbon footprint to model the likely footprint for a project handlig requests for 1000 concurrent users using a serverless system, gradually adding more to the project til they add an always-on cloud SQL service that ends up being the source of 90% of the projected energy use (mainly because it's one part that doesn't scale to zero when not actively used).

https://chronosphere.io/learn/increasing-cloud-native-sustainability-with-observability/

The code for the green coding tool

The green coding metrics tool is called imaginatively enough, "Green Metrics Tool", and is AGPL licensed. you can see it below:
https://github.com/green-coding-berlin/green-metrics-tool

There's a blog here:
https://www.green-coding.org/blog/

And various examples of metrics from other testing runs:
https://metrics.green-coding.org/

My ask - help making a "GOLD" Wagtail

I've been using the mnemonic GOLD ( green, open, lean, distributed) as a way to talk about the qualities of 'greener' development in a few talks in the django community. I've outlined a below with videos

• Greening Django, a talk I did at Django days in 2020 Denmark - https://www.youtube.com/watch?v=upiK4du5vUI
• How to be a Djangonaut in a climate emergency, the most recent talk I did at a DjangoCon in 2021 - https://www.thegreenwebfoundation.org/djangocon-eu-2021/

I like Wagtail, but I'm not as experienced with it as I'd like and it would be useful to get some pointers on how to get a system under test, like the examples above, but using a recent version of wagtail.

Why I think this would be interesting for wagtail users

If this can be set up I think it's then possible to quantify the resource savings with different deployment setups of wagtail, and this would likely be the first significant OSS community project I know where these kind of considerations are well documented with a open and defensible process.

I think this would also make it possible to quantify the impact of optimising certain hot spots in the code inside wagtail for representative user journeys over time too

I'd be very much up for contributing documentation and recommendations to the wagtail project based on what we might learn if there was interest.

So, to recap... if:

1. this would be of interest to you
2. you have worked with the wagtail bakery demo enough to know how to set it up easily
3. you're interested in helping get a version of it set up to test different deployment configurations

Would you please leave a comment below?

If nothing else it might make a fun wagtail space project in future (I'm sorry I missed the last one in June…)

[Scotchester]

I may not be the best person to help you out with this (or have the time), but I wanted to give my support to this really cool idea!

[thibaudcolas]

Sounds pretty cool to me! I don’t have the time to help with instrumentation right now (our next release is due soon) but am generally very interested in Wagtail’s climate impact, and in particular in documentation and recommendations we could make to implementers.

To start with, maybe it’s the right time for a #sustainability channel on our Slack like WordPress? There’s been discussions about the impact of Wagtail in the past, but mostly behind closed doors.

---

Back to the premise of your proposal @mrchrisadams, I have two questions:

• I tried something similar about a year ago with Marmelab’s Argos (repository, blog post), now GreenFrame. Can you say more about how Green Metrics compare? Is it just the appeal of the tooling being open source?
• Do you think there’s really much to learn about comparing different ways to host a site? I’d have expected a baked version to always be orders of magnitude less carbon-intensive to host and serve, and a serverless "scale to zero" setup to similarly always fare better than the equivalent PaaS due to idle CPU power consumption (the more over-provisioned the worse, no matter the tech stack). Or is the value here about quantifying different optimisations as you mention?

[mrchrisadams]

I tried something similar about a year ago with Marmelab’s Argos (repository, blog post), now GreenFrame. Can you say more about how Green Metrics compare? Is it just the appeal of the tooling being open source?

The open source element is of interest here - I really think in this domain that transparency is really important, and basically necessary to inform any sensible policy. Almost every academic paper I read is calling for this, and they largely say "our research is ok, but we can't say if this applies across the rest of the industry", and even if we us the best peer reviewed data available, we keep finding releases that call that into question too.

An example below is a thread showing how one release from China's state Grid Research Institute put the energy use by data centres there as the same use as the global estimates from the IEA.

https://twitter.com/g_roussilhe/status/1519283270973861889

There's also an argument for working to create an ecosystem that allows for a greater number of players to work.

Even on the FaaS / PaaS front, there's research released this month which expands on the tradeoffs associated with serverless, complicating the matter somewhat:

See this paper, Challenges and Opportunities in Sustainable Serverless Computing announced at the recent Hot Carbon conference:

Our preliminary empirical investigation suggests that FaaS applications can be up to 15× more energy hungry than conventional web services. This energy and carbon (in)efficiency is unfortunately a fundamental attribute of serverless functions owing to their programming model and security isolation requirements. As FaaS usage continues its exponential growth, understanding and narrowing this energy gap will be vital for the carbon footprint of the overall computing ecosystem.

So, that's one order of magnitude the other way already!

About the green metrics tool

I'll be honest, I'm still learning more about the various parts of the Green Metrics Tool and how it works in detail.

If there's interest, I'd be happy to write some more on the Green Web Foundation as I explore it, as I have some familiarity with other projects in this domain now like Scaphandre.

I'd also likely join a #sustainability channel in Wagtail - I think being able to demonstrate good practices there, just like in Wordpress is a fairly high leverage place to devote time in this field, and demonstrating the ideas in public projects makes it easy for other projects to learn from.

[thibaudcolas]

So, that's one order of magnitude the other way already!

Damn! 😄

---

Let’s see what others think – personally I’d recommend starting with the channel, and following with a "sustainability considerations" / "climate impact considerations" / "energy considerations" documentation page like we’ve created accessibility considerations. This way we can document everything that impacts emissions on the one page, providing guidance for things that Wagtail doesn’t do too well right now, and documenting good resources on the topic. And once we’ve done the research needed to put this type of page together, we’ll probably be able to put a backlog of potential improvements together.

On the hosting side, I don’t think we document Wagtail’s requirements much because it’s "the same as Django" (paraphrasing our docs) and we generally want people to refer to Django docs where possible, but there’s a case for us to say more if it reduces the impact of Wagtail sites out there.

[gasman]

This is really interesting - happy to help where I can! I've done plenty of setting up of bakery demo instances, although less so when it comes to actual production setups.

On the subject of wagtail-bakery for generating static files, the underlying django-bakery package is currently lagging behind on Django 4 support, and it's a little unclear how much work it's going to take to get that project back up to speed with regular CI testing. As a result, I recently came up with a replacement, wagtail-freezer - currently very minimal, with just enough functionality to get it working on my personal site. Happy to hear any feedback on how it can be made more widely useful!

[mrchrisadams]

Hey Gasman - if I understand this correctly, we'd need to either run this with Django < 4.x to create a test set up for with the Wagtail Bakery Demo, or we'd need to use Django 4.x with wagtail freezer swapped into the Wagtail Backery Demo demo - is that the case?

[RealOrangeOne]

Speaking as someone who has run numerous Wagtail and Django sites in production (both at Torchbox and elsewhere), this sounds like a really interesting avenue of research. If you're optimising purely for carbon efficiency, there definitely a lot of dials to turn, but I'm not sure how many of them are necessarily Wagtail or even Django specific? One could argue if that's what you're chasing, Python might not be the best choice.

With your LAMP stack, what recommendations were you able to give? Do you imagine any of them being adaptable to a Django-based application? Measurements is 1 thing, but actions are presumably the main goal here.

That said, definitely happy to help out where I can! Suspect there might be some interesting links between this and the performance team

[RealOrangeOne]

Is it possible to run the entire common wagtail editing user journeys using an entirely scale-to-zero stack?

Definitely possible (even without hacks)! So long as you don't need scheduled management commands. It's also worth noting that on a high-enough traffic site, 0 is pretty unlikely, and warm up can be quite an expensive operation.

[ArneTR]

Hey all, Arne here from Green Coding Berlin.

We are the developers behind the Green Metrics Tool that @mrchrisadams mentioned earlier.

If we could contribute some work on this topic, we would be very helpful to. So I thought I join the discussion here.

In the topic there are two repositories mentioned and it feels to me like they might be convoluted.

• Chris mentioned the Bakery Demo App, which to my understanding is a Django Reference Implementation
• @gasman mentioned the django-bakery and WagtailFreezer, which to my understanding are static site transfomers for Django.

So far correct?

On the topic of comparing static site builders with the energy cost of requests to a dynamic CMS we have for instance made a small article about to get an idea of the order of magnitude they differ: https://www.green-coding.org/case-studies/wordpress-vs-hugo-cloudflare/

The other topic, what Chris mentioned I guess, is to get an idea of the order of magnitude the general energy consumption a Django project lies in it would be helpful to have a standardized setup & interaction with such a project.
This makes the [Wagtail BakeryDemo app](Wagtail BakeryDemo) so attractive, as it is a reference implementation.

My question: Is there a standard setup of Unit Tests / E2E Tests / Selenium interactions with the [Wagtail BakeryDemo](Wagtail BakeryDemo) app that we could use to run such a measurement?
We would be really happy to put some working hours into this and generate some open data on it.

[thibaudcolas]

hey Arne, glad to have you here :) All correct, and it’s indeed confusing that our demo site is about a bakery, and that there’s a separate "bakery" project that’s essentially just a playful name for "baking" a server-rendered site into static files.

Thank you for sharing this article, that’s exactly the type of resource we need when advising people on the characteristics of different hosting options! In our experience it’s tricky to generalise results like this because,

• For sites serving any significant traffic, we’d use a reverse proxy with caching, so a lot of requests wouldn’t even reach Wagtail (I don’t have access to stats on cache hit rates myself unfortunately so can’t back this up with actual numbers)
• Those sites serving significant traffic also often have a lot of pages, and static sites’ build times tend to increase worse than linearly with page count.

I don’t think we have a standard testing setup for our demo site right now. We’ve occasionally run tests on it but only as one-off efforts. I’d love to put this together if it could enable this work – could you share the kinds of scenarios you would be after?

The bakerydemo website is very simple, so whether we want to produce a sample of pages to navigate one by one, or more realistic user journeys from page to page – it should be possible for us to put this together quickly.

[ArneTR]

Hey thibaudcolas,

Regarding your notes:

1. The reverse proxy thing is common practice, I agree. Nevertheless in a setup like with Cloudflare Pages you have the benefit that the site behind the proxy is actually turned off if not needed.
   Talking about high volume sites this might however never be the case though, as at least one request every 15 minues (which is their spin-down time) should go through still.
   Real numbers here from Cloudflare for instance would be great. We asked them on a similar topic but they did not want to share any such data.

2. Build times in large static sites should increase because of the cross-references of the pages non-linearly. Would it not be the same for a dynamic page,as it contains more cross-references to other content forms like data in joined tables, plugins etc. also their energy profile should increase in the same fashion?

Regarding your question:
The scenarios we use are in line with the concept of the "Standard Usage Scenarios" that the Blue Angel for Software proposes.

It should be a proper representation of what the author assumes (or has measured data on), what the interaction pattern with the website is.

For the Blue Angel application of the KDE PDF Reader Okular we did the interaction pattern with xdotool. Here the repository with the examplary file

For websites we typically use interaction patterns in Selenium / Puppeteer)

All we need to feed it into our tool is therefore a Puppeteer file. It would be optimal if all subpages of the project are queried at least once and if something like forms, authentication flows or APIs is present this should be interacted with.

Also anything that I forgot to mention but would produce a load and would count as a typical user interaction should be included.

This Puppeteer file we containerize then, so our tool can separate this out and do the measurements.

[thibaudcolas]

👍 that sounds excellent to me. I should be able to produce a set of representative usage scenarios for our demo site. I’ll try to base those representative scenarios off real-world sites that have a similar structure if possible. I have a lot of experience with Puppeteer so it should be no problem for me to use that format.

It would be optimal if all subpages of the project are queried at least once and if something like forms, authentication flows or APIs is present this should be interacted with.

We have a few examples of such functionality, will make sure to add them. When you say "all subpages" – do you mean "query at least one page of each page type", or literally "all pages of the site at least once"?

---

Build times in large static sites should increase because of the cross-references of the pages non-linearly. Would it not be the same for a dynamic page,as it contains more cross-references to other content forms like data in joined tables, plugins etc. also their energy profile should increase in the same fashion?

Yes, except in a dynamic site the "cross-reference" pages are only built once requested. For example, if I have a paginated and filtered list of breads – a static site would have to build all the pages of a paginated list, with all filter combinations, even if they’d never end up getting used. A server-rendered site will only build a given combination of "pagination page number + filter" once that specific combination is requested.

---

One last question –– is it helpful at all if we also provide a static version of our demo site, using django-bakery, wagtail-freezer, or even a simpler "wget export"? Would you be interested in comparing that with the normal server-rendered stack?

[ArneTR]

We have a few examples of such functionality, will make sure to add them. When you say "all subpages" – do you mean "query at least one page of each page type", or literally "all pages of the site at least once"?

Yeah, thanks for asking here. I meant that every type of page should occur at least once in the usage scenario. It does not have to be all occurences of that page type. This can be easily extrapolated given the assumption that every page of the same type should behave identical.

One last question –– is it helpful at all if we also provide a static version of our demo site, using django-bakery, wagtail-freezer, or even a simpler "wget export"? Would you be interested in comparing that with the normal server-rendered stack?

We can absolutely compare that. Showing this potential in a bigger scope like this would be really interesting. In order to do that I would containerize the build process to test the building step and then also containerzie the delivery phase by putting a static site server like NGINX in a container.
Would be really interesting if this project does have these multiple filter sub-pages that you were talking about and getting some hard numbers of what the effect really is.

[thibaudcolas]

Hi all, took me a while to get back to this but here we are! Here is the proposed reference configuration as well as tentative automated Puppeteer scripts to benchmark it: https://github.com/thibaudcolas/bakerydemo-gold-benchmark

This is based upon our vanilla bakerydemo, with additional Django and Docker configuration improvements to be more representative of a real-world production site – and a few compromises on top so it can still be run locally. I’ve documented those improvements and key differences with a real site in the README for future reference.

The README also contains instructions on how to benchmark the site,

• There’s a critical warmup crawl of the site, which is fundamental for our image processing to kick in (we only store our source images in git, not their optimised versions).
• I’ve produced 5 different scenarios as Puppeteer testing scripts that are representative of user journeys.
• And while I was at it I’ve also produced 5 equivalent Playwright scenarios for use with the newly-released GreenFrame CLI
• How to export a static site and serve it with nginx – though note only 2 of the 5 scenarios will work.

I’d love any and all feedback on this, and particularly from you @mrchrisadams and @ArneTR whether this seems like it’ll work with Green Metrics.

---

One particular point I could use feedback on is the obvious ways in which this will differ from a real-world site and how (or whether) to account for them.

With this setup, the site is:

• Running locally rather than in a data center
• Not using a reverse-proxy CDN caching requests (at Torchbox all the sites we manage use either Cloudflare or equivalent AWS, Azure, GCP services – for as many requests as possible).
• Serving media files with Django (bad practice) rather than dedicated object storage service (S3, Google Cloud Storage, etc)
• Running over HTTP rather than HTTPS

And for the user journeys – the main shortcoming I believe is how fast an automated script would go through the steps compared to a user. I’ve introduced a few delays so the test cases run in 10-20s rather than 1-2s, but that’s still a good order of magnitude faster than real-world user journeys. Here are sample times people spend on different page types across sites we manage as an illustration:

• Homepage: 2min
• Listing page: 1min
• Search results: 30s
• Blog post: 1min30s to 8min

[ArneTR]

Uhh, very nice!

I will get on these measurements asap ... hopefully before christmas, but if not then def. in the days between NYE.

Regarding the GreenFrame CLI: Nice that you are checking out that tool. We were very happy that they finally open sourced it. Do you have the results somewhere for qualitative comparison?

Regarding your questions:

• Running locally rather than in a datacenter will give you a quantiative difference. However qualitatively the results should everywhere be the same (if the tool is good, which we hope the Green Metrics Tool is :) )
• Caching is obviously a typical live setup and will have a different energy impact that cannot be measured in this setup. However the idea in general would be to compare Django to a different system and thus both systems would be compared without the caching layer.
• Are the samples testing file downloads or video streaming? Or do you mean serving images?
• HTTP vs. HTTPS is generally a concern which will lead to a bit of a missing overhead. However as soon as testing scenarios are kept constant and comparisons also use HTTP/HTTPS respectively this can be evened out.
• User Journey times: Same argument tests have to be stable over time and comparisons equal.

Since no current agreed upon way for a "standard test scenario" for web-frameworks exists we just have to assume one.
Once more people look at it and send remarks it can then be thought of proposing a standard at some point.

I believe this test here is very early and reasonable assumptions are the best for now. I will send remarks however on the setup as it is regarding our internal experience on how we design energy test setups.

[thibaudcolas]

Yay! Thanks for the feedback, I agree on all the above.

Are the samples testing file downloads or video streaming? Or do you mean serving images?

That is serving images. The amount of optimisation done to images will be identical. In a normal Wagtail setup images would be served straight from S3 (or equivalent), so request handling for images would have 0 server impact. Here, they’re served by the server. Not ideal but hopefully a reasonable compromise to keep this benchmarking simpler to run.

The results from GreenFrame are in the README under Playwright scenarios for Greenframe, as well as the exact commands I ran. Copy-pasted:

✅ homepage-landing completed
The estimated footprint is 0.093 g eq. co2 ± 6.1% (0.21 Wh).
✅ search completed
The estimated footprint is 0.041 g eq. co2 ± 4.9% (0.092 Wh).
✅ blog-filtering completed
The estimated footprint is 0.063 g eq. co2 ± 1.8% (0.142 Wh).
✅ contact-us completed
The estimated footprint is 0.035 g eq. co2 ± 6% (0.078 Wh).
✅ admin completed
The estimated footprint is 0.155 g eq. co2 ± 8.8% (0.352 Wh).

Interpreting those results is lost on me. The CLI gives very little output (basically just the above), so I’m not sure whether it correctly measures the right containers.

[ArneTR]

Just a quick update as I did not stick to my initial time planning ... holidays were a bit more relaxed than expected :)

@ribalba is currently working on the task and we expect to have something ready by mid / end next week.

The scenarios are quite straightforward. The only thing we have to do here is to annotate them so they display better in graphs and one can easily see when which step is executed.

A question I have out of curiosity and for comparison regarding particular the Greenframe measurements:

• Did you also measure the cost of creating the static page?
• Did you also measure the cost of warming up the cache?
• Are the Greenframe measurements with or without warmed up cache?
  • Did you also test in the non-warmed up / warmed up variant?
• Did you measure the cost of the migrations?
• Did you also measure the cost of the generation of the static HTML files?

[thibaudcolas]

Thanks for the update! I didn’t measure any of those things (more on that below), mostly because I’d expect all those things to be very insignificant compared to the effort needed to serve the site.

Did you also measure the cost of creating the static page?

No. I did it by running wget on the live server, so the cost per page is essentially the same as serving the page once – plus wget saving the file(s).

Did you also measure the cost of warming up the cache?

Also no. That’s something we’d love to optimise, but in practice this image cache only needs creating once per image over the lifespan of the image on the site. So the carbon impact is very minimal.

Are the Greenframe measurements with or without warmed up cache?
Did you also test in the non-warmed up / warmed up variant?

All warmed up, no testing without it being warmed up because that would be a very unusual instance in a real-world site (since it’s just the very specific caching of the image optimisations), not page-level caching.

Did you measure the cost of the migrations?

No, that’d be interesting!

Did you also measure the cost of the generation of the static HTML files?

Also no.

[hazho]

thanks for this discussion;
measurements should include everything, in order to get an accurate result, for example the CDN for static files should not be exempted, because the static file that been served of course been served for the contents that is in the wagtail based websites, but most importantly the admin site should be measured, because the visitors site can be variously affecting the climate warming due to the page editors various practices, for example in a2z-eco-sys.com family of tenants, there are some of the tenants that doesn't follow the best practices regarding using contents and building their pages, although everything had been explained very well to all of the tenant's admins, but what remain almost the same is the admin site and the number of static files and their weights that been served through the web

[thibaudcolas]

There is a measurement of the admin site in the benchmark! Relatively simple compared to real-world usage, but still.

You’re right about the CDN usage to serve images, it’d definitely be more accurate with it, but the benchmark would also be harder to set up / results harder to replicate. So I thought it’d be a reasonable tradeoff. Certainly if this becomes something we do regularly, we’d look at improving accuracy.

[ArneTR]

Hey @thibaudcolas ,

we have forked the repository to make it clear what changes we made to make it run with the GMT: https://github.com/green-coding-berlin/bakerydemo-gold-benchmark

I hope the Diff shows that only minor touches were necessary:

• we funnily run on the same port with our tool. Since in our current test environment the frontend is hosted on the dev machine itself the come into conflict and had to change that
• We updated the puppeterr flows to add a timestamp to their console.log()
• We added the respective .yaml / .yml files that are needed to instruct our tool what to do. They are extemely similar to the present docker-compose.yml.

Really looking forward to your feedback especially if the usage_scenario.yml file is directly readable for you, as this is the current design goal :)

2. Here are the results:

• Wagtail Bakery Gold Benchmark - Admin
  • YAML File
• Wagtail Bakery Gold Benchmark - Cache Warmups
  • YAML File
• Wagtail Bakery Gold Benchmark - Migrations
  • YAML File
• Wagtail Bakery Gold Benchmark - Contact Us
  • YAML File
• Wagtail Bakery Gold Benchmark - All Routes
  • YAML File

The static tests we have excluded in this first run, as they need a different setup in our tool, that we are working on. I expect to have them ready by end next week.

An note on the test setup and result quality

• In the dev machine we used the GMT with its frontend and database runs on the same machine. Our other setup is currently broken :(
  • Nevertheless the processes have under 1% utilization while running the tests and should have no impact on the qualitative results
• We measured the Cache-Warmups and and Migrations separately. Also the routes single as an example and all together. I thinks this gives a good overview of the qualitative energy distribution
• The absolute values are different from GreenFrame, as different machines / calculation models will always give different results. What we typically look for is that they are in the same ballpark, which is the case here

Please give it a look and ask questions. I think from here we can now go next steps and am interested in what questions you have:

• More measurements?
• More metrics?
• Stronger comparsion with GreenFrame?
• More detailed introspection in some routes?
• etc.

Thank you for your great work! It was very easy to build on your gold benchmark. I was really blown away by the throrough documentation and all the prepared files!!
Also thank you to @ribalba who made all the work in our fork internally

[thibaudcolas]

Thank you @ArneTR and @ribalba 😌. Just wanted to acknowledge I’ve seen this, and it looks super solid, but I’ve not had the time to digest the detailed results yet. It’s on my list to do, at the latest in early Feb. We’ll also have other people at Torchbox (and hopefully wider Wagtail community) dissecting the results.

There seems to be an issue with the results links you provided – I was able to access them as-is around the time you posted your update, but now get an error of "Could not get project data from API
Technical error with getting data from the API - Please contact us: info@green-coding.berlin".

Investigating a bit,

• The provided link is https://metrics.green-coding.org/stats.html?id=2821c396-98f0-4210-8aad-a9fc5a37f01e
• It redirects to https://metrics.green-coding.berlin/stats.html, which has the error
• Manually going to https://metrics.green-coding.berlin/stats.html?id=2821c396-98f0-4210-8aad-a9fc5a37f01e seems to work as expected

[ArneTR]

oh, sry, we changed the domain to .berlin

apparently the ID gets lost in the translation process.

I changed my old post to reflect this change. Please try again

[ArneTR]

Also a small feedback on the results: We have a meeting on Tuesday on the results for the static pages we still have to deliver.

So when you look at it in Feb. you should have it all ready I hope :)

[ArneTR]

Hey @thibaudcolas , sorry for being silent for so long.

We have been working internally not only on making the static measurements work easier, but also on revamping our Green Metrics Tool so that it can natively compare softwares and also incorporate stuff like phases (think of container build, container boot etc.).

We have modified the repo you have created in the following ways:

• Adding a variable to the Puppeteer scripts so that we can inject a domain name (which varies between static and dynamic setup)
• Isolating the process of warming up the caches from the actual measurement by putting it into the "installation" part of our tool
• Separating out the static case

A dynamic site measurement

A static site measurement

To make it easily viewable what we have changed I have opened two pull requests:

• functionality for static page measurements with Green Metrics Tool
• functionality for dynamic page measurements with Green Metrics Tool

Also, the I hope most interesting part: Here would be the comparison between these two cases

Very interested in your feedback!

The idea for the pull requests in particular would be to track the changes of Wagtail over time and see how the energy compares for this reference implementation.

Our tool can track changes over time through a "Repeated Runs" or "Commits" comparison as we call it. Effectively you will the the changes over time aggregated. See an example here: Repeated Runs comparison