-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Chromium Build Discussion #26
Comments
@Alex313031 Windows AVX is thankfully one of the easiest edits to set SIMD level. In the WIndows build config: //chromium/src/build/config/win/BUILD.gn Simply search for -msse3 and replace with -mavx. Clang will propagate the -mavx flag to lld for LTO, so do not worry about an ldflag for AVX. Default SIMD levels aside, the default x86-64 instruction model in LLVM is actually Intel Sandy Bridge, which already includes modelling for AVX instructions when using -mavx. If you are cross building under Linux for Windows, you need to do the same in the compiler config as well, just like if you were natively doing a Linux build. The reason is libc++ pulls its build config from there. //chromium/src/build/config/compiler/BUILD.gn Make sure you disable AVX in the tflite config. It will not build successfully with AVX and above. //chromium/src/third_party/tflite/BUILD.gn Add the following cflags to the tflite config for a x86-64 build:
I have updated the Widevine args. Windows builds just pushed. The Bitmovin demo works. https://bitmovin.com/demos/drm Awaiting a reroll of Linux builds after a Polly config update. Sadly nothing new there, which is one of the reasons I tend to tell people trying Polly to use a very basic Polly config to start. I have a new Ryzen 8C/16T notebook on order. I would not be surprised if it turns out (much) faster for building Chromium than my two old 32-core Opteron systems. |
Hello
I tried Win-32 folder in the archive you've uploaded to the repository here. It turned out to be x64. I started chromium version 97 and the account folder is created in "\AppData\Local" - which for me is totally unacceptable. |
My personal interest is optimizing Chromium performance. I do not care much about portable releases, and I have no personal interest in ungoogled patches. Long story short, ya'll get the same builds I use. ;) Straight copy-and-paste from chrome://version in the current Win32 zip archive: Chromium 97.0.4670.0 (Official Build) (32-bit) That said, you might have snagged the Win32 build that was up for awhile yesterday, and it might have been packaged or pushed incorrectly. Happens occasionally. Anyway, it was already well outdated when pushed, so I went ahead and simply removed it to be sure. Jerry (woolyss) has a server-side script to generate portable releases (using crlauncher) of submitted builds. You might look into that if interested: |
Exactly from his site I came with a link to your repository. ;) In my opinion, the browser should not be as fast as secure, so I think there is a need to fight for the privacy of browsers, not speed. Without a doubt, the developers of Chrome make them cumbersome enough, taking up a lot of memory and so on. Confidential browsers remain on the fingers of one hand. And from this point of view - I have no interest in any speed optimizations, as long as the browser is not permeable to telemetry.. In other words, if it's not a Marmaduke - end with Ungoogled. How and why - because not everyone has the knowledge to compile to do it themselves for each new version. Lastly, I use Chrlauncher, but there is simply no collision of Chromium privacy developments. Only Marmaduke is relied upon, he at least quickly compiled and published them for download, while Elostone stopped publishing new versions of Windows. Because I'm looking for a sure, protective browser as the Ungoogled is for now. If your developments for new versions were what I'm looking for ... but alas - they are not. ;) It is a pity that the trend is such - to regress for consumers and withdrawing them to confidentiality, withdrawing freedoms. |
Thanks as always!! Also how does one contact Jerry. I made a comment but I don't think he will see or do anything. Things are incorrect. Carl's (who makes Bromite) builds for android need to show that they have all codecs, and marmaduke needs to be reverted from all codecs+ to all codecs, because H.265 hasn't worked for a while now. He was only using build flags, which make the browser REPORT mime handling of HEVC, but won't actually play it, as he has not actually modified the proper header files like Nik used to do. However, doing this on my end failed on linux, I'm not sure if it is windows specific. I'm also going to notify Marmaduke. Also, I'm starting to build often enough that I would like to see if he would consider posting my builds for linux. Also I cannot for the life of me get cross building to work!! I would really like that so I don't have to boot into windows. How are you making your zips. I have been opting to manually copy everything to /home/alex/bin, and running my own build artifact cleanup script, and making my own .desktop file in ~/.local/share/applications, but would like it to auto make it into a zip. Lastly, where did you find info on downloading the PGO profile, the setup and mini_installer targets, i.e. how does one list all the available targets for ninja/autoninja. These are notably absent from chromium documentation. Also I attached my script and .desktop file (modeled after debian's) for shits and giggles if you or anyone else finds it useful. I also included a .desktop file to run the content shell if you compile blink_tests, as well as a folder (for content shell) and single .svg (for chromium) to copy to /usr/share/icons/hicolor/scalable for both desktop files. |
Jerry's email address: https://info.woolyss.com/ On Windows with VS2019, WinSDK 19041, and Python3 installed to default directories. Go to: /depot_tools/win_toolchain And run: python3 package_from_installed.py 2019 --noarm -w 10.0.19041.0 Your choice on ARM support. I do not have it installed in my WinSDK config, as I do not target Windows builds for aarch64. WinSDK 10.0.22000.0 might work as well. I know the script can package a toolchain from testing earlier this week, but I have not actually tried cross building with 10.0.22000.0, yet. The script will assemble and spit out of the needed SDK files in a zip archive. For example: 476813973f.zip Copy that to somewhere on your Linux box. I have mine in /home/robrich/vs2019. So the first part would be redirecting the Windows toolchain URL to a local directory: export DEPOT_TOOLS_WIN_TOOLCHAIN_BASE_URL="/home/robrich/vs2019" Next is overriding the default toolchain hash by setting the hash for the generated archive file. export GYP_MSVS_HASH_3bda71a11e=476813973f Having to find and override the hardcoded hash is kind of annoying, but it does not change often. It is currently 3bda71a11e and can be found in the //chromium/build/vs_toolchain.py script file. Make sure you have target_os = ['android', 'win'] in your //chromium/.gclient config. Now when you export those values, running gclient sync should pickup the toolchain change. Alternatively, you can manually do the same from the //chromium/src directory if desired: python build/vs_toolchain.py update --force Add target_os = "win" to your builds.args and proceed as usual with your build process. There are various ways about obtaining PGO data. I just manually run the PGO script to fetch it. python tools/update_pgo_profiles.py --target=linux update --gs-url-base=chromium-optimization-profiles/pgo_profiles There is no 32-bit PGO data for Linux builds. There might be a 32-bit x86 Linux AFDO sample profile generated by the ChromiumOS project, but I have not looked lately. AFAIK, I have never even done a Chromium 32-bit Linux build. There are various installer and packaging scripts under the //chromium/src/chrome/installer/ directory. You might have to skim the scripts to get ideas. For example, here is how to do a deb package when building: ninja -C out/release "chrome/installer/linux:unstable_deb" You probably would want to clean up the Google Chrome stuff if doing a deb package. Remove the lines below remove_udev_symlinks in the following files: //chromium/src/chrome/installer/linux/debian/postinst You can extract Chromium from the deb if desired. For example, something like this: ar x chromium-browser-unstable_97.0.4670.0-1_amd64.deb The data.tar.xz archive has the files you want; already in their recommended directory layout. BTW, in theory, you might be able to even use alien to convert the deb package to various other package formats. Never tried it here, though. For a list of what ninja targets, try: gn args out/release --list > targets.txt The large list of targets are piped to the text file, though I am not sure installer targets are listed. Been awhile. |
Thank you. I know about the different pgo datasets, but where did you find your information, like where is it in the documentation. On cross building, that is EXACTLY what I've been doing and it still fails, but havent tried 'python build/vs_toolchain.py update --force' so I'll try that. If it still fails ill post the error output here. |
Answering to #23 You said -avx encodes both AVX and SSE3 in VEX, but I found that -avx2 does same thing. Dump screenshot attached (built with -avx2 -march=skylake). As you see, vmovups used instead of movups while using 128- and 256-bit registers close enough to each other. And so on. I guess clang is smart enough to encode instructions to minimize penalties. There's another dump snippet of same binary, instructions are not VEX encoded. But there's no mixing with ymm registers so there will be no penalty. Compiler decided to use "old-style" encoding. So, don't underestimate compiler, I think there's a reason to use benchmarks to decide if avx2 builds are better. |
You are overthinking it. Naturally -mavx2 would do the same thing. Why wouldn't it? It includes AVX. Just as if you set -mavx512, and you get AVX and AVX2 as well. All of them enable VEX encoding. The second screenshot does not include mixing of XMM and YMM instructions, which is what we are discussing here. Throw some AVX code in there, then see what potentially happens. Now think about Chromium. Several components have CPU dispatch, which can cause mixed encodings, as LLVM is building part of the code at -msss3 and the dispatched code at -mavx, -mavx2, etc. LLVM is not going to optimize the -msse3 code for VEX encoding using the default x86-64 processor target. If you want an AVX2 build of Chromium, do it. ;) Going a little further back, I used to target Haswell and tune for Skylake. I did not want to pull in any instructions from Skylake or later, as I was pushing public builds for larger consumption, but I did want to target my primary Kaby Lake system with instruction tuning. |
@Alex313031 The Windows zip archives are just reformats and recompresses of the chrome.7z archive generated when building the mini_installer. ;) I started doing the zip archives to appease a few people that complained about installing 7z or similar to use the files. The only thing I do on Linux is generated debs, which can be unarchived if desired. The ungoogled project apparently generates a portable Linux config, but I have not bothered really looking into their process. BTW, make sure you are building without any debug symbols. I do not think symbols are pulled into chrome.7z anyway, but it has been a long time since bothering with them, so just covering the bases here. |
Before I forget, about documentation, AFAIK it simply does not exist for some features. I have picked up lots of build processes over the years through crbug.com reports. git changes, looking at project buildbots, skimming the source, etc. |
@RobRich999 Oh lawd this is half of what I've had to do to learn about building chromium, you'd think PGO and build targets would be documented. Only chrome, chromedriver, blink_tests, and content_shell are well documented. And about the windows archives ah that makes sense, does the installer or build process use its own p7zip or lzma/lib7zip, and yeah duh you don't make linux archives. If you ever do make linux zips check out my cleanup script as I mentioned, it's only meant for linux anyway. For making a portable I have a bash script called cr and a .desktop file and all it really does is this > '--user-data-dir=../.profile --no-default-browser-check --allow-outdated-plugins --disable-logging --disable-breakpad' I use this because I use debians chromium which is sadly I think going to remain stuck on M90 for buster, but mostly use mine, and I want them to run independently of each other with my own builds not touching the system outside of its dir, this is all that is needed on linux to make a portable install. So is there anything else on build targets, bc idk how you're getting a list of those from gn args out/Default --list. Also, are you using is_official_build, if not (and even if you are because there's not good documentation on what is_official_build actually changes, so I'm not sure if it sets these.) you can set these to further remove debugging stuff (reduction of about 40mb when I ran du -h in my out dir after running cleanup script.) A last flags thing is that webui is increasingly being used in chromium and chromium os so this is worthwhile to set but in my testing increases build time by 10 mins (but again im on a FX-8370), probably not an issue for you. . If you wanna know more > https://chromium.googlesource.com/chromium/src/+/HEAD/docs/webui_explainer.md Did you try out my chromium-dev-editor, it doesn't matter if you didn't but if you did how did it run on win 10 avx. One known bug that I can't seem to figure out is that the menu items wont respond to clicks and you gotta use the arrow keys. I majority of it is programmed in dart and idk much about it. Most everything besides updating the icon, about section, and .crx manifest did just took a bajillion google searches and stackoverflow lol. Lastly, since you shared your args.gn, I'll share mine, which attempt to make an "as much like google chrome in features and performance" build as possible. Some of the things in there wouldn't be there without your help so cudos. google_api_key = "" |
Oh shit I put my api keys!! how do I delete a comment??? |
Nvm figured it out, and I just edited it. |
A step furhter, outright disable PDB creaetion by searching //chromium/src/build/config/compiler/BUILD.gn for config("no_symbols") and below set /DEBUG:NONE for Windows builds. Just doing a very quick skim right now. ;) I will try to comment on the other inquiries later today or maybe tomorrow. |
Then why not to build ALL the code with -mavx2 not only dispatched to globally avoid SSE3 code? |
Because building globally with -mavx accomplishes the same in regards to SSE3 code. Plus code dispatch for AVX2, AVX512, etc. still is honored where present in whatever components. The reality is we are talking a project where the bulk of the core code is actually scalar anyway. ;) Also, as I have noted elsewhere, AVX2 was my most annoying build to maintain due to breakages. I was not building it natively, which made it all that more "fun"* when something inevitably broke. I might eventually return AVX2 builds. In the meantime, if someone else wants to maintain an AVX2 or whatever build, then I will try to answer build questions as possible. *Hobbies are supposed to be interesting and fun IMO. When a hobby is no longer either, then it becomes time to reevaluate one's efforts. Admittedly, I have done Chromium builds for so many years now that I am pretty much burnt out on the project; and have been for quite a long time now. It is a big part of why my build intervals have declined in recent times. |
Woow, just tried to build current version, gclient runhooks now requires python 3.10, and depot_tools still contains 3.8 )) |
Maybe it's time to publish a complete howto with as much explainations as possible, so people like me and others will able to make their own "it works for me" builds. Maybe someone will then maintain them or will create own repo. |
@RobRich999 I'm sorry to hear that mate. Also jerry fixed carls by adding all codecs and changes marmadukes from allcodecs+ to allcodecs at my request, and he has accepted my thorium builds for the linux subsection!! He also accepted my chromium os builds but I need to find a place to put them as I dont think github will allow the 6.8gb apiece disk image files. |
Also @RobRich999 Whats your build system i.e cpu, ram, gpu. os, os version. |
@Paukan777 I would be willing to publish an in depth how-to to my github if you want. |
I am proud to announce the first actual release of Thorium, which will be put on chromium.woolyss.com right underneath yours @RobRich999 as soon as Jerry reads his email today. Also uploaded is linux-chromeos, where it runs the whole system UI in an X11 window. Jerry said he might add that to the chromiumos subsection. Also uploaded is full ChromiumOS x86_64 builds with x264, module, and full linux firmware support. If someone here wants a tutorial on API keys to allow sign in on ChromiumOS OR to enable sync on chromium (or any other chromium based browser for that matter), I can provide them sparingly upon request. |
@Alex313031 Congrats! :) Windows 10 build box Kubuntu build box I just obtained a new Ryzen 5700u 8c/16t notebook. I might get around to setting up a dev environment on it. |
@Alex313031 It does not hurt to set them, but several of your builds args already match their default settings, at least with is_official_build = true enabled. ;) A few examples: rtc_use_h264 = true I do use is_official_build = true for building. Primarily it turns off component and debug building. It also turns off ThinLTO optimizations (for now) due to binary bloat, but we turn the optimizations back on with the other ThinLTO build args anyway, so no concern there. |
We can`t turn off new design in 124 avx2 right? |
Hi All, Building Chromium with is_offical=true, getting rendering process crash error on a specific web application apache superset(tab showing error 5) dump file showing: View layout should not be re-entrant any build arguments impacting rendering or blink which could help to resolve this issue? |
@necros2k7 Tends to come up with each new significant UI change. You might dig through the about:flags like the "Refresh 2023" options, but even if whatever flags roll back UI changes, odds are those flags will be eventually deprecated anyway. @zivloaiz That appears to be a debug check. Setting is_offical=true also implies: In particular, DCHECK()s are still enabled for release builds, which can halve overall performance, and do increase memory usage. Always set "is_official_build" to true for any build intended to ship to end-users. While it is not the recommended approach, you can try setting dcheck_always_on=false as a build argument. If that resolves it consider filing a bug report with the project(s) affected. |
@RobRich999 Can we have updated Windows 64 AVX-512 builds? The current builds have an annoying default browser prompt on loading, can't be disabled regardless of what I do. Seems to be fixed in v124.0.6325.0 or later. |
@boyedarat I am working on updated builds today and perhaps into tomorrow. In the meantime or if it happens again in the future, you can add the following command-line flag if desired: --no-default-browser-check |
Usual builds updated. Hoping to get Win64 AVX-512 builds rolled on Monday. FWIW, I updated to the recently updated Windows 11 v10.0.22621.3235 SDK as well. |
Skipped AVX-512 during the last round of builds. Attention was elsewhere. Anyway, I did roll them for my current round of builds. Here ya'll go: https://github.com/RobRich999/Chromium_Experimental/releases/tag/v125.0.6422.0-r1287427-win64-avx512 |
|
RawDraw has been deprecated and will be replaced in the future by SkiaGraphite+DrDc
These are modern Windows standard libraries
There's no benefit, only loss of HDR support, high refresh rate support, MPO support, hardware decoding support, VpSuperRes/TrueHDR support |
@Andarwinux so SkiaGraphite should be more performing? Strange but when I enable it - app screen image becomes like lowres or TV) and when DrDc enabled in settings chrome:gpu says it`s disabled( |
SkiaGraphite is still WIP, but should eventually lead to better performance as D3D12 will be used instead of D3D11. DrDc is currently only available for CrOS and Android, with other platform implementations dependent on the final SkiaGraphite. |
The somewhat older blog post is about WebGPU, but the hardware stats show for why D3D12 and Vulkan have not exactly been the project's top development priorities IMHO. "....on platforms like Windows (31% of Chrome users lack D3D11.1+), Android (23% lack Vulkan 1.1+), and ChromeOS (Vulkan adoption on the rise)" Source: https://developer.chrome.com/blog/new-in-webgpu-122 Admittedly I am mostly a Linux user, as in I am more or less "happy" when far more comparatively mundane graphics features like Wayland support work as intended. ;) |
Both SkiaGraphite and WebGPU are actually driven by Dawn. That's why SkiaGraphite started from D3D11. it doesn't matter if D3D12 is important to Google or not, because it's Intel and Microsoft doing that. As for Vulkan, I don't care, the Vulkan ecosystem on Android and Windows is shit. |
Please be patient, RobRich doesn't have a super build farm cluster like goma so builds take time. |
Sorry but what exactly dxil, dxcompiler libs are used for, i mean app starts without them w/o problem |
The dxcompiler is used to compile HLSL shaders into DXIL, the IR used by Direct3D UMD, which is further compiled by UMD into GPU ISA-specific final machine code. dxcompiler is one of the Windows standard libraries, Chromium bundles it just for compatibility with legacy systems. |
I am getting ready to kick off updated Windows builds in a few minutes. I have a Xeon 96c/192t 256GB build box these days so thankfully build times are not too excessive. It is often more about how I feel on any particular day. ;) |
Hi, @RobRich999 Thanks for the work. Can you write a detailed step-by-step instruction on building chromium with all these flags enabled, or a video as well? It will help a lot of people. |
@boyedarat Probably later today. I did not get to them during my previous round of builds. Think I will drop the Zen4 build as previous feedback show no real performance difference compared to the baseline Win64 AVX-512 build. I might promote Win64 AVX-512 out of "experimental" status as well. @ShreyasJejurkar Think I have been planning to write instructions for like 8+ years now. o.0 If truly interested, get a standard Chromium browser build working. I can help fill in the rest. :) |
Quick FYI if encountering "YUV_420_BIPLANAR" GPU log errors (and possible browser instability) for Chromium on Linux with Wayland support enabled. https://issues.chromium.org/issues/331796411 As I noted in the bug report comments, disabling multi-pane video support should be a temporary workaround until the underlying issue is resolved. chrome://flags/#use-multi-plane-format-for-software-video |
@RobRich999 FYI for Linux/Wayland/VAAPI: Alex313031/thorium#687 |
@ms178 Sure enough. Caught the report at crbug.com earlier today. The change should be integrated in my next round of builds. :) |
Rolled a Linux Zen2 build to verify. VA-API decoding is working as confirmed via about:media-internals diagnostics. AMDGPU on Mesa 24.1 graphics stack. I did not test VA-API encoding, but it shows enabled in the about:gpu stats. --ozone-platform=wayland --use-angle=vulkan --enable-features=Vulkan,VulkanFromANGLE,DefaultANGLEVulkan,VaapiIgnoreDriverChecks,VaapiVideoDecoder,VaapiVideoEncoder,UseMultiPlaneFormatForHardwareVideo Disable multi-plane video if about:gpu is logging YUV_420_BIPLANAR errors. --ozone-platform=wayland --use-angle=vulkan --enable-features=Vulkan,VulkanFromANGLE,DefaultANGLEVulkan,VaapiIgnoreDriverChecks,VaapiVideoDecoder,VaapiVideoEncoder --disable-features=UseMultiPlaneFormatForHardwareVideo,UseMultiPlaneFormatForSoftwareVideo I will try to get to the other Linux builds in a day or few. Of note RawDraw is reportedly working here as well. Ya' can add RawDraw to the enable-features list to test. YMMV. Win64 AVX-512 is churning on the build box right now. I restored my LLVM AVX2 build script patch repo if interested. Doubt I will be dealing with binary releases, though. https://github.com/RobRich999/LLVM_Optimized_AVX2 The Linux Zen2 build above has added optimizations including Poilly enabled. Another YMMV. Looks like my Linux Zen2 build might have picked up an intermittent crasher when using using Wayland. I had RawDraw enabled. Just disabled it. I will continue evaluating. |
Updated Win64 AVX-512 build available. YMMV as usual, since I can not test AVX-512 builds at this time. https://github.com/RobRich999/Chromium_Clang/releases/tag/v127.0.6504.0-r1306219-win64-avx512 |
@Andarwinux Thanks for the feedback. I upped optimization levels for that particular build. FWIW, Chromium VA-API support is working for my latest Zen2 build on Xwayland as well. --use-angle=vulkan --enable-features=Vulkan,VulkanFromANGLE,DefaultANGLEVulkan,VaapiIgnoreDriverChecks,VaapiVideoDecoder,VaapiVideoEncoder,UseMultiPlaneFormatForHardwareVideo,UseMultiPlaneFormatForSoftwareVideo I had to drop Chromium Ozone back from native Wayland due to an intermittent crasher. There is similar a bug report already filed. |
Discussion regarding Chromium builds and related topics.
The text was updated successfully, but these errors were encountered: