Consider installing gnuplot along with Gaston using Julia's artifacts system

[xiaodaigh]

This is the error I get when I try using Gaston

ERROR: InitError: Gaston cannot be loaded: gnuplot is not available on this system.
Stacktrace:
 [1] error(::String) at .\error.jl:33
 [2] __init__() at C:\Users\RTX2080\.julia\packages\Gaston\JtE39\src\Gaston.jl:64
 [3] _include_from_serialized(::String, ::Array{Any,1}) at .\loading.jl:697
 [4] _require_from_serialized(::String) at .\loading.jl:748
 [5] _require(::Base.PkgId) at .\loading.jl:1039
 [6] require(::Base.PkgId) at .\loading.jl:927
 [7] require(::Module, ::Symbol) at .\loading.jl:922
 [8] top-level scope at util.jl:175
 [9] eval(::Module, ::Any) at .\boot.jl:331
 [10] eval_user_input(::Any, ::REPL.REPLBackend) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\REPL.jl:86
 [11] run_backend(::REPL.REPLBackend) at C:\Users\RTX2080\.julia\packages\Revise\XFtoQ\src\Revise.jl:1162
 [12] top-level scope at none:0
during initialization of module Gaston
caused by [exception 1]
IOError: could not spawn `gnuplot --version`: no such file or directory (ENOENT)
Stacktrace:
 [1] _spawn_primitive(::String, ::Cmd, ::Array{Any,1}) at .\process.jl:99
 [2] #550 at .\process.jl:112 [inlined]
 [3] setup_stdios(::Base.var"#550#551"{Cmd}, ::Array{Any,1}) at .\process.jl:196
 [4] _spawn at .\process.jl:111 [inlined]
 [5] _spawn at .\process.jl:106 [inlined]
 [6] success(::Cmd) at .\process.jl:496
 [7] __init__() at C:\Users\RTX2080\.julia\packages\Gaston\JtE39\src\Gaston.jl:62
 [8] _include_from_serialized(::String, ::Array{Any,1}) at .\loading.jl:697
 [9] _require_from_serialized(::String) at .\loading.jl:748
 [10] _require(::Base.PkgId) at .\loading.jl:1039
 [11] require(::Base.PkgId) at .\loading.jl:927
 [12] require(::Module, ::Symbol) at .\loading.jl:922
 [13] top-level scope at util.jl:175
 [14] eval(::Module, ::Any) at .\boot.jl:331
 [15] eval_user_input(::Any, ::REPL.REPLBackend) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\REPL.jl:86
 [16] run_backend(::REPL.REPLBackend) at C:\Users\RTX2080\.julia\packages\Revise\XFtoQ\src\Revise.jl:1162
 [17] top-level scope at none:0

[mbaz]

I don't consider this a bug in Gaston. The documentation clearly states that gnuplot is required. When gnuplot is properly installed, Gaston works fine in Windows.

I will leave this bug open (with edited title) as a reminder that I need to study the artifacts scaffolding and see if it makes sense for Gaston. It's very low priority for me, though, but I will accept a pull request implementing this.

[ghost]

get the same issued when precompiling gaston. but without poping up alarm stacktrace. after "using Gaston" repeated its works fine.

what is Operating System are you used?

[VarLad]

@mbaz or anyone who's willing to give a try

using BinaryBuilder
BinaryBuilder.run_wizard()

for url: https://excellmedia.dl.sourceforge.net/project/gnuplot/gnuplot/5.4.0/gnuplot-5.4.0.tar.gz
After step 2a and 2b, in the terminal

cd gnuplot-5.4.0
./configure --prefix=/workspace/destdir
make 
make install
exit

The binary was created, although there were some dependencies unfulfilled
But almost all of them have their binaries already created.
The dependencies I encountered for linux were
cairo
cairo-pdf
pango
pangocairo
glib-2.0
Qt5Core
Qt5Gui
Qt5Network
Qt5Svg
Qt5PrintSupport
I think that qml.jl already has the qt binary
https://github.com/JuliaPackaging/Yggdrasil/blob/2c223a4511c715985d0ae9e6bc4cd53debfa1bae/Q/Qt/build_tarballs.jl we can skip out the other qt packages... only 5 are needed as dependencies for gnuplot [qt5core qt5gui qt5network qt5svg and qt5printsupport]
Also, I think that the other binaries are available too, I'm sure I saw them being installed with some other packages.
If someone can find them, just use them in step 2b

If someone has enough data, and is willing to give it a go, I hope the above instructions come in handy

[mbaz]

Does this procedure build binaries for Windows and Mac too?

[VarLad]

Yes

Does this procedure build binaries for Windows and Mac too?

Yes
Just after my linux build was over, Windows build started

[VarLad]

Although I had to cancel that one as I've low on internet data here...

[mbaz]

Thanks for the pointers! I will give it a shot soon.

[mbaz]

I'm not going to build for Mac... I'm not willing to agree to any restrictions and legal terms.

[VarLad]

I wanted to implement Gaston in a few Pluto tutorials... although I can't distribute it without artifacts
Gaston is just so good for this, as its lightweight and heavily equipped yet REALLY fast!

[VarLad]

I'm not going to build for Mac... I'm not willing to agree to any restrictions and legal terms.

No problem...
I just need the linux and windows build

[VarLad]

I'm not going to build for Mac... I'm not willing to agree to any restrictions and legal terms.

@mbaz Actually why not just put something like

if(Sys,isapple)
 run(` /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)" `)
 run(` brew install gnuplot `)
else if(Sys.iswindows)
download("https://excellmedia.dl.sourceforge.net/project/gnuplot/gnuplot/5.2.8/gp528-win64-mingw.exe")
run(\`  ./gp528-win64-mingw.exe \`)
end

Since the installation process for Windows and Mac is not going to change
But for Linux/FreeBSD, the process is different, the binaries would be better.
https://docs.julialang.org/en/v1/base/file/#Base.download
https://docs.julialang.org/en/v1/base/base/#Base.run

[mbaz]

@clad26 That's an interesting thought! Questions:

• For apple, do you need to run those commands as root?
• Is 5.4.0 not available for Windows?
• Would it be acceptable to just print the instructions, and ask the user to execute them?

[VarLad]

5.4.0 for windows was in their testing repo, release candidate 1 is out, it seems
I'd wait though, for a stable release

It seems that to install brew, we need sudo
But for installing with brew we don't

Automatically installing for the user is a better option here. It'd be good if there existed a mechanism to check if user already had a version >= 5.2.8... I'm sure run( ｀gnuplot --version｀ ) can work here...
If the user doesn't have a pre-installed version:
You can print what you're doing and ask for a yes/no. If the user types no, and the user doesn't have gnuplot >= 5.2.8 either, give an error.

In short, installing the binaries/Artifacts, if the first run gives an error that gnuplot is not installed makes sense！If the user says he doesn't want to install, throw an error！

[VarLad]

@mbaz Thanks for taking interest！
I hope we get this feature in Gaston.jl soon！

[mbaz]

@clad26 Thanks for your help! I'll try to get something working soon.

[VarLad]

@clad26 Thanks for your help! I'll try to get something working soon.

Much appreciated！
https://github.com/simeonschaub/ArtifactUtils.jl
This might help

[VarLad]

Also, if you're following the Windows method, if the user has an outdated version, it might be better to remove the old version！
run(｀"C:\Program Files\gnuplot\unins000.exe" ｀)
should do the trick！

Also, isn't it possible to just use the Windows build AS THE ARTIFACT？
Its much better than building it yourself every time！
I tried this and it works perfectly.... https://pilotfiber.dl.sourceforge.net/project/gnuplot/gnuplot/5.2.8/gp528-win64-mingw.7z
Using the BinaryBuilder for Windows version might not be the best thing...... The above build works perfectly.... you might have to compress the contents of the .7z file into a .tar.gz file to use as an artifact though....

[VarLad]

Artifact using BinaryBuilder is only required for Linux/FreeBSD

For MAC/OSX, you might wanna look into https://github.com/JuliaPackaging/Homebrew.jl
It just outright simplifies the process of building the artifact for Mac/OSX

[VarLad]

This way we can have the artifacts for all of Mac/OSX, Windows and Linux/FreeBSD....

[VarLad]

From the bin folder of gnuplot windows build

@mbaz I think you can find the cairo, pango, qt libxpm zlib lua libpng libwebp libjpgeg libgif libtiff glib and others on https://github.com/JuliaPackaging/Yggdrasil/
Xpm here: https://github.com/JuliaPackaging/Yggdrasil/tree/9d1ae6ccba6f393bd7dbbc4fd49cdf29003c43b2/X/Xorg_libXpm
Just search the name in the repo and you'll probably find it
Although I think you'll have to build WXwidgets yourself... for wxt terminal
https://github.com/wxWidgets/wxWidgets/releases/download/v3.1.4/wxWidgets-3.1.4.tar.bz2

[VarLad]

@mbaz I hope the above information helps

[mbaz]

@clad26 This is very useful indeed, thanks a lot! It looks like a non-trivial amount of work, though. I think I'll start with the bare minimum dependencies (i.e. no WxWidgets): I'm thinking Qt, X11, GIF, PNG, SVG, Canvas, Cairo and maybe a couple of others. I can then add other dependencies, if needed and as time allows.

I am not comfortable with modifying a user's installation, though. My plan is to have Gaston use the Julia artifact and ignore any pre-installed programs.

For Windows, just including the official binary as an artifact looks like the best solution. Still, I'm sure I have seen 5.4.0 Windows binaries, because I know I tested it before the latest Gaston release. I'll take a look and see if I can find it.

For Mac, it looks like the homebrew.jl package is the way to go. I'll give this lower priority, though, and leave it for a future release.

[VarLad]

I am not comfortable with modifying a user's installation, though. My plan is to have Gaston use the Julia artifact and ignore any pre-installed programs.

I'm sure you've heard of CUDA.jl.
When the users install CUDA.jl, it downloads the latest version of the CUDA artifact and uses it.
But the users have an option if they want to skip the artifact download, and switch to the user's installation
Its achieved by making a folder "config" in .julia folder, and making a startup.jl file in it, and typing in startup.jl,
ENV["JULIA_CUDA_USE_BINARYBUILDER"]=false
Of course, if the user doesn't download the artifacts and doesn't have a version >=10.2 beforehand either, it gives an error.

How about a similar mechanism for Gaston too? Ofcourse, most Julia beginners wouldn't even think about it, while more advanced Julia users will have more freedom

[VarLad]

This is very useful indeed, thanks a lot! It looks like a non-trivial amount of work, though. I think I'll start with the bare minimum dependencies (i.e. no WxWidgets): I'm thinking Qt, X11, GIF, PNG, SVG, Canvas, Cairo and maybe a couple of others. I can then add other dependencies, if needed and as time allows.

I don't think my Fedora build provides me with wxt either, so that isn't a problem personally for me. qt and x11 are sufficient for anyone.

[mbaz]

How about a similar mechanism for Gaston too?

That'd be nice, but I don't want to try to do too much. I don't have a lot of time to devote to this right now. I prefer to get something out that works, and fine-tune over the next releases.

[VarLad]

Also, the size of the collective artifacts matters.... It'd be great if you can keep it somewhere around 20-60mb!

[VarLad]

@mbaz Any chance that, if you're making the binary for linux, then can you build gnuplot 5.5？
It has a new smooth parameter which works on surfaces too. Until this, there was no easy way to do this.
If not, can you give a way to the user to access their version of gnuplot somehow？Thanks

[VarLad]

@mbaz Any update on this? Were you able to get atleast the qt terminal compiled with BinaryBuilder?

[VarLad]

@mbaz Any plans to make the artifact system a milestone for the 1.1 release?🙂

[mbaz]

@clad26 I've had little time to work on this, but I hope to, yes.

[VarLad]

@mbaz I heard Plots was getting a Gaston backend, which I think makes the artifacts all the more important!
Any update on the artifacts?

[mbaz]

It is a lot of work, and I'm unlikely to do it in the short to medium term. Another reason I'm not enthusiastic about it is that it is trivially easy to install Gnuplot in all supported platforms. I will consider a pull request if someone does the work, though.

[VarLad]

@mbaz Whats your experience with wxt terminal?
I'm kinda getting close to the binary, but for now, without qt.
I'm building wxWidgets too, for wxt terminal though
I think that should be a good temporary alternative

[mbaz]

Wxt is a solid terminal in my experience.

[VarLad]

I tried the wxt terminal on Windows yesterday
And it was smoother than qt terminal for 5.2.8
I still don't know how it performs on Linux+GTK3 and Mac+Cocoa, but I'm expecting a similar performance...
I'm ditching the qt terminal for the wxt terminal for now...

[mbaz]

Question: let's say a user has their own installation of Gnuplot, supporting the Windows terminal, Qt, etc. Then they install Gaston with your artifact, which pulls your gnuplot binaries. Which version of Gnuplot will Gaston use in this case?

[VarLad]

It'll use the artifacts, I guess
Similar to CUDA.jl, I want the users to be able to choose, same as https://juliagpu.gitlab.io/CUDA.jl/installation/overview/

You might want to disallow use of artifacts, e.g., because an optimized CUDA installation is available for your system. You can do so by setting the environment variable JULIA_CUDA_USE_BINARYBUILDER to false when importing CUDA.jl.

To troubleshoot discovery of a local CUDA installation, you can set JULIA_DEBUG=CUDA and see the various paths where CUDA.jl looks. By setting any of the CUDA_HOME, CUDA_ROOT or CUDA_PATH environment variables, you can guide the package to a specific directory

I don't wanna install CUDA twice...Its really big... so in .julia/configure/startup.jl

I always do JULIA_CUDA_USE_BINARYBUILDER=false

I propose something similar for Gaston

[VarLad]

I'll properly look into how to do this, once the binaries are built
I initially aimed to do this before 1.6, but I guess it'll take more time than I anticipated...

[VarLad]

By the way, may I ask, are you a MAC user？

[mbaz]

I can go with that scheme, but ideally, it would be opt-in rather than opt-out. Gnuplot is so easy to install in all major platforms, that I think using the artifact will be the less-common scenario.

I don't use Macs at all; I use Linux.

[VarLad]

If your distribution provides it, its great...
But mostly outdated... I use Fedora Linux... and am stuck with 5.2.8-6...
5.4.1 has some cool features like 3D histograms, or so I hear
Even on Windows, they give you the official binary of 5.2.8, but you can get 5.4.1 from chocolatey...

I think the main problem here for me was kinda personal. I wrote a set of tutorials using Gaston for bar plots...
The issue here is that, I kinda gave the instructions to install on MOST platforms, in README.md！But most don't read README.md
I fear that they'll clone the Pluto notebooks, install the necessary packages, and then when the plots don't show up, they quit the browser and don't ever open it again…

[VarLad]

Also, if in case their own build is somewhat messed up, thats another reason

By the way, I've the book Gnuplot in Action, but I really never read it from top to bottom. I only open it, when I need to learn something very specific, and its not in Docs, Examples, Stackoverflow, etc, atleast not in a way I can understand.

[mbaz]

I agree that some users might benefit from having the binary artifacts, and that's why I'll add this feature if it has the necessary flexibility. Thanks for working on it!

[VarLad]

@mbaz Some problems here and there.... Its almost there... I'd say anytime this month
Qt is not available on musl. It has its svg terminal and png/gif terminals. This includes distributions like Alpine Linux. Note that the binary of gnuplot provided with Alpine don't have the qt or wx terminal either

Problems on Windows 32bit. The latest version of gnuplot 5.4.1 for only 64-bit Windows. Even Chocolatey does that.
For Windows 32bit, https://sourceforge.net/projects/gnuplot/files/gnuplot/5.2.7/ can be a temporary compromise

I'd like to hear your opinions on this!