Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pkg precompile error with current master #53196

Open
tecosaur opened this issue Feb 6, 2024 · 9 comments · May be fixed by JuliaLang/Pkg.jl#3782
Open

Pkg precompile error with current master #53196

tecosaur opened this issue Feb 6, 2024 · 9 comments · May be fixed by JuliaLang/Pkg.jl#3782
Labels
backport 1.10 Change should be backported to the 1.10 release kind:bug Indicates an unexpected problem or unintended behavior stdlib Julia's standard library

Comments

@tecosaur
Copy link
Contributor

tecosaur commented Feb 6, 2024

I've recently been seeing the following error with the current master (at this moment, I'm at 28db6c7) and I'm not entirely sure if it's Pkg or Base that's to blame (though I'd guess Pkg from the stacktrace):

# I type `]` to enter the pkg> repl
julia> ┌ Info: Precompiling Pkg [44cfe95a-1eb2-52ea-b672-e2afdf69b78f] (cache misses: wrong dep version loaded (1), invalid header (9), mismatched flags (1))
└ @ Base loading.jl:2576
ERROR: LoadError: MethodError: Cannot `convert` an object of type TOML.Internals.Parser to an object of type Base.TOML.Parser
The function `convert` exists, but no method is defined for this combination of argument types.

Closest candidates are:
  convert(::Type{T}, ::T) where T
   @ Base Base.jl:126
  Base.TOML.Parser(::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any)
   @ Base toml_parser.jl:42

Stacktrace:
  [1] Base.TOMLCache(p::TOML.Internals.Parser, d::Dict{String, Dict{String, Any}})
    @ Base ./loading.jl:268
  [2] top-level scope
    @ ~/Documents/Projects/julia/usr/share/julia/stdlib/v1.11/Pkg/src/Registry/registry_instance.jl:18
  [3] include(mod::Module, _path::String)
    @ Base ./Base.jl:556
  [4] include(x::String)
    @ Pkg.Registry ~/Documents/Projects/julia/usr/share/julia/stdlib/v1.11/Pkg/src/Registry/Registry.jl:1
  [5] top-level scope
    @ ~/Documents/Projects/julia/usr/share/julia/stdlib/v1.11/Pkg/src/Registry/Registry.jl:10
  [6] include(mod::Module, _path::String)
    @ Base ./Base.jl:556
  [7] include(x::String)
    @ Pkg ~/Documents/Projects/julia/usr/share/julia/stdlib/v1.11/Pkg/src/Pkg.jl:3
  [8] top-level scope
    @ ~/Documents/Projects/julia/usr/share/julia/stdlib/v1.11/Pkg/src/Pkg.jl:62
  [9] include
    @ ./Base.jl:556 [inlined]
 [10] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:2427
 [11] top-level scope
    @ stdin:3
in expression starting at /home/tec/Documents/Projects/julia/usr/share/julia/stdlib/v1.11/Pkg/src/Registry/registry_instance.jl:18
in expression starting at /home/tec/Documents/Projects/julia/usr/share/julia/stdlib/v1.11/Pkg/src/Registry/Registry.jl:1
in expression starting at /home/tec/Documents/Projects/julia/usr/share/julia/stdlib/v1.11/Pkg/src/Pkg.jl:3
@tecosaur tecosaur added the kind:bug Indicates an unexpected problem or unintended behavior label Feb 6, 2024
@vtjnash
Copy link
Sponsor Member

vtjnash commented Feb 6, 2024

It sounds like this is a Pkg bug. The Pkg code wants to pass in a Base.TOML object but does using TOML instead, assuming that will give back the same exact module. If you haven't run any Pkg commands, that is likely to give back the same object. But if you have used Pkg, then using TOML might be something different than Base.TOML.

@vtjnash vtjnash added the stdlib Julia's standard library label Feb 6, 2024
@KristofferC
Copy link
Sponsor Member

It sounds like this is a Pkg bug. The Pkg code wants to pass in a Base.TOML object but does using TOML instead, assuming that will give back the same exact module. If you haven't run any Pkg commands, that is likely to give back the same object. But if you have used Pkg, then using TOML might be something different than Base.TOML.

I don't understand that description. The error here is because TOML.Internals.Parser and Base.TOML.Parser are different objects but TOML.Internals.Parser is identically to the one from Base:

julia/stdlib/TOML/src/TOML.jl

Lines 10 to 12 in 94fd312

module Internals
# The parser is defined in Base
using Base.TOML: Parser, parse, tryparse, ParserError, isvalid_barekey_char, reinit!

It feels like somehow the external https://github.com/JuliaLang/TOML.jl has been loaded.

@vtjnash
Copy link
Sponsor Member

vtjnash commented Feb 6, 2024

Sure, but TOML.Internals.Parser isn't expected to have any relationship to Base.TOML.Parser, except by implementation accident. If you use Pkg to load something that depends on TOML, then it is expected that Pkg may choose to upgrade TOML to something newer from that external link and sever that relationship.

@tecosaur
Copy link
Contributor Author

tecosaur commented Feb 7, 2024

A bit more context, I don't see this when I run julia +dev --startup-file=no with my home project, but do with julia +dev --startup-file=no --project=~/.julia/dev/StyledStrings for some reason 😕.

@KristofferC
Copy link
Sponsor Member

Probably using the non-stdlib TOML in that environment. But should be easy to fix on the Pkg side.

@tecosaur
Copy link
Contributor Author

tecosaur commented Feb 7, 2024

What I find quite odd is that this happens consistently with the StyledStrings project, but it has zero dependencies (other than Test for the test environment). Meanwhile, I'm able to run julia +dev --startup-file --project=... with a project that has StyledStrings in its dependencies, and I don't see this issue.

This looks rather funky.

@KristofferC
Copy link
Sponsor Member

What does using TOML; pkgdir(TOML) say when you have that project?

@tecosaur
Copy link
Contributor Author

tecosaur commented Feb 7, 2024

Oh this is interesting, with --project=[..]StyledStrings I see

julia> pkgdir(TOML)
"/home/tec/.julia/packages/TOML/q14aM"

but with the other project I see

julia> pkgdir(TOML)
"/home/tec/Documents/Projects/julia/usr/share/julia/stdlib/v1.11/TOML"

Looking at the Manifest.toml it seems like something I'd done had somehow messed with it, and it had a few unrelated packages in it: including a bad TOML entry for whatever reason 🤷.

Should this bug be closed as a "something very funky happened, but it's a one-off/PEBKAC", or is there something more to it?

@KristofferC
Copy link
Sponsor Member

As Jameson said, this is technically a bug in Pkg in that if you e.g. dev the external TOML.jl you will encounter this.

@KristofferC KristofferC linked a pull request Feb 7, 2024 that will close this issue
Open
@vtjnash vtjnash modified the milestones: 1.10, 1.11 Mar 5, 2024
@vtjnash vtjnash added the backport 1.10 Change should be backported to the 1.10 release label Mar 5, 2024
@KristofferC KristofferC removed this from the 1.11 milestone Mar 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
backport 1.10 Change should be backported to the 1.10 release kind:bug Indicates an unexpected problem or unintended behavior stdlib Julia's standard library
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

use the correct TOML.Parser coupled to Base.TOMLCache JuliaLang/Pkg.jl
3 participants
@vtjnash @KristofferC @tecosaur