Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problem with Zygote.hessian #228

Open
vpuri3 opened this issue Oct 25, 2023 · 1 comment · May be fixed by #229
Open

Problem with Zygote.hessian #228

vpuri3 opened this issue Oct 25, 2023 · 1 comment · May be fixed by #229

Comments

@vpuri3
Copy link
Contributor

vpuri3 commented Oct 25, 2023 •
edited 

julia> using ComponentArrays, Zygote

julia> p = (; a = ones(2), b = zeros(2)) |> ComponentArray
ComponentVector{Float64}(a = [1.0, 1.0], b = [0.0, 0.0])

julia> Zygote.gradient(x -> sum(x.a), p)
((a = [1.0, 1.0], b = [0.0, 0.0]),)
julia> Zygote.hessian(x -> sum(x.a), p)
ERROR: type Array has no field a
Stacktrace:
  [1] adjoint
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/lib.jl:229 [inlined]
  [2] _pullback
    @ Zygote ~/.julia/packages/ZygoteRules/OgCVT/src/adjoint.jl:66 [inlined]
  [3] #19
    @ Zygote ./REPL[12]:1 [inlined]
  [4] _pullback(ctx::Zygote.Context{false}, f::var"#19#20", args::Vector{ForwardDiff.Dual{Nothing, Float64, 4}})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/compiler/interface2.jl:0
  [5] pullback(f::Function, cx::Zygote.Context{false}, args::Vector{ForwardDiff.Dual{Nothing, Float64, 4}})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/compiler/interface.jl:44
  [6] pullback(f::Any, cx::ZygoteRules.AContext, args::Vararg{Any})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/compiler/interface.jl:42 [inlined]
  [7] gradient(f::Function, args::Vector{ForwardDiff.Dual{Nothing, Float64, 4}})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/compiler/interface.jl:96
  [8] (::Zygote.var"#125#126"{var"#19#20"})(x::Vector{ForwardDiff.Dual{Nothing, Float64, 4}})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/grad.jl:64
  [9] forward_jacobian(f::Zygote.var"#125#126"{var"#19#20"}, x::ComponentVector{Float64, Vector{Float64}, Tuple{Axis{(a = 1:2, b = 3:4)}}}, ::Val{4})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/forward.jl:29
 [10] forward_jacobian(f::Function, x::ComponentVector{Float64, Vector{Float64}, Tuple{Axis{(a = 1:2, b = 3:4)}}}; chunk_threshold::Int64)
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/forward.jl:44
 [11] forward_jacobian
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/forward.jl:42 [inlined]
 [12] hessian_dual
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/grad.jl:64 [inlined]
 [13] hessian(f::Function, x::ComponentVector{Float64, Vector{Float64}, Tuple{Axis{(a = 1:2, b = 3:4)}}})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/grad.jl:62
 [14] top-level scope
    @ REPL[12]:1
 [15] top-level scope
    @ ~/.julia/packages/CUDA/nbRJk/src/initialization.jl:205
julia> Zygote.hessian_reverse(x -> sum(x.a), p)                               
ERROR: Mutating arrays is not supported -- called setindex!(Vector{Float64}, ...)
This error occurs when you ask Zygote to differentiate operations that change
the elements of arrays in place (e.g. setting values with x .= ...)               
                                                                                                                                                                     
Possible fixes:                                                                                                                                                      
- avoid mutating operations (preferred)                                                                                                                              
- or read the documentation and solutions for this error                                                                                                             
  https://fluxml.ai/Zygote.jl/latest/limitations                        
                                         
Stacktrace:                                                                                                                                                          
  [1] error(s::String)                                                                                                                                               
    @ Base ./error.jl:35                                                          
  [2] _throw_mutation_error(f::Function, args::Vector{Float64})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/array.jl:88                   
  [3] (::Zygote.var"#555#556"{Vector{Float64}})(::Nothing)                 
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/array.jl:100        
  [4] (::Zygote.var"#2655#back#557"{Zygote.var"#555#556"{Vector{Float64}}})(Δ::Nothing)                                                                                  @ Zygote ~/.julia/packages/ZygoteRules/OgCVT/src/adjoint.jl:71                
  [5] (::Zygote.var"#291#292"{Tuple{Tuple{Nothing, Nothing}, Tuple{Nothing}}, Zygote.var"#2655#back#557"{Zygote.var"#555#556"{Vector{Float64}}}})(Δ::Nothing)        
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/lib.jl:206
  [6] (::Zygote.var"#2169#back#293"{Zygote.var"#291#292"{Tuple{Tuple{…}, Tuple{…}}, Zygote.var"#2655#back#557"{Zygote.var"#555#556"{…}}}})(Δ::Nothing)
    @ Zygote ~/.julia/packages/ZygoteRules/OgCVT/src/adjoint.jl:71                                                                                                   
  [7] macro expansion                                                             
    @ Zygote ~/.julia/dev/ComponentArrays/src/array_interface.jl:0 [inlined]                                                                                         
  [8] _setindex!                                                                  
    @ Zygote ~/.julia/dev/ComponentArrays/src/array_interface.jl:131 [inlined]
  [9] (::Zygote.Pullback{Tuple{…}, Tuple{…}})(Δ::Nothing)                         
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/compiler/interface2.jl:0                                                                                             
 [10] setproperty!                                                                
    @ ComponentArrays ~/.julia/dev/ComponentArrays/src/namedtuple_interface.jl:17 [inlined]                                                                          
 [11] (::ComponentArrays.var"#getproperty_adjoint#87"{ComponentVector{Float64, Vector{…}, Tuple{…}}, Symbol})(Δ::FillArrays.Fill{Float64, 1, Tuple{Base.OneTo{…}}})
    @ ComponentArrays ~/.julia/dev/ComponentArrays/src/compat/chainrulescore.jl:4 [inlined]
 [12] (::Zygote.Pullback{Tuple{…}, Tuple{…}})(Δ::Tuple{Nothing, Vector{…}, Nothing})                                                
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/compiler/interface2.jl:0                                                                                             
 [13] ZBack                                                                       
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/compiler/chainrules.jl:211 [inlined]
 [14] (::Zygote.Pullback{Tuple{…}, Tuple{…}})(Δ::Tuple{Nothing, Vector{…}, Nothing})                                                                       
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/compiler/interface2.jl:0
 [15] #37                                                                         
    @ Zygote ./REPL[22]:1 [inlined]                                               
 [16] (::Zygote.Pullback{Tuple{…}, Tuple{…}})(Δ::Tuple{Nothing, Vector{…}})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/compiler/interface2.jl:0
 [17] #75
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/compiler/interface.jl:45 [inlined]
 [18] (::Zygote.Pullback{Tuple{Zygote.var"#75#76"{…}, Float64}, Tuple{Zygote.Pullback{…}, Zygote.var"#2180#back#303"{…}, Zygote.Pullback{…}}})(Δ::Tuple{Vector{Float6
4}})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/compiler/interface2.jl:0
julia> using CUDA

julia> CUDA.allowscalar(false)

julia> Zygote.hessian(x -> sum(x.a), p |> cu)
ERROR: Scalar indexing is disallowed.
Invocation of getindex resulted in scalar indexing of a GPU array.
This is typically caused by calling an iterating implementation of a method.
Such implementations *do not* execute on the GPU, but very slowly on the CPU,
and therefore are only permitted from the REPL for prototyping purposes.
If you did intend to index this array, annotate the caller with @allowscalar.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:35
  [2] assertscalar(op::String)
    @ GPUArraysCore ~/.julia/packages/GPUArraysCore/uOYfN/src/GPUArraysCore.jl:103
  [3] getindex
    @ Zygote ~/.julia/packages/GPUArrays/EZkix/src/host/indexing.jl:9 [inlined]
  [4] getindex (repeats 2 times)
    @ Zygote ~/.julia/dev/ComponentArrays/src/array_interface.jl:94 [inlined]
  [5] extract(xs::ComponentVector{ForwardDiff.Dual{Nothing, Float32, 4}, ComponentVector{ForwardDiff.Dual{…}, CuArray{…}, Tuple{…}}, Tuple{Axis{…}}})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/forward.jl:23
  [6] forward_jacobian(f::Zygote.var"#125#126"{var"#41#42"}, x::ComponentVector{Float32, CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}, Tuple{Axis{…}}}, ::Val{4})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/forward.jl:29
  [7] forward_jacobian(f::Function, x::ComponentVector{Float32, CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}, Tuple{Axis{(a = 1:2, b = 3:4)}}}; chunk_threshold::Int64)
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/forward.jl:44
  [8] forward_jacobian
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/forward.jl:42 [inlined]
  [9] hessian_dual
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/grad.jl:64 [inlined]
 [10] hessian(f::Function, x::ComponentVector{Float32, CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}, Tuple{Axis{(a = 1:2, b = 3:4)}}})
    @ Zygote ~/.julia/packages/Zygote/gsq4u/src/lib/grad.jl:62
 [11] top-level scope
    @ REPL[25]:1
 [12] top-level scope
    @ ~/.julia/packages/CUDA/nbRJk/src/initialization.jl:205
Some type information was truncated. Use `show(err)` to see complete types.
@vpuri3
Copy link
Contributor Author

vpuri3 commented Oct 25, 2023

However,

julia> f = x -> sum(x.a)                                                                                                                                             
#47 (generic function with 1 method)                                                                                                                                 

julia> ForwardDiff.jacobian(x -> Zygote.gradient(f, x)[1], p)
4×4 Matrix{Float64}:
 0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0

julia> ForwardDiff.jacobian(x -> Zygote.gradient(f, x)[1], p |> cu)
4×4 CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}:
 0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0

Looks like it's a problem with Zygote's forward_jacobian

@vpuri3 vpuri3 linked a pull request Oct 25, 2023 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

add method for zygote.seed(::CA) vpuri3/ComponentArrays.jl
1 participant
@vpuri3