How to make adjoint simulation?

[tylerflex]

I need help making an adjoint simulation for my problem

[jan-david-fischbach]

I would also be interested in this question. Going even further I would be interested in building a drop in replacement for e.g. Ceviche or similar.

[tylerflex]

Yea I actually started working on multi-frequency but abandoned it because it gets a bit tricky. Here is the problem in a nutshell: each contribution to the objective function requires a separate source in the adjoint simulation. For multi-frequency problems, we'll therefore need several sources to be added, each at different frequencies. Furthermore, we need to choose the bandwidth of these sources so their spectrums don't overlap, otherwise there will be error in the gradient. It becomes difficult to set these bandwidths in an automatic way that is reliable. For example, if a user needs outputs at two frequencies very close together, we'd need extremely small bandwidths for each of the sources. As a result, the pulses will start to look almost CW and the simulation will need to run very long. In some sense, doing each frequency as a separate simulation with a shorter pulse / larger bandwidth is not only easier (because they are independent, so no cross talk error) but also we can run them for much shorter. So for now we decided to leave it this way. I'd check out the new tutorial on multi-frequency adjoint if you want to take advantage of running each of these frequencies in parallel using our async run function, which has adjoint / jax support. Let me know if you have any other questions!

[jan-david-fischbach]

Hey, thanks for the detailed response. Do I understand it correctly, that for simulations in which the length of the pulse dominatesthe simulation duration there is little to be gained from "joining" the simulations? Assuming I had a very large simulation volume (that is what tidy3d is good at, right?) the light propagating in the simulation region can take much longer, than the pulse duration, correct? In these cases it would save some time to run multiple frequencies at once, or am I mistaken?
I think i have to brush up on my FDTD-Adjoint knowledge a bit,,, The following paper doesn't seem to have crosstalk problems by carefully designing a single pulse for all frequencies: https://opg.optica.org/oe/fulltext.cfm?uri=oe-30-3-4467&id=468836 @smartalecH. Could the same approach not be applied here? I am probably completely wrong :/

[tylerflex]

I think you have the right idea. And the point about the large simulation volume is an interesting one, I'd need to think about that a bit more.

We can probably design a pulse to avoid crosstalk problems like you said, but at this iteration of Tidy3D is it not currently possible. Instead we'd have to do it through a series of narrowband gaussian pulses. It's a feature we've been thinking about introducing in the future, but at the moment it's not on our immediate roadmap.

In general, the first iteration of the adjoint feature is designed for frequency-by-frequency calculation due to the simplicity and wanting to have at least something working. We added parallel simulation capability to enable a bit more convenience when doing multi-frequency adjoint but it's not the ideal solution. That being said, we're constantly improving and adding features to it, so I anticipate that true multi-frequency support in a single simulation is something we can handle when we get the bandwidth.

@smartalecH or @momchil-flex might have some input on the feasibility or practicality of doing multi-frequency with separate vs. a combined time-domain simulation.

[momchil-flex]

The following paper doesn't seem to have crosstalk problems by carefully designing a single pulse for all frequencies: https://opg.optica.org/oe/fulltext.cfm?uri=oe-30-3-4467&id=468836

This is indeed the correct approach in principle. As Tyler said the reason we currently do it with separate simulations is that we wanted to have something running, but in the future either adding a set of narrow-band sources or directly going for this approach is definitely something we should do. However, even the approach from the paper cannot be free of cross-talk without increasing the time span of the signal, in some cases significantly, and with keeping high accuracy of the injected amplitude at the sampling points. This is just Fourier stuff. You're trying to sample a number of points in frequency space. Their frequency separation, as well as how strongly the desired signal depends on frequency - and it can have quite a strong dependence, and your target accuracy, set a maximum resolution Delta f that in itself imposes a minimum length of the time signal. In some problems this could be significant, but I'd be curious if Alec has a comment on that.

Like I said though, this is the right approach. It's just that it should be done carefully if you want to do this automatically for a broad range of use cases. And we haven't had the time for it yet. :)

[smartalecH]

Great discussion! As expected, meep's hybrid-domain adjoint implementation has the same limitations described by Momchil and Tyler (it's rather hard to beat the Fourier uncertainty principle 😀 ).

That being said, there's still good news! Here's a few reasons why I thought it was worth the effort to implement:

Most of the photonics devices we were designing were intended to exhibit a smooth broadband response (grating couplers, mode sorters, etc). So I was happy with sampling a course subset of the band (about every 5 nm across C+L or O) and optimizing those points directly. In other words, it was rather easy (for my application) to find a balance between sufficient sampling and moderate adjoint runtimes.

Of course, there are some applications that require much finer sampling. But I would argue that those devices (like resonators) probably shouldn't be designed with a hybrid-domain FDTD approach. Rather, it might make more sense to use a Pade approximant (which we have started working on within meep), or even treat the problem as an eigenvalue optimization problem (e.g. with filter diagonalization).

But suppose you are designing "smooth" (or non-resonant) devices. How can we be sure that there's no "weirdness" going on with the points between our samples? Every paper we wrote had at least one reviewer asking this question (understandably). To first order, the physics of broadband optimization tend to mitigate any resonant-like behavior (unless you are trying to make a resonator). So you can lean on this pleasant fact for a bit (and of course, you should run a final simulation with a really dense grid of wavelength sample points to check your work). But if you wanted to get creative, you could "shuffle" the frequency/wavelength points you are optimizing each iteration. This actually fits really well within the paradigm of "continuous minimax optimization." Your optimization "landscape" changes each iteration as a result of this shuffling, but we're used to that 😀.

Again, for me (as a lowly grad student at the time... now I'm just lowly), I thought it was worth it to go through all this work. But for Tidy3D, this may not be the case! I had very different incentives. SWE time is significantly more expensive than compute time (even for FDTD simulations!) so if Tidy3D can just blast through a bunch of independent solves at whatever wavelengths you want, that's probably the way to go (at least for the near future). Getting all the broadband-adjoint machinery to finally work was really tedious (and frankly, impossible without Steven). Getting the proper discretized operator in place was really painful (but made for a fun chapter in my dissertation...shameless plug...)

Finally, I'll add that there are many ways to skin this cat. I'm obviously a fan of our approach. But Andy Michaels' approach using multiple CW sources (and then solving a system of equations) is rather elegant too. As I alluded to earlier with the Pade approximant (which popped out of a discussion I had with Owen Miller a few years ago), there are still several clever tricks with basic time-domain adjoints that could be implemented.

I'll step off my soapbox now. Hope this helps!

[momchil-flex]

Thanks Alec, very interesting additions to this discussion!

Getting all the broadband-adjoint machinery to finally work was really tedious (and frankly, impossible without Steven). Getting the proper discretized operator in place was really painful (but made for a fun chapter in my dissertation...shameless plug...)

Which chapter are you referring to? The discussion in chapter 3 seems very similar to the paper. Or you just mean getting it to work was tedious?

SWE time is significantly more expensive than compute time (even for FDTD simulations!) so if Tidy3D can just blast through a bunch of independent solves at whatever wavelengths you want, that's probably the way to go (at least for the near future).

Yeah good to hear from you that it was tedious before we make the decision to start working on it. :) Tidy3D can indeed blast through a bunch of independent solves, but the other thing to consider though is that the costs are not born by the same entity. SWE cost is born by Flexcompute, while the cost of extra Tidy3D sims is born by the user. On the other hand, we do not apply any premium for using our adjoint plugin. So it feels like if we do put in the SWE effort to improve the multi-frequency scheme, then we should bill a premium for using it. Or we can just... not do either. At least for now.

[smartalecH]

Which chapter are you referring to? The discussion in chapter 3 seems very similar to the paper. Or you just mean getting it to work was tedious?

So for our paper, we settled a bit on our implementation. We decided to say that our work was a friendly "mix" of the two ways one typically performs adjoint optimizations: discretize-then-differentiate, vs differentiate-then-discretize. The reason we went this route was because we had a few "fudge" factors we needed in order to get things right and we weren't confident we knew exactly where they came from (because we hadn't gone through the math in differentiating the full timestepping operator).

Section 3.2 actually defines and differentiates the full operator and identifies where this fudge factor comes from (of course... there could be errors in my math...).

If you are only doing single frequency optimization, then the fudge factor isn't really an issue because you can just normalize it out etc. But with broadband optimization, you really need to get the fudge factor down.

In hindsight, all of this "accuracy" may not have been totally necessary. The whole idea of accuracy in the context of adjoint gradients is relative anyway (accurate compared to what? finite-differences? does that really matter?).

Since I went through the math, I've been more interested in "creatively mixing" discretize-then-differentiate and differentiate-then-discretize approaches. This is what Ian and I were doing with EMEpy (before we got too busy).

So what I'm trying to say, is that you guys will probably find an easier/cleaner way to implement this 😀

SWE cost is born by Flexcompute, while the cost of extra Tidy3D sims is born by the user.

Inevitably, it always trickles back down to the user, right? Either because you sacrificed resources to work on this rather than other important features, or because, like you proposed, you placed an R&D premium on the feature to fund its cost (which sounds very reasonable to me). But I'm no economist 😅

[jan-david-fischbach]

@smartalecH About the weirdness that might or might not be going on in between the frequency points: For many devices it is desirable to have high (very close to maximum) transmission (Grating couplers, waveguide bends, y-splitter just to name a few). AFAIK in the Fourier resolution limit the S-parameter (or whatever quantity we are interested in) at one frequency point of the DTFT does not solely result from that frequency but from a (weighted) "neighborhood". As it is physically impossible for the transmission to be greater than that maximum we want to be close to (e.g. 1) a strong negative outlier in between frequency points cannot be compensated by diversion in the other direction. The question I am trying to formulate is: We indirectly consider all the frequencies in between the frequency sampling points, don't we? And going further: could one optimize a broadband response by a single frequency (FDTD) Adjoint, by lowering the frequency resolution sufficiently?

I am currently reading your thesis. Haven't gotten to the discretization part yet. I'll reconsider what I have written here, when I do. I'll try to PM you about some other questions I had coming up during the read.

Regards JD

[smartalecH]

Hmm I'm not sure I agree (as much as I'd like to). As a simple counterexample, I've indirectly designed devices with resonant features in between my wavelength samples.

I think I understand what you are trying to say about having enough neighbors + a nearby upper bound limiting the freedom of the interior point. But the DTFT (or its resolution limits) doesn't really apply here. Look at any resonant device (including photonic crystals).

But I would like to be wrong 😀