Daniel Kennefick interview with Andrew Abrahams

0:00 I'll just speak into it and say that it's quarter to two on September the 4th, and I'm speaking with Andrew Abrams. So, I guess I might as well start by asking you how you got involved with the science of mathematics. Well, I guess when I was deciding where to go to graduate school, I sort of had these two competing interests to deal with. So, after winning various possibilities, Larry Smar was at the time sort of the famous person in numerical relativity. He was the only one who I had heard of. So, I decided to come here. So, this was at the time and still is a pretty active place in numerical relativity. I was in 85 when I came here and I worked with Larry and with Chuck Evans who is a post-doc here and did a lot of pioneering things. That's how I got involved with and I went to Cornell in 88, post-doc sort of position. During that time there started to be these meetings on black hole collisions, one of the major activities currently that was gearing up and so I got involved in this grand challenge. So that's how I got involved. That was about the time that the Grand Challenge was set up. Did I do anything? Well there was, let's see, my dates may be off by one year, but I think it was in 91 there was an original proposal that wasn't accepted by NSF. And then the next year they started a more serious job then and more computer scientists were brought in and the whole thing was made more politically correct from the CS standpoint. And when was it that you came back here? I came back here a little over a year ago. I was at Cornell for six years and then I went to Chapel Hill to work with Jim York there and do a check-in for a couple years before I decided to sort of go out to other things.

2:30 Well, when I came back here, I didn't come back really to do relativity. It was more to serve as a part-time. It was one of the attractions of coming back here. So, relativity is more of a hobby? Yeah, it's more of a hobby. But I'm still involved. I'm still officially a cognizant scientist and it's been a challenge for whatever that's worth. I'm still working with people. I just don't have much time to do sort of the hands-on stuff anymore, so. Well, I'm sort of going to take on stuff. I'm moving into another case. Relativity is more of a hobby for me this one, too. So, well, let me see. I guess I have... I suppose... I'm kind of curious as to how the grand challenge or anything works. I gather it's a fairly loose alliance between different groups. What is the purpose of the alliance made and intended to accomplish to kind of, I understand, kind of, well, that's what we could make of it. Well, I think originally it was conceived as a way of getting the people who were most active in the mathematical relativity together with computer scientists to work. Number one, pool of the knowledge that has been developed to sort of two-dimensional simulation for most of what has been done for math, and also to sort of raise the level of computer science that has been done in the relativity community to make it possible to do the 3D calculation. Unfortunately, I think that presupposed the level of cooperation which people who are traditionally in pedigrees are not really willing or able, perhaps, to accomplish without, to do without some sort of incentive which wasn't really there, I guess.

5:00 So it has been sort of problem to coordinate the disparate groups? Yeah, I think so. I mean, there's always been, the goal has been clear. I think everyone sort of accepted that as a goal. When it became clear that it was not going to be an easy win, I think a variety of possible approaches became more of a sticking point again. People wanted to explore their own favorite rather than work on a coordinated puzzle. I think that any large scientific collaboration has that same issue really. Yeah, I think that was crucial. I mean I think it's a little different than LIGO or with particle accelerators where there isn't really a lot of choice, right? I mean if you're a scientist who wants to gravitational wave astronomy, you can't say well I'll just go take my business elsewhere, we'll build our own in the backyard, I mean a few people obviously have taken that, but it's not quote unquote a serious decision to take, whereas you know... In numerical relativity, I think the competition was between, you know, putting your effort into this big alliance versus going back to business as usual and working on the things that, you know, sort of, of most scientific interest. So you have the centrifugal forces, but you don't really have a kind of centripetal force pattern? Yeah, maybe there isn't enough of, yeah, there wasn't enough of a, you know, this is going to be the center, this is the... I think the way, sort of, the only analogy is to have... Software that makes programming the parallel computer very easy and, you know, sort of a large central computer where everyone's doing their runs and that, to some extent, that software has been developed but it hasn't taken hold in the light of the night of. So it's not as if everyone is channeling their efforts there again. Also, this is theory, you know, even though it seems like experiment to a lot of people viewing it from the outside, I mean, in principle, you are doing theoretical stuff, and a theoretical problem where the correct paths are far from clear, and so everyone has their own opinion, and it's hard to identify people who don't have the same opinion.

7:30 Was there, at the beginning, was there kind of a general program that... There was some sort of agreement behind that kind of running into trouble, and then it was more of a case of, well, was there ever a... Well, I mean, I think early on, sort of, there was a rather ambitious goal, which was, you know, to have the catalog of waveforms calculated by the end of the five years. And there was certainly a lot of disagreement about the methodology, but I think people at that point were more accepting to sort of go along with... It was a common plan just because, frankly, people were more optimistic that something would come out of it, you know, when Sherman and Amso graduated and it just became more and more of our period. Was it that there were difficulties that were encountered more? Issues, conceptual issues in principle, or a certain lack of computing power? And there are also conceptual issues. Now, if one had enough computing power available easily, in other words, if you could run enough experiments, it might be a lot easier to get by some of these conceptual issues because you wouldn't have to have a theoretical path, essentially, to experiment on what works the best. You really can explore it from every space. And so in a sense the computers are there for the big production runs at the end when you have the code ready to go, but there doesn't seem to be yet, you can't for instance run on your workstation yet, things large enough scale to really involve a lot of vision. I think that's... They knew that when you're going for the gold ring, that no one's going to stand in the way of us getting as much computer time as the biggest machines in the world to get those big runs done. But I'm sure for your day-to-day exploratory work, those machines are unless available.

10:00 And also, if your algorithms are still under development, they haven't been optimized for parallel computers yet, so there's that tension as well. If you work on getting the computer science side of it completely solved, then you can do... It tends to be sort of separate communities of people who are good at those two things, and so... What were the main conceptual problems? Well, maybe I should just talk about what sort of the decisions that were made or the types of things that... It came about when I was at Chapel Hill in the middle of New York, and there was a long show people were on, and there was Arnold Anderson on a hyperbolic formulation of 3 plus 1 relativity, which it sort of came late in the day from a grand challenge standpoint, but some people, particularly at Cornell, got excited about it and worked on developing a code based on these, but instead of the standard 3 plus 1. The main advantage is that you can look at the system that's explicitly hyperbolic. You look at it and you know that these are wave equations. So you can use a lot of the fine difference methods that are appropriate for wave equations. Also certain things like using, excising the black holes, using causality arguments are obvious how to implement in this. They're obvious at least from a mathematical standpoint how to implement. And there's also advantages to how you pull out what the gravitational wave signals are, I believe. So anyway, there was this sort of competing code developed based on this formulation.

12:30 In November, basically December, we decided to stop effort on that quote, at least temporarily, just for the purpose of focusing on the direction of the seminar. And as always with decisions like this, there's some element of politics and some element of scientific reasoning. What proportion you assign to which part depends on who you are. So that's an example. So what I'm pointing out is that what we don't know is as fundamental as which equations are the right ones to solve. But part of the science, and relatively, of course, part of the science of what equations you need to have coordinates to go along with a coordinate system. So the gauge choice is something that also is still an open issue, essentially. In principle, it's a choice of coordinates that allow the black holes to sort of smoothly move through your spatial grid without pulling the coordinates along them, without wrapping up the coordinates, without causing huge gradients where there shouldn't be. You don't want horizons that are expanding out at the speed of light in those particular coordinates, or expanding through your coordinate space at the speed of light eating up your grid. All kinds of mixed numerical issues. Although there are pretty good candidates there for one-dimensional choices, so a single black hole, no one has really at least been able to experiment sufficiently with any three demons to be sure that they're good. So that's certainly... And, you know, making a good choice of those coordinates is probably also going to determine whether or not one has a well-posed mathematical system of equations to work with. So, you know, the fact is if you go back to the ADM equations, if you have a good enough coordinate system, you probably don't need one of these explicitly hyperbolic versions of the equations, which tends to be more complicated. Since these hyperbolic formulations have more freedom, they stay hyperbolic. This one we worked on stays hyperbolic no matter what sort of shift vector we choose, for instance.

15:00 What that sort of indicates is, well, maybe that's a better system for experimenting with different sorts of coordinates. We're not going to run into having an ill-posed initial value problem just by having bad coordinates. There are a lot of actually very needy and interesting theoretical issues which sort of have to be left behind in the more pragmatic world of, I mean, it's like my work at PyEx, you know, sometimes I'll encounter something very interesting and it'd be fun to spend two days on, but there's someone waiting for an answer that they need right now for something, you know, I can't help it. Anyway, so there's certainly a lot, you know. If we understood better, I think it would be a lot clearer what decisions to make. But since we don't, the decisions that we make were less ideal. Probably another major conflict point that comes to my mind is the area which I've been most directly responsible for, the outer boundary and the radiation extraction problem. Basic ideas, well, since you're really close to the source, you don't actually simulate out to infinity, you simulate just a small box around your source, you still need to get some s and o, d, and so on, waveforms, and you still need to put boundary conditions that are sensible. So the two competing approaches in this case have been what's called characteristic matching, which has been something that's mainly advanced by the Pittsburgh group. Although Richard Massner has also been involved in it. So that's Jeff Blumenkirchner, and Roberta does the film as some other people. So they've decided to treat this, in a sense, like a mathematical relativity problem, in that you should be able to match onto a characteristic solution outside on some mold, to extend that out to scribe. The approach that I've been working on, which actually extends back, goes back to my thesis work, involves a much more sort of blue-collar approach to things, which is where you do perturbative expansions around the outside and you match interior solutions, numerical solutions, onto the exterior. This has worked pretty well. I mean, it's used.

17:30 The basic technique has been used, for instance, by Ed Seidel and his group, all along all the waveforms that they've published, essentially, and most of the work I've done and most of the work my parents and I have done have used these techniques to get the waveforms out, and they work, you know, certainly up to the accuracy of the codes that are being used, they work extremely well, you know. And so... For the grand challenge, we developed a somewhat more elaborate version of this which matches onto Schwarzschild's perturbations using, in fact, equations from this new formulation. The reason was that I wanted to have something that was at the extrinsic curvature level so they could easily be matched onto the interior evolution rather than onto... Really, curvature perturbations, which are harder to match onto the 3 plus 1 variables. So, anyway, so we've implemented this also, and we have a common boundary model that evolves the exterior spacetime in lockstep with the interior resolution, except it's just solving one-dimensional equations. And then it feeds data back in at the edge of the mesh. So right now, it seems to be the most stable method for code. We can evolve longer with this than any other method can evolve in a DOA concept. But here again, it's conflict. I'm actually sort of on the other side of it than I am on the equations, interior equations, mathematical purity versus pragmatic realism or something. In this case, I'm on the blue-collar side of it. Sure, well, I guess that kind of bipolarity is something that's fairly familiar in relativity between, if you want to take a more rigorous approach.

20:00 Right. It's not surprising to see it. Well, for instance, previously, I studied the history of the radiation reaction problem, and there, matching between the near zone and the wave zone was a big problem, and there was a lot of questions about, you know, would we want to be pragmatic or rigorous about it, and what you would say. Yeah, actually, one of the, since this is a historical conversation, one of the people who did some more early work on radiation Extraction was sort of inspired a lot of what I did for my thesis was Jim Anderson, who is obviously very active in the radiation reaction stuff. So his work in particular is the Unmatched Acetone? Yeah, so he did some work with David Hobel, who someone actually might also try to talk to at some point. So they started that stuff and when I met them, after I met David I guess, I went out there and spent some time talking to them. Ended up taking a very different direction to solving the problem, but it was the, you know, the goals were exactly the same, you know, to reduce the amount of computation you have to do by, you know, being able to put your families in close and get related ones out and feed data back. Thanks for that, and I gather in general that that's one of the big problems, feeding them. It shows us what other conceptual and computational problems there are that you have to keep with the area of your lecture, whatever you're using, as long as possible. Right, so I mean we're using Cartesian grid, and so essentially you expand it, the amount of memory you need on your computer is going up like the cube of the number of grid points. It becomes very expensive. The fact is that we don't even really have a very good feeling for what is a sensible size of calculation to do.

22:30 People have in their heads numbers mainly from three-dimensional CFD calculations, hydro calculations, 500 cubed is a big grid. We could get 500 cubed and solve this problem, but really there's nothing to do with each other. It could be 5,000 and we would be able to solve it for 10 years. Well, you mentioned, for instance, how, say, the Tibsburg group has their preferred matching technique than Richard Massey, who developed that in the Jewish Bureau on the outside out, so I'm in favor of the methods that you've been working with. Is it, I suppose, is it sort of your experience that one finds kind of loose informal arrangements between the different groups involved over, you know... Yeah, I mean, it's inevitable because people move around. Postdocs are kind of the real way that things get done, of course, and also it's the exchange of information, and so I went from here to Cornell to UNC, and that was sort of a, well, in my mind, a linkage between those groups because of that. I worked with students on putting this stuff into their codes at Cornell, and then... I had already worked quite a bit with Chuck Evans. He was involved in the early modeling stuff, and we had worked on credible phenomena and other things together, so there was, you know, it was just a historical connection. Chuck Evans had a two-dimensional GR code, which I worked with extensively. We did a lot of compilations, including head-on neutron stars. Shakira and Sapolsky had their 2D axisymmetric codes. They're in the same gauge and there's a lot of similarities between those codes. And so, a lot of the early pioneering work was done with one of those codes or some offspring of those codes.

25:00 Then, after I left here, when Ed Seidel came here and started building the group here, I came back and... There's no mystery about why it's happening this way, I'd say, it's just the things they're familiar with using, you know, someone willing to do it, you know, they're always good friends with it. I mean, it's something that these methods are not, how to put it, mittens. They are not on mathematically the same footing as what William Kirk, for instance, wants to do, which is solve exactly the equations on the outside. The question is, what I think the crucial questions are, are A, whether you really want to go to SCRI to get your waveform out anyway, because SCRI isn't a matching construction, at least as far as our university is concerned. So I take at least that. Key terms may include, for example, quantum mechanics. Speakers include, for example, quantum I also feel strongly that there are approximations also in the other methods even when they don't seem to voice them as clearly enough. For instance, the initial data that you're applying outside some radius is sort of always going to be approximate to some extent for a particular type of background. For instance, two black hole background. You're never going to be able to put that initial data out to describe. So the question is whether it's better to use exact evolution of perturbably accurate data or whether you use perturbable evolution of perturbably accurate data.

27:30 I guess you can argue it either way. So these are the main things. I've also always been concerned about the way that data gets fed back into the interior code. There is a cultural divide between the people at Pittsburgh who are traditionally doing characteristic approaches to the people who have traditionally been doing 3 plus 1. They were not really aware of the variety of coordinate system gauges that are used in the 3 plus 1 evolution and the types of demands that you have for instance that you don't have always a strictly hyperbolic system on the interior you may have a mixed elliptic hyperbolic system so you have elliptic equations don't have boundary conditions at scribe they have boundary conditions at i0 so you need to do something different with them. Anyway, so there's just, there's some issues like this right, I just feel, communication wasn't as strong as it could have been to really make sure that people understood the pros and cons in the right context. The, so the, you were saying that the Alliance was funded originally in about 1992 or something? Yeah, I think that's, that must be right. So is it nearing the end of its five year? Right, so the first... Funding cycle of the five years is over, essentially. I think it's May of 2018. So is it planned to continue at all? Well, plans change all the time, but my understanding of the current situation is that there may be multiple proposals from subsets of it.

30:00 I mean, I know more than this, but I'm not sure how much of this is public knowledge, I don't know. Oh, okay. Well, yeah, I don't know either. I mean, Kipper's been very influential, I would say, in holding the Alliance together, actually, because he's come to a number of meetings and all the services addressed, you know, you guys have to work together kind of things, you know, have to make some tough choices, you know. It's important that you get a result. Yeah. I mean, I think that's important. I'm not completely correct, but maybe asking a little too much is what people are involved in. Well, my guess is what's going to happen is that it's going to be more of a loose alliance from now on. Maybe there'll be some, I expect there'll be some continued effort to cooperate, but the funding will be explicitly provided. The way it worked for the first five years is that the grant was to Texas, and Massner had no less. All of these terms are not completely controlled when one of those is divided up. So that probably causes friction. Yeah, and you get very worked up about it. Well, especially in view of the fact that there were differences of opinion over which way to go. So I know Kip is sort of, he's the chairman of the oversight committee. Right. I guess he's had lots of experience with LIGO diplomatically. Given your experience of the first five years being, is it, you indicated that maybe a period at least would be better to have people group together, or is it more practically better to allow people to go their own way? I think that for whatever reason, groups at different institutions don't work together very easily. There's too much implicit or explicit competition going on.

32:30 There are a lot of, they're the principal investigators at each institution, and then there's the postdocs and credit students who really do the work, typically, and the PIs. It seems to spend all their time worrying about funding issues, which is okay, but yeah, they have to worry about it because they're supporting a group, but on the other hand, the amount of money that they're worrying about, you know, it's almost, it's nowhere near proportional to the amount of concern that goes into it. And so, and I always felt this was a very destructive aspect here. There has been more involvement in the science, direct involvement in the science, by the people who are supposedly the leaders, I think, since we've worked out that way. So, kind of regretfully, I think I would have to say that my... If you really want to solve the problem, still your best bet is to focus money at an institute, at a single institution where you can have a bunch of people working together very closely. The physical proximity does seem to be important, because as you were saying, in some sense, there's a connection between the groups who would actually be physically carried as a group of people moving between them. Well, yeah, I was thinking that more as the way things worked before the alliance. I mean, there still has been a lot of cross-fertilization of postdocs. It's a little edgier now, because there's still a very limited pool of postdocs, and so... You're more or less competing for the same money to fund, and it's sometimes like the same person, so someone moves, and it's like, well, so who are you spending your money on now? It's like, well, you know, we didn't ask you to do it, we went back to that thing.

35:00 Yeah, so that all feeds back into the string of resources. I mean, I think the other aspect is that in the next version, there's not going to be, the amount of money is going to be less. I think it'll work, there's something like 900,000 a year in the old. The money that's being scraped together now, I don't even understand what all the sources are, but I know that Isakson is trying to funnel this on. It may not even happen, but if it does happen, it's still going to probably be less than half of what it was available for. So there's no way that things can be supported at the same level. So in view of that, what would you say is the possible experience? To continue with the problem, what would be the time scale? I suppose that, well, let's say the original kind of idea was to get all of these tempos put out by the full 3D problem. Is that still a goal or a realistic goal, or is it for the next five years? For things to happen fairly quickly. That's not impossible at all, but I guess judging from the sort of rate of progress I see now, five years seems difficult to achieve. But it's completely fantastic and I think actually it would be... One could argue that a better approach than having people focus on the problem is going to be to purposely make things more stochastic, but it's going to go back to the way things were before and have people work on more variety of problems and approaches to see if that leads to a breakthrough. And that's also, I don't know, it's like roulette or something. You put your whole stack on the one to get the thing where you have read it or not. Sure, I guess as you say, if you're looking for a graduate, then you should respond.

37:30 But, I mean, it's certainly far from an engineering problem at this stage. But my guess is what will happen is there will be a couple of groups funded to do a service on a smaller scale to continue their approaches. Most of the other people will go back to the old-fashioned way, and maybe there'll be more just general collaboration because people aren't now sort of used to working together and sharing software tools and things like that. It might be a good thing. Sure. Quite conversely, you know, most people are not forced to work together anymore. It'll become very easy also for them to work together. Sure. I mean, there has been sort of this feeling of, you know, people have gone through this initiation by fire or something, having this alliance, and, you know, most people on the inside, I think, have a somewhat jaded view towards it, and so, for that reason, maybe it's a bonding at home. I guess, personally, as far as the science directions, I think there's a lot more hope. If one looks away from kind of the pure brute force numerical relativity approach and looks more at things that are combining perturbation theory and numerical solutions and kind of model of words, I don't know if you'd call it all this, but many industry of black hole perturbation theory being applied to black hole collisions, you know, there was... So, you know, Richard Price and Marty Poulin wrote a paper, and Barry Cook and I very quickly afterwards wrote a paper sort of extending some of their stuff, and it's been sort of, as I said, a mini-series of maybe eight, ten papers I have, and it's kind of, a lot of people are interested in extending these ideas to 3D readers, and I think there's some ways you can start to deal with binary problems. In a interpretive way, where you take into account some strong field effects as far as the dynamics of the particles or black holes, but you don't, as far as the weights are concerned, they're treated more perturbation-wise.

40:00 So, to me, if I, you know, if someone said to me, you've got to do something useful for LIGO in the next five years or your fingers are going to get cut off, I would go in that direction rather than the... I won't know you vaguely, but I was going to ask briefly, Ed Seidel is known to Europe pretty much full-time, so does that kind of place him for the future kind of completely different? I'm just going to have to ask something about that. I talked to some people at the University of Southampton actually, and they said that there were some moves afoot to study kind of a European program, but that was back in January, and I don't know where that stands now, but again, it's in the kind of alliance of groups in different countries. It now has probably the most numerical efforts in Europe working for him in the classroom, or at least a lot of traction on them. I didn't even bring up the NCSA dynamic in this whole thing. In the earlier stages of the Grand Challenge, there was sort of a... One of the tensions was between using the codes that were being developed at NCSA, which were probably most advanced in the sense of approaching the physics problem that we were interested in, but also sort of generally viewed from outside as maybe not being very sophisticated from a computer science standpoint, or at least not having done things very rigorously.

42:30 People also had a lot of doubts about the physics and what do they think would be done practically with mathematics. Yeah, I think that's one of the less fair phrases, but there was a general mood with them. So there wasn't, they weren't sure whether to use the NCSA codes or to come and develop new things. And so sort of both were done, and then... Of course the NCSA group moved off, and so with the idea that there would continue to be this collaboration, and for instance people like Peter Medians would be used as a conduit of information between the Alliance and Potsdam, but that's never really happened. There was a lot of distrust in political things at the time. So again, the lack of physical proximity. Yeah, so when did it begin that the NCSA group moved? Yeah, I'm making it sound like it was a long time ago, but it was really only last summer. But a lot has happened in the last year as far as, for the first couple years of the alliance, the groups were working more or less independently, trying to get something going in one area or another. As the time became more critical, things intensified a lot. There became a lot more pressure to make decisions to force something to happen. Really since probably the second two and a half, two years, or the latter two and a half, three years, we're in a lot more of this activity.

45:00 I was wondering what the computer science people involved in the two days and how many of them are out of it. Originally, there were four principal investigators who were from CS. Two of them are in the CS department here. Jeffrey Fox, who's the, maybe you know from Caltech, or you've probably heard of Caltech, is now the head of the Syracuse NCSA impact. So, there was a student at impact who developed a software called DAG-H, maybe you've heard of it. What it is is a general package for solving... PDs in principle using adaptive meshes on parallel computers.