Interview with Ed Seidel

0:00 Okay, now I think we're getting something. Okay, so now it's going. Okay. Is it picking me up too? I think it is. Okay. Okay. So, yeah, the funny thing was it was making like it was getting the sounds, but the pause must have been on or something strange. That's the trouble about doing mono. Anyway, so just to start off again, it's the 10th of February. It's now about 2.30. and I'm talking with Ed Seidel, and we've just been discussing how you and the Cornell Group had both, I gather in the early days of the Grand Challenge, had both been working on a means for finding horizons, and that they had a solution in which they were going to kind of integrate over all of the light cones to see where the photons went, but that Waimosuan, who you were working with at the time, had come up with a method involving looking backwards along the photon's light pad to see if it was sucked in close to the horizon and find the horizon. Right. So, yeah, this method was clearly a lot more efficient because you save your space-time and then you just integrate backwards once or twice and on a workstation you're done in five minutes and you found it. Whereas with the other method you need a supercomputer. Right. And not only that, but our method actually gives you the generators on the horizon arbitrarily close generators that are arbitrarily close to the real generators, whereas the Cornell method was absolutely unable to give you generators at all. So we could study dynamics of generators and expansion, shear, and all this stuff. In fact, we've been doing a lot of stuff along those lines. Okay, so there was this kind of competition going on, in a sense, and so Paul had been the undergraduate from Cornell, who's now a student here, had been working on that problem with Cornell, and he came, really fantastic you know and he was very excited about it but still he was feeling very weird that he was there working with us and wasn't unable to talk to us about his project because um they said not until the paper was published should he discuss it with us because you never know you know people aren't that honest or something i don't know so anyway um what happened eventually Paul had been describing to Stu and Saul what that was all about and then this got into a bit of a public email just between several of us

2:30 Stu and Saul and me and Moimo and so forth and they sent a note, they were very negative about this saying it could never work basically saying it could never work in fact I still have a record of all these emails to show you how sensitive I am I've kept them so I can sort of prove anything But, so the thing is, at one point they'd sent a note to Paul responding to him saying, where Paul had said, you know, dear Stu and Saul, I'm sorry, I think you didn't understand something about this method. And they sent him this very sort of damning note that they sent to Paul and then blind copied to a number of people, I don't know who, but I got one, Wymo got one, saying, you know, Paul, we fear it is you who does not understand the dynamics of event horizon and so forth. but that really sort of slapped him in the face. And if something, Paul just felt devastated by this. These two guys that he'd sort of idolized as an undergraduate had sort of really slapped him down and put him in his place. And I was really angry about this. They were wrong. They actually hadn't understood something, but Paul was just trying to get into normal discourse like as a scientist. He's growing up to be a scientist, you know, and they really slapped him down. like I felt like it was um what you do to a child if you're a nasty parent like to hang somebody's bed sheets out the window if they've wet the bed you know what I mean to sort of say look this kid has has done something bad you know something terrible to behave like that so I really stepped in and I got into a strong confrontation with students all about this and and they backed down and apologized but I really didn't like that but but anyway so what developed from that was um They eventually realized that our method was really the way to go. It was the best way to go. And then they, for example, during a talk at Illinois, one of our Grand Challenge meetings where KIPP was and a lot of important people were to kind of review the progress, they gave a talk where they implied in their talk that actually they'd been doing this all along. They didn't exactly say it, but they said it so everybody would have thought that that's what they meant. But technically they said it in a way they could always defend. And so people had the idea. And I remember Paul sitting, he was sitting next to Larry Smarr, and I got very angry when I heard this, because I've obviously been sensitized to this. And Paul, Paul, it's kind of red-faced already, you know, and his face just turned red, and his ears get particularly red. And Larry looked at him and said, you know, Paul, what's the matter? And Paul said, he's lying. This was Saul. So he's lying about what he just said. And Larry just said, not surprised.

5:00 so you can see that they're like this is the kind of history that went into this with these kind of feelings about different groups whether or not people are lying or not I don't know you know but these kind of sort of slimy things okay so then then I mean this goes on I want to tell the story in some detail so you have an idea this is only one of which there are a number that are sort of this is like the prototype of a bad experience that came up then there was a paper that stew and saul published where again they made it sound at least to us they made it sound like they had come up with this method of doing backwards integration they and it was like no big deal obvious this is the way you should be doing it you know um and we had the opportunity in a sense for i'll tell you about in a minute to be on a paper with them um where we we had to fight them to say no this is not right we have to insert the And they gave us credit for it. And now I've seen Saul, for example, give talks at big meetings like the Chandrasekhar Memorial last year, where he personally credited Wimo and me for coming up with this. So he knows we're sensitive about this idea. In fact, he gave me the credit for it when he actually should have given it to Wimo. Anyway, that's another thing. Wimo doesn't feel very comfortable because he's always getting this kind of thing. He's not getting the due credit for a lot of his work. Okay. But it gets even worse, because we had, at the same time, we had submitted a FISREV letter describing this work, and it was held up by the referee, and with extremely negative comments, saying that the method was wrong, it couldn't possibly work, blah, blah, blah. And we don't know who the referee actually was. Are we getting anything? Yeah, we're doing good. Okay, good. We don't know who the referee actually was, but we think it was Matzner. told us that he wasn't the referee, but he knew the referee very well. But there were a lot of signs, like he showed us a figure that he had drawn that was exactly the same figure that was in a referee report, hand drawn. It was clearly, you know, his figure. So he didn't cover his tracks very well. But the thing is, the referee report was very negative, and the referee suggested that we should take out the two black hole collision, finally, and that we had traced out the event horizon for the two black holes, and just not make this a FISRAV letter, make it just a FISRAV-D article, okay?

7:30 Now, okay, that's fine. At the same time, Matsner was trying, once people realized that this was really a big deal, was trying to make a grand challenge paper, and he was suggesting, what do you know, that we focus on the two black hole thing and write a paper for science, okay? And only the PIs of this grant should be allowed to be authors on it, okay? So I had a huge battle with Richard for months, literally months over this. I was not going to budge because I wasn't going to let Wymo, who came up with the original idea, not be allowed to be on the paper. And also, I wasn't going to allow Paul Walker, who had been essentially actually doing the work in terms of the evolutions and so forth, to not be on this paper. The paper, by the way, was one where they took a Cornell space time of dust that collapsed to form black holes, and then those two black holes collided, and our calculations of the Misner data that had two black holes existing already, but then collided. Okay, so I want to show you something. It eventually was published. I hope I have a picture of it. It's on the cover of Science. Oh, is that right? No, it's not there. I'd really like to show it to you, because it's a beautiful cover of Science. It was the feature story. It was a Grand Challenge publication. Well, I can show it to you later, probably. Maybe not. I'll show it to you later. Okay. But the point is that, in the end, we did write this joint paper. Madsener backed down and allowed Wimo to be on it after a couple of months of very tense arguing with him about this. In the end, I believe, I don't know lot with Kip. Wymo and I talked a lot with Kip. And I know that Kip had some influence on Richard to get him to back down. So he finally allowed Wymo to be on this paper, but he wouldn't back down to allow Paul and Joan to be on this paper. So in the end, there's this very prestigious publication coming out that Paul actually did a lot of the research and even created the figure that went onto the cover of Science, and he wasn't allowed to be an author. And the feeling was only senior people should be allowed to be on this paper and i was very i was very angry about this and i fought it for uh for many months but in the end i didn't have any support all of the other pis were like um you know well so they're just students and they can publish their work in phys rev d or something like that you know so um they'll get they'll get their due in the end but this is of course after paul had been through this experience with uh with stew and saw with

10:00 this email and these arguments and so forth and this weird feeling about not being allowed to because people might steal it or something like that. So in the end, Paul is leaving physics. And I know that this has been a major part of it, because he thinks, why should I stay in this community? Look at the way people treat each other and so forth. And I'm just focusing on this one example, but it's not the only one. So there are several other things that are kind of equally bad. Well, it varies as interesting. So one thing is that there was a certain kind of, well, rigidity to the hierarchical structure of the thing that kind of the people who were the big people of the thing felt that there were certain kind of perks or, you know, there was a certain kind of prestige attaching to the thing which really should be reserved. Right. And there's also the political expedience of trying to publish a paper to show that, hey, this grand challenge is actually doing something together. Of course, that was totally forced by the head PI, who had nothing to do with it, you know what I mean? But anyway, it was for the face of it, so that funding could continue, frankly, and so that he could look like he was actually being a leader. I know this is, I mean, I just think that's the fact. Probably he would even agree with that if you ask him about it, but, you know, if he's honest with himself. But, yeah, so there are a lot of pressures in these big projects like this. You must have heard lots of stories about LIGO, right? I know there's been a lot of nasty stuff happening there. But it has a lot to do with ego, prestige, funding, and so forth, and it causes people to do things that I think they probably wouldn't do. Before any of this happened, I got along wonderfully with Richard. I mean, you know, I just thought he was a good guy, kind of had fun even with him, you know, going out and drink a beer down in Austin or something. But when the thing really got going and there were these pressures, then people began to behave in ways that, unethical, what happened there. The worst thing is, when we got here, Paul, Paul tends to be kind of an emotional guy, he was, he got, somehow Paul got very, very angry with me, and it kind of went on for a good part of last year, and we weren't getting along well, and you know, I had many long discussions with him, I took him out to dinner a couple of times, went out for a beer to discuss things, and he could never really pinpoint exactly why, but he was behaving very, very angrily towards And in the end, he repeatedly told me, he thinks I really let him down in this science paper by not fighting to the bitter end.

12:30 He thought I should have just withdrawn and said, forget it. I refuse to allow my name to be in any of my results to be in this paper. And I know that this was clearly something, you know, he really felt let down by me. Now, he was never in on any of the phone calls that I had with Richard that went on for literally months where I wasn't budging on this, you know. But the thing is, he didn't really see it. I can tell him, oh, Paul, I really fought hard for you, but, you know, how could he really know? So this has had a serious impact on somebody that I think is probably the most talented student in all of numerical relativity. I mean, he's really great, and I feel terrible that he, I mean, he's almost making me want to cry. I feel terrible that he feels so bad about it. And he had this sort of ideal of, you know, physics is this pure research thing, and I can just go sit and think great thoughts, you know, and come up with these great things, and then to get embroiled in all of these political controversies, and then, as a result, want to leave the field. That's not the only reason. I mean, Paul's very talented at computer things, so of course he can make five times the salary immediately, but I think at the heart of it, it's this, because he would be willing, I think, to take the lower salaries and so forth for the pureness of doing research, and he's not going to. If he felt he could get credit for what he was doing. Right. I really think fundamentally that's at the bottom of it. So, I mean, obviously a lot of these things are details that I would never want to see published, understanding of where uh you know a lot of my attitudes about this grand challenge where they're coming from this is kind of the it is actually very helpful for me to have an idea of what because people will tell me you know that oh they've been very honest very unsatisfactory things but you know people aren't willing to be specific so just uh so that we get this on the tape too i'll reiterate that obviously i wouldn't be publishing or spreading about anything that that you say or saying that you've said such and such without, first of all. Yeah, okay. Well, I would appreciate that. And, of course, everybody will have their different take on what actually happened, so, you know, but this is the way I see it. Yeah, well, that certainly, it's interesting to me. And, as you say, clearly it's a personal shame and also a problem for the field difficulties which arise out of these new groupings and so on are going to cause young people to actually leave the field, were to happen in this. Yeah, right. While we're still on the subject of kind of grand challenges, alliances, and so on,

15:00 when I was in Southampton early last year, but I haven't really had a chance to follow up on this yet, they mentioned that there were some moves afoot to kind of get together a European equivalent of the grand challenge, looking at the binary backhoe from groups in different areas. Do you know if anything has come of that in the meantime? Yeah, yeah, I'm involved in that. And that would be a much looser organization than the Grand Challenge. It's a so-called network proposal. And the basic idea is to focus different research groups a little bit more towards cooperating, but not to force them to come up with a particular end piece of software that solves a particular problem. But it does things like you'd be working in a general area, let's say stellar collapse. And so somebody does this kind of work, somebody does that kind of work, and you exchange postdocs and students often enough that there's real interaction between the groups. And so that is much less focused than, say, one of the American Grand Challenge projects and probably has a better chance of success in that way. There won't be anybody saying, do this, do this. You're not allowed to be on this paper. There should be this overarching paper, and a whole host of things like that. Right, so to justify the existence of the network, there doesn't necessarily have to be a paper with the head of every group. That's right. This is a European Commission funding thing, so I think I've heard of these network things. So, in your experience, that seems likely to be kind of a better funding model, you think, than the idea of a... I think it's kind of potluck. It depends on the people you get, and there's no way to really know, even until they're funded. I mean, for example, the Neutron Star Grand Challenge, I was completely shocked that that rift developed. And I didn't think it would happen at all. I don't think it means that they're all doomed. I know some of the other grand challenges, at least on the outside, not being personally involved in it, and they seem to have had their act together. And I think part of the reason was that those other grand challenges, the more successful ones, in terms of the sociology at least, were much clearer in division of labor and expertise. So say there's one of doing cosmology. You have somebody who's an expert in particle methods for, you know, n-body simulations. Another person is an expert in gas dynamics. Another person is an expert in, you know, stellar birth or something. I don't know, you know. And so all these people bring different expertise. Whereas in our Black Hole Grand Challenge, basically everybody was a numerical relativist

17:30 trying to solve the same problem with different techniques for the last 20 years. And so it's hard to get them to change their ways, basically. so the more engineering oriented ones I think have a better chance of succeeding as a centrally controlled one even though maybe LIGO's not the best example of that, I don't know other grand challenges that appeared to work better looking from the outside when you said they had a more engineering approach, was that made possible because somehow the the science was better understood to begin with to the extent that people already had a clear plan of how they were going to get to their goal. Yeah, right. Yeah, I think that's the case, yeah. Sort of, you know you have to have stars, you know you have to have end-body interactions, you know you have to have gases, and so you just get the expert in each of these areas. You know you need adaptive mesh refinement, you know. And so you might say the same thing about the black hole problem, but even though you know you have to have black holes and you have to have parallel codes, you don't know what shift to use. Do you have to use apparent rising boundary conditions? All you really know is about the initial data, but you don't know what path to get to evolve it. So you've got characteristic evolution, you've got Cauchy evolution, you've got Cauchy characteristic matching, you've got a whole different formulations of the equations, ADM, hyperbolic, York's hyperbolic, John Massot's hyperbolic, and which one's better? Well, we don't really know. We only developed these things the last couple of years. So let's do all of them. But if you do all of them, then you're not focused. So if you only do one, then everybody else is mad that you're choosing the wrong one because it's not the one that they believe in their heart, it's the right one, the one that they came up with, you know what I mean? So it's very natural, I think, that those problems have happened. And I'm actually very pleased at how well they've managed to recover from all of those terrible things that happened during the early part of it. I think a lot of mistakes were made by people who just really didn't know how to behave, I think, but they realized that they were getting a bad reputation, and they've pulled together much better in the last year. So I do truly feel more optimistic about some of the collaborations, some parts of them, and the possibilities for success. But in the end, as I said, they're trying now to get funding to continue for that Grand Challenge project, and they couldn't agree on the approach to take, so it split.

20:00 Texas, Penn State, and, let's see, Texas, Penn State, Pittsburgh, right, alliance, basically, versus the Cornell, now Illinois, now it's due is move to Illinois, North Carolina. And there were very strong disagreements, so they decided to write two proposals that are sort of competing against each other now. So the Cornell, Illinois, North Carolina is a hyperbolic approach, based on Jimmy York's hyperbolic approach. Right. The other one is still ADM, so the other thing you mentioned briefly about the grandchild that one of the successes had been on the computer science side. I gather that for instance this DAG-H code that was developed which is parallel code for doing numerical work on parallel machines. Right. That's been one of the big successes of the thing. Is that, well, basically I don't know too much about how it works and that. Is that something that is kind of a more general tool? It's not something that was really the work of the relativists involved. It was more like a tool that was built to provide the bedrock for the... That's right. That was the plan. unfortunately it wasn't supported heavily enough I think compared to the number of lines of code that were written so there was sort of one guy, Manish Parashar who was working at Texas unfortunately there were also personality conflicts just even within Texas so those operations didn't go that smoothly between the computer scientists and the physicists although it wasn't a disaster but it just could have gone better I think but that was the basic parallel structure people were supposed to build codes on top of. And it also would provide the capability of doing adaptive mesh refinement so you can refine areas, say, near black holes and where waves are going out, but not areas where you don't need extra resolution. The problem is people who actually played the game according to the rules, that is, try to use this thing, were in a sense penalized because the code wasn't heavily supported enough. So there were always bugs and then the people trying to do some physics or testing were being held up and it wasn't efficient and and so forth so that's one thing that happened to the cornell north carolina hyperbolic code they decided to build it completely on dag which i think was the right spirit but in

22:30 the end since dag wasn't really up ready for prime time yet they couldn't they found they couldn't do real work with it and in the end their code has been dropped by the alliance that was another very painful decision that that richard made um at the time i thought it was a wrong decision now in retrospect i think maybe i don't know if it was the right decision but it turned out okay i think um you know um so so but the point is that i think we try to do too many things as a collaboration and didn't focus enough but maybe it was it was just impossible to focus because are too many different ideas about what to do. But even in the success story of DAG, or DAG-H as some people call it, it's still not working as a production tool. And it's very frustrating for all of us. We thought, I can tell you now, to show you how naive we were to show you how incredibly naive we were in 1992 I had a few students Karen Camarda I don't know if you've met her but she's now a postdoc at Penn State and some people at NCSA we were working with another person at NCSA doing adaptive mesh refinement and this is August of 92 before I went away to Australia for six weeks for four weeks and before I went we had a meeting with these people computer scientists doing adaptive mesh refinement and we said okay they said their thing is about ready they want to try on real applications okay we'll rewrite our black hole code so we can use it with your adaptive mesh refinement code and we'll be doing all this great physics by the time i get back from australia okay so of course it didn't happen and it still hasn't happened but we thought we were a couple of weeks away from being able to do that and then dag came up and at first we were a little skeptical but we got to know manish and at least the people at ncsa really hit it off with manish and there's a very good relationship between manish and and and the NCSA group, and now here. In fact, Manish has come to spend two months here working with us on different occasions, two one-month periods, and continues to work closely with us. We've hired him to work on the Neutron Star Grant Challenge, in fact, so another thing. Okay, so he was developing DAG, and he and Joanne also hit it off very well, and there was a meeting in Austin, I think it was in 95, it might have been in 94, but in October, and Manish worked a couple of nights, like all night long, two days in a row,

25:00 and they thought they could get our 3D code running with DAG with adaptive mesh refinement by the meeting, the Austin meeting, you know. So they worked, and like the night before, you guys going to get it to work? I think, yeah, it's really close. It didn't, and it's still not working. Now, even, you know, and there's still been a lot of effort on it since then. So that's just, there was a lot of optimism. I mean, I think at the time, you could think there's just like one more problem, ran much deeper than that so i i don't want to say that dag hasn't been a success but it's still obviously not being used as a production tool that was really needed from the beginning and the plan was to get it done right away and then plug in all these physics modules and we'll be doing all this great physics does too complicated yeah there's a problem well does the fact that you you now have cactus running in that does that kind of release or ease your dependence on tools like DAG or is it sort of... Yeah, well, you see, we were lucky. We weren't dependent on DAG because before the Grand Challenge started, we were already writing 3D codes. We started in January of 93, really going into 3D. The Grand Challenge didn't get going until the following August or something like that. So, we had written codes to do 3D numerical relativity Like the CM5, that was a parallel machine, but it didn't require anything like DAG. And so by the time the Grand Challenge was really forming, we had a lot of experience doing 3D in parallel. And so we were sort of trying to take our existing codes that were really producing work for us, I mean physics, and then we would play with DAG, but we weren't dependent on it to do research. And so that's one reason we were able to do a lot of things in 3D while the other groups were not really able to, just because we started ahead of time. And it's just by luck. We might have started just with Dag right off and would have had trouble as well. Okay, so we've learned a lot about not depending on any one way to do something. And when the Cactus Code was written, we decided Paul has worked very closely with Manish. So Paul knows Dag inside and out probably as well as anyone except for Manish. You know, he knows it the best. So Paul decided to write his own layer of parallel message passing for Cactus that would be very simple and would work efficiently, but wouldn't have any possibility of doing adaptive mesh refinement or whatever. So the plan for us was to just use that as production,

27:30 but then keep trying with DAG. You know, try it again, try it again, use it as a driver, and when it finally works, then we'll switch over to DAG. Okay, but we never wanted to be dependent on one way to do it. We were discussing the differences between the kind of network that you have here in Europe with St. Hampton and other groups, and the model of funding that you had in the Binary Black Hole Grand Challenge. When I talked to Doug Swesty in Illinois, he was mentioning that one difference in the funding that they have with NASA, that you have with NASA, is that they, there's a kind of, that certain levels of performance or something have to be reached after every year or something in order for funding to continue. That's right. And I presume that this is a somewhat unusual method of finding it. I've never seen anything like it before. It does focus you. You've got to get 100 gigaflops? Okay, drop everything and get 100 gigaflops. I mean, we didn't like this at all, and we complained about it. We had this big, long, whole-day negotiation where Doug played the leadership role. His dad apparently taught him how to negotiate buying used cars, And it was just a pleasure to see Doug negotiating with these four guys from NASA, saying, you know, what do you mean? 1.2 million? There's no way we can do it. We've got to have at least 1.4 million. Come on. That's just an extra .2, you know. And he was great. He got it up to like 1.35, and finally you could just tell that they just couldn't go. So in desperation in the end, he said, okay, you've got to at least throw in one of these NASA graduate fellowships, or we just can't do the project. And finally he said, okay, you can have this NASA graduate. So, you know, it was really like car bargaining, you know. I want my radio, and I want it to be FM stereo. No way, or I'm not buying the car, you know. So anyway, so where was I going with this, though? I got distracted with this Doug story. Oh, yeah, about the milestones. So during this whole thing, they kept saying the milestones were really important to us. And they were cutting our budget from what we had proposed. and they basically wanted to cut out all the physicists

30:00 and just make sure that we could get enough computer science support so we could get the milestones achieved and finally it just came to a head and they said look, we're just not interested we figure if you can get a code running 100 gigaflops, you'll find a way to do some physics with it, and I think basically they're right, it's true if we could get the code to go that fast and on a real problem, not just some infinite loop, you know, that's an empty loop They knew that we would find the resources. And so we fought it enough so we were balanced and we had physics personnel for the most part. But still, that was their attitude. So basically the milestones are that you have to have certain amounts of code working at certain speeds. Right. So we had a milestone last August. We had to achieve 50 gigaflops on a Cray T3E on a code that could do both GR and hydro, coupled code. And we did it. And we were very focused on meeting that milestone. We then got 66 gigaflops. It's one of the fastest codes in the world on a real application code. It's really something. Now we have to achieve 100 gigaflops by next summer, and I think we'll probably be able to do that just in about two months on a special machine. so and that's the cactus code so oh yeah so basically cactus does the GR part and then you have another code to do the hydro yeah so the hydro codes are modules that plug into cactus basically so cactus is the GR solver and all the computer science infrastructure that allows the parallelism and so forth all So how close are you to getting some physics out of the... Well, with CACTUS, I mean, you heard some of the discussion today. It's been a big project, and in some ways it's still not up to the level of the code that we call the G-code. I don't know if you've heard about the G-code. That was our code that we started in 1993 on the CM5 and on the C90. That's what allowed us to do these calculations of two black holes colliding, where the waves are coming out and so forth. That was a 3D calculation. So we've continued to push the g-code, a few of us, while trying to bring the cactus code up to the same level.

32:30 Now, it's much more advanced computationally, but in terms of the fine-tuning of computational parameters to get that physics result out, it's just about on the verge of being able to do that. So you heard, we think we have row waves collapsing to a black hole, but our results aren't agreeing with Epley. So maybe Epley was just wrong, and we're just wasting our time, but we're not sure about a few things, you know? So we're just about to be able to reproduce these kinds of results with CACTUS that we've got in this picture here that we could do with the G-code. And with neutron stars, we have single neutron stars that we can evolve for roughly a millisecond or two, but we need to be able to go a couple tens of milliseconds probably to do the calculation. So our first goal is we're going to do the head-on collision of two neutron stars. I mean, the first physics goal for the NASA project. And we hope to be able to do that at least with some degree of success, of really doing a physics problem, not just a demonstration that the code can run, by the summer. So we're getting there. The cactus thing has really taken off. It was something that, again, Paul really designed based on what we all felt were a lot of failings of the Grand Challenge, that there wasn't any real collaboration. There weren't a lot of people working with the same code, contributing a module to it or whatever. So the efforts were quite fragmented. And at NCSA, we had a couple of these different 3D codes, so-called G-code and the H-code, two different 3D codes based on different formulations of the Einstein equations. And the G-code was a code that we tried very much to make a completely open code. So everybody in my group at NCSA and YMO's group at WashU would contribute to the same code. But then we got into the problem of having 15 different versions of it. And we spent a whole year trying to reintegrate those things, all the stuff people contributed to their own versions, and it just failed. We just couldn't do it. Okay. On the other hand, there was the H code that Joanne controlled very, very strictly and hardly let anybody do anything with. So it was very promising. It was much more advanced computationally. But because only Joanne was really using it, it never really accomplished very much, because it's just much more than a one-man job. So we try to take the best of each of these strategies and put it in Cactus. And Paul created this wonderful code that's complicated. You might have heard Doug, in fact, complaining about it.

35:00 Doug is the one person who can't stand the Cactus code. So that's, in fact, that's part of the rift that's happened, unfortunately, with Doug. It's Westy, that is. but anyway so this code is now being used by at least a dozen people here and we have people from Cray who are helping to tune it people from SGI helping to tune it on SGI machines we have people from the Konrad Zuse Institute here in Berlin from Garching down by Munich they've written like parallel isosurface finders that run with the code people at NCSA have written the parallel IO libraries that go with it have contributed to other kinds of modules. The WashU group, we're all contributing everything. You might have seen the Cactus page, you know, we're trying to maintain. We're hiring a research programmer. So now the thing is really picking up steam. And I'd say probably we have 25 people really active using that one code as their main research tool in 3D. And, you know, you can imagine how much you've been accomplished with that if everybody can leverage off of what somebody else has contributed to the code. do. That's the way I think the Grand Challenge should have been run in the first place, the Black Hole Grand Challenge, but there just wasn't that feeling of cooperation like there has been basically with the groups at YMONI developed. And when I came here, I made it very clear to the people here that I consider them, the Wash U people and NCSA, I still keep a close connection there. They were just essential parts of our research operation constantly making trips back and forth, and the project pages help us keep these things going. So anyway, a guy in Wash U can contribute a routine. Somebody here can use it. Somebody in Mallorca can use it. Somebody in Valencia, they're doing 3D hydro with the Cactus Code. There are three different 3D hydro efforts, all being plugged into the same space-time code, and they all use different algorithms. And maybe one of them will be better for colliding neutron stars, and one of them is better for looking at relativistic jets, and it's all available to people. But that does cause a lot of agony among people who are used to thinking, I've got this algorithm, and as long as I keep this algorithm to myself, I can do some physics that somebody else can't do. So, you know, how much people are willing to put into the common pool depends on the person. So there are always these negotiations that we're having, you know,

37:30 with the people who aren't kind of part of the core group of the WashU and the Potsdam group. And it was also, it was difficult, I think, for the people here in Potsdam to get used to this idea, too. The first year, last year, was very difficult, getting people to really trust each other and so forth. And there were problems that really boiled up over the summer while I was gone. I went away to NCSA for two months this last summer. And when I got back, it's like people were talking to each other. You know, there were terrible personality clashes and so forth. and i think it was good that it happened because a lot a lot of stuff out that had been built up over the year so we just got it all out we had an encounter session i took everybody out for a beer you know and uh we tried to get it out and now since that happened i think people have a much better understanding and that the group is working beautifully now but it took the first year to kind of to get things out of their system uh the the bad thing is that there's a two-year postdoc cycle basically. So just when everybody's really getting in a groove, and they really are, then half the people will be going off to their next job next year. Yeah, that's unfortunate. I was going to ask, in fact, how big a role, you've already kind of alluded to it a couple of times, bigger role people moving from one group to another plays in the cohesion between the groups. I mean, will some of the people leaving here be likely to go, So, for instance, to Washington or a university or to... I wish it were that way, but it's not. It's just the way it has worked out. For example, YMOA has got a group of three postdocs and some graduate students, and they're staying right now, so there's no transfer there, and they're not coming here either. But some of the people here are going out to other places. For example, Penn State. Karen, my student, just did a PhD on 3D black holes, went to Penn State. Now Steve Brandt just got a postdoc to go to Penn State as well. So that'll probably bring up collaborations with people at Penn State a little more closely than they have, and that's fine. Karsten Gundlach is going to the University of Chicago next year, and he says he's committed to continuing working with Cactus, so maybe that'll bring people like Bob Wald and Bob Garoche into numerical relativity. Apparently, Wald is interested, and that's one thing he stressed to Karsten, that if he went there, he could continue to do numerical relativity with Cactus and with our group. So I hope that'll work out well. But on the other hand, because of these clashes with the Grand Challenge,

40:00 nobody here has had any interest in, say, going to Texas, for example, or back to Illinois, because Illinois has now sort of changed hands. Stu came as I left, and so people are very kind of weary about going to Illinois. Yeah, it's a shame. I left some people back at Illinois, like Pete. Pete has come here a couple of times and I hoped that we would get past the Grand Challenge rancor and when Stu went to Illinois, you know, I've made a lot of overtures to try to heal things up with Stu and so forth but there's no interaction between say Pete and Stu and now I know, for example people here who had a possibility of doing a post-doc at Illinois they might have had a possibility to apply. So there are these sort of sharp feelings. And so what's happening is the people here who have been kind of in my group or YMO's group are going out, but they're not quite yet filtering back into the, directly into the Grand Challenge, but like Penn State's kind of a neutral camp, so like Karen and Steve feel comfortable going to Penn State, but they wouldn't feel comfortable going to, say, Texas yet. I hope like one more cycle of postdocs and then maybe there'll be much more healing, you know, people can get back That's interesting. Oh, yeah, on the subject of the neutron star binary problem, the one thing that's interested me, because when I was in Kib's group for a while, I looked at, well, you know, possible data analysis methods for extracting information like equation state information from neutron star binary signals. years ago, we were very interested in keeping an eye on what was coming out from people doing hydro code of various kinds. And one of the results that came out more or less just at the end of the period when I was working on it was Wilson and Matthews' work with their code in which they had this unexpected star crushing effect, they called it, where the neutron stars were going to collapse actually before they banged into each other. Right. So I've been sort of following that to some extent. And I guess you're not at the stage yet where you're actually doing the two neutral stars spiraling around each other probably.

42:30 But that's more or less the reason why I was asking when you'd be getting to that point. Because one of the funny things is that that debate seems to have sort of come to a halt with some people feeling that this wasn't, you know, Wilson and Matthews just can't be there and have more or less shown that it can't be there. Basically, nobody believes it. Yeah, whereas Wilson and Matthews are insisting, no, it's still in our code. So everybody's sort of saying, well, obviously somebody else has to go away and write a code to do the same problem. But of course, it's a question of exactly how long it's going to take people to do it. Well, so that's something obviously we're very interested in. And I think they do their calculation in a co-rotating frame. In the co-rotating frame, the neutron stars just collide head-on. So in some ways, our head-on collision should be like a zeroth-order test of that. And so we'll be looking at that very carefully to see if we see some effects like that. So we may be able to have something to say about that this year. I don't know if we will or not, but at least without rotation. Right. But I just commented on another thing about the sociology of that thing. Everybody I've talked to, and I feel the same way, knowing, you know, Wilson and Matthews both, they would have been completely discounted, except for the fact, particularly, that it's Jim Wilson, you know, and everybody says things like, you know, well, you can never quite figure out what he's doing. I mean, have you ever talked to Jim? Yeah, but just briefly, I'm hoping to go and meet him later this year. He's a really nice guy, you know, and he's also very smart, but he never makes himself very clear. So you're talking to him, you never quite know what has he really done, no matter how much you press him. But everybody has said things like, you know, I would think this is complete crap, except that it's Jim Wilson who always, who never can really justify what he does mathematically, but usually he does things that are right. He does the right thing. You know, he's got this incredible intuition about how to treat this or that term in an equation, numerical methods, just sort of by the seat of his pants, knowing, just feeling what the right thing is. And so there's that kind of mystique about him that give people the feeling that maybe there's really something to what he's doing. Whereas if I think some other people had been doing this, they would have just been completely discounted, you know. it does seem to be interesting that there's that as you say kind of sociological yeah well Doug even said something he had a great comment he said you can't really

45:00 attack him because it's like attacking Santa Claus you just can't really do that and he looks a little like Santa Claus so with the with the European network that's going on what is your group's role in that and is there so I guess there's no does it work? I mean, you were saying there's no kind of single approach. I know that the Southampton group were working on some new approach, which I haven't got the details of yet. Well, they're doing Cauchy characteristic matching, so it's similar to what the Pittsburgh group was doing and what their role in the Grand Challenge was. But no, that actually, it's just a hope that it's not funded at the moment. So we went through a proposal last year. It was rejected, and we plan to address kind of some of the referees' comments and try again next year. Oh, so it's, you're still, yeah, you're going far. And, um, so in that, so in that case that the Sunhanton group would be doing, uh, uh, would be this, um, would be similar to what the, what the, what the Pittsburgh group were doing, you were saying. Yeah. They'd be playing a similar role. Yeah. So then you'd be providing the, um, uh, the space-like part of the evolution, the Cauchy evolution, and then they would match on, say, a characteristic evolution. I mean, that's one of the possibilities. They also have some new people there. Nils Anderson will be joining there next year, so he does a lot of work on stellar perturbation theory and so forth. So there could be other directions that they would connect in. I mean, for example, one thing that I'm getting more interested in doing over the last couple of years is applying perturbation theory as a test bed for numerical relativity and as a way to help interpret results. Because you know about the work of Price and Pullen, maybe, doing a perturbative approach to the two-black hole collision, and that turned out to be spectacularly successful, much more so than anybody would have thought a priori. And I was really impressed by that. And also we worked with them. We published a couple of papers together on the comparisons between numerical and perturbative approaches. And so that's been so important in the black hole thing that we're extending that quite a lot now to do comparisons between full 3D numerical relativity now and perturbative approaches to evolution. the same kind of plan in mind to do it with neutron stars so for example you could take a neutron star fully relativistic and hit it and then see how it oscillates and see if you can reproduce the known normal modes of that

47:30 system right or even better take a rotating relativistic neutron star and perturb it and calculate what are the normal modes because they've never even been able to do that perturbatively yet that problem still hasn't really been solved so there are a lot of physics issues there so the point is that I see you know, people might have thought that numerical relativity would just replace perturbation theory because once you run out of perturbation theory, then, of course, you want to solve the nonlinear problem. But it turns out that the perturbative approaches are just as important now as they ever were, and maybe even more so because you can actually use them as calibration tools, or also to understand what's happening. So, I mean, this got a little bit on a sidetrack, but the original point was that NILS is doing perturbative kind of studies, And so that would be a different connection with the Southampton group. But just as a general trend in numerical relativity, I think the perturbative approach is becoming very important, kind of almost having a rebirth. Like in the 70s, everybody was doing perturbation theory. And now many people are doing it again. But thinking now, how does this help me calibrate a numerical code or interpret the results? Because we were getting results that we frankly didn't understand physically from our numerical calculations. And by looking at different perturbative techniques, treating the black holes as if they were close, as if two black holes are actually one, or as if it was a point particle falling into another black hole. Those different calculations provided different regimes of agreement, and then it helped us understand the physics much better. And so those approximations and techniques, I think, have a long way to go. And so you'll be seeing a lot of that, I think, in the next few years. That does seem to be interesting because one of the, for instance, One of the things that sort of became obvious to me already looking at the Wilson and Matthews thing was that there was a sort of a, between them and the Caltech group, it was sort of like Wilson and Matthews would be saying, well, this is what we're getting out of our code. And the Caltech group would want to be saying, well, okay, so how does this show up, you know, in terms of post-Huttonian thing or in this kind of limit? and Wilson and Matthews would be would be they'd be dissatisfied with what Wilson and Matthews would be responding they'd be sort of saying well I could kind of maybe come to this part of our code sort of thing so that there was a certain difficulty in providing the physical interpretation from the point of view of it and I did talk to Jorge Pullen about some of his work a while back so it does sound interesting

50:00 And when you say that this is useful for the calibration of more purely numerical code, you mean that during the process of kind of developing the code it can tell you where you're going right? Right. If you're in a regime where you know the solution should be amenable to a perturbative approximation, then you ought to be able to get the perturbative result unless it's just sort of too low the signal is too small for your numerical code to pick it up or something like that but then it's important to know that what's the minimum gravitational wave amplitude you can actually see in numerical calculation for example so we're using it like that and we've been really surprised at the accuracy of the numerical codes that we've been able to reproduce perturbation theory so much In fact, have you seen the second-order perturbation result? I'll just draw you a cartoon of it, but this is for two black holes colliding. It's just unbelievable. This is the waveform. So you've got two black holes that collide, and so they come together, form one big black hole, and then you extract the wave at some large radius, so the wave kind of comes out. And you want to look at the waveform here. And so this was a calculation I'll just draw here, a cartoon of the numerical result. So it was like this. We did this first. So this is normal mode coming out from the final black hole. Okay. Now, this is something we got, and we were a little nervous about this waveform. We didn't know if it was correct or if it was off by 20% or 50% or 100% or whatever. And then we did this comparison with Price and Pullen with their first order calculation. And we were just amazed that we got a result that was kind of like this. we thought, wow, you know, how could our numerical result be that close to the perturbative result, you know? And on the other hand, Price and Poland were thinking, how could our perturbative answer be so close to the right answer, you know? So we had completely different attitudes. And then they extended the first order calculation to second order, and they made second order corrections. And in this one regime, there's one regime where it really worked, it took their dotted line and it was just perfect right on when they went to second order perturbation theory and it's just staggering that it could have been so good and so Jorge Polin at the GR15 meeting or I guess it was 15 in India

52:30 sorry in India just a couple of months ago gave a talk about this stuff and he said you know it was one of the happiest days of my life when we did this second order perturbation calculation we laid it over the numerical one it was right on was one of the happiest days of my life when i saw that result too because i thought wow how could our numerical results be that great you know so each of us didn't particularly trust our own methods i mean numerical relativity is an approximation and you you do your best there are lots of systematic ways you can estimate your errors and so forth but there could always be something wrong and it took it took a lot of years of fine-tuning the codes to get to a result that we felt confident as confident as we did feel but still there was doubt so this gave us all a big knowing what the perturbative guys were in a regime where they could tell clearly that their method should be breaking down in some way. They sort of knew it should be breaking down, but it was actually correct, but they would never have believed it unless they could see from the fully non-linear calculation how good it was. That is interesting because Jorge did show me the thing and he emphasized how thrilled they were that it came out close. Yeah, he said that during his plenary talk. I wanted to jump up and say it was the happiest day of my life when I saw those things but i i didn't um the uh so so basically the you it's just a new way it's a a new useful means of providing a limit in which you can test the code right i mean that's kind of the most basic level but then the by by diagnosing in what regimes a certain approximation works and it doesn't then it really does help elucidate the physics because you know oh that's really what's going on there, or that's not going on there, because that approximation is not valid. So it's been really helpful. So presumably it probably is a difficult problem when developing one of these complex codes to distinguish between what's arising out of the physics and what's, say, some artifact. Oh yeah, it can be very difficult. So that's one reason why progress is so slow, because you do a 3D calculation, it may take a week to get the result through the queue and then you get the result and it's just some wiggle away from you you just don't know so you try it again, different resolution maybe the thing changes dramatically and maybe not, maybe there's some bug now with the cactus code, you know there are literally tens of thousands of lines of code that people from

55:00 different groups around the world have contributed and you know, it's hard to do quality control in a sense on a code like that, you just don't know but you go through many careful cross-checks and so forth and then finally you're confident but it can take a long time before you're completely sure about a numerical result so having kind of agreement like this gives you a lot of comfort that you really have got something, finally you've got it right and then you try it on a problem where the black holes are further apart or something you mentioned that of course numerical relativity is an approximate method Of course, in one sense, you're actually solving the full Einstein equation, so it's not approximations in the same sense that I usually mean when we're doing analytical work. Where do the main, as it were, sources of error arise? Is it just in the resolution? In principle, it's just a matter of resolution, right? In practice, there are code bugs that may never be detected. So, for example, you can argue, well, my code has to be converging to some solution. But if you have the equations coded up incorrectly, it could be converging to something that's not the Einstein equation, but some other set of equations, basically Einstein plus error terms. Now, of course, there are ways you can cross-check that by looking at constraints. They should only be satisfied if the real Einstein equations are satisfied. So there is a way of cross-checking, but of course you have to go through those steps. Maybe there's some subtlety that comes up, or maybe everything you've checked checks 90% of your terms and then you go to do a physics problem that doesn't have any checks and in that physics problem there's an error in the other 10% of your terms and you don't know that. Or it could be that your analysis tools of the metric, for example, you just evolved G's and K's, you don't evolve physical quantities, so then you have to then process that and they compute some Riemann invariant or some waveform extraction or something like that, the post-processing is often very, very complicated. If you have an error in one of those routines, there are ways to check. You can cross-check with different measures, like maybe you have this really function as one measure of a wave. If you have Tchaikovsky function as another measure of the same wave, or maybe you look at Riemann invariance or psi 4 or psi 0, all of these kinds of things. But even if your space-time is converging absolutely perfectly and you're sure the constraints are converging to zero as you go to higher resolution, there could be something in the analysis routines that could be wrong.

57:30 Or, you know, just many, many things like this. Or maybe they're just effects that you didn't see. We found, for example, some cases where the code was crashing, as I was saying a little while ago, with linear waves. We were just evolving Tkalski waves. This was a few years ago. And just sent a Tkalski wave in. It's weak, very linear wave. What could go wrong? always crashing the wave would come into the origin the metric functions would start to go crazy and then eventually the code would crash it turned out just to be a problem of the slicing it's geodesic slicing it caused focusing of the coordinates and in retrospect it was obvious but the waves were so weak that we didn't really think that the focusing was um was the cause of it even though in the back of our minds we know oh geodesic slicing has this problem that coordinate lines can kind of focus together and it took a lot of careful analysis of time scale so we could were focusing on the timescales that they should based on the strength of the wave. We looked at things like the Riemann invariance that even as the code was crashing, they were sort of converging away to zero. So there wasn't any physics going on there. It had to be some coordinate thing. But this process could take literally, in this case, it took about 18 months before we were completely confident we understood the numerical evolution of weak waves, for which there's an analytic solution. There's two other main things that I wanted to ask you about. One, I guess, is I just wanted to kind of ask you about your personal history and how you got into numerical relativity in the first place. Okay. Well, so I started off, oh, I guess I'll just say, as an undergraduate, I was interested in physics and math. I didn't know much about astronomy, but I thought it was kind of interesting. Black holes, I thought, well, it'd be nice to work on black holes. But I didn't really, I never particularly thought I would actually be able to do that. I was at William & Mary. and I had a very tumultuous undergraduate career it took me six years to get my bachelor's degree because I I wasn't serious the first couple of years I failed out and I I quit I came to Europe hitchhiking I got a job in a hotel in Switzerland and sort of got my head together and realized okay

1:00:00 I really want to go back and study so eventually I went back and I did fine and then I I went to the University of Pennsylvania in graduate school, and at Penn, I was in physics, and I was, I got more and more interested in doing work in astrophysics, but there wasn't really much going on in astrophysics there. Paul Steinhardt just joined when I was there as an assistant professor, but he was doing really particle theory, a little cosmology, although he turned out to do a lot of cosmology. So in the end, I decided I wanted to do astronomy, So I transferred to Yale to the astronomy department and finished up my PhD there. But after about a year of being there, I realized that astronomers really are not physicists. And there was a clear cultural and sort of knowledge difference between the two. And I found I was really interested in the physics. So I went to the physics department and found Vince Moncrief, who was a mathematical relativist. And he had always had an interest in kind of astrophysical problems. So I got a problem doing perturbation theory, which is one reason I'm so fond of perturbation theory, but doing perturbation theory of supernovae and stellar collapse, non-spherical perturbations. It was a numerical project where the perturbation equations on a time-dependent spherical background are very complicated, like the Reggie Wheeler equation times 10, you know, many new variables and so forth. So there I got involved in doing numerical evolution of stellar collapse and the gravitational wave signature. okay so from there i went to washington university as a postdoc cliff will hired me there um and i worked some on black hole normal modes and uh and more perturbation theory kind of things and then i met wymo who had just joined as an assistant professor and then we started doing some numerical work on the collapse of self-gravitating scalar fields boson stars and so forth and then i went to uh to ncsa where david hobill was at that time kind of leading the numerical relativity group and larry smar but he was the director um and also david bernstein was there as a graduate student but that was it at least in that group and um that's when i started really getting into large scale numerical simulations of black holes and then two years after that i mean i really got into that i think it was a very good move for me career-wise obviously because that's now what i And then David left in two years, and I sort of at that point took over the group.

1:02:30 That was in 91, I guess. Yeah, 91. Is that right? Something like that. Yeah. And then I built it up from there, basically, really focusing on full numerical simulations. And that's when we really started getting into the two black hole collision and 3D work. And just being at NCSA made an incredible difference to me. because there was, of course, Larry. Did you talk with Larry at all? I did. Yeah? You know, Larry's a very inspiring person, very enthusiastic, sort of always telling you to go for it, you know, and that influenced me a lot in the way I approached things, I think, and just his support and enthusiasm for what I was doing and for what the group was doing really helped us kind of have a vision, I think, for where we wanted to go. And to get away from just, like, doing your next little problem, publishing a paper, but really getting a direction. It was also good timing that the computer technology was just exploding to the point where it was just impossible to even think about doing 3D work until about 1992, 1993. And so it was just there at the right time with somebody like Larry to really give you the kind of spark to go for it. And we did it, and we all got fired up. So, I mean, all of the things, I think I've just been very lucky that I've had these different influences. Like, I got my founding in perturbation theory from one of the masters, Moncrief. And although that was very mathematical, it had a numerical part to it. And now I realize how important it was to learn that perturbation theory, because it's becoming such an important part of numerical relativity, even now. And then meeting Wymo, we just really hit it off. And in fact, interestingly, he wasn't there the first year, because he stayed in Hong Kong. So we only overlapped for a period of less than a year while I was there, but we just then began collaborating. When I moved to Champaign, the collaboration actually intensified rather than weakening. And so we got used to this mode of working at a distance. And then we built up the groups at both places based on the kind of the fundamental idea that we were working together as one group. And so that, therefore, it was very natural for me to think, and when I come to Germany, so there's a time zone difference, but it's not that different. and telephone to work together so we've had in mind all along for almost 10 years now

1:05:00 that we were working together as a single group even though we're now separated by 5,000 miles Well that was more or less my next topic really the practicalities of organizing groups because one of the things that seems to have become clear both in the history work really that I was doing and starting looking at the current thing is that it is very difficult to organize collaborations between groups that are actually physically separated by a considerable distance. So you've touched on a number of possible ways of overcoming this, use of the web and so on. Yeah, the web's been really important in this. That was another thing. It was great to be at NCSA when Mosaic was being developed. I mean, who knew that it would be so big now? I mean, I think we knew it was really exciting, but we never thought that every company in the world, Even like some guy in Spain just with a little lean-to could put up a web page and suddenly he's got people from all over the world coming to his hotel. Now it's turned into a big hotel and so forth. Yeah, that's interesting. So, yeah, I'd forgotten first. I guess you guys actually saw it being born as it were. Yeah, we watched it from just this little neat toy that you thought, wow, you could really do something with this, and to this incredible multi-billion dollar industry, now with Microsoft and Netscape. In fact, the people who created Mosaic went off to found Netscape and went to work for Microsoft, so I know a lot of people who are multi-millionaires at this point who are making $6 an hour as poor undergraduate programmers just in 1993, or 94 even. Yeah, that's remarkable. so it had a lot of influence on us in the way we approached collaborations because of the web and the technology the use of the technology just being there at that time I think opened up a lot of possibilities like video conferencing we were always working with vendors because at NCSA it's kind of a special place the vendors want to come they want their computers to be there because then people think oh there must be good computers NCSA is using them so we worked with Apple we were always trying to get them to things, SGI. We were some of the first people to try to do video conferencing, both with Apple computers and with SGI. And if you talk to people like Karen, the way that the Carols are set up there, I guess you know, you know, did you go to Larry's office down the hall at NCSA? Okay. So I was in the same hall, but down at the other end of it. Then there's

1:07:30 this kind of Carol in the middle. And then across where Pete Aninos' office is now, that's where Joanne and Pete were. And so we had this whole kind of domain. And all the students So we were always testing out this video conferencing, and Karen was always in the middle hearing us say, Hello, Joanne, can you hear me? And Joanne was saying, No, I can't quite hear you, but of course if we can hear each other, you know, through the doors. But we were always doing this thinking we'll be able to do this, we'll be able to work with the people at WashU once this actually gets mature enough to use it. And we were always experimenting with these kind of whiteboards where you would, like, try to write an equation, you know, with your mouse so that Wymo could see I was running the Schrodinger equation or something because we got tired of trying to talk on the telephone, look at it together you know it was just and say okay don't forget to you know to square that or whatever when we could just use the technology to do it so we've had this idea of working with the developers of the of things like mosaic and other um collaborative tools that were developed like um collage was something you might not have heard of but it was a sort of a a forerunner of a lot of the the whiteboard kind of technology that you see now where people can write on something and it shows up on another computer we were doing this experimenting with the wash 1990 or something like that so so other um is the video conferencing at a stage now over it no it's still like my impression it's not really i mean if you have a dedicated line then it can work but it only works there if you really know the other people because otherwise you just get bored you see these faces and the sounds not that great and you know you lose interest really but if you know the people so like the eye contact is really important to really make you know to make that connection. And if you know the people and you can see them, then I found it worked much better. But most of the video conferences I've been to just end up being painfully boring. But we keep trying it. We haven't done as much of it as I expected with Wash U. I found that the time difference is much more important than I had thought. I never thought that part of it through. But whenever I want to call somebody over there, it's usually inconvenient. Either they're asleep or I'm at home or vice versa. And the phone bills are more expensive. We're lucky here at the moment, our phone bills are not really constrained, so I can call them. But they're much more dependent on grants for phone calls at WashU. And in fact, most American groups, the phone bills can really become important. So they don't call us very

1:10:00 much. And it would be better if that weren't a problem. Yeah, my wife, my wife's family lives in Arkansas, so we're familiar with the six-hour time difference, I guess it's seven in your case, can be quite inconvenient. Yeah, it really can. It's not necessarily a good step, because sort of by the time it's the end of the day in one place, it's the beginning of the day in the other, and you're always... Yeah, or very often, like, I'll be at the stage where I really want to talk to, say, WIMO about something, but it's 6 o'clock in the evening, and I just really have to get home, and I'm tired, and I know my wife wants me to come home, because dinner's going to be ready in a little while, or we're going to go out to dinner, or whatever, so I don't do it, and then it builds up the next day, and a couple days later, and then say, oh, well, you know, it's not that important, you know, whereas, of course, at NCSA, I would just pick up the phone anytime, because we're on the same time schedule. So those little things have been more important than I wish they were. I think maybe Pablo Laguna mentioned that one of the things that he found useful with the web, although I think he complained that he had to spend a lot more time than he would have liked developing web pages for the Grand Challenge Group. But he said one of the ways it was useful was in connection with phone calls, getting around the problem that you mentioned of saying, well look at this equation it's at this URL, do you find that you use it in that way? Oh yeah, we do that the project pages this is another thing we developed when Mosaic was still coming out before Netscape and Internet Explorer were available this is another thing the genius of Paul and Joanne when it comes to computer things they dreamed up the idea of using the web for these collaborative of um reporting pages where you could sort of attach all kinds of media that you normally have in your numerical simulations so it's particularly good for numerical things we can put an image like that or you know a postscript file gif file whatever to go along with a report of something you've done um so anyway they set that up uh and then it's evolved a lot over the last three or four years but the point is that um without it we really wouldn't be able to collaborate very well you, or even within the group. You know, this is a very big group, and there's a big distance between us, and a lot of us are busy a lot of the time with something or other, and having these things automatically be reflected out to a number of people who are interested in their particular

1:12:30 results, so you send it to the black hole page, and all the black holers get a result that they can see. Waymo can send a note, or Malcolm Tobias, or Pete Aninos, or whoever we're working with, Karen or maybe Pablo, in fact, probably start working more with Pablo. Without it, there's no way we could work as effectively as we do. And you have to keep forcing people to do it. I have to keep reminding them, you've got to send something to their project page. So I have a rule that all the students have to at least send something every week, even if they have no progress, something they think is no progress. They can at least say, I worked really hard this week on this thing. I couldn't really solve the problem. But maybe somebody Or, if they don't hear anything, then the people in St. Louis think, oh, I guess the Potsdam guys just lost interest in this. But they don't realize that the people are slaving away on it. So even if you just say, I'm still working on it, every week, I'm still working on it, then it makes a big difference in what the attitude and the perception is from the other side. Has there been much movement of students and postdocs between your group and Wymos group? we have yeah um and and so for example this summer i went to ncsa for two months and karen kamarda who was a student here at that time i mean she's a student at illinois but finishing her degree here um came for a month and tony font who's a postdoc here in hydro came for a month um i guess that was it but so we worked a lot with pita ninos and john shalf i don't know if talk with John at NCSA and Doug. And then, of course, I went to St. Louis a lot, and YMO came up to St. Louis a lot at that time. And then Malcolm Tobias, one of YMO's students and now a postdoc, came here for a month last year. Mark Miller came here for a couple of weeks. I've gone there four or five times in the last year, just to St. Louis, I mean, just for a few days even, just to make reestablished contact. Greg Dawes came. He was a student I was now a postdoc at NCSA, came here for a month. Pete's come here for a two-month period, so a total of two months. Yeah, so without that, the personal connection is really, really important. You mentioned about Paul, for instance, might decide to leave. Has the difficulties of postdocs finding other jobs been a problem for people staying in the field,

1:15:00 available? Well, I think the market's pretty tight, but we've been pretty successful in getting people jobs who want to stay in the field. Either I've been able to keep people on as postdocs if they couldn't find a job, or if they've been able to go out and get jobs. So I guess I have not had anybody who couldn't find a job that they wanted so far. So I felt really happy about that. I'm Our students really coming through have all gotten good jobs, and so I'm very happy with that. It's been very sad to see them go, because they've all been such good students and postdocs, you know, but it's just the way life goes. They've got to go out and do their own thing. Sure, well, that's great if they get to go out and work some things. That's because I met people who were saying that they were worried that if, you know, at the end of the Grand Challenge, then it seems likely that the level of funding for numerical relativity in the U.S. would drop significantly, so that might be sort of a problem for people. Well, yeah, that's a very real possibility. And just because people have always managed to get jobs doesn't mean that they've been very worried that they wouldn't until it finally came through. So, yeah. That was an issue. Yeah, it is always an issue. I think that's not the main issue for Paul. I think it has a lot to do with these other things that happened. No, well, yeah. In his case, he clearly would have had the ability to go on. Yeah, right. Just to, well, go back to what you were saying about some of your personal history in the field, as it tied in then to my main topic of interest gravitational waves, you were discussing the influences of the advances in computing power and, well, personal details such as going to NCSA at the right time and so on, which had led to big developments in the market of relativity. How much of an influence has LIGO and some of the projects like that been, in your experience? I mean, in focusing attention on certain problems that are related to the issues of gravitational waves, like, to say, the binary black hole. Yeah, okay, that's interesting. Well, I remember when I was a graduate student at Yale,

1:17:30 Kip came out one time to give a talk. It was probably 1984, I guess, something like that. And he was very enthusiastic about LIGO and thinking, in fact, he was just going off to Congress or something and he was going to talk. And he was very confident that they'd be building wave detectors within two years or something like that. So I was really excited. Wow, this is going to be great, detect gravitational waves. year after year where people were very enthusiastic, but nothing ever really came of it. The funding never seemed to be there. So it began to get very skeptical, as many people did, whether it would ever actually happen. And it came to a boil even, I guess, with the Bacall Report or whatever it was in the early 90s. LIGO didn't even make it. This is like priorities for astronomy for the next decade. It wasn't even mentioned, you know. And then there was the real possibility of funding but then there was this congressional hearing where people like Tony Tyson were saying I think it won't work and he had a lot of credibility he had a lot of experience with this stuff and then so Cliff Will mounted this basically this write-in campaign to the Congress and Senate and I think that was important so I'm trying to create the backdrop for this constant excitement about it but always feeling let down and then in the end, not thinking that it was really going to have that much of an impact on my own research. So I was interested in doing gravitational waves, but I kind of lost the feeling that these would actually be detected anytime soon. But in the last couple of years, it's obviously being built. I've seen pictures of it. I saw Kip, in fact, the day he came back from the groundbreaking ceremony in Washington, and we were at a conference in Snowmass, had an apartment just next to Kip, where we talked a lot about the Black Hole Grand Challenge and this Event Horizon paper and all of this stuff, but we also talked a lot about LIGO. And Kip came back from that meeting and was telling me how thrilling it was to see. You know, they had actually dug a ditch, and almost, he didn't, I wouldn't say he cried, but he was emotional, you know, he was clearly so happy. And that kind of had an impact on me. And since then, and with many more meetings with Kip, I've gone to MIT, I met Ray Weiss and some people there last year and gave a talk there. I've become much more focused on things that actually are connected to LIGO and thinking about waveforms particularly. So in the last

1:20:00 year I've been thinking a lot about how much can you actually detect I mean compute waveforms accurately in a 3D numerical simulation and that's been driven a lot by the need to actually see waveforms in LIGO and to make predictions from numerical relativity that are actually relevant to LIGO. So anyway, I've had these sort of ups and downs, and in the last couple of years I've become much more focused on waveforms and the connection with LIGO. And so there's a project, the GEO project you know about, probably in Hanover, right? And I would like to take a field trip with the people in the group here and take them and show them the thing, because I think it might have a similar impact on them, just psychologically, to start thinking much more Relativity, or even whatever relativity they're doing, that's how it's related to gravitational wave astronomy. So in the end, I think it is having a lot of impact on me personally, and I think it has a lot to do with just seeing it, seeing the reality of it. And so I hope that when we take people there, we're going to just all go on a bus and see it. I think maybe it'll have an impact on what people choose to do on a daily basis. Should I do this computer thing or just the same old computer code? When am I ever going to get any real physics out of it? They see LIGO, or they see GEO, and they'll think, well, I could really do something that's related to that, and it might sort of give them some enthusiasm or direction. So for you, it's really only in the last couple of years when they really started to, as you say, dig earth in that. Yeah, it was more intangible. I'm not saying it wasn't there, but now it's making me think more on a daily basis. I don't do as many simulations anymore. I'm really doing more cheerleading and helping people write papers like ideas or directions sometimes but it's helping me think about directions that the group needs to be taking that are related to that and I hope that when people are getting tired of their computer code they can think about something like they're actually going to detect these waves and help get somebody through the day because you know research can be very depressing days on end when no results and nothing works you know so you've got to have something I mean of course you have a lot of days when things go right but you have to have something like goal that makes you remember that it's really worthwhile yeah it's interesting to get because even even in kids group where we more or less tended to live and breathe

1:22:30 it's true that it's only in the last two or three years that things have really gotten down to it's true that in the last two or three years things have become much more focused LIGO requirements, you know, I mean, as a general people were it was it was out there, but it's only somehow in the last two or three years, maybe now that I think about it, that it's really become you're thinking specifically in terms of what can I do for LIGO today Yeah, really, that's really true and the papers by Scott Hughes and Anna Flanagan have been really important, at least for the people in this group to see that these guys are really thinking what black hole collisions can do for LIGO as a source and what the need is for numerical relativity. And so we brought Scott here. In fact, I'm kind of sorry Scott isn't interested in coming here because I thought he might be able to help make that connection even more tangible for people if he were to come here as a postdoc. He's going to Illinois, actually. Yeah, I got the time. But anyway, he did come here. Unfortunately, during the week that I had arranged to go to California, so we both sort of switched places. I didn't actually see him here. But I heard that he actually had a very positive impact on the people in the group here just because he gave a talk. And other people have seen these papers, or these massive papers that nobody has really the strength to read all the way, but there's just so chock full of information. But he was able to distill that in kind of an exciting way to people here that it apparently really did have the kind of impact that I hoped that it would, that they're thinking much more about it. So at least Scott's visit, and to a lesser extent the Flanagan Hughes paper, does actually kind of inspire people here and kind of set some sort of an agenda of what's useful for a lot of others. Yeah, there really has. Well, I've more or less run out of the questions that I've thought up and it's about time to let you get down to other things since we've managed to go on for quite a while Thanks