Interview with Alan Wiseman

0:00 Okay, so it's okay, and it's quarter past three, and 9th of February, 2000, first interview of 2000, and I'm speaking with Alan Wiseman, and I guess we were talking about these issues of things like software in a large collaboration. You were telling me a little bit about how you're on this LIGO Science Collaboration software coordinating committee. Is that the full title? Yeah, I think that is the title. Well, to backtrack a little bit on that, in the bylaws of the LIGO scientific collaboration which is something referred to as the white paper. They outline a number of positions of power. One is the software, or one of the positions is the what's called the LSC spokesperson which is Ray Wise who I guess is in essence in charge. Then there are some committee chairs of the data analysis subgroups. Those Keith Riles, Bruce Allen, and actually Albert Lazzarini, who was also employed by the lab, heads up another one. And the other position is the software coordinator. However, there is not a software coordinator at present, and this has created a little bit of friction, but rather than, frankly, because nobody is that thrilled about getting the But they appointed a committee to handle the tasks of the software coordinator, and I was asked to serve on the committee. So the basic idea is to make sure that the software written by various people and groups for the purposes of data analysis will actually work together and with, say, the data formats and so on produced by LIGO. Yeah, there's clearly a very long pipeline between what comes gushing out of the instrument and down at the very nitty-gritty level of looking for some very specific astrophysical event

2:30 or some very minute instrument glitch or event. And that's what the committee's for. We'll see that those all hang together. So, I mean, it's not just, I presume, an aspect, a thing that what people are writing are little things to search for stuff. Obviously, it's not just that people are writing little bits of software that's going to search for something that they're just interested in. And the key problem, I presume, is that some of the groups, like for instance maybe this one, are contracted to write the key bits of software that are actually going to search for the most likely type of signals. Is that? Well, yes, but it certainly has to hang that this group has agreed to write a search code for what they call a hierarchical search for binaries. And actually, they're doing this in collaboration with the Cardiff Group, and it's part of the MOU that they have with the Pligo Project. But there's lots of stuff that will go upstream from that in the data pipeline. Clearly, this type of code, maybe it would run at a computer at the site or maybe it would run off-site or something like that. But there's exactly what the data is going to look like when it gets into it. To what extent will it be conditioned data or some subset of the data. And, you know, so there's a lot of people who are writing software, which is in some sense upstream from the binary search codes that they're writing. And then, of course, there's also compatibility issues about whether or not the software that is being written by this group will actually interface effectively with it. And then the other thing is, the other issue that has not really arisen yet because there is not a coherent package in any given area, say in binary and spiral or instrument diagnostics, that the other thing that ultimately this committee will be charged with

5:00 is the validation of the software, which, of course, consists of, first of all, just checking that individual modules are debugged and pretty much do what the user says they do. just the full integration of whether or not these things actually hook together correctly, whether or not everybody's using meters, kilograms, and seconds, and somebody else's code is in centimeters or something. Make sure that those types of issues are ironed out, too. Actually, those we try to iron out in advance. Right, so yeah, that's a lot. So, for instance, since the Cardiff Group and the Milwaukee Group are collaborating on hierarchical search codes, is it just sort of up to them to figure out how to cooperate between themselves, or is that part of the committee? At the last meeting of the LIGO Scientific Collaboration, Bruce Allen, who's the chair for one of the data analysis subgroups, what is called ACES, which is Astrophysical Source identification and signature group, we kind of began with a list of areas and tasks and really started to, in essence, seek out volunteers to put their name down and to really commit to actually produce the code. And these are pretty well, these are, you know, on the ACES web page, you can see exactly what people are committed to. And it's, to some, at least for significant parts of the entire pipeline from the data to the final outcome, at least the parts that deal with actually writing the code that will do the astrophysical analysis. Most of the bullet points are filled in, that most of them have somebody's name attached to them.

7:30 As I say, this group is working in collaboration with Cardiff, and there was some very specified division of labor, exactly what Cardiff would produce, exactly what this group would produce. That was laid out fairly explicitly, and they've really made a genuine attempt to meet their They laid this out in advance where they would be at certain dates and have pretty much tried to meet that and I think have been fairly successful at it. And so that should produce something that more or less, at least for a significant segment of, at least for a significant part of the data analysis, that ought to hang again there. So I think they're in pretty good shape there. It just occurred to me, since you were mentioning about ACES, and I guess there are different sort of groups and divisions within the MSC. I was just curious as to, obviously somebody beforehand, I suppose, had to come up with some sort of scheme for dividing things up. And, you know, I was wondering how that kind of stands up. I mean, as it... The way the project seems to be organized to me, anyway, this is from my point of view at the bottom of the food chain, is that there is a pretty high-level group of people in charge. I mean, Barry Barish and Ray Weiss, who are very experienced at this kind of stuff, that they've both been involved in big collaborations, and they've been around the block. And so I think everybody's reasonably happy with them. But then they're also not in the position to probably aren't going to sit down and write a thousand lines of C code to do something, he could do some detailed calculations, and so that sort of gets pushed down, and so there are a group of people who are not junior, but who are certainly not as senior as Barry and Ray, and that consists of people like Bruce Allen and Sam Finn and Keith Riles, and these are the three guys who initially started out being in charge of the data analysis

10:00 Bruce Allen is in charge of the ACES, which is Astrophysical Source Identification and Signature. And then also Keith Riles, who has a background in high-energy physics and seems to be a really competent guy, is in charge of the detector characterization. Now, he, of course, wears a couple of hats because he also does some work, Keith with Dick Gustafson do some real, no kidding, development of hardware. So they're interested in detector characterization because they also work on detectors. And the third group is the DCSA, which is Detection, Confidence, and Statistical Analysis. And originally Sam Finn was in charge of that. That I don't think ever had a very high membership. People did not take to that very well. And eventually I think Sam stepped down. I'm not certain exactly what his intention was. I think he really wanted us to dissolve the subgroup. But currently Albert Lazzarini, who also is not only in the scientific collaboration, in the lab, I think, has actually taken that up, too, is now the chairman of that. And I really don't know how healthy that subgroup is. I know that Albert is spread really thin with lots of other commitments at the lab, so I'm not so sure how actively he's working on that. So basically, there was a certain division to begin with, and if I understand you correctly, that people sort of voted, people within the collaboration kind of voted with their feet as to which groups that some groups they wanted to be in. Yeah, that's exactly right. I mean, these were formed at one of the earliest LSC meetings, namely one at Hanford, Washington, and I guess this would have been probably, And, you know, people went to, just broke up into the groups.

12:30 There were certainly lots of instrument-based groups and people involved in suspension systems and things like that. But, yeah, I think that it was pretty clear from the beginning that the astrophysical source identification and signature group would be pretty healthy It's the one that astrophysicists, theorists, relativists themselves are probably the most interested in, and it's certainly how they fit in most effectively into the collaboration. So that's why it was fairly healthy. And, of course, there's just no shortage of work to be done in detector characterization. In fact, I think they actually have a fairly healthy and active membership, but their task list is just a hell of a lot longer than they've got people to do it. I don't know what the overall health of that organization is, but if there's bad morale, it's probably because people just plain old feel swamped. by that Yeah. So is there a split? I mean, is it mostly theorists going into ACES and then more experimental or different type of non-relativity people, for instance, non-astrophysical people going into detective characterization? I would say that's probably a pretty accurate characterization. But that split may begin to break down. Within the LSC, we have begun to circulate a proposal to do some research. This is one of the things that is outlined in the white paper. to ensure that people within the collaboration can actually have access to work on various projects. Things are out in the open. You can also estimate in advance what the impact of any given research is going to be on lab resources. We have these proposals, and to do some very particular task,

15:00 the one that originally started in this group, although it's grown considerably, One, to do a mock coincidence study, to take old 40-meter data, split it into two data segments, and pretend that the data comes from two detectors. But not only to do it that way, we could just run that code on our Beowulf downstairs and could do that. But actually to use this to do some of the analysis, as much of the analysis as possible in the LDAS, that's the LIGO data acquisition system or data analysis system, and to try to exercise some of that. And so at an early stage of circulating this proposal, it was suggested to me that I broaden out the list of proposers. And so I asked quite a few people in the lab, Albert, would he be interested? And did he have any postdocs or people who were working with him? And he said, sure. And so eventually we did. Of course, this is also, in hindsight, this is an incredibly wise thing to do because if you run into any political problems down the road, having a broad base of people is, you know, somebody in that collaboration has got to know everybody. And you have a strong list of proposers that you actually do have some expertise. And so if an issue arises, you do have somebody who can speak to it very eloquently. And where, if I had just tried to ramrod the thing through myself, somebody might have asked me about, you know, interfacing detector characterization with some kind of database hardware. I wouldn't have known virtually nothing about that. And so it was actually quite helpful to me and had a bigger group to work with. So I was, in some sense, kind of reluctant at the output when that was suggested to me. Let's just keep this small and in-house. But in hindsight, I'm glad I paid on that issue because it came in handy.

17:30 And is there a real-world price to be paid as work for that? I mean, does it then become actually unwieldy and difficult to coordinate the number of people involved? I'll let you know. I think this one is small enough that probably most of the work will actually be done by four or five, probably less than ten, but there will be 15 or 20 people to take the credit. And so, yeah, I'd say that's – since we've really just pushed this through the proposal process, it's actually – actually, I'm going to email Ray Weiss and ask him what the status of this thing is, that I think we've jumped through all the hoops in order to even start it. And so I think we should be able to get this thing kicked off in the next few weeks anyway. And I'd like to because part of this is to actually come up with some kind of an interface between astrophysical code. You search it, you know, take a stream of data, search it for an astrophysical event, your filter bank goes ping, yes, we think this is it, and stuff those kinds of events into a database, but also stuff instrument, you know, similarly instrument events and stuff into a database, and then come up with some way to query the database, you know, for what do you think are the really solid candidates for an actual event. And I think that is something we've got to do. And when we first proposed this, I think it sort of came out of the 40-meter analysis that we did that virtually every time we gave a talk on this, somebody, Ray or somebody, would say, you do split the data in two and see how much you gain by having two detectors because you have all these spurious events but clearly they're just that they're spurious events and if you had another detector to overrule and say say well we didn't see anything here and that our detector was working

20:00 fine how much better you do and and that seemed and so in some sense we just kind of thought well that you know that would just do that and but we were encouraged um at various stages to broaden that to really not to do that just in-house but to broaden that to exercise part of the LIGO data acquisition system the database system and stuff and uh and uh you know eventually eventually that That was the proposal we wrote, and I think we also, that this came about sort of at the time that this system of internal proposals was being laid down on the white paper, and so we just decided we'd go ahead and do this, do it that way. And when I began this process last September of submitting this proposal to the powers the side, they determined, as I say, they were sort of making up the rules as we went along because this was the first one to come through the pipeline and stuff. And also, one of the primary reasons for having these proposals is impact on lab resources. They have limited manpower at the sites, at Caltech, and so if something's going to take an exorbitant amount of their effort, they don't want to have anything to do with it. uh albert i think in principle he thought it was a pretty good idea in practice he just didn't know where he was going to be with his hardware and uh so he was kind of well it sounds like a good idea but uh i'm not sure i can commit to it and uh so the the fact that it dragged out into this proposal process actually turned out i think in hindsight to be a good to be a good thing we'll when the thing actually comes about, whether or not it was a good thing. But it gave him about four or five months of breathing room, and so they really have received a lot of their hardware. They've actually installed it. They actually do have something that resembles a database, apparently, and they've got programmers working on it in some sense. And now he's actually one of the strongest proponents of this proposal because he sees it as, yes, this is really live ammunition testing

22:30 all this hardware that we've built and now we have programmers who are stuffing random numbers into this database and querying for random numbers. It will be interesting to see whether this thing really holds up when you stuff astrophysical events or stuff real instrument events into it and whether or not it works or whether or not the interface is suitable for users in the field to actually use, things like that. When you mention the powers that be that you put the proposal to, is that the heads of the LSE or the lab? This is yet another committee. No, it's a pretty wisely chosen bunch, that the people who would rule on these proposals officially is the laboratory directorate, which probably, well, I guess officially would be Barry, but probably as operationally might be Ken Blackburner and Albert Lazzarini. The LICO or the LSC spokesperson, which is Ray Weiss, the data analysis committee chairs, which consist of Bruce Allen, Keith Riles, and now Albert wears this other hat of being one of those and the software coordinator, which in effect we now, instead of a software coordinator we have a committee, but in effect the chairman of that committee will cast any vote. Now they've got to come up with how to decide whether or not a proposal is, I don't want to say funded because they don't give you any money. The only thing they have to give away is some manpower. And I think in this case that it's fairly well received. I mean, in some sense, we got encouragement from above to actually pursue this thing. And so that's what... And ultimately, I suppose what they have to give is data. But in this case... In this case, the only data we've asked for is old data. Yeah, the same data that you used before.

25:00 The same data that we used before. This is now more of a scientific issue, but that idea has been the fact that we're using old data, that has its pros and its cons, and people are certainly willing to point these out to you, particularly the cons. But you might say, well, gee, they still operate the 40-meter, why not use new 40-meter data? Well, in the process of building this collaboration to do this, I enlisted Fred Robb, who knows a lot about 40-meter data and knows a lot about new data, and his claim was that the old 40-meter data is most like what they're going to get in the future. That, you know, they've reworked the 40-meter instrument now, and so it is not much like the data they actually expect. And so, you know, so when people raise this objection to the proposal, I could easily, you know, well, this is what Fred, you know, Fred has spoken to this issue and decided that this is the thing to do. The other thing is, and this often gets lost just in my own conversations, is that the core of why we do this is that it really comes back to comparing one instrument to two instruments. If you were to have one instrument that had this sensitivity, how could you do? And I suppose you had two instruments of similar characteristics and similar sensitivity, how could you do? Well, if somebody wanted to do this same analysis with new data, the first thing they'd have to do is the old analysis that we've already done. We've already analyzed the 40-meter data with one detector, thinking of it as a single detector. And so the first thing, if you were to do this thing from scratch, the first thing you'd have to do is redo that project. And so this makes it a compelling reason to stick with exactly what you did before. And there's also, you know, to what extent we might want to rewrite software or something to an analysis code to search through the thing. Well, on the one hand, we have plenty of commitments here at UWM to write software for lots of things.

27:30 This is not one of them. And besides which, you'd really like to make the best apples-to-apples comparison you can. And so you'd like to use the same old software to the extent you can to do this other project. But you'd really rather not, this is a comparison of one detector to two detectors. It's not new software and old software. It's not the comparison. So that's one of the reasons we've stuck with this. But, you know, there's always plenty of suggestions about what you should do. And we kind of stuck to our guns. This is what we're going to do. There's lots of other projects that one could do. And there's also, right now, and this is politically even much more complicated than anything in the LSC, that they arranged to run the refurbished 40-meter last summer in coincidence with the TAMA detector. And so they actually do have coincident data. Well, so why not do a data analysis? Why not do some kind of an analysis where you examine these two data streams? This is no kidding coincident data. Well, that's a pretty good idea, and there is an internal collaboration being formulated to do that. But all kinds of political problems there, because now you've got to coordinate not only between LSC, the L and LSC stands for LIGO, so everything falls under that umbrella, but now you've got to coordinate with the TAMA researchers, and the LSC is huge, has hundreds of members. You could easily see that it could end up swamping the relatively small number of researchers that the Japanese have working on TAMA. So here in this group, we just decided not to do that. The other reason is that in that case, not only do you have to do the analysis that we have already done before on old 40-meter data, Not only do you have to do that, you've got to do it twice, and you have to learn about two instruments. The other thing is, and again, Fred Robb pointed this out to me.

30:00 He said the real question that you want to answer, or there's two questions you want to answer, and the proposal that we put forward only answers one, but where you are analyzing two segments of data that come from similar detectors, that in the case of the 40-meter detector and atomic detector, quite different instruments, and so how you're going to analyze data and the conclusions you can draw from that are probably a lot different than the case where the detectors are very similar. I mean, suppose you're doing stochastic background detection, people will do that kind of analysis not only including interferometers, but they'll also include bars, bar detectors in their data analysis. So that is a type of analysis. But here we'd like to do a LIGO-to-LIGO type analysis. So, again, kind of stick to this proposal. Somebody or we or whoever may write a proposal and do some of these other things. As I say, there is some collaboration being formulated to do this analysis with the tunnel. projects that's that's going to come about one way or the other I was kind of curious actually about this issue having real data and becoming familiar with what's in it and what it means and presumably learning about that from the actual experimenters people working on the instrument and what the experience of that was like I guess in the case of this 40 meter data that you probably other people could speak to that much more effectively than I could probably when you go back to Europe you probably ought to interview Bruce about that because he would be much more knowledgeable than I would in the end I just I'll just speak in very general terms the one thing that I have noticed, and this is something I think experimenters would tell you always, probably would have said all along, that there, you and I have been to a thousand meetings where we have talked, heard people talk about we can see coalescing binaries out to a gazillion

32:30 gigaparsecs and this kind of event or something, and it is all wrong. Those are always based on the most pristine noise models. Oftentimes, just pure white noise is all they've ever done. Because all they've ever done is called gas dev in numerical recipes to generate, give them a noise floor. And probably don't even bother to color that, to give it any kind of spectral feature or anything. And so, and then other times people have done it with somewhat more advanced methods of actually considering what the shape of the noise curve is. It's all irrelevant, that I, you know, maybe, it probably gives you an order of magnitude estimate of how big a template bank you're going to need, but as real analysis, the thing in the case of the 40 meter was that the only thing you could really count on was taking the noise stream you had, injecting signals into it, and see how effective you were at getting those signals back out. That was really the only way to give yourself any confidence that what you were doing was correct because every few minutes something goes off in the detector that lights up every filter in the bank, and so it's awfully hard to distinguish these things. So it really, it also, in the case of the original 40 meter analysis, we came up with lots of discriminants and we could estimate our efficiency by doing Monte Carlo simulations of injecting signals and doing that. But the one thing, ultimately the statistical analysis was based on the loudest event that passed all our discriminants. And it was reassuring because in the 40 meter they did record other channels. What was the magnetometer doing? What was the seismometer doing? What were the environment variables going on? And so lots of times you could take some loud event

35:00 and perhaps one of our discriminating techniques would eliminate it because it didn't look like a chirp from a coalescing minor, so you could get rid of it. But in the one event upon which the statistical analysis is all based, you can look at the other channels for how the experiment was actually running and they were relatively quiescent, which is somewhat reassuring that nobody was stomping on the floor and stuff. And the interpretation of the information that was embedded in those channels, the experimentalists had plenty to say about that kind of stuff and were actually pretty knowledgeable. In fact, probably they were a lot more interested in that than they were in the astrophysical analysis, which was sort of what got us into the problem in the first place. well I was struck actually in Japan at this TAMO workshop talking to say Hideyuki who Takashi is working on he's their dad yeah he's their LSC he's their scientific collaboration well I think it's Hideyuki Takahiro and I think they have some other guys them is those guys are supremely confident. I don't, you know, if anybody can pull it off, they can. And so that was sort of an issue that came up. This group here has stayed out of this, but that was an issue that came up with doing this 40-meter trauma analysis you know, you have three, you know, like showing up with 300 guests for dinner. Okay, you know, me and my 300 friends would like to collaborate with you three. Yeah, so there was a lot of sensitivity to that. And that's way above my paycheck to worry about. We just bit off what we thought we could chew, and we're sticking with it, and somebody else, we're completely comfortable to let others work on something else.

37:30 Yeah, that was true at Kiriyuki at one point, said to me, well, this sort of, I don't know, not exactly a glum look, but anyway, it's so non-Gaussian, referring to the noise of Tama, and that was certainly striking. So, do you think that, I mean, clearly, you know, I suppose not to fault the theorists too much to begin, you know, before they saw any real data, I suppose they didn't have any experience to go on in terms of making some noise to look at. and you know if you figure that experience then is what's needed and obviously the theorists start to get it if they actually see some real noise but is it enough to actually just see the real noise and talk to the experimentalists occasionally or do you think there would be a big problem with the people doing the data analysis being people who haven't actually worked on the instrument no I think that's why you gotta have the data analysis has to become coherent, and oh, how they do love to use the jargon words, such as, you know, end-to-end testing. You see, that's the kind of mock data challenges. Boy, we have our own vernacular here that people use, but it's much more important now to do that. I guess I wasn't at the last GW DAW, but I was struck by this at the one in Penn State and in Paris, the one before that, where we were beginning to get involved with real data at that time. I just really thought a lot of this work is irrelevant, that the world has actually just moved past that. You know, your conclusions based on white Gaussian noise just aren't that interesting to me anymore because I know in the end they will be largely irrelevant. Oftentimes I guess the real role that they fulfill is that oftentimes in the case of pure Gaussian noise you can often make it do a definitive calculation to show how good a

40:00 technique is statistically and so you can say um you know in if this was pure Gaussian noise this is a good technique and uh and that does give you a certain you know it is something to hang your hat on and you know lord knows it wasn't any good there it's not going to be for shit in the real case but so it you know or maybe it allows you to compare one method with another but I don't know I just I know sort of reluctantly have just gotten away from thinking about that and in the end, the noise is going to be what the noise is. And let's prepare to deal with it. Prepare both emotionally and... And what is the best way to deal with it? I mean, if you have this high non-Gaussian noise, is it actually going to be possible to convince everyone that you're really seeing a signal? and what what have you got to do I mean is it enough just to have it something go thing in both detectors at once there's obviously coincidence is a big thing that would be that's one really good issue that's one really good way to resolve things boy triple coincidence would be another Boy, if you had Virgo or Tama or Geo or something else online, besides two LIGO detectors, that would be good. Something, or better yet, coincident with an astrophysical event like a supernova. Now you're talking. Okay. Okay. But, I remember, I'm now old enough that I can actually reminisce, and that when I went to the GR meeting in Argentina, this was, although many times I've actually had listen to Nobel Prize winners speak or have them give a talk.

42:30 I think Joe Taylor is the only Nobel Prize winner who's ever been to one of my talks. And actually, now that I think about it, I can't remember where this transpired. Actually, wait a minute. This story transpired some years later at the Texas meeting in Munich. But somebody was posing the question, you know, gee, how do you know when you've seen something? And Joe Taylor said, well, when it comes to interferometers that are broadband, I'd sure be impressed if I saw one of Alan's chirps because these are things of reasonable duration that have a very prescribed evolution through them. And so there are tests that you can do to discriminate whether or not that's what you've seen. And that's one of the things we use in the 40-meter paper, that not only does the thing have a lot of power, that it pings the filter, but the power has to add up just right. The first part has to give, and so you basically, how does the power accumulate over time? And so that's another discriminant, and that works reasonably well even in the presence of, in fact, what that does is that allows you to discriminate against a lot of non-Gaussian events. The thing went ba-boom, ba-boom, ba-boom, the detector did. let the record reflect that I raised my hand. The detector can discriminate against, or this type of method can actually discriminate against that because whereas the detector may have oscillated many times, you know, its RMS value for a very short period of time, time, that can give a tremendous amount of power when you beat that against one of your filters, but the power doesn't accumulate over time in the proper form. So that's one of the ways to discriminate, and that's one of the things we use to do

45:00 that. Yeah, so it's nice that it's such a distinctive kind of signal here. Yeah. So in the case of that 40 meter, that analysis, you had some of the people, some of the 40 meter people who were involved in sort of pointing out where... Well, they also played no small role in this, that they took the data. That the author list is pretty large, and some great effort, I think people went to great effort to certainly to offer that even people who were long gone from the project could be authors on that paper because they actually took the data. Now, as for specific interactions with people about what the hell was really going on, that was largely people who are still involved with the project but who were also involved with it back then, namely Fred Robb. It was primarily Fred Robb's graduate student at the time, who you know well who actually took the data so Fred was pretty influential let's see and also Stan Whitcomb who's been around the block too on this but these are the people who were consulted in detail about how do you really go about dealing with these problems yeah I guess I was going to wonder quite a reasonable amount of turnover with the 40-meter people, but basically there were enough who were still on the project that you would mostly interact with them. Yeah, I think, and had quite a few email exchanges with, I can't think of his name, Mr. 40-meter himself but he's now at JPL I'm drawing a back to Bob Sparrow

47:30 yeah but I think in the end he declined authorship on he was on the paper but actually yeah he was you know as things were aired and he probably I don't recall this specifically but it wouldn't surprise me if he participated in some of the phone conferences and stuff to discuss how to go about actually doing the analysis yeah so he was fairly involved with that okay so I guess just to go back to this software coordinating issue, which we've moved away from. I'm still trying to get it straight in my head, what the issues were there. There just seemed to be so many of them, obviously. Well, one thing has got to come up with some, one of the things that's difficult about the task to this committee is that in some sense it really does span the entire gamut of the problem. that, you know, this morning, you know, we had these, you know, that means the bell that you hear in the background is, that means it's tea time, it's department tea time. Actually, yeah, can we suspend for a little bit? Okay, so we're back. And, well, I don't know exactly where we were, but the one thing that occurred to me to ask, going to sort of came up, maybe a little bit during the tea, right, is how you compare, how it compares for you in terms of enjoyment, say, personal satisfaction, what you do now with what you, you know, used to do, insofar as your...

50:00 Um, I guess I don't have a good answer for that. I don't even have an answer for that, let alone a good one. that's a bad question that's a bad question no answer at all I'm honestly not sure this really goes this is more about me than about anything on the one hand I guess I'm torn i do like uh you know working in my office alone close the door that's it you know that's something that i've kind of enjoyed i guess if you don't like that you probably shouldn't have gone into theoretical physics in the first place but uh on the other hand i guess i've always sort of enjoyed thinking about institutional dynamics and and and things political so uh so i guess i and to some participating and you know I'm pretty low on the food chain here but you know to the extent that I do participate in this stuff I guess I kind of enjoy that too. I don't know but then then you then eventually it will eventually reach the stress level where there's not enough time in the day to do both and that could be more problematic. Then I think about that could force the issue, and I'm not exactly sure where to go from there, but for the time being I'm pretty, pretty content I guess. Because I was just able to combine both, so they're… Yeah, yeah, I haven't had to choose. I wrote an NSF proposal in the fall to, you know, it's unclear whether it would be funded And in that, there is such a heavy emphasis around here on LIGO data analysis that Bruce is totally committed to that, Patrick is largely committed to that, many of the post-docs and graduate students around here are heavily involved with that.

52:30 In some sense, to distinguish myself, I was even, from that effort, I was actually encouraged to make it maybe even somewhat less LIGO intensive than I might have otherwise have done. But then again, you know, I guess my interests sort of divide too, so maybe that isn't such a big change. But I went for 60% in a sense sort of research, sort of self-motivated, largely independent research 40% involved with LIGO projects, so we'll see how that shakes down. So what's the nature of the research that you're considering? Issues of radiation reaction would constitute a large part of that, and obviously that does It does have at least potential bearing on LIGO sources, and it would also, the research related to LIGO was, well, it was a number of things, but to participate in some of these proposals that we were talking about earlier and stuff like that, to continue that line of work. So that's what that was about. I know you really like your plot on the door of a circularizing orbit. Where does that come from? That comes from some radiation reaction. We saw a good calculation that I was doing last summer. So, yeah, that shows the orbit circularizing. And to tell you the truth, I'm not sure whether that picture may actually be incorrect. I'm not sure that the code produced that was actually debugged at the time, but it was nice to see. And I'm sure that that took parameters that were just way beyond the physical realm, too, in order to get it to circularize that quickly, you know, in a few tens of orbits to go from rather elliptical to nearly circular. Right, yeah, it was pretty extreme. So that code is looking at... That was scalar charge code. I think that was related to some of these calculations,

55:00 leading order, central mass of the object calculations that I was doing last summer, actually to some extent doing this fall as well. But, yeah, I'd love to continue to work in that field, but, you know, I've got, right now, right now I write LIGO software, so, actually, right now I write LIGO software standards, we write meta software. Right. So, yeah, and you're happy enough doing that as long as... It's keeping my attention, you know, so. And it's, and is it actually, is it mostly that, because you mentioned you like the interaction between the people and the politics and so on, the collaboration. I mean, is that more the interest? I mean, there's not that much interest maybe in the nitty-gritty of writing the standards, as it were, but it's the interaction between the people that's... I don't know. Well, first of all, it's pretty clear which side the bread's buttered on around here in physics now. talk to non-LIGO-related physicists, they would claim that LIGO is just sucking up all the money. And, you know, there's two courses of action. You know, get pissed about it, or if you can't beat them, join them. And so this seems to be the... But that's maybe the cynical way to look at it, a more fruitful way to look at it, is that this is an exciting field. Lots of people are working on it, doing great stuff, and so getting involved in it is maybe not such a bad thing to do. And I think a lot of it, it's not a big stretch from my natural inclination, but it's not exactly down to miss, is to get more involved with data analysis and really writing software and stuff like that. It's not going way out on the limit for me, but maybe I've left completely to my own devices

57:30 that isn't where I'd go, but I don't feel too bad about it. Well, it's something that popped into my head to ask, because I guess I've talked more about this recently. Of course, you meet people from time to time who say, oh, LIGO couldn't possibly work. And, I mean, I must say that as a good undergraduate, I had to get white light fringes on a little fabric parameter parameter and spend a couple of days, you know, getting headaches trying to get these fringes up, you know, before finally succeeding. And, you know, if you asked me to make a four kilometer long fabric parameter parameter work, I'd say it was impossible, except, obviously, that I think the guys working on it are brilliant. I think they've actually got a two kilometer, basically have a two kilometer working now. And the thing was engineered and largely designed to do what they said it was going to do. And apparently, from all indications, I'm sure the information goes through many filters before it gets to me, but it really does what the engineers said it would do, that they turn it on and you know the alignment schemes actually work and so they can actually get the thing into lock pretty quickly. So, hey, it's the wrong time to announce the question. It just occurred to me that, you know, that I was always one of those two sides of your mind thing. I mean I always had no idea whether it was actually going to work because I just On the other hand, I didn't ever think about it as being a worry or something like that. I was just wondering if you ever sit there and think. No, I guess my only worry. I think, first of all, we can give it enough time. We can write the software that will do the data analysis. So it isn't going to fail for that reason. So in some sense, I can hold up my, I think, eventually, maybe not up to the time scale, but eventually I or we can hold up our end of the bargain. more, but to really build something that's got that sensitivity and you've taken into account every conceivable source of noise, that, you know, that, I'm sure everybody has

1:00:00 that in the back of their mind to worry about that and, you know, that, you know, to talk with the experimentalists and stuff, they, you know, that they're concerned about the, Does, you know, does, in Hanford, does tumbleweed rolling across the plain have, you know, in, you know, it piles up, you know, it blows and it piles up against the edge of the building. You know, people have been working on the calculations to figure out what that signature looks like. And, you know, man, you know, it's got to go down to that detail. You know, the probability that somebody missed something is, you know, did you really leave enough margin can they really make the vacuum system the one thing I will say is I do have supreme confidence in Ray and Fred and those people they honest to God know what they're doing so yeah Yeah. Sure. And of course as long as it's out of your hands. So basically, yeah, so I suppose basically there's no way the theorists can screw up this kind of, that sort of thing. No, I don't know, I mean, we've got an easier job too, so the experimentalists are the ones who really got it hanging out there, you know, it's their instrument. I mean, do you think that, you know, waveforms from the actual, say, merger phase of the black hole neutron star so that you need numerical input, do you think that's actually going to really be necessary when push comes to shove so that? Oh, I don't know whether that's it. Whether or not we understand the signals well enough to dig them out of the noise.

1:02:30 I think everybody would like to detect something pretty unambiguous first and then move on. Then you'd probably test other—once you've detected something, you probably wouldn't tend to believe other things more effectively. Be more convinced by other observations. I don't—in some sense, what's going to be the first detection? I don't know. I just couldn't tell you. I don't have a good feel for whether or not it's going to be necessary to put in information from numerical calculations or whether the post-Newtonian calculations or one of their follow-on surrogates is good enough. I really just don't know. Or whether or not you've got to wait and integrate for a year source or stochastic background. I've said, you know, after several beers that stochastic background will be the first source actually detected, but the last one to actually be believed. They'll turn it on, they'll say, oh, there's some correlation between the outputs, but nobody's going to tell you actually get a definitive detection of something that people are just going to say that's just some statistical anomaly in the noise or some common source or vibrations at the center of the earth or something. I don't know exactly what it's going to take to do the analysis. I think whatever it is, that will get done eventually. Building the instrument, that's the really hard part, to get one that's sensitive enough to do it. I mean, in principle, and I guess from a sociological point of view, it could be a problem from the data analysis point of view that because you need input of people who actually work on the instrument and different theorists and so on, that when it comes to the bit, they're all going to have different viewpoints of that.

1:05:00 what constitutes a real detection, and do you have any sense at this point, based on the fact that you've been working on coordinating committees and that kind of thing, of there being real seismic fault lines? I don't know. I guess that the collaboration does have, in anticipation of total meltdown later, publication that we talked about these internal proposals that primarily these are to formulate so that people are not cliquish about their internal collaborations. They're also impact on lab resources. But one of the other things that has people have to comment as they put these things together that you want to do some research involving the data and the equipment is the publication policy. Is you have to, you know, what publication do you think will come out of this research kind of see way off in advance where somebody's heading for publication. And so I think probably, at least from my point of view, that's how this will be controlled is whether or not, you know, what conclusions actually make it to print. And that's going to, boy, I don't know. I think that one, the sociology of that just hadn't been ironed out. and there may be a time when there's a really interesting LSC meeting to go to. There may be a real, you know, a real barn burner. Yeah, sure. That's an obvious possibility. That's, you know, and I think the idea is that it's a good thing to have some discussion of this when the sensitivity is 10 to the minus 19th or something on the 40 meter. And because people start, you know, people, their heads are still clear, feeling objective and stuff. But when somebody's starting to push down around 10 to the minus 21, 10 to the minus 22, and you're borderline expecting to see something, people's opinions might start to change about just how open the publication policy ought to be

1:07:30 or who's got access to the data. So, boy, getting these things ironed out in advance is probably a really, at least trying to. I don't know, maybe it's off or not because it will all fall apart when tempers flare. They really do try to do this. And, of course, that was one of the things we went through with the 40-meter analysis paper was they wanted to sort of test the publication policy, whether internally. of late. So we had to jump through some extra hoops with that publication. It was actually substantially more difficult to get it past the internal review than it was to get it past the physical review letters. That's not to say we didn't have to tussle with PRL too, but nothing out of the ordinary. I mean, just get your referee reports back. One's good, one's bad. Go fight. So you write a letter to the editor, fight for it. They send it out to some more referees. You get some other opinions and go. And that's what we But internally, it was, you know, first of all, they had to make it up as we went along because they really didn't have a publication policy as we went, and it had to be, we did have to make some presentations to the LIGO lab that we had to do that in order to present preliminary results at the Paris G.W. Daw in November of 97, and we had to do this subsequently as the paper was getting close to publication. And that was kind of issues of who's going to be on the paper? Well, no, that was mostly, in this case, nobody really gave a damn, because the paper didn't say much interesting didn't say anything astrophysically interesting but i think they just wanted to have us punch certain tickets so that when something really interesting does come along that you can see we made those guys you know present us the results and we you know

1:10:00 discussed it and we sent it back for revision and to comment on this and um we nixed it and said no you can't, or something like that, they probably would have been happier had they had good cause to reject it, but I suppose they were, I suppose the management was working a delicate balance between, in some sense one, maintaining control, but on the other hand, not wanting to stifle the first one that came down, you know, so this would encourage people to work And so since I presume that will be another, since, you know, who gets access to the data is presumably going to be one of these sensitive issues later on, was that a sensitive issue at the time with that 40-meter prototype data? No, but at the time, and to some extent still to this day, they do keep fairly close tabs on that data, that you can get it, but you can't just download it off the web. You've got to sign an agreement with the LIGO project about what, you know, you've got to have a reason to have it. You've got to tell them what you're going to do with it. And I'd have to look back at all of this, and I wasn't terribly privy to all these negotiations either, but you probably have to tell them what you're going to do with it in terms of publication. They don't want somebody going off and in some sense undermining the integrity of their data by claiming that they've black hole coalescences out beyond Pluto or something in that data, and then having to refute something that was based on their data. It sounds like it would get pretty ugly, so they want to keep but they want to keep it pretty open. Your sidekick, Harry, he's discussed this, and he's actually advised the project. When speaking openly, I guess for the most part he just observes, that it's hard to keep things contained. And in some sense, maybe you ought to just open it up, much more freewheeling about it, that as you try to control everything and keep things

1:12:30 under fairly rigidly constrained, that in the end that's going to fail anyway, so you might as well just open it up from the get-go. I guess another thing, probably the last thing I can think of, is that As in, because I'm sort of interested in this, where a possible future problem might arise based on previous experiences. Again, another thing, I guess, is that I've come across or heard of disagreements about how you look at the data. Like, for instance, Bernie Schutz, I think, the Carter people, of course, looked at data from the... The Glasgow Garcian, 100-hour coincidence. You know, I've heard of cases where Bernie got flack, for instance, I think from Ray Weiss at times, you know, about how they analyze the data. So this is something I heard at second hand. I think one example was that, you know, Bernie was interested in taking, you know, spikes out of the data, looking through it, and then, you know, actually removing spikes. And I think Ray was giving him trouble because he was saying, no, no, no, you don't want theorists to be doing stuff like that. If there's spikes, that means that the data is crap anyway. You want to let the experimenters fix up their instrument and stuff like this. Anyway, that was just sort of a, for instance, at, you know. Oh, yeah, I was there for that exchange. Yeah, that was in Paris at GW Daw in 97. and somebody got up and described, you know, methods for removing 60, or I guess in Europe it's 50 hertz line spikes, and Ray just, you know, God damn it, if I hear another theorist talk about that again, I'm just going to, I don't know what I'm going to do. The experimentalists will remove those spikes. But I don't know.

1:15:00 I don't have a dog in that fight, clear of that. Whether or not you want to clean up the data, fine. You know, there is a lot of effort to look at environmental, you know, that you can come up with some optimized data stream by finding a stream of data that has minimum correlation with the other outputs. You know, occasionally the detector will give some large, you know, will oscillate wildly for a few sample times. But, and you can see that that's clearly correlated with some seismic rattling of the apparatus. And so, and there are methods to basically to come up with a clean data stream that has minimal correlations with the other channels. So whether that's a good idea or a bad idea, I'd let somebody else worry about that. And the other thing, these are all testable hypotheses, too. You know, the worry would be when you clean it up, you take out what you really want to see in the first place. inject some signals in before you clean the data. clean the data, and then look for the signals. If you can see them still, that's a reasonably good indication that it was, if you see them better, then it's a reasonably good indication you did the right thing. If you can't, well, then maybe it wasn't so good, so, I don't know. Sure. Yeah, I more meant as a, for instance, you know, differences of opinion over ways to approach handling the data and so on. Was that something that you... Well, the way that the data analysis is done, so for instance, the way that the Carter group wanted to deal with the data, so the way they wanted to clean it up before they did their data analysis, I guess Ray Weiss didn't predict it like that. Yeah. Well, it's free country, and if you don't like the way somebody does it, do it your own self.

1:17:30 Yeah, as long as everyone has access to it. So that wasn't, like, that was, believe me, you know, in running this 40-meter project through, this 40-meter data analysis project through, that there were a lot of, you know, there was quite a bit of Monday morning quarterbacking on this that, and, I don't know, and that these people, other people have the same access to the data. If you don't like the choices that somebody's made, write an MOU and a piece of software and do it the way you think it ought to be done. I guess people have plenty of time to tell you what you ought to have done. Yes, that's an excellent idea. We will be expecting your proposal shortly. And so there's going to be plenty. There was some, and nothing particularly serious. Actually, I think, frankly, I think also some of it was jealousy, too. I think it was that Bruce put together a coherent effort that I think a lot of people, this is, I don't know why I want this on tape, but I think, I think a lot of people wish they could have done it, but in fact do not have the personality to carry it out. You know, Bruce is just such a hard-driving guy, but, you know, he's always enthusiastic. You know, he's always up and enthusiastic. And he's usually, you know, he can admit his mistakes, you know. Actually, that's one of the funnest things about working with him is to find what he's wrong. you know he's he'll admit his mistakes move on and you know let's go and but he's he's genuinely

1:20:00 enthusiastic about it you always get the impression that he's he's he's in it for he's in it he's in it to actually do the science and you know he just he loves it other people you work with you get you know you just get the feeling they're working an angle and and that um that's distressing that uh sometimes that you know that you get the impression that they're angling for power or position or prestige and with bruce you know he he's got political instincts like anybody else but you know at the end of the day he really wants you know to write that code or you know to do that analysis or do that calculation That's really his sole motivation. And that actually ends up being infectious, and that's why he's pretty competent at getting people to work for him. So, yeah, that and he can run a committee meeting and actually shut people off to that house, too. Okay. So, in principle, everybody has probably got equal access to the data, as you say, if they're writing them on YouTube. Yeah, you know, it takes a little work to get it, but yeah. As near as I know, or certainly any of this old 40-meter data, if somebody wanted to write a proposal and get some, I don't think it wouldn't be hard to do. And as for future data, that's a much trickier issue, and it's way above my paycheck to worry about that. But I think, you know, people within the LSC, or certainly people who are involved in the initial stages of what they call LIGO-1, you know, well, they'll get a crack at it. But nevertheless, if I read it right, the experience of, say, just you're working with the 40-meter data indicates maybe that that's not necessarily going to stop conflicts arising because, for instance, there may be other ways in which people don't have the resources what somebody else is doing like so as you say if they can't organize a sufficiently good large group around them like Bruce can they're probably not going to have the resources so they're still going to be feeling yeah there's still going to be conflict between yeah I think that falls under

1:22:30 the category of tough right yeah I just mean that these are actually yeah or another thing that it falls under is that this is actually difficult problems to deal with within the collaboration because in fact you're not going to be able to eliminate those kind of problems unless you somehow start allocating everybody's postdocs, because they're actually going to be on equal resources anyway. Well, I guess everybody has Rich Isaacson's phone number, so they want to submit a grant And they want to assemble the horses to get something done. They had equal access. Just some people are a lot better at it. You know, in the end, some people are just better at doing things than others. And that's the way of the world. To go back to what you said earlier, for instance, in the case of the ACES working group, you know, the spear carriers who do the work are actually maybe going to vote with their feet, too. Yeah, that's true. That's true. That probably plays a big role. I'm interested in this because doing the quadruple formula controversy and that kind of thing. I mean, some of the people who had been big skeptics and spent a lot of time criticizing the way other calculations were done, you know, would kind of say that, so here I'm thinking of Peter Havis as an example, you know, I mean, he would say that, well, he just never had the resources, he never had the postdocs or whatever to carry through his research program. So I suppose that, you know, that's one way that you get these things springing up because, on the one hand, he's, you know... I don't know him, but, you know, everybody under the sun wanted to work for Kip. Sure, I mean, you have these differences between them, and then I guess at some level it sort of explains some of the conflicts that arise, and I guess that it's, I'm just thinking that there's no obvious way to solve those conflicts then, because as you say, those differences

1:25:00 and resources are always going to be there. Yeah. Well, and this is a big collaborative effort, and some people just are naturally good at collaborating. It's sort of leading a collaboration, and other people just aren't. Some people are just fun to work with. You know, they don't pay us enough to, you know, to really put up with too much bullshit. So, you know, so I guess all you can do is vote with your feet. Yeah. Well, it's an interesting aspect of it. So it's one interesting way that the way things go actually comes up from the bottom. yeah yeah that well that's that it's not all coming from the top down yeah to me that is sort of the fascinating thing about about viewing the whole project and that uh it is that you know i you know when you have a hammer every problem looks like a nail but uh um that for me i i have military background, and I really do see it, and sometimes it's very different than military because you can't issue orders and stuff, but you really do see sort of a detached, in some sense detached, very high level management of the project. And then below that, you do see people like Bruce, who are just sort of the mid-level field commanders, who clearly have the ear of the hierarchy, but also who have the respect of people who are down there writing code. And that without some fairly substantial group of those people, the whole part, because as much respect as I do have for people at the top of this, I really do, I think they just, you know, they really are competent and certainly give that impression and have no issues with them, but they won't write, they probably aren't going to write the code. I mean, I don't think Barry's going to sit down and get 10,000 lines of data analysis

1:27:30 code working. And, you know, somebody's got to get their head around this problem. And I think any mid-level person in this organization whose sole intention is to kiss ass up the chain is doomed because the thing you ultimately need is the respect of people below you. You know, if you do not have postdocs who are itching to work for you itching to work for you um your your empire will crumble from below and uh so so people who can really instill enthusiasm and people coming up from the bottom will be successful in this and people who don't won't and and i i know amount of kissing the hip you know of kissing ass up the change, we'll do you any good. And I think to some extent the model that you and I are familiar with is Kip, you know, that he was, you know, the one thing, you know, the one thing, he's one of the most influential people in relativity, and I think one of the reasons is, you know, he just treated postdocs and students really well, and, you know, and really let them work on what they wanted to encourage them and you know in some sense that he built his empire from below now granted he also had a lot of respect for people you know from from his mentors as well but uh um but that's really you know the the stable empire in in this world is is uh you know what are the people below you think not not what are the people above you think and uh because it because nobody can do this thing alone that there's just too damn much to be done and people who alienate the people they work with will get nothing done and so yeah you know that there always is this tendency to sort of think you know god you know you know you know, so-and-so, you know, they won't do it up to my standard or, you know, they're not, but, you know, sooner or later you've got to trust some other people to do some other things because nobody can do it all. That's a good way of it. Yeah.

1:30:00 Yeah. Yeah, that's an interesting point. Yeah, I certainly like the idea of looking at it like it's the people. it's the people at the bottom besides doing the work well that also just be maybe how I view it being down here down here at the shall we just say in the lower half that's where I that's how I see it you might change your mind as you rise the working class I got the form of the job at last. Yeah, it may be like that. I don't know. I actually do see that the people with the healthy research groups are the people, though, who, and I think, although it's not LIGO, that Kip, I think, is just absolutely the best example of this. that he really did, people wanted to work with him, and so he was very successful. I think that's maybe the model that the people in this project are going to have to follow, because nothing's going to get done until somebody writes the C code to look for it, at least from a software point of view. And I'm sure, I'm sure when it comes to the instrument building, that there's, that there, you know, that there are also a lot, you know, that I guess in some sense I'd put, you know, Nergus and stuff, that there are some people who have been postdocs in recent years who are now very involved in building the instrument. And, you know, they really know what's going on. And, you know, and if the mid-level management really sucked and nobody wanted to work for those, nobody wanted to, you know, the people like that, gee, these are solid state physicists, you know, who can do stuff like polish mirrors and build electronics like they couldn't get a job somewhere else. If this project, you know, if the boss sucks, I'm out of here at three times the money.

1:32:30 Sure. They probably actually even have it worse. Well, I agree. It certainly speaks well to Bruce. I was thinking that I haven't worked with Bruce myself, but I'll start by the fact that most of his group here is people who are at Caltech. Since, as you said about KIPP, we know that they know what it is to be treated, right? So, obviously, we'd love to. He's our main source of conversation. Well, I was going to turn it on now anyway, Alan. There is one aspect of the software that really does have me concerned about where this thing is going to go. On the one hand, there's going to be a lot of testing of the software and stuff like that, but I know that sooner or later somebody, some junior graduate student, is going to write some C module that does something and will use it for months, is going to discover? Because people generally don't find bugs in other people's software. The people who are going to find it is the person who wrote it themselves. It's certainly the most likely candidate. And how is that person going to go up to Ray Weiss and say, oh, by the way, the analysis we've been doing for the last month is meaningless because i fucked up and you know exactly how do you create the climate where people are you know where you know people at the bottom level are going to be writing you know they're going to be writing a lot of the software it's going to be hard to test you know you know how do you create the climate where they you know that they feel free you know call them free when they say tell you know you may have never heard of me but I fucked up that that's I also kind of fear we have this committee that acts as the software coordinator but when they get a software coordinator

1:35:00 whoever wants that job They've got to be incredibly cautious about the climate they create for this reason. First of all, that if they reject all, people submit some code to be validated. Well, if the software coordinator is a prick, this sends everything back and says, well, you didn't dot this I or cross this T or something, that you eventually don't have any software that's really in the pipeline to be validated. And the real test of the software is to get some coherent package that you stick a whole bunch of modules together. And so if the software coordinator is a prick, you don't ever formulate a big enough package for a coherent test. and nobody's going to feel like submitting software if the guy rejects it all the time and you're not going to get it together for a big test. So in some sense, if the person is overly cautious and a prick, you will actually have less well-validated code than you might with somebody who was maybe a little more lenient about it. You know, let's let it slide for now. Now we can fix, you know, what we really need to do is start bashing this stuff together and see whether it hangs together or not. And that is the other thing. But then again, if the person's too lax, you know, kind of, you know, real cowboy, all you have is a bunch of cowboy programmers out there that, you know, anything goes, that you don't end up with a very coherent package either, and that'll suck. So, and, you know, then there's also this issue of kind of creating the climate where you want, you know, you want people to be just extra careful. You know, when you submit the code, you have tested it, you know, and it's got both belt and suspenders and nothing can go wrong. But on the other hand, you want people to feel free that, excuse me, but I did fuck up. How do you meander that narrow? That's got to be a set of measures there all through this parameter space. What would actually be the model if there are, for the way the different cohort and

1:37:30 all these different people will go together. I mean, will there be one main code written by one person or group which does most of the work and everything else sort of should fit in around the edges and should fit into that? Or will it just be patchwork? I don't have a big... I'm not sure that A, that's decided, and B, I'm not sure I have a big enough picture to answer that. I think if you want to follow this aspect of the sociology, you ought to go out to Kel-Tec that he's the one he's certainly the one I don't know whether he's going to do it but he's going to be the one whose neck is on the block to make that decision and figure out how that's going to happen I talked to Albert briefly at the MLD conference but of course people are so busy well that's he spread pretty thin too well great thanks Alan Amen.