WHO HOLDS THE KEYS? Friday, March 20, 1992
Chair: Dorothy Denning, Georgetown University
Panel: Jim Bidzos, RSA Data Security David Bellin, Pratt Institute John Gilmore, Cygnus Support [Now also with EFF] Whitfield Diffie, SunSoft, Inc. John Perry Barlow, Electronic Frontier Foundation
D. DENNING: Welcome to the session on "Who Holds the Keys?"And in case you hadn't decrypted that by looking at the program, the session is actually about cryptography and some of the implications of cryptography in terms of personal privacy, electronic communications, and so on. Behind all the issues is to what extent should the government actually have control over the technology and our use of the technology and so forth.
We have five panelists with us today, so I'm going to keep my comments extremely short because I want to give them the opportunity to speak about the issues, and then I want to have time for the audience to ask questions. But what I would like to do, for those who might have missed the announcement that was in the first CFP Progress newsletter that came out: the Computer Systems Privacy and Advisory Board has voted -- made a recommendation by unanimous agreement -- that there should be a public discussion and evaluation of the cryptography policy for the country. This recommendation was just made, I believe, two days ago, so we don't yet know what the response of the administration will be to it. But I find that extremely encouraging and hopeful.
Our first panelist today is David Bellin, who's on the faculty of the Pratt Institute.
BELLIN: 3FA0 FF48 0ADD 3C. (laughter) Thank you very much. I guess I could just sit down now, knowing that I think there was only one person in this room who understood what I said thus far. One question I want to pose to you is, Does that mean that I should be prohibited from getting up here and uttering what seems to you to be nonsense? And let me point out that my algorith. m to develop the words I just said is quite unique -- it's at least as hard to decrypt as DES, or as easy, depending on your belief in trapdoors. Does this mean I should be restricted in my ability to sell the algorithm to others? Or that I should be forced to furnish the algorithm or the internals of it to a government agency? Or that my use or sale of this encryption algorithm should be permitted only with governmental leave? Should commercial carriers that distribute my words be permitted, or even required, to restrict distribution of my speech if it's presented in a form they cannot understand? Or even more dangerous, of course, if it's presented in a form that some hypothetical government agency can't understand? Well, in my view, all of these proposals, which you've heard in other forms before, are nonsensical. I think that the freedom of speech described in our Bill of Rights gives me the freedom to encrypt my speech in any manner I desire. Of course, that freedom of speech is not absolute. You can't shout "Fire!"in a crowded theater, but of course, you can't accuse me of shouting "Fire!" if you can't understand what I'm saying. Encrypted speech is clearly protected behavior, in this sense. Well, we have a cherished concept in this right to speak our minds freely, and of course, sometimes we want people to understand us.
A closely-associated concept is the right to freely associate with our peers in order to engage in other protected legal activities, such as that of speech people can understand without a key. But privately-encoded means of communication are crucial to the ability to associate freely with our peers in this day and age. How could we associate freely if a real, well-founded fear exists that somebody will be listening as we attempt to get together? Aside from a few persons who do exist who have personality deviations of interest to psychiatrists, few of us would choose to engage, for example, in some of our bedroom activities if we knew.
that a government agency or snoopy neighbor could look in on us. But extend that thought to other spheres of life. Imagine how much harder it would be to organize a local PTA to rally against the policies of the local principal if the school principal was the spouse of a Senor Ortega who Bruce Sterling described yesterday.
The proposals of the Bush administration and law enforcement agencies are constantly in the direction of opening up that metaphorical bedroom door so that they can look in. And of course there are bad guys out there, as they remind us, rapists, terrorists, international subversives, and lately, of course, it's the drug cartels that are the big ones. They all have more money than law enforcement; they're all more flexible and cunning than law enforcement -- perhaps they're smarter, also, I don't know. The argument goes that we need restrictive cryptography policies in order to keep advanced technology out of the hands of these bad guys. Well, as far as I'm concerned, this is really just more nonsense.
The restriction on high-tech exports prevents the Soviet Union from developing an advanced space program, or from developing advanced radar systems, or from developing advanced missile guidance systems. Presently, it's illegal to export a 300-baud modem to certain countries. Do you really believe that the technology is so controllable that those countries cannot now transmit data at 300-baud? Clearly, this begins to enter a realm of ideology, and not reality. As far as I can see, criminals or foreign entities will get advanced encryption technologies no matter what our government does. You can't control free trade in scientific information. The mathematical theory upon which cryptographic techniques are based is openly available. If the government was to attempt to restrict that type of information, it would clearly turn into a trampling of our democratic principles.
I also don't accept an adage that I've heard repeated over and over at. this conference -- that there's a continuum, and at one end there's public safety and at the other end there's freedom. We should think about the social conditioning served by establishing these two as polar opposites, which I don't think they are. Just what are the scales of justice balancing along this continuum? For example, when we speak about public safety, we should give a little bit of thought to whom we pretend to keep safe, and from whom or what we seek to keep them protected. For example, keeping Bankamerica safe from fraud is a lot different than keeping me safe when I go out in my back yard and fall into my swimming pool. Will cryptographic controls really keep me safe from a robbery on my way home tonight? And what if the safety I want is the safety from surveillance, from others seeing what I do in what I consider my private time? And by the way, again, whom am I being kept safe from? The drug cartels? The Communists? Or political opponents of the current administration?
On the other side is supposed to be freedom. We love to toss the word freedom around, and we love to hear how the enslaved and oppressed everywhere else in the world want the freedom we have. It might be useful in a conference like this to give a little consideration to what our freedoms really consist of. Is it the freedom to go to our hypothetical supermarket and buy the hypothetical brand of coffee at a discount? The freedom to have cheap petroleum products? The freedom to go to the mall and listen to Muzak? Well, if those are the freedoms, then maybe this continuum put forth makes some sense. But to me, a consumption-based concept of freedom has little appeal. The freedoms I seek to preserve are the freedom to speak my mind on issues that concern me, the freedom to associate freely with other citizens to exercise some control over government policy and elected officials, and the freedom to insist that government act to promote the general social welfare. Those are the. freedoms I seek to preserve and extend, and in that sense, this safety- freedom continuum seems to make no sense to me.
I was reminded of another analogy often made yesterday when I talked to my old friend Richard Civille, who I met when I was on the board of CPSR. He reminded me of the old adage that when guns are outlawed, that only outlaws will have them. That analogy is clear here, but thinking about that phrase forced me to think about the other analogy which is often made about the new electronic frontier and how those of us inhabiting it now are cowboys on the new frontier. I reflected upon this, and I think really that that's a fundamentally flawed concept. Remember when Columbus got here -- I'm sorry, I may hear about this later -- but remember when Columbus got here, it wasn't really an empty frontier. Of course, there were some people around. And who were the cowboys? Well, to some, they were an extension of Columbus' arrival on the non-empty frontier where there were people living, yes, called native Americans. And in terms of looking at our own cultural history, it might be more interesting to look at things through the eyes of the native Americans, of the Indians out there on the frontier. Of course, the metaphor used about a virgin frontier is also very loaded with machismo and other nuances that I will leave aside for the moment. Look at it from the Indians' point of view. The railroads were coming through, and with them the cowboys and the cavalry weren't far behind.
Well, I would allege in our sphere of computers and telecommunications, the railroad's coming through now also. I would argue that cryptographic techniques should be encouraged, widely disseminated, and that all networks should be forced to include encryption of mail as a default standard available to all users. (applause) Thanks. It's only in this way that we'll be left free to say what we want, to associate with whom we want, and to keep from being forced onto little piec. es of land from which nobody will be able to hear us anyway. Thank you. (applause)
D. DENNING: Our next panelist will be Jim Bidzos, who's the president of RSA Data Security.
BIDZOS: Thank you, Dorothy. The next eight to ten minutes, I'm going to try to give everyone here a little perspective and lesson in cryptography and explain everything. Actually, what I would like to do is try to put this digital telephony issue in a little bit of perspective and talk more about the historical policies of the U.S. government towards cryptography. I think maybe it will give us an idea of where we might want to try to go to make some real progress here because there are certainly some problems.
First of all, there are three ways that I've observed that the US Government manifests its policy in cryptography. The first is in export control; the second is in standards; and a third is in legislative and executive action. For example, the creation of the National Security Agency in 1952 effectively set the cryptographic policy in the United States for at least the next 25 years. The proposal of a digital signature standard by the Commerce Department very recently is an example of an approach to policy through standards. And, of course, the export controls that exist routing cryptographic systems through the State Department are also an example of that policy.
There's something that's very interesting about all this. We can certainly lump the digital telephony issue, and also last year's S.266 -- the proposal to essentially outlaw the use of cryptography that the government couldn't access -- into this category as well. But what's really interesting about all these policies, if you think about them, is that they all put the government in an adversarial role with industry. Instead of industry and government working together, we essentially find ourselves fighting all of the time, whether it's over legislation -- which a lot of us here are obviously very interested in --. whether it's over standards that many of us may not agree with, whether it's over an export policy that many of us continue to fight. It's an ongoing battle that's going on today and has been for a long time. The long-term effects of that adversarial relationship, I think, are much worse than some of things we're seeing now. It's pretty clear that many of these things aren't thought out, they aren't done in consultation with industry. They're usually done in private and they're essentially forced on us. They tend to look like knee-jerk reactions in many cases. I think S.266 is a good example of that. I think this digital telephony proposal is a good example of that. So there's a lot of turmoil and confusion, and I think danger as well. We really have to be careful here. I think that a lot of these proposals are not well thought out, primarily because they're not done in consultation with industry, they're basically done in back rooms. For example, one could argue right now that economic policy is being made by the intelligence agencies. I personally don't think that's a particularly good idea.
Now, why is all of this happening? There are all the obvious reasons -- there's more networking, there are more people concerned about their privacy, the world's moving more and more into the electronic frontier, as we've heard. But there are other things that are changing as well. Cryptography is moving into mainstream products; cryptographic technology is out, it's available. These things aren't going to change. The companies that produce the mainstream products and want to use cryptography are going to continue to do so -- they're going to want to do more of it. We've seen that now in many cases, especially in the area of export laws and some of the battles that are going on right now. Also, a lot of the companies that are building these systems today aren't the traditional kind of company that had a special kind of relationship with the government, in pa. rticular, many of the agencies that made these policies. For example, I don't thing that Microsoft or Apple are companies that are greatly influenced by the Defense Department. They don't derive much of their revenue from them, they aren't influenced by them, so we are seeing more and more of this turmoil. This helps explain a lot of why this is happening, and also the emergence of electronic commerce, which is rapidly becoming global.
I think this is also driving a lot of these clashes between government and industry and causing sparks in the area of cryptographic policy. I think there's room on both sides here for a little more understanding. For example, I think it's very wrong for the Justice Department to believe that it can slip some language into different bills and perhaps keep trying until that works. I think there are too many people, many of them in this room, who understand these techniques, who are prepared to fight them and bring attention to them. I don't think that's ever going to succeed. They're very vague and nonspecific which makes them very, very dangerous. So I don't think that approach is going to work at all. However, they also have some legitimate interests, which I think we need to be more sensitive to.
Now one of the interesting things here that I think many of these proposals fail to recognize is that we technically have a large number of options. We can do a lot of different things. Let me, for example, just describe very briefly one technical approach to possibly addressing some of the concerns of the Justice Department. This is just a relatively simple proposal, but an interesting one to think about, primarily due to Ron Rivest. And that is simply that in a world where public key cryptography may be used extensively, there may be these things called certificates, which are essentially a trusted copy of one's public key. A simple way to protect the interest of both parties in this case -- and again, this is just a proposa. l and to get people to thinking and understanding that there are many, many different options -- is that one's public key could also in fact be signed by some independent party. Now let's say, for the sake of argument, that this might be your bank. What that signature on your public key or on your standard certificate from your bank may say is that the owner of the corresponding private key has in fact registered or escrowed that private key with this organization, and presented with the proper documents, whether it is a warrant or whatever it may be, then that private key could be made available. This is just a simple example of how technology could be applied to provide far less control than a piece of legislation that says anything you do that interferes with what I want to do is illegal and I can fine you money and send you to jail.
When we begin to understand that we have all these different options, then I think the important thing is to realize that we need to have a national policy review. And I in particular welcome the decision action recommendations of the advisory board to Congress formed by the Computer Security Act to have a national policy review in this field. I think it's the single most important thing we can do. First of all, the advisory board, again, advises the Congressional committees who have oversight over the Computer Security Act. I think it's one of our best channels into Congress to actually effect some sort of a positive change. I think it's important that government stop fighting with industry and that they finally start working together in this area because ultimately we're going to go nowhere. It's pretty clear that our economic interests are some of the most important interests that we have today, and the more government and industry fight on this, the less progress we're going to make. I think it's true that traditionally whenever interests dominated by military concerns have controlled certain technologies, then they've . ultimately been released. We found that by the time that tends to happen, the markets have been captured by companies from other countries. I hate to see that happen in this field, so in addition to all the personal privacy issues, we also have a national competitiveness issue. I would encourage everyone to support the recommendations of the advisory board to have a national review on cryptographic policy. I think it's long overdue and it's high time that we did it so we can stop this kind of thrashing conflict between government and industry and start working toward some rational policy that will work for all of us. Thank you. (applause)
DENNING: Next is John Gilmore from Cygnus Support.
GILMORE: Hello. I'm going to keep my talk fairly short this time. I'm giving up some time to let Whit Diffie expand his presentation.
What I'd like to talk about is the need for discussions on cryptography to happen in the public. The NSA has a long history of working these things out in private, using a sort of divide and conquer approach, making regulations that you only find out about when you go and talk to them. You can't just look them up, you can't design your product based on what you know about them. Instead, when you go to get export permission you find out that you have to rewrite your product. There's a strong presumption in our law against something called secret law. I think it was Justice Brandeis, in the process of knocking down some such regulation, who said that secret law is an abomination.
The pro-cryptography use side has been trying to pull this stuff out into the public, into the public eye, and into public debate. One such example is an amendment to the Export Administration Act that has been pushed, in particular by Microsoft, that would transfer control for export of cryptography in mass market software products to the Commerce Department. Moving from the State Department to the Commerce Department may not sound like much -- it's still . the government running it -- but it turns out that the State Department on this issue is basically run by NSA, and the Commerce Department is much more aligned with reality. (laughter and applause) So this has been successfully put into the bills, in both House and Senate, and of course the bills were slightly different, so they've gone to a conference committee. The administration has threatened to veto the bill over three items in it. One of them was this transfer of encryption, so Senator Jake Garn went to the NSA and said, we need to negotiate this out.
It turns out that what has been happening is that the NSA picked the Software Publishers Association and has been negotiating with them over this, without going through a public process and without dealing with the concerns of any other part of the industry. Now, I think there's a lot of danger in this. One of the dangers is that what we'll get is something that meets the interests of these companies, but not the interests of their users. Another danger is that particular threats to civil liberties in general are not addressed in this. Another one is that because these negotiations are happening in secret and by people who aren't used to negotiating with the NSA, there's an opportunity for the NSA to sort of pull the wool over their eyes, to say, no, this is how it has to be, we can't let you get encryption any stronger than this, and that's just the way it is. People who maybe have been dealing with the NSA and the issue longer can go in there and say, this is ridiculous and you know it's ridiculous -- DES is all over the world, and not allowing DES would just be stupid. In addition, there's a potential to produce a sort of government-sanctioned monopoly through which algorithms will be exportable that would benefit particular companies without that choice ever really seeing the light of day in a public process.
I think that the right way to approach this is to open up this negotiation process, open u. p the secret draft agreements that are going back and forth between these people. Bring in the civil liberties people, bring in the computer manufacturers, the network software companies, all the people who are concerned about this besides mass market software publishers. I think this is a really good year for working on export issues. I think there has been a lot of progress with COCOM; there's been a lot of progress with proposals to cut the military budget, to shrink the black budget, and to stop pretending that we're in a cold war. I don't think we should let this chance pass. I think we should work pretty hard either to keep this amendment that gives full authority to Congress to switch it to the Commerce Department, or we should open up the give and take between the public and NSA to public negotiation. Thank you. (applause)
D. DENNING: Next is John Perry Barlow.
BARLOW: John was also liberal with his time on my behalf. He and I have been working together on an article for Communications of the ACM, which I'm going to try to compress in real time, since it's far too long to fit here, but essentially I want to back up to last year's conference. I came away from that conference troubled by a widespread willingness I felt I discerned for many people to trust the government with the regulation of privacy. Everybody understood that Cyberspace is a very public kind of place -- every time you make an economic move, you smear your fingerprints all over it, and a lot of folks were ready to take the sort of European approach in having the government regulating private information. I'm not comfortable with that at all.
The solution to many of these problems obviously involves encryption. Well, there are problems now, as we see, with encryption. I mean, one of the areas where I've expanded my sense of the frontier analogy is that when we first came to Cyberspace, I didn't recognize that it was already fully occupied by a well-established army of occupati. on which had been there a long time in the form of the National Security Administration (sic). And this group of 40,000 troops has got this place wired for sound. I mean, there is very little that goes on in Cyberspace that doesn't go through the big sieve out here at Fort Meade. And they want to be able to have that sieve continue to work.
Now there is some ambiguity about who the enemy might be at the moment. (laughter) That doesn't seem to trouble them. I got particularly rattled about it when I realized that not only could we not dust off our hands in satisfaction, having killed Senate Bill 266 and its provisions to give every cop in America the doorkey to our locks, but we had essentially sown the dragon's teeth, because 266 is coming up in many different forms in many different places. The most discouraging of these to me is a series of meetings that have been taking place on digital cellular standards -- the TR.45 standard-setting committee which has adopted a standard that was basically mandated by the NSA, which will not be published, which is unusual with an encrypted standard. Usually you publish it so that people can try to break it. A lot of people in the cryptology community say that it is not going to be published because it's very easily breakable. In fact, one person -- and I wish this were not a unnamed source because I'd love to be able to say who this is -- but one fellow told me that because of the laxity of the standard, it would be possible for any county sheriff in America to attach a black box to his PC and get the plain text of your cellular phone conversation. Well, you don't need a law when you've done that. If you're really think that ECPA is going to apply, and you don't have to worry about it on those grounds, then you have to wonder why law enforcement is so worried about encryption in the first place, and whether it's going to take away from them something that they already have.
But the real problem, I think, is that we . are using an instrument of foreign policy, supposedly, and foreign jurisdiction, to set the domestic agenda. There is a cultural conspiracy which doesn't involve maligned gray people meeting in dark rooms. It's simply a cultural coincidence of belief between the law enforcement community and the NSA. In fact, when I started trying to find out if there was any kind of direct collusion between the NSA and the FBI on encryption, what I found -- and I talked to people like Stansfield Turner and Bobby Ray Inman -- I found people at very high levels who would immediately, when you started asking them about export of encryption, would immediately go to the war on some drugs. Right away. And this is supposedly a domestic matter. What we are doing in essence is conducting domestic policy with the NSA. I don't believe this is correct; I think the proposals that are being called for here to limit the NSA, to open up the whole standard setting for encryption in the United States are very timely, and we must do this now. Thank you. (applause)
D. DENNING: And finally, our last panelist is Whit Diffie, and we've agreed to let Whit have thirteen minutes instead of ten, in exchange for which he's going to entertain us with slides.
DIFFIE: OK. I gave this talk at one hour's length to the Committee on Communications and Information Policy of the IEEE on Tuesday. Lance heard it and said, "I think Computers, Privacy, and Freedom ought to hear these remarks,"so here you have the fourth time -- this is the speed version.
Insert Fig. 10-1 & caption "Figure 10-1"
I think if you think about it, you'll realize that the most important things any of you ever do by way of security have nothing to do with the guards, the fences, the badges, the safes, or any of that stuff (Figure 10-1). That far and away the most important security measure in anybody's life is that you recognize the people you know, you recognize the people you work with. And you have a mechanism for extending . that recognition by introduction, so you come to know, come to trust people you didn't know before because people you did know introduced them to you. Then you have a transferable form of authentication that you use constantly. You have your written signature and you put your signature on something and the recipient can show it to a third party and say, "Here. Look. Whit Diffie promised to do that. See his signature on this letter." And then finally, you have the possibility of closing your door, stepping aside somewhere, and having a private conversation with somebody.
Insert Fig. 10-2 & caption "Figure 10-2"
Well, the question for us comes down to, What do you do when you move from a world of pen and ink and face-to-face conversations into a world of electronic mail, telephone, video conferences, and electronic data interchange (Figure 10-2)? And make no mistake about it, before the end of this decade, two-person video conference, which for reasons of cost, has not really appeared yet, is just going to sweep this country the way fax has, the way PCs have. It'll make possible collaborative work over a distance, and it will transform every corporation's and every social group's way of operating.
Insert Fig. 10-3 & caption "Figure 10-3"
We are awash in technologies that decrease privacy -- lots of them developed under the umbrella of the war against terrorism, the war against drugs, a lot of them, as a matter of fact, from the Vietnam War (Figure 10-3). There are lots that you know; let me mention two or three you might not know. There was a wonderful article in Aviation Week four months ago or so, sometime last fall, about a gravetomometer so sensitive that it was able to detect a pound of cocaine suspended in the middle of a 50-gallon drum of something like orange juice, I don't remember what it was. You really can't hide things from something that can detect gravitational anomalies that fine. Well, the Vietnam War gave us night vision that would ma. ke your bedroom look as though you were making love on the beach at Monterey at midday. The last one's one called Rafter, and it was mentioned in a book called Spy Catcher. Rafter is a British code name for the technique of listening to radio receivers, that is, listening to the local oscillators of radio receivers to figure out what station you are listening to. Now people are used to the notion that they might be monitored when they are transmitting, I think that's perfectly natural. But think about the fact that people might be listening to hear what you are receiving, what things you're interested in getting.
Insert Fig. 10-4 & caption "Figure 10-4"
Against that I can find very few means of increasing privacy -- I don't guarantee there aren't others -- certainly crypto is a technology with the capacity to increase people's privacy (Figure 10-4). And I think all these cases, the first couple in any event, are fairly obvious and desperately needed -- security in telephone communications even more so, security in electronic mail.
Two far less obvious things have been developing over the last few years, things that require the provision of actual anonymity, and at the same time prevent you from making use of your anonymity to cheat -- those are electronic elections and electronic money.
Insert Fig. 10-5 & caption "Figure 10-5"
It's also true that cryptography has developed capacities that allow it to increase accountability (Figure 10-5). In short, if you look at the previous slide (Figure 10-4), increase in anonymity where appropriate, here increased accountability. Digital signatures give you the capacity to audit, just the way you audit a classical ledger, the same format at least. You look at the handwriting of each individual entry in the ledger, you know who did it, you have accountability for the actions that the ledger covers. Digital signatures give you that capacity in electronic media and auditing is the basis for investigations. It is the c. apacity of investigations to discover what did happen -- to find out who was accountable is the essence of people's being held answerable for their actions.
Insert Fig. 10-6 & caption "Figure 10-6"
Let me get to the policy difficulty (Figure 10-6). I think cryptography offers, or at least appears to offer, something that as far as I can see is unprecedented. I mean, if you get in the spirit of the mathematics -- and I emphasize that nobody knows for sure -- but it almost looks as though an individual might be able to, in a systematic way-- for example, with a mass-marketed piece of software -- protect information in such a way that the concerted efforts of society aren't going to be able to get at it. I mean, no safe you can procure has that property, right? The strongest safes won't stand an hour against oxygen lances. But cryptography may be different from that.
Insert Fig. 10-7 & caption "Figure 10-7"
Before I go any further, I'm going to say that in practice, it might turn out to be not so different from that. As you try to implement cryptography, you find that you begin to think, well, maybe this isn't so much a mathematical problem any more. Maybe this is an arms race, and we've got to develop a better technique and they'll develop a better technique. Because of the problem of electromagnetic radiation out of your equipment, particularly the plain text contaminating and modulating the transmitted cipher text signal, the tamper-resistance of your equipment and the quality of your ability to generate unpredictable numbers are absolutely crucial (Figure 10-7). And those are all issues of physical science. So I put that bee in your bonnets -- worry a little when you write your crypto programs and things. I've heard some fairly cocksure statements around here, you know, "Anybody could implement this," and "How's there any hope of controlling it?", etc. Well, it's not always that easy.
Now we will turn for just one instance to the previous slide (Fig. ure 10-6) and say, from my point of view, that this, in fact, has a lot to be said for it. I understand why the police don't like it. But let me suggest that a very large part, if not the essence, of the distinction between a free society and a totalitarian society consists of the difference between being answerable for your actions and being subject to prior restraint on actions that the society doesn't approve of. (applause) What this means is, in essence, if you look at it, the subpoena sort of model. They've got to come to you and say, "Whit, show us these records or you sit in jail for a while." And I get to decide, as reporters unfortunately have to decide, whether I'd rather sit in jail than show this court the information they want. What I think is utterly inappropriate is they can go to some third party, the keepers of the disk that I have my stuff on, and say, "Either you show us Whit's stuff or you go to jail for a while." And you know, it's not their data, what do they care? OK.
Insert Fig. 10-8 & caption "Figure 10-8"
I really believe, long-term, that there are some serious problems here (Figure 10-8). I think telecommunications violates a locality property that has been the basis of society. If I come into town to negotiate with somebody, I really can't be confident that I won't be noticed going in and out of town. If I have to stay over in the hotel, I leave a record there, etc. If calls can be made that are somehow completely anonymous, completely secure, any two people could be in contact, and the whole way that we understand what groups are in the society would dissolve. I think, in a peculiar way, the lack of security in communication has taken the place of this locality. That is, you can call somebody up and conspire with them by phone, but you can't be sure that you will not have been overheard. You don't have the same confidence that you do if you go sneak off behind the haystack to encounter this person. And so I wonder if soci. ety, in the sense we know it, would exist in the presence of absolutely unaccountable communication between any pair of people, as much as that notion appeals to me tremendously.
Insert Fig. 10-9 & caption "Figure 10-9"
The next point, and I think a very down-to-earth one, is that traditionally we have these principles of freedom of speech, freedom of expression, freedom of information (Figure 10-9). But we are moving into a society where information is a commodity. And societies have always regulated commodities, so what possible hope do we have of keeping the information definitely unregulated? Cryptography is a technology with the potential to make data unregulatable in many cases, so I think we have to expect a lot of opposition to a lot of its uses.
Insert Fig. 10-10 & caption "Figure 10-10"
I think, however, that there's certainly something to be said for the notion that data are less dangerous than matter. I mean, it may be dangerous if I stand up here and explain how an H-bomb works, but it's nothing like as dangerous as if I brought an H-bomb with me. (laughter)
Insert Fig. 10-11 & caption "Figure 10-11"
So now I come to the issue, how well could cryptography be regulated? I think this is a case where -- I think John Barlow quoted this slogan -- if guns are outlawed, only outlaws will have guns. I think that actually applies much better in this case because cryptography grows much more naturally out of mainstream computer science than ordnance technology does out of mainstream home machining. Virtually everybody does cryptography on standard microprocessors or digital signal processors unless they want to go really, really fast. I mean, people build lots of special purpose crypto chips but lots of people also do it on standard processors for the same reason that everybody does everything on standard processors -- that's the cheap way to do it. You can do crypto in very small programs. Even DES, which is a rather large complicated crypto system. , is not a very large program, as programs go. And I think now, looking at that slogan, right?, if you pass a law against using cryptography in some circumstances, all of the people who are normally law-abiding will abide by it, but somebody who really wants to get messages through will build covert channels to carry them. We discovered in computer security that even within the rather controlled environment of computer operating systems, bits move through covert channels rather freely and they are very hard to find and very hard to limit. So, as a result, the enforcement of any such rules would probably require very drastic measures.
Insert Fig. 10-12 & caption "Figure 10-12"
Well, down to the last two points here. I think that if you do what seems to be proposed in these various bills that come up, which is to limit the quality of security technology available to individuals, that you risk limiting two things (Figure 10-12). In the first place, you risk the accountability of the government and the society itself. That is to say, that everybody worries, and I think correctly so, that warrants may not be necessary to some people. The fact they can't present the evidence they gained by their warrant-less wiretaps doesn't mean they can't then build other evidence based on them.
The second point is that a lot of the cases are going to be exposing you to the actions of somebody other than the courts and the sheriff and the FBI, so they're effectively saying, you have to accept the level of protection we provide you in whatever way -- you aren't allowed to protect yourself. Now it seems to be one thing to say that you can't have a tommy gun in your house to protect yourself against burglars, and quite another thing to say that you can't have a really strong door. I think that's a much more similar analogy. There's also a very interesting precedent for this, I think, in the copy protection technology of things like digital audiotapes. In some sense, putting cop. y protection in digital audiotapes supports not only the interest of Columbia Records but it supports the interest of the black market. If I'm selling black market audiotapes which I copied from legitimate ones by building an illegal -- I don't even know if it's illegal, but you build from scratch -- a nonrestricting copier. Now I've copied copy-protected tapes. I mean, I don't want my customers copying their own tapes -- I want them coming to me. (laughter)
Insert Fig. 10-13 & caption "Figure 10-13"
So, what could we do? The answer is, I don't know. I have in mind a compromise (Figure 10-13), but I don't know how to do it. I think if you could guarantee accountability of the society, that is to say, you could say, data always could be subject to search warrants, but nobody could ever do it covertly. It would be like having perfect signet ring seals to go on envelopes so that if your letter was opened you would know that some warrant had been served and your letter had been opened. The trouble is, unfortunately, I have no idea how to do that. Thanks for your patience. (applause)
D. DENNING: OK, before opening it up to questions from the floor, I'd like to ask the panelists if any of them have very short comments that they want to make in response to each other said. John?
BARLOW: I'd just like to say that if people in the audience or if people who speak to people in the audience are interested in working on this and being part of the negotiations about the transfer of mass market software, talk to Jerry Berman of the EFF Washington Office, or John Podesta.
D. DENNING: Yes?
AUDIENCE MEMBER: I just had a very brief comment on something John Barlow said. He commented on the oddity of a secret cryptographic standard. I think in fact secret standards of various kinds aren't that uncommon. What's especially odd about this one is that this one's an ANSI standard, and they had to bend the rules very hard to get around the fact that it's ANSI rules th. at their standards have to be publicized. There is a wonderful story, but it's too long, the things it went through, but it's too long to give at the moment.
D. DENNING: OK. Well, we're open to questions from the floor.
AUDIENCE MEMBER: Knowing that both CFPs have made a concerted effort to provide a level playing field for discussion of controversial issues, I may have missed this in the beginning in your introduction, if so, I apologize. Why isn't anyone from the NSA, CIA, and the FBI on this platform?
D. DENNING: OK, that's a good question. I was not the person who actually organized the session, however, after I was invited to be chair of the session, I talked with the person who organized it, who's Marc Rotenberg, about that very issue. And Marc had already talked to some people in the government and hadn't gotten anywhere in terms of their interest in participating, so I called up somebody I know at the NSA. That person put me in contact with somebody else in the agency who would be in a position to make the decision, or maybe even be the appropriate person to have on the panel, and he explained to me that they would decline because they didn't feel that they could say the things in a public forum that they would need to say. He also, though, on the positive side of it, which I interpreted as somewhat encouraging, said he thought that maybe a year from now that we would see the NSA participating more in public discussions about this, that at this time it was premature. Now, as far as the FBI goes, I was under the impression that maybe Marc or somebody had approached the FBI as well, but I'm not exactly sure. I'd be delighted to have somebody from the FBI join us right now and either make a few comments or respond to questions that come up. It is certainly not an intention to exclude people from the government at all.
PANEL MEMBER: Oh, by the way, there is a representative from NSA here, she's Sandra Ray, and she's sitting in the audience taki. ng notes. (laughter)
D. DENNING: OK, you're next.
PANEL MEMBER: But your sponsors are taping. (laughter)
ERIC HUGHES: I'm Eric Hughes of DigiCash. I have a question for Jim Bidzos which is rather impertinent. We know from experiences of the last year that, as in the Soviet Union and Eastern Europe, governmental power is in large cases what people believe it to be. Now we know that Phil Zimmerman has challenged the legality of patent law in response to software patents by writing PGP -- Pretty Good Privacy. When will RSA Data Security simply export RSA technology out of the boundaries of the country and see what the government does? (applause)
BIDZOS: I'm not sure I see a relationship between the two things you said -- patent law and exporting cryptography don't...
HUGHES: They're both challenges to the existing form of legality that's been created by the government.
BIDZOS: Let me just understand your position -- you're anti- patent law. You wish there were no patents...
HUGHES: I don't have a particular position on this. DigiCash, as a corporate representative, has a patent on the blind signature, just like you have a patent on the RSA signature.
BIDZOS: There are lots of patents and lots of companies that have them. I'm not sure what your patent question is.
HUGHES: My question is really about export law -- that's my real question.
BIDZOS: I see. When are we going to ignore export law and ship RSA, say, out of the country?
HUGHES: As a form of civil disobedience made by a corporation rather than an individual. (hoots)
BIDZOS: Well, maybe I already have. (laughter and applause)
HUGHES: Are you willing to certify that with a public key signature? (laughter)
BIDZOS: A DSS signature? Absolutely. (laughter) Yes, I'll sign anything with one of those. (laughter and hoots) But I ... (interruption) Let me just repeat that maybe I already have, and if I did, I don't think it would be a good idea for me to admit it.
HUGHES: Well, the . whole point is to admit it and see what the government does. (laughter and hoots)
BIDZOS: Easy for you to say.
HUGHES: Cryptography is the civil liberties issue of the 21st century and we might as well start now.
BIDZOS: Now that I understand your question, let me answer it this way. We've taken the first step to doing that. We have made a piece of software called RSA REF, which is RSA and DES together -- not for the first time, but together again (laughter) -- available in source code form to anybody who wants it who is willing to acknowledge that they won't break the law. I mean, I don't care what you do, but you have to tell me that you're not going to break the law. So, all you have to do is send electronic mail to RSAREF-REQUEST at RSA.COM and you can take that step yourself, too.
HUGHES: From CWI.NL, I take it, as well.
D. DENNING: I'd like to go on to somebody else. Thanks. OK, Steve Cisler?
STEVE CISLER: My question is for anyone, including Dorothy. With a lot of federal agencies really searching around how to mold swords into plowshares, what do you think the NSA will be doing maybe in the next year that they won't be here?
BIDZOS: I have an answer, because I get asked this a lot. I have a view that may be different from what most people feel. I think NSA's budget is likely to be increased, and that is simply because if I were perhaps a Congressional representative or an NSA representative sitting in the meetings where money is appropriated, I think my question would be, well gee, now we've agreed to cut the defense budget by $50-100 billion over the next five or ten years, that means we'll have a leaner, meaner military and our response time would be longer. Who would like to be the first to vote against a modest increase in our ability to give ourselves more lead time? I don't think anybody would. So I suspect that if anything, NSA and their interests are likely to be as strong or stronger than they have in the past.
CISLER: Do you . think there'll be any kind of agreement such as the Energy Department just signed with the Computer Systems Policy Project? Sort of joint development of research?
BELLIN: If there is, I don't think we're going to know about it. (laughter)
D. DENNING: Vint Cerf?
VINT CERF: Hello, hello. I have a couple of things to mention, and one question to ask. First of all, I wanted to make sure before I ask my question that everyone understands I have a great deal of interest in making it possible for privacy-enhanced mail using various cryptographic techniques to be readily available essentially everywhere, at least throughout the Internet. So steps that move in that direction are in fact helpful. However, I want to ask you all to think a little bit for a moment before you decide to totally damn various parts of the US Government. A lot of those people who are out there who have to work in secret do so at great risk. And they do that to defend each of us. So before you decide that some of the policies that in fact go against our grain and our natural desire for openness, before you decide that those are completely wrong and unacceptable, I hope you'll give a little bit of thought to the people who go out there and defend us in secret and do so at great risk. We don't pay them for that, at least nothing except perhaps our taxes, but you know, you can't pay anybody to risk their lives. So I hope you'll give a little bit of thought to that before you decide those guys are all bad guys. Thanks. (applause)
GILMORE: I don't think we should destroy our privacy to protect it, the way in Vietnam we destroyed our villages to protect them. (applause)
D. DENNING: Richard?
AUDIENCE MEMBER: OK, thank you very much. We hear a lot of arguing about the First Amendment foundation for cryptographic speech. I'm beginning to wonder about the Second Amendment implications. We've heard several times on the panel here today, and I think some of us through other conversations . during the last couple of days, that if guns are outlawed, only outlaws will have guns, and by analogy if crypto is outlawed, only outlaws will have crypto. One of the purposes of cryptographic technique is to protect an individual from outside intrusion, either from others or perhaps from a government run amok, which is certainly an undercurrent that I've sensed here. Taking a look at some of the fundamental historical purposes of the Second Amendment to the Bill of Rights, I'm curious what the application of the Second Amendment would be to cryptographic protection. And I'll just throw that open to the panel. "The right to keep and bear arms shall not be abridged."
BELLIN: Maybe I'll take a first stab at that, since I was the first one to introduce the analogy today. I would reflect back on why the right to keep and bear arms was included in the Bill of Rights -- that might be one way to approach it. Was it to keep taxes low so you wouldn't have to have a standing army? Or was it as another mechanism of preserving what I talked about as the fundamental freedoms of our society which I see, more personally, in First Amendment terms -- to speak freely, the right to associate in order to affect the course of government, those whom we've turned over the reins to, to some extent? So, if that was the purpose, in other words, in order to prevent a British tyranny from coming back, or something along those orders, then we have a better parallel to what we're talking about in cryptographic techniques, I think.
BARLOW: I have an anecdote for you on this point -- this does keep coming up. When I was talking to Admiral Inman about this, he equated robust encryption with Saturday night specials. He said, "My answer would be legislation which would make it a criminal offense to use encrypted communication to conceal criminal activity." So I said, "Wouldn't that render all encrypted traffic somewhat suspect?" And he said, "Well, you could have a registry of institutions. which could legally use ciphers. If you get somebody using one who isn't registered, then you go after him." I said, "You can have my encryption algorithm when you pry my cold, dead fingers from its private key." (laughter and applause)
D. DENNING: Marc Rotenberg?
MARC ROTENBERG: I have an announcement, a comment, and a question. First, I am pleased to announce that the Second Conference on Cryptography Policy will be held in Washington, DC, on June 1, and is being jointly sponsored by CPSR, EFF, and RSA. Second, it seems that these issues are beginning to mushroom, which is to say that a year ago we were talking about S.266. Now we're talking about a vastly expanded proposal regarding the digital network. The next level, I suspect, has to do with the intelligence overhaul which is now being discussed in both the House and Senate. In particular, there are proposals to give statutory authority to the National Security Agency in the area of information security and communications security. The Senate bill gives authority to NSA for all communications security for the federal government; the House bill gives authority to the NSA for all information security for the federal government. My question, I guess, may be to John Gilmore: What should we anticipate as this clash between emerging technologies and protection of the national security state plays itself out? What are the next battles going to be over?
D. DENNING: Could I just make a comment first, and that is that my understanding in talking with Willis Ware yesterday was that there was a mistake in those bills that is being rectified. I'm not sure what the actual intent of the bills was, but the intent of the bills was not actually to give NSA more authority over security than they already had, and the Computer Security Act gives NIST authority for security in the commercial sector. There are some people from NIST here, so maybe they can correct me on this.
UNKNOWN PANEL MEMBER: Is Willis here? .
D. DENNING: Willis is not here any more. Is there anybody here who is more knowledgeable?
ROTENBERG: Dorothy, let me just say I've read both of the bills -- it's S.2198 and it's H.R.4165, section 314...
D. DENNING: No, they were clearly written, but I guess the question is they were written without somebody having done the right research, and the research having been done, the plan is to change that. Can you speak to that?
WAYNE MADSEN: I'm Wayne Madsen. What I heard yesterday was that they said inadvertently some staffers who had prepared the legislation used some old language because they weren't up on what was going on here now. I'll leave that up to you to believe or not, but they're saying mea culpa.
D. DENNING: Anyway, I guess the question was actually directed to John, so let's let John go next, and then Whit, if you want say something, too.
GILMORE: I think the question is just sort of, where's the next battleground going to be? Overall, I think that the battleground is on what are we losing by following these cold war policies. It's clear that there was no reason to doubt what the NSA said over the last 40 or 50 years, because there was no harm if you followed their regulations. Maybe 10 people got hurt, or 20 people got hurt that year, and you could afford that for national security. But when the privacy of millions of people who have cellular telephones and the integrity of our computer networks and our PCs against viruses are up for grabs here, then I think the battleground is going to be counting up the harm and in the public policy debate trying to strike a balance. The real challenge there will be to get the people who can show harm on the other side, who can show harm to our national security by freeing the technology, to speak up and tell us what this harm is. They're so unused to having to defend the need for it that I'm afraid they will just sort of attempt to push it to the wire, and whether they win or lose is not the issue..
The result will be not what's best for the country, just because we don't know what's best for the country, because we can't evaluate both sides of the equation. (applause)
D. DENNING: OK, and then Whit?
DIFFIE: Marc, I'm not so convinced that's a bad thing. My experience of NSA is that the people who work in COMSEC seem to me really have the security of American communications as their interest. And it's not obvious to me that a group of the sort we've had for the last several years, without particular resources in the Department of Commerce, is any less subject to the influence of the offensive side of NSA -- which might want some compromise for the sake of intelligence sources -- that that's any less subject to influence than the COMSEC side of NSA is to influence. There was originally scheduled the STU-3 program would have supplied Type 2 STU-3s rather freely within the United States. At least the excuse that is given for the program having been changed to where now only approved people with law enforcement, big companies, etc., can get them, is that the Computer Security Act allegedly deprived them of the authority to have such a broad program with influence over the civil sector.
GILMORE: It'll be quick. I think there's no problem with having the computer security people at NSA do the work. The problem is to move them out from under the NSA umbrella so that they are subject to the FOIA, they are subject to all the reviews and all the other constraints that every other government agency is subject to.
D. DENNING: And Jim Bidzos wanted to make a brief remark, too.
BIDZOS: Marc, I think this legislation is somewhat unique. I mean, this basically says, in the interest of law enforcement, give me the right to do whatever I need to and accuse whoever I need to of breaking the law. It could be abused. There's no question about that. And I think people are a lot smarter today, and they're saying, "Prove to us that there's some protection in here f. or law-abiding people before we implement this thing." This is a wonderful form of government, but there's no way to ensure that we don't elect corrupt officials, and history is full of examples where that's happened before, so these things tend to be abused. The language in S.266 in particular, and also in this digital telephony bill, seems to me to just open the door to possible abuse. An example is in Cincinnati, Ohio. For about five or six years running, the local police and Cincinnati Bell worked together -- colluded -- to make literally thousands of illegal wiretaps. It happens all the time. And all we're saying is, "Gee, we're smarter these days, you know, we're not so dumb -- we're not going to just take this thing the way it is. Why don't you prove to us that it can't be abused?" We're not willing to trust the government any more. So maybe the next battlefield ought to be the next election -- let's make this a major issue and let's lobby. (applause)
D. DENNING: Ross Stapleton?
ROSS STAPLETON: Ross Stapleton, Central Intelligence Agency, and I have to say up front I don't speak for the CIA and this is just me. I can tell you I did my doctoral work studying Soviet-East European personal computing. I have seen export controls and all that close at hand, and actually kicked the tires and things and all that. I can say I agreed nearly 100% with what Mr. Diffie said, up until he said something that surprised me, in that this room didn't shout it down. That was when he said information is less dangerous than physical things. Good God! If you believe that, I'll give you a choice. I can go to your school district and give out one hit of PCP, or I can cover the area with instructions on how to make it. All I'd ask you to keep in mind is to have some sympathy for the foreign policy-niks who know that, in a sort of frustrated air, when it's hard to move information around, it's unlikely that someone can even get an atomic bomb plan, despite the fact tha. t we've got tens of thousands in both the former Soviet Union and in the United States. But I would fear someone giving out the Princeton dissertation and broadcasting it over the nets to all and sundry in that form -- now, given plutonium, we can make a bomb. So, information is a dangerous thing, in the right hands. I think we're all selling ourselves short if we think information is an unempowered commodity.
All that said, I have to agree with everything else. Cryptography is not magic, it's math, and DES is not only here, it's on a server in Helsinki, so we have to live with the fact that information moves around. We may want to be sympathetic to the fact that there are people, in fact people without the tools that we all have here, trying to enact the current and past foreign policy. Help educate them, help tell them why these things are happening, but realize that we're disrupting a lot of things real fast.
BARLOW: The problem with trying to regulate the flow of information, which is a little like trying to regulate the flow of wind, is that it's quite possible to keep it out of the hands of individuals and small institutions; it's very difficult to keep it out of the hands of large institutions. So you have, in effect, a situation where the Soviets are using RSA in their large codes, and have for a long time, and yet we can't use it as individuals in the United States. And that's just dumb.
BIDZOS: My revenue forecasts are being revised downward, by the way. (laughter)
BARLOW: You weren't getting royalties on that anyway, were you? (laughter)
BIDZOS: Maybe. (laughter and applause)
D. DENNING: John Podesta?
JOHN PODESTA: I have no speech, but I actually have two somewhat unrelated questions, if you will bear with me. Whit Diffie talked about using encryption to enhance the capacity to audit, but in the case of the cellular standard, I think the NSA spent most of their time worrying about encryption used in the signaling channel, rather than . the voice channel on cellular, because they wanted the capacity to audit or do traffic analysis, transactional analysis. I wanted to ask Whit whether the use of encryption can enhance privacy in that area, which is to eliminate, or build up privacy on transactional analysis. Secondly, to Jim Bidzos, you talked about going from a situation where government and industry are fighting with one another to one in which there's more cooperation, and I wanted a comment on whether you thought the SPA negotiation with NSA was of the latter category.
DIFFIE: My answer is that cryptography can certainly be applied in such a way as to protect transactions from auditing, that is in effect what electronic money does -- it creates anonymous transactions. On the other hand, when you want that effect, it can be used to enhance the auditability of a sequence of transactions.
BIDZOS: Let me try to set the record straight on this SPA business -- I'm not a member of SPA, I haven't been part of these negotiations. Apparently someone at the companies involved has a fondness for some of our DES alternatives and uses them. These alternative algorithms that we're in the business of designing and selling have variable key sizes. We give them to our customers who choose a key size suitable to their application and go and attempt to get their complete package exported. I play no role in the process once they buy these particular algorithms from us. I've only recently -- very recently -- heard about this business in SPA. I have no detailed knowledge of what's going on, what's been negotiated, and what the key sizes are, so I'm afraid I really can't answer your question without understanding all the details. But, I mean, if it's a secret meeting, it doesn't sound very good.
PODESTA: You would at least support John's notion that it ought to be put out in the open and opened up to public discourse, right?
BIDZOS: Well, I support my own notion that one ought to learn a lot about some. thing before one comments on it. (laughter)
D. DENNING: OK, I guess the clock just ran out and Lance said he had a couple of comments he wanted to make before the break.
HOFFMAN: Well, that clock says zero, but don't go away, you people who are at the microphones. I have a couple of comments. This says 2:56, so let's take two or three more...
D. DENNING: OK. Two more?
AUDIENCE MEMBER: I have no questions, but I have two comments. Briefly, it seems to be clear-cut lunacy for DES to be spread all over the world and for us not to be able to export it legally. It seems lunacy to me for public key crypto to be published worldwide and be used worldwide while US citizens cannot use it for communication with the rest of the world that already uses it. And it simply makes the government look dumb. So why don't they let it go? Just let go. OK, that's one comment.
Second, I have a lot of sympathy for law enforcement officers who are having to face the DDR&M -- that is, drug dealers, rapists, and murderers. But it is a serious comment, I do have a lot of sympathy for them. They are under severe duress from us, the law-abiding citizens who want protection from these serious crimes and this serious danger. Maybe we should move to another society where it's OK to pry information out of suspects' brains by beating them until they tell us. That's done all over the world. Why don't we do it? We decided that was going too far. Maybe we need to decide that the protection of communications between presumed innocent individuals supersedes the need of law enforcement for the ability to conduct surveillance and investigation on information and communications between any individuals that they want to when authorized by law, or by political process, or by whoever's in power, or by whatever other means there is, authorized or unauthorized. We as a society really need to make a decision whether we want everybody protected, even those damn DDR&Ms, or whether we want every. body endangered. Thank you.
HOFFMAN: I have a couple of announcements before the break. One -- plans are underway to establish an Illinois Privacy Council. Interested people should contact George Trubow -- stand up, anybody who hasn't seen George, this is who he is - - 312-987-1445. No 800 number. Second announcement -- some of you may not have had a chance to look at the followup program to the initial brochure. The last session at 3:15 -- in fifteen minutes -- it's going to be dynamite. Mara Liasson from NPR is going to chair, and the panelists are Peter Denning from George Mason University, Mitch Kapor from Electronic Frontier Foundation, Simon Davies from Privacy International, Roland Homet from Executive Ink, and Esther Dyson from EDventure Holding, so it should be quite a session. I look forward to seeing you all there. Finally, Bruce Koball has a very quick announcement to make.
KOBALL: For the folks that weren't here this morning when I was formally drafted with my arm twisted behind my back into chairing CFP-3, I'd like to make a request that people that have an interest in participating in CFP-3, which looks like it's going to be held in San Francisco next year at around this time. Please talk to me before you leave here and give me some of your ideas. We hope we can include you in the program planning process. Thank you. (applause)
HOFFMAN: Thank you, Bruce, and finally, Molly Raphael reminds me, if you have not filled out your evaluation form, this might be an interesting time to consider it and drop it off at the registration desk. Thank you very much to Dorothy and the panel for a very interesting discussion. (applause).
