0001 1 TRANSCRIPT OF PROCEEDINGS 2 BEFORE THE 3 TEXAS LOTTERY COMMISSION 4 AUSTIN, TEXAS 5 6 REGULAR MEETING OF THE § TEXAS LOTTERY COMMISSION § 7 WEDNESDAY, SEPTEMBER 20, 2006 § 8 9 COMMISSION MEETING 10 WEDNESDAY, SEPTEMBER 20, 2006 11 BE IT REMEMBERED THAT on Wednesday, 12 the 20th day of September 2006, the Texas Lottery 13 Commission meeting was held from 9:00 a.m. to 14 12:58 p.m., at the Offices of the Texas Lottery 15 Commission, 611 East 6th Street, Austin, Texas 78701, 16 before CHAIRMAN C. TOM CLOWE, JR., and COMMISSIONER 17 JAMES A. COX, JR. The following proceedings were 18 reported via machine shorthand by Aloma J. Kennedy, a 19 Certified Shorthand Reporter of the State of Texas, 20 and the following proceedings were had: 21 22 23 24 25 0002 1 APPEARANCES 2 CHAIRMAN: 3 Mr. C. Tom Clowe, Jr. 4 COMMISSIONER: Mr. James A. Cox, Jr. 5 GENERAL COUNSEL: 6 Ms. Kimberly L. Kiplin 7 EXECUTIVE DIRECTOR: Mr. Anthony J. Sadberry 8 CHARITABLE BINGO EXECUTIVE DIRECTOR: 9 Mr. Billy Atkins 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 0003 1 TABLE OF CONTENTS 2 PAGE NO. 3 AGENDA ITEM NO. I - Meeting Called to Order...... 6 4 AGENDA ITEM NO. II - Report, possible discussion and/or action on lottery sales 5 and revenue, game performance, new game opportunities, market research and trends......... 77 6 AGENDA ITEM NO. III - Report, possible 7 discussion and/or action on transfers to the State and the agency budget................... 104 8 AGENDA ITEM NO. IV - Report, possible 9 discussion and/or action on statistical analysis of the Mega Millions megaplier 10 feature........................................... 6 11 AGENDA ITEM NO. V - Consideration of and possible discussion and/or action on 12 external and internal audits and/or reviews relating to the Texas Lottery Commission 13 and/or on the Internal Audit Department's activities........................................ 107 14 AGENDA ITEM NO. VI - Report, possible 15 discussion and/or action on the agency's contracts......................................... 108 16 AGENDA ITEM NO. VII - Report, possible 17 discussion and/or action on the 79th Legislature....................................... 108 18 AGENDA ITEM NO. VIII - Commission may meet in 19 Executive Session: A. To deliberate the duties and 20 evaluation of the Executive Director and/or Deputy Executive Director 21 pursuant to Section 551.074 of the Texas Government Code 22 B. To deliberate the duties and evaluation of the Internal Audit 23 Director pursuant to Section 551.074 of the Texas Government Code 24 25 0004 1 TABLE OF CONTENTS (continued) 2 PAGE NO. C. To deliberate the duties and 3 evaluation of the Charitable Bingo Operations Director pursuant to 4 Section 551.074 of the Texas Government Code 5 D. To deliberate the duties of the General Counsel pursuant to Section 6 551.074 of the Texas Government Code E. To receive legal advice regarding 7 pending or contemplated litigation and/or to receive legal advice 8 pursuant to Section 551.071(1)(A) or (B) of the Texas Government Code 9 and/or to receive legal advice pursuant to Section 551.071(2) of the 10 Texas Government Code, including but not limited to: 11 Cynthia Suarez v. Texas Lottery Commission 12 Shelton Charles v. Texas Lottery and Gary Grief 13 Stephen Martin vs. Texas Lottery Commission 14 Employment law, personnel law, procurement and contract law, 15 evidentiary and procedural law, and general government 16 law Mega Millions game and/or 17 contract F. Return to open session for further 18 deliberation and possible action on any matter discussed in Executive 19 Session ................................ 112 20 AGENDA ITEM NO. IX - Report by the Executive Director and/or possible discussion and/or 21 action on the agency's operational status and FTE status................................... 114 22 AGENDA ITEM NO. X - Report by the 23 Charitable Bingo Operations Director and possible discussion and/or action on the 24 Charitable Bingo Operations Division's activities...................................... 115 25 0005 1 TABLE OF CONTENTS (continued) 2 PAGE NO. 3 AGENDA ITEM NO. XI - Public Comment............. 116 4 AGENDA ITEM NO. XII - Adjournment............... 116 5 REPORTER'S CERTIFICATE.......................... 117 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 0006 1 P R O C E E D I N G S 2 WEDNESDAY, SEPTEMBER 20, 2006 3 (9:00 a.m.) 4 AGENDA ITEM NO. 1 5 CHAIRMAN CLOWE: We'll come to order, 6 please. Good morning. Today is September the 20th, 7 2006. It's 9:00 a.m. Commissioner Cox is here. My 8 name is Tom Clowe. We'll call this meeting of the 9 Texas Lottery Commission to order. 10 AGENDA ITEM NO. IV 11 CHAIRMAN CLOWE: And we have some 12 outside appearances today. We'll go immediately to 13 Item IV on the agenda, I believe which covers that 14 subject: Report, possible discussion and/or action on 15 statistical analysis of the Mega Millions megaplier 16 feature. 17 Mr. Anger. 18 MR. ANGER: Good morning, Mr. Chairman, 19 Commissioner. For the record, my name is Michael 20 Anger, and I'm the Lottery Operations Director. 21 Commissioners, you may recall that there 22 was public comment at the June 28th Commission meeting 23 offered by Prof. Gerald Busald relating to his 24 analysis of the results of the megaplier feature of 25 the Mega Millions game since its introduction in 0007 1 December of 2003. I will not attempt to speak for 2 Prof. Busald. And I'm pleased to note that Prof. 3 Busald is in the audience, so I'm sure he can speak 4 for himself. 5 However, I believe it is fair to say 6 that he expressed concerns that he felt that the 7 megaplier might be operating in a statistically biased 8 manner. At that meeting, I offered to share Prof. 9 Busald's analysis with Dr. Randy Eubank, who is 10 joining me here at the table this morning. Dr. Eubank 11 is the agency's vendor for statistical consulting 12 services. 13 By way of background, Dr. Eubank is a 14 Ph.D statistician with over 25 years of work 15 experience in the area of statistics. He currently 16 works as a professor in the Department of Mathematics 17 and Statistics at Arizona State University. 18 Commissioner Cox, you had also suggested 19 that we arrange a direct discussion between Prof. 20 Busald and Dr. Eubank to discuss the results. I 21 arranged for Dr. Eubank to review the information 22 provided by Prof. Busald, and I participated in a 23 conference call with both parties on July 6th. 24 Without speaking for Prof. Busald or 25 Dr. Eubank, it was my perspective of the discussion 0008 1 that the parties agree on the results of the analysis 2 performed on the data that Prof. Busald presented in 3 his public comment during the June 28th Commission 4 meeting. However, there may not be complete accord 5 between the parties as to whether there is reason to 6 be concerned with those results. 7 Dr. Eubank has refreshed his analysis of 8 the megaplier to include results that have occurred 9 through August 31st of this year. Dr. Eubank is here 10 today to present his current analysis and results and 11 also to share his assessment of the significance of 12 these results and what they might tell us with regard 13 to the performance of the megaplier. 14 And with that, I'll turn the things over 15 to Dr. Eubank to present his information. 16 CHAIRMAN CLOWE: Just a minute before 17 you start. 18 Any comments at this point? 19 COMM. COX: No, sir. 20 CHAIRMAN CLOWE: Mr. Anger, it will be 21 our intention to hear from Dr. Eubank first -- he has 22 filed a witness affirmation form which I have received 23 first -- and then Prof. Busald. And that way, I think 24 we can hear from both of these gentlemen and ask them 25 questions. 0009 1 If that's agreeable with you? 2 COMM. COX: Of course. 3 CHAIRMAN CLOWE: Thank you, Dr. Eubank. 4 DR. EUBANK: For the record, my name is 5 Randy Eubank. I'm a Professor of Mathematics and 6 Statistics at Arizona State University -- and former 7 Aggie. 8 Good morning. It's nice to be here. 9 It's always nice to get out of Arizona for a few days; 10 in fact, the heart of the world here in Austin. 11 What I want to do today -- or what I've 12 been asked to come here and do today is to talk a 13 little bit about the analyses that I do on the data 14 that's produced by the automatic drawing machines that 15 produce the numbers that are drawn for the megaplier 16 for the Mega Millions game. In the process of doing 17 that, I also want to talk about some of the concerns 18 and issues that have been raised in public testimony 19 before the Commission. 20 So let me start off -- by the way, I 21 have been teaching statistics since the seventies, so 22 I don't really feel good unless people interrupt me at 23 some point in time. So if anybody has any questions, 24 please just stop me and you'll make me feel like 25 somebody is listening. Teaching undergraduates 0010 1 statistics, that's not often. 2 All right. So the first thing is, let 3 me give just sort of a brief statement about sort of a 4 state of the megaplier address for a moment or two. 5 I'm going to say some things which may seem a little 6 bit obscure, but the goal of this presentation is to 7 come back and make these things make sense. 8 I just finished analyzing the results of 9 the megaplier machine output as of August 31, 2006. 10 Now, what comes out of these machines is a whole lot 11 of information. There are pre-tests that are done 12 before every draw, 250 pre-tests that are done before 13 every draw is done. A machine is selected at random. 14 There are different random number generators that live 15 under the hood of these automatic drawing machines 16 that are selected at random, lots of information. So 17 there's something like about 70,000 values just in 18 terms of the pre-test alone that I look at. So 19 there's a lot of things that I look at here, and I do 20 diagnostic analysis on all of that. 21 In terms of the things which have 22 appeared before this Commission in testimony is 23 focused on really sort of one small part of that data. 24 So let me sort of address that today. And the 25 conclusion is that in terms of the analyses that I 0011 1 have done, I don't see any issues or any problems 2 whatsoever in terms of the way these ADMs are 3 performing. In terms of the specific thing that has 4 come up before the Commission has to do with the 5 numbers that are actually drawn for the megaplier -- 6 2, 3 or 4. So let me address that specifically. And 7 I'll make a statement here that seems a little 8 obscure, got some buzz words in it, but I'll hopefully 9 clear all that up as I continue to talk. 10 So the first thing that I'll say here is 11 that a test of the null hypothesis that the megaplier 12 values of 2, 3 and 4 are being selected with the 13 correct frequency. That means 2/21 -- 2/21's, 7/21's, 14 12/21's. Okay. That's the fractions they should be 15 selected with. A test of that hypothesis would be 16 significant at the 23 percent level. 17 Now, there's a number of words here that 18 people that haven't seen statistics wouldn't 19 necessarily know -- null hypothesis, test of 20 significance level. I want to make those things clear 21 in terms of the material that I'll discuss today. But 22 as just a brief sort of preview of coming attractions, 23 let me say that any statistician that would do -- any 24 credentialed statistician that would look at that, if 25 you said, "Well, it's significant at the 23 percent 0012 1 level," would say, "Well, there's nothing going on. 2 Everything looks fine." As we'll come to talk about 3 this eventually, the question is, we expect to see 4 something like this, and this says you would see it 5 about 23 percent of the time. Things that happen 6 23 percent of the time happen all the time. 7 I want to talk a little bit about 8 hypothesis testing today. And there's three aspects 9 of statistics. Hypothesis testing is one of the 10 three, one of the big three. There's a lot of terms 11 that pop up in here. It's just like any science, any 12 field, even the lottery, everybody has some. 13 ADM is used for automatic drawing 14 machines, and it becomes second nature to use these 15 words if you're familiar with them. Well, in 16 hypothesis testing, some of the words that come up are 17 "null hypothesis," "decisional rule," "level of 18 significance" and "P-value." So I want to try and 19 talk about these words. Just like any sort of buzz 20 word that appears in science or any field, they've got 21 basic sort of common sense things that they tie back 22 into. 23 And one of the best ways that I've 24 always found to tie these things back into something 25 that we can all connect with is to make an analogy to 0013 1 a trial by jury, because hypothesis testing is 2 really -- it's sort of a scientific parallel to trial 3 by jury, or maybe trial by jury is a scientific 4 parallel of hypothesis testing, depending on how you 5 want to think about it 6 Well, I have been teaching statistics 7 since, oh, sometimes in the late seventies. And the 8 way I have probably always done this is, I have always 9 made connections to trial by jury. I'm sure I'm not 10 the one that thought about it. I haven't found 11 anything that I discovered uniquely it seems like 12 applies. I'm still waiting. But I'm sure somebody 13 taught that to me. 14 But you might say, "Well, you know, 15 maybe this is just sort of Randy's unique, sort of 16 quirky way of thinking about things." So I decided, 17 "Well, I'm going to put in "hypothesis testing, trial 18 by jury" into Google and see what comes out the other 19 side. Well, I had 273,000 hits. Now, as you know, I 20 get paid on an hourly basis, so you'll appreciate that 21 I withstood the temptation to read all 273,000 of 22 these things. So the State of Texas probably owes me 23 a cup of coffee at some point in time. 24 But I did look at some of them. I did 25 look at the first few. And what the conclusion is 0014 1 that I drew from that is that if you go and look at 2 some of these, you'll see very much the same thing 3 that I'm going to talk about today. So this 4 information is certainly out there, and it's not just 5 on the Web. You can find it in a classroom. There 6 are a number of books that -- there are many, many 7 books that are used in elementary statistics. There 8 are lots of these things. 9 And, again, it would be an effort that 10 would take a long time to go through every one of 11 them. But I went through a few of these things to see 12 what sort of connections they have into this trial-by- 13 jury sort of point of view. Is this unique or is it 14 sort of common? And I found a lot of these textbooks 15 where you can find this sort of thing. And, you know, 16 statistics is taught in every type of setting. It's 17 taught in the big universities, it's taught in the 18 community colleges 19 And here are a couple of books there 20 that I found. One book that I really liked is a book 21 by Prem Mann. And Prof. Busald know that because he 22 uses that book in his Math 1442.001 stat class that he 23 teaches at San Antonio College. And the Prem Mann 24 book, honestly, I very much like this book in terms of 25 what it does for hypothesis testing. It very much 0015 1 parallels the way I think about the subject. And so 2 I'm going to quote from this thing frequently, or I'll 3 indicate that I'm borrowing from it frequently as I go 4 along. The point to be made here, I think, is that 5 this is not a unique point of view. It is a standard, 6 maybe even proper way to think about the concept of 7 statistics. 8 All right. Well, the key word here that 9 gets us all started is the concept of the null 10 hypothesis. And the simplest way of thinking about a 11 null hypothesis is a statement about probabilities, 12 some sort of a model for the way probabilities, 13 associated with some real-world event, happened. 14 In the context of the megaplier, 15 selecting the megaplier numbers, a null hypothesis, 16 specifically the one that we have been testing here, 17 is at the integers 2, 3 and 4 are being chosen at 18 random. That's part of the hypothesis; they're being 19 chosen at random with the probability 2/21, 7/21 and 20 12/21, respectively. So that's our null hypothesis. 21 Now, I want to connect this into trial 22 by jury. And the way we'll think about this from that 23 perspective is that the null hypothesis is basically 24 the person on trial. Trial by jury is a complicated 25 process. There's lots of things involved in it, but 0016 1 I'm going to break it down to a very sort of simple 2 thing in terms of the way I want to think about it 3 today. And let's just do sort of a barebones sketch 4 of what trial by jury is all about. 5 It starts off with a defendant claiming 6 that they're innocent. The prosecutor collects 7 evidence, with the goal being to prove guilt or 8 convict the defendant. And then both of those people 9 sort of step away. And the verdict is rendered by an 10 impartial third party or a jury. The jury takes in 11 all the evidence, all the information, but supposedly 12 is not influenced directly in the sense that nobody is 13 twisting their arm to decide one way or the other. 14 They try to make an impartial call on the basis of the 15 evidence. 16 And here is a quote out of this book by 17 DeVeaux, Velleman and Bock that I like. It says, "The 18 jury considers the evidence in light of the 19 presumption of innocence and judges whether the 20 evidence against the defendant would be plausible if 21 the defendant were in fact innocent." Could these 22 things have happened if the defendant were innocent? 23 Another way of saying this is innocent until proven 24 guilty. 25 Now, what could happen? Well, the fact 0017 1 is that the person is either -- the actual situation, 2 the person is either guilty or innocent. One of these 3 two things is true. The verdict can be either not 4 guilty or guilty. And we don't make our call, unlike 5 on the late night Perry Mason reruns where the person 6 finally breaks down and says, "I did it." In the real 7 world, that usually doesn't happen. So we had to make 8 our call, as the people in the jury, they have to make 9 their call on the basis of incomplete information. So 10 their decision can be wrong. 11 Now, if the person is not guilty and the 12 Court's decision is that the person is not guilty, 13 that's good, the processes worked. If the person is 14 guilty and the verdict is that the person is guilty, 15 that's a good decision and the processes work. But 16 there are two other options. One is that the person 17 is guilty and the verdict is that the person is not 18 guilty, so now a guilty person gets to walk out the 19 door. And the last option, and the one that's most 20 important in terms of the way the judicial system 21 works is that the person is not guilty but that the 22 verdict is that the person is guilty, so that means 23 that an innocent person is sent to jail. 24 The types -- we call these things -- 25 I'll call them Type I and Type II errors. And, as we 0018 1 know, the Type I error is what we're worried about. 2 The goal is to not send any innocent people to jail, 3 and that goes back to the statement I made a minute 4 ago. So the whole process is done under the 5 presumption that the person is innocent. So you ask, 6 "Could this have happened if the person were really 7 innocent, if there was no guilt?" 8 And the catch phrase that comes out of 9 all of this is that you want to come back with a 10 guilty verdict only if evidence indicates guilt beyond 11 a reasonable doubt. Now, that doesn't mean beyond a 12 doubt because if it was beyond a doubt, then nobody 13 would ever get convicted. It's beyond a reasonable 14 doubt. So you can say, "Well, there is some doubt," 15 but it has to be a reasonable doubt. And the fact 16 that there can be a reasonable doubt, which is defined 17 in the judicial system itself, but the fact that there 18 can be a reasonable doubt is what makes it possible to 19 have errors. We all know that innocent people have 20 been sent to jail and that guilty people have been let 21 free. 22 Now, how does this relate to the 23 hypothesis testing thing? So let me bring that phrase 24 back to the subject at hand. Well, as I said at the 25 outset here, the person on trial here, the parallel of 0019 1 hypothesis testing is the null hypothesis. That's the 2 model that we're testing. Or, in the context of the 3 megaplier that 2, 3 and 4 are occurring with the 4 probability of 2/21, 7/21 and 12/21. That is just a 5 case in point. 6 The prosecutor, the person that plays 7 the prosecutor role in this is us, the people that are 8 analyzing the data or, if you want to point the 9 finger, the finger at me, the statistician who is 10 actually analyzing the data. Now, I don't get to 11 fiddle with the data; I don't get to tamper with the 12 evidence. I step away from it. I collect the data, 13 put it in the situation where it can be processed and 14 then I hand it on to an impartial third party, 15 something called a decision rule. A decision rule is 16 just a numerical rule that processes the data in a 17 certain way and returns values which tell me whether 18 or not the null hypothesis should be rejected. 19 So it's an impartial third party that 20 looks at the data without any direct arm-twisting from 21 me to make a call one way or the other. And the rule 22 that's been brought up here or the decision rule 23 that's been talked about in this forum in the past is 24 what we call the chi-squared test. I want to talk 25 about that more in a minute. 0020 1 All right. Well, what can happen? 2 Well, it's exactly the same situation as trial by 3 jury. Okay? In the real world, either the null 4 hypothesis is true or it's false. The person is 5 either innocent -- either not guilty or guilty. And 6 our decision is either to not reject the null 7 hypothesis, come back with a not guilty verdict, or 8 reject the null hypothesis, come in with a guilty 9 verdict. 10 The same sorts of errors can happen. So 11 you can have a Type II error, which means that you're 12 going to let a false hypothesis get away from you, or 13 you're going to let a guilty person walk out the door. 14 But what you're worried about is rejecting a true null 15 hypothesis or sending an innocent person to jail. So 16 we worry about a Type I error. 17 So just like the legal system, we worry 18 primarily about Type I errors, we're willing to let 19 some false hypotheses get away from us, let some 20 guilty people walk out the door in order to be sure 21 that we don't make any mistake, sending any innocent 22 people to jail. 23 Now, unlike trial by jury where the idea 24 of a reasonable doubt is determined essentially by the 25 judge -- that's where you sort of get that information 0021 1 from -- we don't have a judge here in this setting. 2 And that is where you get a little bit of difference 3 in that I actually -- as a statistician, I'm at 4 liberty to choose my level of reasonable doubt. But 5 you can't choose it to be anything you want to because 6 there's standards that are out there in this industry. 7 This industry has been around for a long time, and 8 there are standards out there that everybody sort of 9 goes by. 10 And the standard hypothesis level -- by 11 the way, so the probability of a Type I error, the 12 chance of a Type I error, the chance of sending an 13 innocent person to jail is something called a level of 14 significance of the test. So that's the phrase, 15 "level of significance," that I talked about earlier. 16 And the standard choices for this level of reasonable 17 doubt, level of significance, are 10 percent, 18 5 percent and 1 percent. 19 Now, that's what I've been teaching for 20 years. You find it in all the textbooks. But just to 21 prove to myself that I wasn't the only one that ever 22 thought about this, I put it into Google again, my 23 favorite source for all types of information now, some 24 of it -- well, some of it that I would probably be 25 better off not knowing. But in this case I know it's 0022 1 right because it came back from Wikipedia. And if 2 it's on Wikipedia, it's got to be good. And this was 3 probably the first one or two hits. And it says -- 4 it's a quote -- "Popular levels of significance are 5 10%, 5% and l%." 6 Now, the choice that we make for level 7 of significance, that's something I get to choose. So 8 how do I decide do I want 10 percent, 5 percent, 9 1 percent? Or do I want to be a maverick? Maybe I 10 want to put in 50 percent, be wrong half the time. 11 I'm an Aggie. I mean, maybe that's an okay thing to 12 do if you're from Aggieland. 13 But how do you come up with this? Well, 14 it will depend on the context of what you're doing. 15 So to sort of mix up some things, some metaphors in a 16 certain sense here, I have friends who do consulting 17 with lawyers on like discrimination cases, for 18 example. Well, in that setting, 5 percent is sort of 19 the standard, that's what you go by. So if it's 20 significant at 5 percent, then you talk about it. 21 If I was doing something like -- let's 22 suppose that we have two drugs that we're looking at. 23 We have a standard drug that we use to treat some 24 illness and we have a new drug that's come on the 25 market. And the question is being asked, the null 0023 1 hypothesis that we want to test is that the new drug 2 is no better than the old one. Well, what happens if 3 we reject that null hypothesis? We start using a new 4 drug in contrast to the old one. 5 And if we make a Type I error here, if 6 we wrongly reject, then we're using a new drug that's 7 not as effective as the old one. If it's something 8 for just, you know, a case of the sniffles, then maybe 9 it's no big deal, no biggie, a few more people have 10 sniffles than before. But if it's for some serious 11 life-threatening illness, then that's problematic. 12 We've gone away from something that we understand that 13 works to something that doesn't work as well. So in a 14 case like that, maybe you want to use 1 percent or 15 maybe even half a percent, something where you're sure 16 that you don't go away from something that you know 17 and you understand. 18 Finally, what are the consequences here 19 for the lottery? And not just in the context of the 20 megaplier but in any of the other settings here where 21 I do the statistical analysis, where I'm looking at 22 the ball sets, the machines and so on and how they're 23 performing. What are the consequences in terms of a 24 Type I error here? 25 Well, what does it mean? It means that 0024 1 if there is an issue that comes up, is there going to 2 be additional testing? In the case of the megaplier 3 machines, they were -- I believe Robert and Michael 4 know better than me -- but I think they were sent out 5 to Battelle originally and were tested there, or some 6 portion of the code or whatever was tested, or the 7 machines or ball sets were placed. 8 Every one of these things has a 9 financial consequence. They don't handle them for 10 free. If you ask Battelle to do something, they're 11 going to charge -- they don't -- as nice a bunch of 12 people as they are at Battelle, they're not going to 13 do it for free. And the same -- I would say the same 14 thing is true for me. 15 So all these have financial consequences 16 and there is no free lunch. If you make a decision on 17 these sorts of things, somebody has to pay up. So a 18 Type I error, sending an innocent person to jail, 19 wrongly rejecting a null hypothesis or moving away 20 from the thing that you know and understand has 21 financial consequences that affect the bottom line in 22 terms of revenue of the lottery. So that's the sort 23 of thing that I -- that's the sort of thing that I 24 keep in mind when I'm looking at these things. 25 So let's go back to the problem at hand. 0025 1 Again, I said for the megaplier data, there's lots of 2 information that comes in here. But let's focus on 3 just this one thing, the numbers 2, 3 and 4 that are 4 picked by the ADMs. So let's go through the whole 5 process of actually testing this hypothesis, now 6 thinking about it from the point of view of this trial 7 by jury. 8 So we've got megaplier values 2, 3 and 9 4. The appropriate null hypothesis here is that the 10 numbers 2, 3 and 4 are being selected at random with 11 the probability of 2/21, 7/21 and 12/21. All right. 12 Now, at this point in time -- as of August 31st, I 13 should say -- there have been 286 drawings. And if 14 two occurred with the probably of 2/21, then you 15 should see 286 times 2/21, or about 27 twos come up, 16 about 95 threes and 163 fours. 17 So the first question that comes to mind 18 possibly is, "Well, why don't these numbers match up 19 exactly? Why haven't I seen exactly 27 twos?" That 20 would not be a good thing. Okay? And here is the 21 reason why: So we don't want these things to match up 22 too slavishly. We don't want them to always be 23 exactly the same. And the reason why is because it 24 speaks to the randomness part of this. We don't want 25 things to occur in a predictable manner. 0026 1 As a simple example, let's suppose that 2 we have had 285 drawings, and you know that the 3 numbers have to match up exactly. And after 285 4 drawings, we have seen 95 threes and 163 fours. 5 What's the next number going to be? 6 COMM. COX: Two. 7 DR. EUBANK: That's right. Okay? So I 8 know with probability one what it's going to be. And 9 if that's true, then that means I just predicted it, 10 and I want it to not be predictable. So if I don't 11 want them to match up exactly, then what is it I want 12 them to do? Well, I want them to fluctuate around 13 these expected values in some sort of a random way. 14 Too much fluctuation certainly wouldn't be good. 15 How much is too much? Well, that's for 16 the jury to decide, and that's where the decision rule 17 steps in. So the decision rule is this impartial 18 third party that comes in and looks at these numbers 19 and are asked the question: Are these things 20 fluctuating too much or are they too slavish? 21 Fluctuating too much means that the probabilities are 22 not being matched up correctly. Too slavish means 23 that we're too close -- we're staying too close to the 24 expected values to where the thing is predictable. I 25 want a jury to come in and make that call for me. 0027 1 And there are, in fact, lots of juries 2 that people can call in here, just like there's lots 3 of juries in the real world that you can select. But 4 one that's a standard here in this part of statistics 5 called goodness-of-fit testing is something called a 6 chi-squared test. So the chi-squared test, the first 7 thing is, you get a distance measure. It's a measure 8 of distance between the observed and expected 9 frequency, so it measures the distance between 17 and 10 27 and so on. You don't need to -- unless you just 11 thirst for knowing exactly what the formula is, I will 12 be glad to give it to you, but we can skip that for 13 now. 14 All right. So the chi-squared test is 15 by an impartial third party that's going to come in 16 and look at these frequencies. And there's two things 17 that can happen. Either the frequencies sort of match 18 up too much or they don't match up enough. And "don't 19 match up enough" means that we get large value of the 20 chi-squared statistics. Small value near zero means 21 we're too slavish and we're looking at something 22 predictable. 23 So the bottom line here, one of the 24 things that comes out of what I just said is that 25 you've got to look at both sides of this thing. 0028 1 You've got to look both for small and large values. 2 In some of the testimony that's been presented here, 3 that's not been done, and that's a fundamental flaw. 4 And there's one more thing to say about 5 this. So as we all know, in the ADMs, they use a 6 random starting point when they actually initialize 7 random number generators. So there's something that's 8 fundamentally different about them than just using, as 9 I would, sitting at home using one of my random number 10 generators that I start up and give it a seed. They 11 use a random process to pick what the seed is. 12 But, nonetheless, under the hood of all 13 the ADMs are random number generators. There's three 14 of them -- Elkin's, KISS, and MT, which stands for 15 Mersenne Twister. And all these are good random 16 number generators. Right? But being a devil's 17 advocate here, if you've read the literature on random 18 number generation -- and I have -- if you've read 19 that, then you know that what -- most random number 20 generators work fine in terms of nailing down what the 21 probabilities look like. They do okay for that. 22 The problem can be predictability. So 23 that's the thing that you have to -- if you gave me a 24 random number generator that I knew nothing about, 25 what I would expect to find, if I found anything, was 0029 1 some poor predictability. Okay. So that's the reason 2 I say that you've got to look to -- this is in 3 particular a case where you have to look at both 4 sides. You have to look for predictability as well as 5 something that doesn't match up in terms of the actual 6 probabilities or proportions. 7 All right. So the decision rule then 8 that comes out of this -- this is using a chi-squared 9 test, so now this is out of my hands. Right? There 10 is a table that I go to or a virtual table, if you 11 have statistical analyses software, that gives me what 12 my decision rule is. 13 The only input I give here is 5 percent 14 reasonable doubt, my 5 percent level of significance. 15 And the decision rule then says -- again, this is 16 mechanics, so this is not in my hands anymore, 17 something that comes out a textbook or a table. And 18 it says that what I should do is, I should reject the 19 null hypothesis that the megapliers are being selected 20 at random with the correct relative frequencies if the 21 observed value of my chi-squared distance measure, the 22 observed difference I see, is larger than 7.38 or 23 smaller than .05. 24 Okay. Now I calculate the chi-squared 25 value as 4.31. 4.31 is not larger than 7.38 and it's 0030 1 not smaller than .05. So with a 5 percent reasonable 2 doubt, there is not sufficient evidence to say that 3 the null hypothesis is not true or to say that the 4 megaplier values are not being selected at random with 5 the specified probabilities. 6 Now, you say, "Okay. Well, that's your 7 5 percent level of significance. What if I wanted a 8 different level of significance? What if I wanted to 9 do it at the 10 percent level? Would that change my 10 findings?" 11 Well, if you keep on along those lines, 12 you're eventually led to something called a P-value, 13 and that's also something that's come up here in some 14 of the testimony here. And if you go to Prem Mann's 15 book, for example, he'll say the standard level -- or 16 the standard definition for P-value is that it's the 17 smallest significance level at which the null 18 hypothesis can be rejected. 19 So I keep looking and I keep asking, 20 "Well, if I change my reasonable doubt to 10 percent 21 or 15 percent or 20 percent, could I still reject a 22 null hypothesis?" And eventually when you get to 23 23 percent, you find out that you can actually reject 24 the null hypothesis. 25 Now, here is the interpretation, and 0031 1 this goes back to the statement I made earlier. The 2 jury says, "Are the things that we've seen consistent 3 with innocence or not being guilty?" And so this says 4 that there is a 23 percent chance -- so that's the way 5 I think about this. This is the correct 6 interpretation for this P-value -- there is a 23 7 percent chance of seeing something like the 8 chi-squared statistic that we saw if the null 9 hypothesis was true or the numbers are being selected 10 at random with the correct frequency. 11 So there is a 23 percent chance of 12 seeing something like what we saw if this person's 13 hypothesis is not guilty -- true. I think this would 14 be a -- this would be a challenging country to live in 15 if 23 percent was okay for reasonable doubt for 16 convicting somebody. And I certainly would be 17 probably a better citizen than I have been in the 18 past, certainly wouldn't jaywalk anymore. Let's just 19 put it that way. Right? 20 Now, that's the correct way to think 21 about this. But there have been some 22 misinterpretation of these numbers that has appeared 23 in this forum, and so I want to address that for just 24 a second. So one phrase -- and this is a paraphrase 25 of a direct quote that appeared here in turn -- but 0032 1 I'll say this is a paraphrase and kind of in the 2 context of what I've just talked about. 3 This is the wrong way of talking about 4 this number, and it says there that is a 23 percent 5 chance the model is right and a 77 percent chance that 6 it's wrong. That's a gross misinterpretation of this 7 number. It has nothing to do with statistics. And 8 there's no way that you can think about the things 9 that we've done and come up with that conclusion. All 10 right. 11 Now, I've written a document that 12 Michael has that certainly explains that in great 13 detail. But rather than go into all the ins or outs 14 as to why that's true -- because there's a number of 15 reasons why this is not right -- but let me just go to 16 sort of the simplest one, and here is the way it is: 17 The person is either innocent or guilty. When they 18 walk in the courtroom, they either did the crime or 19 they didn't. Okay? So the model is either right or 20 wrong. There is no chance involved in it. This is 21 just a fundamental fact of the universe. Okay? 22 So as I walk across the room, my guilt 23 or innocence does not change. Right? I don't 24 suddenly look like Brad Pitt and then look like 25 Angelina Jolie later. All right. So that just 0033 1 doesn't happen. Okay? So there are some things that 2 are constants in the universe. And whether or not the 3 null hypothesis is true or false, it's a constant. 4 Where does the probability come in? 5 Well, we make a decision, and our decision can be 6 either right or wrong. So the only thing that's 7 random here is whether or not our decision is right or 8 wrong, not whether or not the model is right or wrong 9 or the person is innocent or guilty. We may or may 10 not be right in judging the person innocent or guilty. 11 But the fact is, one of those two things is true. All 12 right. 13 So if I look back here, there is a 14 23 percent chance of seeing something like what we 15 saw. There is a 77 percent chance of seeing something 16 different. That's the right way to think about this. 17 Is 23 percent chance unusual? No, certainly not. 18 Okay. Things that happen with 23-24 percent, 19 probability they happen all the time. Okay? 20 And as a statistician, I would certainly 21 be like -- I would certainly be running around saying, 22 "The sky is falling" all the time if I had jumped up 23 and down every time I got excited about something that 24 happened 23 percent of the time, "Just happens all the 25 time." Is this consistent with innocence? In my 0034 1 opinion, yes 2 All right. So let me sum up. I do 3 quarterly analysis on all the data that comes in from 4 the megaplier machines and the ADMs. And I just 5 finished the analysis for this last quarter. And as a 6 result of that -- I looked at all the data -- I see no 7 indication, any reasonable level of significance, 8 anything that I think any statistician that you would 9 talk to would say caused any cause for concern. I 10 don't see any indication that the megaplier values are 11 not being chosen correctly, according to the model we 12 have. 13 Now, there is a little more that I want 14 to say about this before I finish, because the lottery 15 provides -- the data that comes in from the lottery is 16 sort of unique in a certain sense. In lots of 17 applications in the world, you see one set of data; 18 that's all you ever see. So if you're doing -- you do 19 like a marketing study, you go out and you collect 20 your data, and you make your call on the basis of this 21 one set of data that you collected. If somebody is 22 writing a master's thesis or a doctoral dissertation, 23 you go out and collect data, that's all the data you 24 ever got and you've got to make a call on it. 25 In contrast, with all of the things that 0035 1 are on-line, like your on-line games and also for 2 these megaplier values, all the things I monitor, this 3 is an ongoing process. The data continues to 4 accumulate. So I have been doing this since about 5 '92, and so there is no telling how many -- I should 6 have probably worked by the penny for every number I 7 got or something like that. That would have been a 8 wiser decision. 9 But there's no telling how many numbers 10 I've seen over the course of the year, and they 11 continue to accumulate. We have the luxury here -- 12 what I guess I'm saying is that we have the luxury 13 here of continuing to observe these processes. We can 14 sample them again and again and again. And that means 15 that one of the fundamental laws of the universe, the 16 law of large numbers, plays a role. 17 And I hope you'll forgive me for saying 18 this, but given sort of the theme of this talk or this 19 presentation, I have to say this: You cannot escape 20 the long arm of the law of large numbers. It's going 21 to get you eventually. So if something happens, if 22 anything were ever to be wrong, eventually it has to 23 show itself. Okay? That's what the law of large -- 24 you can prove it mathematically, you can prove it 25 empirically, any way you want to go. 0036 1 So let me sum up by showing you what I 2 mean, this particular application of what I'm saying 3 here. Here is the -- these are the P-values starting 4 for the -- megaplier values specifically that come up 5 in the draw is 2, 3, and 4. So these are the 6 megaplier P-values corresponding to the chi-squared 7 test of this hypothesis, that the numbers are being 8 selected at random with the probability of 2/21, 7/21, 9 12/21. And this goes back, starting at about Draw 55 10 when the chi-squared approximation sort of first is 11 actually sort of semi-valid, to this point now where 12 we're at Draw No. 286. 13 So this very last one over here on the 14 far right corner, that's that .23 that I gave you a 15 minute ago. So if you look at this, what happens 16 here? So you see that this is kind of a -- it looks 17 like sort of a drunkard's walk. Somebody is walking 18 along and kind of changing at random and moving back 19 and forth, no particular direction necessarily, just 20 kind of wondering around. 21 Down here at the bottom, that red line 22 down there is the 5 percent line. That's where I 23 begin to have a little doubt. That's where my 24 reasonable doubt, where it drops below my reasonable 25 doubt when it drops down below that 5 percent. You 0037 1 can see that it hasn't done that. Okay? 2 Now, if something were wrong -- okay? -- 3 if something had happened to the ADM or things weren't 4 working correctly or whatever, any imaginary scenario 5 you want to come up with, or any scenario you want to 6 come up with, if something had happened there where 7 this process is no longer working correctly, the null 8 hypothesis is false, what will happen here is that 9 eventually these little dots will drop down below that 10 5 percent line and they will never come back. Okay? 11 Now, that doesn't mean that they can't 12 drop below that 5 percent line. They can take an 13 occasional sojourn down below and come back up. But 14 if something is wrong, eventually they've got to drop 15 below that line and they never come back, at some 16 point in time. So we get to monitor this stuff. If 17 something were to ever go wrong, I am confident that 18 these statistical techniques will show it. Okay? We 19 will see it with Probability 1. 20 Okay. That's all I have to say. And 21 nobody asked me a question, so I'm a little bit 22 disappointed, but I suppose I'll live. 23 CHAIRMAN CLOWE: I think we have some 24 questions. 25 DR. EUBANK: Good. 0038 1 CHAIRMAN CLOWE: Well, we were 2 listening. 3 DR. EUBANK: Okay. Well, good. You're 4 going to get an A for today, then. 5 COMM. COX: You did a great job, 6 Dr. Eubank, of making something that seemed 7 incomprehensible comprehensible. Thank you very much. 8 DR. EUBANK: Most of my students have 9 said the opposite of that, by the way. 10 (Laughter) 11 COMM. COX: Well, this was a great 12 little refresher for me. I think I understood this at 13 one time, but it was quite some time ago. 14 Now, there are a couple of ways that you 15 can go about this kind of an analysis it seems like, 16 and it seems like we've done them both. One is to go 17 inside the black box and see if the black box works, 18 and the other is to look at what goes in the black box 19 and what comes out of the black box and, using the 20 statistical techniques such as you did, determine 21 whether it looks like to you as a statistician it 22 works. 23 DR. EUBANK: Right. 24 COMM. COX: Now, if either of those -- 25 could either of those be totally conclusive? Could 0039 1 you go inside the black box and establish that beyond 2 a reasonable doubt, no matter what any statistician 3 says ever, this baby works? 4 DR. EUBANK: That's a really good 5 question. But I think the answer -- here is my answer 6 as a statistician: I would never be convinced until I 7 saw the statistics, until I saw it work. 8 COMM. COX: Okay. 9 DR. EUBANK: So as a statistician -- I 10 mean, I understand. I'm beyond just a layman status 11 in terms of understanding the way this might be coded 12 up. I've never seen the code, and so I've never 13 actually looked inside the black box. I don't know 14 whether Battelle actually did that or not, but they 15 did something related to that. 16 I've never actually seen the code, but 17 I've written code. I've written my own code for 18 random number generators and so on. The only way that 19 I've ever seen these things verified in the real world 20 is using statistical analysis. So I've seen people 21 come up with, you know, these very complicated random 22 number generators and then people look at them 23 statistically and they find out that: Hey, they're 24 predictable. That's basically what people see, 25 there's some sort of a pattern in the data that they 0040 1 produce. 2 So I guess my answer is that without 3 looking at the statistics, I don't think I would 4 ever -- I don't think I could ever be convinced until 5 I actually saw this thing work and I saw the numbers 6 coming out, yes, until I did what you said, we put 7 data in and we see what comes out the other side. 8 And I as a statistician or somebody 9 that's a good statistician looks at these things, I 10 don't think I would ever be convinced. I think it's 11 both of these things. I don't think you can just -- 12 you know, I think somebody has got to look at what's 13 in the black box and you've got to know that there's 14 good stuff in there, but you've got to have some sort 15 of proof on the other side. That's what I'm saying. 16 COMM. COX: So even if the algorithm 17 looks perfect, which it does -- 18 MR. WESTERBURG: Right. 19 COMM. COX: -- you have the possibility 20 that a part might fail or the current might vary or 21 something like that -- 22 DR. EUBANK: That's exactly right. 23 COMM. COX: -- that would cause 24 something to not come out as the black box formula was 25 generated? 0041 1 DR. EUBANK: That's right. 2 COMM. COX: Well, what we've done here, 3 Michael, it seems to me, is that we have been vigilant 4 to determine that what's inside the black box works, 5 and we've also looked at outside the black box to see 6 if the results are statistically reasonable. And it 7 looks like that we're passing both tests. Would your 8 conclusion be any different from that? 9 MR. ANGER: I agree with that statement, 10 Commissioner. To clarify -- I did go back -- this 11 question came up when we participated in the 12 discussion on this at the June 28th meeting. And one 13 of the things that I committed to do was to go back 14 and find out exactly what type of testing Battelle had 15 done to certify the ADMs before we put them into 16 production for use for the megaplier drawings. 17 What we learned was that they conducted 18 with generated output, if you will, similar analysis 19 to what Dr. Eubank did. And to follow up on his 20 comments -- and certainly I will qualify myself as a 21 layman in this -- but the fact is, we understand how 22 the ADM is constructed and we understand the levels 23 of -- I guess layers of randomness that have been 24 introduced. 25 Dr. Eubank talked about the fact that 0042 1 the automated drawing machine that we use contains 2 three random number generators. And there is a 3 process by which -- which random number generator will 4 be selected to be used for any nightly drawing is 5 conducted randomly. 6 So first off, the random number 7 generator that we use on any given drawing is random. 8 And then that random number generator is continuing to 9 generate numbers in anticipation of the drawing. And 10 what we have is, is another layer of randomness, and 11 that is human intervention. A number of our draw team 12 actually hits a button at the appropriate time to 13 select the number that will be the megaplier number 14 for that evening's drawing. There's a couple of 15 layers on top of that. 16 So in answer to your question, we 17 understand the mechanics of how the equipment works. 18 We do not have the code reviewed by Battelle. And, in 19 fact, they advised us that the method that they would 20 use, looking at the output, was the proper way to 21 certify and check this equipment to ensure it was 22 operating properly, to the point that you made. 23 There's a lot of moving parts here. And 24 I think Prof. Busald in his testimony back in June 25 talked to this as well. You've got the random number 0043 1 generators, you've got the software code that 2 interacts with them. And in spite of what our 3 technology folks here at the agency might say, you 4 know, you also have the potential for, you know, the 5 ghost in the machine, if you will. 6 You know, what if some anomaly occurred 7 in the machine, like an electrical current issue or 8 something like that, that could interrupt or cause 9 that to operate in a way that it's not intended to 10 operate? You can't see all of those things or you 11 can't look through all of those things. What you can 12 do is, you can look at the output and you can see if 13 the output is coming out in a way that's appropriate. 14 And so we did that certification process with 15 Battelle. And then as a follow-up in actual 16 production, what we're doing with Dr. Eubank is 17 revisiting that on a quarterly basis to ensure that 18 there isn't anything to indicate that we have an area 19 for concern. 20 COMM. COX: Okay. So we have down this 21 top-down and bottom-up? 22 MR. ANGER: Yes, sir, I believe so. 23 DR. EUBANK: Let me just say one thing. 24 And I commented on this, but it's probably worthwhile 25 to say it again. 0044 1 So the random number generators 2 themselves that are used inside here are not new. 3 These are well-known, been around for a long time. 4 The Mersenne Twister is sort of like the industry 5 standard now. I think I talked about this maybe one 6 of the times I was here before, but I think you 7 basically called it the Cadillac of random number 8 generators, so those are standard. 9 And I'm not sure how the code for these 10 things was created. But you can find code -- I mean, 11 you can actually go to the website of the people who 12 developed the random number -- the Mersenne Twister, 13 for example, and I can download their code for that. 14 So both the code itself -- now, implementing it inside 15 something is a different story. But what I'm saying 16 is that the code itself is out there for all these 17 things. So if I want to get a random number 18 generator, I don't have to think up my code on my own. 19 I can go find it. 20 COMM. COX: And I think we had that 21 discussion with the gentleman from Battelle a couple 22 of years ago. 23 MR. ANGER: Yes. And he also had a 24 presentation about a year ago here at the Commission 25 meeting where we brought in representatives from 0045 1 TeleCom Game Factory and their subcontractor, Counter 2 Concepts, Dave Ablett, who is actually the person who 3 constructs the machines with regard to the random 4 number generators and the software codes -- 5 COMM. COX: That may be what I recall. 6 MR. ANGER: -- that interact, yes. 7 COMM. COX: Okay. Now, Dr. Eubank, I 8 may be mixing up my P-values and my levels of 9 significance, but let me see if I understood in those 10 terms what you were talking about. 11 DR. EUBANK: Okay. 12 COMM. COX: A level of significance is 13 something that you select as a professional as this 14 reasonable doubt percentage, if you will? 15 DR. EUBANK: Right. 16 COMM. COX: And the P-value is the 17 resulting statistic that you compare with your level 18 of significance? 19 DR. EUBANK: That's correct. 20 COMM. COX: Okay. So a 23 percent, you 21 say that happens all the time. You would never have 22 selected 23 percent as your level of significance 23 because it happens all the time? 24 DR. EUBANK: Correct. Let me sort of go 25 off on a tangent here that sort of makes the point, I 0046 1 think sort of lays it open. So one of the things that 2 I've seen in testimony that appeared here was somebody 3 that said that they wanted the P-value to be 1? What 4 does P-value 1 mean? Well, that's basically a good 5 old Texas tradition called the hanging judge. Okay? 6 Everybody that comes in is convicted. You don't even 7 need to give them a trial. You do it just because you 8 want to make them feel good about being hung. 9 COMM. COX: All right. 10 DR. EUBANK: So this says that you want 11 to be wrong all the time. Now, it's going to be a 12 great -- there's not going to be any guilty people get 13 out the door, but there's going to be a whole lot of 14 innocent people that are going to go down the tubes. 15 COMM. COX: All right. 16 DR. EUBANK: So I want my P-value to be 17 small. I don't like making -- even as an Aggie, I 18 don't like making mistakes. And I'm sorry, I don't 19 want to make mistakes 25 percent of the time. I want 20 my P-values to be small. And 5 percent is -- the 21 standard are 10 percent, 5 percent, 1 percent. 22 Ten percent gives me too many false positives. I've 23 got too many innocent people that I'm sending to jail. 24 One percent -- well, even 5 percent to me -- because 25 we do so much testing here -- 5 percent is something 0047 1 that's probably too small. That's 1 in 20 innocent 2 people that you send to jail. That's probably a 3 little -- that's probably not -- what did I say? It's 4 probably not small enough. Okay? 5 So even 5 percent is probably -- I think 6 you have to -- what I'm saying here about this -- so 7 when I talk about this chart here, I said, "Well, 8 okay. Every once in a while, if things are working 9 right, this thing is supposed to dip below that 10 5 percent line because everything is supposed to 11 happen, but it's not supposed to stay below the 12 5 percent line." Okay? 13 So by using this 5 percent with a little 14 bit of common sense, realizing the way things fall in 15 a random way, then I think you eventually come up to 16 what I believe is the right decision. I mean, just 17 like I said, so with Probability 1, this is going to 18 show up. 19 But to address your question, yes, 20 23 percent is way too big a chance for error. That 21 happens all the time. I choose 5 percent because it's 22 sort of a middle ground in terms of the things that 23 are standards. It's a good guideline. 24 COMM. COX: Do you use 5 percent for all 25 the testing that you do for the Texas Lottery? 0048 1 DR. EUBANK: Yes, sir. 2 COMM. COX: Is that a decision that you 3 made as a professional or was that made in 4 consultation with Lottery management? 5 DR. EUBANK: That's something I made as 6 a professional. 7 COMM. COX: Would it be appropriate for 8 Lottery management to review that with you? 9 DR. EUBANK: Absolutely. 10 COMM. COX: Because what I think we've 11 got there is a question of the integrity of the 12 lottery versus the cost of that integrity. 13 DR. EUBANK: Sure. 14 COMM. COX: So I would think that it 15 would be good if we looked at that. I bet we come 16 right where you did -- 17 DR. EUBANK: Well, maybe so, but I think 18 that's a good suggestion because that's really the 19 sort of -- as a rule, I mean, that's the input that 20 you want from your client. 21 So this goes back to 1992. And so none 22 of the people that were my clients back then are here 23 anymore. But I think that's a good suggestion, 24 because that's the kind of thing that I look for in 25 terms of guidance from people that I work for. 0049 1 COMM. COX: Thank you, Mr. Chairman. 2 CHAIRMAN CLOWE: Thank you, 3 Commissioner, for your question on level of 4 reasonableness and your answer in that regard, 5 Dr. Eubank. That was an area that I wanted to hear 6 some more about and appreciate that follow-up. 7 Was there any significance on your slide 8 about observed and expected results that they didn't 9 put? 10 DR. EUBANK: Say that again. 11 CHAIRMAN CLOWE: They don't add up to 12 the same number? 13 DR. EUBANK: Let me go back. Okay. 14 Well -- 15 CHAIRMAN CLOWE: You just passed it. 16 DR. EUBANK: You got two buttons, and so 17 you've got a 50 percent chance of being wrong. And as 18 an Aggie, that means you've got 100 percent -- 19 COMM. COX: There you are. 20 DR. EUBANK: There you are. You said 21 they don't add up to the same number. What did you 22 mean, that they're not exactly the same? 23 CHAIRMAN CLOWE: Yes. 24 DR. EUBANK: Okay. They're not supposed 25 to -- my -- 0050 1 CHAIRMAN CLOWE: The total is not the 2 same. 3 DR. EUBANK: That's probably because I 4 rounded over on the other side. So that's -- 5 CHAIRMAN CLOWE: Okay. 6 DR. EUBANK: -- that's my bad. But 7 these expected values are actually fractional. They 8 have some fractions involved, and I rounded them off. 9 CHAIRMAN CLOWE: My balance sheet 10 always -- 11 DR. EUBANK: Yes, sir, I know. I took a 12 course in accounting, so I should know better. 13 CHAIRMAN CLOWE: I would like an 14 asterisk there. 15 (Laughter) 16 DR. EUBANK: Okay. I'll go home and do 17 that. 18 CHAIRMAN CLOWE: I just wanted to prove 19 to you I was awake. 20 COMM. COX: And that you're -- 21 DR. EUBANK: So the bad thing is, I have 22 to admit that I've complained about that exact same 23 thing to somebody else, so it's good to be humbled. 24 CHAIRMAN CLOWE: Thank you for your 25 explanation. I would like to ask you to stay. And if 0051 1 you will, we're going to hear from Prof. Busald. And 2 not that we want to have a debate, but there might be 3 some issues that y'all might like to discuss for our 4 edification. 5 And before we call Prof. Busald to the 6 table, we'll take a short break. 7 (Off the record: 9:55 a.m. to 10:05 8 a.m.) 9 CHAIRMAN CLOWE: We'll come back to 10 order, please. We'll come back to order. And we'll 11 ask Prof. Busald to come up and address the 12 Commission. 13 PROF. BUSALD: Good morning, 14 Commissioners. For the record, my name is Gerald 15 Busald, and I'm a professor of mathematics at 16 San Antonio College. And for the record, I would like 17 to admit that I am a mathematician, not a 18 statistician. I just happen to teach introductory 19 statistics, so I certainly yield on statistical 20 expertise to Dr. Eubank. 21 However, there are some things that we 22 disagree on. One of them is -- and one of the things 23 is not what the P-values have been over time. I think 24 he agreed, and we had exactly the same P-values 25 because they're calculated exactly the same way. 0052 1 What we disagree on probably is 2 significance level. I found it interesting that 3 August 29th was chosen as the last drawing analyzed 4 because, as Dr. Eubank points out, this is something 5 that occurs. It changes every time there is a 6 drawing, the P-values change. He happened to pick it 7 at a time when the P-value was at a peak because we 8 just had had a 2 drawn. And so a great deal of 9 significance is a term not in statistical significance 10 but in real world significance, was given to the 11 23 percent number. 12 I would like to point out that today, 13 after last night's drawing, that number is 9.25 14 percent. As you saw on his graph, those values go 15 down; they may go in cycles. And so the question 16 becomes -- and quite frequently it has been below 17 10 percent -- and so my concern that I raised is and 18 the statement that I make and Dr. Eubank -- I don't 19 want to speak for him, but I think we agreed on the 20 phone that I could say there is only a -- today I 21 could say there is only a 9.25 percent chance that I 22 am wrong if I say the numbers we see would not have 23 occurred if the program were working perfectly. So 24 that number goes up and down. 25 So, you know, we're not trying to send 0053 1 someone to jail. What I have suggested is a concern 2 over a computer program. There is a great deal of 3 difference between sending someone to jail and a 4 computer program, especially this being the Texas 5 Lottery Commission. As Commissioner Cox said, we have 6 costs versus integrity issues. And if I were in 7 charge, I would be very worried about that, looking at 8 just results, which is what this did. Of course, I 9 have no input in the pre-test, post-test, anything 10 else inside of the program, so I can only look at the 11 results. And as of last night's drawing, that P-value 12 is down to 9.25 percent 13 And I thought it was interesting that he 14 said, "Well, the program is innocent until proven 15 guilty." I don't really think that's true with the 16 lottery, because I think the lottery, unlike most 17 things, bears the burden of proof of innocence, 18 because we are a public agency, or the Lottery 19 Commission is a public agency that needs to have that, 20 not a, "Well, you can't prove us guilty." I think you 21 need to look at it from the other side of the coin. 22 And that's just my opinion as a citizen 23 bringing -- and it just so happened that we came into 24 this just because of something, a statistical test 25 that my students ran. And if you look at it, that 0054 1 first time was over a year and a half ago. And we're 2 still sometimes hovering around below the 10 percent 3 level; sometimes it's above. 4 Now, whether it would ever go below the 5 5 percent level in the long run, I would hope it 6 wouldn't, that that would mean there's only a 7 5 percent chance I'm wrong when I say the program -- I 8 wouldn't see these numbers if the program is working 9 correctly. I don't know. As a citizen, I wouldn't 10 want to see that happen. 11 So I think we disagree on how or what 12 you do with these numbers. And so, you know, I agree, 13 5 percent is a common significance level. One 14 percent -- if they're testing a drug, I hope their 15 significance level is less than 1 percent because I 16 don't want to take a drug that is not proven in the 17 statistical manner to be effective. 18 But on the other hand, this is not that. 19 This is a computer program that was instituted 20 probably on a cost effectiveness basis and with an eye 21 toward doing all drawings on that basis, which you 22 know is done in many states. And so I think it's just 23 a different level of proof that you need to consider. 24 And that's just my opinion. 25 But I don't think Dr. Eubank has 0055 1 disputed any of the numbers that I brought forward to 2 you. And, you know, my numbers -- because I didn't 3 round, my numbers, my observed and my expected do add 4 up, and that's just a rounding error. I mean, that 5 happens with rounding. But I do have the current 6 numbers through last night's drawing. I didn't 7 reproduce it because it was through last night's 8 drawing. And since I don't get paid by the hour, I 9 don't have a lengthy presentation presented for you. 10 And I'm not asked to be in a situation. 11 But my whole concern is that the Lottery 12 Commission not say, "Well, there is not enough proof 13 to convict us," just like I hoped on so many issues 14 that the Lottery Commission do everything to give 15 every piece of information possible to the players, 16 whether they have to or not. I haven't been 17 successful in a lot of those efforts, but that's where 18 my emphasis is. 19 And so I would be concerned seeing these 20 numbers if I were in your chair. Obviously, I'm not 21 in your chair. But I would be concerned about the 22 perception of the players if the players really 23 perceived much of risk. Fortunately for the Lottery 24 Commission, most of this is never perceived by the 25 players. And that's why I said no matter what you do, 0056 1 no matter what you tell the players, if you are more 2 honest and give them more information, it has no 3 negative impact on sales, just like when we finally 4 got the issue of stating that a breakeven prize wasn't 5 the winner, despite the initial staff conclusion that, 6 "No, we don't want to say that." And, of course, that 7 was also Dr. Eubank's conclusion at the time, "Well, 8 it's okay to call them a winner if they break even." 9 And, of course, the Commissioners finally decided 10 otherwise. 11 But I think that we need to give players 12 as much as possible. We need to do everything to 13 build confidence and to give them as much information 14 as possible. There's nothing wrong with that. So I 15 guess it's just a matter of that's my take on how this 16 should be perceived. 17 But I am happy to say that my numbers 18 are not in dispute. Immediately, the time that 19 Dr. Eubank happened to choose was right after a 2 came 20 up. And, of course, you can see the 2's were under- 21 represented. Since that time we've had more 3's than 22 expected. Obviously, it bounces up and down in a 23 range, looking at his graph, the P-values. But if you 24 notice the trend on the P-values is -- overall from 25 the beginning is down, and that's the probability I'm 0057 1 wrong if I say the program is not working as it was 2 designed. Now, the significance level, yes, that's 3 the choice that you have to make. But I think 4 significance level for the Lottery Commission in this 5 instance is different than if we're testing a drug. 6 And so that's my comment. I would be 7 happy to answer any questions. 8 COMM. COX: Thank you, Prof. Busald. I 9 have a couple of questions. One is a factual question 10 and the other is just to be sure I understand, to see 11 whether I understand the difference between you and 12 Prof. Eubank. 13 But I heard you say that quite 14 frequently the P-value has been below 10 percent. 15 PROF. BUSALD: Yes. 16 COMM. COX: And, Dr. Eubank, I thought 17 on your schedule it didn't get that low. 18 DR. EUBANK: Something in the twenties. 19 PROF. BUSALD: It's never gotten below 20 23 -- 21 CHAIRMAN CLOWE: Just a minute. 22 PROF. BUSALD: I'm sorry. 23 CHAIRMAN CLOWE: Just a minute. I want 24 this on the record. Dr. Eubank, if you'll come up -- 25 PROF. BUSALD: I'm sorry. 0058 1 CHAIRMAN CLOWE: -- and answer 2 Commissioner Cox's question. And then if you wish to 3 converse with each other -- 4 PROF. BUSALD: Yes. I'm sorry. 5 CHAIRMAN CLOWE: -- that's going to be 6 permitted, but let's do it slowly so the recorder can 7 get all the comments. 8 PROF. BUSALD: I apologize. 9 COMM. COX: Dr. Eubank, maybe you could 10 put that slide back up. 11 DR. EUBANK: Well, this goes back to the 12 way I said that you have to compute P-values here. So 13 this goes to the way I said you have to compute 14 P-values here. So you look for being too slavish or 15 that the probabilities don't match up correctly. So 16 what people call this is a two-tailed test, so this 17 can be a two-tailed chi-squared test. So it goes back 18 into that statistical jargon for two-tailed tests. 19 There's some issue about how you can 20 compute P-value here, what it means in a case of 21 something like a chi-squared test where the 22 distribution is not symmetric. If you look in this 23 Prem Mann book, for example, they only do p-values for 24 things like the standard test about the mean where the 25 distribution of the test statistics are symmetric. 0059 1 But if you're going to be fair here and 2 actually think about this the right ways in terms of a 3 two sample -- in terms of a two-tailed sort of 4 approach to the problem, then the number that was just 5 quoted, 9 percent, becomes 18 percent. So we disagree 6 in sort of a fundamental way in terms of the way that 7 what's being called a P-value is being used here. 8 So finally if you look at my plot down 9 here, you see that it drops somewhere back in -- I 10 can't -- I don't have an arrow, but it drops somewhere 11 down into something like the -- oh, maybe this might 12 be about 18 percent, 17 percent or something like that 13 at one point -- I'm sorry. I did have an arrow after 14 all. Thank you, Robert. 15 All right. So I'm not exactly sure what 16 that is. That might be 11-12 percent. And that's 17 probably -- at one point I remember this got close -- 18 it might have reached down into about the 6 or 19 7 percent level in terms of what Prof. Busald would 20 say. But from my point of view, I do this as a 21 two-tailed experiment because I look -- I do this as a 22 two-tailed test because I look for both too small and 23 too large values here -- 24 COMM. COX: So you and Prof. Busald were 25 computing it differently, you were using a different 0060 1 test? 2 DR. EUBANK: We were using exactly the 3 same test statistic number. So my 4.31 -- at the 4 point when I computed my 4.31, he would have gotten 5 4.31, too. But the P-value that we got out of the 6 other side of that would have been different. 7 COMM. COX: Because you were using a 8 two-tailed and he was using a one-tailed? 9 DR. EUBANK: Sure. 10 COMM. COX: Now, which tail were you 11 using, Prof. Busald? 12 PROF. BUSALD: I was using the tail that 13 the numbers differed from what was expected without 14 concern that they're too close to what's expected. 15 COMM. COX: Okay. And this didn't have 16 to do, then, with whether the number that would reward 17 the player at a higher level was either higher than 18 lower, it had to do with just -- 19 PROF. BUSALD: Just the numbers. Right. 20 COMM. COX: Okay. 21 PROF. BUSALD: And let me see if I can 22 gather my thought here. It was just: Are these 23 numbers what was the expected? The reason over the 24 long run I would expect at some point, if the program 25 is working correctly, it would get near, very near 0061 1 what was expected, and the P-value would be -- the 2 probability of being wrong would be very large, just 3 like the hanging judge. 4 It's just like flipping a coin. If I 5 flip a coin 5,000 times, somewhere along the line the 6 number of heads will be less than a half and sometimes 7 it will be more than a half. But in the long run, 8 it's got to get near the half. And the same thing 9 here. At some point these numbers, if the program is 10 working correctly, need to get near the values. And 11 they may go to the other side if it's working 12 correctly. Right now the 2's are under-represented at 13 some point. If the program is working correctly, you 14 would expect them to be more than what was expected. 15 And so -- 16 COMM. COX: So let me see then if I 17 understand. I think you said, Dr. Eubank, that on 18 population you used a P-value of .23. 19 DR. EUBANK: That's correct. 20 COMM. COX: And, Prof. Busald, I think I 21 heard you say that you agreed with that, that the 22 P-value was .23. 23 PROF. BUSALD: I don't agree with -- it 24 all depends on -- 25 COMM. COX: Obviously, at a point in 0062 1 time. But what I'm trying to determine here is at a 2 point in time, did y'all agree that it was .23, at the 3 same point in time? 4 DR. EUBANK: We had the same numbers. 5 PROF. BUSALD: I have a different 6 P-value because I'm doing a different test. 7 COMM. COX: Okay. So -- 8 PROF. BUSALD: I'm doing a one-tailed 9 test. 10 COMM. COX: So when you say that your 11 number was quite frequently below 10 percent and 12 Dr. Eubank's chart doesn't show anything getting close 13 to 10 percent, we're not talking about the same 14 numbers? 15 PROF. BUSALD: No. He would be talking 16 about below 20 percent because he's doing a two-tailed 17 test. And quite frequently you can see it is below 18 20 percent. 19 COMM. COX: Okay. Do you agree with 20 that, Dr. Eubank? 21 DR. EUBANK: Yes. 22 COMM. COX: Okay. Now, the other issue 23 that I'm looking for is trying to find some simple box 24 to put this difference of opinion into. Earlier I 25 asked Dr. Eubank whether the 5 percent that he used as 0063 1 a level of significance was something that we should 2 sit down with him and be sure that it reflected our 3 view as to what we wanted as integrity of the lottery 4 versus the cost of integrity. Is that what we're 5 talking about here? Do you have a different view of 6 the level of significance than Dr. Eubank does? 7 PROF. BUSALD: Yes, and because of that 8 reason, because of the integrity of the lottery. 9 COMM. COX: Okay. 10 PROF. BUSALD: That is the reason I have 11 that difference of opinion. 12 DR. EUBANK: Could I just say one thing? 13 The statistics that's being used here -- let me go 14 back for a second. The statistic that's being used 15 here is something called a chi-squared statistic. 16 Now, what goes on here is that this number for large 17 sample sizes -- 286 is a relatively large sample size. 18 286 is our sample size here. 19 For large sample sizes, this statistic, 20 this number behaves like a number that's been drawn 21 from a chi-squared distribution. As the sample size 22 grows larger, it behaves more and more like something 23 drawn at random from a chi-squared distribution. Now, 24 a chi-squared distribution is a number -- is a 25 distribution that lives on the line. It's 0064 1 non-negative. And the numbers for this thing have to 2 go between zero and infinity. And they occur at 3 random. What that means for the P-values is that the 4 P-values sort of fluctuate at random between zero and 5 one. All right. That's what it translates into, and 6 that's what should happen here if everything is 7 behaving correctly. 8 The P-value, if things are behaving 9 correctly, absolutely will not go to one. Okay? And 10 it absolutely will not go to zero and stay there if 11 this thing is behaving correctly. The whole reason 12 that this can be used is because the distribution of 13 this test statistic is approximately something that 14 behaves like a random variable and has a chi-squared 15 distribution test, and that's the thing that lives on 16 the non-negative rule, zero to infinity, and just 17 takes values at random there. Okay? And so that's 18 what I expect to see. I don't expect to see this 19 thing peg at one end of this P-value range or the 20 other. Okay? 21 COMM. COX: Okay. Now, I heard Prof. 22 Busald say that he is a mathematician and you are a 23 statistician. And I have heard it said before, when 24 you disagreed with GTECH, that their statistician is a 25 mathematician and you're a statistician. Dr. Eubank, 0065 1 how do you look at that? You're a mathematician as 2 well as a statistician, I assume. You probably had a 3 good -- 4 DR. EUBANK: I'm in the Department of 5 Mathematics and Statistics, so I can call the 6 mathematics name when I need -- 7 COMM. COX: You've probably had a few 8 hours of math. 9 DR. EUBANK: One or two. 10 COMM. COX: Yes. So should we be 11 getting a professional statistician and a professional 12 mathematician and then a nuclear physicist? What 13 should Chairman Clowe and I be doing up here? 14 DR. EUBANK: You can get me in a lot of 15 trouble when I answer this -- okay? -- because I have 16 mathematicians on the same floor that I live on. But 17 there is a difference in training. My Ph.D. is -- I 18 took a lot of mathematics courses during the course of 19 my Ph.D., but none of that counted towards my Ph.D. 20 program. Only the courses that I took in statistics 21 counted. 22 What you see in a statistics program are 23 things just like what I quoted. So I took course in 24 large sample theory, so I can actually prove to you 25 mathematically the things that I just said about this 0066 1 chi-squared statistic. I can actually go through 2 calculations, and I've taught courses in large sample 3 theory, for example, at both Southern Methodist 4 University and Texas A&M, where I've done those sorts 5 of things. 6 So there is a difference in training 7 that somebody that sees only elementary statistics 8 books would never know. Okay? My students that I've 9 taught for years as undergraduates, they wouldn't know 10 this, and there would be no reason that they would 11 necessarily need to know this. 12 So there is a difference in training and 13 expertise. I think you can -- it's like a mechanic. 14 So I'm okay with changing the air filter on my car, 15 but I better not mess with the fuel injection system. 16 All right. And so I think there is a difference in 17 terms of the background, and it makes a difference in 18 terms of the way you think about these things. So I 19 don't know. I'm trying to be as politically correct 20 as I can in my answer, so if this ever appears in the 21 Phoenix paper, that they don't burn me in effigy 22 outside my office. 23 COMM. COX: I remember, I took 24 statistics in business school and I remember that we 25 used z-values because we didn't know calculus. So I 0067 1 understand a little bit about what you're talking 2 about, about changing the air filter versus designing 3 the air filter. But I also know that statistics are 4 taught in Departments of Mathematics. So I'm trying 5 to understand, is it fair to say that any of this is 6 attributable to a difference between a mathematician's 7 view and a statistician's view, or is it just a 8 differences of opinion about what's important? 9 DR. EUBANK: I think it's a difference 10 in training, because there is a fundamental difference 11 between someone who has a Ph.D. in mathematics, for 12 example, and somebody who's got a Ph.D. in number 13 theory or somebody that works in partial differential 14 equations or whatever. I work with people, for 15 example, who are math biology people. And these 16 people look at data all the time, but they don't know 17 anything about statistics, and that's why I interact 18 with them. They have no concept about how to even 19 think about randomness. 20 So I think there is a difference in 21 training. I know how to look under the hood of these 22 things, and I know what it rests on top of. So, for 23 example, I mean, I can take this apart and show you 24 from the very beginning in a certain sense where all 25 this stuff came from if you've got the -- if your life 0068 1 is sufficiently dull at this point to where you're 2 willing to listen to me do all my mathematics. 3 So I think there is a difference in 4 training in that when you understand things at a 5 fundamental level, you can reach conclusions. There 6 are people that are the actual -- something like an 7 elementary statistics course. So what you say is 8 true. There's statistics course that are elementary. 9 As I showed on one of my early transparencies, in 10 fact, there are elementary statistics courses that are 11 taught everywhere, and oftentimes they are taught by 12 people who are non-statisticians. 13 You don't need calculus to be able to 14 teach something like out of this Prem Mann book, for 15 example. You just don't need to know that. And so it 16 would be better if they had somebody that was a 17 statistician teaching that, but there's only so many 18 statisticians that you can hire, and there's only so 19 many statisticians that are out there. There is a 20 shortage. 21 And sometimes a statistician is not 22 going to be willing to go to a Department of 23 Mathematics. 24 COMM. COX: Sure. 25 DR. EUBANK: Okay? So somebody has to 0069 1 teach these courses, and I think they're usually done 2 quite well. But there is still a difference in 3 training in a person that teaches them, for somebody 4 that has a degree in mathematics versus somebody that 5 has a Ph.D., for example, or even a master's in 6 statistics. 7 COMM. COX: Well, as a layman, I'm 8 persuaded by the idea that this should be a two-tail 9 test. It's intuitively appealing that there shouldn't 10 be autocorrelation here. 11 DR. EUBANK: Well, that's right. 12 COMM. COX: There should be some 13 dispersion or the results of the lottery are going to 14 be predictable -- 15 DR. EUBANK: Right. 16 COMM. COX: -- and we don't want them to 17 be predictable; and, yet, we want them to be fair. 18 DR. EUBANK: Right. And it goes back to 19 really -- I thought about this in great depth, 20 actually. But over the last number of years, I spent 21 a lot of time reading things about random number 22 generators, and I've read a couple of books on it and 23 a lot of literature on it. Both because of my 24 interaction with the lottery but also because I teach 25 it in a class, in a class on statistical computing, so 0070 1 I have an interest in this that's two-pronged. And I 2 know from reading that literature that there are both 3 issues that come up here. So particularly with random 4 number generation mechanisms, they generally do a 5 pretty good job of producing the probabilities that 6 you want. But there can be, just like you said, some 7 sort of autocorrelation where you can predict what's 8 going to happen again. 9 So you can look at patterns of these 10 things, and there's some papers that have appeared in 11 some of the good journals where they have done some 12 pretty elaborate techniques and you find out there 13 was -- in fact, there is a random number generator -- 14 I think its name is Randu, R-a-n-d-u or something like 15 that. It's a random number generator that's been 16 around for a long time. It's still out in some places 17 and they've shown that it's actually really not very 18 good. It does a good job of giving you the 19 probabilities you want, but it's very predictable. It 20 has patterns that appear in the numbers when you look 21 at them the right way. It's not always easy to find 22 those patterns, and you have to have some pretty 23 sophisticated techniques for doing it. But certainly 24 you should not ignore that possibility. 25 COMM. COX: And, Prof. Busald, your idea 0071 1 that it shouldn't get too far from the statistical 2 probabilities is intuitively appealing as well. 3 Clearly, that is an important thing. That would be 4 the thing that I think at a lower level of numbers 5 would be more likely to occur so, as a practical 6 matter, might be the more relevant of the two tests if 7 you had to pick one tail or the other. 8 And I think I heard you say that on 9 yours, a 20 percent level would be the same as his 10 10 percent level. That's because it's cut in half or 11 because it -- 12 DR. EUBANK: The other way around. 13 COMM. COX: -- in two halves. 14 DR. EUBANK: The other way around. 15 PROF. BUSALD: Yes. 16 COMM. COX: So, Mr. Chairman, I think 17 what I'm hearing here is that we've got some 18 reasonable approaches to this thing on both sides -- 19 they both think they're right -- and that the real 20 issue is the one that we identified after Dr. Eubank's 21 presentation, which is: Should we be sitting down 22 with him and determining where you draw the line as 23 between the integrity in cost and integrity of the 24 lottery? So I would suggest that this has been a very 25 helpful discussion and that that might be our next 0072 1 focus. 2 CHAIRMAN CLOWE: I agree. And I wanted 3 to go back to that phraseology that you used, and I 4 think Prof. Busald quoted you in his opening remarks. 5 In my mind, there is no conflict there; there never 6 has been a conflict. The integrity of the operation 7 of the lottery in my mind doesn't have a price tag on 8 it. We're not talking about the expense of doing the 9 right thing at all, and I want clarification in that 10 regard. There is no conflict there whatsoever. 11 What we're talking about is two 12 different approaches, based in both cases on good 13 research, good work on the numbers and a conclusion 14 that parties differ on. I understand Dr. Eubank's 15 example of trial by jury. I'm not sure that I would 16 have chosen that, but I understand it was your way of 17 showing the Commission how you were trying to get to 18 the right conclusion. That's basically what it boiled 19 down to. 20 Prof. Busald's approach, as I see it, 21 somewhat different, and you are more on the side -- 22 and I think you traditionally represent a position of 23 full disclosure and one of, you know, let's tell 24 everything we can to the players. And you said this 25 morning yourself, "It doesn't make any difference to 0073 1 them. They're going to play anyway, so why not tell 2 them." And I understand that those are not 3 necessarily positions in conflict in every instance. 4 In some instances they are. 5 And, Commissioner Cox, my answer to your 6 question is, which I think you posed to Dr. Eubank is: 7 What is our role? Our role is simply to make the 8 decision of what's the right way to go. We're going 9 to rely on Director Sadberry and his staff to continue 10 to advise us. And if we think it's the right policy 11 that Operations Director Anger is implementing, we're 12 going to continue on that basis. If we think we want 13 a change and it can be made better, more fair to the 14 players, we're going to direct that change to be made. 15 That's where I come down on this. And I 16 think the discussion has been helpful. It's certainly 17 been a revelation to those who sat here and listened 18 to both of these learned instructors. It reminds me 19 that statistics was the only course I ever flunked at 20 The University of Texas. 21 (Laughter) 22 COMM. COX: You're making that up. 23 CHAIRMAN CLOWE: No, I'm not. And the 24 instructor who flunked me was Stella Trawick 25 (phonetic). I'll never forget her name. The only F 0074 1 I ever made at the university, carrying off a good 2 average. I went back and took it over again and 3 scored very highly, learned my lesson. It's been 4 educational. 5 Do you have anything to add, Director 6 Sadberry? 7 MR. SADBERRY: Mr. Chairman, thank you, 8 and Commissioner. I concur and express our 9 appreciation both to the instructors who have come 10 today as well as our staff and Operations Director 11 Anger and his staff on the thoroughness and 12 appropriateness of this discussion. 13 As you have indicated, these are matters 14 that will be instructional to me and my office as I 15 coordinate our efforts with our staff to address as 16 appropriately and to advise you or make 17 recommendations. 18 I might add, if it's appropriate, 19 that -- and this was a topic of discussion 20 previously -- there are, in addition to this matter, 21 which as you can see required significant attention -- 22 there are other matters that have been presented to us 23 for consideration, and they will as well be 24 undertaken, and you will get a report from us on those 25 in due course as we gather those. Because of this 0075 1 matter and the fact that you have specifically 2 directed attention to it and that collaboration occur, 3 we want this brought on as soon as possible, which 4 today is the first occurrence. 5 CHAIRMAN CLOWE: And you're referring to 6 matters raised by Prof. Busald -- 7 MR. SADBERRY: I am. 8 CHAIRMAN CLOWE: -- which are not 9 published on this item in the agenda? 10 MR. SADBERRY: That is correct. And I 11 didn't want to lose sight or have the record be silent 12 on those, but they're not appropriately noticed for 13 today's presentation, but they will be the subject of 14 discussion. 15 And we would advise Prof. Busald and any 16 other interested parties of which we are aware of the 17 time in which those will be brought on. So if they 18 choose to be present, they can participate or at least 19 be in attendance as those matters are discussed as 20 well. 21 I think I now understand the specific 22 focus of a particular matter with respect to this 23 presentation that staff should address, and we will do 24 that. 25 COMM. COX: Mr. Chairman, I would like 0076 1 to rephrase -- thank you for your thoughtful 2 observation -- on the term I used which was integrity 3 versus the cost of integrity. You're absolutely 4 right. The cost of integrity is irrelevant to the 5 concept of the integrity of the Texas Lottery. 6 I think what I was trying to say is, how 7 often do we want Dr. Eubank waving a red flag that may 8 turn out to be crying wolf, but we're just being very, 9 very careful? And I would call that maybe vigilance 10 versus the cost of vigilance. And I still would like, 11 if you're agreeable, to look at that with Dr. Eubank 12 and just -- we'll probably come out right where he 13 did, but I think it's a decision that we should 14 participate in. 15 CHAIRMAN CLOWE: Well, I think that 16 point has been well made, and I thank you for 17 clarification on that. I think Dr. Eubank said we 18 started in '92 with the parameters that we've been 19 using. And we should reexamine that; there is no 20 question about that. 21 And we ought to take Prof. Busald's 22 views into consideration, and we certainly have. We 23 will continue to do so. But we are questing to the 24 right position. That's where we really want to get to 25 and be comfortable with. And because these learned 0077 1 individuals disagree to some extent -- I'm not sure I 2 understand the one-tail and the two-tail -- but I 3 think there's room for honest disagreement there. 4 That's where we get paid the big bucks. 5 COMM. COX: There you go. 6 CHAIRMAN CLOWE: So we will make that 7 determination when the time comes. 8 Anything further, gentlemen? 9 PROF. BUSALD: Yes, I just have one 10 additional comment. And that is, as far as fairness 11 to the players, the program that is operating right 12 now is more than fair to the players. 13 COMM. COX: Right. 14 PROF. BUSALD: So I haven't disputed 15 that we're cheating players in any way. 16 CHAIRMAN CLOWE: Thank you for that 17 comment. 18 COMM. COX: And we've got an obligation 19 to the players and to the school children of Texas 20 that we have to balance. 21 CHAIRMAN CLOWE: Thank you, gentlemen. 22 Thank you, Mr. Anger. 23 AGENDA ITEM NO. II 24 CHAIRMAN CLOWE: Next we'll go to Item 25 No. II, report, possible discussion and/or action on 0078 1 lottery sales and revenue, game performance, new game 2 opportunities, market research and trends. 3 Ms. Pyka. 4 MS. PYKA: Good morning, Commissioners. 5 My name is Kathy Pyka, Controller for the Lottery 6 Commission. 7 Commission, this morning I'm going to 8 provide a quick overview of Fiscal Year 2000 9 activities, starting with our Fiscal Year 2006 sales, 10 which were $3.78 billion, the highest level of sales 11 in Commission history. This is a -- 12 COMM. COX: We finally caught up with 13 '97. 14 MS. PYKA: Yes, sir. 15 COMM. COX: Congratulations. 16 MS. PYKA: This is a $112.2 million or 17 3.1 percent increase over our sales for Fiscal Year 18 2005. And our cash revenue -- our cash transfers to 19 the state for Fiscal Year 2006 were just over one 20 billion. We do plan to provide a detailed overview on 21 all of our sales data and our revenue data at our 22 second meeting in October. And we'll give you quite a 23 bit more information about our final accrual numbers 24 with that. 25 And now what I would like to do is -- 0079 1 COMM. COX: Kathy? 2 MS. PYKA: Yes, sir? 3 COMM. COX: You said record sales. Do 4 you think the transfers are going to be a record as 5 well? 6 MS. PYKA: They are not. 7 COMM. COX: Okay. 8 MS. PYKA: They are not. What we're 9 doing right now is just to finalize all of the unspent 10 administrative dollars. We'll give you a final 11 accounting of the accrued transfers so that we can 12 look at it on both a cash basis transfer as well as an 13 accrued basis transfer. So we'll have that for you in 14 detail at the second meeting in October. 15 And with that, Robert has some follow-up 16 information that he is going to present from our last 17 Commission meeting on sales data. 18 MR. TIRLONI: Good morning, 19 Commissioners. For the record, my name is Robert 20 Tirloni. I am the Products Manager for the 21 Commission. 22 As Kathy just said, we have two slides 23 for you today. These are follow-up from the late 24 August meeting that we had. This first slide looks at 25 the percentages of on-line and instant ticket sales in 0080 1 the states that we are typically compared to, that 2 being California, Florida, Georgia, Massachusetts and 3 New York. You see the three columns -- instant, 4 on-line and other. "Other" in this case refers to 5 keno and/or video lottery. Obviously, Florida and 6 Texas do not offer either of those product categories. 7 All the other states do. 8 The information that's most important I 9 think to point out is the progression that has been 10 made in some of these states. And we've highlighted 11 on-line in green to try to get it to stand out because 12 we do realize there are quite a bit of numbers on this 13 slide for you. 14 But, for example, in the Year 2000 in 15 the State of Florida, they were realizing almost 75 16 percent of their sales from on-line, with 25 percent 17 coming from instants. And in the five-year period 18 that has lapsed, you now see that Florida is getting 19 almost 47 percent of their sales from the on-line 20 product category, with 53 coming from the instant 21 category. 22 Similar situation in Georgia. In the 23 Year 2000, almost 59 percent coming from on-line, with 24 about 39 coming from instants. And again, there has 25 been a pretty dramatic progression. Those numbers 0081 1 have almost completely flipped, to 38 percent coming 2 from on-line and 60 percent, almost 61 percent, coming 3 from instants five years later. 4 I'll continue, and then I'll go back to 5 some of the other states. New York, very similar 6 situation. In the Year 2000, 62/28 breakdown. And 7 five years later, on-line only represents about 8 40 percent of the sales in New York. 9 Massachusetts, which back in the Year 10 2000, has always been -- it's not on my text, but they 11 have been very heavy on instant. So in the Year 2000, 12 they were realizing almost 67 percent of their sales 13 from the instant category, only about 18 percent from 14 on-line. There has been more progression to instant 15 five years later, but the difference is not as 16 dramatic. 17 And that's very true of our own 18 experience in Texas. Back in the Year 2000, we were 19 seeing about 38 percent of our sales from on-line, 20 almost 63 percent from instants. And we have followed 21 that trend as well, with instants increasing, as y'all 22 are well aware. We discuss that pretty regularly in 23 these meetings. And in the Year 2005, we had almost 24 74/26 percent supply split. 25 CHAIRMAN CLOWE: Do you want to talk 0082 1 about this? 2 COMM. COX: I've got one question, I 3 guess. And as I look at these, I see things are all 4 over the lot. And one of the things I think probably 5 is distracting me is the keno, because it makes the 6 yellow numbers on the left and the green numbers vary 7 in ways that are not related to a mix between instant 8 and on-line. But there hasn't been much change in 9 Massachusetts in any of their categories; whereas, for 10 the most part, change has been pretty significant. 11 Is Massachusetts the oldest lottery on 12 that page? 13 MR. TIRLONI: I believe that's true -- I 14 don't know if they're older than New York. 15 COMM. COX: But they're certainly older 16 than Texas, they're certainly older than Georgia and 17 Florida, and for sure California? 18 MR. TIRLONI: Yes, that's correct. 19 COMM. COX: Okay. So their relative 20 stability could be related to maturity; whereas, more 21 change might be expected in areas where it's new? 22 MR. TIRLONI: Newer lotteries, yes. 23 COMM. COX: That's about all I can see 24 up there. Probably if I saw this again, I would want 25 to ask you to do two things, and that is round to even 0083 1 percentages, so we don't have so many numbers up 2 there, and footnote the other but make the percentages 3 of instant and on-line the percentage of the total of 4 those two rather than of the three. And I might be 5 able to see other things. 6 MR. TIRLONI: Okay. 7 CHAIRMAN CLOWE: I've never been 8 hesitant to make generalizations, so I'm going to make 9 some. And I invite -- 10 COMM. COX: Good. 11 CHAIRMAN CLOWE: -- your disagreement, 12 and yours as well, Robert and Gary and Kathy. It 13 would be meaningful to me to have the per capita 14 percentage up there with this, because my sense is 15 that Texas probably has one of the lowest per capita 16 playership of any of the states you've chosen here. 17 MR. TIRLONI: I have that for you on the 18 next slide. 19 CHAIRMAN CLOWE: Okay. You know, I just 20 think there's a culture in New York and Massachusetts 21 of gaming that has been in place for a long time. And 22 I know that because I've been to those states and I've 23 seen what I've seen. And I think the culture in Texas 24 is still developing. 25 And we have conflicts in this state that 0084 1 are real and significant regarding people who are 2 opposed to legalized gaming, and we are respectful and 3 mindful of that body of influence. And I don't think 4 you see that in Massachusetts or New York. I don't 5 think you see it in Florida or California. It's not 6 as clearly defined as it is in this state. 7 I think that there has been a 8 concentrated effort in Texas on making the instant 9 tickets attractive, new, innovative, and I think the 10 on-line games in Texas are old hat. I don't think 11 they're really appealing to players. The Pick 3 has a 12 culture I think that follows it. I think the two big 13 on-line games Mega Millions -- well, I guess now 14 Kathy's numbers are going to show us in October that 15 maybe Pick 3 is bigger than Lotto Texas as a 16 percentage -- Lotto Texas down around 6 percent, Mega 17 Millions just over 8 percent without the multiplier. 18 And I think Pick 3 may be larger than Lotto Texas as a 19 percentage of overall revenue. The game is tired. 20 Even with the last change, we didn't see a kick. 21 And, you know, I saw a program, 60 22 Minutes I think last Sunday, about Internet gaming. 23 It's amazing what those people are doing with Internet 24 gaming and what a draw that is on the gaming dollar. 25 We can't compete for that dollar; we're not competing. 0085 1 That dollar is leaving the State of Texas, and that's 2 just the fact. 3 So, Commissioner Cox, my reaction to 4 this is that it's interesting to me that our sales 5 have topped now any year that we've had since this 6 operation was legalized. You hit the hot button when 7 you asked about the contribution. The answer is: No. 8 We're working harder, we're spending more money, we're 9 making more effort, and we're holding that 10 contribution to the school children of Texas to about 11 a billion dollars a year, and it's not going to get 12 any better in my mind the way we are. 13 And, you know, I think it's a tribute to 14 the staff and to the people who are charged with 15 running this operation that they have been able to 16 achieve the sales, maintain that level of a billion to 17 the Foundation School Fund, where we have to be 18 realistic. And certainly as a Commissioner, I see a 19 very broad picture, and I'm not having any lack of 20 understanding about where we are and what our problems 21 are in this state. 22 I'm very comfortable with the job that's 23 been done, and I don't think we can do much better, 24 quite frankly, and not have a change in the statute or 25 not offend those people who are opposed to gaming. We 0086 1 cannot compete against interstate Internet gaming. We 2 can't compete against casinos in other states. It's 3 beyond us. It's not legal, and we're not going to 4 entertain it. We don't have the authority, the legal 5 right to entertain it. 6 You know, when we see these numbers, I 7 think I understand them. And I'm not upset or 8 frustrated by: This is the game that we're playing, 9 and we have to stay within the bounds and play by the 10 rules. What's your thought? 11 COMM. COX: Well, I've over the years 12 looked at things with the staff that have been 13 intended to address those kinds of concerns. One of 14 the things that we've got out there -- and the 15 Internet is exemplary of that -- is that our culture 16 has moved away from the deferred gratification that 17 your and my generation prided itself in working for, 18 to the instant gratification of the younger people. 19 Well, somebody who is interested in 20 instant gratification isn't going to be playing Lotto 21 Texas. They might be playing a Scratch Off ticket 22 because they can find out right then and now. But as 23 far as putting down a dollar and coming back an hour 24 later or a day later or three days later, they're not 25 going to play that game. 0087 1 So the on-line games in large part are, 2 by definition, out of line with cultural preferences 3 in this country. Whether anything can be done about 4 that, I don't know. You know, I've talked to the 5 staff about, "Can you make the lotto game an instant 6 game, somehow using a table back behind the game to 7 say, 'Hey, by golly, you just one $63 million, and 8 here it is,' as opposed to having to wait until 9 Wednesday night?" 10 It's not a promising prospect because 11 there are too many hurdles it looks like to overcome 12 to get there. So I think we've done -- our staff has 13 done a lot of thinking about how to deal with the 14 issues that you addressed. And, as you say, there are 15 constraints out there. Some of them are legal. I 16 think some of them are the culture of this state that 17 just says the people of Texas aren't going to go for 18 that even if it is what you would ideally do if you 19 were guided only by profit and not by other 20 considerations. 21 CHAIRMAN CLOWE: And I think that's what 22 guides the States of Massachusetts and New York. I 23 point them out over California, Florida, or even 24 Georgia -- maybe Georgia would be in that group -- you 25 know, they're really out for more revenue and more 0088 1 contribution to whatever the destination is. 2 We have a unique situation in this state 3 in my mind that is very sensitive, I guess is the 4 right word, and we as Commissioners must be aware of 5 that at the highest level, where Director Sadberry and 6 his staff are more concerned with day-to-day and doing 7 the best job they can within their job descriptions. 8 And that's a role we Commissioner play, as I see it. 9 Director Sadberry has great experience along those 10 lines. 11 But this is an interesting problem to 12 deal with. And if we were in it just to make the 13 contribution to the school fund grow as a result of 14 increased revenues, there would be things that we 15 would be considering that are out of bounds for us as 16 it now stands. 17 COMM. COX: Well, one way of looking at 18 this is on the product life cycle basis. The product 19 life cycle curve tends to rise sharply as a product is 20 introduced, then flatten out and ultimately decline. 21 And the challenge I think that we have is that while 22 the individual products within our product line are 23 subject to the product life cycle -- and Lotto Texas 24 has clearly gone up, has reached a peak and it's 25 declining. Our goal is to try to find new products, 0089 1 if we can, that will keep the product life cycle of 2 the Texas Lottery portfolio as a whole on the rising 3 side of the graph. 4 CHAIRMAN CLOWE: That's exactly right. 5 And we're doing that in my mind within the instant 6 ticket venue. 7 COMM. COX: Yes. 8 CHAIRMAN CLOWE: I think an excellent 9 job has been done there. The payout is higher. 10 That's a basic that you begin with. And it goes back 11 to the very point that you made, that your mind seized 12 on with Kathy's presentation: You know, what's the 13 bottom line? And the result is that we're working 14 harder, we're spending more money, and we're getting 15 less for the school children of Texas in an area that 16 is now 73.87 percent of our total sales. I thought it 17 was -- it's higher than that in Fiscal '06, isn't it? 18 MS. PYKA: Yes. 19 CHAIRMAN CLOWE: It's over 76 percent? 20 MS. PYKA: Right. In Fiscal Year '06, 21 it's 75.8. So this is actual 2005. 22 CHAIRMAN CLOWE: See, we're looking at 23 your numbers. 24 MS. PYKA: Very good. 25 COMM. COX: Kathy, now that was a pretty 0090 1 meaningful increase and payout percentage. 2 MS. PYKA: Right. 3 COMM. COX: Sales were up and 4 contribution was flatter down. I haven't asked you 5 yet which. 6 MS. PYKA: On a cash basis, it's 7 slightly up over Fiscal Year 2005. 8 COMM. COX: But it's not the best we've 9 ever done? 10 MS. PYKA: It's not the best we've ever 11 done. Back in the years when the prize payout was 12 capped -- 13 CHAIRMAN CLOWE: When we were doing 14 3.7 billion without a lot of scratch-off tickets? 15 MS. PYKA: Right. So as you look at the 16 prize payout expense and looking at the cash-based 17 transfer, our prize payout in '05 was 60.8 percent. 18 It goes up to 61.2 in '06. 19 COMM. COX: And I don't think it would 20 surprise anyone in this room to know what I'm thinking 21 is that we're paying GTECH more than the school 22 children of Texas are getting anymore. 23 If we had had the contract amendment 24 with GTECH in place at the beginning of the fiscal 25 year here and it had gone from 73.8 to 75.8, sales had 0091 1 gone up, revenues had been basically flat, what would 2 the rebate, if you will, have been? 3 MS. PYKA: I would need to go back and 4 calculate that. I mean, we're looking at the first 5 rebate, the one, and looking at 1.2 percent. So I can 6 certainly look that up. 7 COMM. COX: Maybe for next meeting, if 8 you could run that. 9 MR. GRIEF: It would be some helpful 10 information -- and for the record, my name is Gary 11 Grief, Deputy Executive Director. 12 Kathy, do you recall -- or, Ben, you may 13 recall -- what the difference in the prize payout 14 percentage is from '05 to '06? 15 MS. PYKA: That's the number here. The 16 prize payout for '05 was 60.8 and for '06, 61.2. 17 COMM. COX: So that's the number we 18 would be looking at? 19 MR. GRIEF: Yes. 20 MS. PYKA: Right. And then I'll take 21 that and calculate the rebate from it. 22 MR. GRIEF: And just for the record, I 23 want to be sure that the Commission understands, yes, 24 revenue year-over-year did increase in '06. And I 25 believe this is the fourth or fifth consecutive year 0092 1 of increase -- 2 MS. PYKA: Of increase. 3 MR. GRIEF: -- net revenue at the bottom 4 line. 5 MS. PYKA: Right. 6 MR. GRIEF: We'll have those numbers for 7 you in -- 8 COMM. COX: So it did increase -- 9 MR. GRIEF: Every year. 10 COMM. COX: -- and has increased every 11 year -- 12 MR. GRIEF: Right. 13 COMM. COX: -- for four or five years. 14 It just hasn't gotten back to matching the '95, '96, 15 '97 kinds of years? 16 MS. PYKA: That's correct. 17 COMM. COX: So we're still improving? 18 MS. PYKA: Right. 19 CHAIRMAN CLOWE: An interesting number, 20 Commissioner, would be in '97, if that was the record 21 year until '06, what was the contribution that year 22 compared to the contribution this year? 23 MS. PYKA: In Fiscal Year 1997, the cash 24 basis transfer, which was the highest, was a billion 25 189 million. And we're looking at a billion 0093 1 28 million this fiscal year. So, I mean, we're 2 getting back up there. We're just not up to that 3 level in '97. 4 COMM. COX: Well, clearly there is a 5 reality in there that the contract amendment with 6 GTECH recognizes and addresses. 7 MS. PYKA: Right. 8 CHAIRMAN CLOWE: I appreciate this 9 conversation and this discussion. Thank you-all for 10 your input. I think it's important that we just not 11 look at these numbers and don't talk about them. I 12 think we need to talk about them in the public so that 13 we all understand what it is we're doing and what we 14 have to work with. 15 Did you have something further, Robert? 16 MR. TIRLONI: I did, sir. I had two 17 more slides. This next slide also -- again, there's a 18 lot of numbers. We're giving you follow-up based on 19 your request last month for per cap figures. And so 20 these percentages are very similar to what you just 21 saw on the previous slide. In fact, they're 22 identical. 23 But you, Mr. Chairman, asked for per cap 24 numbers. And, Commissioner Cox, you asked for 25 transfer percentage based on sales in these states. 0094 1 And so the highest per capita is, as you said a minute 2 ago, Chairman, Massachusetts does have the highest per 3 cap. 4 And you asked for two figures last 5 month, the per cap for all of the total sales, which 6 for Massachusetts is 698. And then we did drop out 7 the "Other" category, which is keno and VLTs, so that 8 you could see an apples-to-apples comparison when you 9 were comparing those per cap numbers to Texas numbers. 10 And when you do that, the per cap for 11 Massachusetts is 582. Our per cap is 160. See, 12 California has the lowest per cap. I should note for 13 you-all that California has a legislative mandate on 14 their return to the state, which limits their prize 15 payout and limits their marketing and sales efforts, 16 which has an impact on their success. 17 In terms of revenue -- 18 COMM. COX: Robert, excuse me, but we 19 have the same thing. Theirs must be in a different 20 form than ours, so it's more constraining than ours 21 is? 22 MR. TIRLONI: It is. Their highest 23 prize point on the instant ticket side is $5.00. 24 Sigames is the instant ticket vendor in California. 25 They have to monitor the number of games that are out 0095 1 on the street for sale very closely. And it's my 2 understanding that as they monitor that payout on 3 almost a daily basis, at different points in the year, 4 they actually have to go out and pick up games so that 5 retailers cannot sell games, to be able to manage that 6 payout and meet their legislative mandate. 7 COMM. COX: What a way to run a 8 railroad. 9 MR. TIRLONI: Yes, sir. 10 CHAIRMAN CLOWE: That's a real 11 constraint. 12 MR. TIRLONI: Which -- 13 CHAIRMAN CLOWE: Where do the proceeds 14 of their lottery, go, Robert? 15 MR. TIRLONI: In California? 16 CHAIRMAN CLOWE: Yes. 17 MR. TIRLONI: Education. 18 CHAIRMAN CLOWE: To the education fund? 19 MR. TIRLONI: Yes, sir. 20 CHAIRMAN CLOWE: That's interesting. 21 MR. TIRLONI: I believe that's the 22 reason that you'll see that California has the highest 23 percentage in terms of transfer. And I should note 24 that -- 25 COMM. COX: And the lowest per capita? 0096 1 MR. TIRLONI: Yes, yes. The government 2 transfer percentages that you see here, that 3 calculation that Kathy's group did is on the total 4 sales. They were not able to pull out the keno and 5 VLT transfers, so that's on the total. And you see 6 Massachusetts has the lowest percentage in terms of 7 transfer, at 21 percent. 8 CHAIRMAN CLOWE: But probably one of the 9 highest actual dollar amounts? 10 MR. TIRLONI: I think Kathy has that. 11 MS. PYKA: Looking at sales by state, 12 that is -- actually, New York has total sales by 13 state, is the 6.2. And then Massachusetts is at 4.5. 14 COMM. COX: I think the Chairman was 15 talking about revenue. 16 MS. PYKA: Oh, I'm sorry. On revenue of 17 the state, again New York is the highest. 18 MR. TIRLONI: Commissioners and 19 Chairman, you touched on this a second ago with your 20 comments. You-all had asked us to provide this 21 follow-up information. We kind of took the liberty of 22 coming up with a list of variables, and you discussed 23 some of those moments ago. You talked about the 24 culture in Massachusetts and New York. 25 And we identified a whole list of actual 0097 1 variables that we feel influences this data and, as 2 you said, makes it somewhat difficult to generalize or 3 apply the findings from Massachusetts, let's say, to 4 Florida or from Florida to Texas. And, actually, some 5 of these variables that we identified were part of the 6 Dr. Huff and Dr. Jarrett study. 7 I'll quickly run through these for you. 8 Product mix -- 9 CHAIRMAN CLOWE: Unless you want to 10 discuss them, we can read. 11 MR. TIRLONI: Okay. I was going to 12 elaborate on some of them, but -- 13 CHAIRMAN CLOWE: Go ahead. 14 COMM. COX: Well, let me suggest this, 15 Robert: You've got a report from Prof. Huff and 16 Dr. Jarrett that I don't believe they have presented 17 to this board. I think they were here once to answer 18 questions on it, but then they had to leave because we 19 had a big show before that. 20 MR. TIRLONI: I believe that's correct. 21 COMM. COX: I would really like, rather 22 than your going into these, to have -- Mr. Chairman, 23 with your permission -- have Prof. Huff and 24 Dr. Jarrett come and present that report to us. 25 MR. GRIEF: That's an excellent idea. 0098 1 The Commission will be glad to make that happen. 2 CHAIRMAN CLOWE: And, Robert, I didn't 3 mean to cut you off at all, because I think this is 4 really an excellent list and I would like to hear more 5 about it. And I think Commissioner Cox's suggestion 6 is an excellent one. Let's go into this in more depth 7 at a future meeting. 8 And I see Ms. Trevino in the audience. 9 My suggestion, Nelda, is that you think about this in 10 regard to your legislative briefings. I think some of 11 the questions that I have heard from members of the 12 Legislature is, "So what's really going on at the 13 Lottery? What's happening over there?" They have 14 some questions which I think some of the issues we've 15 had this discussion and some of the items that are 16 listed there as variables -- this is an excellent 17 list -- might broaden the understanding of some 18 members who want to know more about where we are in 19 the lottery operation. 20 MR. TIRLONI: We'll work with Gary and 21 Mike Fernandez and his research group on scheduling 22 something for a future meeting where Drs. Huff and 23 Jarrett can be here and go into more depth on these. 24 CHAIRMAN CLOWE: In the meantime, give 25 us this list, if you will. I want to think about 0099 1 it -- 2 MR. TIRLONI: Sure. 3 CHAIRMAN CLOWE: -- put it under my 4 pillow. 5 COMM. COX: You probably already sent me 6 one. But if you haven't, send me a copy of their 7 report because I would like to review it before they 8 make their presentation. 9 MR. TIRLONI: Okay. We'll do that. 10 And, Commissioners, Director Sadberry -- 11 COMM. COX: Let me go back on one 12 thing -- and, Gary, this goes to a discussion we were 13 having earlier -- I think where you were headed on 14 that was that there's so many variables by state that 15 it's difficult to draw conclusions looking at sales, 16 one state versus the other, and I totally agree with 17 that. 18 And I think there's a difference there 19 that I would point out between knowing what the 20 differences are and managing our business based on 21 those differences. I think it's our obligation to 22 know what those differences are. That doesn't mean 23 that we manage our business to try to narrow those 24 gaps or increase those gaps, as the case may be. 25 MR. GRIEF: That's exactly my point. 0100 1 Thank you for stating it better than I could. 2 CHAIRMAN CLOWE: And I think that's a 3 very important point because in my time on this board, 4 we are closer today to really understanding our 5 business than I think we've ever been. 6 COMM. COX: Yes. 7 CHAIRMAN CLOWE: I have seen in past 8 meetings, Commissioner Sadberry, these numbers be 9 reported and the Commissioners kind of said, "Oh, you 10 know, we sure hope it rolls the next time." And 11 that's been sort of the depth of understanding that 12 I've had myself at times. I think we're really 13 getting to understand what it is that's influencing 14 our business better than we ever have in the past. 15 MR. SADBERRY: Mr. Chairman, I concur. 16 One of the tremendous advantages that I enjoy is 17 having seen it from both sides in that respect and 18 also having seen two other things. One is the 19 progression of time and the maturity of the lottery 20 and the Lottery Commission that has accompanied that, 21 as well as the forward progression in staff and the 22 staff of dedication to these functions. 23 And I would state without question in my 24 mind that this is at a high point of our abilities to 25 analyze and to function effectively. And I think I've 0101 1 never seen it better, and I would rate it at 2 excellence. And that's from having a day-to-day and 3 hands-on experience in this capacity and comparing 4 that to prior times. 5 CHAIRMAN CLOWE: Thank you. 6 MR. TIRLONI: And, Commissioner, just to 7 follow up on your comment about noting the 8 differences, we take a pretty active approach. My 9 staff, once a quarter, participates in conference 10 calls with all of the other Mega Millions member 11 states. We talk with their product marketing teams in 12 all of those states. And it's basically a sharing of 13 new game launches, game strategies, promotional 14 strategies, so basically sharing information amongst 15 states and getting that knowledge and getting that 16 learning. 17 And then, as you said, you may not be 18 able to generalize that or directly apply it, but you 19 can take that learning and maybe tweak it a little bit 20 for what we're doing and how we're doing it. 21 CHAIRMAN CLOWE: Have you planted any 22 seeds, Robert? 23 MR. TIRLONI: Possibly. We are focused 24 on on-line. I will make that point. That's our goal, 25 is to increase on-line. We realize if we can do that, 0102 1 that will make a difference to the bottom line and to 2 our contributions to the school fund. 3 COMM. COX: Well, one of the things that 4 it seems to me is, if you're going to increase 5 on-line, you've got to be targeting your games to 6 people that are not oriented toward instant 7 gratification. And I think -- I don't know what that 8 demographic is, but I've got some suspicions about it. 9 And I think I know a lot about what demographics are 10 not oriented toward instant gratification as I tried 11 to improve the on-line game, because it's slow. 12 MR. TIRLONI: Yes, sir. 13 COMM. COX: My son wouldn't. No chance; 14 never happen. 15 MR. TIRLONI: I agree. It will be 16 interesting. 17 And, Commissioners, Director Sadberry 18 has a brief update for you on the Ipsos Reid research 19 data. 20 MR. SADBERRY: Commissioners, for the 21 record, I am Anthony Sadberry, Executive Director. 22 And usually at this point on this particular agenda 23 item, you have presented to you slides and some 24 comments on the Ipsos Reid tracking results. That is 25 not included in today's presentation. And I want to 0103 1 advise you that the agency's Research Department is 2 working on a presentation which we anticipate 3 presenting in the first Commission meeting in October, 4 and that will give you an historical perspective on 5 the results of the Ipsos Reid monthly tracking reports 6 related to the public's overall opinion of the Texas 7 Lottery and their agreement or disagreement with the 8 statement that the Texas Lottery is operated fairly 9 and honestly. 10 I've asked staff -- and I note that you 11 have made comments on this item at the last two or 12 three Commission meetings -- I've asked the staff to 13 analyze the data collected over a broad time period so 14 that we can determine the full value of the 15 information being provided and also determine the 16 optimal reporting intervals for this information. 17 Additionally -- and Ms. Pyka has already 18 commented on this and alluded to it -- the agency's 19 Office of the Controller and Products Department are 20 working on a presentation for the second Commission 21 meeting in October that will provide you with the 22 final sales and transfer data for Fiscal Year 2006, as 23 well as a comparison to Fiscal Year 2005 sales data by 24 game. 25 MR. TIRLONI: That concludes our 0104 1 presentation for this morning. We would be happy to 2 answer any other questions. 3 CHAIRMAN CLOWE: Are there any 4 questions? 5 COMM. COX: No, sir. 6 CHAIRMAN CLOWE: Thank you -- 7 MR. TIRLONI: Thank you, sir. 8 CHAIRMAN CLOWE: -- lady and gentlemen. 9 AGENDA ITEM NO. III 10 CHAIRMAN CLOWE: And then next, Item 11 III, report, possible discussion and/or action on 12 transfers to the State and the agency budget. 13 Ms. Pyka. 14 MS. PYKA: Thank you. Again for the 15 record, my name is Kathy Pyka, Controller for the 16 Lottery Commission. 17 Tab 3 includes our information on the 18 agency's financial status. The first report in your 19 notebook reflects transfers and allocations to the 20 Foundation's School Fund, the allocation of unclaimed 21 prizes for Fiscal Year 2006. 22 Our total transfers to the state 23 amounted to $1.1 billion. This represents a 24 .8 percent increase in the total amount transferred 25 over Fiscal Year 2005. 0105 1 The second page in your notebook 2 provides you the detailed information on the monthly 3 cash transfer. We've noted for you the total amount 4 transferred to the Foundation's School Fund, which was 5 $1.029 billion, and the balance of $55.6 million 6 related to the transfers from unclaimed lottery 7 prizes. 8 The next page in your notebook is simply 9 the calculation of the transfer to the Foundation's 10 School Fund. 11 The last page in your notebook related 12 to transfers provides you the cumulative data from 13 Fiscal Year 1992 to the present. This notes cash 14 basis transfers to the Foundation's School Fund for 15 Fiscal Year 2006 totaled $1.029 billion, with a 16 cumulative transfer to the Foundation's School Fund of 17 $8.7 billion. 18 I will now move on to our budget data. 19 The next document in your notebook includes the Fiscal 20 Year 2006 monthly finance summary for the Commission, 21 and this is a document in which I provided you a 22 revised copy this morning. Our Lottery account budget 23 the Fiscal Year 2006 was $191.6 million. This budget 24 includes the increases that we placed in for Rider 13 25 for the increase to the GTECH lottery contract. Of 0106 1 this total amount, 93.1 percent has been expended 2 through Fiscal Year 2006, and that reflects our major 3 vendor accruals. We will update it further to reflect 4 all of our final accruals and encumbrances. 5 Our bingo operations budget, funded by 6 GR, was $14 million for the fiscal year. And of that 7 total amount, we expended 98.8 percent of the budget. 8 With that, that concludes my 9 presentation. I would be happy to answer any 10 questions. 11 COMM. COX: No questions, Mr. Chairman. 12 CHAIRMAN CLOWE: Thank you, ma'am. 13 MS. PYKA: You're welcome. 14 COMM. COX: How about a break, 15 Mr. Chairman? 16 CHAIRMAN CLOWE: We'll take a short 17 break. 18 Ms. Melvin, we'll call on you when we 19 come back to order. 20 (Off the record: 11:18 a.m. to 11:25 21 a.m.) 22 CHAIRMAN CLOWE: We'll come back to 23 order. 24 25 0107 1 AGENDA ITEM NO. V 2 CHAIRMAN CLOWE: Next, Item V, 3 consideration of, possible discussion and/or action on 4 the external and internal audits and/or reviews 5 relating to the Texas Lottery Commission and/or the 6 Internal Audit Department's activities. 7 Ms. Melvin. 8 MS. MELVIN: Good afternoon, 9 Commissioners. For the record, my name is Catherine 10 Melvin. I'm the Director of the Internal Audit 11 Division. I have a very brief item of update. It's 12 related to the annual financial audit. I just want to 13 keep you apprised of that. 14 We're still on schedule to have the 15 auditors on site beginning in October. An entrance 16 conference is scheduled for Monday at 9 o'clock. And 17 I believe, Commissioner Cox, you're invited to that 18 entrance conference. 19 As with last year, the state auditor's 20 office will be managing the audit. Actually 21 conducting the work will be the firm Maxwell, Locke 22 and Ritter. 23 And that's all I have. 24 COMM. COX: No questions. 25 CHAIRMAN CLOWE: Thank you, ma'am. 0108 1 MS. MELVIN: Thank you. 2 AGENDA ITEM NO. VI 3 CHAIRMAN CLOWE: Next, Item VI, report, 4 possible discussion and/or action on the agency's 5 contracts. 6 Mr. Jackson. 7 MR. JACKSON: Good morning, 8 Commissioners. For the record, my name is Tom 9 Jackson. I'm the Purchasing and Contracts Manager for 10 the Commission. 11 Commissioners, in your notebooks under 12 Tab No. 6 is a report on prime contracts that has been 13 updated for your review. 14 I will be happy to answer any questions. 15 CHAIRMAN CLOWE: Are there any 16 questions? 17 COMM. COX: No, sir. 18 CHAIRMAN CLOWE: Thank you, sir. 19 AGENDA ITEM NO. VII 20 CHAIRMAN CLOWE: Next, Item VII, report, 21 possible discussion and/or action on the 79th 22 Legislature. 23 Ms. Trevino. 24 MS. TREVINO: Good morning, 25 Commissioners. For the record, I'm Nelda Trevino, the 0109 1 Director of Governmental Affairs. 2 We are preparing for our next agency 3 legislative briefing scheduled for September the 27th. 4 We intend to provide updates on various matters, 5 including fiscal year and lottery sales and revenue, 6 the agency's legislative appropriations request 7 submission, the recent GTECH contract amendment and 8 the status of various audit reports related to the 9 agency. 10 On September the 28th, the agency will 11 appear before a joint budget hearing conducted by 12 staff of the Legislative Budget Board and the 13 Governor's Office of Budget Policy and Planning. We 14 will present the agency's legislative appropriation 15 request and respond to any questions that the staff 16 may have. We are scheduled to appear before the 17 Senate Committee on Finance on October the 3rd, also 18 for a presentation on the agency's LAR. 19 In preparation for these hearings, 20 Governmental Affairs has facilitated meetings between 21 the Executive Director and committee members and key 22 staff of the House Appropriations and Senate Finance 23 Committees. 24 The Governmental Affairs staff continues 25 to monitor interim House and Senate committee -- 0110 1 interim hearings, and we will keep you posted on 2 developments related to any of these legislative 3 interim activities. 4 This concludes my report, and I'll be 5 happy to answer any questions 6 CHAIRMAN CLOWE: Commissioner Cox, I am 7 committed elsewhere on October the 27th and 28th and 8 will be unable to attend any of the meetings 9 Ms. Trevino mentioned. I am available for the 10 October 3rd meeting if it is your desire to have me go 11 to that meeting. So you might check your schedule and 12 advise Ms. Trevino what your plans are. 13 COMM. COX: That would be for the 27th 14 and 28th? 15 CHAIRMAN CLOWE: Yes, sir. 16 You will be attending the meeting on the 17 3rd of the Senate Finance Committee? 18 COMM. COX: I will unless you prefer to 19 be there yourself. 20 CHAIRMAN CLOWE: I prefer that you be 21 there. 22 COMM. COX: Okay. Very good. 23 CHAIRMAN CLOWE: Thank you, Nelda. 24 COMM. COX: October 27th and 28th, did 25 you -- 0111 1 MS. TREVINO: September. 2 CHAIRMAN CLOWE: September. 3 COMM. COX: September 27th -- 4 MS. TREVINO: September 27th. 5 COMM. COX: -- and 28th. That's when 6 you're going to be out, Mr. Chairman? 7 CHAIRMAN CLOWE: Yes, sir. 8 COMM. COX: Okay. Now, what's happening 9 on those days again? 10 MS. TREVINO: September 27th is when we 11 will be holding our agency legislative briefing, 12 beginning at 10 o'clock. And then on the 28th is the 13 agency's appearance before the Joint Budget hearing 14 before staff of the Legislative Budget Board and the 15 Governor's Office of Budget Planning and Policy. And 16 that, Commissioner Cox, I believe begins at 17 11 o'clock. 18 COMM. COX: And again, Mr. Chairman, was 19 it your pleasure that I attend both of those meetings? 20 CHAIRMAN CLOWE: Only if you care to. 21 And I thought you might want to discuss both of those 22 meetings with Nelda to determine what her plans are. 23 And I just want to let you know that I wouldn't be 24 able to attend either one of those, depending on, you 25 know, how you end up after you visit with her in 0112 1 regard to your feelings. 2 COMM. COX: Very good, sir. 3 CHAIRMAN CLOWE: I'm not asking you to 4 attend. It's simply whatever you think, along with 5 her, is best. 6 COMM. COX: All right, sir. 7 CHAIRMAN CLOWE: But I will try to be 8 there on the 3rd. 9 MS. TREVINO: We'll follow up with you, 10 Commissioner Cox, about those two -- 11 CHAIRMAN CLOWE: Great! Glad to see you 12 back, Nelda. 13 MS. TREVINO: Thank you very much. I 14 appreciate that. 15 AGENDA ITEM NO. VIII 16 CHAIRMAN CLOWE: Commissioner, if you're 17 agreeable, I move we go into executive session. 18 COMM. COX: Yes, sir. 19 CHAIRMAN CLOWE: At this time I move the 20 Texas Lottery Commission go into executive session to 21 deliberate the duties and evaluation of the Executive 22 Director, Deputy Executive Director, Charitable Bingo 23 Operations Director and Internal Audit Director and 24 deliberate the duties of the General Counsel pursuant 25 to Section 551.074 of the Texas Government Code, to 0113 1 receive legal advice regarding pending or contemplated 2 litigation and/or to receive legal advice pursuant to 3 Section 551.071(1)(A) or (B) of the Texas Government 4 Code and/or to receive legal advice pursuant to 5 Section 551.071(2) of the Texas Government Code, 6 including but not limited to: 7 Cynthia Suarez vs. Texas Lottery 8 Commission; 9 Shelton Charles vs. Texas Lottery 10 Commission and Gary Grief; 11 Stephen Martin vs. Texas Lottery 12 Commission; 13 Employment law, personnel law, 14 procurement and contract law, evidentiary and 15 procedural law and general government law. 16 Is there a second? 17 COMM. COX: Second. 18 CHAIRMAN CLOWE: All in favor, please 19 say "Aye." 20 COMM. COX: Aye. 21 CHAIRMAN CLOWE: Aye. 22 The vote is 2-0. The Texas Lottery 23 Commission will go into executive session. The time 24 is 11:32 a m. Today is September 20, 2006. 25 (Off the record: 11:32 a.m. to 12:55 0114 1 p.m.) 2 CHAIRMAN CLOWE: We'll come back to 3 order. The Texas Lottery Commission is out of 4 executive session. The time is 12:55 p.m. 5 Is there any action to be taken as a 6 result of the executive session? 7 AGENDA ITEM NO. IX 8 CHAIRMAN CLOWE: If not, let's move on 9 to Item No. IX, report by the Executive Director 10 and/or possible discussion and/or action on the 11 agency's operational status and FTE status. 12 Director Sadberry. 13 MR. SADBERRY: Commissioners, for the 14 record, my name is Anthony Sadberry, Executive 15 Director. 16 We received an official announcement 17 from GTECH on Monday regarding some organizational 18 changes to GTECH's upper management that affect Texas. 19 Larry King has been named Vice President of Sales, 20 GTECH Americas, reporting to Alan Eland, Senior Vice 21 President at GTECH Americas and will no longer be 22 directly responsible for oversight for the GTECH Texas 23 account. 24 Scott Gunn has been promoted to Regional 25 Vice President, Region 2, replacing Larry King. Scott 0115 1 brings an array of experience to this role, including 2 12 years at GTECH, most recently as California Account 3 Development Manager where he was responsible for the 4 leadership and financial performance of GTECH's 5 business operations. We look forward to working with 6 Scott. 7 There are materials in your notebooks 8 for your review regarding status and positions as of 9 September 12, 2006. 10 I would be happy to answer any questions 11 at this time. 12 COMM. COX: No questions. 13 CHAIRMAN CLOWE: Thank you, 14 Mr. Sadberry. 15 AGENDA ITEM NO. X 16 CHAIRMAN CLOWE: Next, Item X, report by 17 the Charitable Bingo Operations Director, possible 18 discussion and/or action on the Charitable Bingo 19 Operations Division's activities. 20 Mr. Atkins. 21 MR. ATKINS: Commissioners, I don't have 22 anything to add to the information that's contained in 23 the notebook, but I would be happy to answer any 24 questions you may have. 25 COMM. COX: No, sir. 0116 1 CHAIRMAN CLOWE: Thank you, sir. 2 AGENDA ITEM NO. XI 3 CHAIRMAN CLOWE: Item No. XI. Are there 4 any persons wishing to make public comment to the 5 Commission at this time? 6 Commissioner Cox, any additional 7 business? 8 COMM. COX: No, sir. 9 AGENDA ITEM NO. XII 10 CHAIRMAN CLOWE: Thank you all very 11 much. We are adjourned. 12 (Texas Lottery Commission meeting 13 adjourned: 12:58 p.m.) 14 15 16 17 18 19 20 21 22 23 24 25 0117 1 C E R T I F I C A T E 2 STATE OF TEXAS ) 3 COUNTY OF TRAVIS ) 4 I, Aloma J. Kennedy, a Certified 5 Shorthand Reporter in and for the State of Texas, do 6 hereby certify that the above-mentioned matter 7 occurred as hereinbefore set out. 8 I FURTHER CERTIFY THAT the proceedings 9 of such were reported by me or under my supervision, 10 later reduced to typewritten form under my supervision 11 and control and that the foregoing pages are a full, 12 true and correct transcription of the original notes. 13 IN WITNESS WHEREOF, I have hereunto set 14 my hand and seal this 4th day of October 2006. 15 16 ________________________________ 17 Aloma J. Kennedy Certified Shorthand Reporter 18 CSR No. 494 - Expires 12/31/06 19 Firm Certification No. 276 Kennedy Reporting Service, Inc. 20 Cambridge Tower 1801 Lavaca Street, Suite 115 21 Austin, Texas 78701 512.474.2233 22 23 24 25