Audio Transcript:
LAURIE HOWELL: Up next on Science 360 Radio, an exclusive interview with aviation security expert, Sheldon Jacobson, an NSF-funded scientist whose operations research continues to influence policy and practice, including screening activities at airports. Jacobson's studies have found that comprehensive airport screening may actually be less effective than more targeted screening based on passenger activities.
SHELDON JACOBSON: In our very early research when we first began working on our NSF grant, we had estimated that 60 to 70 percent of the population can be viewed as being very, very low risk and subject to much less screening. Since the TSA has launched a TSA pre-check, their estimates, from what I've heard and it seems to vary, is they're hoping to get a target of 50 to 60 percent. So, it's amazing that our predictions of ten years ago are relatively close to the predictions that they have today.
JOSH CHAMOT: And most of what you're doing is looking at the activities of individuals, for example, in an airport and trying to figure out what factors would raise red flags. How does your approach tackle the almost countless factors that can be involved with any individual walking into an airport, let alone the thousands of individuals or hundreds of thousands of individuals who go through an airport in a month?
SHELDON JACOBSON: Well, the aviation security system is really a collection of smaller components and our research tries to focus on decomposing the bigger problem into these more manageable components. So, we look at risk assessment, we look at security resource allocation, and trying to match the two. One of the issues that came out relatively recently from the TSA has been risk-based systems. We've actually been looking at models and gaining insights into risk-based systems for around eight years, well before it became introduced by the TSA and many of the ideas and concepts that we have, in fact, disseminated in research papers and shared at research conferences are now at the foundation of the thinking that we see at airports and will see over the next three to five years.
JOSH CHAMOT: So, when you talk about risk-based assessment, what exactly are you talking about in terms of the execution of what's going on?
SHELDON JACOBSON: What it means is that we attempt to match the risk of the passenger pool, the risk of the passenger pool with the resources that are available. So, for example, if we have uniform screening, which means that everybody is screened the same, we have actually looked at those models and demonstrate that they are, in fact, very, very unsafe compared to using resources in a more targeted manner. And we've actually been able to quantify the measures of risk and the costs associated with those.
JOSH CHAMOT: So, when you say that you're focusing on individuals in a targeted manner, you're not picking somebody out of a crowd because of what they're wearing or something like that. What are some of the factors you're actually looking for in this type of analysis?
SHELDON JACOBSON: It's basically information analysis. We're simply looking at the information about people who travel. When you get on an airplane, you've bought an airplane ticket. How you bought your airplane ticket is an important factor as part of that information. Do you, in fact, buy your ticket an hour before the flight or do you buy it three months before the flight? This is important information. The more information that we as travelers are able to, in fact, provide to the TSA, what are models have demonstrated is that those people show a marked drop in their risk and, as a result, we want to use security resources that are commensurate with that risk level. On the other hand, people who either choose not to give information or are trying to hide, and sometimes it's hard to discern between the two, then we have to subject them to more screening. That's perfectly understandable but it becomes a choice. The vast majority of people really aren't a risk so we are trying to find a needle in the haystack. What we have done with our research is said that it's much easier to simply partition the haystack to the group that we know is safe rather than trying to find the needle. And by doing that, we actually end up with a more secure system at lower cost with more convenience to the general public. It's a win-win-win situation.
JOSH CHAMOT: So if somebody wants to cause harm, can they rig the system? Can they look for factors that would be picked up and try to avoid those factors as they go through their process?
SHELDON JACOBSON: Well, no system is 100 percent foolproof but one of the things that we've found in our research is that when you treat everybody the same, the system is actually more vulnerable because if you're going to treat people the same, you tend to treat them at a very high level of risk. What that means is that the needles in the haystack, the really true risky people don't stand out as much. If we lower the overall perception of risk in the entire passenger population, then the people who are risks stand out like a sore thumb, basically. And our research demonstrates you can end up having a more secure system. It's very counterintuitive.
JOSH CHAMOT: In addition to looking at airport security, you've been looking at vaccines and how they're distributed and stored and a lot of activities around vaccine usage. You've been looking at sports. You've been looking at a whole bunch of different things. How do you decide that a problem warrants a statistical approach and then how do you take your techniques and bring them into that environment?
SHELDON JACOBSON: Well, I view life as a laboratory for discovery and, quite often, when young people, new graduate students, potential graduate students, I get contacted by people outside of our university saying, "How do you start working on a research problem?" And, literally, the world that we live in is our laboratory. If we look around, the issues to explore are before us. The question is what tools, what technical abilities do we have to actually shed light and make a difference on these problems? And I've been very fortunate to work on problems before they've become bigger than life, more prevalent in the view of society as a whole. Our work on vaccine is a great example of that. We had met someone from the CDC back in 1996 and he had talked about a problem that he viewed to become very, very relevant in the next 15 years but it wasn't relevant at that time. So, we were really handed this problem and it turned out to be how do you design pediatric vaccine formularies, which are the sets of vaccines that you administer to children in the United States when you have more complex vaccines being added to the schedule, the recommended childhood immunization schedule? And we ended up building some very interesting mathematical programming models to solve the problem and we did it so well that we solved the problem and we still had 12 years to spare. Eventually, the National Science Foundation started to fund this work because there were some very fundamental modeling issues that required attention beyond vaccines. They were just fundamental modeling issues, and we were able to make some important advances. It eventually led, in fact, to some work on vaccine stockpiling and we were able to, in fact, assess the pediatric vaccine stockpile policies of the United States and demonstrate where they had strengths and where they had more limitations. The interesting thing about that 15-year problem is that that problem actually became very relevant in 2008 when there were multiple combination vaccines and the public health community said, "What do we do?" And our papers were the foundational reference to what people should do.
JOSH CHAMOT: So what did you recommend?
SHELDON JACOBSON: What we recommended is that there are ways that you can evaluate and compare these combination vaccines. We provided a methodology. We eventually created a website that people were able to use for a while and we presented this work at conferences, the National Immunization Conference a few years ago. We actually gave a talk specifically about this issue to help people in the public health community with this decision. The thing is there's no right and wrong answer. There's just different answers based on different immunization environments, and that's what our research demonstrated. But it required us to build models that were not obvious to the naked eye. People would say, "What do those models have anything to do with vaccination?" But, ultimately, we did publish much of this work in both the medical community as well as the engineering community and it was well received in both.
JOSH CHAMOT: So it seems like you can apply this almost anywhere. What are you looking at next? Where are you going to be applying some of these models and then developing new models?
SHELDON JACOBSON: Well, the theme of decision-making under uncertainty is something that we're still working on and we're looking at a certain type of system where you make these decisions in real time under very highly dynamic and changing conditions. The potential application of this would be certainly in aviation security with matching passengers as they come in with resources. You can even do it with dynamic investment allocation but much more relevantly, it can even help people figure out when they should buy a home because there's a timing issue and there's many choices that are coming on and off to the market. How do you make that decision? So, once again, you'd look at the final product and say, "Oh, this helps people buy a home," but within it are very complex models which go into that decision-making process.
JOSH CHAMOT: Which project that you've worked on has really surprised you the most? It seems like you've delved into so many different areas. Have there been things that have jumped out?
SHELDON JACOBSON: Around 2005, we started to work on the relationship between transportation and obesity and what we wanted to do is try and understand obesity, the epidemic which is crippling our nation. Can we get some insights that maybe most people are not seeing? So, we tried to, in fact, quantify the impact of obesity on fuel consumption in non-commercial vehicles and it turns out the number was on the order of around one billion gallons per year in the United States. But the interesting thing is that the reverse question is does driving, in fact, cause obesity? We were able to come with a model to predict obesity based on how much we drive and the irony is that the lag in time is on the order of six years. So, when people drive more today, it takes around six years for that to show up in obesity. So, around six years ago, when driving levels went down because the economy was a little softer and gas prices started to spike upward, we now see an obesity drop and, in fact, there was big news of this just in 2012 that the obesity rates are leveling off and our models, in fact, predicted that.
JOSH CHAMOT: With the work that you're doing, you're taking a complex world and you're creating simplified models that we can then use to advantage and to actually accomplish some goal or see something in a way that we wouldn't have seen it before. Are there situations that you've run into where the modeling simplifies a little too much or maybe misses some of the key factors and then does that go back and guide how you apply models in the future? Everything can't be modeled 100 percent. We can never have--otherwise it's not a model, it's a recreation of reality.
SHELDON JACOBSON: Correct.
JOSH CHAMOT: So, if you're always going to have these little differences, how do those gaps guide what you do next?
SHELDON JACOBSON: Well, the world is very complex and when we create models, it is ultimately a simplification of a very complex system. We then do our analysis, our manipulation, our algorithmic design around those models but ultimately we then have to apply it back to the real system. If, when we do that, we find that our insights are flawed or not optimal, we then have to go back and look and say, "What have we missed in our model?" And that, in fact, creates the opportunity for new insights, either by extending the model or potentially creating a new one, a so-called transformational focus of the research, because sometimes we have a preconceived idea of what the model should look like. And I've been in many situations where, quite frankly, I was wrong and it was only after we tried to apply the results from our modeling exercise did we realize, oh, it's not making sense and we saw what the issue was, brought it back and realized that our model did not contain certain components and we had to create a completely new view of the modeling process.
JOSH CHAMOT: And then define a little bit what these models are because there are a lot of mathematical approaches to not just the numbers but the numbers represent certain things so you have to pick certain variables, you then have to pick certain mathematical techniques to look at those variables and look at how they're interacting with each other. What exactly goes on when you design these models?
SHELDON JACOBSON: The models that I work with tend to be within the rubric of the field of operations research, which is now often referred to advanced analytics. And the kinds of tools that are available in our field include models like linear programs, integer programs. I deal a lot with stochastic models which involve things like Markov decision processes, for example. These are the kinds of models. If you simply look at them, they often look like equations but when you put them together and manipulate them and see insights and relationships between the variables and the models. That is when you start to get certain insights. For example, I may come up with a model and establish monotonicity of a certain variable as we approach optimality, which means that the variables will be either non-increasing or non-decreasing. They're either going up or they're going down and that may help me figure out what the optimal parameter is because of that structure. Those are the kinds of things that we like to look at and extract from the models. And it may sound very simplified but the fact is for complex systems, although models are flawed, they can be very, very useful and quite often point in the direction to making significant and substantial changes in the real system.
JOSH CHAMOT: So, these are all tools. I mean, these are the tools in your toolkit that you can then use to get just enough information out of this complex world we live in to really come up with some conclusions.
SHELDON JACOBSON: Along the way, we're simply not focusing on solving a specific problem. We do have a bigger picture view because when I am looking at a particular complex system and I do create a model, I'm always thinking in the back of my mind the potential thought that this model could be useful somewhere else, and sometimes I will build a model that may apply, in fact, to aviation security but then it may also apply in another completely unrelated domain involving vaccines, for example. And although it's the same basic structure of the model, the fact of the matter is it could be adapted and applied in many different domains and that, in fact, is the science of discovery in engineering.
JOSH CHAMOT: And that's led you to some pretty creative applications, including the bracketology work that you've been doing.
SHELDON JACOBSON: Yeah. I got interested in college basketball when I was a graduate student and just watched the games and followed March Madness. And then a few years ago, I asked a very simple question. It always seems that a team seeded number one, number two, or number three wins the national championship, and the statistics bear that out. So, we said, "Is it true that these teams really are dominant compared to the rest of the field and how are they relative to each other?" If a team is seeded number one, do they really have an advantage over a two or a three? So, we looked at the numbers and we analyzed the statistics for the modern era of the tournament, which is since 1985, and noticed that as the tournament progresses, the advantage of being a so-called top seed, a one versus a two, versus a three begins to wane. Then what we did is we developed some probabilistic models to, in fact, assess the probability that certain seed combinations will promulgate through the bracket. And we were able to, in fact, come up with a very, very strong model. It fit extremely well and we published both of these works and it's gotten a lot of national attention. We created a website which was called BracketOdds, which gets thousands of visitors for the 48 hours after selection Sunday and before the tournament actually begins. It's fun research. And, this year, I had the opportunity to kind of write something for USA Today about it and the letter that I sent in focused on the fact that, yes, we look at the basketball as being an athletic event but it's really also a mathletic event that we have all these people that they're trying to figure out their brackets and it's a STEM activity in that way and I really do believe strongly that we can use sporting events like this to help promote science, technology, engineering and math because it's interesting and exciting to the very people we're trying to reach.
JOSH CHAMOT: Well, and then you can turn around and tell people that this toolkit they're using to figure out whether Kentucky makes it to the Final Four is the same toolkit that you can use to actually solve some very complicated problems in the world.
SHELDON JACOBSON: Exactly. And hopefully they will be inspired to say, "Gee, if this can help me with my bracket, what other exciting things can it do?" When I look at a problem, you can see it in a way that anybody can interpret it but as an engineer, as a person who deals with numbers and mathematics, everything could be represented with zeros and ones and when you transform a real system into zeros and ones and put it onto a computer though modeling and analysis, you can literally transform a situation and remove yourself from something that seems good to something that's better and hopefully that something will be the best. And the transition is the exciting work of the science and once you get to the final solution, people will say, "Oh, well, why wouldn't you do that before?" It's because the path was unclear. What research enables and facilitates is the path of discovery. That's the greatest thing that I've discovered and the excitement which is why I like doing research.
JOSH CHAMOT: All the activities we've talked about, I imagine there are some problems you'd still like to solve. What would they be?
SHELDON JACOBSON: Well, there are many problems I would like to solve. In the world of computation of complexity, if I can solve the classic P=10P problem, that would be kind of nice and it also has a $1 million prize but I don't think--
JOSH CHAMOT: What is the P=10P?
SHELDON JACOBSON: Oh, that's a problem whereby certain common autorial optimization problems are classified in a group called NP-complete, which means that they're really, really hard problems and there are no known algorithms that will run efficiently on a computer. It's a very classic problem. It's very well known. It's one of those great open problems that's been around for decades now and there's a prize of $1 million. Every computer scientist and operations researcher would love to solve it and I don't think I'm going to solve that one. But I will give you a real one, and this is a problem I've only been working on for a few years. In fact, some initial work with it was funded just a few years ago by the National Science Foundation and it has to do with the relationship between causality and observational data. Just to give you an idea, in the medical world, a very important theme right now is comparative effectiveness, which means that when you introduce a new procedure, a new technique, does it, in fact, improve existing methods? Well, in the medical world and in the social sciences, we often run double blind design of experiments but you can't always do that. So, you then have to resort to observational data, which means you have a group of data and then you classify them and meld them together in a way that looks like it's randomized but it's not quite. We've been working on this problem, and it's a very classic problem that's been around for over 40 years, and we've come up with a new approach using, of all things, discreet optimization models to solve the statistics problem, and we're really excited about this. We just think it's a way that in this big data environment that we're working in when you have so much data, our methodologies have the potential to transform the way potentially drug companies do their testing, how the social sciences do their testing of cause and effect, how the Food and Drug Administration makes decisions. The reach of that is so great and we're just so excited because the preliminary work that we've done is so positive that we're convinced that it really could make a difference.
JOSH CHAMOT: Sheldon, this has been really cool. Thank you for joining us.
SHELDON JACOBSON: Well, thank you very much for having me.
