Have you ever wondered about the logistics of what it takes to pull off transporting you and your bags from your home to your destination and back again? Have you ever imagined what it might be like to work for the airlines and be in charge of scheduling flights? Believe it or not, the computational complexity of airline travel planning is some of the most complicated in the world. Every traveler understands the criticality of time in flying around the globe. And if your business is time-critical, then you understand the importance of rapid decision making in the face of complex data sets.
As all of us have seen, an airline that is not driven by the numbers will bleed to death very fast. As a way of understanding the fundamental challenge and opportunities of big data, consider the complexity of the airline transportation business and the combinatorial explosion of flight-planning:
- Commercial airliners take off about once per second worldwide, meaning there are approximately 87,000 flights per day and about 30 million flights per year.
- If you were trying to calculate what flight would be the most profitable say from San Francisco to Boston, arriving the same day, you would have close to 30,000 possible flight combinations from which to choose.
The only way to solve these problems is with fast, sophisticated algorithms. Business Intelligence tools that shovel data in your face are simply not an option. What’s called for is optimization – meaning automatically sift through all the possible alternates and give me the best recommendation based on the business objective (cost, profit, service level, etc.). BI simply cannot do that, because it is not designed to! For an airline, offering the best schedule based on passenger flows, competition and economics, and being able to respond to disruption is critical to survival.
Even for something that may seem as simple as seat availability, using a BI tool would have airlines bankrupt in less than a week. Seat availability is much more complicated than just the question of whether the number of reserved seats equals the capacity of an aircraft because airlines dynamically adjust prices according to demand and adjust their responses to seat availability queries as they estimate demand for flights. Think how many questions an airline must answer related to seat availability: If every passenger poses 100 searches before buying a ticket (a number in line with actual behavior) and each search looks at 1,000 flights, then the airlines would need to answer 15,000,000 questions per second. Neither their networks nor their computers can handle this.
Why is this important? Because data makes all the difference in the world in a time-critical environment. I know because this is the world I come from. In my previous life, I worked with a team at Delta Air Lines solving some of the most difficult problems in transportation. I come from the field of computational mathematics and Operations Research.
When storms hit, when pilots get sick, when connections are missed, when technical difficulties arise, you don’t need a reporting tool. You don’t need to query a database. You need algorithms that can automatically sift through the swarm of possible choices and recommend the best, and you need it immediately. The algorithm does all the analysis and the human only reviews it. That is the only way big data can be converted to intelligence in time. That is the reason algorithms exist. And that is the reason optimization models exist. Data needs to be analyzed in its totality. And now that big data is a reality, algorithms and optimization are absolute imperatives for business survival.
To me, data translates to questions such as “What are the problems we need to solve? What is the insight? How can the data provide guidance to make the best decisions?” Doesn’t everyone else think like that? If not, get ready because big data will force that train of thought.