What can you do during an airline computer meltdown?

Catastrophic computer outages that paralyze an entire airline are few and far between. Except this summer.

Last month, Southwest Airlines canceled 2,300 flights after a router in one of its data centers failed, delaying hundreds of thousands of passengers. And last week, Delta Air Lines suffered a massive computer outage, which triggered the cancellation of 451 flights in a single morning.

A rare look behind the curtain at Southwest’s meltdown offers several important customer-service lessons for passengers who experience similar delays in the future. And in an industry that depends on finicky information systems, these incidents are bound to repeat themselves. They’ve left customers wondering how to avoid getting stuck in another IT collapse, and what, if anything, an airline can do to make up for such an event.

Jack Russell, who was scheduled to fly from St. Louis to Las Vegas last month, had a front-row seat for Southwest’s IT issues, which an employee euphemistically blamed on a “software problem.” The airline’s proposed fix: Fly him to Vegas four days later.

As the executive vice president of a software company in St. Louis, Russell knows a thing or two about computers that go on the blink. But he’s less understanding about Southwest’s IT implosion, which he says left him with little choice but to pay an extra $1,800 to reach his destination.

“I spent twice as much money as I thought I would to get to Las Vegas,” Russell says. “If my customers had an outage created by my company and I said, ‘Sorry, it was a freak occurrence,’ they would be waiting at my doorstep with their lawyer.”

The Southwest systems problem suggests how fragile even the best-run airlines can be. It started in the early afternoon of July 20, when one of its small Cisco routers, out of about 2,000 such pieces of hardware that direct the airline’s network traffic, failed.

This router broke in an unusual way. Instead of registering the error, which would have allowed network administrators in Southwest’s Dallas data center to take it offline immediately and replace it with a working router, it behaved as if it was still operating normally. Only, it wasn’t directing any traffic.

Although network administrators spotted the error within half an hour, enough traffic had backed up that critical systems needed to be rebooted — a process that took a full 12 hours and affected critical functions, including the airline’s website, its smartphone app and several internal systems used by Southwest employees to handle reservations. It was as if someone had turned off the lights for half a day.

When the systems flickered back to life, the problems continued. The airline still didn’t have enough information to restart all flights. Because its systems had been down for so long, it couldn’t be sure whether some of its crews had taken enough rest, as required by the Federal Aviation Administration. That forced Southwest to cancel more flights on July 21 and 22.

Brandwatch, a social-tracking service, charted a corresponding tsunami of anger on Southwest’s social media channels. The airline drew 36,905 mentions in a single day on July 21, an almost 20-fold increase from normal levels.

“The spike in incoming volume that this received was incredible,” says Joshua March, the chief executive of Conversocial, which offers customer-service software to travel companies. “But the really significant piece in this instance was the inability to effectively scale the response.”

Southwest had no script for handling an event of this magnitude.

“It was really rough,” says Robert Jordan, the airline’s executive vice president and chief commercial officer, who describes the IT catastrophe as a “thousand-year flood.” The airline sent 50-percent-off vouchers to passengers affected by the outage, and in some cases paid for them to fly to their destinations on other airlines. All told, he says Southwest spent “tens of millions of dollars” trying to make amends.

“We know we messed up,” he adds. “We know we have to work really hard to regain our passengers’ trust.”

Southwest is still cleaning up. Russell’s delayed flight to Las Vegas is among the thousands of cases still being processed. Under most circumstances, a full refund for a replacement flight would be a tall order, but these are not normal circumstances.

IT disasters of this scale are unusual. Back in 2012, United Airlines experienced several days of delayed flights and sluggish customer service as it struggled to integrate the IT systems of United and Continental Airlines. Last July, United also suffered an outage that made it cancel hundreds of flights after a network router stopped working.

Asked if passengers could have done anything to get to their destinations faster during such a systems collapse, Jordan paused. So many things went wrong during the event that the normal tricks didn’t work. You couldn’t fall back on calling the airline because even the call-center employees didn’t have access to their IT systems.

“There just isn’t a good answer,” he says.

That’s the consensus of the customer service experts, too.

Elaine Allison, a former flight attendant and on-board service manager who now offers training courses in customer service, says passengers are powerless to negotiate their way around a total systems failure. She happened to be in Las Vegas during the week of Southwest’s outage, but was lucky enough to be flying on another airline.

“Pack at least one day of clothing and small amenities, plus all medications, in a carry-on, in the event luggage is checked and immediately not retrievable,” she says. Russell handled the situation correctly by re-booking his flight on another airline, she says. Southwest must refund a ticket when it cancels a flight.

The trick, says customer service expert Teri Yanovitch, is to look forward and not back. Southwest needs to figure out how to say it’s sorry without losing its shirt, and customers need a game plan should they get caught in a future systems failure.

“Southwest needs to explain the situation and how Southwest will prevent it from happening again,” she says. “As a customer, the best you can do when all critical IT systems are down is to keep calm, don’t take it out on the employee — it is not their fault — and consider your options for alternate transportation based on the situation.”

Research suggests that Southwest can make a full recovery, Yanovitch says. When a recovery is handled correctly, 96 percent of the customers will return. And when it’s not? In 2012, when United Airlines suffered its first meltdown, it was the world’s largest airline. Today, it’s No. 3.

Underwritten by

Elliott Advocacy is underwritten by AirportParkingReservations.com

With nearly 20 years in the industry, over 128 airport covered in the U.S. and Canada, and over 1,000 Hotel and Parking Partners-we provide travelers the best options on how to get to the airport when flying. Whether you want to; drive yourself and park near an Airport (Airport Parking), stay the night before your flight at an airport hotel and leave your car (Hotel and Parking Package), or take a shared Shuttle/Private Car/Limo to the Airport- we got you covered. The best deals can be found online, and booking a reservation has never been easier. You can explore all of our options by visiting us at AirportParkingReservations.com, ParkSleepFly.com, AirportParking.com, and ShuttleFinder.com.

Elliott Advocacy is underwritten by Cavalry Travel Insurance

Cavalry takes the worry of out travel by providing 24/7 access to medical and security professionals combined with the best medical evacuation and security extraction services. Cavalry gets you home safely when you need it most. Learn more at Cavalrytravelinsurance.com.

Elliott Advocacy is underwritten by Seven Corners

Seven Corners has helped customers all over the world with travel difficulties, big and small. As one of the few remaining privately owned travel insurance companies, Seven Corners provides insurance plans and 24/7 travel assistance services to more than a million people each year. Because we’re privately held, we can focus on the customer without the constraints that larger companies have. Visit Seven Corners to learn more.

Elliott Advocacy is underwritten by Sodexo North America

Sodexo North America Sodexo North America is part of a global, Fortune 500 company with a presence in 80 countries. Sodexo is a leading provider of integrated food, facilities management and other services that enhance organizational performance, contribute to local communities and improve quality of life for millions of customers in corporate, education, healthcare, senior living, sports and leisure, government and other environments daily. Learn more at Sodexoinsights.com.

Elliott Advocacy is underwritten by Travelex Insurance Services

Travelex Insurance Services is a leading travel insurance provider in the United States with over 55 years combined industry expertise of helping people dream, explore and travel with confidence. We offer comprehensive travel insurance plans with optional upgrades allowing travelers to customize the plans to fit their needs. Compare plans, get a quote and buy online at Travelexinsurance.com.

Elliott Advocacy is underwritten by AirHelp

AirHelp is the world’s leading flight delays compensation company, helping passengers apply for compensation following a delayed or canceled flight or when boarding has been denied. It is AirHelp’s mission to fight for passenger rights by holding airlines accountable for flights disruptions that are out of passengers’ control. AirHelp has already helped 5 million people, taking the stress out of applying for compensation and making it as hassle-free as possible for travelers around the world.

Elliott Advocacy is underwritten by Allianz Travel Insurance

The Allianz Travel Insurance company has built its reputation on partnering with agents all around the world to provide comprehensive travel insurance for their clients. Contact Allianz Travel Insurance for a comprehensive list of coverage.

Elliott Advocacy is underwritten by Chubb

Chubb is the world’s largest publicly traded property and casualty insurance company, and recognized as the premier provider of insurance for successful individuals and families in the U.S. and selected international markets, offering coverage for high-value automobile, homeowners, recreational marine/aviation, valuables and umbrella liability coverage. As an underwriting company, Chubb assesses, assumes and manages risk with insight and discipline, and combines the precision of craftsmanship with decades of experience to conceive, craft and deliver the best insurance coverage and services to individuals, families and business of all size.

Elliott Advocacy is underwritten by Fareportal

Fareportal’s portfolio of brands, which include  CheapOair and  OneTravel, are dedicated to helping customers enjoy their trip. Whether you want to call, click, or use one of our travel apps, one thing is clear: We make it easy to take it easy.

Elliott Advocacy is underwritten by Insuremyrentalcar.com

An independent provider of low cost CDW/LDW insurance for use with rental cars. Up to $100,000 cover with no deductible. Policies available on a per day, per trip or per year basis. Also works with overseas rentals. Try  Insuremyrentalcar.comnow.

Elliott Advocacy is underwritten by G1G.com

Since its inception over three decades ago, G1G has continued to revolutionize the travel insurance industry by being the only aggregator to operate a customer portal, placing all of the user's primary needs in one place. We have continued to innovate and disrupt the market by reimagining the way travel insurance can be delivered to the end user in ways no competition can. Simply put, no one knows the market as well as its founders and no one else shares G1G values and mission.

Elliott Advocacy is underwritten by Mediacom Communications

The nation’s fifth-largest cable operator, serving the smaller cities and towns in the Midwest and Southern regions of the United States. We are a high-performance broadband, entertainment, and communications company that brings the power of modern technology and quality customer experience to life inside the connected home by combining ultra-fast gigabit speeds with personalized local and over-the-top entertainment choices that fit your lifestyle. Details at  Mediacomcable.com.

Elliott Advocacy is underwritten by Arch RoamRight

Arch RoamRight is one of the fastest growing, most-highly rated travel insurance companies in the United States. Travel advisors love working with us, and travelers feel protected with our trip cancellation and travel medical insurance coverage. We also make it easy to file a claim online with our fast, paperless claims website. Learn more about RoamRight travel insurance.

Elliott Advocacy is underwritten by Virtuoso

The leading global network for luxury and experiential travel. This invitation-only organization comprises over 1,000 travel agency locations with 17,500 advisors in over 45 countries, and holds preferred relationships with 1,700 of the world’s finest travel companies. Virtuoso advisors collaborate with their clients to create personalized itineraries featuring exclusive perks, while also providing advice, access, advocacy, and accountability. For more information, visit  Virtuoso.com.

Elliott Advocacy is underwritten by VisitorsCoverage

As a company that is constantly striving to simplify travel insurance, VisitorsCoverage, is on a mission to help travelers make the better decisions about purchasing travel insurance, quickly. VisitorsCoverage has helped millions of travelers globally to buy the suitable travel insurance and explore the world with confidence. Get insurance for your next trip at  VisitorsCoverage. Lowest Price Guaranteed.

Elliott Advocacy is underwritten by MedjetAssist

Medjet is the premier global air-medical transport, travel security and crisis response membership program for travelers. With a MedjetAssist membership, if you become hospitalized more than 150 miles from home, we will get you from that unfamiliar hospital all the way home to the hospital you trust. All you ever pay is your membership fee. MedjetHorizon members add 24/7 personal security and crisis response benefits. Elliott.org readers enjoy discounted rates. Travel safer with  MedjetAssist.

Elliott Advocacy is underwritten by Southwest Airlines

The mission of Southwest Airlines is dedication to the highest quality of Customer Service delivered with a sense of warmth, friendliness, individual pride, and Company Spirit. We are committed to provide our Employees a stable work environment with equal opportunity for learning and personal growth.

Elliott Advocacy is underwritten by TravelInsurance.com

TravelInsurance.com makes it fast and easy to compare and buy travel insurance online from top rated providers. Our unbiased comparison engine allows travelers to read reviews, compare pricing and benefits and buy the right policy with a price guarantee, every time. Compare and buy travel insurance now at  TravelInsurance.com.

Elliott Advocacy is underwritten by Squaremouth

Squaremouth helps travelers easily and instantly compare travel insurance policies from all major providers. Only companies that meet the strict requirements of Squaremouth’s Zero Complaint Guarantee are available on the website. Compare policies on  Squaremouth.com to save over 70 percent on your next purchase.

Elliott Advocacy is underwritten by Travel Leaders Group

Travel Leaders Group is transforming travel through its progressive approach toward each unique travel experience. Travel Leaders Group assists millions of travelers through its leisure, business and network travel operations under a variety of diversified divisions and brands including All Aboard Travel, Andrew Harper Travel, Colletts Travel, Corporate Travel Services, CruCon Cruise Outlet, Cruise Specialists, Nexion, Protravel International, SinglesCruise.com, Travel Leaders Corporate, Travel Leaders Network and Tzell Travel Group, and its merger with ALTOUR. With more than 7,000 agency locations and 52,000 travel advisors, Travel Leaders Group ranks as one of the industry’s largest retail travel agency companies.

Elliott Advocacy is underwritten by Travelers United

If you’ve been mistreated by the airlines, Travelers United is your voice in Washington. Join the #1 travel advocacy organization working with Congress to improve and protect travelers. Plus, get $400 of annual benefits you can use for travel for only $29/year. Add your voice to ours. Make travel better.  Join today.

Send this to a friend