Texas Winter Storm: IT Channel, Clients Face Rolling Blackouts, Internet Outages ‘Disrupting Everything’

‘It’s hard to prepare for it unless you are, say, a hospital with a generator. We help businesses prepare for hurricanes all the time. They have data replicated off-site, and they have evacuation plans to work from other data center. But that’s a regional disaster. We’re now in a state-wide disaster,’ says Chris Case, president of Sequel Data Systems.

The winter storm that has sent snowfalls rising and temperatures falling to record levels in Texas has wreaked havoc with business’ IT systems thanks to a string of unscheduled power blackouts, according to several Texas-based IT solution providers.

The unusually cold weather, which has blanked most of Texas with layers of snow and claimed the lives of more than a dozen people, caused power outages affecting more than 3 million homes and businesses earlier this week. As of early Wednesday morning, 2.7 million households were without power, according to The Electric Reliability Council of Texas, or ERCOT, which manages the flow of electric power to over 26 million Texas customers, or about 90 percent of the state’s electric load.

ERCOT started implementing rolling outages on Sunday due to the falling temperatures.

[Related: Silicon Valley: One Earthquake Away From IT Disaster?]

Businesses throughout most of Texas have experienced rolling blackouts where the power goes off and on for unexpected times, making it difficult to handle normal business issues, said Chris Case, president of Sequel Data Systems and Shadow IT Services, an Austin, Texas-based solution provider.

Businesses are also suffering from downed power lines and water cutoffs as well as lost internet services, forcing users to rely on their cell phone data plans to get work done, Case told CRN.

“This is a unique situation,” he said. “There’s only so much I can do. We’ve prepped customers for disaster recovery, and then AT&T and Verizon go down. It disrupts everything.”

The Texas winter storm has gone well past the extreme of what a business can expect, Case said.

“The storm is disrupting everything,” he said. “It’s hard to prepare for it unless you are, say, a hospital with a generator. We help businesses prepare for hurricanes all the time. They have data replicated off-site, and they have evacuation plans to work from other data centers. But that’s a regional disaster. We’re now in a state-wide disaster.”

Given the uncertain nature of the power grid, Case said Sequel Data is helping many of its clients shut down their offices “gracefully” when they can.

“We have people telling us their power is up and down,” he said. “Maybe their UPSs give 30 minutes of uptime. We suggest they just gracefully shut down and leave everything off so they don’t have to worry about hard shutdowns.”

For customers and channel partners, trying to navigate the storm is like playing a big game of Whac-A-Mole, said Michael Strong, co-owner and chief operating officer at Blue Layer IT, a Lubbock, Texas-based solution provider and managed service provider.

“Who’s in the office,” Strong told CRN. “Who’s not? Who’s working remote? Who’s not? There’s more red on our operating panel than ever. Green is good. When red is more than green, we’re dealing with a lot of issues.”

Businesses with more than one internet connection continue to operate as long as they have power, but some businesses have been without power for a couple days, Strong said.

While Blue Layer IT has a full-sized generator to run its two buildings, others are not so lucky, Strong said.

“I look out my window, and I’m surrounded by clients,” he said. “They don’t have generators. Their power may turn on for an hour or two in the morning, and then turn off. We’ve done proactive shutdowns from some clients who aren’t running essential businesses so that if they are struck by a potential blackout, they don’t have servers powering themselves off after the backup batteries die. It’s a big balancing act.”

In addition to having its own generator, Blue Layer IT also has a lot of its equipment running in a top-tier data center in Dallas with 24-hour monitoring and auditing, Strong said.

“We have multiple power and internet feeds there,” he said. “They told us their generators are on and their fuel tanks were recently topped off. So we haven’t had any power or internet disruption. It’s important to pick a good data center.”

Faisal Bhutto, president of cloud and cybersecurity at Computex Technology Solutions, a Houston-based solution provider acquired in 2019 by Pensare Acquisition, told CRN via text message that business is now difficult in Texas because of the winter storm.

“It’s been a mess here in Texas especially with electricity outages that are now going on for third day straight,” Bhutto wrote. “Fortunately from a business standpoint, we built a robust facility with natural gas generators that power data centers as well as our entire facility so we are fully operational and all the services we host for our clients are up.”

Work from home has been severely challenged due to the lack of power and internet at employees’ or customer‘s homes, Bhutto wrote.

“So work volume is low, and there is a clear impact on customers’ businesses,” he said. “Several customers have experienced outages throughout this weather due to power being out in their branches. Our staff has been able to come to our solution center where we have our NOC (network operations center), SOC (security operations center), and data center. Our secondary operation center in Minneapolis is also live.”

Other IT services providers have fared better.

Howie Evans, vice president of Dallas Digital Services, a Colleyville, Texas-based solution provider, said that he and his business have not lost power. Any employees that lost power found a friend or relative with power and extra space, Evans told CRN. He even drove to the office Tuesday.

With temperature lows expected to reach the forties by next Tuesday, Evans is hopeful that the storms will pass quickly. Clients of DDS, which has 14 employees, may want to talk shop about backup services. “This will be on everyone’s mind, absolutely,” he said.

(Second photo showing a snow-covered San Antonio courtesy Rachel Toppel)