Texas Ransomware Responders Urge Remote Access Lockdown
Lessons Learned From Crypto-Locking Malware Attack That Hit 22 Municipalities
Mathew J. Schwartz (euroinfosec) • September 6, 2019
Credit Eligible
Texas Ransomware Responders Urge Remote Access Lockdown
Photo: M&R Glasgow, via Flickr/CC
Three weeks after a ransomware attack slammed 22 Texas municipalities' systems, leaving them crypto-locked, state officials say more than half of the cities have returned to normal operations. The remainder, they say, are continuing to advance their system restoration (see: Texas Says 22 Local Government Agencies Hit by Ransomware).
See Also: Webinar | The Future of Adaptive Authentication in Financial Services
Based on the experience, the state's information security leaders have published a series of recommendations - including how to lock down endpoints and infrastructure being remotely managed, for example, by managed service providers.
Although state officials have declined to go into details, based on their lessons learned, it appears that some or all of the affected municipalities' systems may have been managed by a single MSP.
Incident Response Concludes
Texas state officials say their incident response effort was launched just hours after the 22 municipalities got hit by a ransomware attack on Aug. 16.
By Aug. 23, just one week after the attack, all of the victims "had transitioned from assessment and response to remediation and recovery with business-critical services restored," the Texas Department of Information Resources says.
While a federal investigation into the ransomware attack is ongoing, Texas officials characterize the joint federal and state response to this "statewide, multi-jurisdictional cybersecurity event" as being "a tremendous success."
Amanda Crawford, executive director of the Texas Department of Information Resources
Amanda Crawford, executive director of the state's Department of Information Resources, says this success was due in large part to having a response plan already in place.
"I am proud of the work of Department of Information Resources' information security team and grateful for the partnership with the many state and federal agencies who joined in our response to this incident," she says. "I also want to recognize the impacted entities for working with our responders to get this resolved quickly while still protecting the integrity of the federal investigation. It was this team effort along with advanced preparation that allowed a very critical situation to be resolved quickly and with minimal impact for Texans."
State officials have declined to specify which strain of ransomware infected systems or how the systems were infected.
But Gary Heinrich, the mayor of one of the affected municipalities - Keene, Texas, with a population of 6,100 - last month told NPR that the attacker demanded a total ransom worth $2.5 million to restore all crypto-locked systems across the 22 municipalities.
Heinrich also said the city outsources its IT operations. "They got into our software provider, the guys who run our IT systems," Heinrich told NPR. "A lot of folks in Texas use providers to do tha, because we don't have a staff big enough to have IT in house."
State officials have declined to confirm if a single managed service provider served all 22 municipalities that got hit by ransomware. But they did say that to the best of their knowledge, no ransom got paid to the attacker, which they described as being apparently "one single threat actor."
5 Lessons Learned for MSP Users
Nancy Rainosek, the CISO of Texas, has offered five lessons learned from the attack that she says are applicable to any organization that uses remotely administered IT services or managed security providers.
Here are her verbatim recommendations:
"Only allow authentication to remote access software from inside the provider's network;
Use two-factor authentication on remote administration tools and virtual private network tunnels - VPNs - rather than remote desktop protocols;
Block inbound network traffic from Tor exit nodes;
Block outbound network traffic to Pastebin;
Use endpoint detection and response to detect Powershell running unusual processes."
Based on Rainosek's recommendations, it appears that the Texas municipality ransomware attacker was able to remotely access - via the anonymizing Tor network - poorly secured remote-administration tools installed by an MSP on the endpoints of up to 22 Texas municipalities, using PowerShell to move laterally in targeted networks and exfiltrating stolen data or perhaps a list of infected systems to the text-sharing site Pastebin.
Sodinokubi Targets MSPs
Managed security providers are an obvious target for ransomware attackers, because one MSP will manage IT for numerous organizations. If hackers can break into the MSP's infrastructure, they can potentially infect numerous victim organizations with little additional effort. For example, if an MSP has 100 clients, and each client as 100 endpoints, that's potentially 10,000 endpoints that a single attacker might be able to infect.
Bill Siegel, CEO, Coveware
Bill Siegel, CEO of Coveware, a Connecticut-based ransomware incident response firm, says ransomware-as-a-service affiliates will target remote management tools - also known as remote-administration tools - used by MSPs or IT service providers, which get installed on every endpoint they manage.
"It's been devastating, because when they do get into an MSP, they hit hundreds of companies, sometimes simultaneously, [generating] very high return on the attack, rather than just hitting the MSP, which is also a small business," he tells Information Security Media Group. "They're hitting hundreds of small organizations at a time." (See Do Ransomware Attackers Single Out Cyber Insurance Holders?)
MSPs use remote management tools to scale. But some tools have vulnerabilities - several widely used RMM tools have received critical security updates in recent months - and some RMM tools get deployed in a poorly secured manner, for example, without two-factor authentication, Siegel says. As a result, hackers can sometimes remotely push malicious software directly to an RMM tool for immediate installation on an endpoint.
Common industries targeted with ransomware in Q2 2019 (Source: Coveware)
Siegel says his firm wasn't involved in the Texas municipality investigation. But he says attackers using the Sodinokibi ransomware-as-a-service offering continue to target MSPs, and that it's likely that ransomware strain was used against the Texas municipalities (see Ransomware: As GandCrab Retires, Sodinokibi Rises).
WHAT THEY NEED IS STRIKEFORCE'S OOBA AND GUARDEDID
Lessons Learned From Crypto-Locking Malware Attack That Hit 22 Municipalities
Mathew J. Schwartz (euroinfosec) • September 6, 2019
Credit Eligible
Texas Ransomware Responders Urge Remote Access Lockdown
Photo: M&R Glasgow, via Flickr/CC
Three weeks after a ransomware attack slammed 22 Texas municipalities' systems, leaving them crypto-locked, state officials say more than half of the cities have returned to normal operations. The remainder, they say, are continuing to advance their system restoration (see: Texas Says 22 Local Government Agencies Hit by Ransomware).
See Also: Webinar | The Future of Adaptive Authentication in Financial Services
Based on the experience, the state's information security leaders have published a series of recommendations - including how to lock down endpoints and infrastructure being remotely managed, for example, by managed service providers.
Although state officials have declined to go into details, based on their lessons learned, it appears that some or all of the affected municipalities' systems may have been managed by a single MSP.
Incident Response Concludes
Texas state officials say their incident response effort was launched just hours after the 22 municipalities got hit by a ransomware attack on Aug. 16.
By Aug. 23, just one week after the attack, all of the victims "had transitioned from assessment and response to remediation and recovery with business-critical services restored," the Texas Department of Information Resources says.
While a federal investigation into the ransomware attack is ongoing, Texas officials characterize the joint federal and state response to this "statewide, multi-jurisdictional cybersecurity event" as being "a tremendous success."
Amanda Crawford, executive director of the Texas Department of Information Resources
Amanda Crawford, executive director of the state's Department of Information Resources, says this success was due in large part to having a response plan already in place.
"I am proud of the work of Department of Information Resources' information security team and grateful for the partnership with the many state and federal agencies who joined in our response to this incident," she says. "I also want to recognize the impacted entities for working with our responders to get this resolved quickly while still protecting the integrity of the federal investigation. It was this team effort along with advanced preparation that allowed a very critical situation to be resolved quickly and with minimal impact for Texans."
State officials have declined to specify which strain of ransomware infected systems or how the systems were infected.
But Gary Heinrich, the mayor of one of the affected municipalities - Keene, Texas, with a population of 6,100 - last month told NPR that the attacker demanded a total ransom worth $2.5 million to restore all crypto-locked systems across the 22 municipalities.
Heinrich also said the city outsources its IT operations. "They got into our software provider, the guys who run our IT systems," Heinrich told NPR. "A lot of folks in Texas use providers to do tha, because we don't have a staff big enough to have IT in house."
State officials have declined to confirm if a single managed service provider served all 22 municipalities that got hit by ransomware. But they did say that to the best of their knowledge, no ransom got paid to the attacker, which they described as being apparently "one single threat actor."
5 Lessons Learned for MSP Users
Nancy Rainosek, the CISO of Texas, has offered five lessons learned from the attack that she says are applicable to any organization that uses remotely administered IT services or managed security providers.
Here are her verbatim recommendations:
"Only allow authentication to remote access software from inside the provider's network;
Use two-factor authentication on remote administration tools and virtual private network tunnels - VPNs - rather than remote desktop protocols;
Block inbound network traffic from Tor exit nodes;
Block outbound network traffic to Pastebin;
Use endpoint detection and response to detect Powershell running unusual processes."
Based on Rainosek's recommendations, it appears that the Texas municipality ransomware attacker was able to remotely access - via the anonymizing Tor network - poorly secured remote-administration tools installed by an MSP on the endpoints of up to 22 Texas municipalities, using PowerShell to move laterally in targeted networks and exfiltrating stolen data or perhaps a list of infected systems to the text-sharing site Pastebin.
Sodinokubi Targets MSPs
Managed security providers are an obvious target for ransomware attackers, because one MSP will manage IT for numerous organizations. If hackers can break into the MSP's infrastructure, they can potentially infect numerous victim organizations with little additional effort. For example, if an MSP has 100 clients, and each client as 100 endpoints, that's potentially 10,000 endpoints that a single attacker might be able to infect.
Bill Siegel, CEO, Coveware
Bill Siegel, CEO of Coveware, a Connecticut-based ransomware incident response firm, says ransomware-as-a-service affiliates will target remote management tools - also known as remote-administration tools - used by MSPs or IT service providers, which get installed on every endpoint they manage.
"It's been devastating, because when they do get into an MSP, they hit hundreds of companies, sometimes simultaneously, [generating] very high return on the attack, rather than just hitting the MSP, which is also a small business," he tells Information Security Media Group. "They're hitting hundreds of small organizations at a time." (See Do Ransomware Attackers Single Out Cyber Insurance Holders?)
MSPs use remote management tools to scale. But some tools have vulnerabilities - several widely used RMM tools have received critical security updates in recent months - and some RMM tools get deployed in a poorly secured manner, for example, without two-factor authentication, Siegel says. As a result, hackers can sometimes remotely push malicious software directly to an RMM tool for immediate installation on an endpoint.
Common industries targeted with ransomware in Q2 2019 (Source: Coveware)
Siegel says his firm wasn't involved in the Texas municipality investigation. But he says attackers using the Sodinokibi ransomware-as-a-service offering continue to target MSPs, and that it's likely that ransomware strain was used against the Texas municipalities (see Ransomware: As GandCrab Retires, Sodinokibi Rises).
WHAT THEY NEED IS STRIKEFORCE'S OOBA AND GUARDEDID
4
