ConnectWise Control 'Attack Chain' Exploit: 20 Questions For Security Researcher Bishop Fox

"One of our security researchers found these vulnerabilities. He deemed them severe enough to rate them as critical and the attack chain so bad that we had to report them."

Security consultancy Bishop Fox has discovered eight vulnerabilities in ConnectWise Control.

“One of our security researchers found these vulnerabilities. He deemed them severe enough to rate them as critical, and the attack chain so bad that we had to report them,” said Bishop Fox Associate Vice President of Consulting Daniel Wood.

ConnectWise was tied to numerous security breaches in 2019, starting with reports in April that the company’s Control product was used in the now-infamous Wipro hack. Then an MSP using ConnectWise Control had its network hacked in August, resulting in the compromise of 22 municipal web sites in cities and towns across Texas.

ConnectWise told CRN Tuesday that while it “struggled” to reproduce the flaws, it has patched 75-percent of them.

Here, Wood answers 20 questions about how Bishop Fox discovered the vulnerabilities and the response it received from ConnectWise.

Why did you look at ConnectWise Control?

It's actually not the most interesting part of the story, but essentially our security researcher, Matt Hamilton [who has since left the company], he was looking into using the product himself for some personal purposes, for his family, for remote IT troubleshooting, so he can help his family out when they needed it, similar to LogMeIn.
So, you know, being a security researcher the first thing you do when you think about remote management tools and software is, ‘How secure is it?’ So he just started to take a look at it, to make sure that before he started using it personally, that it wasn't super vulnerable or had a lot of issues with it. So he just started conducting the research on his own to make sure, again, that it was secure before using. So that's kind of why we started looking at.
What was it you found?

The kind of things that we found are commonplace to what you would see with a web application or a web service, whether it’s cross-site scripting or cross-site request forgery (CSRF}. A lot of web applications and services have those vulnerabilities.
The run of the mill kind of vulnerabilities that you would expect on any application or service that doesn’t have security taken into serious consideration during the development life cycle of it is basically what was discovered.

This is more of a general statement, but when companies take on other technologies or acquire other companies for their technology and use a lot of third-party dependencies, a lot of the time they'll get a lot of tech debt with it. That tech debt will result in vulnerabilities. So if this was more of a legacy application that they just gave a facelift to then, they probably inherited the vulnerabilities.

Specifically, what were the flaws that you discovered?

The flaws in general, they’re serious in their own right, but when you start taking a look at chaining the vulnerabilities together -- we call them basically attack chaining. There are pretty much two, big sub-optimal outcomes that can occur here: you have code execution on the actual Control server by leveraging the cross-site scripting and the cross-site request forgery vulnerabilities, which then allows you to execute arbitrary code execution on that Control server. That’s for the SaaS service, when you are talking about cloud. Then you are looking at what could an attacker access after having, basically, arbitrary code execution on that Control server. In this case, it could be access to things like [Amazon Web Services] S3 buckets or EC2 instances, or anything else that the Control server may give you access to.

The other avenue of attack could be on the actual client desktop, where the application is installed, where if an attacker were to create a JavaScript payload that exploits the lack of CSRF protection, they can then start to execute code within the context of that web browser, and at that point as soon as the malicious JavaScript starts to execute within the web browser it starts to exploit that cross-site scripting vulnerability.

From that point, if you don’t have cross-site forgery protection, or if CORS is misconfigured on the server, basically it gives you a connection to the client desktop that the software is installed upon. So if you were to install it on your desktop and somehow you were able to go to a website that had malicious JavaScript, like a drive-by download, at this point if it was known that you had this software installed on your end point, it would execute in the context of your browser and could exploit your personal computer if it was installed on it.

You wouldn’t need to have initial access on the endpoint itself as long as someone who has the Control server installed were to browse a malicious website, the JavaScript could execute in that browser and start sending all those requests to the Control server API on basically, your client. At that point its able to pull out the information it needs from the API, then would provide you access to the endpoint itself.

What was ConnectWise's response?

When we first reported the vulnerability [to ConnectWise] we had radio silence. We gave it a day or two and then at the same time I requested our team get CVE [Common Vulnerabilities and Exposures] numbers assigned so we could officially track the vulnerabilities from MITRE and at that point decided to reach out to their help support, or help desk, and we provided them the disclosure of the vulnerabilities as well as the CVE numbers and basically offered to help them understand the vulnerabilities that we had reported. At that time, that’s when they replied back and wanted to have a conversation with us.

Was this part of a sales pitch by Bishop Fox to try to get ConnectWise to use your services?

No. Absolutely not.

So you weren’t saying ‘For $50,000 or $100,000 we can make this all go away?’

We did not have that conversation at all. It was literally: One of our security researchers found these vulnerabilities. He deemed them severe enough to rate them as critical and the attack chain so bad that we had to report them.

When [Hamilton] was doing his research, he was searching more about the company to better understand the impact, and that’s when he came upon [CRN's] previous reporting about the Texas Ransomware incident and how it could be connected. So that piqued our interest. There was an active ransomware incident ongoing. We found vulnerabilities that could be potentially involved in this sort of incident. That’s when we decided to kick our game up a notch and make sure we were working with ConnectWise, the vendor, so they could understand the vulnerabilities, but also I ended up making the decision to contact authorities as well, to provide any support that maybe needed there as well.

So you began a conversation with the FBI field office investigating the Texas ransomware outbreak. Did you talk to them and explain what you found?

We talked with them and walked them through the vulnerabilities. We had multiple conversations, I would say, maybe over a month or two.

Can you tell me about the conversations you had with ConnectWise?

So the initial conversation occurred with the team, [former ConnectWise CISO] John Ford, [Bishop Fox's] Matt Hamilton, and some other people on the vendor side. They were all part of the conversation. Then after they responded back to the ticket, I believe we had a phone conversation with them the next day. Then we had one or two follow-up meetings over the span of that week, essentially where we went back and forth helping them understand the vulnerabilities. They had questions. They were working to reproduce some of the vulnerabilities and confirming whether they existed or not. One of the vulnerabilities they weren’t able to reproduce. We had our final point of contact with them at that time.

How did that end? Did you leave satisfied that they were making plans to fix the security vulnerabilities that you found?

I don’t know if we were satisfied with that conversation. The conversation turned a little contentious. A threat of defamation and libel did come up in that conversation. That immediately concerned us. I personally engaged John Ford (pictured) to try to get a meeting set up. We waffled back and forth. Between the end of September and the beginning of December I tried to get a meeting with [Ford] and we were missing each other. I reached out to him a couple more times after that, but I ceased emailing him after three or four times without a response. [Note: Ford left ConnectWise in December and took a new role at security vendor IronNet Cybersecurity in January.]

So what made you feel comfortable coming out and publicizing the Zero Day vulnerabilities after the threat of litigation?

There are two things. First and foremost, at Bishop Fox we absolutely stand behind the researchers we have and support them. As long as they follow our policies and procedures, and we do things by the book, then we’re always going to support them and stand up for them. If someone is threatening litigation, that’s only going to make us double down on protecting our researchers, consultants, and our company. The other thing is we’re not doing anything that’s unique in the industry. We actually follow a vulnerability disclosure policy that is industry standard, where we try to work with the vendors, and its very similar to the Google disclosure policy that a lot of different companies follow.

We believe in responsible disclosure. We’re not going to drop Zero Day’s all over the place. We are going to work with affected vendors, make sure things are fixed.

If it becomes egregious, the vulnerability’s impact and severity could override not disclosing a vulnerability. We want to make sure we are helping to protect the public that could be using this product that could be vulnerable to the issues that we outlined.

After that threat of litigation, did you go over the work again and just make sure that you were correct?

Yeah. Absolutely. So as soon as this occurred and that was relayed to me, I immediately rounded the wagons on our side and made sure that we were covered from a legal perspective and did 20 passes over the research that [Hamilton] had written up and discovered to make sure that everything we were saying was factual, and we truly understood the problem.

Is this something you do frequently?

It's becoming more frequent. [At] Bishop Fox, our mission is to become the best offensive security firm in the universe. That’s kind of our internal mantra. A lot of that revolves around the security research that we do. We have started to discover more and more vulnerabilities and we’re releasing them through our responsible disclosure policy more frequently.

Are these difficult vulnerabilities to fix?
No, not at all. I mean, they're kind of like the basic web service and web applications vulnerabilities that you typically see in web applications. There's definitely going to be involvement from ConnectWise to fix those.

I know it was brought up about enabling multifactor or two-factor authentication, that it would potentially address it. It absolutely would do nothing to mitigate any of the vulnerabilities because with cross-site scripting and cross-site forgery, you're going to be riding the session of the victim or the user, so at that point, they're already basically authenticated. So that's not going to mitigate any of the vulnerabilities.
There are a couple things that could. There are some kinds of "hacky things" that could be done, as well, like making sure that the Control server doesn't have Flash installed on the ConnectWise Control Web Server and some other things. It's not too complicated.

Is there any way a user or an MSP could patch this without ConnectWise’s help?

No. By its nature, they wouldn’t be able to patch the vulnerability itself, nor would they be able to mitigate it.

What is typically the response you get from companies when you find a vulnerability in their product?

Typically, the response is pretty positive. Companies don’t want to be in the news, and companies don’t want to expose their clients to vulnerabilities. Typically, we’ll see them want to understand the issues and then proactively work to address the vulnerabilities and put a patch out, within a relatively short time frame.

And then there are also companies like Amazon—Amazon Web Services— where they were actively seeking to understand the issue and then seeking almost a partnership to better understand the research that we do in the future to make sure that they're protecting their customers going forward.

So there's the run of the mill, ‘Thanks, but no thanks’ all the way up to active partnerships. This is one of the rare ones where a company will become almost adversarial in nature.

Has anyone looked to see if ConnectWise took this to heart and fixed what you found since you last talked?

If you had asked that question two months ago, the answer would have been yes. But after not seeing the vulnerability patching happening, we decided it wasn’t the best use of time to keep giving ConnectWise a free pen test over and over, if they weren’t addressing the issues.

Have you looked at other RMM tools to get a baseline, or to see if they are just as flawed, or less flawed?

Yes. That’s basically as far as I’ll expand on that. Yes, we have.

Are they all dangerous?

It's hard to generalize. If done incorrectly, not just the programming of the architecture of the application, but also the configuration. If that's done incorrectly then yes, you can expose yourself to a whole bunch of different security concerns with remote access.

You’re essentially opening the attack surface of whatever endpoint you install the software on because it's literally saying ‘Hey, I want to offer remote administration to this endpoint,’ and so now you're trusting the authorization model of the software, as well as just how secure it's been designed or configured to prevent anyone else who does not have authorization to connect to it. You’re taking a risk by using RMMs because you’re basically knowingly increasing that attack surface, hoping that the software actually has done this due diligence.

Are the other RMM tools that you guys have looked at equally flawed?
I can’t make an assertion in that way because I haven't looked at all the different RMM tools that we have assessed to kind of do apples-to-apples. I could make a statement that, for a commercial tool, the vulnerabilities that exist in it, you would expect them not to exist in a commercial tool of this caliber.
Is that a reference to the ConnectWise product or all RMM products?

The ConnectWise product. A lot of the time these developers are not security developers or security-conscious developers or even have security professionals in their organization. So they open up that attack surface and they don't really understand the risks of doing so. So, going back to RMMs, usually what you'll find are these types of vulnerabilities in them.