Threat Hunting with Datadog

Keeping your systems secure requires constant vigilance. Malicious processes can sneak into your infrastructure, stealing data, corrupting files, or launching denial-of-service attacks. There are many tools out there to do this process but I chose to utilize Datadog's Windows Agent and platform.

This blog post will guide you through the process of finding malicious processes using Datadog. We'll cover:

• Identifying suspicious activity: How I identified suspicious activity on my network.
• Drilling down with process context: Once you've identified a potential threat, Datadog allows you to examine the process in detail, including its resource consumption, network connections, and associated files.
• Further Investigation: We'll explore how to use Sysinternals's TCPView to investigation the process network traffic.
• Taking action: Finally, we'll talk about how to remove the process.

Identifying Suspicious Activity

The first step is to identify processes that exhibit suspicious behavior is to look for properties that exhibit malicous behavior. How I detected this malious process:

To monitor traffic flows and data tranversing my network I use several tools but the main tool is NTop. With NTop I can capture the traffic at the firewall to understand where systems are going and the amount of traffic they are producing. The main dashboard I use is live flows, I like to keep the dashboard up so that I can see what systems are doing the most talking. Over a week's time period I noticed that one of the systems on the network kept sending data to several domains owned by Datacamp Limited.

NTop Flows keepingquick.eu:5001 (UDP)

If we do a lookup on the two above domains we can see they resolve to 149.36.49.77, 138.199.57.213, 149.36.49.90 and 138.199.57.214.

~$ nslookup keepingquick.eu 
Server: 127.0.0.53 
Address: 127.0.0.53#53

Non-authoritative answer: 
Name: keepingquick.eu 
Address: 149.36.49.77 
Name: keepingquick.eu 
Address: 138.199.57.213 
Name: keepingquick.eu 
Address: 149.36.49.90 
Name: keepingquick.eu 
Address: 138.199.57.214

A whois search on Arin shows they are owned by Datacamp Limited

This system was constantly sending data to those and several other domains on UDP port 5001 which prompted me to look closer at what was going on.

Drilling Down With Process Context

There are several tools that exist to dig into these processes, so I decided to install Datadog’s Windows agent. I collected the process and network data to make it easier to see what process on the computer was sending out the traffic to the domains in question.

Once the data was being sent to Datadog it made it easier to correlate the traffic to a process that was generating the traffic. I could utilize the NPM data to look up each domain, I started by looking for “keepingquick.eu” over the past 2 days to see if any traffic would come up.

Query: server_domain:keepingquick.eu

As you can see in the screenshot the client in question has reached out to that domain.

If we click on the entry we start to dig a little deeper into the process to see if we can identify which one is the culprit.

The screenshot shows that the client has been sending data to the server on UDP port 5001 which is the same flow that I see in NTop-ng. Under the Client PID column we can see the process ID that is generating this traffic

Now we click on the process id and view the related processes this will bring us to the process in question.

Below we see that process id 20752 is TaskbarSystem.exe, the process seems legitimate but why is it contacting those domains and what is it sending to them.

A quick look at traffic generated by this process shows that the process is contacting several domains including Walmart and AirBnB. If I had to guess this might be adware but what the process is doing is beyond the scope of this document.

Further Investigation

Now that I have tracked down the process I wanted to jump on the system to see if the traffic is showing up there.

To do this I downloaded the SysInternals tool TCPView to see the traffic coming from this process. I could see that it was reaching out to several IP addresses on TCP port 443. I won’t be able to see the UDP traffic since there is no state information with UDP traffic.

If I take a look at one of the IP addresses (76.223.42.213) I can see it belongs to Amazon. It looks like the infrastructure behind this application is hosted in AWS.

Looking at the properties of this program we can see that it runs out of the Users folder, something that with system programs never happens.

Before moving on to removing the process from the system I looked for the process on Joe’s Sandbox to see what it found out about this particular process.

Joe’s Sandbox marks this as suspicious but not malicous. If you want to see the full analysis of this process and what it does you can see it at this link.

Taking Action

Now that we have found the process and validated that it is not a system process we can move on to removing it from the system. The good thing about this software is it comes with its own uninstaller, I ran the uninstaller and confirmed that it did actually uninstall the software from the system. I watched NTop for the next few days but never saw the traffic from that system again.