How to Onboard AWS GuardDuty Data into Splunk
Recently one of our customers asked us to onboard data AWS GuardDuty threat intelligence data into Splunk. Since the process was not trivial, we decided to publish this for everyone’s benefit.
What is AWS GuardDuty?
AWS GuardDuty is a managed threat detection service that continuously monitors for malicious or unauthorised behaviour to help you protect your AWS accounts and workloads.
AWS GuardDuty is a service provided by AWS that monitors activities such as unusual API calls OR the deployments which would be potentially unauthorized and avoid any account compromise. It has the intelligence to also detect compromised instances/services etc.
Each of Splunk’s serverless application lambdas are designed to enable you to deploy all the AWS infrastructure needed to start streaming a variety of AWS data sources into Splunk in a scalable, straightforward and automated manner. This blog provides high level information for the POC and the steps required by it:
Requirements:
Splunk Enterprise Version 6.5.x or later.
AWS account (where we will enable sample findings generate for GuardDuty)
HEC enabled on the instance
Splunk Add-on for GuardDuty (This is just used to parse and store the Knowledge Objects and not for creating any inputs)
Cloudwatch Service (For setting up Rule to collect the GuardDuty Findings)
Lambda Function ( for using splunk-logging: Log events from AWS Lambda itself to Splunk’s HTTP event collector)
Steps to On-board Logs:
In order to get the logs from GuardDuty service from AWS, we have to use a serverless approach. To break it down further, let’s look at one of Splunk’s serverless applications provided on Serverless Application Repository – in particular: splunk-logging. This method in brief leverages Splunk’s HEC capability to send data via an AWS Lambda. There are alternatives to this blueprint such as splunk-kinesis-stream-processor which also streams the data to Splunk’s HEC endpoint.
In this article, method used in particular to create Lambda is splunk-logging blueprint, where we will integrate our HEC server “url” and token associated with required index. To understand how to create HEC token and get a url from Splunk refer this.
Ok, Let’s Begin…
Let’s start with step one, which is deploying the serverless app. A serverless application from Serverless Application Repository is a CloudFormation template that uses the Serverless Application Model (SAM) to make deployment and readability easy. The user can use this Serverless Application Repository with a Lambda function from the available template options.
1. INSTALL the add-on. Install the Splunk Add-On for GuardDuty on the server that you intent to receive data on. Please note that other than installing the Add-on and creating a HEC Token to be used in the steps below, no other setup is needed from Splunk Side.
2. Login to AWS management console and launch the EC2 instance where the user will need to install Splunk and enable HEC inputs to get the data. (This is just for POC, in real time, we already have HEC server so we do not need to launch any EC2).
3. Create the function for Lambda which will have a Serverless Application Repository of Splunk-Logging.
4. Select the appropriate serverless repository that will be used to integrate the HEC url and token, here we are using splunk-logging.
5. After selecting splunk-logging, create the lambda function. The user will be able to see a form that asks for the Splunk HEC token and the Splunk Collector URL. Please fill in the token generated and the server’s FQDN for the serverless application to reach the ingestion endpoint.
6. Once click on “Deploy”, the function should look as follows.
7. Once the function is created, note down the Amazon Resource Name (ARN) (from the top right corner) visible from the the Lambda function. A lambda function is also a resource in use and the ARN here will be used later to get the data in.
8. Scroll down to the function code and edit the “index” and “sourcetype”. Add the “index” name that was created to which the user wants to send the data into. Add the “sourcetype” as “aws:cloudwatch:guardduty”. This is because we want the Splunk Add-on for GuardDuty to parse the data coming in and the sourcetype “aws:cloudwatch:guardduty” is the sourcetype defined in the add-on.
9. To capture the findings from GuardDuty, create a CloudWatch Event Rule from the CloudWatch console (CloudWatch console > “Events” > “Rules” > “Create Rule”). The rule creation will ask to set the Target. Set the splunk-logging function that you have created in the above steps.
10. Verify while linking the rule that your splunk-logging function is same as the ARN noted down earlier.
11. Let’s try the setup! Try generating some sample findings from the settings page in the AWS console for GuardDuty. After these sample findings are received by Splunk, the user should see events flowing to our Search head. Before that, let’s take a quick understanding that how to do this. Open the GuardDuty console from the AWS Services menu. Click on “Get Started”
12. For a personal account there won’t be any notable events, so click on “Generate sample findings” under the “General Settings” to proceed. This will generate demo findings to test out the setup and see the data in Splunk:
13. Once the sample findings are generated per the above screenshot, verify in the Splunk platform by running a search against the intended index and see if you can find the events there.
14. There are some dashboards with default available in the Splunk add-on for GuardDuty which can also help verify the availability of the data into Splunk.
For any more information or guide on AWS GuardDuty contact our Splunk Cloud Experts! or visit us at https://crestdata.ai/solutions/splunk
Author
RISHABH GUPTA
Rishabh is currently working as Splunk and Security Professional Services Consultant at Crest Data. Rishabh has been consulting with Fortune 500 and global enterprise customers and has been an active member of Cyber Security and Infrastructure Reliability communities for 6+ years. Rishabh is an accredited consultant for Splunk Core, Splunk Enterprise Security, Splunk UBA for SIEM and Phantom for SOAR based applications. He has been a frequent speaker at DefCon and other Cyber Security Conferences.