Implement Cloud Compliance policies using OPA Policies and protect your cloud infrastructure. Integrate OPA policies in CICD pipeline and validate your code , OPA is open source and it is easy to implement and only requires basic programming skills.
Cloud compliance using OPA Policies
When we provision the infrastructure and deploy resources into cloud , it is important that we evaluate the IaaS code for compliance and security policies before we deploy them. Tools such as AWS config & Fugue only help after resources are deployed. Ideally we wanted to validate the code during the process of CICD pipeline in an automated way without having to go through the code reviews which can be error prone.
There are few tools out there in market such as Hashi Corp Sentinel which only work if you have Hashicort enterprise version of terraform or Cloud version , sentinel is effective choice but it is not available with opensource version of terraform.
Open Policy Agent ( OPA ) is an open source version which is easy to adapt and parses through terraform json plans and evaluates them for compliance violations. I have not used Cloudformation for this exercise and I am not sure if it is a good combination, ever since I started working with terraform, I really don’t want to go back to Cloudformation, it was so painful.
Fugue is another choice which recently open sourced their product regula which is built on OPA ,Regula offer similar functionality and abstracts low level coding. as of now there is not much activity in the project.
How does OPA work :
First create your own opa policies and keep them in a separate project or in the root of the terraform project ,next run terrraform plan on your project, convert the plan into JSON format and run opa command with two arguments, 1. opa file location , terraform json file location.
Pre requisites
Setup and run locally on windows 1. Knowledge of Terraform 2. Exposure to JSON files and OPA framework 3. AWS account. Setup and run in enterprise environment 1. Knowledge of Terraform 2. Knowledge of Gitlab CICD pipelines 3. Basic knowledge of Docker containers to create a custom container for OPA 4. Knowlege of Linux (Ubuntu) 4. Exposure to JSON files and OPA framework 5. AWS account.
Source code @ GIT
OPA workflow in enterprise environment using Gitlab CICD pipeline.
Setting up and running OPA on Windows development environment locally
- Download windows version of opa from Github page
- Unzip the file “opa_windows_amd64.exe” and rename it to “opa” , then add the folder containing the opa exe file to windows environment variable PATH
- Test if OPA is iconfigured properly by running “opa version” from command prompt
- Refer to the repo for a complete opa example which validates following compliance policies
- s3 buckets should always be created in a specific region
- bucket’s ACL property should always set to ‘private’ unless it is a website hosting bucket
- Sample code :
- S3 encryption should be enabled.
- Minimum required tags should always be set
- S3 logging should always be enabled.
- Copy the above example to your local terraform project root as referred in this example
- For ease of testing , we can keep both terraform and rego files in the same project and in the root folder as shown in the git repo. Ideally you would want to keep a seperate repo for all your policies.
- Set ( Export) aws access key , secret key and session tokens in the command prompt.
- From the command prompt, CD into terraform project root folder Run below commands in sequence to test the policies. You will have to run plan on terraform templates, convert the output binay into JSON files and then validate the policies using OPA.
terraform init
terraform plan --out tfplan.binary
terraform show -json tfplan.binary > tfplan.json
opa eval --format pretty --data s3-validate.rego --input tfplan.json "data.terraform.analysis.score"
opa eval --format pretty --data s3-validate.rego --input tfplan.json "data.terraform.analysis.authz"
"false" flag indicates the terraform code didn't meet one of the compliance policies.
- “data.terraform.analysis.authz” returns true / false indicating success of failure of evaluation of policies
“data.terraform.analysis.score” : returns either zero or total count of score indicating weight of evaluation of policies . This weight (score) can be used to set a threshold in the CICD pipeline.
for example : if score > 20 then fail the pipeline , where 20 is you minimum allowed threshold value.
Setting up OPA in enterprise environment using GitLab
- Create a new repo in Gitlab for creating a custom docker image
- Create a custom docker image , this will be used in creating Gitlab pipeline – here is a working example : Dockerfile
- Create .gitlab-ci.yml file in the root for the repo – here is a working example :.gitlab-ci.yml
- Running pipeline on this project(.gitlab-ci.yml) will create an image at specified location in Gitlab server as specified in the variable –destination
- Create a terraform project ,refer to the S3 project we have used in the above steps.
- Refer to the repo for a complete opa example which validates following compliance policies
- s3 buckets should always be created in a specific region
- bucket’s ACL property should always set to ‘private’ unless it is a website hosting bucket Sample code :
- S3 encryption should be enabled.
- Minimum required tags should always be set
- S3 logging should always be enabled
- Copy the above example to your git lab repo
- Refer to the .gitlab-ci.yml file from the s3 sample repo, we are using the custom docker image that was created earlier.
- This gitlab pipeline has three stages where each stage has a dependency on the previous stage
validate : validates terraform YAML file and configuration file for syntax and programming errors
plan : runs terraform init, runs terraform plan and generates output file
compliance : evaluates terraform output json file for compliance & security validations using opa . we are using "data.terraform.analysis.score" to determine if pipeline can proceed or not
apply(deploy) : manually invoke apply stage to deploy resources into cloud as per the generated plan - Create environment variables with in the CICD pipeline for authenticating with you AWS account, these variables are used in gitlab pipeline code.
AWS_ACCESS_KEY_ ID
AWS_SECRET_ACCESS_KEY
AWS_SESSION_TOKEN
- Run the pipeline
Refer to this article to learn more about terraform providers and modules