AWS Lambda to update SES bounce notification Topics

AWS Lambda to update SES bounce notification : If you are using AWS SES ( Simple Email Service ) to send out emails to your customers, partners through your cloud applications, over the time you may come across bounce email issue which need to be resolved immediately. I advise you to prepare for the worst case scenario and have a process built for handling bounce emails.

Amazon will put your account in probation period if the email bounce rate goes over 10%, so it is very important to have a process developed to handle the scenario if it ever happens to you. refer to this AWS FAQ page for more information about the probation and bounce emails.

The easiest and simplest solution is to create a SNS topic and have an email group ( which sends issues to a ticketing system ) subscribe to the Topic.

Most time emails bounce either due to

• Invalid/terminated email (hard bounce)
• Blocked from receiver/inbox full/server down (soft bounce) .

The message you receive from bounce SNS topic contains the email address which bounced and evaluate each email and remove it from your system / database. here is a sample response you will receive from SES bounce email into your SNS bounce Topic

{
    "Records": [
        {
            "EventSource": "aws:sns",
            "EventVersion": "1.0",
            "EventSubscriptionArn": "arn:aws:sns:us-east-1:XXXXXXXXXX:YOUR-SNS-BOUNCE-TOPIC:3-04a1-46b7-a707-4453",
            "Sns": {
                "Type": "Notification",
                "MessageId": "fdfd-3ds-er-91b4-33d",
                "TopicArn": "arn:aws:sns:us-east-1:XXXXXXXXXX:YOUR-SNS-BOUNCE-TOPIC",
                "Subject": null,
                "Message": "{\"notificationType\":\"Bounce\",\"bounce\":{\"feedbackId\":\"010701737f8a0ab1-4cbf-496d-b33a-343432-000000\",\"bounceType\":\"Permanent\",\"bounceSubType\":\"OnAccountSuppressionList\",\"bouncedRecipients\":[{\"emailAddress\":\"test@test.com\",\"action\":\"failed\",\"status\":\"5.1.1\",\"diagnosticCode\":\"Amazon SES did not send the message to this address because it is on the suppression list for your account. For more information about removing addresses from the suppression list, see the Amazon SES Developer Guide at https://docs.aws.amazon.com/ses/latest/DeveloperGuide/sending-email-suppression-list.html\"}],\"timestamp\":\"2020-07-24T06:37:00.049Z\",\"reportingMTA\":\"dns; amazonses.com\"},\"mail\":{\"timestamp\":\"2020-07-24T06:37:00.049Z\",\"source\":\"sender@test.com\",\"sourceArn\":\"arn:aws:ses:us-east-1:XXXXXXXXXX:identity/yourdomain\",\"sourceIp\":\"0.0.0.0.0\",\"sendingAccountId\":\"XXXXXXXXXX\",\"messageId\":\"0009090-898989-98398989\",\"destination\":[\"test1@test.com\",\"test2@test.com\"]}}",
                "Timestamp": "2020-07-24T06:37:00.075Z",
                "SignatureVersion": "1",
                "Signature": "jljfdkljgldfj948[edklkl;ds08239edlflsdj",
                "SigningCertUrl": "https://sns.us-east-1.amazonaws.com/SimpleNotificationService-test.pem",
                "UnsubscribeUrl": "https://sns.us-east.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-1:XXXXXXXXXX:EMAIL-BOUNCE-SNS-TOPIC:hkhkj-hdjkhkf-mkeje-eere",
                "MessageAttributes": {}
            }
        }
    ]
}

To receive the bounce notification into the SNS topic, update the Bounce notification SNS Topic for each of the email / domain in the account as shown below.

However is is not practical to keep updating the SNS topics manually every time someone creates an email or a domain.

Here is a solution using Lambda that checks all emails and domains in the given region daily ( CW schedule) and checks if an identify (email) is configured with a SNS topic or not , and updates the identify with a given SNS Topic if it is not already configured.

Complete SES bounce email lambda code ( python ) is provided on my Github project and refer to this link for boto api for SNS service

First get list of all the emails and domains in the region ( where lambda is running )

 # get list of all emails and domains (Identities)
    identityList = client.list_identities(
 )

Loop through list of identities and get Notification attributes for each identity, this response contains all the notification attributes and tell us if they are already configured or not.

for identity in identityList['Identities']:
    print("identity=", identity)
    response = client.get_identity_notification_attributes(
        Identities=[
            identity,
        ]
)

Following is sample JSON that returns as response for each identity, doing a simpel print (response) will provide something as follows.

'test@test.com': {
	'BounceTopic': 'arn of the SNS topic if it is already configured',
	'ForwardingEnabled': True,
	'HeadersInBounceNotificationsEnabled': False,
	'HeadersInComplaintNotificationsEnabled': False,
	'HeadersInDeliveryNotificationsEnabled': False
}

Next step is, check if the BounceTopic is configured for the email. variable bounceTopicExists returns True or False

bounceTopicExists = "BounceTopic" in response['NotificationAttributes'][identity]
print(bounceTopicExists)

Final step is to make an API call to update the Topic if bounceTopicExists returns as False

#if identity doesn't have a BounceTopic, configure it
if bounceTopicExists == False:
    bounce_notif = client.set_identity_notification_topic(Identity=identity, NotificationType="Bounce", SnsTopic=sns_topic)

To automate the lambda to run daily, go to Cloud Watch , select Events -> Rules – > Create Rule -> Schedule. provide Cron job expression or select fixed rate schedule using provided dropdown.
Select your Target on right hand side as the Lambda function that we just created.

Cloud watch schedule can be automated using terraform in just few lines as shown below

resource "aws_cloudwatch_event_rule" "every_day" {
  name                = "${local.tag_prefix}-SES-NOTIFICATION-LAMBDA-RULE"
  description         = "Fires every day"
  schedule_expression = "rate(1 day)"
}

resource "aws_cloudwatch_event_target" "ses_notification_every_day" {
  rule      = aws_cloudwatch_event_rule.every_day.name
  target_id = "your Lambda Name"
  arn       = "Your Lambda ARN"
}

Multi account setup

This solution can be implemented with in a multi account setup as well, you will deploy the lambda in a shared services / master account and then grant necessary permissions for the lambda execute role and create a assume role in each of the member accounts. follow this Amazon’s documentation for more details

This solution can be implemented in multi region as well. just pass the region during the api call as shown below

ses_client = boto3.client(
        'ses',
        region_name=region,
        aws_access_key_id=ACCESS_KEY,
        aws_secret_access_key=SECRET_KEY,
        aws_session_token=SESSION_TOKEN,)

also take look at the other popular article on this blog site