AWS DevSecOps Architecture & Implementation Report

---

1. Introduction

This document details the activities I performed as an AWS Solution Architect and DevSecOps Engineer during the design and deployment of a real‑world infrastructure automation project. The primary goals of the project were:

• To provision a secure and scalable environment using AWS services.
• To automate the deployment of EC2 instances and associated infrastructure using Terraform (IaC).
• To automatically install and configure a mandatory security agent on all provisioned EC2 instances.
• To use AWS Systems Manager (SSM) to orchestrate agent installation post‑provision.
• To implement Amazon Simple Notification Service (SNS) to deliver automated notifications about the deployment and installation processes.

The following sections describe the architectural decisions, implementation steps, security considerations, and DevSecOps practices applied throughout the project and the AWS Solution I used

---

2. Requirements Analysis & Planning

Before beginning implementation, I gathered requirements and constraints to define the correct AWS services, IaC design, and operational workflows.

2.1 Functional Requirements

• Deploy multiple EC2 instances automatically.
• Ensure instances meet security and compliance baselines.
• Install a specific security agent on every EC2 instance immediately after provisioning.
• Track installation and system compliance status.
• Provide automated notifications of success/failure.

2.2 Non‑Functional Requirements

• Use Infrastructure as Code for complete environment reproducibility.
• Ensure security hardening and least privilege access.
• Ensure modular Terraform code design.
• Minimize manual operations by leveraging AWS Systems Manager.
• Ensure auditable workflow for installation tasks.

2.3 Architecture Constraints

• Must rely on AWS‑native services.
• Must support future scaling and modularity.
• Must support fully automated re‑creation of environment.

---

3. High‑Level Architecture

Below is the conceptual architecture used in the project.

                   +-----------------------------+
                   |        Terraform IaC        |
                   | (Provision Entire Stack)    |
                   +-------------+---------------+
                                 |
                                 v
                +----------------+-----------------+
                |         AWS Infrastructure        |
                +-----------------------------------+
                | VPC + Subnets + Security Groups   |
                | IAM Roles + Instance Profiles     |
                | EC2 Instances (x N)               |
                +----------------+------------------+
                                 |
                                 v
                   +-------------+-------------+
                   | AWS Systems Manager (SSM) |
                   |  - Run Command            |
                   |  - SSM Agent              |
                   +-------------+-------------+
                                 |
                +----------------+----------------+
                | Install Security Agent on EC2    |
                +----------------------------------+
                                 |
                                 v
                      +----------+----------+
                      | Amazon SNS (Email) |
                      | - Deployment Status |
                      +---------------------+

This architecture represents the interaction between Terraform, EC2, SSM, and SNS throughout the full lifecycle of deployment and configuration.

---

4. Terraform Infrastructure Deployment Activities

As the DevSecOps Engineer/Solution Architect, my first major activity was designing and implementing the infrastructure using Terraform.

4.1 Terraform Project Structure

To maintain scalability and modularity, I used a multi‑module IaC structure:

/terraform
  /modules
    /networking
    /compute
    /iam
    /security-groups
  main.tf
  variables.tf
  outputs.tf

4.2 Networking Module Activities

• Designed a secure and isolated VPC.
• Created public and private subnets across multiple availability zones.
• Implemented routing through IGW/NAT gateways.
• Ensured EC2 instances intended for agent installations had SSM access.

4.3 IAM Roles & Policies Activities

• Created an EC2 IAM Role with required SSM permissions:
  • AmazonSSMManagedInstanceCore
  • CloudWatchAgentServerPolicy
• Created least‑privilege Terraform execution roles.
• Ensured AWS credentials were managed securely.

4.4 Compute Module Activities

• Automated creation of required EC2 instances.
• Embedded tags for identification and automation (SSMTarget=true).
• Attached SSM‑compatible AMIs (Amazon Linux 2 / Ubuntu with SSM pre‑installed).
• Configured user data for basic initialization.
• Associated instance profiles for SSM access.

4.5 Security Group Activities

• Implemented least‑privilege inbound/outbound rules.
• Ensured no public ports were open unless explicitly required.
• Restricted SSH access to trusted IPs only.

4.6 SNS Resource Deployment

• Created SNS topic dedicated to deployment notifications.
• Configured email subscriptions.
• Added SNS topic ARN as Terraform output for integration with SSM Run Command.

---

5. Security Agent Installation Using AWS Systems Manager

After provisioning infrastructure with Terraform, the next critical phase was automated installation of the security agent.

5.1 Why AWS Systems Manager?

• No need for SSH access.
• Provides centralized automation.
• Supports command execution across multiple instances.
• Auditable through AWS CloudTrail.
• Agent‑installation scripts can be version‑controlled.

5.2 SSM Run Command Workflow

I performed the following sequence to deliver the security agent to each instance:

1. Verified SSM Agent connectivity.
2. Created an SSM Document (optional) for repeatable installation steps.
3. Triggered AWS-RunShellScript to run the installation commands.
4. Used SSM parameters to define command arguments.
5. Captured output logs.
6. Forwarded results to SNS.

5.3 Example Installation Logic (Descriptive)

• Download security agent package from vendor URL / internal S3 bucket.
• Validate checksum.
• Install package using OS‑specific installer.
• Validate installation using process/services check.

5.4 Error Handling and Resilience

• SSM retries configured.
• SNS sends failure email if installation fails.
• Logs stored in CloudWatch for analysis.

---

6. SNS Notification Implementation

SNS was integrated into the workflow to provide real‑time updates to stakeholders.

6.1 Activities Performed

• Created SNS topic (security-agent-install-status).
• Configured email protocol subscription.
• Integrated topic with SSM Run Command output.
• Configured Terraform to output topic ARN.

6.2 Notification Lifecycle

Notifications were sent for:

• Deployment started.
• Agent installation in progress.
• Successful installation.
• Failure with full error detail.

This transparency ensures operational awareness for DevSecOps and security teams.

---

7. End‑to‑End DevSecOps Pipeline Flow

Below is a simplified workflow diagram showing how all components work together.

                Developer Commits Terraform Code
                                |
                                v
                      Terraform Plan & Apply
                                |
                                v
            +----------------------------------+
            | AWS Infrastructure Provisioned   |
            +----------------------------------+
                   | EC2 Instances Online
                                |
                                v
                 AWS Systems Manager (SSM)
                                |
       Run Command Executes Agent Installation Script
                                |
                                v
                     Agent Installed on EC2
                                |
                                v
                 SNS Sends Notification Email

---

8. Security Considerations

As a DevSecOps Engineer/Solution Architect, ensuring security was embedded into each layer of the solution.

8.1 Infrastructure Security

• Private subnets used where possible.
• No open ports except necessary.
• IAM least privilege enforced.
• SSM used instead of SSH.

8.2 Data Security

• No hardcoded credentials.
• Terraform state stored securely (remote state backend recommended).

8.3 Compliance & Governance

• Logs were stored in CloudWatch.
• SSM execution recorded in CloudTrail.
• Mandatory agent reinforces compliance posture.

---

9. Challenges and Solutions

Challenge 1: Ensuring EC2 Instances Were Ready for SSM

Solution: Used AMIs with SSM agent preinstalled and validated via metadata checks.

Challenge 2: Coordinating Terraform and SSM Run Command

Solution: Added creation‑time triggers and dependencies to ensure SSM executed only after instance provisioning.

Challenge 3: Handling Security Agent Failures

Solution: Implemented retry logic and detailed SNS alerts.

---

10. Conclusion

This project demonstrates a complete DevSecOps‑driven AWS architecture, combining Infrastructure as Code, automation, security enforcement, and notification workflows. By leveraging Terraform, AWS Systems Manager, and SNS, I created a fully automated, scalable, and secure solution that meets real‑world enterprise operational standards.

The solution ensures:

• Full automation of resource provisioning.
• Zero manual installation steps for security agents.
• Clear operational visibility via notifications.
• Strong security through IAM, SSM, and controlled networking.

This design and implementation showcase the best practices expected of an AWS Solution Architect working within a DevSecOps environment.