SparkBoard - AWS Serverless Task & Announcement Platform

Project Introduction

SparkBoard is a complete AWS Serverless full-stack application demonstration project that provides task management, announcement publishing, file sharing, and real-time notification capabilities on a serverless collaboration platform. The project adopts Infrastructure as Code (IaC) design, building complete cloud architecture through AWS CDK, integrating 12 AWS services including API Gateway, Lambda, DynamoDB, Cognito, S3, SQS, SNS, and EventBridge, showcasing enterprise-grade Serverless application best practices.

Core Philosophy

• Fully Serverless Architecture: Zero operational costs, auto-scaling, pay-per-use billing
• DynamoDB Single Table Design: Optimized data access patterns, reduced API calls, lower costs
• Event-Driven Microservices: 9 independent Lambda functions with single responsibility and independent scaling
• Complete Observability: CloudWatch Dashboard, X-Ray tracing, alarm systems
• Production-Grade CI/CD: GitHub Actions + AWS OIDC automated deployment

My Responsibilities

As the sole developer of the project, I am responsible for:

System Architecture Design

• Designed five-layer Stack architecture: Storage → Auth → API → Messaging → Monitoring
• Implemented DynamoDB single table design: 2 GSI indexes supporting multiple query patterns
• Established 9 Lambda microservices: Items, AuthMe, Uploads, Users, Health, Monitoring, NotificationHandler, AutoArchive, PostConfirm
• Designed SQS + SNS asynchronous notification architecture for decoupling and reliability

Infrastructure as Code (AWS CDK)

• Storage Stack (storage-stack.ts): DynamoDB single table + S3 file storage
• Auth Stack (auth-stack.ts): Cognito User Pool + PostConfirmation Trigger
• API Stack (api-stack.ts): API Gateway REST API + 6 synchronous Lambdas
• Messaging Stack (messaging-stack.ts): SQS Queue + SNS Topic + Notification Lambda
• Monitoring Stack (monitoring-stack.ts): CloudWatch Dashboard + Alarms + X-Ray

Lambda Function Development (Node.js 18)

• Synchronous Services (6):
  • Items (733 lines): Task/announcement CRUD, state management, archive functionality
  • Users (470 lines): User management, RBAC permission control
  • Monitoring (417 lines): Monitoring dashboard, metrics queries
  • Uploads (218 lines): S3 presigned URL generation
  • AuthMe (312 lines): User data query and update
  • Health (27 lines): Health check endpoint
• Asynchronous Services (3):
  • NotificationHandler (378 lines): SQS-triggered email sending
  • AutoArchive (50 lines): EventBridge scheduled archiving
  • PostConfirm (50 lines): Cognito-triggered auto group join

Frontend Development (React 19 + TypeScript)

• Vite build tool configuration and optimization
• TanStack Query data fetching and caching strategy
• Amazon Cognito Identity JS authentication integration
• TailwindCSS 4 + Radix UI component design
• Recharts 2 monitoring dashboard visualization

CI/CD & Deployment

• GitHub Actions workflows: ci.yml (testing) + cdk-deploy.yml (deployment)
• AWS OIDC keyless authentication integration
• PowerShell deployment scripts (deploy-all-stacks.ps1)
• Multi-stack dependency management and ordered deployment

Core Features

1. User Authentication System (Cognito)

• Registration & Login: Email verification, password strength validation
• JWT Token Authorization: API Gateway Cognito Authorizer
• User Group Management: Users (regular users), Admins (administrators)
• PostConfirmation Trigger: New users automatically join Users group
• Personal Profile Management: GET/PATCH /auth/me endpoints

2. Task & Announcement Management (Items CRUD)

• Create Items: POST /items - supports type (task/announcement), title, description, due date, S3 attachments
• Query List: GET /items - supports pagination, type filtering, user filtering
• Update Status: PATCH /items/{id} - completion status, archive status
• Delete Items: DELETE /items/{id} - soft delete mechanism
• DynamoDB Design:
  • PK: ITEM#{ulid}, SK: METADATA
  • GSI1: userId + createdAt (query user items)
  • GSI2: ITEM#ALL + createdAt (global list)

3. File Upload System (S3 Presigned URL)

• Secure Upload: POST /uploads - generates 15-minute valid presigned URLs
• File Type Restrictions: supports image/*, application/pdf, .docx, .xlsx
• Size Limit: maximum 10MB
• S3 Bucket Configuration:
  • Auto-delete objects (development environment)
  • CORS configuration supports direct frontend upload
  • Lifecycle policies (future expansion)

4. Asynchronous Email Notification (SQS + SNS)

• Trigger Condition: When task is marked as completed

• Message Queue: SQS Queue decouples Items Lambda from notification system

• Message Format:

  {
    "eventType": "TASK_COMPLETED",
    "itemId": "01HXY...",
    "title": "Complete Report",
    "userId": "user-123"
  }
  

• SNS Email Sending: Notification Lambda queries user email then sends via SNS

• Error Handling:

  • Visibility Timeout: 30 seconds
  • Max Receive Count: 3 attempts
  • Dead Letter Queue (DLQ)

5. Scheduled Auto-Archive (EventBridge)

• Schedule Rule: rate(1 minute) - executes every minute
• Archive Logic:
  • Query completed tasks past due date
  • Batch update archived: true
  • Record autoArchiveAt timestamp
• DynamoDB Query:
  • Use GSI2 to query ITEM#ALL partition
  • FilterExpression: status = COMPLETED AND dueDate < NOW

6. Monitoring & Observability

• CloudWatch Dashboard:
  • Lambda Invocations, Duration, Errors
  • DynamoDB Read/Write Capacity Units
  • SQS Messages Available/In Flight
  • API Gateway 4XX/5XX Errors
• X-Ray Tracing: End-to-end request tracing, service maps
• CloudWatch Alarms:
  • Lambda error rate > 5%
  • DynamoDB read/write capacity exceeds 80%
  • SQS DLQ message count > 0

Technologies Used

Infrastructure (AWS CDK TypeScript)

• AWS CDK (2.x): Infrastructure as Code
• TypeScript (5.x): Type-safe IaC
• Core AWS Services:
  • API Gateway (REST API)
  • Lambda (Node.js 18)
  • DynamoDB (On-demand billing)
  • Cognito (User Pool)
  • S3 (File storage + Static website)
  • CloudFront (CDN)
  • SQS (Message queue)
  • SNS (Email notification)
  • EventBridge (Scheduled tasks)
  • CloudWatch (Logging & monitoring)
  • X-Ray (Distributed tracing)
  • IAM (Permission management)

Backend Lambda Functions (Node.js)

• AWS SDK v3: @aws-sdk/client-dynamodb, @aws-sdk/client-s3, @aws-sdk/client-sns
• UUID Generation: ulid (time-sortable IDs)
• Testing Framework: Jest + @aws-sdk/client-dynamodb (unit tests)

Frontend (React + TypeScript)

Development Tools

• GitHub Actions: CI/CD automation
• ESLint: Code quality checks
• Jest: Unit testing
• PowerShell: Deployment scripts

Project Status

Current Version: Production-Grade Complete Project

• Branch Status: feature/SQS_SNS (development branch)
• CDK Stacks: All 5 stacks successfully deployed

Feature Completion

• ✅ Completed:
  • Complete AWS CDK infrastructure (5 stacks)
  • 9 Lambda microservices (2,605 lines of code)
  • DynamoDB single table design + 2 GSIs
  • Cognito authentication integration
  • S3 presigned URL file upload
  • SQS + SNS asynchronous email notification
  • EventBridge scheduled auto-archive
  • CloudWatch Dashboard + Alarms
  • X-Ray distributed tracing
  • React 19 frontend application
  • GitHub Actions CI/CD
  • Complete documentation (692-line final report)
• 📋 Extensible Features:
  • WebSocket real-time notifications
  • Lambda Layers shared code
  • DynamoDB Streams data synchronization
  • Step Functions complex workflows
  • CloudFront frontend deployment

Development Challenges & Learnings

1. DynamoDB Single Table Design

Challenge: How to support multiple query patterns in a single table?

Solution:

• Primary Key Design: PK = ITEM#{id} | USER#{id}, SK = METADATA | PROFILE
• GSI1 Design: Query all items for specific user
  • PK: userId
  • SK: createdAt (time-sorted)
• GSI2 Design: Query global item list
  • PK: ITEM#ALL (fixed value, shared by all items)
  • SK: createdAt

Learnings:

• Understanding NoSQL data modeling best practices
• Mastering GSI index design techniques
• Learning to balance query performance and costs

2. SQS + SNS Asynchronous Architecture

Challenge: How to implement a reliable email notification system?

Solution:

• Decoupled Design: Items Lambda doesn't directly call SNS, sends messages to SQS instead
• Reliability Guarantees:
  • SQS guarantees at-least-once delivery
  • 30-second Visibility Timeout prevents duplicate processing
  • Max 3 attempts before DLQ
• Error Tracking: DLQ with CloudWatch Alarm monitors failed messages

Learnings:

• Deep understanding of event-driven architecture
• Mastering message queue best practices
• Learning distributed system fault tolerance design

3. Lambda Cold Start Optimization

Challenge: Lambda cold start times affect API response speed.

Solution:

• Microservice Splitting: Smaller Lambda functions start faster (Items: 733 lines still reasonable)

• SDK v3 Modularization: Import only needed AWS SDK modules

  // ✅ Correct: Import only DynamoDB Client
  import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
  
  // ❌ Wrong: Import entire SDK
  import AWS from 'aws-sdk';
  

• Provisioned Concurrency: Future option for high-traffic endpoints

Learnings:

• Understanding Lambda execution environment lifecycle
• Learning JavaScript bundle optimization techniques
• Mastering serverless performance tuning

4. API Gateway Cognito Authorization

Challenge: How to integrate Cognito JWT Token validation?

Solution:

• API Gateway Authorizer: Configure Cognito User Pool as authorization source

• Token Passing: Frontend passes JWT in Authorization Header

• Lambda Context: Retrieve user info from event.requestContext.authorizer.claims

  const userId = event.requestContext.authorizer.claims.sub;
  const email = event.requestContext.authorizer.claims.email;
  

Learnings:

• Deep understanding of OAuth 2.0 / OpenID Connect flow
• Mastering JWT Token validation mechanism
• Learning API Gateway authorization design patterns

5. GitHub Actions OIDC Deployment

Challenge: How to securely deploy AWS resources in GitHub Actions without storing Access Keys?

Solution:

• OIDC Trust Relationship: AWS IAM Role trusts GitHub's OIDC Provider
• Short-term Credentials: GitHub Actions obtains temporary STS Token (15-minute validity)
• Least Privilege Principle: IAM Role grants only CDK deployment permissions

Learnings:

• Understanding OIDC federated identity authentication
• Mastering AWS STS AssumeRoleWithWebIdentity
• Learning keyless CI/CD best practices

6. CloudWatch Observability Design

Challenge: How to effectively monitor 9 Lambda functions?

Solution:

• Unified Dashboard: Single panel displays all key metrics
• Tiered Alarms:
  • P0: DLQ has messages (immediate action)
  • P1: Lambda error rate > 5% (handle within 1 hour)
  • P2: DynamoDB capacity > 80% (evaluate scaling)
• X-Ray Service Map: Visualize service dependencies

Learnings:

• Understanding three pillars of observability (Metrics, Logs, Traces)
• Mastering CloudWatch alarm strategy design
• Learning distributed tracing best practices

Project Highlights

Technical Innovation

• ✅ Complete Serverless architecture demonstration (12 AWS services integrated)
• ✅ DynamoDB single table design best practices (2 GSIs optimize queries)
• ✅ SQS + SNS asynchronous processing architecture (decoupling and reliability)
• ✅ EventBridge scheduled tasks (Cron scheduling)
• ✅ X-Ray end-to-end tracing (distributed system observability)

Engineering Practices

• ✅ Infrastructure as Code (AWS CDK TypeScript)
• ✅ CI/CD automated deployment (GitHub Actions + OIDC)
• ✅ Complete monitoring and alarm system (CloudWatch Dashboard + Alarms)
• ✅ Microservice separation of concerns (9 independent Lambdas)
• ✅ Comprehensive documentation (692-line final report, 465-line README)

Cost Optimization

• ✅ On-demand DynamoDB (no idle costs)
• ✅ Lambda billed only on requests (serverless operations)
• ✅ S3 lifecycle policies (auto-delete expired files)
• ✅ CloudWatch Logs retention (auto-delete after 30 days)

Learning Outcomes

• ✅ Deep implementation of AWS serverless architecture
• ✅ Mastering DynamoDB NoSQL design patterns
• ✅ Understanding event-driven microservice architecture
• ✅ Learning cloud application observability design