TRACKFLOW - CI/CD Pipeline Monitoring System

A comprehensive real-time CI/CD pipeline monitoring and visualization system designed to provide unified visibility into build and deployment processes across multiple platforms.

🚀 Features

Core Modules (17 Total)

1. Webhook Integration Module - Receives events from GitHub Actions, Jenkins, GitLab CI, and custom CI/CD tools
2. Backend API Module - RESTful APIs for data access and management
3. Database and Persistence Module - MongoDB-based data storage
4. Real-Time Communication Module - Socket.IO for live updates + 2s polling fallback
5. Frontend Dashboard Module - React-based responsive interface with search & tool badges
6. Data Visualization Module - Recharts-based dynamic HistoryChart + pipeline analytics
7. Notification Module - Real-time alerts, audio notifications (Web Audio API), browser desktop notifications, tab title badges
8. Demo and Integration Support Module - Testing and demonstration tools
9. Authentication and User Management Module - JWT-based secure authentication
10. Advanced Analytics and Reporting Module - Performance insights and reports
11. Tool-Specific CI/CD Adapters Module - GitHub Actions, Jenkins, GitLab CI, and auto-detection
12. Error Handling and Retry Mechanism Module - Fault-tolerant operations
13. Notification Enhancement Module - Multi-channel notification support
14. Security and Configuration Module - Security headers, rate limiting, sanitization
15. Scalability and Performance Optimization Module - Caching and optimization
16. Code Analysis Module - Interactive interface for static code analysis
17. AI-Powered Assistant Module - Chatbot for CI/CD pipeline insights and user assistance

🛠️ Tech Stack

• Frontend: React.js, Socket.IO Client, Recharts, React Router
• Backend: Node.js, Express.js, Socket.IO
• Database: MongoDB
• Authentication: JWT (JSON Web Tokens)
• Real-time: WebSocket / Socket.IO
• Version Control: Git

📋 Prerequisites

• Node.js (v16 or higher)
• MongoDB (v5 or higher) - Running locally or MongoDB Atlas
• npm or yarn

🔧 Installation

1. Clone or Navigate to Project

cd test

2. Backend Setup

cd backend
npm install

Create or verify .env file in backend directory:

PORT=5000
MONGODB_URI=mongodb://localhost:27017/trackflow
JWT_SECRET=your-super-secret-jwt-key-change-this-in-production
JWT_EXPIRE=7d
FRONTEND_URL=http://localhost:3000
GITHUB_WEBHOOK_SECRET=
JENKINS_WEBHOOK_SECRET=
GITLAB_WEBHOOK_SECRET=

3. Frontend Setup

cd ../frontend
npm install

Create .env file in frontend directory (optional):

REACT_APP_API_URL=http://localhost:5000/api
REACT_APP_SOCKET_URL=http://localhost:5000

🚀 Running the Application

Start MongoDB

Make sure MongoDB is running:

# Windows
net start MongoDB

# Linux/Mac
sudo systemctl start mongod

Start Backend Server

cd backend
npm run dev

Backend will start on http://localhost:5000

Start Frontend Application

cd frontend
npm start

Frontend will start on http://localhost:3000

📖 Usage

1. Register/Login

• Navigate to http://localhost:3000
• Register a new account or use demo credentials:
  • Email: demo@trackflow.com
  • Password: demo123

2. Seed Demo Data (Optional)

Use the demo scripts to populate with sample pipelines:

# Seed realistic pipelines from GitHub/Jenkins/GitLab
node backend/scripts/seed_pipelines.js

# Or use the demo API endpoint
curl -X POST http://localhost:5000/api/demo/seed

3. Run Integration Tests

# Test all webhook endpoints (GitHub, Jenkins, GitLab, auto-detect)
node backend/scripts/test_integrations.js

4. Simulate Webhook Events

GitHub Actions Webhook:

curl -X POST http://localhost:5000/api/webhooks/github \
  -H "Content-Type: application/json" \
  -d '{"workflow_run": {"id": 123, "name": "CI", "status": "completed", "conclusion": "success"}, "repository": {"full_name": "user/repo"}}'

Jenkins Webhook:

curl -X POST http://localhost:5000/api/webhooks/jenkins \
  -H "Content-Type: application/json" \
  -d '{"name": "My Job", "build": {"number": 42, "phase": "COMPLETED", "status": "SUCCESS"}}'

GitLab CI Webhook:

curl -X POST http://localhost:5000/api/webhooks/gitlab \
  -H "Content-Type: application/json" \
  -d '{"object_kind": "pipeline", "object_attributes": {"id": 1, "status": "success"}, "project": {"name": "my-app"}}'

Auto-Detect (sends to any platform):

curl -X POST http://localhost:5000/api/webhooks/auto \
  -H "Content-Type: application/json" \
  -d '{...any CI/CD payload...}'

5. Monitor Pipelines

• View real-time pipeline updates on the dashboard
• Search pipelines by name, repository, branch, or tool
• Filter by status (All, Success, Failure, Running)
• View tool badges (🐙 GitHub, 🤖 Jenkins, 🦊 GitLab, ⚙️ Generic)
• Click on pipelines for detailed information
• View historical trends with the Recharts-based HistoryChart
• Receive audio alerts on pipeline failures (double-beep) and successes (chime)
• Tab title shows failure count: (N FAILED) TrackFlow

📡 API Endpoints

Authentication

• POST /api/auth/register - Register new user
• POST /api/auth/login - Login user
• GET /api/auth/profile - Get user profile (protected)

Webhooks

• POST /api/webhooks/github - GitHub Actions webhook
• POST /api/webhooks/jenkins - Jenkins webhook
• POST /api/webhooks/gitlab - GitLab CI webhook (NEW)
• POST /api/webhooks/auto - Auto-detect tool from payload (NEW)
• POST /api/webhooks/generic - Generic webhook

Pipelines

• GET /api/pipelines - Get all pipelines (with pagination)
• GET /api/pipelines/:id - Get single pipeline with events
• GET /api/pipelines/:id/events - Get pipeline events/stages (NEW)
• GET /api/pipelines/stats/overview - Get statistics
• GET /api/pipelines/history/:name - Get pipeline history by name (NEW)
• GET /api/pipelines/recent/:count - Get recent pipelines

Analytics

• GET /api/analytics/overview - Overall metrics
• GET /api/analytics/trends - Success/failure trends
• GET /api/analytics/performance - Performance insights

Notifications

• GET /api/notifications - Get user notifications
• PUT /api/notifications/:id/read - Mark as read
• POST /api/notifications/mark-all-read - Mark all as read

Demo/Testing

• POST /api/demo/seed - Seed database with sample data
• POST /api/demo/trigger-github - Simulate GitHub webhook
• POST /api/demo/trigger-jenkins - Simulate Jenkins webhook
• GET /api/demo/payloads - Get sample payloads

🔌 Webhook Integration

GitHub Actions

Add a notification step to your .github/workflows/*.yml:

on:
  workflow_run:
    types: [completed]

jobs:
  notify-trackflow:
    runs-on: ubuntu-latest
    steps:
      - name: Notify TRACKFLOW
        run: |
          curl -X POST http://your-trackflow-server/api/webhooks/github \
            -H "Content-Type: application/json" \
            -d "${{ toJSON(github) }}"

Jenkins

Configure webhook in Jenkins job settings:

• URL: http://your-trackflow-server/api/webhooks/jenkins
• Method: POST
• Content-Type: application/json

GitLab CI (NEW)

Configure in GitLab → Settings → Webhooks:

• URL: http://your-trackflow-server/api/webhooks/gitlab
• Secret Token: (optional, set GITLAB_WEBHOOK_SECRET in .env)
• Trigger: Pipeline events

Auto-Detect (NEW)

For any CI/CD tool, use the auto-detect endpoint:

• URL: http://your-trackflow-server/api/webhooks/auto
• TrackFlow will automatically detect GitHub, Jenkins, GitLab, or generic payloads

🎨 Features Highlights

Real-Time Updates

• Live pipeline status changes via Socket.IO
• 2-second polling fallback when WebSocket disconnects
• Smart deduplication prevents duplicate notifications
• Connection status indicator (Connected/Polling)

Audio & Browser Notifications (NEW)

• 🔊 Failure double-beep alert (Web Audio API)
• 🎵 Success rising chime (C5→E5→G5)
• 🖥️ Browser desktop notifications on failure
• 📑 Tab title badge: (N FAILED) TrackFlow
• Mute/test sound buttons in navbar

Historical Trends (NEW)

• Recharts-based dynamic line chart
• Pipeline selector dropdown
• Color-coded dots (green/red/blue by status)
• Duration tracking over time

Multi-Platform Support

• 🐙 GitHub Actions
• 🤖 Jenkins
• 🦊 GitLab CI (NEW)
• ⚙️ Generic/Custom
• 🔍 Auto-detect from payload (NEW)

Advanced Analytics & Code Analysis

• Success/failure trends
• Performance metrics
• Duration analysis
• Tool-specific insights
• Static code analysis

AI-Powered Assistant

• Interactive chatbot interface
• Instant CI/CD insights
• Pipeline assistance and querying

Security

• JWT authentication
• Rate limiting
• NoSQL injection protection
• Secure password hashing
• CORS configuration
• Webhook signature validation (HMAC for GitHub, token for GitLab)

Developer Experience

• Clean, modern UI
• Dark theme design
• Responsive layout
• Real-time feedback
• Pipeline search (name, repo, branch, tool)
• Tool badge emojis
• Comprehensive error handling

📁 Project Structure

test/
├── backend/
│   ├── adapters/              # CI/CD tool adapters
│   │   ├── adapterFactory.js  # Factory + auto-detect
│   │   ├── githubAdapter.js   # GitHub Actions parser
│   │   ├── jenkinsAdapter.js  # Jenkins parser
│   │   └── gitlabAdapter.js   # GitLab CI parser (NEW)
│   ├── config/                # Configuration
│   ├── controllers/           # Route controllers
│   ├── middleware/
│   │   ├── authMiddleware.js
│   │   ├── errorHandler.js
│   │   ├── securityMiddleware.js
│   │   └── webhookLogger.js   # Payload logger (NEW)
│   ├── models/                # MongoDB models
│   ├── routes/                # API routes
│   ├── scripts/               # Demo & test scripts (NEW)
│   │   ├── test_integrations.js
│   │   └── seed_pipelines.js
│   ├── services/              # Business logic
│   ├── sockets/               # Socket.IO handlers
│   ├── .env                   # Environment variables
│   ├── package.json
│   └── server.js              # Entry point
│
├── frontend/
│   ├── public/
│   ├── src/
│   │   ├── components/
│   │   │   ├── Dashboard/
│   │   │   │   ├── Dashboard.jsx
│   │   │   │   ├── Dashboard.css
│   │   │   │   ├── HistoryChart.jsx    # Recharts chart (NEW)
│   │   │   │   └── StagePopup.jsx      # Stage modal (NEW)
│   │   │   ├── Layout/
│   │   │   │   ├── Navbar.jsx          # + audio controls
│   │   │   │   └── Navbar.css
│   │   │   └── ...                     # Analytics, Auth, etc.
│   │   ├── context/
│   │   │   ├── LiveStatsContext.js     # REAL data (REWRITTEN)
│   │   │   └── AuthContext.js
│   │   ├── services/
│   │   │   └── audioNotifications.js   # Web Audio API (NEW)
│   │   ├── utils/
│   │   │   ├── api.js
│   │   │   └── socket.js
│   │   ├── App.js
│   │   └── index.css
│   └── package.json
│
├── CHANGELOG.md               # (NEW)
├── MIGRATION_PLAN.md
└── README.md

🐛 Troubleshooting

MongoDB Connection Error

• Ensure MongoDB is running
• Check connection string in .env
• Verify MongoDB port (default: 27017)

Port Already in Use

• Change PORT in backend .env
• Update REACT_APP_API_URL in frontend

CORS Issues

• Verify FRONTEND_URL in backend .env
• Check browser console for specific errors

🔒 Security Notes

• Change JWT_SECRET in production
• Use environment variables for sensitive data
• Enable webhook signature validation
• Configure proper CORS origins
• Use HTTPS in production

📈 Future Enhancements

• Email notifications
• Slack/Discord integrations
• Bitbucket Pipelines adapter
• CircleCI adapter
• Custom dashboards per user
• Team collaboration features
• Audit logs
• Pipeline comparison view

👥 Contributing

This is a demonstration project. Feel free to fork and customize for your needs.

📄 License

MIT License - feel free to use for personal and commercial projects.

🙏 Acknowledgments

Built with modern web technologies and best practices for real-time monitoring systems.

---

TRACKFLOW - Empowering teams with real-time CI/CD visibility 🚀