Checkout in GitHub 🔗

RSS Aggregator

This is a robust RSS scraper service built with Golang. It is designed to efficiently aggregate and manage RSS feeds from various sources, allowing users to easily access and consume content from their favorite websites.

Key Features

• Multi-Feed Support: Scrape and aggregate feeds from multiple sources simultaneously.
• RSS Feed Parsing: Efficiently parse and aggregate RSS feeds.
• Database Integration: Store scraped data in a PostgreSQL database for persistent access and management.
• Concurrency: Utilize Golang's concurrency model for faster processing.
• Error Handling: Robust error handling for network and parsing errors.
• Logging: Comprehensive logging for monitoring and debugging.

Technologies

• Golang: The core programming language used for building the service.
• PostgreSQL: Database for storing feed and post data.
• Goose: Goose is a database migration tool Resource Sharing.
• Sqlc: sqlc generates type-safe code from SQL
• Chi Router: Lightweight, idiomatic and composable router for building Go HTTP services.
• UUID: For generating unique identifiers for feeds and posts.
• CORS: Middleware for handling Cross-Origin Resource Sharing.

Getting Started

To get started with this project, you can clone the repository and build the project:

git clone https://github.com/haroonalbar/rss-aggregater
cd rss-aggregater
go build

Once the project is built, you can run the service:

./rss-aggregater

The service will start and begin scraping RSS feeds as configured.

Conclusion

This RSS scrapper is a powerful tool for aggregating and processing RSS feeds using Golang. By leveraging Go's concurrency model and robust standard library, this service is both efficient and easy to maintain. Whether you are building a news aggregator or monitoring multiple feeds, this project provides a solid foundation.