Creating Your Abstract Backend Message Queue Service Implementation

This guide provides step-by-step instructions for setting up your message queue service project using this template.

Method 1: Using GitHub Template (Recommended)

Step 1: Create Repository from Template

1. Navigate to the Abstract Backend Implementation Template repository
2. Click the "Use this template" button (green button)
3. Select "Create a new repository"
4. Configure your new repository:
   • Repository name: abstract-backend-<your-message-queue-name>-service (e.g., abstract-backend-redis-service)
   • Description: Message queue service implementation for Abstract Backend using <Your Message Queue Service>
   • Visibility: Choose Public (recommended for community plugins) or Private
   • Include all branches: Leave unchecked

Step 2: Clone Your New Repository

git clone https://github.com/YOUR-USERNAME/abstract-backend-YOUR-MESSAGE-QUEUE-service.git
cd abstract-backend-YOUR-MESSAGE-QUEUE-service

Method 2: Manual Setup

Step 1: Download or Clone Template

Git Clone

git clone https://github.com/Chisanan232/abstract-backend.git abstract-backend-your-message-queue-service
cd abstract-backend-your-message-queue-service
rm -rf .git  # Remove template's git history
git init     # Initialize new git repository

Download ZIP

1. Download the template as a ZIP file from GitHub
2. Extract to your desired directory
3. Rename the folder to your project name
4. Initialize a new git repository:

cd your-project-directory
git init

Step 3: Project Customization

Update Project Metadata

Edit pyproject.toml to customize your plugin:

[project]
name = "abstract-backend-YOUR-MESSAGE-QUEUE-service"  # Replace YOUR-MESSAGE-QUEUE
version = "0.1.0"
description = "📮 A YOUR-MESSAGE-QUEUE service implementation for Abstract Backend."
authors = [{ name = "Your Name", email = "your.email@example.com" }]

# Update URLs to your repository
[project.urls]
Homepage = "https://github.com/YOUR-USERNAME/abstract-backend-YOUR-MESSAGE-QUEUE-service"
Repository = "https://github.com/YOUR-USERNAME/abstract-backend-YOUR-MESSAGE-QUEUE-service"
Documentation = "https://YOUR-USERNAME.github.io/abstract-backend-YOUR-MESSAGE-QUEUE-service/"

# Configure your service entry point
[project.entry-points."abe.backends.message_queue.service"]
your_message_queue_name = "abe_plugin.backends.message_queue.service.your_queue_service:YourMessageQueueBackend"

Rename Source Files

Update the source file structure to match your message queue implementation:

# Rename the main implementation file
mv abe_plugin/backends/message_queue/service/your_queue_service.py abe_plugin/backends/message_queue/service/<your_message_queue_service>.py

Update Documentation

1. README.md: Replace template content with your queue-specific information
2. Documentation: Update docs in docs/contents/document/ with your implementation details
3. Changelog: Document your initial version in docs/contents/document/changelog.mdx

Step 4: Development Environment Setup

Initialize the Project

# Create virtual environment and install dependencies
uv sync

# Activate the virtual environment
source .venv/bin/activate  # Linux/macOS
# or .venv\Scripts\activate  # Windows

Install Development Dependencies

# Install all development dependencies
uv sync --group dev

# Install pre-commit hooks
uv run pre-commit install

Verify Installation

# Run tests to ensure everything is working
uv run pytest

# Check code quality
uv run black --check .
uv run pylint src/
uv run mypy src/

Step 5: Implement Your Queue Backend

Basic Implementation Structure

Replace the template implementation in your main source file:

"""
Your Message Queue Backend Implementation

This module provides a YOUR-MESSAGE-QUEUE implementation of the MessageQueueBackend protocol
for Abstract Backend.
"""

import os
from typing import Any, AsyncIterator, Dict
from abe.backends.message_queue.base.protocol import MessageQueueBackend


class YourMessageQueueBackend(MessageQueueBackend):
    """YOUR-MESSAGE-QUEUE implementation of MessageQueueBackend."""

    def __init__(self, connection_url: str):
        """Initialize the YOUR-QUEUE backend.
        
        Args:
            connection_url: Connection string for YOUR-QUEUE service
        """
        self.connection_url = connection_url
        # Initialize your queue client here

    @classmethod
    def from_env(cls) -> "YourMessageQueueBackend":
        """Create instance from environment variables.
        
        Expected environment variables:
        - YOUR_MESSAGE_QUEUE_URL: Connection URL for YOUR-MESSAGE-QUEUE service
        
        Returns:
            Configured YourMessageQueueBackend instance
        """
        connection_url = os.environ.get("YOUR_MESSAGE_QUEUE_URL", "default://localhost")
        return cls(connection_url)

    async def publish(self, key: str, payload: Dict[str, Any]) -> None:
        """Publish a message to YOUR-MESSAGE-QUEUE."""
        # Implement your publish logic here
        pass

    async def consume(self, key: str) -> AsyncIterator[Dict[str, Any]]:
        """Consume messages from YOUR-MESSAGE-QUEUE."""
        # Implement your consume logic here
        while True:
            # Yield messages as they arrive
            yield {}

Step 6: Configuration Examples

Environment Variables

Create example configuration for users:

# .env.example
YOUR_MESSAGE_QUEUE_URL=redis://localhost:6379/0
YOUR_MESSAGE_QUEUE_USERNAME=optional_username
YOUR_MESSAGE_QUEUE_PASSWORD=optional_password
YOUR_MESSAGE_QUEUE_SSL=false
MESSAGE_QUEUE_BACKEND=your_message_queue_name

Docker Compose (Optional)

If your queue service can run in Docker, provide a docker-compose.yml:

version: '3.8'
services:
  your-queue:
    image: your-queue:latest
    ports:
      - "6379:6379"  # Adjust port as needed
    environment:
      - YOUR_MESSAGE_QUEUE_CONFIG=value

Step 7: Testing Your Implementation

Write Tests

Create tests in test/unit_test/ for your implementation:

import pytest
from your_module import YourMessageQueueBackend

@pytest.mark.asyncio
async def test_publish_consume():
    """Test basic publish/consume functionality."""
    backend = YourMessageQueueBackend.from_env()
    
    # Test publishing
    await backend.publish("test-key", {"message": "hello"})
    
    # Test consuming
    async for message in backend.consume("test-key"):
        assert message["message"] == "hello"
        break

Run Tests

# Run all tests
uv run pytest

# Run with coverage
uv run pytest --cov=src --cov-report=html

Step 8: Documentation Setup

Build Documentation

cd docs
npm install  # or pnpm install
npm run build

Update Documentation Content

1. Edit docs/contents/document/introduction.mdx with your queue-specific details
2. Add configuration examples in docs/contents/document/quick-start/
3. Include API documentation for your implementation

Next Steps

Now that your project is set up:

1. 🔧 Implement your queue logic - Build the core functionality
2. 🧪 Test your implementation - Ensure reliability and performance
3. 📚 Document your service - Help users understand how to use it
4. 🚀 Publish your package - Share with the broader Abstract Backend community

Common Setup Issues

Import Errors

# If you get import errors
uv sync --reinstall

Pre-commit Hook Failures

# If pre-commit hooks fail
uv run pre-commit run --all-files

Test Failures

# If tests fail due to missing dependencies
uv add --group dev your-missing-dependency

Ready to start implementing? Head to the implementation guide!