Merge pull request #12 from Hestia-Homes/main

Completed template setup and deployment of backend - testing automated deployment

.github/workflows/deploy_fastapi_backend.yml

@@ -0,0 +1,36 @@
+name: Serverless Deploy
+
+on:
+  push:
+    branches: [ dev, prod ]
+
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v3
+
+      - name: Set up Python
+        uses: actions/setup-python@v2
+        with:
+          python-version: 3.10
+
+      - name: Install Serverless
+        run: npm install -g serverless
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements/base.txt
+
+      - name: AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
+          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+          aws-region: eu-west-2
+
+      - name: Deploy to AWS Lambda via Serverless
+        run: sls deploy --stage ${{ github.ref_name }}

.gitignore

@@ -242,3 +242,6 @@ fabric.properties
 
 *.DS_Store
 infrastructure/terraform/.terraform*
+
+# Don't commit packages up serverless packages
+.serverless

backend/.env.example

@@ -0,0 +1,4 @@
+API_KEY = example-api-key
+ENVIRONMENT = local
+SECRET_KEY = YOUR_SECRET_KEY
+ALGORITHM = HS256

backend/README.md

@@ -0,0 +1,143 @@
+# Backend
+This is the api service that will supply the frontend with the insights that are driven by the machine
+learning and data modelling services.
+
+# Usage
+
+## Prerequisites
+Python 3.8+
+Poetry for managing project dependencies and virtual environment.
+
+## Installation and setup
+1. Clone this directory and navigate into the project directory.
+
+```commandline
+git clone https://github.com/Hestia-Homes/Model.git
+cd backend
+```
+
+2. For environment management, I'm using conda with pycharm which is a convenient setup for development
+on a mac M1 however using tools such as poetry or pipenv is also fine.
+
+For example, to install conda and create a virtual environment for this project, run the following commands:
+
+```commandline
+conda create -n backend python=3.10
+conda activate backend
+```
+
+then enter the virtual environment and install the dependencies using conda.
+
+```commandline
+conda install --file requirements/base.txt
+```
+
+3. Duplicate .env.example and rename it to .env
+```commandline
+cp .env.example .env
+```
+
+4. Open .env and fill in the required environment variables.
+
+## Running the Application
+
+from within  the application you can run with the following command:
+
+```commandline
+uvicorn app.main:app --reload
+```
+
+You application will be available at the designated url
+
+## API Documentation
+
+FastAPI automatically generates interactive API documentation for your application. To access the docs, start your 
+server and visit <yourlocalurl>/docs in your browser. Alternatively, you can go to
+<yourlocalurl>/redoc to view the documentation in the ReDoc format.
+
+## Testing
+To run tests, run the following command from the root of the project directory:
+
+```commandline
+pytest
+```
+
+## Local Development
+During local development, you may need to generate and use a dummy JWT to 
+test protected endpoints of the application.
+
+# Generating a Dummy JWT
+
+FastAPI provides a convenient way to generate a dummy JWT for testing. 
+To generate a dummy JWT, follow the steps below:
+
+Make sure your application is running in a local environment. 
+The dummy token endpoint is only available in a local environment.
+
+While your application is running, visit the /dummy-token endpoint using a tool 
+like curl or any HTTP client like Postman.
+
+For instance, if your server is running locally on port 8000, you can use curl 
+to get a dummy token:
+
+```commandline
+curl http://localhost:8000/dummy-token
+```
+
+You will receive a response containing the dummy JWT
+
+```json
+{
+    "dummy_token": "<Your Dummy Token>"
+}
+```
+
+### Using the Dummy JWT
+Once you've obtained a dummy JWT, you can use it to make requests to 
+protected endpoints in your application:
+
+1. When making a request, include an Authorization header with the value Bearer 
+<Your Dummy Token>. Replace <Your Dummy Token> with the token you 
+received from the /dummy-token endpoint.
+
+2. Now you can make requests to the protected endpoints of the application.
+
+Remember, the dummy JWT is meant for testing purposes only and should not be 
+used in production environments. The /dummy-token endpoint is not available 
+in non-local environments.
+
+### Thoughts for authenticating the frontend with the backend
+To provide secure communication between your frontend Next.js application and your backend FastAPI service, you have several options. Here are a few popular approaches:
+
+- JWT (JSON Web Tokens): Since you're already using JWT for authentication in the frontend, you can also use this to authenticate requests to your FastAPI backend. This involves passing the JWT token in the Authorization header of the request from your frontend to the backend. Then, you can use a JWT decoder on the backend to validate the token. This can be done using libraries such as PyJWT in your FastAPI application.
+
+- API Keys: This is another common approach where you issue unique keys for each user/service that needs to access the backend API. Each API call then includes this key in the request header. FastAPI can easily validate these keys. While this approach is simpler than JWT, it provides less flexibility and security, as it doesn't allow for claims or scopes.
+
+- OAuth2.0: OAuth2 is a protocol that allows applications to request authorization to access resources on behalf of a user. FastAPI has direct support for OAuth2 using the OAuth2PasswordBearer class, which can be used for issuing access tokens to clients. Note that this could be overkill if you're already using JWT and the calls to your backend are not on behalf of a user.
+
+- Mutual TLS (mTLS): Mutual TLS is a method of two-way communication encryption where both client and server authenticate each other. This can be more complex to setup but can provide an additional layer of security in some scenarios.
+
+No matter which method we choose, you should always serve your applications over HTTPS to ensure that all data, including tokens or keys, is encrypted during transmission.
+Also, ensure that you handle the JWT tokens carefully, especially if they are stored in the client's browser, as they could be vulnerable to Cross-Site Scripting (XSS) or Cross-Site Request Forgery (CSRF) attacks. Consider httpOnly cookies for storing tokens if your use case allows it.
+
+### I think that we could use both JWT + API Key.
+
+# Notes:
+Using both JWT and API keys can provide an additional layer of security and 
+could be a good approach for our requirements.
+
+1. JSON Web Tokens (JWT) are useful for carrying user context between services. With JWT, you can embed user-specific data (like user ID, roles, permissions, etc.) in a secure, tamper-proof token. This can be validated by your FastAPI backend to authenticate and authorize the user.
+2. 
+3. API Keys can serve as an identifier for the client application (in this case, your Next.js frontend). It can provide a straightforward way to track and control how the client application is calling the backend API.
+
+Here's a rough workflow of how these can be used together:
+
+A user logs in to the Next.js frontend using NextAuth and receives a JWT.
+This JWT is stored securely in the client's browser.
+For each request from the frontend to the backend, the JWT is included in the Authorization header.
+In addition to the JWT, an API key unique to the frontend application is included in each request (possibly in a custom header like X-API-Key).
+The backend service validates both the JWT (for user authentication and authorization) and the API key (for client application validation).
+This approach provides a double check for each request:
+
+The JWT verifies that the request comes from a legitimate, authenticated user.
+The API key verifies that the request comes from a trusted client application.

backend/app/config.py

@@ -0,0 +1,18 @@
+from functools import lru_cache
+from pydantic import BaseSettings
+
+
+class Settings(BaseSettings):
+    API_KEY: str
+    API_KEY_NAME: str = "X-API-KEY"
+    SECRET_KEY: str
+    ALGORITHM: str
+    ENVIRONMENT: str
+
+    class Config:
+        env_file = ".env"
+
+
+@lru_cache()
+def get_settings():
+    return Settings()

backend/app/dependencies.py

@@ -0,0 +1,58 @@
+from fastapi import Depends, HTTPException, status, Request
+from fastapi.security import APIKeyHeader, OAuth2PasswordBearer
+from jose import jwt, JWTError
+from app.config import get_settings
+
+
+api_key_header = APIKeyHeader(name=get_settings().API_KEY_NAME, auto_error=False)
+oauth2_scheme = OAuth2PasswordBearer(tokenUrl="token")
+
+
+async def validate_api_key(api_key_header: str = Depends(api_key_header)):
+    if api_key_header != get_settings().API_KEY:
+        raise HTTPException(
+            status_code=status.HTTP_403_FORBIDDEN, detail="Could not validate credentials"
+        )
+    return api_key_header
+
+
+def get_user(user_id: str):
+    # Define here how to fetch a user from your database
+    # using the user_id. Here's a simple placeholder implementation:
+    # TODO: Update this function to fetch a user from your actual database
+    if get_settings().ENVIRONMENT == "local":
+        return {"id": user_id, "name": "Dummy User"}
+    else:
+        user = None
+        if user_id == "known_id":
+            user = {"id": user_id, "name": "Known User"}
+        return user
+
+
+def validate_jwt_token(token: str = Depends(oauth2_scheme)):
+    credentials_exception = HTTPException(
+        status_code=status.HTTP_401_UNAUTHORIZED,
+        detail="Could not validate credentials",
+        headers={"WWW-Authenticate": "Bearer"},
+    )
+    try:
+        # The SECRET_KEY should match the NEXTAUTH_SECRET in the front end
+        payload = jwt.decode(token, get_settings().SECRET_KEY, algorithms=[get_settings().ALGORITHM])
+        user_id: str = payload.get("sub")
+        if user_id is None:
+            raise credentials_exception
+        user = get_user(user_id=user_id)
+        if user is None:
+            raise credentials_exception
+        return user
+    except JWTError:
+        raise credentials_exception
+
+
+async def validate_token(token: str = Depends(oauth2_scheme), request: Request = None):
+    token_data = validate_jwt_token(token)
+    if not token_data:
+        raise HTTPException(
+            status_code=status.HTTP_403_FORBIDDEN, detail="Could not validate credentials"
+        )
+    return token

backend/app/local/router.py

@@ -0,0 +1,28 @@
+from fastapi import APIRouter, HTTPException, status
+from jose import jwt
+import datetime
+from app.config import get_settings
+
+router = APIRouter(
+    prefix="/local",
+    tags=["local"],
+)
+
+
+def create_dummy_token(secret: str, algorithm: str):
+    data = {
+        "sub": "known_id",
+        "name": "Test User",
+        "iat": datetime.datetime.utcnow(),
+        "exp": datetime.datetime.utcnow() + datetime.timedelta(minutes=30)
+    }
+    return jwt.encode(data, secret, algorithm=algorithm)
+
+
+@router.get("/dummy-token")
+async def dummy_token():
+    settings = get_settings()
+    if settings.ENVIRONMENT != "local":
+        raise HTTPException(status_code=status.HTTP_403_FORBIDDEN,
+                            detail="Dummy token can only be generated in local environment")
+    return {"dummy_token": create_dummy_token(settings.SECRET_KEY, settings.ALGORITHM)}

backend/app/main.py

@@ -0,0 +1,17 @@
+from fastapi import FastAPI, Depends
+from mangum import Mangum
+from app.portfolio import router as portfolio_router
+from app.dependencies import validate_api_key
+from app.config import get_settings
+
+
+app = FastAPI(dependencies=[Depends(validate_api_key)])
+
+
+app.include_router(portfolio_router.router)
+
+if get_settings().ENVIRONMENT == "local":
+    from app.local import router as local_router
+    app.include_router(local_router.router)
+
+handler = Mangum(app)

backend/app/portfolio/router.py

@@ -0,0 +1,19 @@
+from fastapi import APIRouter, Depends
+from app.dependencies import validate_token
+
+router = APIRouter(
+    prefix="/portfolio",
+    tags=["portfolio"],
+    dependencies=[Depends(validate_token)],
+    responses={404: {"description": "Not found"}}
+)
+
+
+@router.get("/{portfolio_id}")
+async def get_portfolio(portfolio_id: int):
+    return {
+        "portfolio_id": portfolio_id,
+        "name": "My Portfolio",
+        "description": "This is my portfolio",
+        "data": "some data"
+    }

backend/docker/Dockerfile

@@ -0,0 +1,23 @@
+# Pull base image
+FROM python:3.10.12-slim-buster
+
+# Set environment variables
+ENV PYTHONDONTWRITEBYTECODE 1
+ENV PYTHONUNBUFFERED 1
+
+# Set work directory
+WORKDIR /app
+
+# Install system dependencies
+RUN apt-get update && apt-get install -y netcat-openbsd
+
+# Install python dependencies
+COPY ./requirements/base.txt ./requirements/base.txt
+RUN pip install --upgrade pip
+RUN pip install -r requirements/base.txt
+
+# Copy project
+COPY . .
+
+# command to run on container start
+CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

backend/package.json

@@ -0,0 +1,6 @@
+{
+  "devDependencies": {
+    "serverless-offline": "^12.0.4",
+    "serverless-python-requirements": "^6.0.0"
+  }
+}

backend/requirements/base.txt

@@ -0,0 +1,27 @@
+anyio==3.7.1
+cffi==1.15.1
+click==8.1.3
+cryptography==41.0.1
+ecdsa==0.18.0
+exceptiongroup==1.1.2
+fastapi==0.99.1
+h11==0.14.0
+httptools==0.5.0
+idna==3.4
+mangum==0.17.0
+pyasn1==0.5.0
+pycparser==2.21
+pydantic==1.10.11
+PyJWT==2.7.0
+python-dotenv==1.0.0
+python-jose==3.3.0
+PyYAML==6.0
+rsa==4.9
+six==1.16.0
+sniffio==1.3.0
+starlette==0.27.0
+typing_extensions==4.7.1
+uvicorn==0.22.0
+uvloop==0.17.0
+watchfiles==0.19.0
+websockets==11.0.3

backend/serverless.yml

@@ -0,0 +1,43 @@
+service: fastapi-lambda
+
+provider:
+  name: aws
+  runtime: python3.10
+  region: eu-west-2
+  environment:
+    API_KEY: ${env:API_KEY}
+#    ENVIRONMENT: ${self:provider.stage}
+    ENVIRONMENT: 'local'
+    SECRET_KEY: ${env:SECRET_KEY}
+    ALGORITHM: ${env:ALGORITHM}
+
+package:
+  individually: true
+  include:
+    - Model/backend/**
+
+plugins:
+  - serverless-python-requirements
+  - serverless-offline
+
+custom:
+  pythonRequirements:
+    dockerizePip: true
+    dockerFile: docker/Dockerfile
+    useDocker: true
+    dockerSsh: true
+    fileName: requirements/base.txt
+
+functions:
+  app:
+    handler: app.main.handler
+    events:
+      - http:
+          path: /{proxy+}
+          method: ANY
+#    vpc:
+#      securityGroupIds:
+#        - sg-0abcd1234efgh5678
+#      subnetIds:
+#        - subnet-a1b2c3d4
+#        - subnet-e5f6g7h8