Hugging Face's logo

Spaces:
can-org
/
canspace
Sleeping

App Files Community
canspace / Machine-learning /README.md
Pujan Neupane
Project : pushing all the files to hugging face
e9f0d54
preview code
|
raw
history blame contribute delete
7.81 kB

FastAPI AI

This FastAPI app loads a GPT-2 model, tokenizes input text, classifies it, and returns whether the text is AI-generated or human-written.

install Dependencies 

pip install -r requirements.txt

This command installs all the dependencies listed in the requirements.txt file. It ensures that your environment has the required packages to run the project smoothly.

NOTE: IF YOU HAVE DONE ANY CHANGES DON'NT FORGOT TO PUT IT IN THE REQUIREMENTS.TXT USING bash pip freeze > requirements.txt

Functions

  1. load_model()
    Loads the GPT-2 model and tokenizer from specified paths.

  2. lifespan()
    Manages the app's lifecycle: loads the model at startup and handles cleanup on shutdown.

  3. classify_text_sync()
    Synchronously tokenizes input text and classifies it using the GPT-2 model. Returns the classification and perplexity.

  4. classify_text()
    Asynchronously executes classify_text_sync() in a thread pool to ensure non-blocking processing.

  5. analyze_text()
    POST endpoint: accepts text input, classifies it using classify_text(), and returns the result with perplexity.

  6. health_check()
    GET endpoint: simple health check to confirm the API is running.

Code Overview 

executor = ThreadPoolExecutor(max_workers=2)
  • ThreadPoolExecutor(max_workers=2) limits the number of concurrent threads (tasks) per worker process to 2 for text classification. This helps control resource usage and prevent overloading the server.

Running and Load Balancing:

To run the app in production with load balancing:

uvicorn app:app --host 0.0.0.0 --port 8000 --workers 4

This command launches the FastAPI app with 4 worker processes, allowing it to handle multiple requests concurrently.

Concurrency Explained:

  1. ThreadPoolExecutor(max_workers=20)

    • Controls the number of threads within a single worker process.
    • Allows up to 20 tasks (text classification requests) to be handled simultaneously per worker, improving responsiveness for I/O-bound tasks.

  2. --workers 4 in Uvicorn

    • Spawns 4 independent worker processes to handle incoming HTTP requests.
    • Each worker can independently handle multiple tasks, increasing the app's ability to process concurrent requests in parallel.

How They Relate:

  • Uvicorn’s --workers defines how many worker processes the server will run.
  • ThreadPoolExecutor limits how many tasks (threads) each worker can process concurrently.

For example, with 4 workers and 20 threads per worker, the server can handle 80 tasks concurrently. This provides scalable and efficient processing, balancing the load across multiple workers and threads.

Endpoints

1. /analyze

  • Method: POST
  • Description: Classifies whether the text is AI-generated or human-written.
  • Request:
    { "text": "sample text" }
  • Response:
    { "result": "AI-generated", "perplexity": 55.67 }

2. /health

  • Method: GET
  • Description: Returns the status of the API.
  • Response:
    { "status": "ok" }

Running the API

Start the server with:

uvicorn app:app --host 0.0.0.0 --port 8000 --workers 4

🧪 Testing the API

You can test the FastAPI endpoint using curl like this:

curl -X POST http://127.0.0.1:8000/analyze \
  -H "Authorization: Bearer HelloThere" \
  -H "Content-Type: application/json" \
  -d '{"text": "This is a sample sentence for analysis."}'
  • The -H "Authorization: Bearer HelloThere" part is used to simulate the handshake.
  • FastAPI checks this token against the one loaded from the .env file.
  • If the token matches, the request is accepted and processed.
  • Otherwise, it responds with a 403 Unauthorized error.

API Documentation

  • Swagger UI: http://127.0.0.1:8000/docs -> /docs
  • ReDoc: http://127.0.0.1:8000/redoc -> /redoc

🔐 Handshake Mechanism

In this part, we're implementing a simple handshake to verify that the request is coming from a trusted source (e.g., our NestJS server). Here's how it works:

  • We load a secret token from the .env file.
  • When a request is made to the FastAPI server, we extract the Authorization header and compare it with our expected secret token.
  • If the token does not match, we immediately return a 403 Forbidden response with the message "Unauthorized".
  • If the token does match, we allow the request to proceed to the next step.

The verification function looks like this:

def verify_token(auth: str):
    if auth != f"Bearer {EXPECTED_TOKEN}":
        raise HTTPException(status_code=403, detail="Unauthorized")

This provides a basic but effective layer of security to prevent unauthorized access to the API.

Implement it with NEST.js

NOTE: Make an micro service in NEST.JS and implement it there and call it from app.controller.ts

in fastapi.service.ts file what we have done is

Project Structure 

nestjs-fastapi-bridge/
├── src/
│   ├── app.controller.ts
│   ├── app.module.ts
│   └── fastapi.service.ts
├── .env

Step-by-Step Setup

1. .env

Create a .env file at the root with the following:

  FASTAPI_BASE_URL=http://localhost:8000
  SECRET_TOKEN="HelloThere"

2. fastapi.service.ts 

  // src/fastapi.service.ts
  import { Injectable } from "@nestjs/common";
  import { HttpService } from "@nestjs/axios";
  import { ConfigService } from "@nestjs/config";
  import { firstValueFrom } from "rxjs";

  @Injectable()
  export class FastAPIService {
    constructor(
      private http: HttpService,
      private config: ConfigService,
    ) {}

    async analyzeText(text: string) {
      const url = `${this.config.get("FASTAPI_BASE_URL")}/analyze`;
      const token = this.config.get("SECRET_TOKEN");

      const response = await firstValueFrom(
        this.http.post(
          url,
          { text },
          {
            headers: {
              Authorization: `Bearer ${token}`,
            },
          },
        ),
      );

      return response.data;
    }
  }

3. app.module.ts 

// src/app.module.ts
import { Module } from "@nestjs/common";
import { ConfigModule } from "@nestjs/config";
import { HttpModule } from "@nestjs/axios";
import { AppController } from "./app.controller";
import { FastAPIService } from "./fastapi.service";

@Module({
  imports: [ConfigModule.forRoot(), HttpModule],
  controllers: [AppController],
  providers: [FastAPIService],
})
export class AppModule {}

4. app.controller.ts 

  // src/app.controller.ts
  import { Body, Controller, Post, Get, Query } from '@nestjs/common';
  import { FastAPIService } from './fastapi.service';

  @Controller()
  export class AppController {
    constructor(private readonly fastapiService: FastAPIService) {}

    @Post('analyze-text')
    async callFastAPI(@Body('text') text: string) {
      return this.fastapiService.analyzeText(text);
    }

    @Get()
    getHello(): string {
      return 'NestJS is connected to FastAPI ';
    }
  }

🚀 How to Run

Run the server of flask and nest.js:

  • for nest.js

    npm run start

  • for Fastapi

    uvicorn app:app --reload

Make sure your FastAPI service is running at http://localhost:8000.

Test with CURL 

curl -X POST http://localhost:3000/analyze-text \
  -H 'Content-Type: application/json' \
  -d '{"text": "This is a test input"}'