Skip to main content

Refactor Explanation

This document explains the current refactored design of the audio-anomaly-detection repo, why each module exists, how inference flows through the system, and exactly how to add new models safely.

1) Refactor Goal​

The original API draft had endpoint logic, model loading, feature extraction wiring, and inference behavior in one place. The refactor moved to a modular layout so that:

  • API endpoints stay thin and HTTP-focused.
  • Model selection is dynamic and config-driven.
  • New models can be onboarded mostly via configuration.
  • Core inference logic is reusable and testable.

2) Current API Contract​

Endpoints​

  • GET /
    • Health/root message.
  • GET /models
    • Returns available model names and default model.
  • POST /predict-anomaly
    • Accepts file upload (.wav, .mp3, .flac, .ogg) and optional model_name query param.

Prediction behavior​

  • If model_name is provided, API tries that model.
  • If omitted, API uses DEFAULT_MODEL_NAME from env.
  • If model is unknown, API returns HTTP 400.

3) Project Structure and Responsibilities​

main.py
app/
  main.py
  config.py
  dependencies.py
  schemas.py
  api/
    routes.py
  services/
    errors.py
    inference_service.py
    registry.py
    pipeline.py
features/
  feature_extractor.py
tests/
  test_registry.py
  test_api_routes.py

main.py​

Compatibility run entrypoint. Imports app from app.main and runs uvicorn.

app/main.py​

Application factory and composition root.

  • Configures logging.
  • Loads settings.
  • Builds ModelRegistry and InferenceService.
  • Stores service objects in app.state.

app/config.py​

All environment parsing and config validation.

  • ModelConfig: per-model config object.
  • AppSettings: top-level app settings.
  • Supports two config modes:
    1. MODELS_CONFIG_JSON (multi-model)
    2. Legacy single-model env fallback
  • Resolves relative artifact paths to absolute paths.
  • Ensures default model is present in configured model set.

app/dependencies.py​

FastAPI DI adapter to retrieve InferenceService from request.app.state.

app/schemas.py​

Response schema contracts.

  • PredictionResponse
  • ModelsResponse

app/api/routes.py​

HTTP-facing layer only.

  • Validates request shape/file extension.
  • Calls service layer.
  • Maps service/domain exceptions to HTTP status codes.
  • Keeps endpoint logic minimal.

app/services/errors.py​

Domain-level exceptions.

  • UnknownModelError
  • InferenceFailureError

app/services/registry.py​

Model resolution + lazy pipeline cache.

  • Decides which model to use.
  • Validates requested model name.
  • Builds pipeline only on first request per model.
  • Uses lock for thread-safe lazy initialization.

app/services/pipeline.py​

Concrete model runtime pipeline (SklearnModelPipeline).

  • Feature extraction
  • Scaler transform
  • predict_proba
  • label mapping
  • response payload assembly

app/services/inference_service.py​

Inference orchestration from bytes to result.

  • Resolves model.
  • Decodes audio with librosa.
  • Executes model pipeline.
  • Adds latency metric.

features/feature_extractor.py​

Feature engineering logic retained from existing implementation.

  • Silence trim
  • pre-emphasis
  • MFCC/formants/spectral features
  • optional dropped-feature filtering

4) End-to-End Request Flow​

  1. Client sends POST /predict-anomaly with multipart audio file and optional model_name.
  2. Route validates extension.
  3. Route reads bytes and calls InferenceService.predict_from_bytes(...).
  4. Service resolves model name through ModelRegistry.
  5. Registry returns cached pipeline or builds one lazily.
  6. Service decodes audio bytes into mono waveform at configured sample rate.
  7. Pipeline extracts features, scales them, runs classifier probabilities.
  8. Pipeline returns structured prediction payload.
  9. Service adds latency_ms and returns response.
  10. Route serializes to PredictionResponse.

5) Model Configuration​

Core env keys​

  • API_TITLE
  • API_VERSION
  • SAMPLE_RATE
  • DEFAULT_MODEL_NAME
  • MODELS_CONFIG_PATH (preferred)
  • MODELS_CONFIG_JSON

Legacy fallback env keys (when JSON is not set)​

  • MODEL_PATH
  • SCALER_PATH
  • LABEL_ENCODER_PATH
  • DROPPED_FEATURES_PATH

MODELS_CONFIG_PATH (preferred)​

Point this to a readable JSON file (for example: config/models.json).

JSON model config shape​

Whether read from file or env string, model config is a JSON object keyed by model name. Each value should include:

  • model_path
  • scaler_path
  • label_encoder_path (optional)
  • dropped_features_path (optional)
  • feature_extractor (currently supports default)
  • predictor (currently supports sklearn_proba)

6) How to Add a New Model (No Code Changes)​

Use this when new model follows current sklearn probability flow.

  1. Add new artifacts (model/scaler/optional encoder/optional dropped-features) under models/.
  2. Add model entry in config/models.json.
  3. Ensure feature_extractor is default and predictor is sklearn_proba.
  4. Call API with ?model_name=<new_model_name>.
  5. Verify visibility via GET /models.

7) When Code Changes Are Required for New Models​

Case A: Different feature extraction​

Expected updates:

  1. Add new extractor implementation.
  2. Add branch in ModelRegistry._build_pipeline(...) by feature_extractor value.
  3. Extend tests for that branch.

Case B: Different preprocessing/transform​

Expected updates:

  1. Add new transform logic in pipeline.
  2. Add config switch and registry branch.
  3. Add tests for transform output shape/behavior.

Case C: Non-predict_proba predictor behavior​

Expected updates:

  1. Add new predictor execution path.
  2. Map output to stable API response schema.
  3. Add tests for probability and label mapping behavior.

8) Error Handling Strategy​

  • Unknown model_name -> UnknownModelError -> HTTP 400.
  • Extraction failure (e.g., too-short audio) -> InferenceFailureError -> HTTP 422.
  • Unsupported extension -> HTTP 400.
  • Unexpected internal exception -> HTTP 500.

This cleanly separates domain failures from transport-level responses.

9) Modularity Assessment (Current State)​

What is modular and healthy:

  • Clear boundary between HTTP (routes.py) and inference orchestration (inference_service.py).
  • Model loading concerns isolated in registry.py.
  • Concrete pipeline isolated in pipeline.py.
  • Config parsing and validation centralized in config.py.

What remains intentionally simple:

  • One concrete pipeline class (no unnecessary abstraction layers).
  • One prediction endpoint only (/predict-anomaly) to reduce API surface complexity.

10) Testing Coverage​

Current tests​

  • tests/test_registry.py
    • requested model resolution
    • fallback to default model
    • unknown model handling
  • tests/test_api_routes.py
    • model_name query routing behavior
    • unknown model returns 400
  • GET /models response test.
  • invalid extension returns 400 test.
  • default-model path test via API (no model_name passed).

11) Operational Notes​

Run app​

uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload

List models​

curl http://127.0.0.1:8000/models

Predict​

curl -X POST "http://127.0.0.1:8000/predict-anomaly?model_name=<model_name>" \
  -F "file=@/path/to/audio.wav"

Run tests​

python -m pytest -q

(Use python3 -m pytest -q if python is unavailable in your shell.)

12) Migration Notes​

  • utils/model_handler.py is legacy and not used by the new route path.
  • Refactored flow relies on app package modules under app/.
  • Existing clients should call /predict-anomaly only.

13) Summary​

The refactor now provides:

  • Dynamic multi-model selection via query + env default.
  • Lazy, cached model pipelines.
  • Cleaner modular boundaries.
  • Simpler, lean architecture (without over-engineered abstractions).
  • A clear path to extend behavior when model families diverge.