Real-Time Platform & Multi-Agent AI for Predictive Maintenance
End-to-end real-time data platform and multi-agent AI system supervising an acid cooler in a sulfuric acid production unit: streaming ingestion, medallion lakehouse, online anomaly detection with drift-triggered retraining, and a LangGraph agent team answering operators in natural language.
- Streaming ingestion chain (Apache Kafka → TimescaleDB) with Pydantic contract validation, physical-range checks, quality coding (good/suspect/bad) and a dead-letter table; medallion architecture (Bronze/Silver/Gold) over TimescaleDB and a MinIO/Parquet data lake, orchestrated with Dagster (Polars batch jobs, Spark Structured Streaming aggregates); automated per-tag data quality controls for completeness, ranges, staleness and outages.
- Online anomaly detection with River (Half-Space Trees) scoring each sensor stream in real time; offline Isolation Forest baseline benchmarked in MLflow, with the online model versioned through a champion alias so rollback needs no rebuild or redeployment; concept-drift monitoring (ADWIN) with a Dagster sensor auto-triggering retraining.
- LangGraph supervisor exposed through FastAPI, orchestrating three specialized agents: an analyst turning natural language into read-only SQL with allowlist validation, a documentalist running RAG over technical documentation and incident history with Qdrant and locally computed embeddings, and a diagnostician correlating anomalies, drift, data and docs.
- Observability with Langfuse (traces, costs, evaluations) over a provider-agnostic LLM layer (Anthropic, Groq, Ollama) and a deterministic evaluation set replayed in CI; React/TypeScript web app with real-time dashboards, alerting and a conversational interface, packaged as a single Docker Compose stack deployed on a VPS.