The Stack

Data Engineering and AI/LLM technologies I use to build production-ready systems

Data Engineering

Data Orchestration

Apache Airflow Dagster Prefect Kubernetes Argo Workflows

Data Processing

Python Pandas Polars Apache Spark Ray Dask

Cloud Infrastructure

AWS GCP Azure Docker Kubernetes Terraform

Model Evaluation & ETL

MLflow Weights & Biases Great Expectations dbt SQLGlot

AI & LLM Ops

LLM Ops

LangChain LlamaIndex OpenAI Ollama vLLM

Vector Databases

Pinecone Weaviate Chroma FAISS Milvus Qdrant

ML Frameworks

PyTorch TensorFlow scikit-learn XGBoost LightGBM

Embeddings & Inference

Sentence Transformers Hugging Face OpenAI Embeddings Cohere Together AI

AI & Data Engineering Projects

Production systems built for scalability, performance, and real-world impact

RAG Pipeline Architecture
RAG

Enterprise RAG Pipeline

LangChain Pinecone OpenAI AWS

Built a production-ready Retrieval-Augmented Generation system processing 50K+ documents with hybrid search capabilities, achieving 94% retrieval accuracy and sub-200ms response times.

Vector Search | Embedding Store | LLM Orchestration
LLM Fine-tuning Infrastructure
LLM Ops

LLM Fine-tuning Platform

Hugging Face PyTorch MLflow Kubernetes

Developed an end-to-end fine-tuning pipeline for domain-specific LLMs, reducing model inference costs by 60% while improving domain accuracy by 35% compared to base models.

Distributed Training | LoRA Adapters | Model Registry
Real-time Data Pipeline
Streaming

Real-time ML Inference Pipeline

Apache Kafka Flink Redis Docker

Architected a real-time ML inference pipeline processing 100K+ events per second with end-to-end latency under 100ms, featuring automatic model rollback and A/B testing capabilities.

Event Streaming | Stream Processing | Low-Latency Inference
Vector Database Architecture
Vector DB

Scalable Vector Search System

Weaviate Python Docker GCP

Implemented a distributed vector database cluster handling 10M+ embeddings with 99.99% availability, featuring automatic sharding, replication, and intelligent caching strategies.

Distributed Cluster | HNSW Index | Multi-Replica
Data Lake Optimization
Optimization

Data Lake Performance Optimization

AWS S3 Glue Athena Python

Optimized enterprise data lake reducing query latency by 75% and storage costs by 40% through partitioning strategies, file format optimization (Parquet), and intelligent data lifecycle management.

Columnar Storage | Partition Pruning | Lifecycle Policies
ML Experiment Tracking Platform
MLOps

ML Experiment Tracking Platform

Weights & Biases MLflow Airflow FastAPI

Built a comprehensive experiment tracking and model registry system, enabling data scientists to compare 1000+ experiments, deploy models with one click, and monitor production performance metrics.

Experiment Tracking | Model Registry | CI/CD Deployment

Experience

My professional journey in data engineering and AI

2024 - Present

Senior AI Engineer

ComplianceLab X

Building and shipping AI systems used in real production - from early prototypes to scalable, monitored services. Designed and deployed end-to-end AI pipelines (data ingestion -> modeling -> inference -> monitoring) with a strong focus on reliability and performance. Built LLM-powered features including RAG systems, agents, and internal tools that automate workflows and improve decision-making.

Python LLM Ops RAG AWS
2022 - Present

Data Developer

BetterCollective

Driving data engineering projects focused on scalability and performance optimization. Implementing data pipelines and optimizing infrastructure for production workloads.

Python LLM Ops GCP Airflow
2024 - 2025

Data Engineer

index.dev

Built scalable data infrastructure and ETL pipelines. Implemented data quality frameworks and optimized warehouse performance for real-time analytics platforms.

Airflow AWS Python dbt
2020 - Present

Freelance Data Engineer

Upwork

Delivering data engineering and AI solutions for clients worldwide. Specializing in ETL pipelines, data warehousing, RAG systems, and ML infrastructure. Maintained 100% job success score across 50+ completed projects.

ETL RAG LLM Ops Python

Let's Build Something Together

Available for consulting, freelance projects, and full-time opportunities

Email

ldavid797@gmail.com

LinkedIn

Connect with me

GitHub

Check out my code

Upwork

Hire me on Upwork

Ready to scale your AI infrastructure?

I specialize in building production-ready data pipelines, RAG systems, and LLMOps platforms that scale.

Start a Conversation Download Full CV