MLX RAG

Retrieval Augmented Generation (RAG) example using Langchain (loader), Chroma (vector database), Hugging Face sentence-transformers (embeddings) and MLX (LLM).

This example code was tested with these models:

• mlx-community/gemma-3n-E2B-it-lm-4bit
• mlx-community/gemma-2-9b-it-4bit
• mlx-community/Phi-3-mini-128k-instruct-4bit
• mlx-community/WizardLM-2-7B-4bit
• mlx-community/gemma-1.1-7b-it-4bit
• mlx-community/Mistral-7B-Instruct-v0.2-4bit
• mlx-community/Mixtral-8x7B-Instruct-v0.1-hf-4bit-mlx

Usage

pip install -r requirements.txt
python main.py

Example document

• Chemical strategies to mitigate electrostatic charging during coffee grinding (Access Paper -> View PDF)