You can use the Example or Template
You simply need to
git clone https://github.com/Gresliebear/Mistral-Example.git
This should be used for example or good starting off point for your project.
https://github.com/Gresliebear/Mistral-Example
then you create .env file and Add your API key
create .env file
put this in the .env file
MISTRAL_API_KEY="XXXXXXXXXXXXXXX"
Mistral-Example
1. Create Mistral API
Example of using Mistral API and locally running the model
Create a account on Mistral https://auth.mistral.ai/ui/login
We will then go to Mistral API and generate a key you can just sign with google is the easiest way.
You will then name your organization
Then on the left hand side you click API
This new plan you will choose Plan
Then you will subscribe, verify through a phone number, and other requirements you should be taken back to this page
Building this from scratch
2. You will mistral API key for app.py to work
create .env file
put this in the .env file
MISTRAL_API_KEY="XXXXXXXXXXXXXXX"
3. create a virtual environment
python -m venv venv
enter your virtual environment
source venv/bin/activate (Linux/MacOS)
or
.\env\Scripts\activate (Windows)
3. Let’s write the basic flask app
import os
from io import BytesIO, StringIO
from dotenv import load_dotenv
from flask import Flask, request, jsonify
from mistralai import Mistral
import base64
import pandas as pd
import PyPDF2
load_dotenv()
app = Flask(__name__)
MISTRAL_API_KEY = os.environ["MISTRAL_API_KEY"]
@app.route("/uploadToLLM", methods=["POST"])
def upload():
return("Hello World")
if __name__ == "__main__":
app.run(host="0.0.0.0", port=5000, debug=True)run the app to test
python app.py
Test the endpoint
import requests;
r = requests.post("http://localhost:5000/uploadToLLM", json={"foo":"bar"})
print(r.status_code, r.text)4. Now lets add file processing
import os
from io import BytesIO, StringIO
from dotenv import load_dotenv
from flask import Flask, request, jsonify
from mistralai import Mistral
import base64
import pandas as pd
import PyPDF2
load_dotenv()
app = Flask(__name__)
MISTRAL_API_KEY = os.environ["MISTRAL_API_KEY"]
@app.route("/uploadToLLM", methods=["POST"])
def upload():
# This first block of code is when we past data to flask app endpoint
try:
# 1) get PDF bytes from either multipart or JSON-base64
try:
if "file" in request.files:
print(request.files)
pdf_bytes =request.files["file"].read()
else:
data = request.get_json(force=True)
b64 = data.get("file") or data.get("base64")
if not b64:
return jsonify({"error": "No file provided"}), 400
pdf_bytes = base64.b64decode(b64)
# 2) extract text
document_text = extract_pdf_text(pdf_bytes)
except Exception as e:
app.logger.exception("Upload failed")
return jsonify({"error": str(e)}), 500
if __name__ == "__main__":
app.run(host="0.0.0.0", port=5000, debug=True)5. Create prompt
import os
from io import BytesIO, StringIO
from dotenv import load_dotenv
from flask import Flask, request, jsonify
from mistralai import Mistral
import base64
import pandas as pd
import PyPDF2
load_dotenv()
app = Flask(__name__)
MISTRAL_API_KEY = os.environ["MISTRAL_API_KEY"]
@app.route("/uploadToLLM", methods=["POST"])
def upload():
# This first block of code is when we past data to flask app endpoint
try:
# 1) get PDF bytes from either multipart or JSON-base64
if "file" in request.files:
pdf_bytes = request.files["file"].read()
else:
data = request.get_json(force=True)
b64 = data.get("file") or data.get("base64")
if not b64:
return jsonify({"error": "No file provided"}), 400
pdf_bytes = base64.b64decode(b64)
# 2) extract text
document_text = extract_pdf_text(pdf_bytes)
# then we need to build a prompt to pass mistral
prompt = (
"Extract all the key facts and their numeric values from the document below. "
"Output only CSV with exactly two columns: Year, Invention, Fact, Value. "
"Include units with Value column"
"No headers, no commentary. "
"If you cannot parse it, reply exactly “failed to structure”.\n\n"
"-----DOCUMENT START-----\n"
f"{document_text}\n"
"-----DOCUMENT END-----"
)
if __name__ == "__main__":
app.run(host="0.0.0.0", port=5000, debug=True)6. Then lets pass the data and the prompt to mistral
import os
from io import BytesIO, StringIO
from dotenv import load_dotenv
from flask import Flask, request, jsonify
from mistralai import Mistral
import base64
import pandas as pd
import PyPDF2
load_dotenv()
app = Flask(__name__)
MISTRAL_API_KEY = os.environ["MISTRAL_API_KEY"]
@app.route("/uploadToLLM", methods=["POST"])
def upload():
# This first block of code is when we past data to flask app endpoint
try:
# 1) get PDF bytes from either multipart or JSON-base64
if "file" in request.files:
pdf_bytes = request.files["file"].read()
else:
data = request.get_json(force=True)
b64 = data.get("file") or data.get("base64")
if not b64:
return jsonify({"error": "No file provided"}), 400
pdf_bytes = base64.b64decode(b64)
# 2) extract text
document_text = extract_pdf_text(pdf_bytes)
# then we need to build a prompt to pass mistral
prompt = (
"Extract all the key facts and their numeric values from the document below. "
"Output only CSV with exactly two columns: Year, Invention, Fact, Value. "
"Include units with Value column"
"No headers, no commentary. "
"If you cannot parse it, reply exactly “failed to structure”.\n\n"
"-----DOCUMENT START-----\n"
f"{document_text}\n"
"-----DOCUMENT END-----"
)
# 4) call Mistral
client = Mistral(api_key=MISTRAL_API_KEY)
chat = client.chat.complete(
model="mistral-large-latest",
messages=[{"role": "user", "content": prompt}],
)
raw_csv = chat.choices[0].message.content
if __name__ == "__main__":
app.run(host="0.0.0.0", port=5000, debug=True)7. then lets clean our data response
import os
from io import BytesIO, StringIO
from dotenv import load_dotenv
from flask import Flask, request, jsonify
from mistralai import Mistral
import base64
import pandas as pd
import PyPDF2
load_dotenv()
app = Flask(__name__)
MISTRAL_API_KEY = os.environ["MISTRAL_API_KEY"]
@app.route("/uploadToLLM", methods=["POST"])
def upload():
# This first block of code is when we past data to flask app endpoint
try:
# 1) get PDF bytes from either multipart or JSON-base64
if "file" in request.files:
pdf_bytes = request.files["file"].read()
else:
data = request.get_json(force=True)
b64 = data.get("file") or data.get("base64")
if not b64:
return jsonify({"error": "No file provided"}), 400
pdf_bytes = base64.b64decode(b64)
# 2) extract text
document_text = extract_pdf_text(pdf_bytes)
# then we need to build a prompt to pass mistral
prompt = (
"Extract all the key facts and their numeric values from the document below. "
"Output only CSV with exactly two columns: Year, Invention, Fact, Value. "
"Include units with Value column"
"No headers, no commentary. "
"If you cannot parse it, reply exactly “failed to structure”.\n\n"
"-----DOCUMENT START-----\n"
f"{document_text}\n"
"-----DOCUMENT END-----"
)
# 4) call Mistral
client = Mistral(api_key=MISTRAL_API_KEY)
chat = client.chat.complete(
model="mistral-large-latest",
messages=[{"role": "user", "content": prompt}],
)
raw_csv = chat.choices[0].message.content
# 5) clean & parse CSV
cleaned = clean_csv_response(raw_csv)
df = pd.read_csv(
StringIO(cleaned),
names=["Year", "Invention", "Fact", "Value"],
engine="python", # more tolerant parser
skip_blank_lines=True,
skipinitialspace=True,
on_bad_lines="skip" # drop any row that doesn’t split into exactly 4 fields
)
# 6) write Excel directly to disk
output_path = "key_facts.xlsx"
with pd.ExcelWriter(output_path, engine="xlsxwriter") as writer:
df.to_excel(writer, index=False, sheet_name="KeyFacts")
if __name__ == "__main__":
app.run(host="0.0.0.0", port=5000, debug=True)8. Now let’s do some error handling
import os
from io import BytesIO, StringIO
from dotenv import load_dotenv
from flask import Flask, request, jsonify
from mistralai import Mistral
import base64
import pandas as pd
import PyPDF2
load_dotenv()
app = Flask(__name__)
MISTRAL_API_KEY = os.environ["MISTRAL_API_KEY"]
@app.route("/uploadToLLM", methods=["POST"])
def upload():
# This first block of code is when we past data to flask app endpoint
try:
# 1) get PDF bytes from either multipart or JSON-base64
if "file" in request.files:
pdf_bytes = request.files["file"].read()
else:
data = request.get_json(force=True)
b64 = data.get("file") or data.get("base64")
if not b64:
return jsonify({"error": "No file provided"}), 400
pdf_bytes = base64.b64decode(b64)
# 2) extract text
document_text = extract_pdf_text(pdf_bytes)
# then we need to build a prompt to pass mistral
prompt = (
"Extract all the key facts and their numeric values from the document below. "
"Output only CSV with exactly two columns: Year, Invention, Fact, Value. "
"Include units with Value column"
"No headers, no commentary. "
"If you cannot parse it, reply exactly “failed to structure”.\n\n"
"-----DOCUMENT START-----\n"
f"{document_text}\n"
"-----DOCUMENT END-----"
)
# 4) call Mistral
client = Mistral(api_key=MISTRAL_API_KEY)
chat = client.chat.complete(
model="mistral-large-latest",
messages=[{"role": "user", "content": prompt}],
)
raw_csv = chat.choices[0].message.content
# 5) clean & parse CSV
cleaned = clean_csv_response(raw_csv)
df = pd.read_csv(
StringIO(cleaned),
names=["Year", "Invention", "Fact", "Value"],
engine="python", # more tolerant parser
skip_blank_lines=True,
skipinitialspace=True,
on_bad_lines="skip" # drop any row that doesn’t split into exactly 4 fields
)
# 6) write Excel directly to disk
output_path = "key_facts.xlsx"
with pd.ExcelWriter(output_path, engine="xlsxwriter") as writer:
df.to_excel(writer, index=False, sheet_name="KeyFacts")
# 7) report success
return jsonify({
"message": "File saved successfully",
"path": output_path
}), 200
except Exception as e:
app.logger.exception("Restructure failed")
return jsonify({"error": str(e)}), 500
if __name__ == "__main__":
app.run(host="0.0.0.0", port=5000, debug=True)Now let’s make sure we have the functions
def extract_pdf_text(pdf_bytes: bytes) -> str:
reader = PyPDF2.PdfReader(BytesIO(pdf_bytes))
return "\n".join(page.extract_text() or "" for page in reader.pages)
def clean_csv_response(resp: str) -> str:
lines = [line.strip().strip('"') for line in resp.splitlines() if line.strip()]
return "\n".join(lines)Now let’s make script to upload our PDF to flask app
#!/usr/bin/env python3
import requests
import base64
import os
BASE_URL = "http://127.0.0.1:5000"
def post_pdf_base64(filepath):
"""Send the PDF base64-encoded inside JSON (Flask: request.json['file'])."""
with open(filepath, "rb") as f:
b64 = base64.b64encode(f.read()).decode("utf-8")
payload = {
"filename": os.path.basename(filepath),
"file": b64
}
resp = requests.post(f"{BASE_URL}/uploadToLLM", json=payload)
resp.raise_for_status()
print("application/json:", resp.status_code, resp.json())
if __name__ == "__main__":
pdf_path = r"E:\MistralAI Exv2\AAR-Chronology-Americas-Freight-Railroads-Fact-Sheet.pdf"
# Option B: JSON + base64
post_pdf_base64(pdf_path)
Run the test script to past data flask api
python test_script.py
This should be able to give you good prototype test concept out
Local usage
you need to use Huggingface API key to download and access model weights
Huggingface tracks who is using the model they want maintain accountability for submissions and allow the community to identify the authors of models
Some of these models are not open source and require a license to use so always check the license before using a model
create a https://huggingface.co/join and create API key
HUGGINGFACE_TOKEN = os.getenv("HUGGINGFACE_TOKEN")
All you need to do is find model you want and put its model tag below in gpu_app.py
model_name = "mistralai/Mistral-7B-v0.3"
Hardware requirements
Always check the model requirements for hardware it won’t work if don’t have enough memory or VRAM
7 B-parameter models (Mistral 7B, Mamba, Mathstral, Nemo, Small):
FP16 ≈ 14 GB VRAM
4-bit ≈ 3.5 GB VRAM
GPU: ≥ 12 GB (e.g. RTX 3060 12 GB); ≥ 16 GB for headroom
Mixtral 8×7 B (MoE, ≈45 B params):
FP16 ≈ 90 GB VRAM
4-bit ≈ 22.5 GB VRAM
Setup: A100 80 GB (or offload)