Module 13 of 16
Module 12 • Hands-On Engineering Lab
Hands-On Generative AI: Production Code Lab
Put theory into practice. This practical laboratory guides you through building four core production generative workflows: local offline execution with Ollama, real-time token streaming, multimodal visual analysis, and persistent vector storage with ChromaDB.
Lab 1: Local Zero-Cost Inference with Ollama & Python
Running models locally gives you 100% data privacy, zero recurring cloud API bills, and offline reliability. Ollama packages model weights (Llama 3, Mistral, DeepSeek-R1) and serves an OpenAI-compatible REST API:
Terminal Setup
# Download and run Llama 3 locally
curl -fsSL https://ollama.com/install.sh | sh
ollama run llama3:8b
curl -fsSL https://ollama.com/install.sh | sh
ollama run llama3:8b
import os
from openai import OpenAI
# Connect to local Ollama server running on port 11434
client = OpenAI(
base_url="http://localhost:11434/v1",
api_key="ollama" # API key not needed for local server
)
response = client.chat.completions.create(
model="llama3:8b",
messages=[
{"role": "system", "content": "You are an expert Python performance engineer."},
{"role": "user", "content": "How does Python 3.12's specialized adaptive interpreter optimize bytecode execution?"}
],
temperature=0.2
)
print(response.choices[0].message.content)Lab 2: Real-Time Token Streaming (Minimizing TTFT)
Waiting 10 seconds for a full 500-token paragraph destroys user experience.Streaming uses Server-Sent Events (SSE) to deliver tokens the instant they are generated:
import time
from openai import OpenAI
client = OpenAI()
start_time = time.time()
first_token_time = None
stream = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "user", "content": "Write a concise explanation of how FlashAttention speeds up Transformer inference."}
],
stream=True
)
print("--- Streaming Output ---")
for chunk in stream:
token = chunk.choices[0].delta.content or ""
if token and first_token_time is None:
first_token_time = time.time()
ttft = (first_token_time - start_time) * 1000
print(f"\n[Time to First Token (TTFT): {ttft:.1f} ms]\n")
print(token, end="", flush=True)
total_duration = time.time() - start_time
print(f"\n\n[Generation Completed in {total_duration:.2f} seconds]")Lab 3: Multimodal Vision & Chart Inspection
Vision-Language Models allow passing raw base64-encoded images or URLs to extract structured tabular data from complex diagrams:
import base64
from openai import OpenAI
client = OpenAI()
def encode_image(image_path: str) -> str:
with open(image_path, "rb") as image_file:
return base64.b64encode(image_file.read()).decode("utf-8")
# Inspect a financial candlestick or architecture diagram
base64_image = encode_image("architecture_diagram.png")
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": "Identify all single points of failure in this cloud architecture diagram and provide a bulleted remediation list."},
{
"type": "image_url",
"image_url": {"url": f"data:image/png;base64,{base64_image}"}
}
]
}
],
max_tokens=600
)
print("Architectural Analysis:\n", response.choices[0].message.content)Lab 4: Persistent Vector Storage with ChromaDB
Chroma is an open-source, lightweight embedding database designed for rapid RAG development:
import chromadb
from chromadb.utils import embedding_functions
# 1. Initialize persistent ChromaDB instance
chroma_client = chromadb.PersistentClient(path="./my_chroma_db")
# 2. Use OpenAI text-embedding-3-small as the vectorizer
openai_ef = embedding_functions.OpenAIEmbeddingFunction(
model_name="text-embedding-3-small"
)
collection = chroma_client.get_or_create_collection(
name="developer_docs",
embedding_function=openai_ef,
metadata={"hnsw:space": "cosine"}
)
# 3. Insert technical documentation records
collection.upsert(
documents=[
"Next.js utilizes static site generation (SSG) with getStaticProps for high performance.",
"PostgreSQL pgvector provides HNSW index support for high-throughput semantic search.",
"Docker containerization packages applications with their exact runtime dependencies."
],
metadatas=[{"category": "frontend"}, {"category": "database"}, {"category": "devops"}],
ids=["doc_01", "doc_02", "doc_03"]
)
# 4. Execute similarity query
results = collection.query(
query_texts=["How do I optimize vector lookups in relational databases?"],
n_results=1
)
print("Top Retrieved Document:", results["documents"][0][0])
print("Metadata:", results["metadatas"][0][0])Lab Takeaways
- Ollama enables offline, zero-cost development using standard OpenAI SDK clients.
- Always stream tokens in user-facing web apps to reduce perceived latency down to a few hundred milliseconds.
- Multimodal vision models natively inspect charts, diagrams, and UI wireframes.
- ChromaDB provides lightweight local vector persistence with cosine distance metrics.
Explore Related Tools
Hand-picked utilities and calculators related to this tool.
