HO - AI Engineer Expert

## Lộ trình Ôn thi & Chuẩn bị cho AI Engineer Expert ### 📅 Gợi ý Timeline: 2-4 tuần trước phỏng vấn --- ### Tuần 1: Foundations Review **Mục tiêu**: Đảm bảo fundamentals vững #### 1. Python Advanced (Ngày 1-2) ```python # Async patterns - MUST KNOW import asyncio from aiohttp import ClientSession async def batch_embed(texts: list[str]) -> list[list[float]]: """Production async embedding call""" async with ClientSession() as session: tasks = [embed_single(session, text) for text in texts] return await asyncio.gather(*tasks) async def embed_single(session: ClientSession, text: str) -> list[float]: # Implementation with retry logic, timeout pass # Pydantic models - MUST HAVE from pydantic import BaseModel, Field, field_validator from typing import Optional class RetrievalResult(BaseModel): chunk_id: str = Field(..., description="Unique chunk identifier") text: str = Field(..., min_length=10, max_length=2000) score: float = Field(..., ge=0.0, le=1.0) metadata: dict = Field(default_factory=dict) @field_validator('chunk_id') @classmethod def validate_chunk_id(cls, v: str) -> str: if not v.startswith('chunk_'): raise ValueError('chunk_id must start with "chunk_"') return v ``` **Practice**: Implement async retry decorator với exponential backoff #### 2. RAG Deep Dive (Ngày 3-4) **Đọc/B看不完 tài liệu**: - RAG tutorial của LangChain: https://python.langchain.com/docs/tutorials/rag/ - Chunking strategies article (từ Pinecone) - Hybrid search implementation **Hands-on project** (clone và chạy): ```bash git clone https://github.com/langchain-ai/rag-from-scratch cd rag-from-scratch pip install -e . ``` **Key concepts cần nắm vững**: | Concept | Depth level cần đạt | |---------|---------------------| | Chunking | Fixed size, semantic, recursive character split | | Embedding models | OpenAI, BGE, e5, nomic - so sánh trade-offs | | Vector DB ops | CRUD, metadata filtering, ANN algorithms (HNSW, IVF) | | Re-ranking | Cross-encoder vs bi-encoder approaches | | Hybrid search | Dense + sparse (BM25) combination | #### 3. LangGraph Mastery (Ngày 5-7) ```python # Full LangGraph workflow - PHẢI CODE ĐƯỢC from langgraph.graph import StateGraph, END from typing import TypedDict, Annotated import operator class AgentState(TypedDict): messages: Annotated[list, operator.add] current_node: str retrieved_docs: list hallucinations: int def create_agent_graph(): graph = StateGraph(AgentState) # Add nodes graph.add_node("planner", planner_node) graph.add_node("retriever", retriever_node) graph.add_node("generator", generator_node) graph.add_node("verifier", verifier_node) # Define edges graph.add_edge("planner", "retriever") graph.add_edge("retriever", "generator") graph.add_edge("generator", "verifier") # Conditional edge from verifier graph.add_conditional_edges( "verifier", should_continue, { "regenerate": "generator", # hallucinations detected "end": END # quality acceptable } ) return graph.compile() def should_continue(state: AgentState) -> str: if state["hallucinations"] > 2: return "end" # Max retries, return anyway elif state.get("quality_score", 0) < 0.7: return "regenerate" return "end" ``` **Must-complete tutorials**: 1. LangGraph quick-start 2. RAG with LangGraph 3. Agentic patterns (multi-agent) --- ### Tuần 2: Production Systems **Mục tiêu**: Hiểu sâu về deployment và operations #### 1. vLLM Deployment (Ngày 1-2) ```bash # Basic vLLM deployment pip install vllm # Test locally python -m vllm.entrypoints.openai.api_server \ --model Qwen/Qwen2-7B-Instruct \ --dtype half \ --port 8000 # Test API curl http://localhost:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "Qwen/Qwen2-7B-Instruct", "messages": [{"role": "user", "content": "Hello!"}] }' ``` **Advanced topics cần hiểu**: - Tensor parallelism (multiple GPUs) - Quantization (AWQ, GPTQ, GGUF) - Batching strategies (continuous vs static) - Prefix caching - Speculative decoding #### 2. Vector DB Deep Dive (Ngày 3-4) **Qdrant (Khuyến nghị vì self-hosted, Rust-based)**: ```python from qdrant_client import QdrantClient from qdrant_client.models import Distance, VectorParams, PointStruct # Initialize client client = QdrantClient("localhost", port=6333) # Create collection with specific config client.create_collection( collection_name="banking_kb", vectors_config=VectorParams( size=1024, # BGE-large dimension distance=Distance.COSINE ) ) # Upsert with payload points = [ PointStruct( id=idx, vector=embedding, payload={ "text": doc_text, "doc_type": "faq", "category": "loan", "created_at": "2024-01-15" } ) for idx, (doc_text, embedding) in enumerate(documents) ] client.upsert( collection_name="banking_kb", points=points ) # Metadata filtering results = client.search( collection_name="banking_kb", query_vector=query_embedding, query_filter=models.Filter( must=[ models.FieldCondition( key="doc_type", match=models.MatchValue(value="faq") ) ] ), limit=5 ) ``` **Compare Vector DBs**: | Feature | Qdrant | Milvus | Pinecone | Weaviate | |---------|--------|--------|----------|----------| | Deployment | Self-hosted | Self-hosted | Cloud/On-prem | Both | | Performance | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐ | ⭐⭐⭐⭐ | ⭐⭐⭐⭐ | | Filtering | Excellent | Good | Good | Excellent | | Ecosystem | Growing | Large | Managed | Good | | Recommendation | ✅ Best for banking | High scale | Quick setup | Flexible | #### 3. MCP & Tool Calling (Ngày 5-6) ```python # MCP Server Example from mcp.server.fastmcp import FastMCP from pydantic import BaseModel mcp = FastMCP("BankingTools") @mcp.tool() def get_account_balance(account_id: str) -> dict: """Get current balance for a customer account. Args: account_id: Account ID in format ACC######## Returns: Dictionary with balance and currency """ # Validate format import re if not re.match(r'^ACC\d{8}$', account_id): raise ValueError("Invalid account ID format") # Call banking API return {"balance": 50000000, "currency": "VND"} @mcp.tool() def transfer_funds(from_account: str, to_account: str, amount: float) -> dict: """Transfer funds between accounts. Requires: - Both accounts must exist - Sufficient balance - Within daily limit """ # Implementation with compliance checks pass ``` #### 4. Observability Setup (Ngày 7) ```python # Langfuse setup from langfuse import Langfuse from langfuse.decorators import langfuse_context, observe langfuse = Langfuse() @observe() def retrieval_step(query: str): # Langfuse tự động trace results = vector_db.search(query) langfuse_context.update_current_trace( metadata={ "retrieval_count": len(results), "top_score": results[0].score if results else 0 } ) return results @observe() def generation_step(context: str, query: str): response = llm.generate(context, query) langfuse_context.update_current_trace( metadata={ "model": model_name, "tokens_used": response.usage.total_tokens, "latency_ms": response.latency } ) return response ``` **Metrics phải track được**: | Metric | Target | Alert threshold | |--------|--------|------------------| | Retrieval Recall@5 | > 0.85 | < 0.75 | | Hallucination Rate | < 5% | > 10% | | Tool Success Rate | > 98% | < 95% | | Latency p95 | < 3s | > 5s | | Cost per Query | < $0.01 | > $0.05 | --- ### Tuần 3-4: Mock Interviews & Portfolio Building #### 1. Coding Practice **LeetCode problems recommended** (medium/hard): - 239. Sliding Window Maximum (async batching concept) - 146. LRU Cache (memoization patterns) - 200. Number of Islands (graph traversal) - 690. Employee Importance (tree/graph) **Thực hành practical coding**: ```python # Write from scratch - không dùng LangChain helper functions def simple_rag(query: str, docs: list[str], top_k: int = 3) -> str: """ Simple RAG implementation without LangChain: 1. Embed query 2. Embed documents 3. Compute similarities 4. Retrieve top-k 5. Generate response (mock) """ pass ``` #### 2. System Design Mock **Practice scenarios**: 1. **"Design a customer service chatbot for ACB mobile app"** - Multi-turn conversation - Integration với banking APIs - Compliance requirements - Fallback strategy 2. **"How would you improve an existing RAG system with 30% hallucination rate?"** - Diagnose root causes - Propose layered solution - Prioritize by impact - Timeline estimation 3. **"Design evaluation framework for agent system"** - Unit tests per component - Integration tests - End-to-end benchmarks - Monitoring production #### 3. Portfolio Preparation **GitHub profile cần có**: ``` 📁 ai-engineering/ ├── 📁 projects/ │ ├── 📁 rag-banking-kb/ # Main project │ │ ├── README.md │ │ ├── src/ │ │ ├── tests/ │ │ └── ADR.md # Architecture Decision Records │ ├── 📁 langgraph-workflow/ │ │ └── Multi-agent implementation │ └── 📁 vllm-deployment/ │ └── Production deployment configs ├── 📁 experiments/ │ └── Benchmark results └── 📁 contributions/ └── Open source contributions (if any) ``` **ADR template**: ```markdown # ADR-001: Vector Database Selection ## Status Accepted ## Context We need to select a vector database for our RAG system serving 1000+ daily internal queries. ## Decision We chose Qdrant for the following reasons: - Self-hosted (data sovereignty for banking) - Excellent filtering capabilities - Good Rust-based performance - Active community ## Consequences - Pro: Full control over data - Pro: Cost-effective at our scale - Con: Operational overhead - Con: Need DevOps expertise ``` --- ### 📚 Tài liệu tham khảo **Books (nếu có thời gian)**: 1. "Designing Data-Intensive Applications" - Martin Kleppmann 2. "Building LLM Applications" - (sắp ra) 3. "The Pragmatic Programmer" - hồi tưởng **Courses**: 1. DeepLearning.AI: LLMOps Specialization 2. LangChain: LLM Application Development 3. Weights & Biases: ML Engineering **Papers**: 1. RAG survey papers (2024) 2. Agentic AI architectures 3. Vector DB comparison studies **Blogs/Articles**: 1. Lilian Weng's blog (OpenAI) - Agentic systems 2. Pinecone blog - RAG best practices 3. Qdrant blog - Vector search advanced topics 4. Hugging Face blog - vLLM, fine-tuning **YouTube channels**: 1. AssemblyAI 2. Sentdex 3. DataEngineering Weekly --- ### ✅ Final Checklist trước ngày phỏng vấn - [ ] Có thể implement RAG từ scratch (không dùng LangChain) - [ ] Hiểu sâu về LangGraph state machine - [ ] Deploy được vLLM với basic optimization - [ ] Design được observability cho AI system - [ ] Viết được ADR cho 2-3 decisions - [ ] Có 2-3 project stories với measurable outcomes - [ ] GitHub profile clean và có content - [ ] Đã research về ACB và banking industry - [ ] Prepared cho behavioral questions về mentorship - [ ] Researched salary range cho level này