ByteByteGo5 min readintermediate
EP228: How SSH Works
Summary
This article provides concise overviews of several engineering topics, including the SSH handshake and authentication process, six techniques for efficient AI system serving, the architectural components of an AI agent, and a three-step process for evaluating AI applications. It covers fundamental concepts across networking and AI/ML.
- SSH uses a key exchange to derive session keys and public-key cryptography for client authentication, with the private key never leaving the client.
- LLM serving efficiency techniques include streaming, quantization, continuous batching, prefix caching, paged KV cache, and speculative decoding.
- AI agents are structured around an LLM 'brain' that uses planning, external tools, memory (short/long-term), and guardrails within a loop.
- Evaluating AI applications involves picking a task, collecting relevant data, and developing graders (code-based, model-based, or human) for assessment.
This article offers quick, high-level refreshers on foundational concepts in secure networking and emerging practices in AI system design and evaluation, useful for engineers needing a broad understanding.
5/10
