05 · The AI Tools Landscape

There are hundreds of AI tools, and new ones appear every week. Instead of memorising products, learn the categories and the layers they sit on. Then any new tool takes five minutes to place.

Level 135 minTrack 5 · ToolsLast verified 2026-10

By the end of this page you can

Place any AI tool into one of the layers of the AI stack.
Know the main categories: models, chat apps, IDE assistants, terminal agents, cloud agents, local runners, frameworks, data stores, evaluation tools and protocols.
Choose tools using sound criteria rather than hype.

About product names Product names below are examples to make categories concrete. This is not an endorsement or a complete list. Features and names change often, so check each tool's current docs. Your company may also have an approved list. Follow it.

The AI stack in layers

When you meet a new tool, ask: which layer is it? What does it sit on top of? What does it replace?

Models (layer 2)

Kind	What it is	Examples (illustrative)
Hosted frontier models	The most capable general models, used through an API or app.	OpenAI GPT family, Anthropic Claude family, Google Gemini family
Open-weight models	Downloadable weights you can run yourself.	Meta Llama, Mistral, Qwen, DeepSeek, Google Gemma, Microsoft Phi, OpenAI gpt-oss
Coding-specialised models	Tuned for code generation and agentic coding.	Code-tuned variants from most model families
Embedding models	Turn text or code into vectors for search and RAG.	Provider embedding APIs, open models on Hugging Face
On-device models	Small models running on phones and laptops.	Apple's on-device Foundation Models (Apple Intelligence), small open models via MLX / Core ML
Multimodal / media	Image, speech, audio and video generation or understanding.	Speech-to-text (e.g. Whisper), image and video generators

Model hubs such as Hugging Face host hundreds of thousands of open models, datasets and demos.

Chat assistants

General-purpose apps (web, desktop, mobile) for conversation, writing, analysis, research and learning. Typical features: file upload, web search, code execution/analysis, image understanding, "projects" with shared files, memory, and connectors (often via MCP).

Examples: ChatGPT, Claude, Gemini, Microsoft Copilot, Perplexity (search-first), NotebookLM (research over your own sources).

IDE assistants

AI built into the editor: completion, chat, inline edits and agent mode. Covered in depth on page 06.

Type	Examples (illustrative)
Built into the IDE	Xcode coding intelligence (with built-in model integrations and agents), JetBrains AI Assistant and the Junie agent (IntelliJ IDEA), Copilot Chat in VS Code
Extensions / plugins	GitHub Copilot (VS Code, JetBrains, Xcode, others), Gemini Code Assist, Continue (open-source), Cline (open-source), Tabnine
AI-first editors	Editors designed around AI agents, e.g. Cursor, Zed and others (often forks of VS Code)

Terminal (CLI) agents

Agents that run in your terminal. They read and edit files, run commands and tests, and use git. They work with any editor, and many integrate with IDEs and with each other through protocols.

Examples: Claude Code, OpenAI Codex CLI, Gemini CLI, GitHub Copilot CLI, Aider (open-source), OpenCode (open-source).

Cloud / background agents

Agents that run on remote machines. You hand them a task (often from an issue or chat), they work in a sandbox, and they come back with a pull request. Good for parallelising well-specified work.

Examples: GitHub Copilot coding agent, OpenAI Codex (cloud), and similar offerings from other vendors.

Local model runners

Tool	What it does
Ollama	Simple CLI/server to download and run open models locally with an OpenAI-compatible API.
LM Studio	Desktop app to discover, run and chat with local models. Exposes a local API server.
llama.cpp	The high-performance C/C++ engine underneath many local tools (GGUF format).
MLX	Apple's open-source array framework for machine learning on Apple silicon. Runs and fine-tunes models efficiently on Macs.
vLLM / TGI	High-throughput servers for hosting models on GPUs (Level 4).

When local makes sense Sensitive data that must not leave the machine, offline work, experiments, cost control, or learning how models behave. Local models are usually less capable than frontier hosted models, so match the model to the task.

Frameworks & SDKs

Category	Python	JVM / Scala
Provider SDKs	Official SDKs from model providers	Official Java SDKs from several providers (usable from Scala); `sttp-openai` (Scala)
LLM app / agent frameworks	LangChain & LangGraph, LlamaIndex, Pydantic AI, DSPy, vendor agent SDKs	LangChain4j, Spring AI, Semantic Kernel (Java)
MCP SDKs	Official Python SDK (incl. FastMCP-style API)	Official Java/Kotlin SDKs (usable from Scala)
Classical ML / DL	scikit-learn, PyTorch, JAX, Hugging Face Transformers	Spark MLlib, DJL (Deep Java Library)

Frameworks are optional Many experienced teams call model APIs directly and add a framework only when it clearly saves effort. Learn the raw API first (Level 2). Then you will understand what any framework is doing for you.

Data: vector databases & search

Store embeddings and search by similarity. This is the backbone of RAG.

Examples: pgvector (Postgres extension), Qdrant, Weaviate, Milvus, Chroma, Pinecone, and vector search in Elasticsearch/OpenSearch, Redis and cloud databases.

Evaluation & observability

Measure quality, trace agent runs, track cost and catch regressions (Level 3).

Examples: promptfoo, Langfuse, LangSmith, Arize Phoenix, Weights & Biases, and OpenTelemetry's semantic conventions for GenAI.

Protocols and open standards

Standard	Connects	Page
MCP (Model Context Protocol)	AI apps ↔ tools & data sources	09
ACP (Agent Client Protocol)	Editors/IDEs ↔ coding agents (bring your own agent into an editor)	06
A2A (Agent2Agent)	Agents ↔ other agents across systems	Level 3
AGENTS.md	A conventional Markdown file of project instructions read by many agents	07
Agent Skills (`SKILL.md`)	An open format for portable, on-demand agent know-how	07
OpenAI-compatible API	A de-facto HTTP API shape many providers and local runners support	Level 2

How to choose a tool

Criterion	Questions to ask
Policy & privacy	Is it approved by my organisation? Where does my code go? Is data used for training? Is there an enterprise or zero-retention option?
Fit to stack	Does it support my IDE (IntelliJ for Scala, Xcode for Swift, VS Code)? Does it understand my build tool (sbt, Mill, Gradle)?
Model choice	Can I pick models? Bring my own key? Use local models?
Agent capabilities	Can it run tests and commands? Is there a permission model? Does it support MCP, skills and AGENTS.md?
*Quality on your* code**	Try the same 3 real tasks in each candidate. Benchmarks don't predict your codebase.
Cost	Seat price vs usage-based pricing. Limits on premium models.
Lock-in	Are configs in open formats (AGENTS.md, SKILL.md, MCP) that I can carry to another tool?

Expert habit: a personal benchmark Keep 5–10 real tasks from your own work (a tricky Scala type error, a Spark optimisation, a refactor, a test suite to write). When a new tool or model appears, run your benchmark. In 30 minutes you will know more than any review could tell you.

Key takeaways

Think in layers: infrastructure → models → frameworks → protocols → customisation → experiences.
The main categories are chat apps, IDE assistants, terminal agents, cloud agents, local runners, frameworks, vector DBs, evals and protocols.
Open standards (MCP, AGENTS.md, Agent Skills) make your setup portable between tools.
Choose with policy, fit, quality on your own tasks, cost and lock-in in mind.