Ask HN: What’s Your Current AI Agentic Setup? (Self-Hosted Edition)

If you spend any time browsing Hacker News, you’ve probably seen a variation of the thread title: “Ask HN: What’s your current AI agentic setup?” It’s a question that pops up every few weeks, and every time it does, the comments section turns into a goldmine of configurations, workflows, and creative ways people are pushing local AI to its limits.

Here at Self-Hosted Tech, we love a good homelab project. But as local LLMs have evolved from simple chat interfaces into autonomous, tool-using agents, the complexity of our setups has grown right alongside them. Today, we’re going to break down what an AI agentic setup actually looks like in a homelab environment, explore what the community is running, and walk through a practical, beginner-friendly setup using Ollama and CrewAI.

Whether you’re running a beefy dual-GPU rig or just a modest mini-PC, there’s an agentic setup out there for you. Let’s dive in.

What is an AI Agent, Anyway?

Before we start slinging code and Docker containers, let’s define what we mean by an “agent.”

When you spin up a local LLM using something like Ollama or LM Studio and chat with it via Open WebUI, you’re interacting with a standard language model. It takes text in, predicts the next best sequence of tokens, and spits text out. It’s incredibly smart, but it’s ultimately a very fancy, very fast autocomplete.

An AI Agent, on the other hand, is an LLM equipped with:

  • A Memory: It remembers past interactions or has access to a vector database (RAG) to pull in external knowledge.
  • Tools: It can execute actions. This might be running a Python script, querying a database, browsing the web, or sending an API request to your smart home hub.
  • Autonomy: You give it a high-level goal, and it uses a reasoning loop (like ReAct—Reasoning and Acting) to figure out how to achieve that goal using its tools.

If an LLM is a brain in a jar, an agentic setup is a brain with hands, eyes, and an internet connection.

The Core Homelab Agentic Stack

When the HN crowd starts sharing their setups, a distinct pattern emerges. A solid local agentic stack usually consists of three layers: the hardware, the inference engine, and the orchestration framework.

1. Hardware

You don’t need an NVIDIA H100 cluster to run agents locally, but you do need to be realistic.

  • The Sweet Spot: A single used RTX 3090 (24GB VRAM) remains the undisputed king of homelab AI. It allows you to run 8B to 14B parameter models with massive context windows, which agents desperately need.
  • The Budget Option: 16GB of VRAM (like an RTX 4080 or 4060 Ti 16GB) or even relying on Apple Silicon Unified Memory (M1/M2/M3 Macs with 32GB+ RAM). You’ll be leaning on smaller models like Llama 3 8B or Phi-3, but they are surprisingly capable agents.
  • RAM is King: If you have to offload to CPU RAM, make sure it’s fast (DDR5) and plentiful (64GB+).

2. The Inference Engine

This is the software that actually loads the model weights and generates the text.

  • Ollama: The easiest way to get started. It runs as a background service, exposes a local API, and makes pulling models as easy as ollama run llama3.
  • vLLM: For those who need raw speed and high throughput. It’s a bit more complex to set up but supports advanced features like continuous batching.
  • LM Studio: A great GUI option for desktop users, though less ideal for headless homelab servers.

3. The Orchestration Framework

This is where the magic happens. The framework connects your LLM to your tools and manages the reasoning loops.

  • LangChain / LangGraph: The old reliable. LangGraph is the newer, more robust way to build stateful, multi-agent workflows.
  • CrewAI: Currently the community favorite for its simplicity. You define “Crews” of agents with specific roles, give them tasks, and let them collaborate.
  • AutoGen: Microsoft’s framework, excellent for having multiple agents converse and write code together.
  • Flowise / Langflow: Visual, drag-and-drop builders. Perfect if you prefer Node-RED style interfaces over writing Python.

A Practical Walkthrough: Building a Local Research Agent

Let’s get practical. One of the most common setups shared on HN is a two-agent research and writing crew. We’re going to build this locally using Ollama and CrewAI.

Our goal: We want one agent to search the web for the latest homelab news, and a second agent to take that research and write a short blog post outline.

Step 1: Get Ollama Running

First, you need your inference engine. If you haven’t installed Ollama, you can do so with a simple curl script on Linux:

1

curl -fsSL https://ollama.com/install.sh | sh

Once installed, pull a model capable of agentic reasoning. Meta’s Llama 3 8B is a fantastic starting point because it follows instructions well and fits comfortably on most modern GPUs.

1

ollama pull llama3

Verify it’s running by checking the local API endpoint (usually http://localhost:11434).

Step 2: Set Up Your Python Environment

CrewAI is a Python library. To keep your homelab clean, create a virtual environment.

1
2
3
4

mkdir local-agent-setup
cd local-agent-setup
python3 -m venv venv
source venv/bin/activate

Now, install CrewAI and LangChain’s community tools (which we’ll use for web searching).

1

pip install crewai langchain-community

Step 3: Write the Agent Script

Create a file named homelab_crew.py. We are going to define two agents: a Researcher and a Writer. We’ll give the Researcher a web search tool, and we’ll force both agents to use our local Ollama instance instead of sending data to OpenAI.

For the web search tool, we’ll use DuckDuckGo so you don’t need to fumble with API keys just to get started.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60

from crewai import Agent, Task, Crew, Process
from langchain_community.tools import DuckDuckGoSearchRun
from langchain_community.llms import Ollama

# 1. Setup the local LLM via Ollama
# Make sure Ollama is running and you have pulled 'llama3'
local_llm = Ollama(model="llama3")

# 2. Setup the tool the agent will use
search_tool = DuckDuckGoSearchRun()

# 3. Define your Agents
researcher = Agent(
    role='Senior Homelab Research Analyst',
    goal='Uncover cutting-edge developments in self-hosting and local AI',
    backstory='You are a meticulous researcher who loves digging through search results to find the best open-source homelab tech.',
    verbose=True,
    allow_delegation=False,
    llm=local_llm,
    tools=[search_tool] # Giving the researcher web access!
)

writer = Agent(
    role='Tech Content Writer',
    goal='Write a concise, engaging blog post outline based on the research',
    backstory='You are a skilled writer who translates complex tech topics into easy-to-understand articles for the Self-Hosted Tech community.',
    verbose=True,
    allow_delegation=False,
    llm=local_llm
    # Notice the writer has no tools, it only processes the text from the researcher
)

# 4. Define the Tasks
research_task = Task(
    description='Search the web for the latest trends in local AI agents and self-hosting for 2024. Focus on tools like Ollama, CrewAI, and new open-source models.',
    expected_output='A detailed list of 3-5 bullet points summarizing the latest trends with brief explanations.',
    agent=researcher
)

write_task = Task(
    description='Using the research provided by the analyst, write a short, punchy blog post outline for a homelab audience.',
    expected_output='A markdown formatted blog post outline with a catchy title, intro, and 3 main sections.',
    agent=writer
)

# 5. Create the Crew
# We set process to sequential, meaning the research task happens before the write task
ai_crew = Crew(
    agents=[researcher, writer],
    tasks=[research_task, write_task],
    process=Process.sequential
)

# 6. Kickoff the crew!
print("Starting the Homelab AI Crew...")
result = ai_crew.kickoff()

print("######################")
print("FINAL OUTPUT:")
print(result)

Step 4: Run It and Watch the Magic

Execute the script from your terminal:

1

python homelab_crew.py

Because we set verbose=True, you will see the agents thinking in real-time. The researcher will decide to use the search tool, formulate a query, read the results, and compile its findings. Then, it will pass those findings to the writer agent, who will generate the outline.

All of this happens locally. The only external request is the DuckDuckGo search. Your data never leaves your network to hit an OpenAI or Anthropic API.

If we look back at those Hacker News threads, here are a few other agentic setups that homelabbers are running right now:

  • The Personal Knowledge RAG Agent: Using AnythingLLM connected to a local Ollama model. Users point AnythingLLM at a directory of their Obsidian notes or PDFs. The agent uses vector embeddings to answer questions about their own life, codebase, or research papers.
  • The Smart Home Automator: Using LangFlow to build a visual pipeline. An LLM agent is given a custom tool that makes HTTP requests to a Home Assistant API. The user can type, “Is the garage door open, and if so, close it,” and the agent parses the intent, queries Home Assistant, and triggers the relay.
  • The Multi-Agent Coders: Using AutoGen to spin up a “Coder” agent and a “QA” agent. The Coder writes a Python script to solve a math problem, and the QA agent runs the script, reads the error output, and tells the Coder how to fix it. This all runs in a local Docker container.

Challenges and Pitfalls

Running agents locally isn’t all sunshine and rainbows. If you’re going to build an agentic homelab, keep these hurdles in mind:

  • Context Window Limits: Agents eat context windows for breakfast. The system prompt, the tool descriptions, the intermediate reasoning steps, and the tool outputs all add up. If you exceed your model’s context window, it will forget what it was doing. You need a model that supports at least 8k tokens, and ideally more.
  • Smaller Models Get Confused: GPT-4o is incredibly good at knowing when to use a tool. A local 8B model might hallucinate the tool’s output or get stuck in a loop trying to call a tool incorrectly. You will need to write very explicit, strict prompts.
  • Speed: Agentic loops require multiple LLM calls. If you are running on CPU, a task that takes GPT-4 five seconds might take your local setup five minutes. Patience is a virtue, but VRAM is better.

Summary

The “Ask HN” threads are proof that we are entering a new era of homelabbing. We aren’t just self-hosting web apps and media servers anymore; we’re self-hosting autonomous digital workers.

By combining an inference engine like Ollama with an orchestration framework like CrewAI, you can build a fully local, privacy-respecting AI agent that can search the web, read your files, and execute tasks. It takes a bit of Python, a decent GPU (or a lot of patience), and a willingness to tweak prompts, but the payoff is immense.

What’s your current AI agentic setup? Are you using Flowise, AutoGen, or something entirely custom? Drop your stack, your hardware specs, and your favorite local model in the comments below. We read every single one, and who knows—your setup might just be the star of our next video!