When I first started building my mental model for agentic AI, I identified two types of gaps that you need to fix when working with agents:

1. Knowledge gaps : agent doesn’t know something you want it to know, like your name or what type of equipment you have in your home gym.
2. Behavioral gaps : agent doesn’t behave the way you’d like or expect. For example, it guesstimates character counts instead of counting characters.

I’ve since realized there’s a third type: capability gaps.

Capability gaps are when the agent isn’t able to do something you want it to do. For example, you might want your agent to help you manage your email, but it’s not able to connect to your email program. Or you might want it to help you edit videos, but it isn’t able to work with video directly.

I’ve noticed this is the first gap people at work run into when I help them set up an agentic AI tool, like Kiro or Claude Code. They normally have a use case in mind that they want to use agents to help them with, and they quickly run into the agent not being able to do that specific thing (email, calendar, and Slack have been the most common).

There are multiple options to address capability gaps, in increasing order of difficulty:

• Built-in tools: agentic AI apps, like Amazon Quick desktop app or Claude desktop app, that target a broader non-technical audience have built-in options for connecting and enabling additional functionality. They might be called plugins, connectors, or something similar. This is the first thing you should check for, as they will cover the most universal use cases, like email and calendar.
• Command-line tools: agents can use command-line tools to get things done with your computer, like ffmpeg to work with video, pandoc to convert documents, or Python for doing just about anything. This option often means looking for open source packages that you can install on your computer (or asking your agent to do it for you).
• Model Context Protocol servers (MCPs): MCPs are lightweight programs that allow your agent to connect to and use other apps and services, like Figma or email. Unlike command-line tools, where you just install and the agent can use it, MCPs often require you to install and then configure your agentic AI harness to enable them. One thing to watch: every MCP loads a set of tools into your agent’s context window, which eats into its capacity to do work. Only enable MCPs you’re actually using.
• Do-it-yourself: Lastly, you can use the agents to create a custom solution to enable what you need. The ability to do this depends on a variety of factors, like what specifically you need and how the app/service is built. For example, I use OmniFocus at work to manage my to do list, and it has become the place where all of the todos my agents triage for me are created. In order to do this, I first worked with my agent to create an MCP so any agent could use OmniFocus on my computer.

If you run into a capability gap and aren’t sure how to proceed, start by asking your agent. It can often figure it out or find a solution on its own.

If that doesn’t work, search online, in Slack, or in available wikis. Someone may have already solved it, and if so, you can paste the solution directly into your session and have your agent take it from there.

It introduces the elements of the framework, explains why it works given the importance of context engineering, and provides more detail on what it actually looks like to work with AI agents daily.

Stay tuned for the final posts in my Managed AI framework series.

It isn’t that every tool has its own AI agent baked in. It’s that every tool can be used by YOUR agents.

I’ve been spouting this to everyone at work. In a world where I can have my own personal Jarvis (AI agent), why would I want to use a generic agent that someone else provides.

And the more you use agents, the more you realize that having to touch a GUI becomes the bottleneck in getting work done. This is something developers already know; and I learned it from my experience with scripting languages. Programmatic usage is unparalleled for speed.

I can’t wait for the rest of the world to catch up to this new reality.

A few years ago, I set up a new blog. I wanted to do it cheaply, and I landed on using Jekyll and GitHub. It took an entire weekend of reading the Jekyll docs, tinkering, and troubleshooting to launch the blog and configure it to my liking. I spent a lot of hours on it, but I wound up with a setup I fully understood and could easily manage.

For this blog, I was initially going to fire up Jekyll again. I’d have to relearn it in my limited free time, but I was familiar with it and had previously liked it. Best of all, I knew it was free.

Before starting, I used Claude to research other options. It prepared a report that included Hugo , its top recommendation given my requirements. After a few more questions to learn how Hugo compared to Jekyll, we were off to the races.

Setting up the Hugo blog was nothing like my prior experiences. It took ~40 minutes. I never looked at the documentation. Most of the work was done by Claude Code, which provided detailed step-by-step instructions for the parts I had to do.

It was incredibly easy to go from nothing to a live Hugo blog with Claude.

Over the coming weeks, I continued to use Claude to tweak and improve the blog…from my phone!

I learned I could point Claude Code in the iOS app to my blog’s GitHub repo, ask it to change something, and get it done from anywhere, anytime.

I’ve done this a lot.

Whenever I want to change something, I just have to describe it in plain English.

Occasionally I add a screenshot to the chat to help illustrate the issue I want to address.

With agentic AI, it’s so much easier to tackle projects, including ones that would require a lot of initial research to get done. That’s great for someone like me who has very limited free time. And I’m having a blast playing around with it.

The flip side is that I’m not very familiar with how Hugo works.

I’m completely dependent on Claude for my blog. I have to rely on Claude to publish blog posts, update content, and make changes to the site. If I lose access to it, I’m going to have to bite the bullet and do the work that Claude saved me from doing to learn how my blog works.

I’m now gradually learning Hugo because I’m uncomfortable with that dependency for my personal blog. I’m also starting to experiment with running LLMs locally for situations where I don’t have internet access.

This experience has also given me a better sense of the tradeoff of using AI agents to get things done quickly.

With AI agents, you can save a lot of time, but you sacrifice learning and understanding, especially when you are working in an area that falls outside of your expertise.

It’s good to think about what skills and what knowledge are worth the effort to build. Once you identify those items, you can then pivot to using LLMs to help you learn and overcome any blockers you run into along the way.

This is one of my favorite things to do now; kicking off a Claude research project for something I want to learn and using the artifact it creates as a primer.

To address the first issue, you need to update your Copywriter’s persona. The second issue is already covered in a team rule, but it needs to be sharpened. You can fix the last one in the Plan my day skill. Your agent makes the required changes so that next week you are less likely to hit these issues again.

This is what managing your team of AI agents looks like in practice. It’s not a big project or one-time effort. It’s a small, regular process that helps you identify and correct issues with your team as you work with them. Those small fixes compound into big improvements over time.

Why you need a feedback loop

AI agents have limited mental energy per session (the context window), and their work gets worse as they use it up. Therefore, to get the best output possible, you need to work with agents on focused sessions that are tackling a specific task (or clear the session context to move on to the next task).

That means over the course of a week, you’ll have a lot of short, working sessions with your agents. In those sessions, you’ll give them guidance and corrections to improve what they’re doing. The issues you correct in one session will pop up again in a future session if you don’t address them permanently in your setup.

You might be tempted to address every issue as you go, but then you are using the session to improve your setup rather than get the task done. That’s why it’s better to do all of the improvements together in a separate session that you run periodically. You stay focused on the task, and your setup changes get the attention they deserve.

Every fix you apply to your setup takes up some of the available context window at the start of the session, so you should focus on fixing issues that you encounter repeatedly across sessions.

That’s why you can’t just keep the corrections in your head. You’ll forget what the core issues are.

A formal feedback loop ensures you are collecting the necessary information between review sessions to make informed changes to your setup.

Capturing the feedback

Feedback is the fuel that drives the process, so you need to capture it consistently across all of your sessions.

To do this, I moved away from capturing individual pieces of feedback in the moment to instead making it part of my routine to end a working session. I now have a close-shop skill at work and at home that I run at the end of every session or before I clear session context to move on to the next task. The close-shop skill runs multiple skills, including one that logs session feedback. See the bottom of this post for the session-feedback skill I use at work.

The agent decides what goes into the feedback log, not you. That allows you to stay focused on the task at hand, collaborating with your agent to get the work done. At the end, the agent picks up feedback that it thinks would help it work better in the future.

Not every session will result in a feedback entry, and that’s fine. The point is that you get in the habit of giving your agents an opportunity to reflect and look for feedback worth saving across every session.

To simplify things, keep all of the feedback in one file in a central location. You can set it up so that every entry specifies what agent recorded the feedback. That’s helpful for when you do the review to determine if this needs to be addressed via a team rule, a persona update, a skill change, or a knowledge base update.

Addressing the feedback: the weekly audit

Once a week on Fridays, work with an agent to review the feedback logs to determine what to fix and how. Think of it as a weekly 1-on-1 with your team where you review the past week and prepare for the coming week.

There are two components to the review, both of which your agent will help you do: triaging the feedback and addressing it. In the first step, you are looking for issues that are worth addressing in the logs. In the second step, you are deciding at what level to address the issue and implementing the fix via your agent.

The nature of the issue helps determine the best way to fix it.

• Does it apply to every agent, every session? Team rule
• Is it specific to one agent? Persona prompt
• Is it specific to a repeatable task? Skill
• Is the issue that your agent(s) were missing information they needed? Knowledge base

The first time you review the feedback, you can go through the entire process manually. Afterward, ask your agent to create a skill so you have a repeatable process for it in the future. You can see my audit-feedback skill below.

The agent will make recommendations about what to address and how, but push back on anything that doesn’t make sense. Remember, you are managing the team, and that takes work.

Watch this video to see me go through the process at home, working through real issues with real fixes.

This is a management practice, and you’ll get out of it what you put into it. The underlying tools are simple: a feedback log, an audit cadence, and a framework for deciding what to fix and how. By doing this consistently, though, you ensure that your team keeps getting better because you’re investing in their development the same way a good manager invests in their people.

The Managed AI Framework is tool-agnostic. The implementation isn’t. Are you ready to meet my teams?

session-feedback skill example

This is the session-feedback that I use at work. I worked with my LLM Expert agent to create this skill.

---
name: session-feedback
description: Reflect on the current session and append generalizable feedback to the agent's feedback log. Invoke at the end of a working session to capture learnings that could improve the agent's persona prompt, steering rules, or skills. Focus on patterns and principles, not task-specific details.
---

# Session Feedback

## When to Use

The user asks you to log feedback, reflect on the session, or invokes this skill at the end of a working session.

## How to Run

1. **Identify the agent.** Determine which agent persona the session's work was primarily done by. Check the conversation for subagent delegations — if the corrections and learnings came from a delegated agent (e.g., marketing-strategist, writer), the feedback belongs in that agent's log, not the executing agent's log. When in doubt, infer from the suggested fixes (which persona do they target?).
2. Review the conversation history from this session.
3. Identify moments where:
   - The user corrected you or pushed back on your output
   - You had to be asked to do something differently than your default behavior
   - The user provided a preference that isn't captured in your persona or steering rules
   - Your output required multiple rounds of revision to get right
   - Something worked particularly well that isn't explicitly codified
3. For each observation, ask: **"Is there a general principle here, or is this specific to this task?"** Only log observations that point to a reusable principle — something that would improve your performance across future tasks, not just this one.
4. Append the entry to your feedback log. No approval needed — just do it and show the user what you logged.
5. If the feedback also qualifies as a memory entry (user preference or working pattern), append it to `~/.kiro/steering/memory.md` as well.

## Feedback Log Location

Append to: `~/ai/logs/feedback/feedback.md` — a single consolidated log shared by all agents.

If the file doesn't exist, create it with this header:

```
# Feedback Log

Generalizable learnings from working sessions. Used periodically to inform persona prompt and steering rule updates.
```

## Entry Format

Tag each entry with the agent identified in step 1 (not necessarily the agent executing this skill).

```
## <Date> — <agent-name>

- **Observation:** <What happened — one sentence, no task-specific details>
- **Principle:** <The general rule or preference this points to>
- **Suggested fix:** <Where this should be addressed (persona, steering, or skill), why that scope is right (e.g., "steering — applies across all document-creation personas" vs. "persona — only relevant to requirements writing"), and a brief idea of the change>
```

## Rules

- **Abstract, don't narrate.** Don't describe what the task was. Describe the behavioral pattern that needs to change or be reinforced.
- **One principle per bullet.** If an observation points to two different principles, split them.
- **Skip if nothing generalizable emerged.** Not every session produces feedback. If the session went smoothly and all corrections were task-specific, say so and don't append anything.
- **Keep entries concise.** Each entry should be scannable in under 30 seconds. The LLM expert will review these in batch — density matters.
- **Don't duplicate.** Before appending, read the existing log. If the same principle is already captured, note that it recurred (add a date) rather than creating a new entry.
- **Cross-log patterns.** If a principle seems like it would apply to other personas too, note that in the suggested fix. When the same principle appears across multiple persona logs, it's a signal for a steering file rather than a persona-level fix.

## When Reviewing Logs

Before promoting feedback into a persona or steering change, each entry should pass two tests:

1. **Recurrence:** Will this fire frequently enough to justify every agent (or this agent) reading it on every interaction?
2. **Specificity:** Is the fix concrete enough to change behavior, or is it vague advice the model would already "know"?

Drop entries that fail either test.

audit-feedback skill example

---
name: audit-feedback-logs
description: Audit all agent feedback logs and recommend changes to steering files, personas, or skills. Filters entries through recurrence and specificity tests, cross-references existing config to avoid duplicates, and proposes changes at the right level.
---

# Audit Feedback Logs

## When to Use

The user asks to audit feedback, review feedback logs, or promote feedback into setup changes.

## How to Run

### Step 1: Read all feedback logs and existing config

Read in parallel:
- `~/ai/logs/feedback/feedback.md` (the consolidated feedback log)
- All files in `~/.kiro/steering/`
- All persona files in `~/ai/personas/`

Skim skill files only when a feedback entry's suggested fix references a specific skill.

### Step 2: Filter entries

For each feedback entry, apply two tests:

1. **Recurrence:** Will this fire frequently enough to justify every agent (or this agent) reading it on every interaction? If it addresses a rare edge case, skip it.
2. **Specificity:** Is the fix concrete enough to change behavior? If it's vague advice the model would already "know," skip it.

Drop entries that fail either test.

### Step 3: Deduplicate against existing config

For each surviving entry, check whether the principle is already covered by an existing steering rule, persona instruction, or skill rule. If it is, skip it. If it's partially covered, note what's missing.

### Step 4: Determine the right level

Place each fix at the narrowest scope that covers its recurrence pattern:

| Level | Use when... |
|---|---|
| Steering file | The principle applies across multiple agents. All agents read steering files on every interaction, so the token cost is shared. |
| Persona | The principle applies to one agent across many tasks. |
| Skill | The principle applies to one agent in one specific workflow. |

When in doubt, prefer the narrower scope — it's cheaper and easier to promote later than to demote.

### Step 5: Present recommendations

Group recommendations into three categories:
- **Worth implementing** — passes both tests, not already covered, clear placement
- **Borderline** — passes tests but low recurrence or partially covered; flag for user decision
- **Not worth implementing** — fails a test; briefly explain why

For each recommendation, state:
- The principle (one sentence)
- Where it goes (specific file name and section)
- Why that level (one sentence)

Wait for user approval before making any changes.

### Step 6: Implement approved changes

Make the approved edits. After all changes are applied, list what was changed with file paths.

### Step 7: Mark processed entries

After implementation, do NOT delete feedback log entries — they're the historical record. The deduplication in Step 3 prevents re-processing on future audits.

## Rules

- Never auto-implement. Always present and wait for approval.
- Don't rewrite existing steering rules to absorb new feedback — add to them. Rewriting risks losing nuance from the original.
- If two feedback entries from different agents point to the same principle, that's a strong signal for steering level.
- If a feedback entry suggests a fix that contradicts an existing steering rule, flag the conflict for the user rather than resolving it yourself.

Your Copywriter gives you character counts for ad copy that are inaccurate. Your Data Wizard states unproven claims about why a metric went up or down as facts. You continue to have to tell your Marketing Strategist to look at audience insights to inform their messaging recommendations. Your team creates files in the wrong place.

These are all examples of the second type of gap you’ll regularly encounter: behavioral gaps. That’s when your agent doesn’t behave the way you’d like or expect. You can’t fix this gap with context files in a knowledge base. Instead, you’re going to have to use team rules and skills.

Team rules: the employee handbook

Team rules are where you lay out how your agents are expected to operate, always. That includes both behavioral standards, like principles around tone and accuracy, and operational context to help them work more effectively in your environment, like information about your knowledge base structure.

Your team rules are context that all of your agents load all of the time. That makes them ideal for addressing persistent, universal behavioral issues, like LLMs’ tendency to make up information (hallucination) and to be overly agreeable and supportive (sycophancy).

That also means they use your agent’s limited mental energy (context window) in every session. This happens regardless of whether the rules are relevant to the task at hand. The more team rules you have, the less mental energy your team has to work on anything. Therefore, you want to be very careful about what makes it into your team rules.

Reserve your team rules for rules that apply to every agent, every session. If a rule only applies to one agent, then it belongs in the persona prompt.

Here’s an excerpt from one of my team rules at work (full version below the post) that sets the standard for any work with data and metrics.

# Analytical Rigor Rules

## Numerical Verification

When performing any analysis involving numbers, calculations, or quantitative claims:

- After completing calculations, re-derive key figures from source data before stating them
- When citing numbers from provided data, quote the exact source value alongside any derived metric
- If performing multi-step calculations, show intermediate steps — do not skip to final answers
- When comparing values (percentages, ratios, deltas), state both raw values and the derived comparison

This rule helps reduce the likelihood my team will make a mistake when they work with numbers.

Different tools implement team rules in different ways. Kiro uses steering files. Claude uses CLAUDE.md files. Codex uses AGENTS.md files. Different names, same concept.

Skills: the standard operating procedures

Skills enable your agents to perform the same task the same way to deliver consistent results. You can define, in plain English, the steps you want them to perform, the tools you want them to use, the quality checks they should perform, and anything else that is important to completing a task the way you’d like it done. They are like standard operating procedures (SOPs) for tasks your AI team will do often.

Unlike team rules, skills are only loaded by the agent when they’re needed, so they use less of your agent’s available mental energy. Your agents are given just enough information to know what skills are available and what they’re for so they can determine when they should use them. It works pretty well in practice, but I do sometimes have to nudge an agent on my team to use an available skill.

I use a skill at work to help me process tutorial videos that I’ve started capturing of my Kiro CLI sessions. The skill tells my LLM Expert agent to:

1. Create a second copy of the video to work with.
2. Reduce the file size of the video.
3. Perform an audio pass to soften any pops or loud percussive sounds if there’s a voiceover.
4. Extract the video contents. If there’s audio, use a local Whisper speech-to-text model to transcribe it. Otherwise, sample frames from the video every few seconds and read the images to determine what was covered.
5. Call the Copywriter agent to provide several options for a title, subtitle, and key takeaways for the video based on the video script created in the last step.
6. Ask me what options to use.
7. Create a title card and key takeaways card using those options to add to the video. Ensure the copy has good left and right margins and spacing before continuing. If not, adjust and recreate the cards.
8. Add the cards to the beginning and end of the video.

I added the content extraction step after realizing it was taking too long to describe the video contents to my agent. I also added the margin check for the title cards after it created a few videos with the copy going edge-to-edge.

Now whenever I record a video tutorial at work, I simply have to ask my LLM Expert agent to create title cards for the video. It runs the skill above and produces something that is ready for me to share with minimal additional input from me. This has saved me a lot of time as I’ve started to record more videos.

Let your agents do the work

When you want to create team rules or skills, delegate the work out to your agents. You don’t need to know what should go in the files or how they should be implemented. Simply describe to your agent what you are trying to accomplish and ask for its help to get it configured. You can have it walk you through what it’s doing so you learn how it works and can refine its output to suit your needs. This is what you did to create your persona prompts and knowledge base, and hopefully you’re starting to pick up on the pattern. You are managing your agents so they help you accomplish your goals.

You’ve given your AI team the foundation they need to be successful, but that’s not enough. Things are going to change. New issues are going to come up. Your team needs ongoing management, not just a good setup. That’s what the feedback loop is for.

Team rules example

This is an example of one of the team rules I use at work. It’s in a markdown file that sits in Kiro’s ~/.kiro/steering/ folder.

# Analytical Rigor Rules

## Numerical Verification

When performing any analysis involving numbers, calculations, or quantitative claims:

- After completing calculations, re-derive key figures from source data before stating them
- When citing numbers from provided data, quote the exact source value alongside any derived metric
- If performing multi-step calculations, show intermediate steps — do not skip to final answers
- When comparing values (percentages, ratios, deltas), state both raw values and the derived comparison

## Self-Audit on Analytical Outputs

Before finalizing any analysis, data summary, or document containing quantitative claims:

1. List every specific number or metric stated in the output
2. For each: trace it back to either source data or a shown calculation
3. Flag any number that cannot be traced — restate it as an estimate or remove it

## Prefer Code for Computation

When analysis requires arithmetic beyond simple operations (addition, subtraction, single-step percentages), write and execute code to compute results rather than performing mental math. State that code was used.

## Source-to-Output Sync

When updating a derived document (BRD, narrative, report) from a source file (CSV, spreadsheet, data export), run a programmatic diff that checks: new/deleted items, changed values, structural changes, and count mismatches. Present the diff summary before making updates. Don't rely on the user to enumerate what changed.

Resource: Skill creation prompt

I want to turn a recurring workflow into a skill — a step-by-step standard operating procedure that an AI agent can follow to produce consistent, high-quality output every time.

Help me build this skill. Interview me with the following questions, one at a time. Ask each question, wait for my response, then move to the next.

1. What is the workflow you want to turn into a skill? Describe what it produces and why consistency matters for this particular task.
2. What are the inputs? What information or files does the agent need before it can start?
3. Walk me through how you do this task today, step by step. Include any steps where you review, check, or adjust before moving on.
4. What are the most common mistakes or failure points? Where does quality tend to drop when this task is done inconsistently?
5. Is there an example of a good output from this workflow? If so, describe what makes it good.

After the interview, produce a skill document in markdown with the following structure:

- **Name:** A short, descriptive name for the skill.
- **Purpose:** One or two sentences explaining what the skill does and when an agent should use it.
- **Inputs:** What the agent needs before starting (files, data, context, prior outputs).
- **Steps:** A numbered sequence of steps the agent should follow. Each step should be a clear, specific instruction — not a vague goal. Write steps as actions, not descriptions.
- **Quality gates:** After any step where errors are likely or quality matters most, add a checkpoint that tells the agent what to verify before continuing. Base these on the failure points I described.
- **Output:** What the final deliverable should look like, including format, length, and any standards it should meet.

Keep the skill focused on one workflow. If what I describe is actually multiple workflows, flag that and suggest how to split them into separate skills.

After you’ve used the prompt above to help you create the skill, you can also ask the agent to help you implement it (assuming it doesn’t do it automatically).

As you work with your team, you’ll quickly notice that you’re giving them the same facts and details over and over again. They’re good in their roles, but they don’t know anything about your specific job. Your Marketing Strategist doesn’t know what products you’re working on. Your Data Wizard doesn’t know what the metrics mean in the data they’re analyzing. Your Copywriter doesn’t know which value props to highlight in your ad copy. They’re missing information they need to do the work. You’ve run into the first type of gap: a knowledge gap. Now you need to fix it.

To close a knowledge gap, you need to start building a knowledge base that has the type of information a new hire would get during onboarding.

Think of the knowledge base as the wiki you’re building for your AI team. It contains relevant and useful content that’s easy to find whenever they need it. That includes information they’ll refer to all the time, like a style guide, and information they’ll need for specific projects, like a project brief.

In practice, a knowledge base is just a collection of files your agents can access, organized so the right information is easy to find.

No skimming, no stamina

This will be incredibly useful for your team as long as you keep in mind two constraints.

AI agents don’t skim or skip. They have to read everything in order to find anything. It doesn’t matter if what they need is in the first sentence of a document. They still have to read the entire document, which is not ideal.

This is a problem because AI agents have limited mental energy. The more they read, the worse they get. You don’t want to give them a book and ask them to find the three facts they need for a task. You want to give them a one-pager with those three facts.

Pages, not books

Instead of using one file that has everything in it, create a knowledge base that has a lot of files in it. Every file should cover a distinct and unique topic. Some of these will contain information that is more permanent and broadly applicable (role-specific). Others will contain information that is for specific projects or tasks (project-specific). With separate files, your agents can mix and match to get exactly what they need.

For example, I work with my Marketing Strategist across multiple products I support. They always need to know what marketing channels and tactics are available regardless of what product we’re working on. But if we’re working on Product A, they don’t need to know about the value props, target audience, or positioning of Product B.

Signposts, not search bars

When it comes to setting up your knowledge base, the overall principle is that if it would help you find a piece of information, it would also help your agents. This should guide how you organize, name, and store your information.

Imagine if you kept all of your files in one folder, and you had to find a specific file without being able to search for it. That would be a nightmare (or at least really time-consuming). Instead, you probably organize your files across different folders that have descriptive names. If you go into a folder named “Product A,” you know that you are going to find more files and folders in it that are related to that product.

You should do the same for your knowledge base: create a folder hierarchy that makes it easier for an agent to browse through all of the available files. For example, I keep my role-specific and project-specific information in separate folders. Every project gets its own folder where I can keep the project-specific files.

This is a simplified version of my knowledge base folder at work.

~/ai/
├── context/                                    # Persistent reference knowledge — rarely changes
│   ├── products/                               # One folder per product
│   │   ├── product-a/
│   │   │   ├── product-a-product-overview.md
│   │   │   ├── product-a-messaging-framework.md
│   │   │   └── product-a-key-metric-definition.md
│   │   └── product-b/
│   ├── marketing/                              # Domain knowledge (not product-specific)
│   │   ├── channels/
│   │   │   └── marketing-channel-overview.md
│   └── document-examples/                      # Few-shot examples for document generation
│       ├── business-requirements-documents/
│       │   └── product-a-brd.md
│       └── strategy-documents/
│
└── projects/                                   # Active and completed work — organized by product
    ├── product-a/
    │   ├── reports/
    │   │   ├── key-metrics/
    │   │   │   ├── key-metrics-report-2026-01.md
    │   │   │   └── key-metrics-report-2026-02.md
    │   │   └── business-reviews/
    │   └── project-1/                           # Time-scoped project folders
    └── misc/                                    # Cross-product or exploratory work
        └── project-1/

The names of the files in your knowledge base are another important clue your agents can use to determine if they’re relevant. You are less likely to know what’s in a file called “document” than you are with a file called “product-a-overview-and-positioning.” It’s the same for an agent considering whether to read that file.

Once you open a file, you probably don’t want to read all of it to know if it’s useful. A summary at the top is helpful to get the gist of the content and determine if you should continue reading. You can do the same in your knowledge base files with a few lines at the top that tell the agent what this file is about and whether it’s worth reading, like:

---
title: "Project Orion Launch Brief"
product: "Orion Analytics Dashboard"
status: active
date_updated: 2026-03-10
summary: Redesign of the analytics dashboard to support real-time data streaming. Goal is reducing time-to-insight for enterprise customers by 40%. Use this file when working on any Orion-related marketing, messaging, or launch planning tasks.
---

You can see a full example of a knowledge base file at the bottom of this post.

You used LLMs to help you create your persona prompts, and you can also use them to set up your knowledge base. Whether you’re a type-A person who has all of your MBA notes from 15 years ago scanned and searchable, like me, or more of a go-with-the-flow type, this is another place to let an LLM take the first pass. Don’t get too hung up on the details or strive for perfection. That’s why there is a feedback loop in the process: so you can move quickly and make improvements that address real issues.

It’s the same approach as in the last post: you let the LLM take the first pass and then refine as you go. The earlier you build that habit, the faster the whole framework is going to pay off.

Your team is more knowledgeable, but that isn’t enough to guarantee you get consistent, high-quality work. You still need to give them an employee handbook and a set of standard operating procedures to fix the second type of gap: behavioral gaps.

Resource: prompt you can use to design a knowledge base for your AI agents

I'm building a knowledge base for a team of AI agents. The knowledge base is a collection of markdown and CSV files that any agent on the team can access. Not every agent needs every file — they'll pull in what's relevant to each task.

Help me figure out what files I need. Interview me with the following questions, one at a time. Ask each question, wait for my response, then move to the next.

1. What agents did you set up? For each one, briefly describe the role and the kinds of tasks they'll handle.
2. What information comes up repeatedly across your agents' work, regardless of which agent is doing it? Think about context that any of them might need.
3. Are there specific products, projects, or workstreams that your agents support?
4. Are there reference documents or standards that multiple agents would need access to?

After the interview, propose a knowledge base structure:
- A folder layout with descriptive names, separating role-specific files (broadly useful across projects) from project-specific files (tied to a particular product or workstream)
- A list of recommended files, each with a one-line description of what it should contain and whether it's role-specific or project-specific

Based on what you learn about my setup, propose a YAML frontmatter format for my knowledge base files. Every file should have a title, status, date updated, and a short summary describing what the file contains and when an agent should use it. Beyond those basics, add fields that make sense for my situation — for example, a product field if I support multiple products, or a domain field if my agents span different areas of expertise. Explain why you chose the fields you did.

Include the proposed frontmatter in each recommended file.

Keep every file focused on a single topic. Aim for files that are 1-2 pages, not 10. If a topic is too broad for one file, split it.

Example: what a knowledge base file looks like

---
title: "Project Orion Launch Brief"
product: "Orion Analytics Dashboard"
status: active
date_updated: 2026-03-10
summary: Redesign of the analytics dashboard to support real-time data streaming. Goal is reducing time-to-insight for enterprise customers by 40%. Use this file when working on any Orion-related marketing, messaging, or launch planning tasks.
---

## What is Project Orion?

Orion is a redesign of the existing analytics dashboard for enterprise customers. The current dashboard refreshes data every 15 minutes. Orion introduces real-time streaming so customers see their data as it happens.

This is not a new product. It's a major upgrade to an existing product that enterprise customers already use daily.

## Business goal

Reduce time-to-insight for enterprise customers by 40%. The current delay between data generation and dashboard visibility is the #1 support complaint and the #1 reason prospects cite for choosing competitors.

## Target audience

- Primary: existing enterprise customers (upgrade path)
- Secondary: mid-market prospects evaluating analytics platforms for the first time
- Not targeting: SMB or self-serve customers (Orion is enterprise-tier only)

## Key messaging pillars

1. **Real-time, not near-time.** Competitors claim "real-time" but deliver 5-minute delays. Orion streams data in under 10 seconds.
2. **Zero migration effort.** Existing dashboards carry over. No rebuilding, no re-learning.
3. **Built for the analysts, not just the admins.** The redesign focuses on the daily experience of the people who actually use the dashboard, not just the people who set it up.

## Competitive context

- Competitor A offers real-time but requires a full dashboard rebuild on migration
- Competitor B has a faster refresh rate (5 min) but no true streaming
- Our advantage is real-time streaming with zero migration friction

## Launch timeline

- Beta: April 2026 (50 enterprise customers)
- GA: June 2026
- Marketing launch campaign begins two weeks before GA

## What this file does not cover

- Pricing and packaging (see `orion-pricing-and-tiers.md`)
- Technical architecture (see `orion-technical-specs.md`)
- Full competitive analysis (see `competitive-landscape.md`)

What started as a sketch is now core to how I use AI agents to do things faster and better at work and at home. It’s an approach that naturally guides you toward the context engineering best practices that improve LLM output.

The reason this approach works is that it uses one of the two available levers to improve how well a best-in-class large language model (LLM) works for you.

1. Fine-tuning: this is where you take an LLM and train it further using your own data so it becomes more specialized for your needs.
2. In-context learning: giving the LLM the right expertise (persona), knowledge (context files), workflows (skills), and rules in each session (team rules).

For most people, fine-tuning is going to be out of reach. Even if you could fine-tune a model, you’d have to retrain it repeatedly to keep up with changes in your work. Otherwise, the model would grow stale. In-context learning is how you keep the model relevant between retraining cycles, and for most people, it’s the only lever available.

It all starts with building your team.

Define the roles

The first step is to define the roles for your team by identifying the groups of similar tasks you do over and over again. It might help to start with pen and paper like I did.

List out the things you do at work or at home on your computer. Don’t overthink it; just write them down. Then group the ones that are similar in terms of how you approach them (required behavior) and the information you need to do them (required context).

The groups of items you do most often and that take the most time are the best candidates for roles because they’ll benefit the most from ongoing improvement. On those tasks, you can work with an agent frequently enough to spot gaps that lead to improvements that you’ll continue to benefit from.

As you build your team, keep in mind that the ideal number of direct reports for a manager tends to be 8-9 . This principle also applies to AI agents. The more you have, the more complex it gets to keep up with the feedback and improvement loop for each one. Remember, you’re not building a department. You’re building a team.

In my role as a Sr Product Marketing Manager, I’ve landed on five agents that I work with daily:

I’m setting up a different team at home: an editor, financial advisor, and personal trainer.

Create the personas

Creating the personas will be quicker than you think, because you’re going to use AI to help create them.

Start with the role that you feel the most comfortable defining. Spend a little time thinking about how you’d want the agent in that role to behave. What should it do? What should it never do? Don’t overthink it. You’re not going for perfection. You’re going for something that you can provide an LLM, like ChatGPT or Claude, to help it create a persona prompt for you. Keep it simple so you don’t get hung up on this step. The feedback loop will improve it over time.

Next, start a chat with the best-performing model you have access to. Regardless of what you are using, if you have the option to select a model, select the latest frontier model from that provider. Starting with a better quality model means you’re more likely to start with a good persona prompt. That’s less distance to close with the feedback loop to get to an agent that starts to materially improve the work it was created for.

In the chat, ask it to help you create a persona prompt. Let it know the role you have in mind, the type of work you’re going to use it for, and how you want the agent to behave. I’ve included a prompt at the end of this post that you can copy into Claude, ChatGPT, Gemini, or your tool of choice to walk you through creating your persona prompt.

Review what the model writes for you, and iterate on it as needed. If something doesn’t sound right, let the model know what the issue is and ask it to update the prompt. You don’t need to use any kind of special language to get this done. Treat it like a conversation you’re having with a coworker to improve a document. And remember what I mentioned before: there’s no need to be precious about this. This is a starting point that you’re going to refine through the feedback loop.

A good prompt is going to define the agent’s identity briefly (1-2 sentences) and focus primarily on behavioral guidance for the agent. This includes how to approach tasks, standards to enforce, and what to prioritize. It’s also helpful to include specific things the agent shouldn’t do in this type of role. For example, I don’t want my data wizard to ignore a sudden spike or decrease in a metric, because I’ve learned that generally doesn’t happen without some external factor causing it.

After you create your persona prompts, take a step back and think about how you created them. You delegated the persona draft to an LLM. That’s not a shortcut. You’re not cheating. That’s the whole point of creating your AI team. You’re going to be delegating more and more work to them, and this is the first point in the framework where you do that.

As you build trust with your agents, you’re going to start to delegate more to them: bigger tasks, more autonomy, more trust. This is exactly what it’s like to be a manager when you’re working with a new employee. You’re initially close to what they’re doing, you build trust, and then you start to give them the room to run. That’s when you start to really see the benefits of adding that employee to your team. It’s the same thing here. The earlier you get comfortable delegating work to the AI agents, the faster everything in the framework will start to pay off.

Set up the agents

The last thing you need to do to build your team is set up the agents by loading the persona prompt into whatever tool you’re using. The specifics are going to vary based on the tool you’re using, e.g. Claude Code versus Kiro CLI. I’ll cover how to do this in more detail in an upcoming post in this series. For now, you just need to remember that the persona prompt is the foundation for each agent on your team.

Building your AI team is straightforward. You’re the expert at what you do and how to do it well. Use your experience and expertise to guide an LLM to build persona prompts for AI agents to fill your open roles. That gets them hired. The knowledge base you’ll create is what gets them up to speed and delivering high-quality work for you.

Resource: prompt you can use with an LLM to help create your persona prompts

I need you to help me write a persona prompt — a set of instructions that will shape how an AI agent behaves every time it runs. Think of it as a job description the AI reads before every conversation.

Before writing anything, interview me. Ask these three questions one at a time, waiting for my response before moving on:

1. **What role does this agent play?** What's the domain and who does it serve? (If you know what platform or tools the agent will use, mention them — but don't worry if you're not sure.)
2. **What kinds of work will it do?** Describe the typical tasks or situations the agent will help with. Think about what a good day looks like — what does the agent do well?
3. **What behaviors matter most?** How should the agent approach its work? What should it do when it's unsure? Are there things it should always or never do?

After the interview, generate the persona prompt. Use what I told you as the foundation, but add your own recommendations — behaviors or guidelines that would make this agent more effective for the role, even if I didn't mention them. Call out anything you added so I can review it.

Follow these rules when writing the prompt:

### Focus on behaviors
- Describe what the agent should *do*, not what it *is*. "Start by understanding the full situation before proposing solutions" is a behavior the agent can act on. "You are thorough and thoughtful" is not — it's a personality trait, and the agent won't know how to translate that into action.
- Frame instructions as conditional guidance: "When X, do Y." This gives the agent concrete decision points rather than abstract qualities to live up to.
- If a behavior only applies sometimes, state the condition.

### Hit the right altitude
- Write at the level of a clear team lead briefing a competent new hire — not a legal contract, not a vague mission statement.
- Be specific enough to prevent the mistakes that actually happen, but flexible enough to let the agent use judgment in novel situations.
- Prefer "when X, prefer Y" over rigid step-by-step procedures. The agent needs guidance it can apply across situations, not a script that breaks the moment something unexpected comes up.

### Structure for the role
- Let the role dictate the structure. A coding agent needs different sections than a research agent or a writing coach. Don't force a template.
- Always lead with identity and scope — one or two sentences that establish who this agent is and what it does.
- After that, organize the remaining instructions into whatever sections make sense for this specific role. Use headers and bullets so the instructions are easy to scan.

### Keep it short
- A persona prompt competes with the user's actual questions and content for the agent's attention. The longer the prompt, the less room the agent has to focus on the real work.
- Aim for the shortest prompt that fully captures the desired behavior. If a line doesn't change how the agent acts, cut it.
- Leave out anything the agent would already know or can figure out from context.
- When in doubt, leave it out. A lean starting point that the user can build on is far more useful than a bloated prompt full of rules that haven't been tested. The user will discover what's missing by working with the agent and can add rules as needed.

### Output format
- Output only the final persona prompt, ready to use.
- After the prompt, add a short "Additions" section listing anything you added beyond what I described, with a one-line rationale for each. This section is for my review — it's not part of the persona prompt itself.

When you are managing a team, you have to scope the roles for your team and fill them with people who can be successful in those roles. To do that, you hire folks with the right backgrounds and experience, both of which inform how they’ll do the work. Every member of my AI team has a persona prompt with specialized behavioral guidelines I want for the role they’re filling. For example, my Data Wizard prompt has guidance around digging into irregularities in data, like when a metric suddenly spikes up or down.

You want to ensure your team has the information they need to do their work, like a wiki with product information, target audience insights, document templates, and standard operating procedures. You’re giving them the context they lack when they step into that role, and I do the same with a structured knowledge base, project-specific context files, and reusable skills that describe how to do specific tasks.

Lastly, you want to develop your team with feedback and guidance tailored to them. I’m using feedback loops to capture issues and improve their work via the persona prompts, team rules, skills, and knowledge base.

I built my team through trial and error, but I now have a framework for how to do this, which I’m breaking down into two phases: build your team and manage your team. Each phase contains three steps:

Build your team

1. Define the roles
2. Create the personas
3. Set up the agents

Manage your team

1. Work with your team
2. Spot the gaps
3. Apply fixes at the right level

In order to build the right team, you need to figure out what roles you need to fill. Start by identifying similar types of tasks that you have to do often in your role; those groups of tasks represent job openings that you could fill with an AI agent.

I’ve found that my most useful agents are the ones that I work with often, since that supports the ongoing cycle of improvement, so I try to avoid creating a custom agent with too narrow a scope that I won’t work with often. I also don’t want to have to juggle a team of 30 agents every day. I think the best practice of keeping a manager’s span of control to ~8 or fewer employees also makes sense in this context.

You’ll then build a persona prompt for each job opening that defines the ideal candidate’s identity and how they work. Once you have the persona, that serves as the foundation for the AI agent you’ll set up in a tool like Kiro or Claude Code to be on your team.

To manage your team, you have to understand their strengths and weaknesses, and that means working closely with them. You’ll want to work with the right agent for the task at hand.

As you work with them, you’ll start noticing recurring gaps that you need to address to improve your team. Some of those will be knowledge gaps, where the team needs more information, and others will be behavioral gaps, where your agents aren’t doing something the way you’d like them to or expect them to.

Based on the gap, you’ll want to address the situation at the right level. That might entail adding a new context file to the team’s knowledge base or updating an agent’s persona. These small tweaks will start to lead to big improvements, but this isn’t a set it and forget it kind of deal. It’s a continuous management process that never really ends.

I’ve noticed that my team is producing better work in less time with this approach, but I don’t have an objective way to measure or validate that. I want to learn more about LLM evaluation techniques so I can get to objective measurement, but in the meantime, I have some validation from others at work.

First, I’ve started to get compliments from copywriters on the draft marketing copy I’m writing with my Copywriter AI agent. Second, I shared the first draft of a monthly business review document my team wrote with my counterpart on the product side. I let her know it was all AI-generated (the analysis and the write-up), and I asked her to review and check if there was potential there. She was so impressed with the quality of the MBR, she started asking me questions about how I’d put it together and what my setup was. Lastly, I was able to write a good business requirements document from scratch in one day because the infrastructure was already in place.

Building and managing an AI team is much harder than just using a chatbot or the default agent that you get with something like Kiro or Claude Code. It takes upfront work to scope the roles, build the personas, and create the infrastructure to support the ongoing improvement.

A lot of that work will happen before you start to see the results, but it will start to compound. Pieces will build on top of other pieces, and things will get faster, both because you’ll start to optimize the process to your work style and because your agents will get better. I don’t have the data yet to prove this is better, but I’ve seen enough to think the effort is worth it. So I’m currently setting up the same approach at home with Claude Code.

That’s a high-level overview of my managed AI framework. I’m going to dive deep into each area of the framework with separate posts on building your team, setting up the knowledge base, working with skills and team rules to set expectations and requirements, and creating a feedback loop to drive the ongoing improvement. I’ll then cover how I’ve implemented this in Kiro CLI at work and Claude Code at home. I’ll end the series with a post on the learnings and best practices I’ve picked up along the way. By the end, you should have a good roadmap with explicit examples to allow you to set up your own team.

I want to develop my skills, deliver better results, and spend more time with my family. This framework is how I’m doing that. It’s a tool-agnostic approach that can help move you away from using one-size-fits-all tools to building an AI team that’s tailored to your needs and able to deliver better results for you.

I began learning about LLM-based gen AI in earnest in 2024. I read all the most popular books at the time, but my exposure remained primarily theoretical. I learned how LLMs work fundamentally, but the biggest practical takeaway was the idea of assigning a persona to chatbots to improve their output. That’s basic prompt engineering, e.g. “You are a copywriter with 15+ years of experience in consumer tech. Help me write a marketing email about this product.” On the rare occasion I used a chatbot, I always remembered to assign it a persona.

Last August, I joined a project at work that was the turning point for my AI enthusiasm. In that project, I had to manually build a large JSON file that would require a lot of ongoing updates. On a whim, I decided to see if I could use a chatbot to write a Python script to go from JSON to Excel and vice versa. That would allow me to make updates in Excel, which would be faster, and then generate the JSON programmatically, reducing the risk of errors. Within 30 minutes, I had a working prototype that ultimately saved me countless hours over the coming months.

I’m OK at Python, but I realized LLMs are much better. So, I began to write a lot of Python scripts this way to automate repetitive or time-consuming tasks, like resizing images or creating Word docs from copy I had in Excel files, in order to stay on top of the workload for the project.

I was soon using chatbots weekly for other things. I began paying attention to what model the chatbot was using, switching to the latest frontier models whenever possible. That had a noticeable impact on the quality of the copy and ideas that I was getting from the chatbots. Especially when I paired better models with a well-crafted persona and a collaborative approach.

I got tired of typing different versions of the same persona prompts whenever I started a new chat. Often I was too busy and moving too quickly to write something better than “You’re a [blank] with X years of experience.” It happened enough times that I realized I could save some time without sacrificing quality by creating reusable persona prompts for different types of tasks. At first I wrote them myself, and they were ok, but not great. By asking the chatbot to help me craft the persona, I was able to take them to the next level. I kept the prompts in Word docs so I could copy and paste them into the start of my chat sessions depending on what I was working on. To save a little more time, I’d pin the chat in the sidebar and rename it to something like “Copywriter” or “Data Wizard,” so I could quickly return to the right chat based on what I was working on.

I’d work with the same chat for up to a week because I wasn’t aware of context rot (where long, ongoing conversations with LLMs start to produce worse results). That’s ok though, because it led to another breakthrough for me. I started to ask the chatbots at the end of each week how we could improve the persona prompt I’d initially started the conversation with based on our interactions. The chatbot would suggest some ideas, and after a few revisions and back-and-forths, it would write an updated version that I would use in the next week’s chat. That created a feedback loop to improve my personas on an ongoing basis.

For example, I learned that LLMs guesstimate how long copy is after noticing that their character counts for marketing copy were often wrong. That’s not great when you’re writing ad copy that has specific character constraints. I updated my Copywriter persona prompt with instructions to count each individual character when writing against copy constraints. After that, I no longer had to worry about getting copy options that were too long for the character constraints I’d provided. It was like giving an employee feedback, except the chatbot immediately incorporated that feedback into how it worked.

That’s the core idea in the framework that is now guiding my AI usage: that I’m the manager of a team of AI agents. They’re incredibly smart, but also kind of dumb. They have a lot of expertise, but they’re also clueless about the specifics of where I work and what I’m working on.

The more effort I put into developing my team and providing what they need, the better the quality of work I get from them. And the benefits are compounding over time. I spend less time correcting simple issues and more time refining and improving what we’re working on. I spend less time providing the same context over and over again to my agents. Instead, they have a growing knowledge base to inform their work. I spend less time tweaking the documents they create because they have actual examples to refer to of the various types of documents I have to write. I’m capturing feedback and improving every aspect of my setup daily, and that makes it even better the next day and miles ahead of using a run-of-the-mill chatbot.

I’m going to go into more detail about this framework in coming posts and explain how I’ve implemented it in Kiro CLI , an AI coding tool I use at work primarily for non-coding tasks, and how I’m now implementing it in Claude Code at home.