OffSec's New OSAI+

... and I passed! 80/100 points, with 75 needed for a pass. Nice! Now it is time to share my adventure with the world!

Pre-course

Back in November 2025, I realized I urgently needed to start bringing AI into my pentesting work. I was already using AI back then, just not specifically for work. I played with Gemini CLI when it came out, and I think I installed it on my Kali VM faster than anyone else (at least I want to think so :D). It was not that great at pentesting. It could not find basic issues and would burn through all the free tokens just trying to find the initial entry point into an easy HTB machine. So I put it aside and mostly used it for report writing and fixing my grammar.

Everything changed when the hype around vibe coding started growing. I realized there would soon be plenty of people not only vibe coding, but bringing AI into their existing infrastructure and code. Around the same time, OffSec came out with its new AI red teaming course, OSAI. It immediately caught my attention, especially its coverage: MCP testing, RAG testing, embeddings testing... wow! I was sold. So I bought it almost right away and got an extra month of lab access.

I waited a couple of months, then on April 1st (what a coincidence) I jumped into the course.

The course itself

I must admit, the material itself is amazing! I learned a lot of new technologies and techniques covering pretty much everything AI related, starting with bypassing security restrictions in a regular AI chat and finishing with testing the infrastructure where LLMs are trained and hosted. The most interesting topics for me were A2A, MCP, and RAG testing. The vector embeddings section was the hardest to understand. That one twisted my mind a bit :D

It took me roughly 1.5 months to go through the material and complete the course. Studying nonstop after work and over the weekends took its toll, but the course was so interesting that I could not stop. I started on April 1st and finished on May 18th. It felt like I flew through the material and all the labs. I will talk about the labs in the next section, but first, the biggest downside of the course itself was...

... the lack of videos. When I pay that much money for a course, I expect some video content to go with the written material. I am sorry, but not having videos for topics like vector embeddings was not acceptable. I know everyone has their own learning preferences, but the lack of options is not okay at this price. In the end, I did not need the videos, but they would have saved me at least 10 hours of studying.

Labs

And this is where most of my complaints about OSAI+ begin...

The labs themselves were decent enough. They helped me build practical skills based on the material I had learned. Some also made me think outside the box or research a topic further to provide written answers. After completing them, I gathered the commands and methodologies I had picked up into a single file, which did help during the exam, by the way (more on that later). So yeah, the labs were okay... when they worked.

During my first pass through the material, I noticed that around 20% of the labs had issues. Either they were broken and the intended solution did not work, or they simply would not boot properly. I am not entirely sure how QA is handled on OffSec's side, but it was pretty frustrating. I had to skip broken labs and come back once they were fixed. The longest one took two weeks, I believe.

Before getting into the challenges, I want to mention the capstone challenge. It was really good, except for the last flag, which I did not get. I think I know the intended path I was supposed to take, but I was too exhausted to spend more time on it, so I gave up. It might have involved some Windows post-exploitation step, but I was there to learn how to red team and pentest AI and related systems, not Windows... which brings me to the challenges.

Challenges

When I finished the course material and labs, there was literally nothing left to do. Nothing! The challenges were unavailable from the time I started on April 1st until the last week of June. That was almost three months down the drain. If someone bought the course after March 31st and could not get an extra month of lab access, they would have had only a week or two at most to tackle them. People complained about this in OffSec's Discord, and I agree with them. I wasted over a month waiting for the challenges to be released. When they finally appeared, I got so nervous that I started on them immediately.

The challenges themselves were decent enough. Some were really hard, while others were straightforward. I asked questions in Discord to solve a few of them, and I learned a lot of new AI-related and Windows-related techniques.

Some of the machines in the labs did not work properly... which I had come to expect by then. Restarting a lab n times would eventually help, but it was annoying. In my opinion, some labs also focused too much on Windows for an AI testing course.

Another issue was that a new lab appeared two or three days before the exam, and I did not do it. I felt ready enough, so I went into the exam as I was, with four of the labs completed.her issue was that a new lab appeared two or three days before the exam, and I simply did not do it. I felt ready enough, so I went into the exam as I was, with four of the labs completed.

I completed the last two labs with a different approach. I wanted to try hacking with AI. Instead of human-in-the-loop testing, I let the AI test on its own while I steered the ship. And wow, it was good. Sonnet 5 had just come out, and I mainly used that for the labs. I must admit: WOW! It was really good. I switched to Opus only once or twice, but Sonnet did a lot of cool work on them.

That brings us to the main topic of this article: the exam. Before that, though, let's talk about how I prepared.

Preparing for the Exam

When I finished the labs, the exam guide came out. After reading it, I learned that full AI usage was permitted. The exam was fully open book, and I could use LLMs in any way I wanted. I confirmed this with OffSec, then got to work.

I prepared two AI providers for the exam. At the time I took it, the model that worked best for my setup was GPT-5.6-Sol, with Opus 4.8 as the runner-up. GPT-5.6 was my main driver during the exam, while Opus was the helper. I think that was one of the best decisions I made... more on that later. I used them through Codex and Claude Code. For this kind of exam, I found agentic tools much more useful than regular chat interfaces.

These are some of the skills I installed:

1. Caveman for both (token saving)
2. Humanizer for both (report writing)
3. Awesome Claude Code for CC

This stack of skills and plugins worked well overall. Caveman helped me save tokens, Humanizer helped with the report prose, and Awesome Claude Code gave CC some guardrails with partial HIL in place. It did not magically solve the reporting problem, though. I still had to tell the agents exactly how I wanted the report structured and what evidence they needed to preserve.

My recommendations

Check the Current Rules

The rules I described above applied to my exam attempt. Read the current exam guide and confirm anything unclear with OffSec before building your setup around AI. And, obviously, apply for Anthropic's CVP and OpenAI's ChatGPT Cyber program.

Test the Whole AI Workflow

Models and agent runtimes change so quickly that I would not unthinkingly copy my choices. Test them again shortly before your own exam, make sure they work inside your prepared workspace, and have a second, or even a third provider ready in case the first one starts struggling.

Most of the people I spoke with struggled to get LLMs to work through the labs. A big part of the problem was missing proper context for AI to use. The models did not know enough about the legitimate testing workflow and methodology they were expected to follow. You need to solve that before the exam. Use the labs to build and tune your workflow, then validate it against the challenges. By exam day, it should work at least 95% of the time (a little statistical hypothesis for you there :D). If you want AI to help during the exam, you need to do your homework on it first. Otherwise, you will be banned from using the top-tier models.

Prepare Your Infrastructure

It is really, and I mean REALLY, important to prepare your infrastructure for this exam. Choosing powerful LLMs is only one part of it. The agents need a clean workspace, persistent instructions, a place to store their state, and a reporting pipeline that already works. You do not want to discover halfway through the exam that an agent ignored your instructions, forgot a credential, or cannot build the final PDF properly.

The workflow matters just as much as the models. Set up separate files for targets, credentials, routes, evidence, and report progress. Do not trust a long chat session to remember everything for you. I also recommend giving each agent persistent project instructions through AGENTS.md, CLAUDE.md, or whatever file it reads natively. That way, a fresh session can pick up from the files instead of trying to reconstruct half the exam from chat history.

Once the exam starts, focus on a reliable passing route before spending too much time on optional branches. Reconsider your priorities whenever you gain a new credential, route, or level of access.

Write the Report as You Go

And please, actually write the report as you go. After every foothold, new credential, privilege change, pivot, or flag, save the exact commands and useful output, take the screenshots, and update the relevant report section. Give the screenshots boring but predictable names so you can find them later. Leaving all of that until 4 a.m. was easily my biggest mistake.

Test the Boring Parts

Before the exam, make sure the report builds, the screenshots render, code blocks fit on the page, and the final archive contains exactly what it should. A successful PDF build does not mean the report actually looks good. Open it and check it yourself.

My Setup

I originally built my OSAI Exam Copilot for my own exam (which I am open-sourcing for anyone to use). Afterward, I cleaned up the reusable parts and left out all private exam material so I could share the setup with the community. It contains the same foundation I used: agent instructions, empty ledgers and report templates, and the report tooling. It is not a solution pack, just the workflow that worked for me. If it saves someone else from rebuilding everything under exam pressure, or from sorting out their report at 4 a.m., then sharing it was worth it.

On to the Exam Itself

The exam itself was fair. It was hard, yes, but that was expected from an open-book exam. I got access to an internal environment pretty quickly. It took me some time to understand what was going on, but I earned 30 points through Chain 1 and another 50 through Chain 2. Unfortunately, that is where I stopped. Because I had let AI write my report, I spent three or four hours rewriting it and collecting screenshots. I finally finished at 4 a.m., gave up, and went to sleep.

My AI usage was 48% on GPT and 30% on Claude, which is decent. But again, I did not complete two boxes, and they could have eaten up the remaining allowance.

I must admit, my mistake was not letting AI write the report. It was failing to prepare the AI for how I wanted the report written. Once it understood what I needed, I flew through the PoC collection and report finalization. Preparation really is key here.

The worst part for me was not the exam. It was not being able to enjoy music during it... You are not allowed to wear headphones anymore, so I had to use my monitor speakers. They sounded awful, and I ended up turning them off. Just why...

Confirmation of the Certification

After submitting the report, I waiting for 2 days to have the report checked on and verified. On July 29th, I got the confirmation that I have successfully passed the exam and became OSAI certified!

In Conclusion

Welp, that is how it went! The whole experience was the proper OffSec treatment. If you think this course will be something different, nope. It is still OffSec developing it.

In my opinion, the material was worth going through. It is extensive and very interesting. The labs and challenges were decent enough, though I would prefer not to restart them a million times to get them working.

If you plan to take the exam, do not stop at studying the course material. Prepare your infrastructure and rehearse the reporting workflow before the timer starts. A little extra work there would have saved me hours when I was already exhausted.