(Ab)using AI to attack M365 and other services to conduct plethora of attacks
It's no secret that AI is being leveraged for attacks but multiple nation state actors. In this post, I'll take you through few tactics of abusing AI for attacking services and how to detect them.
FAQ: Are you encouraging people to attack by disclosing an attack vector?
It's fairly possible that threat actors are already abusing these tools. The intention of this blog post is to highlight an previously undiscovered/non-popular attack. This research hopefully will push defenders to implement measures to thwart this attack. Disclosure of attack techniques is a grey area that is often debated but my intention clearly is for educational purposes and for the betterment of the information security space. I'm not responsible if you decide to be evil and screw other people's lives. Seriously, get a life and stop being evil.
😎 Introduction to workflow automation using LLMs
Browser automation using Large Language models (LLMs) such as GPT-4 is a technique where AI models are leveraged to automate a chain of tasks over a website. While it's possible to automate certain things as filling forms or clicking buttons, etc. The idea is to understand user's instructions in plain English, convert them into automation commands. The following are the phases in a typical browser workflow automation using LLMs:
LLM for understanding user input:
The user's input can be hard to comprehend for a non AI machine as it expects structural information. LLMs can understand what a user wants to achieve and convert that into structural commands that underlying browser automation tools such as Selenium or Playwright can understand.
Process automation using multiple tools:
There are multiple browser automation frameworks like Selenium or Playwright. They typically expect a structured instruction set for a task. A sequence of tasks is generated from previous step.
Decision making (Feedback loop):
Sometimes, they need to go back and understand output and take a decision based on the output. For instance, in our use case, we simply ask them to check if a login is successful and if it's not, we will need to test another password. How does it know that the login failed? Well, it'll take the output and analyze according to our requirement.
The following platforms are a few browser workflow automation that leverage LLMs: (Please note it's not an exhaustive list)
Open AI Operator
Proprietary
Cloud
LangChain + Playwright
Open Source (MIT/Apache)
Self Hosted
AutoGPT + Browser Agent
Open Source (MIT)
Self Hosted
Browser Flow
Proprietary
Cloud
Zapier AI Actions
Proprietary
Cloud
Browser Base
Proprietary & Open source
Cloud & Self Hosted
🌬️ Consideration while choosing the right platform
There are so many platforms out there that can support browser workflow automation and for choosing the right platform for attack, here are a few considerations
Self Hosted v/s Cloud: If you're performing an attack, you want to keep your OpSec tight. There are advantages to both self hosted and cloud. Self Hosted gives you advantage of moving from one place to another. Cloud does it automatically for you or at least easy for you.
Free v/s Paid: Well, it depends on your budget. If you can spare some money, you should try to opt for a premium platform because a few premium platforms actually have some really good features such as Captcha solving, etc. So it all depends on the budget for your Red Team assessment.
Integrations: A few platforms allow for integrating via an API or custom scripts. This is a big thing for folks who want to integrate with Notebooks or SOAR's (for automated remediation of vulnerable credentials, etc.)
The reason for choosing Browser Base is:
It's free. Doesn't need to host it anywhere with an option to do so later.
No Sign up. Low traceability. (which is bad from a defense PoV but can't help it)
Good UI & UX - trust me it is really good UX.
😊The attack
While there are multiple platforms that can be abused, for the sake of this blog post, I have chosen Browser base's Open Operator platform.
Steps for performing this attack:
Enter this prompt:
While I have chosen this prompt, feel free to mess around with this.
Watch the attack unfold. The prompt might need some change but you get the idea.
🚨Detecting the attack
While the following detection might not be universal for detecting all attempts using Browser Base, it is of medium fidelity from my observations. Assuming that you have Azure Signin logs backed up, here's a simple query to start detecting attempts of password spray.
🚓General Monitoring guidance:
If you don't expect your users to login from Amazon's EC2 service, you can surely be suspicious about the usage of this or any such offensive tools. The reason is that a few services in AWS allows for rotation of IP addresses allowing for bypassing of rate-limiting controls.
Check for risky signin events. These are available in Azure Portal inside the Microsoft Entra ID section. Microsoft Identity algorithm use several parameters to identify suspicious sign in attempts and they can come in handy to investigate compromised credentials.
Make sure your logs are backed up to a central location and ensure that you have at least 30 days of data.
It's most certainly possible that this will not capture all the threat actor activity but the idea is to profile the activity and start somewhere. Watch this space for updated detection content while I work on making this better.
🥳 Mitigation for Microsoft Entra ID
If you are an Microsoft Entra ID customer, the following mitigation will help to ensure that your organization will not fall victim to password spray attack.
Enable Conditional Access Policy:
This can be used to block sign-ins from unknown or suspicious locations. Alternatively, you can enforce MFA for risky login requests.
Enforce MFA for all signins:
MFA can greatly improve resilience against password spray attacks. Learn more about how to enable MFA here.
Implement custom lockout policy:
If you are organization is being a victim of multiple password spray campaigns, you can implement smart lockout and rate-limit controls.
Adopt Passwordless & Phish resistant MFA:
You can integrate FIDO2 compliant MFA such as Windows Hello for Business, Microsoft Authentication with number matching, FIDO2 security keys, etc.
In addition to the above, you can look at the official guidance from Microsoft regarding password spray attacks: Password spray attacks Investigation cookbook.
🥹Final thoughts
AI-based browser workflow automation tools can be abused for malicious purposes, including password spraying attacks, credential stuffing, automated phishing, and web scraping for sensitive data. In a password spraying attack, attackers use AI-driven automation tools like Selenium, Puppeteer, or AI-integrated RPA frameworks to systematically attempt common passwords across multiple accounts without triggering account lockouts. These tools can bypass CAPTCHA using AI-based solvers and mimic human-like browsing behavior to evade detection.
While these tools are very useful for several productivity related things, these can be abused too. It's advised to apply the mitigations outlined above (or relevant things for your platform, e.g.. Okta, etc.)
And finally, don't be evil. ✌️
Last updated
Was this helpful?
