# Autopilot+
## β *`What`* is it?
> ## *`AI-powered browser automation tool`* that uses advanced LLMs and computer vision to automate web-based workflows.
### ...Features
* [x] **Automates** complex web-based processes.
* [x] **Human-in-the-Loop** setup!
* [x] **AI-Agent Swarms** based workflows
* [x] **100% SaaS** based subscription model!
* [x] **100% AI-driven browser automation** tool.
* [x] Instead of relying on traditional DOM/XPath-based automation, it uses **visual understanding** and **LLMs to interact** **with websites directly**.
* [x] Uses swarm of agents to fully understand websites then plans and executes AI actions as instructed by you:
1. **Interactable Element Agent**: This agent is responsible for parsing the HTML of a website and extracting the interactable elements.
2. **Navigation Agent**: This agent is responsible for planning the navigation to complete a task. Examples include clicking buttons, inserting text, selecting options, etc.
3. **Data Extraction Agent**: This agent is responsible for extracting data from a website. It's capable of reading the tables and text on the page, and extracting the output in a user-defined structured format
4. **Password Agent**: This agent is responsible for filling out password forms on a website. It's capable of reading the username and password from a password manager, and filling out the form while preserving the privacy of the user-defined secrets.
5. **2FA Agent**: This agent is responsible for filling out 2FA forms on a website. It's capable of intercepting website requests for 2FAs, and either requesting user-defined APIs for 2FA codes or waiting for users to feed 2FA codes into it, and then completing the login process.
6. **Dynamic Auto-complete Agent**: This agent is responsible for filling out dynamic auto-complete forms on a website. It's capable of reading the options presented to it, selecting the appropriate option based on the user's input, and adjusting its inputs based on the feedback from inside the form. Popular examples include: Address forms, university dropdowns, and more.
### ...more Technical Details
* [x] It **leverages large language models (LLMs)** and computer vision to streamline web workflows.
* [x] By integrating **advanced machine learning techniques**, it can intelligently interact with web interfaces UI, mimicking human behavior with remarkable accuracy.
* [x] **Workflows** are a way to chain multiple tasks together to form a cohesive unit of work.
For example, if you wanted to download all invoices newer than January 1st, you could create a workflow that first navigated to the invoices page, then filtered down to only show invoices newer than January 1st, extracted a list of all eligible invoices, and iterated through each invoice to download it.
* [x] Another example is if you wanted to automate purchasing products from an e-commerce store, you could create a workflow that first navigated to the desired product, then added it to a cart.
* [x] Second, it would navigate to the cart and validate the cart state.
* [x] Finally, it would go through the checkout process to purchase the items.
* [x] **Supported workflows** include:
* [x] Navigation
* [x] Action
* [x] Data Extraction
* [x] Loops
* [x] File parsing
* [x] Uploading files to block storage
* [x] Sending emails
* [x] Text Prompts
* [x] Tasks (general)
* [x] (Coming soon in Beta now) Conditionals
* [x] (Coming soon in beta now) Custom Code Block
* [x] **Core components:** implemented in the main source code directory
* [x] task orchestration
* [x] model integration
* [x] and visual element detection
* [x] **2FA methods** to allow you to automate workflows that require 2FA:
* [x] QR-based 2FA (e.g. Google Authenticator)
* [x] Authy Email based 2FA
* [x] SMS based 2FA
* [x] **Password manager** integrations:
* [x] Bitwarden
* [x] 1Password
* [x] LastPass
* [x] **Downloading files from a website** capabilities. All downloaded files are automatically uploaded to block storage (if configured), and you can access them via the UI.
* [x] **Data Extraction:** You can also specify a `data_extraction_schema` directly within the main prompt to instruct the AI exactly what data you'd like to extract from the website, in jsonc format. The output will be structured in accordance to the supplied schema.
* [x] **Form Filling:** natively capable of filling out form inputs on websites. Passing in information via the `navigation_goal` will allow the AI to comprehend the information and fill out the form accordingly.
* [x] **Livestreaming:**
* [x] Allows you to livestream the viewport of the browser to your local machine so that you can see exactly what the AI is doing on the web.
* [x] This is useful for debugging and understanding how the AI is interacting with a website, and intervening when necessary
### ...Usage Tips
* [x] The AI Agents are really good at **carrying out single target goals**.
* [x] **Too many instructions** will confuse the AI.
* [x] **Be explicit about goals** is very important.
* [x] e.g. if you're generating an insurance quote, let it know very clearly how it can identify it has accomplished its goals.
* [x] Use words like "COMPLETE" or "TERMINATE" to indicate success and failure modes, respectively.
* [x] **Chaining:** Workflows can be used if you'd like to do more advanced things such as chaining multiple instructions together, or securely logging in.
* [x] If you need any help with this, please feel free to book some time with us. We're always happy to help you on your journey!
{% embed url="" %}
.png?alt=media&token=1ad2b236-6d57-4d47-8ef9-7aac47cf4870)
