SwarmAgentic delivers the first fully automated pipeline for agentic system generation, jointly optimizing from-scratch agent creation, role specialization, and collaboration through language-driven particle swarm optimization (PSO).
<p align="center">
<img src="misc/main.png" alt="SwarmAgentic overview diagram" />
</p>
<p align="center"><em>Figure 1. Overview of the SwarmAgentic pipeline. Starting from only a task description and objective, the system autonomously generates and optimizes multi-agent architectures through language-driven particle swarm evolution. Each iteration involves execution, LLM-based failure analysis, and targeted refinement, yielding a fully functional and collaboratively optimized system built from scratch.
</em></p>
cd travelplanner/swarm
python test.py --particle_idx -1
Train (Creative Writing)
cd creative_writing
python pso.py --max_iteration 10 --dataset data/data_100_random_text.txt
Common flags
For pso.py:
--max_iteration: maximum number of PSO iterations.
--settings: temperature settings for each particle (e.g., --settings [0.2 0.4 0.6 0.8 1.0]).
--model: model name to use (default: gpt-4o-mini).
--max_workers: maximum number of worker threads (default: None).
--save_dir: directory to save results.
For TravelPlanner only:
--resume: restarts optimization from an existing checkpoint.
--state_idx: chooses which saved state to resume from (-1 for the latest).
For test.py:
--particle_idx: chooses which saved particle to evaluate (-1 grabs the latest entry in save.jsonl).
--model: model name to use for execution (default: gpt-4o-mini).
--eval_model: model name to use for evaluation (default: gpt-4o-mini).
Data and Evaluation
Data layout: Each domain ships with a data/ directory containing train, and test splits. File formats are typically plain text or JSONL with tab-separated fields for prompts and expected responses.
Preparing new data: Drop cleaned files into the domain data/ folder.
Evaluation artifacts: test.py writes metrics to results/<run_name>/metrics.json and stores generations for manual review. During PSO, aggregate scores and trace summaries accumulate in save.jsonl.
You can easily adapt SwarmAgentic to new domains by following these steps:
1. Define Your Task (prompt/base.py)
Define the high-level task description that will guide the multi-agent team generation:
TASK_MINI = '''Your domain-specific task description here...'''
FUNCTION_DESCRIPTION = """
Description of what the forward function should do for your task.
The function signature should be 'def forward(team):'.
Define what the function should return.
"""
Example: For creative writing, TASK_MINI describes the narrative coherence task with specific constraints.
2. Implement Domain-Specific Evaluation (eval.py)
Create an evaluate() function that scores agent outputs. You'll also need to format task instances in pso.py:
def evaluate(llm, task, response):
"""Evaluate the quality of the agent's response.
Returns:
(score, problem): score is the evaluation metric,
problem describes issues if score is not perfect
"""
# Implement your domain-specific evaluation logic
# For creative writing: coherence scoring (1-10)
# For reasoning tasks: correctness checking (0/1)
# For planning tasks: constraint satisfaction
The framework will automatically generate specialized multi-agent teams for your task and optimize team structure and coordination through PSO.
3. Register the Domain Pipeline (pso.py)
Load prompts and evaluators defined above.
Configure PSO hyperparameters (max_iters, population_size, velocity damping) to match the complexity of your task.
Ensure pso.py exports the artifacts required by test.py (save.jsonl).
4. Prepare Your Dataset
Place your training and test datasets under data/ with consistent formatting (plain text or JSONL).
5. Run Training and Evaluation
Train with PSO: python pso.py --max_iteration 10 --dataset data/your_train.jsonl
After training completes, PSO saves all particles to save.jsonl. Choose the best particle and evaluate: python test.py --particle_idx -1
Safety Considerations
[!WARNING]
The code in this repository involves executing untrusted model-generated code. We strongly advise users to be aware of this safety concern. While it is highly unlikely that model-generated code will perform overtly malicious actions in our current settings and with the models we use, such code may still act destructively due to limitations in model capability or alignment. By using this repository, you acknowledge and accept these risks.
Run experiments inside sandboxed containers or restricted user accounts to reduce blast radius.
Scope API keys with least privilege and rotate them periodically.
Inspect artifacts in results/ and logs/ before deploying agent teams in production environments.
Add new domain folders following this template to reuse the PSO drivers and evaluation tooling.
Citing
If you find this project useful, please consider citing:
@misc{zhang2025swarmagenticfullyautomatedagentic,
title={SwarmAgentic: Towards Fully Automated Agentic System Generation via Swarm Intelligence},
author={Yao Zhang and Chenyang Lin and Shijie Tang and Haokun Chen and Shijie Zhou and Yunpu Ma and Volker Tresp},
year={2025},
eprint={2506.15672},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2506.15672},
}