Open-source AI agents are achieving State-of-the-Art results on two different Android AI agent benchmarks.
text
Open-source AI agents are achieving State-of-the-Art results on two different Android AI agent benchmarks.
new_line
new_line
Yesterday, I finished evaluating my Android agent model, deki, on two separate benchmarks: Android Control and Android World. For both benchmarks I used a subset of the dataset without fine-tuning. The results show that  image description models like deki enables large LLMs (like GPT-4o, GPT-4.1, and Gemini 2.5)  to become State-of-the-Art on Android AI agent benchmarks using only vision capabilities, without relying on Accessibility Trees, on both single-step and multi-step tasks.
text
Yesterday, I finished evaluating my Android agent model, deki, on two separate benchmarks: Android Control and Android World. For both benchmarks I used a subset of the dataset without fine-tuning. The results show that  image description models like deki enables large LLMs (like GPT-4o, GPT-4.1, and Gemini 2.5)  to become State-of-the-Art on Android AI agent benchmarks using only vision capabilities, without relying on Accessibility Trees, on both single-step and multi-step tasks.
new_line
new_line
All the information is available on GitHub: 
text
All the information is available on GitHub: 
https://github.com/RasulOs/deki
https://github.com/RasulOs/deki
link
new_line
new_line
I have also uploaded the model to Hugging Face:
text
I have also uploaded the model to Hugging Face:
new_line
new_line
Space: 
text
Space: 
https://huggingface.co/spaces/orasul/deki
orasul/deki
space
resource
https://huggingface.co/spaces/orasul/deki
new_line
(Check the analyze-and-get-yolo endpoint)
text
(Check the analyze-and-get-yolo endpoint)
new_line
new_line
Model: 
text
Model: 
https://huggingface.co/orasul/deki-yolo
orasul/deki-yolo
model
resource
https://huggingface.co/orasul/deki-yolo
new_line
Open-source AI agents are achieving State-of-the-Art results on two different Android AI agent benchmarks.

Yesterday, I finished evaluating my Android agent model, deki, on two separate benchmarks: Android Control and Android World. For both benchmarks I used a subset of the dataset without fine-tuning. The results show that  image description models like deki enables large LLMs (like GPT-4o, GPT-4.1, and Gemini 2.5)  to become State-of-the-Art on Android AI agent benchmarks using only vision capabilities, without relying on Accessibility Trees, on both single-step and multi-step tasks.

All the information is available on GitHub: https://github.com/RasulOs/deki

I have also uploaded the model to Hugging Face:

Space: https://huggingface.co/spaces/orasul/deki
(Check the analyze-and-get-yolo endpoint)

Model: https://huggingface.co/orasul/deki-yolo

🚀
John6666