Despite advancements in artificial intelligence, state-of-the-art vision-language models like GPT-4o, Claude Sonnet 3.7, and Gemini 2.5 Pro still struggle to play video games, including the classic shooter Doom. A new research project introduced VideoGameBench, which tests AI capabilities across 20 popular games using only visual inputs. The research indicates that these models face significant challenges due to high inference latency—by the time the AI decides on an action based on a screenshot, the game state may have changed drastically. This delay is particularly detrimental in fast-paced games like Doom. The study found that AI models had trouble with basic in-game actions and failed to control the mouse accurately in games that require precise movements. The research underscores the limitations of current AI systems in dynamic and complex gaming environments, showing that while playing video games is not particularly complex, AI still has a long way to go in mastering them.

Source 🔗