I’ve also been exploring this idea. What if you could bring your own (or pull in a 3rd party) “CPU player” into a game?
Using an LLM friendly api with a snapshot of game state and calculated heuristics, legal moves, and varying levels of strategy in working out nicely. They can play a web based game via curl.
That calculates the ELOs for each AI implementation, and I feed it to different agents so they get really creative trying to beat each other. Also making rule changes to the game and seeing how some scripts get weaker/stronger is a nice way to measure balance.
I know visualization is far from the most important goal here, but it really gets me how there's fairly elaborately rendered terrain, and then the units are just unnamed roombas with hard to read status indicators that have no intuitive meaning. Even in the match viewer I have no clue what's going on, there is no overlay or tooltip when you hover or click units either. There is a unit list that tries (and mostly fails) to give you some information, but because units don't have names you have to hover them in the list to have them highlighted in the field (the reverse does not work). Not exactly a spectator sport. Oh, but there is a way to switch from having all units in one sidebar to having one sidebar per player, as if that made a difference.
I find this pretty funny because it seems like a perfect representation of what's easy with today's tools and what isn't
Yeah, it's all what you get when you basically ask an agent "Build X" without any constraints about how the UI and UX actually should work, and since the agents have about 0 expertise when it comes to "How would a human perceive and use this?", you end up with UIs that don't make much sense for humans unless you strictly steer them with what you know.
Wouldn't it be interesting if the LLMs would write realtime RTS-commands instead of Code? After all it is a RTS game.
This would bring another dimension to it since then quality of tokens would be one dimension (RTS-language: Decision Making) and speed of tokens the other (RTS-language: Actions Per Minute; APM).
This reminds me of this yearly StarCraft AI competition (since 2010), however I think it uses a special API that makes it easy for bots to access the game
Very interesting project. I'm a bit confused about the lack of hardware specification. The rules make it clear that one's bot has defined deadlines:
> Make sure that each onframe call does not run longer than 42ms. Entries that slow down games by repeatedly exceeding this time limit will lose games on time.
But I'm missing something like: "Your program will be pinned to CPU cores 5-8 and your bot has access to a dedicated RTX 5090 GPU." Also no mention about whether my bot can have network access to offload some high-level latency insensitive planning. Maybe that's just a bad idea in general, haven't played SC in ages.
Yay, I love how we just keep coming up with magic tricks, like toddlers playing with velcro.. These magic tricks do nothing but convince people who don't know any better that LLMs are the real deal, when they simply aren't.
This is just free propaganda for Anthropic && OpenAI who will leverage these (useless) capabilities to convince your boss to give your salary to them, or at least a substantial portion of it.
> I’ve been an engineer for almost 40 years and love seeing what Claude Code can do.
You would say that because otherwise you'd be afraid as being seen as "too old for this job", and hence risking getting kicked out of it all, meaning no future employment opportunities. I know that feeling, because I myself have been doing this programming job for 20+ years already (so not a young one by any means), but let's just cut the crap about it all and let's tell it how it is.
I'm pretty young and hate this technology with a passion. I didn't spend 100k on education, and studying for a decade to have my job reduced to being a project manager for a bot or to play with a prompt slot machine all day. This crap is reducing the thing I genuinely love doing more than anything, writing code, into nothing.. Reviewing code that lacks any sweat, any intention. I really can't stand this garbage.
I can't stand you old heads, I'm very happy for you that you got to stash away 40 years of SWE salaries. Its just ladder kicking behavior to be honest. Typical boomer, you got your nut and don't care what happens after.
25% of new college grads in STEM are unemployed and a bunch of companies (controlled by people in your age group) have laid off 400k Americans over the last 16 months while equities and profits are at an all time highs.
The replies : ItS NoT Ai, ItS cUz FrEe MoNeY fRoM CoViD HaS DrIeD uP.
Yeah, I guess the tens of thousands of PhDs who are working on LLMs full time are just collectively wasting their lives. Everyone except you is simply too dumb to see it.
Using an LLM friendly api with a snapshot of game state and calculated heuristics, legal moves, and varying levels of strategy in working out nicely. They can play a web based game via curl.
Maybe they are already doing this? Are there logs of the model's thinking?
I was proud for getting the highest-ranked JavaScript-based implementation, but got absolutely crushed by the eventual winner.
1. https://github.com/aichallenge/aichallenge
https://egeozcan.github.io/unnamed_rts/game/
I occasionally run my tournament script: https://github.com/egeozcan/unnamed_rts/blob/main/src/script...
That calculates the ELOs for each AI implementation, and I feed it to different agents so they get really creative trying to beat each other. Also making rule changes to the game and seeing how some scripts get weaker/stronger is a nice way to measure balance.
Funny thing, Codex gets really aggressive and starts cheating a lot of times: https://bsky.app/profile/egeozcan.bsky.social/post/3mfdtj5dh...
I find this pretty funny because it seems like a perfect representation of what's easy with today's tools and what isn't
Love the idea though
This would bring another dimension to it since then quality of tokens would be one dimension (RTS-language: Decision Making) and speed of tokens the other (RTS-language: Actions Per Minute; APM).
Also there are a lot of coding benchmarks, that way it would test something more abstract, similar to AlphaStar https://en.wikipedia.org/wiki/AlphaStar_(software)
You could just use the exposed APIs of OpenAI, Anthropic etc. and let them battle.
Edit: Forgot link: https://davechurchill.ca/starcraft/
> Make sure that each onframe call does not run longer than 42ms. Entries that slow down games by repeatedly exceeding this time limit will lose games on time.
But I'm missing something like: "Your program will be pinned to CPU cores 5-8 and your bot has access to a dedicated RTX 5090 GPU." Also no mention about whether my bot can have network access to offload some high-level latency insensitive planning. Maybe that's just a bad idea in general, haven't played SC in ages.
This is just free propaganda for Anthropic && OpenAI who will leverage these (useless) capabilities to convince your boss to give your salary to them, or at least a substantial portion of it.
I’ve been an engineer for almost 40 years and love seeing what Claude Code can do.
Like it or not, young people will not know a world where this technology doesn’t exist. It is just part of their toolset now.
You would say that because otherwise you'd be afraid as being seen as "too old for this job", and hence risking getting kicked out of it all, meaning no future employment opportunities. I know that feeling, because I myself have been doing this programming job for 20+ years already (so not a young one by any means), but let's just cut the crap about it all and let's tell it how it is.
I can't stand you old heads, I'm very happy for you that you got to stash away 40 years of SWE salaries. Its just ladder kicking behavior to be honest. Typical boomer, you got your nut and don't care what happens after.
25% of new college grads in STEM are unemployed and a bunch of companies (controlled by people in your age group) have laid off 400k Americans over the last 16 months while equities and profits are at an all time highs.
The replies : ItS NoT Ai, ItS cUz FrEe MoNeY fRoM CoViD HaS DrIeD uP.