r/ProgrammerHumor • u/PokeRestock • 2d ago

Other vibeCodeMasterClass

Post image
9.3k Upvotes

96% Upvoted

606 comments sorted by

View all comments

64

u/CalmLotus 1d ago

I don't think you need ai for spoken instructions. (Like yeah- the game may be vibe coded but->)

You just need to find key words for certain actions and extract those from audio input. More work definitely, but would actually be doable for a game.

...wait, Nintendo DS games were already doing that I think. With the built in microphone.

42

u/audigex 1d ago

Yeah speech recognition doesn't require a full scale frontier level cloud LLM

You can run something like Whisper https://github.com/openai/whisper locally to transcribe the speech in about 1 second on a MiniPC, then check that for keywords

Then you can use something like Piper https://github.com/OHF-Voice/piper1-gpl to convert the reply back to speech. It's not quite as pretty sounding as something from eg ElevenLabs, but it works - on a mini PC it can handle voice in real time

I use both of those locally to run my own voice assistant for my smart home (indeed, Piper is made by Home Assistant for that exact purpose)

Obviously for a game they'd need more than one Mini PC - but you can scale it up for a lot less than $1k/day!

23

u/shawnikaros 1d ago

The system could be run on the computer it's being played at, no need to phone home. It all just speaks of really garbage design.

5

u/audigex 1d ago

Yup, or at the very least they could run it on a single decent server for the volume of players they've seen (looks like a few hundred)

2

u/Tornado547 1d ago

yeah but that would require this thing called "effort" and thats in short supply these days

2

u/welcomefinside 1d ago

Wait is that not the joke here?

1

u/shawnikaros 1d ago

Yes, it is.

11

u/Tristan_Cole 1d ago

If you want to interpret language, LLMs are much better than keyword scanning. The same keywords can indicate opposite things with a single “not” keyword, and there are about 50,000 generally well-known English words to consider.

10

u/audigex 1d ago

That's true but beside the point here

Nobody is saying keyword detection is as good as an LLM. We're just saying a frontier cloud LLM is very expensive overkill for this kind of project

Keyword detection or basic local transcribing and a MUCH smaller local LLM, would do the job just as well

Whisper + a very small model could handle this without $1000/day in OpenAI API costs

1

u/Virgurilla 1d ago

My Final Degree Project is going to be centered around this, starting it this week :)

1

u/tozpeak 1d ago

People already forgetting that single language gpt-2 was working fine on consumer devices.

1

u/Tristan_Cole 1d ago

How are you going to implement keyword detection with enough logic to accurately parse the meaning behind all of the possible things that could be said to describe or correct someone’s driving?

That is literally a quintessential Natural Language Processing problem, for which LLMs are the best solution.

5

u/Vesuvius079 1d ago

There’s also jev now which is supposed to be super cheap for classification. You could classify the sentences into actions instead of keyword search.

0

u/articulatedbeaver 1d ago

Don't need jev at all, just some work with a traditional classifier.

1

u/Ran4 1d ago

The point of jev and similar models is that it's a lot cheaper.

2

u/Forsaken_Ad8120 1d ago

i mean there is even VoiceBox, which is voice cloning locally can pretty quickly generate responses.

1

u/Unclematttt 1d ago

One caveat on your comment about Whisper: you need a GPU to really let it shine. Using CPU only is not performant, depending on what you are trying to achieve, and how much time you are willing to wait for your results