ChatGPT for SNES

(bxfoundry.com)

125 points | by evan_ 1 day ago

17 comments

  • ectoloph 1 day ago
    I really enjoy the way the typing is mapped.

    Whilst incredibly frustrating on TV remotes, it's cute with the sound effects.

  • soulofmischief 1 day ago
    That's a great project, thanks for sharing.
  • vjay15 1 day ago
    this is so cute
  • 4lx87 1 day ago
    Wild to think we could have implemented transformers in 1990 with consumer hardware. I guess we needed the web first for training, but small transformers are surprisingly effective. What other discoveries are right under our nose?
    • kqr 1 day ago
      As someone else pointed out, training something like this requires much more hardware than running it.

      It's similar with other forms of machine learning, too. A logistic regression can make calibrated decisions in complex situations with nearly no compute at all, but training it on lots of data to give it good weights is more expensive. And even if we could submit a training batch job on 1990s hardware and wait a few years for it to complete, there's a level of interactivity required at the start to explore one's way to the right parameters and tokenisation etc. that probably wouldn't have been economically feasible at the time.

      In a wider sense, a general purpose computer is a general purpose computer. Hypothetically, we "could have" run transformers in the 1940s on vacuum tubes. It just would have (a) required much space, (b) produced much heat, and (c) required tube replacements all the time. We "could have" run transformers on human computers in the late 1800s or early 1900s – all the maths was there, I think? – but it would have been even slower!

      • soulofmischief 15 hours ago
        The maths wasn't all there before the 50s. We needed McCulloch/Pitts in '43, Shannon in '37 and '48, Hebb in '49, and Turing in '50. Then in '56 Shannon, McCarthy, Minsky and Rochester's research bringing it all together, leading to the coinage of Artificial Intelligence, and Rosenblatt's perceptron in '57. But it wasn't until the first AI winter and the discovery of backpropagation in '86.

        From there you could maybe argue that LeCun's later CNN work and all the other things leading up to the development of GPUs, better data pipelines, AlexNet and Word2Vec, the following attention experiments and finally Google's landmark 2017 paper aren't per se required math, but they were important discoveries which got us over the hump from nonsensical Markov chains and digit analysis to LLMs writing poetry or telling me about Mario on my SNES. LTSM is probably the only thing we could really skip since the entire point of Attention Is All You Need is that you could skip LTSM in favor of self-attention.

        Some of this prevailing research is an artifact of the hardware lottery and maybe not strictly necessary, but anyway hopefully this timeline helps some people orient themselves. We're mostly scaling 80's techniques on modern parallel hardware but your other point stands that we could essentially have run transformers on rocks, had we had enough time and patience. However before backprop we didn't know how to train hidden layers. That was the key turning point to scaling the ideas behind the perceptron.

      • anthk 1 day ago
        The difference between 1992 hardware and 1998 was really huge. High end Sparc servers on 1998 (or double CPU based Xeons) could create something usable.
      • handoflixue 1 day ago
        > We "could have" run transformers on human computers in the late 1800s

        I love the idea of using humans to train an auto-complete model that produces human-like text. I think those are called authors ;)

    • phire 1 day ago
      I'm not sure we would have found this impressive back then, the output is pretty nonsensical. Back then, we had other (more manual) approaches that could produce somewhat equivalent results.

      It's a bit of an illusion, it looks quite impressive to us because we know how good the output from a full-sized LLM can be. We are willing to give it a lot of leeway because of its small size as long as it produces output that looks kind of like chatgpt.

      Well... computational linguistics academics should have been impressed at how well it understands language with nothing more than ingesting a huge amount of untagged data.

      • remembergod 1 day ago
        In fact, I know for a fact we wouldn't, because... This existed.

        Well, not something made the same way this was. And yet, still precisely this in nature. There was at one point a website with what was basically a chatbot pretending to be God. It had pretty evasive responses, very frequently got off topic, but sometimes, and I am not sure how they technically achieved it, it would give spot on answers to what you told them. Never deep trivia knowledge, but like, casual talk expressions and very simple queries, and other follow ups that made complete sense. Like it would know how to follow up on a "What's up dog?" and bark as a joke when you tried to say "I meant dog literally".

        And crucially: We found it utterly unimpressive. It was a stupid toy for kids in computer class.

        • piltdownman 1 day ago
          Heuristic Pattern Matching via AIML. Probably a variant of ALICE

          https://en.wikipedia.org/wiki/Artificial_Linguistic_Internet...

          So not even comparative to be honest.

          • phire 11 hours ago
            The technology wasn't comparable. But IMO the result for a typical end user was roughly the same.

            Sometimes they would produce wonderfully on-topic responses, and other times they would go off the rails. And the end user would only tolerate so much off-the-railsness before dismissing the product as pointless.

            Yes, the results here are quite a bit more impressive than historical equivalents if you analyse them at an academic level. But that doesn't matter to an end user.

            And we saw the same with LLMs. Most people completely dismissed LLMs like GPT2/GPT3, despite how much progress they were making. It was only once OpenAI successfully made them follow instructions and wrapped it up as ChatGPT that the general population started paying attention. GPT3.5 wasn't any smarter than GPT3. But everyone perceived it as smarter simply because it could follow instructions (which... brings up some interesting questions about humans)

    • solyin 1 day ago
      The ancient Romans could have been the ones to discover electricity and steam engines but they didn't. Room temperature superconductors could have already been found. The right person in the right place is what's needed.
    • alnwlsn 1 day ago
      There was an article the other week - turns out, we didn't need Deep Blue to beat Kasparov, a off the shelf regular desktop PC could have done it at the time if it was running a modern chess engine of today. Though you never would have been able to train the model in the first place.
    • LeoPanthera 1 day ago
      Running them isn't the same as creating them.
      • DoesntMatter22 1 day ago
        Sure but it could still have been done, we just didn't realize it.
        • paxy 1 day ago
          By that logic we could have invented it at any point in history with a turing machine made of rocks.
          • staplehorse6553 1 day ago
            And now you're re-reading the Three-body Problem.
    • mk89 1 day ago
      Plenty of research papers out there that get no love.

      Go through them and find the next big thing :)

    • piltdownman 1 day ago
      I mean take a step back and look at in context of the early 90s and it was fairly evident that we could. But it was just as evident that it was a research and sentiment dead-end.

      1993 was the AI winter. Not dissimilar to the naive forecast and scope of the famous 1966 Summer Vision Project in MITs fledgling AI department, they had massively underestimated the complexity of the problem domain. Funding had dried up, and the LISP machine industry was circling the drain.

      The jump from human mandated logic and decision trees was far in the future. The original Neural Networks (Elman & Jordan Networks -> RNN) were crippled by context limitations due to stepping through the data in a linear fashion and hitting the vanishing gradient brickwall almost immediately.

      An agentic AI could trivially can build and run a native Lisp interpreter or bytecode VM on both SNES and Sega Genesis hardware, and a human team could feasibly port an implementation over. The Genesis ran on a 68k with support for 24-bit address space and 32-bit registers, so it was easy to compile 68k C/Assembly to reside within 64kb of RAM.

      But it's not 'real' AI in the sense of Transformer or precursor Architecture. Without an FPU you've no avenue to do parallel matrix multiplication in any sort of reasonable cycle time, even if you managed to somehow deploy a 500mb Model into a custom cartridge and optimised the I/O and swap to make it mechanically feasible. So you're talking hours per token.

      So the results were incredibly slow, erratic, with a tiny context window it still couldn't retain context from, and all the use-cases for console gaming or add-ons like Sewing Machines were handled and implemented in a much saner and deterministic manner via State Machines.

      So in short, we could have done so. Even limited to the knowledge of yesteryear. But why on earth would you have even considered it without the foresight of the contemporary industry outcomes? Even with the hindsight it's simply a novelty... for now anyway.

  • HAL9500TURBO 1 day ago
    Why do CRT screens look so good? Someone send me back, please.
    • piltdownman 1 day ago
      Because the pixel art was optimised for scanline blur on 480p 4:3 Screens. It was pretty awful for every other usecase in the modern context.

      You can get all sorts of period-accurate software and hardware scanline and CRT emulation nowadays. Many are built directly into retrogaming emulators. RetroTINK make a very full featured hardware version.

      https://www.retrotink.com/shop/retrotink-6x-ce

      • HAL9500TURBO 1 day ago
        Very interesting, I had no idea this existed. I was watching some youtube demos of the 4K version, and it looks so cool, especially when zooming in pixels. Apparently that's a good usage of the 4K pixel density, since you can fake the individual pixels really nicely there. I've tried CRT filters on emulators over the years and was always disappointed, though (at least @1440p/1080p). OP's ChatGPT demo made me wonder how my own vibecoded pixelart game would look like with a custom filter now. For the filters I've tried with emulators, I always end up turning it off because it just doesn't feel right, but maybe that's just lack of fine tuning.
        • vunderba 23 hours ago
          Yeah, I would agree. It's especially frustrating to me when I see somebody do a Let's Play video using one of those Super Eagle effects or HQ filters that sort of anti-alias everything. While it does sharpen things, it also really ruins the pixel art charm. RetroArch does have some decent filters that can work well in some cases (e.g., CRT-Consumer).

          If you want to look up some really neat effects, there have been some very cool attempts at building CRT shaders with scan lines and screen warping on Shadertoy. Highly recommend taking a look at those for some inspiration.

          https://www.shadertoy.com/view/WsVSzV

          https://www.shadertoy.com/view/X3ccDN

        • epiccoleman 22 hours ago
          Animal Well had a really nice scanline filter that I tried turning off and immediately switched back, it really helped set the vibe.

          Semi-relatedly I did some hacked-in CRT effects on a recent video I made, and it was interesting. Resolve has some good tools for it, but I had to make my own scanline overlay as the built-in one in Resolve is a little too inflexible.

          I wound up with a scanline transparency, a slight fisheye for "monitor curve", and then the built in "CRT" effect in Resolve, which provided some static and a rolling bar (which, really, is not so much a feature of a CRT itself but more an artifact of recording one, but it fit the vibe).

          Definitely kinda cool to try and replicate the feel of old tech in the context of the new, I really liked my results.

    • Cthulhu_ 1 day ago
      I'm pretty amazed there's no company making them anymore at small scale, there's plenty of enthousiasts out there, you'd think there would be a market for it. But then, shipping alone would cost a small fortune compared to modern screens.
      • alex0015 14 hours ago
        I don't know anything about making CRTs but I had the same thought about high quality electromechanical film cameras from the 1980s and 1990s. Turns out there's just not any of the manufacturing capacity left to make anything like that at a scale that makes it affordable anymore. Like if you wanted to make an autofocus film camera with auto wind, programmable exposure, everything standard from 1995, it would cost a huge amount up front to develop the manufacturing process again and test everything to the required tolerances, then you'd still end up with costs high enough that the end product would cost several hundred dollars where used models often go for under $50.
  • deliciousnutcra 13 hours ago
    Imagine this cartridge could be transported to 1992 and landed on the hands of a Nintendo engineer. They should do a movie about it.
  • matt3210 1 day ago
    you beat me to it
  • jubilanti 1 day ago
    At this point, it's so small it's just a glorified Markov chain? It produces meaningless English-shaped sentences.
  • lanewinfield 1 day ago
    "How many 'r's are there in the word strawberry?"

    > 1. The opposite of strawberries

    AGI achieved

  • adiabatichottub 1 day ago
    I asked it the answer to the question of life, the universe, and everything. It gave me a nonsensical response. I'd say it works perfectly.
  • ronsor 1 day ago
    "1 plus 1?"

    > The answer is 9.

    It tried.

    • redundantly 1 day ago
      Nah, it just used a different meaning for the symbols involved.
  • charlieyu1 19 hours ago
    1.5 MB… make it smaller so it could fit on a floppy
  • amelius 1 day ago
    Post-train it on law texts and make it fight against Nintendo's lawyers.
  • neuroelectron 1 day ago
    The music is great and the way it types out stuff for you is absolutely aesthetic. I'd be interested to know more about how it uses the processors to figure out what part of the network to process. I asked ChatGPT to take some guesses and all it did was find me an AI enabled CRT television on Etsy. I think it's starting to reproduce little children.
  • teaearlgraycold 1 day ago
    I appreciate that out of a vocabulary of 8178 words, "San" and "Francisco" made the cut.

    Also, it has a 128 token context window. Brutal!

  • erickhill 1 day ago
    Hm, I asked it questions about Metroid and the answers were gibberish. "A couch is a type of fruit" etc.

    Eh?