Compiling models to megakernels

(blog.luminal.com)

14 points | by jafioti 1 day ago

1 comments

  • measurablefunc 56 minutes ago
    There are only 4 optimizations in computer science: inlining, partial evaluation, dead code elimination, & caching. It looks like AI researchers just discovered inlining & they already knew about caching so eventually they'll get to partial evaluation & dead code elimination.
    • mxkopy 18 minutes ago
      AI actually has some optimizations unique to the field. You can in fact optimize a model to make it work; not a lot of other disciplines put as much emphasis on this as AI
      • tossandthrow 7 minutes ago
        Can you list these optimizations?
        • mxkopy 1 minute ago
          RLHF is one that comes to mind
    • fragmede 38 minutes ago
      Dead code elimination is already a technique in AI when someone takes an MoE model and removes an unused "E" from it.