Show HN: OCR Arena – A playground for OCR models

(ocrarena.ai)

58 points | by kbyatnal 3 days ago

9 comments

  • wener 5 minutes ago
    Really hope there is a layout mode or ocr with bbox mode, I want to see the model restore the whole page.
    • rubikscubeguy 4 minutes ago
      yeah, that would be a cool long term goal
  • ArcaneMoose 1 hour ago
    I've been really impressed with this model specifically because of how insanely cheap it is: https://replicate.com/ibm-granite/granite-vision-3.3-2b

    I didn't expect IBM to be making relevant AI models but this thing is priced at $1 per 4,000,000 output tokens... I'm using it to transcribe handwritten input text and it works very well and super fast.

  • zzleeper 2 hours ago
    Love this! Would have liked to see something like textract for a pre-LLM benchmark (but of course that's expensive), and also a distinction between handwritten text and printed one.

    But still, this is incredibly useful!

  • fzysingularity 2 hours ago
    FYI one of the models on the battle was pretty slow to load. Are these also being rated on latency or just quality?
    • kbyatnal 49 minutes ago
      Ultimately, there’s some intersection of accuracy x cost x speed that’s ideal, which can be different per use case. We’ll surface all of those metrics shortly so that you can pick the best model for the job along those axes.
    • andrewlu0 58 minutes ago
      ideally we want people to rate based on quality - but i imagine some of the results are biased rn based on loading time
  • krashidov 2 hours ago
    I would be curious to see how Sonnet does. Their models are pretty solid when it comes to PDFs
    • rubikscubeguy 5 minutes ago
      sonnet and opus are live now :)
    • kbyatnal 1 hour ago
      Sonnet/Opus is being added shortly!
  • ianhawes 2 hours ago
    Please add Chandra by Datalab
  • codeddesign 1 hour ago
    Most of these are general LLM’s and not specifically OCR models. Where is Google Vision, Mistral, Paddle, Nanonets, or Chandra??
    • kbyatnal 59 minutes ago
      We wanted to keep the focus on (1) foundation VLMs and (2) open source OCR models.

      We had Mistral previously but had to remove it because their hosted API for OCR was super unstable and returned a lot of garbage results unfortunately.

      Paddle, Nanonets, and Chandra being added shortly!

    • rubikscubeguy 4 minutes ago
      nanonets is live now!
  • arathis 3 hours ago
    Claude would be good!
    • rubikscubeguy 4 minutes ago
      claude is live now!
    • kbyatnal 59 minutes ago
      Claude coming shortly (in the next ~1 hour)
  • dang 3 hours ago
    [under-the-rug stub]

    [see https://news.ycombinator.com/item?id=45988611 for explanation]

    • ylhert 3 days ago
      We've got like 10 LLM arenas but nothing for OCR yet, really hope this takes off!
    • athoscouto 3 days ago
      Nice! Would love to see Azure Document Intelligence on this
    • profburial 3 days ago
      This is a killer idea!