Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Inference.net: Wholesale LLM Tokens (inference.net)
5 points by funfunfunction on Aug 14, 2024 | hide | past | favorite | 2 comments


Looks cool. To help the pitch, I think you should show the equiv price on anyscale/together for the 10B tokens. Also is there a reason you're not selling at more granular amounts? Why $400? Just speaking personally, I reckon I'd rather invest in a 3090 so I have more security/privacy. I can't think of many genuinely _useful_ 10B+ uses that won't include a bunch of private data. But nonetheless, I reckon there's a niche here, so nice work.


It’s unlikely an individual would need this much capacity. Folks who need tokens at this level are apps with lots of users that don’t have their own GPUs. Think character.ai type apps.

10B tokens on together.ai is ~$2,000.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: