OctoAI is now NVIDIA
“Working with the OctoAI team, we were able to quickly evaluate the new model, validate its performance through our proof of concept phase, and move the model to production. Mixtral on OctoAI serves a majority of the inferences and end player experiences on AI Dungeon today.”
CEO & Co-Founder @ Latitude
“We are excited about working closely with OctoAI to help enterprises build and deploy generative AI applications at scale. OctoAI complements our industry-leading infrastructure offerings and makes it easy for developers to quickly and cost-effectively take their generative AI applications from idea to production and take advantage of the broadest choice of compute options optimized for generative AI and global infrastructure regions.”
Director of Product Management @ AWS
“Speed is key to the AI art experience we deliver. We’ve been able to increase our image generation speeds by 5x with OctoAI’s low latency inferences, and this has resulted in even more usage and growth for our platform!”
Founder @ NightCafe
“Through this partnership, we’re bringing together our AI-optimized infrastructure with OctoAI’s platform for serving generative AI models, giving developers more ways to use Google Cloud for inference and AI applications. This partnership is a testament to Google Cloud’s commitment to supporting a vibrant ecosystem of startups and an open stack of AI tooling and applications.”
President, Global Field Org @ Google
“The LLM landscape is changing almost every day, and we need the flexibility to quickly select and test the latest options. OctoAI made it easy for us to evaluate a number of fine tuned model variants for our needs, identify the best one, and move it to production for our application.”
CEO & Co-Founder @ Otherside AI
“For our performance and security-sensitive use case, it is imperative that the models that process call data run in an environment that offers flexibility, scale and security. OctoStack lets us easily and efficiently run the customized models we need, within environments that we choose, and deliver the scale our customers require.”
CEO @ Apate AI
“We want to be constantly evaluating new models and new capabilities, but running these in-house was a lot of overhead and limited our momentum. OctoAI simplifies this, allowing us to build against our choice of models and fine-tuned variants through one simple interface.”
Co-Founder & CTO @ Capitol AI
“Our top priority was to get the product to market quickly using an open source image solution. We also wanted to ensure that our imagery provided an outstanding experience to our users, while maintaining consistency with our guardrails and themes. OctoAI simplified how we achieved both of these goals while providing the highest level of speed and reliability.”
Founder & CEO @ Storytime AI
“OctoAI’s integration has been instrumental in making it possible for CALA to power the ability for our customers to fine-tune their image generation. OctoAI has allowed us to accelerate our development and time to market with these new features while eliminating the typical costs that we would have faced by running multiple parallel model variants.”
Co-Founder & CTO @ CALA
“OctoAI made the process of deploying our custom voice dubbing models and taking our application live into production easy. We've been able to deploy and optimize multiple models as we launched and scaled our application.”
Co-Founder @ DubDub.AI
GenAI production stack: SaaS or in your environment
The foundation of OctoAI is systems and compilation technologies we’ve pioneered, like XG Boost, TVM, and MLC, giving you an enterprise system that runs in our SaaS or your private environment.
Enterprise-grade inference
Predictable reliability
99.999% uptime with consistent latency SLAs.
Optimize Performance & Cost
Run GenAI inference at the lowest price and latency on our optimized serving layer.
Future Proof Applications
Rapidly iterate with new models and infrastructure without rearchitecting anything.
Customize Freely
Mix and match models, fine tunes, and LoRAs at the model serving layer.
SOC 2 Type II & HIPPA certified
Your data security and privacy is a top priority for OctoAI. We continually invest in security capabilities and practices in our platform and processes.
Powerful capabilities for your GenAI apps
Build using state of the art solutions for your products with multiple models, thousands of LoRAs, your datasets, and orchestration logic.
Fine tune models for your use cases and serve the best quality model into production for the same cost as the base model.
Build using Retrieval Augmented Generation (RAG) with embeddings and your data to provide contextual accuracy for your users.
Automation with AI agents created with function calling ensures quality, reduces tedious tasks, and allows access to real-time data.
JSON mode provides structured outputs to simplify systems integrations and connect different components in your app, with no performance loss.
OctoStack from OctoAI: GenAI in your environment
OctoStack is a turnkey GenAI serving stack to run your optimized models in your environment on your GPUs. Lower your total cost of ownership and deploy models with greater agility while ensuring data privacy.
What’s New at OctoAI
Latest Models
Phi 3.5-Vision
The newest from the Phi-3 family is a lightweight state-of-the-art multimodal model. This model comes with 128k context length, and was built with a focus on high quality reasoning for both text and vision. This model can use it's reasoning on both text and image inputs, and is available for commercial use.Mistral NeMo
Built in collaboration with NVIDIA this state-of-the-art models has a 128k context window and has an Apache 2.0 license. This model excels at reasoning, coding accuracy, world knowledge, and is multilingual.FLUX.1 [Schnell]
A 12 billion parameter model used to create high quality images from text prompts. FLUX models showcase superiority in creating text in images, high accuracy for human features, and multi-element spaces or landscapes. With super fast generation speeds and a commercial license it can be used for all your GenAI image products.Llama 3.1 Instruct
The Meta Llama 3.1 models are instruction tuned and optimized for multilingual dialogue. Currently, they outperform many open source and closed chat models on several industry benchmarks. Supported languages: English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.
Customer & Product Updates
Natural Language Query Engine powered by Llama 3.1 on OctoAI
OctoAI’s Inference Engine: enterprise-grade, dynamically reconfigurable, natively multimodal
Automating your customer support: Function Calling on OctoAI
In Defense of the Small Language Model
Demos & Webinars
Optimizing LLMs for cost and quality
This technical webinar will review fine tuning models for performance, model quality optimization, devops for LLM apps, and a full demo showing how to fine tune OSS models for better quality than closed models.
Harnessing Agentic AI: Function Calling Foundations
Watch our on-demand webinar about how to create AI agents using function calling for your AI apps. This technical deep dive has a presentation, demo, and example code to follow.
All about fine-tuning LLMs
Listen on-demand to a panel of experts talking about various fine-tunes available, how to create your own fine-tune, alternatives to custom fine-tunes, and more.
Selecting the right GenAI model for production
Watch our on-demand webinar as our engineers review all steps of model evaluation, testing, when to use checkpoints vs LoRAs, and how to get the best results.
Your choice of models and fine tunes
Start building in minutes. Gain the freedom to run on any model or checkpoint on our efficient API endpoints.