OctoML makes it easy to put AI/ML models into production – Meczyki.Net

OctoML, the well-funded machine learning startup that helps enterprises customize and deploy their models, today launched a major update to its product, making it easier for developers to integrate ML models into their applications. will be far easier. With this new release, OctoML can now convert ML models into portable software functions with which developers can interact through a consistent API. With this, it will also be easier to integrate these models into existing DevOps workflows.

As Luis Says, the founder and CEO of OctoML, told me, he believes this is a turning point for the company. Cease co-founded the company with the company’s chief technologist Tianqi Chen, CPO Jason Knight, CTO Jared Roche, and Thierry Moreau, VP of Technology Partnerships. TVMAn open-source machine learning compiler framework that helps ML engineers optimize their models for specific hardware.

image credit: OctoML

“Andchicken We started octoml, We Told: let’s go Make TVM as One sservice,” said Cees.I learned One very From He But Then it became cleans We Worked with More customers He ai/ml deployment Even now very Tough.”

He notes that as tools for ingesting data and building models have improved over the years, the gap between those models and actually integrating them into applications has only increased. So by essentially turning models into functions, that difference mostly disappears. This new system takes away that complexity for developers, which will certainly help bring more models into production. Currently, depending on which numbers you trust, more than half of trained ML models never make it into production.

Since OctoML already provides tools to run those models essentially anywhere, a lot of choices about how to deploy models can now be automated as well. “Andhat What sets us apart from any other solution Qualification To Received pattern For deploy, integrate it application – And Then run Feather any endpoint,” Cease said and noted that it is also a gamechanger for autoscaling, as it allows engineers to build autoscaling systems that can move models across CPUs and accelerators with different performance characteristics as needed.

However, model-as-functions capability is only part of the company’s announcements today. Also new to the platform is a new tool that helps OctoML use machine learning to optimize machine learning models. The service can automatically detect and resolve dependencies and clean and optimize model code. There’s also a new native OctoML command-line interface and support for Nvidia’s Triton Intent Server that can now be used with the new model-as-function service.

“NVIDIA Triton is a powerful abstraction that allows users to take advantage of multiple deep learning frameworks and acceleration technologies in both CPUs and NVIDIA GPUs,” said OctoML CTO Jared Roche. “In addition, by combining NVIDIA Triton with OctoML, we are enabling users to more easily select, integrate, and deploy Triton-powered functionality. OctoML Workflow, Triton-based by seamlessly integrating OctoML acceleration technology Further increases the user value of deployment, allowing you to get the most out of both the serving and model layers.

Looking ahead, Seiz noted that the company, which has grown from 20 employees to over 140 since 2020, will focus on bringing its service to more cutting edge devices, including smartphones, and Qualcomm, among other Snapdragon-powered devices. Thank you for your partnership with.

“TeaHe Time seem Correct That’s why as We chat To friends He Huh deployed To Cloud, Now he they All they say passed plans To deploy Feather corner, too,” he said.