Digital rendering billing and licensing issues have been reduced by a time series solution
PipelineFX helps organizations of all sizes better manage the cost of rendering for digital media applications and programs. For more than 16 years, organizations of all sizes in the film, visual effects, post production, broadcast, design and education industries have been using PipelineFX’s Qube! to manage their render pipeline and make critical business decisions, faster and smarter than the competition.
Time Series Solves Customer Problems
PipelineFX wanted to help its customers meet their digital media deadlines without having to spend the capital to build and maintain their own render farms. The steps a graphics system needs to perform to render a 3D scene to a 2D screen is a very “bursty process,” so one customer may need 10,000 cores one night and only a couple of hundred the next night. In addition, customers’ deadlines are not variable, yet the time needed to render each project is very different, based on the level of computer graphics, layers per shot, resolution of the rendered project and the number of frames per second. To meet a deadline, customers may need to scale from 500 to 1,500 machines for three days. But after those three days, customers don’t want to pay for rendering licenses that are not being used. Their job is finished, so they should not need to pay!
What makes the PipelineFX offering so compelling is the ability to measure utilization every minute and bill at a similar time interval. This high level of precision ensures that PipelineFX customers get a great return on their investment. With this data, PipelineFX is able to provide a very flexible pricing model that can accommodate this high-frequency charging for any license utilization over a certain per-paid minimum. In addition, the system needs to provide reports and insights into what average utilization rates are and whether a customer should consider increasing its minimums.
In addition to solving the problem of rendering licenses during bursty processes, PipelineFX wanted to enhance its SaaS infrastructure to optimize customer demand and server costs. PipelineFX customers need to be able to reserve the right number of servers and nodes to hit their peak demands, but they don’t want (or need) to purchase perpetual licenses for these short-lived servers.
Since PipelineFX wanted to measure and bill at a high-frequency time interval, as well as determine real-time peak demands, its traditional relational database was no longer proving effective.
To gather this high-frequency data, PipelineFx deployed a time series solution. Customers’ rendering data today is collected each minute and transmitted to the PipelineFX service every 15 minutes. If customers lose network connectivity, the application accumulates the batch and, once connectivity is restored, the whole batch is transmitted. Often, eight to 12 hours of data is uploaded—some late in arriving, but the system takes care of this as well. PipelineFX uses this data store to generate customer dashboards and provide reporting of real-time usage.
PipelineFX’s extensive experience with monitoring tools and software development made it an easy decision to use open source projects in its product. The company found early success using open source time series projects to collect and analyze the data needed for its real-time billing offering. Although it was easy to use, PipelineFX was spending valuable development cycles maintaining the solution and decided to review the cost of hosting the solution themselves versus purchasing a hosted version. In its review, PipelineFX found it made better financial sense to go with a hosted version and rely on vendor expertise to keep the software up to date, keep the clusters fully optimized and rely on a 24×7 uptime.
PipelineFX has been able to help its customers achieve maximum efficiency and optimize their existing and future infrastructure. The combination of solutions has allowed for metering and billing at minute-level precision. In addition, customers now have a real-time view of their current usage and billing charges, which enables them to maximize their investment in Qube! without having to commit in perpetual licenses that may go unused.