Hosting methods

LitServe is fully open-source and can be hosted on your own machines, DIY or fully hosted on Lightning Studios.

Host on your own

Feel free to pip install LitServe on any computer you own and host a LitServer yourself. LitServe is Apache 2 and fully open-source!

Host on Lightning Studios

For those looking for a fully managed hosted solution, Lightning AI's Studio platform can deploy LitServe models automatically. Studios automatically handle:

• Autoscaling (min, max replicas)
• All the networking, security things
• Monitoring (uptime, rps, etc...)
• Health checks
• Releases (release roll-backs)
• Custom metric monitoring (monitor anything you want inside the model)
• Collaborative deployment management
• Fast cold-start times
• Logs
• Cost management
• Serverless (scale to zero)
• Deploy on your own VPC
• Consume your own cloud credits
• Multi-cloud
• GPU auto-scaling
• Mission-critical latency

Expert acceleration

Our team can also work with you to increase your model's throughout to the desired range. Our team has experts who have been core contributors to PyTorch Lightning, PyTorch, TorchScript, RedisAI and more.