Hello,
I recently completed a small project where I explored the potential improvements in model serving by switching from naive PyTorch setups to the ones utilizing ONNX Runtime, as well as ONNX Runtime with Rust's Actix web server.
Here are the findings:
https://github.com/martynas-subonis/model-serving
Additionally, I wrote a broader article on Substack for anyone interested:
Hope somebody finds it interesting/useful as well.
Hello,
I recently completed a small project where I explored the potential improvements in model serving by switching from naive PyTorch setups to the ones utilizing ONNX Runtime, as well as ONNX Runtime with Rust's Actix web server.
Here are the findings:
https://github.com/martynas-subonis/model-serving
Additionally, I wrote a broader article on Substack for anyone interested:
https://github.com/martynas-subonis/model-serving
Hope somebody finds it interesting/useful as well.