Excited to sponsor the biggest AI event in India: Nvidia AI Summit (Mumbai) on 24th & 25th Oct ✨

Cloud Native & DevOps Blog

Guide to GPU Sharing Techniques: vGPU, MIG and Time Slicing

Guide to GPU Sharing Techniques: vGPU, MIG and Time Slicing

vGPU vs MIG vs Time-Slicing techniques to optimize GPU usage for AI and ML. Learn how these methods reduce GPU costs and enhance project scalability.

Sameer Kulkarni

Sameer Kulkarni

7^th August 2024 13 mins

Show More

Running Phi 3 with vLLM and Ray Serve

How to deploy the Phi-3 model using vLLM with Ray Serve on Kubernetes and understand the key differences between inference and serving.

Sudhanshu Prajapati

Sudhanshu Prajapati

22^nd October 2024 27 mins

Primer on Distributed Parallel Processing with Ray using KubeRay

What is Distributed Parallel Processing? How to achieve it with Ray using KubeRay in Kubernetes to handle large-scale, resource-intensive tasks.

Sudhanshu Prajapati

Sudhanshu Prajapati

30^th September 2024 15 mins

Improving RAG Accuracy with Rerankers

What are rankers? Learn about the types of rerankers, how to select a reranker model, and how to use it to improve the Retrieval Augmented Generation framework.

Shreyas Mocherla

Shreyas Mocherla

26^th August 2024 12 mins

Running Llama 3 with Triton and TensorRT-LLM

What are inference and inference servers? Learn about a popular inference server, Triton, and deploy Meta’s Llama 3 model (an LLM) using it.

Bhavin Gandhi

9^th August 2024 22 mins

Guide to GPU Sharing Techniques: vGPU, MIG and Time Slicing

vGPU vs MIG vs Time-Slicing techniques to optimize GPU usage for AI and ML. Learn how these methods reduce GPU costs and enhance project scalability.

Sameer Kulkarni

Sameer Kulkarni

7^th August 2024 13 mins

Developing an AI Agent for Smart Contextual Q&A

Learn how Agents work and how you can integrate LLMs with specialized tools to create a practical and powerful AI Agent for smart contextual Q&A applications.

Shreyas Mocherla

Shreyas Mocherla

6^th August 2024 12 mins

Essential Guide to NVIDIA GPU Operator in Kubernetes

What is NVIDIA GPU Operator? Learn its installation, features, and how it manages GPU resources efficiently in Kubernetes environments to scale AI/ML workloads.

Sameer Kulkarni

Sameer Kulkarni

Sanket Sudake

InfraCloud Team

22^nd July 2024 13 mins

Introduction to NVIDIA Network Operator

Explore NVIDIA Network Operator, understand its functionality, key features, and how it maximizes AI application potential within the Kubernetes ecosystem.

Sameer Kulkarni

Sameer Kulkarni

Bhavin Gandhi

InfraCloud Team

9^th July 2024 12 mins

What are Vector Databases? A Beginner's Guide

Understand vector databases, types of data, and why we need them to store unstructured data. Discover the use cases of vector databases with examples.

Sameer Kulkarni

Sameer Kulkarni

31^st May 2024 12 mins

Exploring AI Model Inference: Servers, Frameworks, and Optimization Strategies

This blog post explores key decisions for deploying models in production, incl. infrastructure, optimization & frameworks to streamline operations.

Aman Juneja

6^th May 2024 18 mins

Retrieval-Augmented Generation: Using your Data with LLMs

What is Retrieval-Augmented Generation (RAG)? Learn how to use RAG to train LLMs on specific data within the demo app for accurate and contextual responses.

Sanket Sudake

26^th March 2024 10 mins

--- --- --- ---