Getting Started with Jarvislabs

Welcome to Jarvislabs! This guide will walk you through the fundamental features and capabilities of our product. Whether you're new to GPU computing or an experienced user, we'll help you get up and running quickly.

What You'll Learn

By following this guide, you'll:

• Create your first GPU-powered instance using our template system →
• Set up and verify your development environment →
• Run a large language model (Qwen) on your GPU →
• Create a simple web interface using Gradio →
• Learn best practices for managing your instance and resources →

Creating GPU-Powered Instances

Jarvislabs offers two main ways to create GPU-powered instances, each designed for different use cases:

1. Templates (Recommended for Most Users)

Templates provide a streamlined experience with:

• Containers based
• Pre-configured JupyterLab environment
• VS Code integration
• SSH access
• Optimized settings for common AI/ML workflows

Template Limitations

Since templates are built using containers, you cannot run Docker inside a template instance. If you need to run Docker containers, please use the VM option instead.

2. Virtual Machines (For Advanced Users)

Virtual Machines offer:

• Direct SSH access
• Complete system control
• Maximum flexibility
• Limited GPU options

Choose the option that best fits your needs - templates for a quick start with common tools, or VMs for complete system control.

Creating Your First Instance

Let's walk through creating a PyTorch instance using our template system:

Step 1: Select Template and Configuration

1. Navigate to the Templates section
2. Choose the PyTorch template
3. Select GPU Type: A5000 (recommended for this tutorial)
4. Set Storage: 100 GB (sufficient for tutorial purposes)
5. Click "Launch Instance"

Your GPU instance will be ready in less than a minute!

Step 2: Connecting to Your Instance

You have three ways to connect to your instance:

1. JupyterLab (Recommended for this tutorial)

   • Web-based interface
   • Perfect for interactive development

2. VS Code Online

   • Full VS Code experience in your browser
   • Integrated terminal support

3. SSH Access

   • Direct terminal connection
   • Requires SSH key setup
   • Follow this guide to add your SSH keys if not already configured

For this tutorial, we'll use JupyterLab. Click the JupyterLab option to begin working with your instance.

Verifying Your Setup

Before we dive into development, let's verify that everything is working correctly. We'll run some quick checks to ensure your GPU is properly configured and accessible.

Checking GPU Configuration

Run these commands in your JupyterLab notebook to verify your GPU setup:

# List available GPUs
!nvidia-smi -L

# Check GPU utilization and memory
!nvidia-smi --query-gpu=utilization.gpu,memory.total,memory.used --format=csv

# Verify Python environment and hardware
import torch, platform, psutil
print("Torch:", torch.__version__,
      "| GPU:", torch.cuda.get_device_name(0),
      "| Python:", platform.python_version(),
      "| CPU cores:", psutil.cpu_count(logical=False))

These commands will show you:

• Available GPUs and their status
• Current GPU utilization and memory usage
• Installed versions of key software
• Available CPU resources

CPU Core Count Discrepancy

When checking CPU cores using psutil.cpu_count(), you might notice that the number of cores shown is higher than what's displayed in the Jarvislabs UI. This is because:

• The instance is a Docker container
• The command shows all CPU cores available on the host machine
• The UI displays the cores specifically allocated to your instance

This difference is normal and won't affect your instance's performance or resource allocation.

Managing Python Environments

Need a different Python version or PyTorch version? No problem! We use uv, a modern Python package installer and resolver, to manage environments efficiently.

Why We Use uv

We recommend uv for Python package management because it offers significant advantages:

• Speed: Resolves and installs packages 5-10× faster than pip/conda
• Persistence: When you place your .venv in the persistent workspace volume, your environment automatically restores after instance pauses/resumes
• Efficiency: No need to reinstall packages after instance restarts

This combination of speed, reproducibility, and persistence makes uv the ideal choice for GPU development workflows.

To create a new environment with specific versions:

1. Create Virtual Environment

Change python3.12 to your desired Python version (e.g., python3.11, python3.10).

uv venv myenv --python=python3.12 --seed

2. Activate Environment

source myenv/bin/activate

3. Install PyTorch

uv pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128

Choosing the Right PyTorch Version

When installing PyTorch, it's important to select the correct version for your environment. Here's how to ensure compatibility:

1. Check CUDA Version

   nvidia-smi

   This command will show your current CUDA version in the top right corner.

2. Select Installation Command

   • Visit the official PyTorch website
   • Choose these options:
     • PyTorch Build: Stable
     • Your OS: Linux
     • Package: Pip
     • Language: Python
     • Compute Platform: CUDA (match your container's version)

3. Verify Installation After installation, verify PyTorch can access your GPU:

   import torch
   print(torch.cuda.is_available())  # Should print True
   print(torch.version.cuda)  # Should match your CUDA version

Using Your Environment in Jupyter

To use your new environment in Jupyter notebooks:

1. Install the Jupyter kernel from terminal:

uv pip install ipykernel
python -m ipykernel install --user --name=my-env --display-name "Python3.12"

2. Refresh your browser and click the "+" button to create a new notebook
3. Select your new environment (e.g., "Python3.12") from the kernel list

Your notebook will now use the new environment by default, with all the packages you installed.

Running an LLM Model

Let's run a large language model using the Hugging Face Transformers library. First, install the required packages:

source myenv/bin/activate
uv pip install transformers accelerate hf-xet jupyter ipywidgets tqdm

Now, create a new notebook "Python 3.12" and run this code to load and use the Qwen model:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline

# Load the model in FP16 for better performance
model_id = "Qwen/Qwen3-8B"
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,   # Use FP16 weights
    device_map="auto"           # Automatically use available GPU
)

# Initialize tokenizer
tok = AutoTokenizer.from_pretrained(model_id)

# Create text generation pipeline
chat = pipeline(
    "text-generation",
    model=model,
    tokenizer=tok,
    device_map="auto",
    max_new_tokens=1000,
    do_sample=True,
    temperature=0.7,
)

# Generate text
print(chat("Tell me 2 jokes about an AI")[0]["generated_text"])

Model Loading

• The model is loaded in FP16 precision to reduce memory usage
• device_map="auto" automatically handles GPU allocation
• You can adjust temperature and max_new_tokens to control generation

Creating a Web Interface with Gradio

Let's create a simple web interface using Gradio. First, install Gradio:

source myenv/bin/activate
uv pip install gradio

Now, create a new notebook and run this code to create an interactive web interface:

import gradio as gr
import torch

def gpu_name(_):
    return torch.cuda.get_device_name(0)

# Create a simple interface
demo = gr.Interface(
    fn=gpu_name,
    inputs="text",
    outputs="text",
    title="Hello Friend 👋, Welcome to Jarvislabs",
    description="What GPU I am running!"
)

# Launch the interface
demo.launch(server_name="0.0.0.0", server_port=6006, share=True)

Gradio Interface

To access your Gradio interface, click the API endpoint button in the Jarvis Labs dashboard. This will open the interface in a new tab.

Port Configuration

Default Port (6006)

• The container has port 6006 pre-configured and mapped to the API endpoint
• This port is automatically exposed through Cloudflare
• Use this port when you want to expose your application to the outside world

Using Port 6006

• Perfect for web applications like:
  • FastAPI servers
  • Gradio interfaces
  • Other web services
• Simply run your application on port 6006 to make it accessible

Additional Ports

• You can configure additional ports during instance creation
• These ports can be used for other services or applications
• Check the instance creation interface for port configuration options

Managing Your Instance and Billing

To avoid unnecessary charges, remember to:

1. Stop your instance when you're done working for the day - you can resume it later and all your data in /home will be preserved
2. Destroy your instance if you don't need the data anymore - this will completely remove the instance and its data
3. Configure auto-pause - if you want your instance to automatically pause after a period of inactivity, check out our instance management documentation

Your instance will continue to incur charges as long as it's running, so make sure to stop or destroy it when you're not using it.

Conclusion

Congratulations! You've successfully:

• Created your first GPU-powered instance on Jarvislabs
• Set up and verified your development environment
• Run a large language model (Qwen)
• Created a web interface with Gradio

Next Steps

Here are some recommended next steps to continue your journey:

1. Explore More Templates: Check out our other templates for different ML frameworks and use cases
2. Read More Tutorials: Explore our other tutorials for advanced topics.

Happy coding! 🚀