Shared Filesystems

Shared Filesystems provide persistent storage that remains available across instance launches and terminations. You can attach the same filesystem to multiple instances in the same region, making it easy to maintain and share data independently of compute instances.

Overview

Each filesystem can be used to store:

• Large datasets and model weights
• Source code repositories
• Configuration files

info

Jarvislabs supports filesystems of up to 10 TB, allowing you to scale your storage needs as your projects grow. If you need more storage, please contact us at support@jarvislabs.ai.

Getting Started with Shared Filesystems

Creating and Attaching a Filesystem

1. When launching a new instance, create or attach a filesystem during instance configuration.
2. You can also create one from the Shared Filesystems dashboard and attach it later.
3. Select your desired storage capacity (up to 10 TB).
4. Once your instance launches, the filesystem is mounted at /home/jl_fs.

Using a Filesystem

• Each attached instance mounts the filesystem at /home/jl_fs.
• Data stored in /home/jl_fs persists independently of the instance.
• The same persistent filesystem can be attached to multiple instances, enabling data sharing.

note

You can attach a filesystem only to instances in the same region as the filesystem.

Common Use Cases

• Training Data: Store and access large datasets across multiple training runs
• Model Artifacts: Save trained models and checkpoints
• Collaboration: Share data and code between team members using different instances

Pricing and Billing

Shared Filesystems are billed based on provisioned capacity:

• Rate: $0.00014/GB/Hour (approximately $0.1008/GB/month)
• Example: 500GB storage for one month (720 hours) = $50.40

Important Notes:

• Pricing is for the allocated capacity, regardless of actual usage
• Storage size can only be increased, not decreased
• Current pricing is displayed during filesystem creation

Best Practices for Shared Filesystems

Shared Filesystems use a distributed filesystem designed for scalability and reliability. Follow these practices to optimize your usage:

1. Large Files vs Small Files

   • Performs best with larger files (>1MB)
   • Avoid storing many small files (few KB) in a single directory
   • Consider combining small files into archives (tar, zip) when possible

2. Directory Organization

   • Keep the number of files per directory under 1000 for optimal performance
   • Use subdirectories to organize large collections of files
   • Avoid deep directory hierarchies (keep under 8 levels when possible)

3. Performance Tips

   • For read-heavy workloads, consider using /home temporary files
   • Avoid running intensive file operations on multiple instances simultaneously

4. Data Management

   • Regularly clean up unnecessary files to optimize storage usage
   • Use .gitignore-style files when working with git repositories
   • Back up critical data to external storage periodically

Note: While Shared Filesystems are highly reliable, they're designed for general-purpose storage. For workloads requiring extremely high IOPS, consider using /home storage instead.

Troubleshooting

Common issues and solutions:

1. Slow Performance

   • Check if you're following the best practices for file organization
   • Consider using home for temporary files and intensive I/O operations

2. Storage Full

   • Regular cleanup of unnecessary files
   • Consider increasing storage capacity if needed

For additional support, please contact our support team through the dashboard.