MLOps Setup – OVHcloud + DVC + MLflow (Audio Dataset)

Architecture

Audio + Labels → OVHcloud Object Storage (S3)
Versioning     → DVC
Tracking       → MLflow
Code           → Git

Why this split? Audio files are large binary files — Git was not designed for them. Git handles your code and configuration. DVC (Data Version Control — an open-source tool that brings Git-like versioning to data files) handles your data, using the same mental model (add, push, pull) but storing files on a cloud backend (here OVHcloud S3). MLflow (an open-source platform for tracking ML experiments) records what happened during each training run: which dataset version, which hyperparameters, which metrics.

Together, these three tools give you full traceability: code + data + results.

1. Create Object Storage on OVHcloud

OVHcloud is a European cloud provider whose Object Storage is compatible with the S3 protocol — the standard storage API originally created by AWS. This means any tool that supports S3 (DVC, boto3, the AWS CLI…) works with OVHcloud out of the box, just by pointing it to a different endpoint URL.

1.1 Create Public Cloud Project

• Log into OVHcloud Manager
• Create a Public Cloud project
• Activate billing

1.2 Create Object Storage

• Go to Object Storage
• Create a container (bucket), e.g.: ml-audio-datasets
• Select region (EU recommended)

The right bucket configuration depends on your project phase:

R&D phase — versioning only is sufficient. You will frequently delete mislabeled files, reorganize the dataset structure, or correct errors in bulk. Object Lock would prevent all of this, leaving you with files you cannot clean up. Versioning alone already protects you against accidental deletions — deleted files remain recoverable.

Production phase — when your dataset is stable and used to train models that go to clinical or production use, create a dedicated bucket with Object Lock enabled. This guarantees that any dataset version linked to a deployed model can never be altered or deleted. Note that Object Lock also requires setting a retention policy (a minimum duration during which files are protected) to be fully effective — without it, the lock has no enforcement period.

Once the bucket is created, enable versioning:

aws s3api put-bucket-versioning \
  --bucket ml-audio-datasets \
  --versioning-configuration Status=Enabled \
  --endpoint-url https://s3.<region>.io.cloud.ovh.net

1.3 Generate S3 Credentials

In OVHcloud, S3 credentials are created via OpenStack Users (not IAM as in AWS — the concept is equivalent but the interface differs). Create a dedicated user with restricted permissions following the principle of least privilege: only grant the permissions your pipeline actually needs.

You will obtain:

BUCKET_NAME=ml-audio-datasets
ACCESS_KEY=xxxxxxxx
SECRET_KEY=xxxxxxxx
ENDPOINT_URL=https://s3.<region>.io.cloud.ovh.net

Save these securely (password manager, never in Git).

2. Install Dependencies

Install all required packages in one command:

pip install dvc[s3] mlflow PyYAML

Initialize your Git + DVC repository:

git init
dvc init
git commit -m "init dvc"

3. Configure DVC Remote (OVH S3)

# Public config — committed to Git
dvc remote add -d ovh s3://ml-audio-datasets
dvc remote modify ovh endpointurl https://s3.<region>.io.cloud.ovh.net

# Credentials — local only, never committed
dvc remote modify ovh --local access_key_id <ACCESS_KEY>
dvc remote modify ovh --local secret_access_key <SECRET_KEY>

4. Structure Dataset

dataset/
│
├── audio/
│   ├── file1.wav
│   └── file2.wav
│
└── labels/
    ├── file1.txt
    └── file2.txt

Create a .dvcignore file at the root of your repository to exclude temporary or system files from DVC tracking:

# .dvcignore
.DS_Store         # macOS system files
Thumbs.db         # Windows thumbnail cache
*.tmp
*.log

5. Add Dataset to DVC

dvc add dataset/
git add dataset.dvc .gitignore
git commit -m "dataset v1"
dvc push

6. Run MLflow Locally

mlflow ui

Access at: http://localhost:5000

7. Log Dataset Version During Training

import subprocess
import yaml
import mlflow
import dvc.api

# --- Dataset version (DVC) ---
with open("dataset.dvc") as f:
    dvc_file = yaml.safe_load(f)

# When versioning a directory (dvc add dataset/), DVC creates an internal
# manifest file listing all files with their individual hashes.
# The hash stored in dataset.dvc points to that manifest — it ends with .dir
out = dvc_file["outs"][0]
dvc_hash = out.get("md5") or out.get("etag")  # md5 for local/SSH, etag for S3

if not dvc_hash.endswith(".dir"):
    print(f"Warning: unexpected hash format: {dvc_hash}")
    # This may indicate you versioned a single file instead of a directory

# Optional: resolve the full S3 URL of the dataset on the remote.
# Specify the remote explicitly to avoid ambiguity.
# Wrapped in try/except because this requires network access —
# it will fail in offline or CI environments without remote access.
try:
    dataset_url = dvc.api.get_url("dataset/", remote="ovh")
except Exception:
    dataset_url = "remote-unavailable"

# --- Code version (Git) ---
try:
    git_commit = subprocess.check_output(
        ["git", "rev-parse", "HEAD"]
    ).decode().strip()
except subprocess.CalledProcessError:
    git_commit = "no-git-commit"  # safety: repo with no commit yet

with mlflow.start_run():
    # Reproducibility triangle: data + code + results
    mlflow.log_param("dataset_hash", dvc_hash)     # which data (version identifier)
    mlflow.log_param("dataset_url", dataset_url)    # where data lives on the remote
    mlflow.log_param("git_commit", git_commit)      # which code
    mlflow.log_artifact("dataset.dvc")              # full descriptor — enables exact dvc pull

    # Training parameters
    mlflow.log_param("model", "my_model_name")
    mlflow.log_param("learning_rate", 0.001)

    # Results
    mlflow.log_metric("accuracy", 0.91)
    mlflow.log_artifact("model.pkl")

8. Dataset Update Workflow

When labels are corrected or new audio is added:

# Modify your files locally, then:
dvc add dataset/       # recompute the hash, update dataset.dvc
git commit -m "dataset v2 – corrected labels for session 3"
dvc push               # upload new/changed files to OVHcloud

Cleaning up old versions with dvc gc

Over time, old dataset versions accumulate in your OVHcloud bucket and can significantly increase storage costs. DVC provides a garbage collector to remove unused versions:

# Preview what would be deleted (dry run — safe to run)
dvc gc --workspace --cloud --remote ovh --dry

# Actually delete unused versions from the remote
dvc gc --workspace --cloud --remote ovh

9. Sharing Access

Recommended: create a dedicated read-only OpenStack user in OVHcloud for collaborators.

Result