Run 100,000 tasks with one API call: batches on a pool of GPUs

Some work is thousands of small, independent pieces: embedding a large dataset, scoring a model on thousands of prompts, rendering frames, sweeping hyperparameters, processing files. A batch hands the whole job to RentGPUPower in one request — up to 100,000 tasks — and spreads it over the free machines in the pool.

How a batch works

You send one command, plus either a number of tasks or a list of inputs, one per task. Each task runs your command on a machine, which tells it who it is through environment variables:

A failed attempt is tried again on another machine. Each task is billed like a job, by the second, and the batch stops starting tasks at its deadline, at its budget, or when your credits run out.

Try one from the command line

With the gpurent command line and a file words.json holding ["cat", "dog", "fox"]:

gpurent batch run 'echo "task $GPURENT_TASK_INDEX: $GPURENT_TASK_INPUT"' --inputs words.json --parallel 50
gpurent batch status bat_...
gpurent batch results bat_... --output

batch run waits and shows progress; add --no-wait to come back later.

A real one: a hyperparameter sweep

Each task reads its settings from GPURENT_TASK_INPUT and prints one line of results:

# /app/sweep.py — one task of the batch
import json, os

cfg = json.loads(os.environ["GPURENT_TASK_INPUT"])  # e.g. {"lr": 0.001, "layers": 4}
attempt = int(os.environ["GPURENT_TASK_ATTEMPT"])

score = train_and_evaluate(**cfg)                   # your own code
print(json.dumps({"config": cfg, "score": score, "attempt": attempt}))

Bake the script into your own public Docker image and send the batch:

curl https://rentgpupower.com/api/v1/compute/batches \
  -H "Authorization: Bearer $GPURENT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"image": "ghcr.io/you/sweep:1", "command": "python /app/sweep.py",
       "inputs": [{"lr": 0.001, "layers": 4}, {"lr": 0.0003, "layers": 8}],
       "max_parallel": 50, "budget_cents": 2000, "min_vram_gb": 16}'

Inputs that are not strings reach the task as JSON. Follow the batch with GET /batches/ID, page through each task's status and output with GET /batches/ID/tasks?output=1, and stop it with POST /batches/ID/cancel.

The options

Set image or template_id whenever your command needs Python: the machine's default is usually a plain CUDA image.

Good uses — and how to split them

• Embeddings over a big dataset: one input per shard — a file's URL — and each task stores its vectors.
• Evaluating a model on thousands of prompts: one input per slice of the prompt set; each task prints its scores.
• Rendering: one input per frame range, such as "1-50".
• Hyperparameter sweeps: one JSON config per task, as above.
• Data processing: one file or partition per task.

Tips

• Keep inputs small. Pass a URL or an ID, not the data itself.
• Make tasks safe to run twice. A retry starts over on another machine; GPURENT_TASK_ATTEMPT tells you which try it is.
• Send big results to your own storage, with a short-lived upload link per task rather than a long-lived key — rented machines are no place for secrets.
• Print a short result line; the results pages show each task's output.

FAQ

What does a batch cost?

Each task is billed like a job: by the second, at the hourly price of the machine it ran on. budget_cents caps the total, and GET /batches/ID shows what has been spent so far.

What happens when a task fails?

It is tried again on another machine, up to max_attempts (2 by default). After that it is marked failed and the rest carry on; gpurent batch results ID --status failed lists them.

How do I know when it is done?

Poll GET /batches/ID, wait with gpurent batch run, or give a webhook_url to be told when it finishes. Every endpoint is in the docs.