---
title: "Workload Shapes: When (and How) to Use staRburst"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Workload Shapes: When (and How) to Use staRburst}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(eval = FALSE)
```

Whether staRburst helps or hurts is not about your *field* — it's about the **shape**
of your work. This guide walks through the levers that decide it, each with a runnable
pattern and, where it matters, **real measured numbers** from `bench/benchmark.R` (see
`bench/README.md`).

> **Measurement setup (all tables below).** staRburst 0.3.9; AWS us-east-1; EC2
> `c7i.xlarge` Spot workers; cold start unless noted. **Local baseline machine:**
> Apple M4 Pro (Mac16,11) — 12 cores (8 performance + 4 efficiency), 48 GB RAM, macOS
> 26.5.2, R 4.6.1. Local baselines are run **sequentially** (single core), so they are
> a *conservative* reference: your laptop's multi-core `future`/`parallel` time would
> be lower, and the cloud's advantage correspondingly narrower. Tested 2026-07-21.

There are four levers. Learn to read them and you can predict the outcome for any
workload:

1. **Task length** — how much work each task does.
2. **Task count** — how many tasks you create (and whether to batch).
3. **Fan-out width** — how many workers you spread across.
4. **Cost vs. time** — what you're actually optimizing.

Plus one pro tip on **where each task writes its output**.

---

## Lever 1 — Task length: is each task worth shipping?

Every task pays a fixed tax: it's serialized, uploaded to S3, pulled by a worker, and
its result uploaded back. Plus the pool pays a one-time ~75–90 s cold start. If the
task's actual compute is smaller than that tax, the cloud only makes things slower.

Two measured runs, same 20–30 workers, **opposite verdicts** — the only thing that
changed is per-task work:

| Shape | Tasks | Local (seq) | Cloud total (cold) | Verdict |
|---|---|---|---|---|
| **Heavy** — ~10 s of SVD each | 30 | 319.3 s (5.3 min) | **125.8 s** (87.5 s startup + 38.4 s compute) | ✅ ~2.5× faster, $0.06 |
| **Light** — sub-second each | 20 | 0.1 s | 102.6 s | ❌ ~1000× slower, $0.03 |

```{r}
# Heavy tasks (minutes each) — the cloud wins
results <- starburst_map(scenarios, run_expensive_model, workers = 30)

# Trivially cheap tasks — just do it locally
results <- lapply(inputs, quick_function)
```

**Rule:** if each task is sub-second, stay local. Seconds-to-minutes per task, with
many of them, is the sweet spot.

---

## Lever 2 — Task count: batch, or fall off a cliff

This is the one that surprises people. staRburst creates **one task per element of
`.x`**, and each task is a separate client-side S3 submit *and* collect — in a serial
loop. It does **not** batch for you. So a "big parameter walk" submitted naively as one
task per point doesn't just get slow — it hits a wall on the *client*, before any
compute happens.

Same 10,000-point sweep, two ways — **both measured on live AWS, identical work**:

| Approach | Tasks | Items | Startup | Compute + collect | Cloud total | Est. cost |
|---|---|---|---|---|---|---|
| ✅ **Batched** (100 tasks × 100) | 100 | 10,000 | 105.0 s | **70.4 s** | **175.4 s** (2.9 min) | **$0.13** |
| ❌ **Naive** (1 task per point) | 10,000 | 10,000 | 77.0 s | **4,571.9 s** | **4,648.9 s** (77.5 min) | **$3.46** |

Read that again: the **same 10,000 evaluations** took **2.9 minutes for $0.13** batched,
versus **77 minutes for $3.46** naive — a **~65× slowdown and ~27× cost blowup**, purely
from task count. The naive run spent over an hour with the client submitting 10,000
individual tasks to S3 one at a time (~1.5/sec) while 50 workers sat mostly idle — you
pay for the workers the whole time. Nothing about the computation changed; only how it
was packaged.

This is the trap behind "just fan out my 10M-point sweep": at 1M or 10M tasks the naive
path doesn't take minutes longer, it takes **hours-to-days** and never gets cheaper.
Batching isn't an optimization here — it's the difference between working and not.

```{r}
# ❌ DON'T: 10,000 tasks — ~10,000 serial S3 submits from your laptop
results <- starburst_map(1:10000, evaluate_point, workers = 50)

# ✅ DO: batch into ~100 chunks — 100 tasks, same 10,000 evaluations
batches <- split(1:10000, ceiling(seq_along(1:10000) / 100))
results <- starburst_map(batches, function(ids) lapply(ids, evaluate_point),
                         workers = 50)
results <- unlist(results, recursive = FALSE)  # flatten
```

Pick a batch size so each task runs for a couple of minutes:

```{r}
per_item_seconds <- 0.5              # measure this on a small local sample
target_task_seconds <- 120           # aim for ~2 min per task
batch_size <- ceiling(target_task_seconds / per_item_seconds)  # ~240 here
```

**Rule:** thousands of tiny tasks is an anti-pattern. Batch fine-grained work so you
have ~dozens-to-hundreds of tasks, each doing real work. A 1M- or 10M-point sweep
*must* be batched — never submitted one point at a time.

---

## Lever 3 — Fan-out width: more workers is not always better

Once you've sized your tasks, how many workers should you request? More workers cut
wall-clock **only up to the number of tasks**, and with diminishing returns:

- **Workers > tasks:** the extra workers sit idle — you pay for them, they do nothing.
  30 tasks on 100 workers still only uses 30.
- **Stragglers:** the batch isn't done until the *slowest* task is. Past a point,
  adding workers doesn't help because you're waiting on one long task, not queue depth.
- **Quota & startup:** more workers = more instances to launch (longer, more likely to
  hit limits) and, on Fargate, more vCPU quota consumed.
- **Cost scales with workers, speed doesn't (past #tasks).** Doubling workers beyond
  your task count doubles cost for zero speedup.

```{r}
# 200 tasks: ~50 workers is a reasonable ratio (each worker handles ~4 tasks)
results <- starburst_map(param_sets, fit_model, workers = 50)

# 200 tasks on 500 workers: 300 workers idle, you pay for all 500 — don't
```

**Rule of thumb:** workers ≈ number of tasks (capped at what your quota and budget
allow), so each worker gets roughly one to a few tasks. Beyond `workers == tasks`
there is no speedup left to buy.

---

## Lever 4 — Cost vs. time: what are you optimizing?

staRburst reports an estimated cost per run. The key intuition: **wall-clock and cost
trade off differently.**

- Adding workers (up to #tasks) **buys wall-clock at ~constant total cost** — the same
  compute-seconds are just spread over more machines for less time. That's the good
  trade: finish sooner for roughly the same dollars.
- Adding workers **beyond #tasks** buys nothing and **raises cost** (idle instances).
- **Spot instances** (staRburst's default, `use_spot = TRUE`) are ~70% cheaper than
  on-demand for interruption-tolerant batch work — almost always the right default here.
- **Warm pools** amortize the ~90 s startup across many runs, so repeated jobs
  (parameter sweeps, daily pipelines) get dramatically better cost/time than a single
  cold run implies.

```{r}
# Cheapest for a one-off: enough workers to cover the tasks, spot on (default)
starburst_map(tasks, fn, workers = length(tasks), use_spot = TRUE)

# Repeated runs: a warm pool pays startup once — see the detached-sessions guide
```

**Rule:** optimize wall-clock by matching workers to tasks; optimize cost by using
spot (default) and warm pools for anything you run more than once.

---

## Pro tip — shard your S3 writes across prefixes

If your tasks write their *own* output to S3 (large results you don't want to serialize
back through the result channel), have **each task write to a distinct prefix**, not
all to the same one. S3 scales throughput per prefix, so hundreds of tasks hammering a
single prefix serialize against each other, while spreading them across prefixes lets
S3 parallelize:

```{r}
# ❌ Hundreds of tasks writing under ONE prefix — throughput bottleneck
key <- sprintf("s3://bucket/results/%d.parquet", i)

# ✅ Shard across prefixes so S3 parallelizes the writes
shard <- i %% 100
key <- sprintf("s3://bucket/results/shard=%02d/%d.parquet", shard, i)
```

A cheap, high-leverage change for write-heavy fan-outs: same data, far more aggregate
write throughput, because you're no longer funneling every worker through one hot
prefix.

---

## Putting it together: a diagnosis checklist

Before you burst a workload, ask:

1. **Is each task more than a second of work?** If no → stay local (Lever 1).
2. **Do you have thousands of tiny tasks?** If yes → batch to ~dozens–hundreds (Lever 2).
3. **Are you requesting more workers than tasks?** If yes → cap at #tasks (Lever 3).
4. **Will you run it more than once?** If yes → warm pool + spot (Lever 4).
5. **Do tasks write big outputs to S3?** If yes → shard across prefixes (pro tip).

Get these right and staRburst turns a multi-hour local grind into minutes for pennies.
Get them wrong and you'll pay more to go slower — the mechanics don't care which field
you're in, only about the shape of the work.

See also: `vignette("performance")` for the underlying overhead model, and
`vignette("detached-sessions")` for warm pools and long-running jobs.
