--- title: 'How the Play-By-Play Pipeline Works' output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{How the Play-By-Play Pipeline Works} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r setup, include = FALSE} knitr::opts_chunk$set( collapse = TRUE, comment = '#>', fig.align = 'center', out.width = '92%', fig.width = 7, fig.height = 4.5 ) make_table <- function(x, caption, digits = 3) { knitr::kable(x, caption = caption, digits = digits) } ``` ## Why This Article Exists The play-by-play functions are the package's most opinionated tools. They do not simply flatten an NHL endpoint and hand the result back. They reconcile multiple public sources, repair a few known event-order problems, derive a large public schema, and add event-level on-ice player IDs from the official HTML report. That makes `gc_play_by_play()` and `wsc_play_by_play()` extremely useful, but it also means the output deserves a mental model. This article is that model. It is strictly informational: the goal is to explain what the pipeline trusts, what it derives, what it refuses to infer, and where `shift_chart()` fits. ## Two Feeds, One Public Schema `gc_play_by_play()` and `wsc_play_by_play()` start from different event feeds, but both return a cleaned event table. The source split looks like this: ```{r source-table, echo = FALSE} source_table <- data.frame( source = c( 'GameCenter API', 'World Showcase API', 'HTML play-by-play report', 'Stats shift-chart API', 'HTML shift reports' ), used_by = c( 'gc_play_by_play(), game metadata for WSC', 'wsc_play_by_play()', 'gc_play_by_play(), wsc_play_by_play()', 'shift_chart()', 'shift_chart() fallback' ), role = c( 'Authoritative event stream for GameCenter rows and common metadata.', 'Alternate event stream with UTC timestamps and fewer clip fields.', 'Primary source for event-level on-ice goalie and skater IDs.', 'Preferred source for player shift start/end clocks.', 'Fallback source for shift start/end clocks when stats API rows are missing.' ), stringsAsFactors = FALSE ) make_table( source_table, caption = 'Public sources used by the play-by-play and shift-chart pipeline.' ) ``` The important separation is identity versus timing. HTML play-by-play reports are used to recover who was on the ice for an event. Shift charts are used later to calculate how long those players had been on the ice. ```{r basic-use, eval = FALSE} pbp <- nhlscraper::gc_play_by_play(2023030417) shifts <- nhlscraper::shift_chart(2023030417) pbp <- nhlscraper::add_shift_times(pbp, shifts) ``` ## Pipeline Map The high-level flow is deliberately conservative. The API event feed remains the backbone. The HTML report is matched back onto that backbone. Derived features are added only after the event order and public naming are stable. ```{r pipeline-plot, echo = FALSE, fig.width = 7.5, fig.height = 5.6, fig.cap = 'Conceptual order of the play-by-play cleanup pipeline.'} steps <- c( 'Fetch API and HTML', 'Normalize source names', 'Repair clear order defects', 'Parse situationCode', 'Normalize coordinates', 'Add shot context', 'Parse HTML on-ice rows', 'Match HTML to API rows', 'Finalize public schema' ) box_x <- c(1, 2, 3, 3, 2, 1, 1, 2, 3) box_y <- c(3, 3, 3, 2, 2, 2, 1, 1, 1) box_w <- 0.78 box_h <- 0.34 draw_box <- function(x, y, label, number) { wrapped_label <- paste(strwrap(paste0(number, '. ', label), width = 18), collapse = '\n') graphics::rect( xleft = x - box_w / 2, ybottom = y - box_h / 2, xright = x + box_w / 2, ytop = y + box_h / 2, col = '#edf6f9', border = '#006d77', lwd = 1.5 ) graphics::text( x = x, y = y, labels = wrapped_label, cex = 0.74, col = '#1f2933' ) } draw_arrow <- function(i, j) { x0 <- box_x[i] y0 <- box_y[i] x1 <- box_x[j] y1 <- box_y[j] graphics::arrows( x0 = x0 + sign(x1 - x0) * box_w / 2, y0 = y0 + sign(y1 - y0) * box_h / 2, x1 = x1 - sign(x1 - x0) * box_w / 2, y1 = y1 - sign(y1 - y0) * box_h / 2, length = 0.08, lwd = 1.3, col = '#6c757d' ) } graphics::plot( NA_real_, NA_real_, type = 'n', axes = FALSE, xlab = '', ylab = '', xlim = c(0.45, 3.55), ylim = c(0.55, 3.45) ) for (i in seq_len(length(steps) - 1L)) { draw_arrow(i, i + 1L) } for (i in seq_along(steps)) { draw_box(box_x[i], box_y[i], steps[i], i) } ``` ## What Gets Repaired The package is not trying to rewrite the game. It fixes only issues that are plainly incompatible with the event clock or with the package's public schema. ```{r repair-table, echo = FALSE} repair_table <- data.frame( issue = c( 'Source-specific column names', 'Goal row without shootingPlayerId', 'Blocked-shot zone perspective', 'Impossible period-boundary order', 'Missing public output columns' ), action = c( 'Rename to stable public names such as periodNumber and eventTypeDescKey.', 'Use scoringPlayerId as shootingPlayerId when the scorer is known.', 'Normalize blocked shots to the shooting-team perspective.', 'Drop or repair only the small set of rows that violate clock boundaries.', 'Allocate typed NA columns so GC and WSC outputs keep stable schemas.' ), stringsAsFactors = FALSE ) make_table( repair_table, caption = 'Conservative repairs made before public output is finalized.' ) ``` The design goal is auditability. When something can be derived directly, the package derives it. When something would require a hockey assumption that cannot be verified from the public sources, the package leaves the public state alone. ## `situationCode` Means State, Not Identity The raw `situationCode` is parsed into manpower state columns: - `homeIsEmptyNet` - `awayIsEmptyNet` - `homeSkaterCount` - `awaySkaterCount` - `isEmptyNetFor` - `isEmptyNetAgainst` - `skaterCountFor` - `skaterCountAgainst` - `manDifferential` - `strengthState` Those columns describe the intended rules state of the event. They are not recalculated from the HTML on-ice player list. That distinction is important because the official HTML report can list extra skaters around line changes, delayed penalties, bench situations, or reporting quirks. ## HTML Matching The HTML report is not joined to the API feed only by clock. A single game can have multiple events in the same second, and a clock-only join would create bad matches. The package builds matching signatures from: ```{r match-table, echo = FALSE} match_table <- data.frame( feature = c( 'Period', 'Elapsed seconds', 'Event type', 'Event-owning team', 'Primary actor', 'Secondary and tertiary actors', 'Local event order' ), why_it_matters = c( 'Prevents same-clock matches across periods.', 'Keeps the match near the official event time.', 'Separates faceoffs, shots, hits, penalties, goals, and stoppages.', 'Prevents same-time events by opposite teams from swapping.', 'Anchors the event to the shooter, scorer, hitter, winner, or penalized player.', 'Disambiguates goals, blocks, faceoffs, hits, and penalties.', 'Avoids backwards matches in duplicate clusters.' ), stringsAsFactors = FALSE ) make_table( match_table, caption = 'Signals used when matching HTML rows back to API events.' ) ``` Once a match is accepted, the HTML on-ice IDs are written into scalar columns: `homeGoaliePlayerId`, `awayGoaliePlayerId`, `homeSkater1PlayerId`, `awaySkater1PlayerId`, `skater1PlayerIdFor`, `skater1PlayerIdAgainst`, and so on. Five skater slots are guaranteed; additional `skater6`, `skater7`, and later slots appear only when a game requires them. ## One-On-One Rows Are Special Penalty shots and shootouts are not ordinary strength states. Rows such as `0101` and `1010` are constrained to the shooter and defending goalie. Even if the HTML report shows extra players, the public on-ice identity columns are kept to the one-on-one context because that is the play being recorded. ## Delayed-Penalty Backfill Some delayed-penalty marker rows do not appear as ordinary event rows in the HTML report. When that happens, the package can backfill on-ice IDs from the nearest prior compatible event in the same period. The backfill is allowed only when the state signature is unchanged, the time gap is tiny, and the prior row already has a populated on-ice set. This is intentionally narrow. ## GC and WSC Differences The public schemas are aligned, but not identical. GameCenter rows include GameCenter clip fields such as `highlightClip`, `discreteClip`, and `pptReplayUrl`. WSC rows include `utc` immediately after game elapsed seconds and omit the GC-only clip fields. That source-specific difference is intentional because the source feeds do not expose exactly the same metadata. ## Practical Checklist Use this sequence when building event-level analysis: ```{r workflow, eval = FALSE} # Load cleaned event stream. pbp <- nhlscraper::gc_play_by_play(2023030417) # Add shift timing when fatigue or shift length matters. shifts <- nhlscraper::shift_chart(2023030417) pbp <- nhlscraper::add_shift_times(pbp, shifts) # Add movement context when previous-event geometry matters. pbp <- nhlscraper::add_deltas(pbp) # Score shots when chance quality matters. pbp <- nhlscraper::calculate_expected_goals(pbp) ``` Use `gc_play_by_play_raw()` or `wsc_play_by_play_raw()` only when you want to inspect the upstream feed itself. Use `gc_play_by_play()` or `wsc_play_by_play()` when you want the package's public analysis schema. ## Key Takeaway The pipeline keeps three ideas separate: - API events define what happened and in what order. - `situationCode` defines the intended manpower state. - HTML reports define who was listed on the ice. That separation is what makes the output practical for research while still being honest about where each piece of information came from.