---
title: "Vignette 3: Accessing Interim Objects from Cluster Estimation" 
output: rmarkdown::html_vignette
vignette: > 
  %\VignetteIndexEntry{Vignette 3: Accessing Interim Objects from Cluster Estimation}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  echo = FALSE, # hide code
  message = FALSE, # hide messages
  warning = FALSE, # hide warnings
  fig.width = 12,
  fig.height = 7,
  out.width = "100%"
)
```

```{r setup}
library(gsClusterDetect)
```

# Overview

The `find_clusters()` function is the main entry point for this package, and under typical conditions, the function either returns a message indicating that no clusters were found, or returns an object of class "clusters", which is a two element list containing 1) the clusters found and 2) information about the locations in each of those clusters. However, callers can pass `return_interim=True` to obtain the interim or intermediate objects that are estimated under the hood when `find_clusters()` is called.

## Functions called by `find_clusters()`

`find_clusters()` is a wrapper function around a number of other package functions, and it calls the below functions, in order. When the user passes `interim_results=TRUE` the outputs of each of these functions are stored separately within an expanded output list.

+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Function                                                                                                                                                                                                                                                                                                                 | Output                                                                                                                                                                                                                                                                                            |
+==========================================================================================================================================================================================================================================================================================================================+===================================================================================================================================================================================================================================================================================================+
| `generate_case_grids()` : this function take the case counts and the detect date (see `?generate_case_grids` for more options) and separates out the counts by baseline and test date interval, and estimates cumulative sums of cases by location.                                                                      | An object of class `CaseGrids`, which is a list of items:                                                                                                                                                                                                                                          |
|                                                                                                                                                                                                                                                                                                                          |                                                                                                                                                                                                                                                                                                   |
|                                                                                                                                                                                                                                                                                                                          | - baseline_counts_by_location: a dataframe of counts by location during the baseline interval                                                                                                                                                                                                     |
|                                                                                                                                                                                                                                                                                                                          |                                                                                                                                                                                                                                                                                                   |
|                                                                                                                                                                                                                                                                                                                          | - case_grid: a frame of cases during the test period, with reverse cumulative counts within location, by date                                                                                                                                                                                     |
|                                                                                                                                                                                                                                                                                                                          |                                                                                                                                                                                                                                                                                                   |
|                                                                                                                                                                                                                                                                                                                          | - case_grid_totals_by_date: a frame of the reverse cumulative sum of counts over all locations, by date                                                                                                                                                                                           |
|                                                                                                                                                                                                                                                                                                                          |                                                                                                                                                                                                                                                                                                   |
|                                                                                                                                                                                                                                                                                                                          | - test_cases: a frame of cases during the test period, by and location                                                                                                                                                                                                                            |
|                                                                                                                                                                                                                                                                                                                          |                                                                                                                                                                                                                                                                                                   |
|                                                                                                                                                                                                                                                                                                                          | - detect_date: the detect date passed to this function                                                                                                                                                                                                                                            |
|                                                                                                                                                                                                                                                                                                                          |                                                                                                                                                                                                                                                                                                   |
|                                                                                                                                                                                                                                                                                                                          | - baseline_total: the total number of cases over all locations and dates                                                                                                                                                                                                                          |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| `gen_nearby_case_info()` : This function takes an object of class `CaseGrids` distance matrix, and a distance threshold, and estimates, for each location, the estimated cumulative sum of cases in nearby locations. Two frames for these estimates are provided: one for test period and one for baseline              | An object of class `NearbyClusterGrids`, which is a list of two data frames, including:                                                                                                                                                                                                           |
|                                                                                                                                                                                                                                                                                                                          |                                                                                                                                                                                                                                                                                                   |
|                                                                                                                                                                                                                                                                                                                          | - "baseline", holding the nearby information for baseline counts                                                                                                                                                                                                                                  |
|                                                                                                                                                                                                                                                                                                                          |                                                                                                                                                                                                                                                                                                   |
|                                                                                                                                                                                                                                                                                                                          | - "test", holding the nearby information for the test interval                                                                                                                                                                                                                                    |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| `generate_observed_expected()`: This function takes objects of classes `NearbyClusterGrids` and `CaseGrids` and derives the observed and expected calculation for all target locations                                                                                                                                   | An object of class `ObservedExpectedGrid` , which is a frame that contains the observed and expected value for all target locations.                                                                                                                                                              |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| `add_spline_threshold()`: this function ingests an object of class `ObservedExpectedGrid` and used either an internal or user provided spline lookup table to filter the `ObservedExpectedGrid` by those that meet the spline classification threshold                                                                   | An object of class `ClusterAlertTable` which is a dataframe of all target candidate clusters that are reduced to those meeting the spline threshold                                                                                                                                               |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| `compress_clusters_fast()`: This function is a faster version of the deprecated (but accessible `compress_clusters()`. Both this function and the deprecated version take the `ClusterAlertTable` from the previous step and find only the most significant, non-overlapping clusters from the large table of candidates | An object of class `clusters`, which is a named list of two items:                                                                                                                                                                                                                                |
|                                                                                                                                                                                                                                                                                                                          |                                                                                                                                                                                                                                                                                                   |
|                                                                                                                                                                                                                                                                                                                          | - The first item is a data frame named "cluster_alert_table", which holds one row per cluster, including a column "id" which enables linking to the list in the second item (see below)                                                                                                           |
|                                                                                                                                                                                                                                                                                                                          |                                                                                                                                                                                                                                                                                                   |
|                                                                                                                                                                                                                                                                                                                          | - The second item is a data frame named "clust_loc_list" which is a list of cluster centers, indexed by values "id" in the first item (see above) and containing a vector of locations that are included in that cluster.                                                                         |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| `add_location_counts()`: this function ingests an object of type `clusters` and obtains location counts and sums for all locations within the clusters                                                                                                                                                                   | Returns an object of type `clusters` which has a similar structure as above, but adds total number of locations to the first item (i.e. the table of identified clusters) and converts the second item from a list to a data frame that contains the sums for all locations within the clusters.  |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

: Table: Functions called by `find_clusters()`

See help via `?<function>` for further details on each of these above functions, including optional and required parameters and examples of how to use them.

## Example Usage:

Note: we set `return_interim` to `TRUE`

```{r, echo=TRUE}
result <- find_clusters(
  cases = example_count_data,
  distance_matrix = county_distance_matrix("OH")[["distance_matrix"]],
  detect_date = example_count_data[, max(date)],
  distance_limit = 50,
  return_interim = TRUE
)

# objects included in the output
for (n in names(result)) {
  cat(
    "Name: ",
    n,
    "\nClass: ",
    paste(class(result[[n]]), collapse = " "),
    "\n\n",
    sep = ""
  )
}
```
