---
title: "Getting Started"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Getting Started}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
can_decrypt <- httr2::secret_has_key('KHIS_KEY')

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  error = TRUE,
  purl = can_decrypt,
  eval = can_decrypt
)
options(tibble.print_min = 5L, tibble.print_max = 5L)
```

```{r, eval = !can_decrypt, echo = FALSE, comment = NA}
message("No token available. Code chunks will not be evaluated.")
```

```{r, eval = can_decrypt, include=FALSE}
khisr:::khis_cred_docs()
```

```{r, include=FALSE}
library(khisr)
```

The `khisr` R package simplifies interaction with the District Health Information System 2 (DHIS2) platform. Designed for researchers and public health professionals, `khisr` streamlines data retrieval and analysis, saving you valuable time compared to manual methods.

## Authentication

`khisr` prioritizes security by operating in authenticated mode by default. This ensures you interact with DHIS2 as a recognized user. To begin exploring DHIS2 data, you'll need to establish your credentials.

### Setting Your Credentials:

1. **Obtain Credentials**: Secure your DHIS2 username and password through appropriate channels within the DHIS2 organization.

2. **Store Credentials Securely**: `khisr` offers a convenient way to store your credentials within your R environment. Refer to the comprehensive guide, [Set Your Credentials](https://khisr.damurka.com/articles/set-your-credentials.html), for detailed instructions on setting and managing credentials effectively.

```{r, eval = FALSE}
# Set the credentials using username and password
khis_cred(username = 'your-dhis2-username', password = 'your-dhis2-password', server = 'https://<your dhis2 instance>')

# Set the credentials using a Personal Access Token (DHIS2's recommended
# method for scripts and integrations) instead of username/password
khis_cred(token = 'your-dhis2-token', server = 'https://<your dhis2 instance>')

# Set credentials using configuration path
khis_cred(config_path = 'path/to/secret.json')
```
**Note**: Replace placeholders like '*your-dhis2-username*' and '*path/to/your/secret.json*' with your actual credentials and file path.

## Metadata

DHIS2 utilizes metadata to define the structure and meaning of its data. Explore the [data dimensions](https://khisr.damurka.com/articles/data-dimensions.html) resource for a deeper understanding.

### Metadata helpers in khisr

`khisr` provides a set of high-level functions — one per DHIS2 metadata
type (`get_organisation_units()`, `get_data_elements()`, `get_programs()`,
and around 25 others) — that all share the same interface and can be
filtered the same way. See [`?metadata-helpers`](https://khisr.damurka.com/reference/metadata-helpers.html)
for the full list, and your R IDE's auto-complete for faster typing.

### Metadata object filter

`khisr` filters retrieved metadata using DHIS2's **property:operator:value**
pattern, exposed through [`metadata_filter()`](https://khisr.damurka.com/reference/metadata-filter.html)
and a matching set of infix operators (`%.eq%`, `%.like%`, `%.in%`, and
about 20 more — see `?metadata_filter` for the complete list with
descriptions).

### Working with metadata filters

Basic usage of the metadata filter

```{r, eval = can_decrypt}
# Retrieve organisation units by province (level 2)
province <- get_organisation_units(level %.eq% '2')
province

# Retrieve province by name (Vientiane Capital)
province <- get_organisation_units(level %.eq% '2',
                                   name %.like% 'vientiane capital')
province

data_element_id <- c('lYsfXxCw6Qi', 'GxlrIgMyEf4')

# Retrieve data elements by ID using operator in
data_elements <- get_data_elements(id %.in% data_element_id)
data_elements

# Retrieve data elements by filtering using dataElementGroups
data_elements <- get_data_elements(dataElementGroups.name %.like% 'malaria')
data_elements
```

## Data analytics

The analytics resource in DHIS2 empowers you to access and analyze aggregated data
across various dimensions. To effectively leverage this resource, let's explore the
key functions and parameters involved:

### Key Functions

- `get_analytics()`: Retrieves aggregated data based on specified dimensions and
  filters.
- `get_data_value_sets()`: Retrieves the individually entered raw data values
  behind the aggregates — useful for data-quality auditing.
- `analytics_dimension()`: Constructs dimensions for queries, ensuring accurate
  data retrieval.
- `%.d%` (infix operator): Convenient shorthand for creating dimension filters.
- `%.f%` (infix operator): Convenient shorthand for creating filter dimensions.
- `get_event_analytics_aggregate()`/`get_enrollment_analytics_aggregate()`:
  pivot-table style totals over Tracker data — see [Tracker
  Data](https://khisr.damurka.com/articles/tracker.html).

### Dimension (dx)

The `dimension` query parameter defines which dimensions should be included in the
analytics query. Any number of dimensions can be specified. The dimension parameter
should be repeated for each dimension to include in the query response. The query
response can potentially contain aggregated values for all combinations of the
specified dimension items. The fixed dimensions are the **data element** _(dx)_
**period (time)** _(pe)_ and **organisation unit**  _(ou)_ dimension. You can 
dynamically add dimensions through categories, data element group sets and 
organisation unit group sets.

| Dimension ID | Dimensions                                                                                                                                     |
|:-------------|:------------------------------------------------------------------------------------------------------------------------------------------------|
| `dx`         | Data elements, indicators, data set reporting rate metrics, data element operands, program indicators, program data elements, program attributes, validation rules |
| `pe`         | ISO periods and relative periods (see [Date and Period Format](https://khisr.damurka.com/articles/date-format.html))                            |
| `ou`         | Organisation unit hierarchy: organisation unit identifiers, or keywords `USER_ORGUNIT`, `USER_ORGUNIT_CHILDREN`, `USER_ORGUNIT_GRANDCHILDREN`, `LEVEL-<level>`, and `OU_GROUP-<group-id>` |
| `co`         | Category option combo identifiers (use `all` to get all items)                                                                                   |
| `ao`         | Attribute option combo identifiers (use `all` to get all items)                                                                                   |


### Filter (filter)

The `filter` parameter defines which dimensions should be used as filters for the
data retrieved in the analytics query. Any number of filters can be specified. The
filter parameter should be repeated for each filter to use in the query. A filter
differs from a dimension in that the filter dimensions will not be part of the
query response content, and that the aggregated values in the response will be
collapsed on the filter dimensions. In other words, the data in the response will
be aggregated on the filter dimensions, but the filters will not be included as
dimensions in the actual response.

### Constructing Queries

- **Specify Dimensions:** Use the dimensions parameter to define the dimensions 
  you want to include in the query response. Leverage the infix operators %.d% for 
  concise and readable code.

```{r, eval = can_decrypt}
# To include a list dimensions for data elements id, dataset ids
dx %.d% c('dimension-id-1', 'dimension-id-2')

pe %.d% 'LAST_YEAR'

ou %.d% 'USER_ORGUNIT'

# showing in the analytics
get_analytics(
    dx %.d% c('lYsfXxCw6Qi', 'vTRrNdOOT9g', 'GxlrIgMyEf4'),
    pe %.d% 'LAST_YEAR',
    ou %.d% c('W6sNfkJcXGC')
)

# Using the startDate and endDate with organisation unit keyword 'USER_ORGUNIT'
get_analytics(
    dx %.d% c('lYsfXxCw6Qi', 'vTRrNdOOT9g', 'GxlrIgMyEf4'),
    ou %.d% 'USER_ORGUNIT',
    pe %.d% 'all',
    startDate = '2023-07-01',
    endDate = '2023-12-31'
)
```

- **Apply Filters:** Use the filters parameter to specify dimensions for filtering
  data without including them in the response.
  
```{r, eval = can_decrypt}
# Filter by period
pe %.f% 'LAST_YEAR'

# Filter by organisation unit
ou %.f% 'USER_ORGUNIT'

# showing in the analytics. filter by organisation unit with id 'W6sNfkJcXGC'
# and period 'LAST_YEAR'
get_analytics(
    dx %.d% c('lYsfXxCw6Qi', 'vTRrNdOOT9g', 'GxlrIgMyEf4'),
    pe %.f% 'LAST_YEAR',
    ou %.f% 'W6sNfkJcXGC'
)
```

### Data quality

Alongside the aggregated values themselves, DHIS2 exposes a few endpoints for
checking the *quality* of reported data:

| khisr function                             | Retrieves                                              |
|:--------------------------------------------|:--------------------------------------------------------|
| `get_complete_data_set_registrations()`     | Raw completeness records — who marked a data set complete, and when. |
| `get_analytics_outliers()`                  | Data values flagged as statistical outliers.           |
| `get_validation_results()`                  | Violated validation rules for an org unit/period range. |
| `get_data_value_audits()`                   | Change history for a data value.                       |

```{r, eval = can_decrypt}
# Completeness registrations for a data set at a province and everything
# below it, for a single period
get_complete_data_set_registrations(
    data_sets = 'VEM58nY22sO',
    org_units = 'W6sNfkJcXGC',
    children = TRUE,
    periods = '202301'
)
```

`get_analytics_outliers()` and `get_validation_results()` require the
authenticated user to have the corresponding DHIS2 authority (outlier
detection or validation analysis); without it, DHIS2 returns an
authorisation error rather than empty results.

## Tracker data

Alongside aggregate analytics, DHIS2 also stores case-based, person-level data
through its Tracker API. `khisr` provides `get_tracked_entities()`,
`get_enrollments()`, and `get_events()` for reading it, plus
`tracked_entity_filter()` for filtering tracked entities by attribute value. See
[Tracker Data](https://khisr.damurka.com/articles/tracker.html) for a full guide.

```{r, eval = can_decrypt}
# Tracked entities enrolled in a program, at an org unit and everything below it
get_tracked_entities(
    program = 'PREnRHSp3be',
    org_units = 'IWp9dQGM0bS',
    org_unit_mode = 'DESCENDANTS'
)
```

## System & utilities

A handful of functions cover the DHIS2 instance itself, rather than its
health data:

| khisr function                  | Retrieves                                                        |
|:----------------------------------|:-------------------------------------------------------------------|
| `get_system_info()`               | DHIS2 version, build, and server info.                           |
| `get_geo_features()`              | Organisation unit coordinates/boundaries, for mapping.           |
| `get_sql_views()`/`get_sql_view_data()` | Predefined SQL views, and their data.                        |
| `get_data_store_namespaces()`/`get_data_store_keys()`/`get_data_store_value()` | The system or user key/value data store. |
| `get_file_resources()`            | Metadata (not contents) of files stored in the instance.          |

```{r, eval = can_decrypt}
get_system_info()$version

# Coordinates/boundaries for every province (level 2)
get_geo_features(org_units = 'LEVEL-2')

# Reading the key/value data store
namespaces <- get_data_store_namespaces()
namespaces
get_data_store_keys(namespaces[1])
```

`get_sql_view_data()` requires the authenticated user to be authorised to
read the specific SQL view; DHIS2 returns an error rather than empty
results if not.