19 Receiver Operating Characteristic

Up until now, we have dealt with tests that are categorized as either positive or negative. However, many tests are quantitative rather than qualitative. For instance, the blood urea and nitrogen, serum cholesterol and serum protein among many others are measured on a continuous scale often ranging from 0 to infinity. Such tests pose different challenges as we often need a cut-off point to determine which range of values can be considered “normal” or “abnormal”. As we have learned above the sensitivity and specificity of a test are often used to define how good it is. For tests on a continuous scale the sensitivity and specificity change depending on the cut-off provided for the measure.

In this section, we use the lbw.csv data collected in a study conducted in a cohort of 350 newborns in Ghana. The identification of babies born with a weight of less than 2.5 kg is important because of the special needs they require. In many underdeveloped countries, however, the unavailability of a reliable weighing scale makes this a challenge. This has prompted the search for other surrogate measures easily available in rural areas for determining if a baby is a low birth weight (<2.5kgs). The study aimed to determine how well the length or chest circumference of a baby could be used as a surrogate indicator for low birth weight in newborns. If they turn out to be good tests they can easily be deployed in any rural area as the only instrument needed here would be a measuring tape. The variables collected include their study ID (sid), birth weight (bweight), sex (gender), chest circumference (chc) and length of baby (lgth).

First, we read the data after clearing the workspace

df_lbw <- 
    read_csv("./Data/lbw.csv") %>% 
    mutate(gender = factor(gender)) %>% 
    mutate(bwcat = ifelse(bweight < 2.5, 1, 0) %>% 
               factor(levels = c(0,1), labels = c("Normal","Low")))

And then summarize it

df_lbw %>% dfSummary(graph.col = F)
Data Frame Summary  
df_lbw  
Dimensions: 350 x 6  
Duplicates: 0  

---------------------------------------------------------------------------------------
No   Variable    Stats / Values              Freqs (% of Valid)    Valid      Missing  
---- ----------- --------------------------- --------------------- ---------- ---------
1    sid         Mean (sd) : 175.5 (101.2)   350 distinct values   350        0        
     [numeric]   min < med < max:                                  (100.0%)   (0.0%)   
                 1 < 175.5 < 350                                                       
                 IQR (CV) : 174.5 (0.6)                                                

2    bweight     Mean (sd) : 2.9 (0.6)       33 distinct values    350        0        
     [numeric]   min < med < max:                                  (100.0%)   (0.0%)   
                 1 < 2.9 < 4.5                                                         
                 IQR (CV) : 0.7 (0.2)                                                  

3    gender      1. Female                   166 (47.4%)           350        0        
     [factor]    2. Male                     184 (52.6%)           (100.0%)   (0.0%)   

4    chc         Mean (sd) : 31.7 (3.2)      115 distinct values   350        0        
     [numeric]   min < med < max:                                  (100.0%)   (0.0%)   
                 20 < 32 < 42.1                                                        
                 IQR (CV) : 4 (0.1)                                                    

5    lgth        Mean (sd) : 46.8 (4.3)      109 distinct values   350        0        
     [numeric]   min < med < max:                                  (100.0%)   (0.0%)   
                 28 < 47 < 60.3                                                        
                 IQR (CV) : 5.5 (0.1)                                                  

6    bwcat       1. Normal                   268 (76.6%)           350        0        
     [factor]    2. Low                       82 (23.4%)           (100.0%)   (0.0%)   
---------------------------------------------------------------------------------------

Next, I introduce the pROC package. The function roc() from the package will be used extensively in this section.

19.1 Sensitivity, specificity and cut-offs

As mentioned above when one has a gold standard which is bivariate (indicating presence or absence) and a quantitative test, the sensitivity and specificity of the test depends on the cut-off chosen. For our lbw.csv data our gold standard for a low birth weight baby is the birth weight categorised into low birth weight (LBW) or normal birth weight (NBW). Two continuous measures, the chest circumference and the length of the baby were used as our tests.

To illustrate the relationship between the various cut-offs with sensitivity and specificity we generate some arbitrary cut-offs.

cut.off <- c(28, 42, 44, 46, 47, 49, 50, 51, 61)

From these cut-offs, we generate the categories from the length of the babies and tabulate the resultant categorical variable.

df_lbw <- 
    df_lbw %>% 
    mutate(lgthcat = cut(lgth, br=cut.off, include.lowest=T))

And then count the various groups

df_lbw %>% 
    gtsummary::tbl_summary(
        include = lgthcat,
        digits = lgthcat ~ c(0,1)
    ) %>% 
    gtsummary::bold_labels()

Characteristic	N = 3501
lgthcat
[28,42]	49 (14.0%)
(42,44]	36 (10.3%)
(44,46]	59 (16.9%)
(46,47]	35 (10.0%)
(47,49]	67 (19.1%)
(49,50]	39 (11.1%)
(50,51]	25 (7.1%)
(51,61]	40 (11.4%)
1 n (%)

Next, we determine the sensitivities and specificities at the various cut-offs by using the roc() function from the pROC package. Since this package requires an ordered categorical variable we convert lgthcat into an ordered factor variable. And then go ahead to impute the relevant information into the roc() function