Type: Package
Title: Detection of Grey Zones in Two-Way Inter-Rater Agreement Tables
Version: 0.0.5
Date: 2024-09-18
Author: Haydar Demirhan ORCID iD [aut, cre, cph], Ayfer Ezgi Yilmaz [aut]
Maintainer: Haydar Demirhan <haydar.demirhan@rmit.edu.au>
Description: Grey zones locally occur in an agreement table due to the subjective evaluation of raters based on various factors such as not having uniform guidelines, the differences between the raters' level of expertise or low variability among the level of the categorical variable. It is important to detect grey zones since they cause a negative bias in the estimate of the agreement level. This package provides a function for detecting the existence of grey zones in two-way inter-rater agreement tables (Demirhan and Yilmaz (2023) <doi:10.1186/s12874-022-01759-7>).
Depends: irrCAC (≥ 1.0)
License: GPL-3
RoxygenNote: 7.2.3
NeedsCompilation: no
Repository: CRAN
Packaged: 2024-09-18 01:53:48 UTC; haydardemirhan
Date/Publication: 2024-09-18 13:40:02 UTC

Detection of Grey Zones in Two-way Inter-rater Agreement Tables

Description

Detects the existence of grey zones in in two-way inter-rater agreement tables.

Usage

detectGreyZones(table)

Arguments

table

A matrix object that has equal number of columns and rows showing the agreement table for two raters.

Details

See Demirhan and Yilmaz (2023) for the details of the approach implemented by this function.

If a grey zone is detected in the agreement table, the function reports AC2 and BP with quadratic weights.

If no grey zone is detected in the agreement table, the function reports kappa, AC2 and BP with linear and quadratic weights.

Value

delta

A matrix that shows the relative magnitude of deviance from the perfect agreement to the level of agreement for each cell of the given agreement table. Corresponds to Eq. (6) of Demirhan and Yilmaz (2023).

Delta

A value that shows the maximum of the values in delta. Corresponds to Eq. (7) of Demirhan and Yilmaz (2023).

tau_Delta

The value of the threshold for the detection of grey zones. If Delta > tau_Delta then there is a grey zone exists in the agreement table. Corresponds to Eq. (10) of Demirhan and Yilmaz (2023).

result

Verbal result of the comparison of Delta and tau_Delta.

agreement

Weighted agreement coefficient with linear and quadratic weights. See details.

Author(s)

Haydar Demirhan and Ayfer Ezi Yilmaz

Maintainer: Haydar Demirhan <haydar.demirhan@rmit.edu.au>

References

Demirhan, H., Yilmaz, A.E., 2023, Detection of grey zones in inter-rater agreement studies, BMC Medical Research Methodology, 23, Article number: 3, DOI: 10.1186/s12874-022-01759-7.

Tran, D., Demirhan, H., Dolgun, A., 2021, Bayesian approaches to the weighted kappa-like inter-rater agreement measures, Statistical Methods in Medical Research, 30, 10, 2329-2351, DOI: 10.1177/09622802211037068.

Tran, D., Dolgun, A., Demirhan, H., 2021, The impact of grey zones on the accuracy of agreement measures for ordinal tables, BMC Medical Research Methodology, 21:70, DOI: 10.1186/s12874-021-01248-3.

Examples

table <- matrix(c(10, 2, 7, 1, 3, 3, 0, 8, 80), nrow = 3, ncol = 3, byrow = TRUE) 
detectGreyZones(table)