Skip to main content

Advertisement

Table 4 “Out-of-the-box” individual PHI recall results for partial and fully-contained matches using the VHA evaluation corpus

From: Evaluating current automatic de-identification methods with Veteran’s health administration clinical documents

RULE-BASED SYSTEMS
PHI type #Inst. PARTIAL MATCHES FULLY-CONTAINED MATCHES
   HMS Scrubber MeDS MIT deid HMS Scrubber MeDS MIT deid
Patient Name 206 0.83 0.99 0.98 0.57 0.69 0.69
Relative Name 30 0.76 0.95 1 0.57 0.67 0.77
Healthcare Provider Name 492 0.74 0.96 0.94 0.43 0.47 0.38
Other Person Name 20 0.66 0.81 0.74 0.30 0.25 0.35
Street City 137 0.90 0.96 0.81 0.70 0.78 0.78
State Country 161 0.45 0.49 0.85 0.43 0.45 0.84
Deployment 43 0.34 0.49 0.27 0.07 0.02 0.05
ZIP code 4 1 1 1 1 1 1
Healthcare Unit Name 1453 0.45 0.51 0.12 0.24 0.23 0.03
Other Org Name 86 0.33 0.50 0.27 0.03 0.20 0.03
Date 2547 0.74 0.87 0.80 0.34 0.27 0.46
Age > 89 4 0 0 1 0 0 1
Phone Number 90 0.73 0.79 0.80 0.42 0.5 0.48
Electronic Address 4 0 0.86 0.75 0 0 0.75
SSN 16 1 1 1 1 1 1
Other ID Number 123 0.66 0.82 0.41 0.43 0.61 0.27
MACHINE LEARNING-BASED SYSTEMS
PHI type #Inst. PARTIAL MATCHES FULLY-CONTAINED MATCHES
   MIST HIDE MIST HIDE
Patient Name 206 0.51 0.54 0.42 0.50
Relative Name 30 0.13 0.13 0.13 0.13
Healthcare Provider Name 492 0.53 0.59 0.44 0.53
Other Person Name 20 0 0.20 0 0.15
Street City 137 0.26 0.29 0.26 0.27
State Country 161 0.14 0.22 0.14 0.21
Deployment 43 0.07 0.05 0.07 0.02
ZIP code 4 0 0.75 0 0.75
Healthcare Unit Name 1453 0.09 0.09 0.06 0.05
Other Org Name 86 0.09 0.07 0.06 0.06
Date 2547 0.72 0.73 0.39 0.38
Age > 89 4 0 0 0 0
Phone Number 90 0.34 0.61 0.24 0.51
Electronic Address 4 0 0 0 0
SSN 16 0.56 0.87 0.56 0.87
Other ID Number 123 0.32 0.69 0.20 0.63