Detecting the patient’s need for help with machine learning based on expressions

Lahti, Lauri

doi:10.1186/s12874-021-01502-8

BMC Medical Research Methodology

Table 4 Expression statements (ES) having statistically significant the “need for help” rating differences in the grouping based on the answer values of each background question (BQ), and evaluation about how well the convolutional neural network model can learn a labeling that matches the grouping (n = 673). M = mean, Mdn=median, SD=standard deviation

From: Detecting the patient’s need for help with machine learning based on expressions

	Statistically significant rating differences	Training and validation metrics ² of the convolutional neural network model to learn a labeling that matches the grouping					Comparison of the validation accuracy with the probability of pure chance
Grouping based on the answer value (x) of the background question (BQ)	Expression statements (ES) having statistically significant rating differences in the grouping (the difference of mean ratings ¹ about the need for help)	Epoch step	Training loss	Training accuracy	Validation loss	Validation accuracy	Probability of pure chance of classifying the rating profiles correctly ³ (based on the size of the greatest group)	Difference of the mean validation accuracy and the probability of pure chance of classifying the rating profiles correctly
BQ1, two groups: x < 7 (n₁=263), x>=7 (n₂=410)	ES6: diff_g1&g2=0,07 (95% CI [0,03; 0,11], p = 0.0011); ES8: diff_g1&g2=-0,08 (95% CI [-0,14; -0,02], p = 0.0073); ES9: diff_g1&g2=-0,08 (95% CI [-0,14; -0,02], p = 0.0068); ES10: diff_g1&g2=-0,09 (95% CI [-0,15; -0,02], p = 0.0049); ES7: diff_g1&g2=-0,05 (95% CI [-0,10; 0,00], p = 0.0384); ES16: diff_g1&g2=-0,06 (95% CI [-0,12; 0,00], p = 0.0403); ES17: diff_g1&g2=-0,08 (95% CI [-0,14; -0,02], p = 0.0143); ES18: diff_g1&g2=-0,05 (95% CI [-0,10; 0,00], p = 0.0358);	M = 11.26 Mdn=11 SD=2.39	M = 0.55 Mdn=0.55 SD=0.03	M = 0.73 Mdn=0.72 SD=0.02	M = 0.59 Mdn=0.59 SD=0.01	M = 0.69 Mdn=0.69 SD=0.02	0.61	0.08
BQ1, three groups: x < 6 (n₁= 218), 6<=x < 8 (n₂=207), x>=8 (n₃=248)	ES5: diff_g1&g3=-0,01 (95% CI [-0,05; 0,03], p = 0.446), diff_g1&g2=-0,05 (95% CI [-0,09; 0,00], p = 0.054), diff_g2&g3=0,04 (95% CI [0,00; 0,08], p = 0.102), p_g1&g2&g3=0.0449; ES8: diff_g1&g3=-0,08 (95% CI [-0,15; -0,02], p = 0.067), diff_g1&g2=-0,07 (95% CI [-0,14; 0,00], p = 0.067), diff_g2&g3=-0,01 (95% CI [-0,07; 0,06], p = 0.929), p_g1&g2&g3=0.0489; ES9: diff_g1&g3=-0,09 (95% CI [-0,16; -0,02], p = 0.050), diff_g1&g2=-0,08 (95% CI [-0,15; 0,00], p = 0.058), diff_g2&g3=-0,01 (95% CI [-0,08; 0,06], p = 0.891), p_g1&g2&g3=0.0355; ES11: diff_g1&g3=0,08 (95% CI [0,02; 0,14], p = 0.015), diff_g1&g2=0,01 (95% CI [-0,05; 0,07], p = 0.858), diff_g2&g3=0,07 (95% CI [0,02; 0,13], p = 0.015), p_g1&g2&g3=0.0108;	M = 4.85 Mdn=4 SD=1.8	M = 1.02 Mdn=1.03 SD=0.03	M = 0.48 Mdn=0.47 SD=0.03	M = 1.06 Mdn=1.06 SD=0.01	M = 0.40 Mdn=0.40 SD=0.02	0.37	0.03
BQ2, two groups: x < 2 (n₁=219), x>=2 (n₂=454)	ES11: diff_g1g2=-0,08 (95% CI [-0,13; -0,03], p = 0.0014); ES6: diff_g1g2=-0,06 (95% CI [-0,10; -0,02], p = 0.0039); ES3: diff_g1g2=0,06 (95% CI [0,01; 0,12], p = 0.0476); ES14: diff_g1g2=0,06 (95% CI [-0,01; 0,12], p = 0.04); ES15: diff_g1g2=0,08 (95% CI [0,01; 0,14], p = 0.0189);	M = 5.55 Mdn=6 SD=2.91	M = 0.57 Mdn=0.56 SD=0.04	M = 0.69 Mdn=0.69 SD=0.02	M = 0.63 Mdn=0.63 SD=0.01	M = 0.66 Mdn=0.66 SD=0.02	0.67	-0.01
BQ4, two groups: x < 2 (n₁=364), x>=2 (n₂=309)	ES6: diff_g1g2=-0,06 (95% CI [-0,09; -0,02], p = 0.0064); ES11: diff_g1g2=-0,06 (95% CI [-0,10; -0,01], p = 0.0165);	M = 3.44 Mdn=3 SD=1.28	M = 0.67 Mdn=0.67 SD=0.01	M = 0.59 Mdn=0.60 SD=0.02	M = 0.68 Mdn=0.67 SD=0	M = 0.57 Mdn=0.57 SD=0.02	0.54	0.03
BQ5, two groups: x < 7 (n₁=274), x>=7 (n₂=399)	ES6: diff_g1g2=0,06 (95% CI [0,02; 0,10], p = 0.0024); ES9: diff_g1g2=-0,08 (95% CI [-0,14; -0,02], p = 0.0043); ES10: diff_g1g2=-0,08 (95% CI [-0,15; -0,02], p = 0.0036); ES11: diff_g1g2=0,06 (95% CI [0,01; 0,11], p = 0.0168); ES16: diff_g1g2=-0,06 (95% CI [-0,12; -0,01], p = 0.0271); ES17: diff_g1g2=-0,07 (95% CI [-0,13; -0,01], p = 0.0303);	M = 3.27 Mdn=3 SD=1.65	M = 0.64 Mdn=0.63 SD=0.02	M = 0.65 Mdn=0.65 SD=0.03	M = 0.66 Mdn=0.67 SD=0	M = 0.60 Mdn=0.60 SD=0.02	0.59	0,01
BQ5, three groups: x < 6 (n₁=190), 6<=x < 8 (n₂=271), x>=8 (n₃=212)	ES9: diff_g1&g3=-0,15 (95% CI [-0,22; -0,07], p = 0.0005), diff_g1&g2=-0,09 (95% CI [-0,17; -0,02], p = 0.0230), diff_g2&g3=-0,05 (95% CI [-0,12; 0,02], p = 0.0965), p_g1&g2&g3=0.0007; ES10: diff_g1&g3=-0,15 (95% CI [-0,23; -0,07], p = 0.0004), diff_g1&g2=-0,09 (95% CI [-0,17; -0,02], p = 0.0112), diff_g2&g3=-0,06 (95% CI [-0,13; 0,02], p = 0.1699), p_g1&g2&g3=0.0005; ES6: diff_g1&g3=0,07 (95% CI [0,03; 0,12], p = 0.016), diff_g1&g2=0,02 (95% CI [-0,02; 0,07], p = 0.393), diff_g2&g3=0,05 (95% CI [0,01; 0,09], p = 0.023), p_g1&g2&g3=0.0093; ES8: diff_g1&g3=-0,11 (95% CI [-0,18; -0,04], p = 0.013), diff_g1&g2=-0,09 (95% CI [-0,16; -0,02], p = 0.013), diff_g2&g3=-0,02 (95% CI [-0,08; 0,05], p = 0.985), p_g1&g2&g3=0.0117; ES16: diff_g1&g3=-0,09 (95% CI [-0,16; -0,02], p = 0.04), diff_g1&g2=-0,08 (95% CI [-0,15; -0,01], p = 0.04), diff_g2&g3=-0,01 (95% CI [-0,08; 0,05], p = 0.76), p_g1&g2&g3=0.0301; ES17: diff_g1&g3=-0,10 (95% CI [-0,18; -0,03], p = 0.04), diff_g1&g2=-0,08 (95% CI [-0,15; -0,01], p = 0.06), diff_g2&g3=-0,02 (95% CI [-0,09; 0,04], p = 0.53), p_g1&g2&g3=0.0329; ES20: diff_g1&g3=0,04 (95% CI [-0,02; 0,09], p = 0.022), diff_g1&g2=-0,01 (95% CI [-0,07; 0,04], p = 0.928), diff_g2&g3=0,05 (95% CI [0,00; 0,10], p = 0.022), p_g1&g2&g3=0.0139;	M = 3.63 Mdn=4 SD=1.33	M = 1.05 Mdn=1.05 SD=0.02	M = 0.44 Mdn=0.44 SD=0.03	M = 1.07 Mdn=1.07 SD=0.01	M = 0.42 Mdn=0.43 SD=0.03	0.40	0.02
BQ6, two groups: x < 7 (n₁=318), x>=7 (n₂=355)	ES11: diff_g1&g2=0,08 (95% CI [0,04; 0,13], p = 0.0006); ES6: diff_g1&g2=0,06 (95% CI [0,02; 0,09], p = 0.0056);	M = 5.35 Mdn=5 SD=1.61	M = 0.62 Mdn=0.63 SD=0.02	M = 0.63 Mdn=0.63 SD=0.03	M = 0.65 Mdn=0.65 SD=0	M = 0.60 Mdn=0.60 SD=0,02	0.53	0.07
BQ6, three groups: x < 6 (n₁=240), 6<=x < 8 (n₂=229), x>=8 (n₃=204)	ES11: diff_g1&g3=0,09 (95% CI [0,03; 0,15], p = 0.0077), diff_g1&g2=0,03 (95% CI [-0,03; 0,08], p = 0.3516), diff_g2&g3=0,06 (95% CI [0,00; 0,12], p = 0.0649), p_g1&g2&g3=0.0098; ES6: diff_g1&g3=0,07 (95% CI [0,02; 0,12], p = 0.019), diff_g1&g2=0,04 (95% CI [0,00; 0,09], p = 0.141), diff_g2&g3=0,03 (95% CI [-0,02; 0,07], p = 0.141), p_g1&g2&g3=0.0199;	M = 3.89 Mdn=4 SD=1.8	M = 1.05 Mdn=1.06 SD=0.03	M = 0.41 Mdn=0.41 SD=0.03	M = 1.08 Mdn=1.08 SD=0	M = 0.39 Mdn=0.39 SD=0.03	0.36	0.03
BQ7, two groups: x < 7 (n₁=201), x>=7 (n₂=472)	ES6: diff_g1&g2=0,08 (95% CI [0,04; 0,12], p = 0.0005); ES11: diff_g1&g2=0,07 (95% CI [0,02; 0,12], p = 0.0078); ES19: diff_g1&g2=0,07 (95% CI [0,02; 0,11], p = 0.0048);	M = 7.26 Mdn=7 SD=1.63	M = 0.53 Mdn=0.54 SD=0.02	M = 0.75 Mdn=0.74 SD=0.01	M = 0.59 Mdn=0.59 SD=0	M = 0.72 Mdn=0.72 SD=0.01	0.70	0.02
BQ7, three groups: x < 6 (n₁=143), 6<=x < 8 (n₂=214), x>=8 (n₃=316)	ES6: diff_g1&g3=0,09 (95% CI [0,04; 0,13], p = 0.0051), diff_g1&g2=0,04 (95% CI [-0,01; 0,09], p = 0.1801), diff_g2&g3=0,05 (95% CI [0,00; 0,09], p = 0.0619), p_g1&g2&g3=0.0042; ES11: diff_g1&g3=0,10 (95% CI [0,04; 0,16], p = 0.0086), diff_g1&g2=0,03 (95% CI [-0,04; 0,09], p = 0.4526), diff_g2&g3=0,07 (95% CI [0,02; 0,12], p = 0.0186), p_g1&g2&g3=0.0035;	M = 1.31 Mdn=1 SD=0.61	M = 1.05 Mdn=1.06 SD=0.02	M = 0.45 Mdn=0.45 SD=0.02	M = 1.07 Mdn=1.07 SD=0.01	M = 0.47 Mdn=0.48 SD=0.02	0.47	0.00
BQ8, two groups: x < 2 (n₁=123), x>=2 (n₂=550)	ES4: diff_g1&g2=-0,11 (95% CI [-0,17; -0,05], p = 0.0001); ES12: diff_g1&g2=-0,13 (95% CI [-0,20; -0,06], p = 0.0002); ES14: diff_g1&g2=-0,20 (95% CI [-0,27; -0,13], p = 0.0000); ES15: diff_g1&g2=-0,20 (95% CI [-0,28; -0,12], p = 0.0000); ES3: diff_g1&g2=-0,10 (95% CI [-0,16; -0,03], p = 0.0031); ES10: diff_g1&g2=-0,12 (95% CI [-0,20; -0,04], p = 0.0050); ES11: diff_g1&g2=-0,08 (95% CI [-0,14; -0,02], p = 0.0058); ES8: diff_g1&g2=-0,09 (95% CI [-0,16; -0,02], p = 0.0225); ES9: diff_g1&g2=-0,10 (95% CI [-0,18; -0,03], p = 0.0142); ES13: diff_g1&g2=-0,07 (95% CI [-0,14; -0,01], p = 0.0223); ES16: diff_g1&g2=-0,10 (95% CI [-0,17; -0,02], p = 0.0159); ES17: diff_g1&g2=-0,09 (95% CI [-0,17; -0,02], p = 0.0319); ES18: diff_g1&g2=-0,08 (95% CI [-0,14; -0,01], p = 0.0242);	M = 6.14 Mdn=6 SD=1.69	M = 0.42 Mdn=0.42 SD=0.02	M = 0.83 Mdn=0.83 SD=0.01	M = 0.48 Mdn=0.48 SD=0.01	M = 0.79 Mdn=0.78 SD=0.01	0.82	-0.03
BQ9, two groups: x < 51 (n₁=333), x>=51 (n₂=340)	ES1: diff_g1&g2=0,09 (95% CI [0,05; 0,12], p = 0.0000); ES2: diff_g1&g2=0,10 (95% CI [0,06; 0,13], p = 0.0000); ES3: diff_g1&g2=0,13 (95% CI [0,08; 0,18], p = 0.0000); ES4: diff_g1&g2=0,10 (95% CI [0,05; 0,15], p = 0.0006); ES5: diff_g1&g2=0,08 (95% CI [0,04; 0,11], p = 0.0000); ES14: diff_g1&g2=0,14 (95% CI [0,08; 0,19], p = 0.0001); ES15: diff_g1&g2=0,14 (95% CI [0,08; 0,20], p = 0.0001); ES7: diff_g1&g2=0,06 (95% CI [0,01; 0,10], p = 0.0133); ES8: diff_g1&g2=0,09 (95% CI [0,04; 0,15], p = 0.0466); ES11: diff_g1&g2=-0,05 (95% CI [-0,09; 0,00], p = 0.0485); ES13: diff_g1&g2=0,05 (95% CI [0,01; 0,10], p = 0.0297); ES19: diff_g1&g2=0,04 (95% CI [0,00; 0,08], p = 0.0193);	M = 5.79 Mdn=6 SD=1.44	M = 0.58 Mdn=0.58 SD=0.02	M = 0.69 Mdn=0.69 SD=0.02	M = 0.61 Mdn=0.61 SD=0.01	M = 0.68 Mdn=0.68 SD=0.02	0.51	0.17
BQ9, three groups: x < 40 (n₁=225), 40<=x < 60 (n₂=231), x>=60 (n₃=217)	ES1: diff_g1&g3=0,10 (95% CI [0,06; 0,14], p = 0.0000), diff_g1&g2=0,07 (95% CI [0,03; 0,11], p = 0.0002), diff_g2&g3=0,03 (95% CI [-0,01; 0,06], p = 0.0716), p_g1&g2&g3=0.0000; ES2: diff_g1&g3=0,12 (95% CI [0,08; 0,16], p = 0.0000), diff_g1&g2=0,07 (95% CI [0,03; 0,11], p = 0.0007), diff_g2&g3=0,05 (95% CI [0,01; 0,09], p = 0.0162), p_g1&g2&g3=0.0000; ES3: diff_g1&g3=0,17 (95% CI [0,11; 0,23], p = 0.0000), diff_g1&g2=0,06 (95% CI [0,01; 0,12], p = 0.110), diff_g2&g3=0,10 (95% CI [0,04; 0,17], p = 0.003), p_g1&g2&g3=0.0000; ES4: diff_g1&g3=0,13 (95% CI [0,07; 0,19], p = 0.0011), diff_g1&g2=0,02 (95% CI [-0,04; 0,07], p = 0.5450), diff_g2&g3=0,11 (95% CI [0,05; 0,18], p = 0.0011), p_g1&g2&g3=0.0004; ES5: diff_g1&g3=0,09 (95% CI [0,04; 0,13], p = 0.0000), diff_g1&g2=0,03 (95% CI [-0,01; 0,08], p = 0.042), diff_g2&g3=0,05 (95% CI [0,01; 0,10], p = 0.012), p_g1&g2&g3=0.0000; ES14: diff_g1&g3=0,17 (95% CI [0,11; 0,24], p = 0.0002), diff_g1&g2=0,05 (95% CI [-0,01; 0,12], p = 0.6097), diff_g2&g3=0,12 (95% CI [0,05; 0,19], p = 0.0031), p_g1&g2&g3=0.0002; ES15: diff_g1&g3=0,18 (95% CI [0,11; 0,25], p = 0.0004), diff_g1&g2=0,06 (95% CI [0,00; 0,13], p = 0.5430), diff_g2&g3=0,12 (95% CI [0,04; 0,19], p = 0.0082), p_g1&g2&g3=0.0006; ES7: diff_g1&g3=0,09 (95% CI [0,03; 0,15], p = 0.0064), diff_g1&g2=0,01 (95% CI [-0,04; 0,06], p = 0.7293), diff_g2&g3=0,08 (95% CI [0,02; 0,14], p = 0.0120), p_g1&g2&g3=0.0043; ES11: diff_g1&g3=-0,08 (95% CI [-0,14; -0,03], p = 0.0069), diff_g1&g2=-0,10 (95% CI [-0,15; -0,04], p = 0.0033), diff_g2&g3=0,02 (95% CI [-0,04; 0,07], p = 0.5752), p_g1&g2&g3=0.0017; ES8: diff_g1&g3=0,13 (95% CI [0,06; 0,20], p = 0.016), diff_g1&g2=0,03 (95% CI [-0,04; 0,09], p = 0.956), diff_g2&g3=0,11 (95% CI [0,04; 0,18], p = 0.016), p_g1&g2&g3=0.0116; ES10: diff_g1&g3=0,14 (95% CI [0,07; 0,22], p = 0.034), diff_g1&g2=0,02 (95% CI [-0,05; 0,09], p = 0.995), diff_g2&g3=0,12 (95% CI [0,04; 0,20], p = 0.034), p_g1&g2&g3=0.0245; ES19: diff_g1&g3=0,06 (95% CI [0,01; 0,11], p = 0.034), diff_g1&g2=0,03 (95% CI [-0,01; 0,08], p = 0.198), diff_g2&g3=0,03 (95% CI [-0,03; 0,08], p = 0.198), p_g1&g2&g3=0.0351; ES20: diff_g1&g3=-0,07 (95% CI [-0,12; -0,02], p = 0.127), diff_g1&g2=0,01 (95% CI [-0,04; 0,06], p = 0.268), diff_g2&g3=-0,08 (95% CI [-0,13; -0,02], p = 0.026), p_g1&g2&g3=0.0253;	M = 7.13 Mdn=7 SD=1.45	M = 0.93 Mdn=0.93 SD=0.03	M = 0.54 Mdn=0.54 SD=0.03	M = 0.98 Mdn=0.98 SD=0.01	M = 0.50 Mdn=0.50 SD=0.03	0.34	0.16

¹ For groupings of two groups the difference of mean ratings (each mean rating in the range 0.0-1.0) is computed by the formula (M₁-M₂), and for groupings of three groups by the formula max({(M₁-M₃),(M₁-M₂),(M₂-M₃)}). Wilcoxon rank-sum test (for two groups) and Kruskal-Wallis test (for three groups) indicate the statistically significant rating differences (p < 0.05) between groups, each rating difference (diff_g1&g3, diff_g1&g2 and diff_g2&g3) supplied with the corresponding 95% confidence interval (CI) and the p-value of the Wilcoxon pairwise comparison. The parameter p_g1&g2&g3 shows the p-value of the Kruskal-Wallis test for three groups
² Training and validation metrics of the convolutional neural network model are averaged from 100 separate training and validation sequences to learn a labeling that matches the grouping (n = 673)
³ For groupings of two groups the probability of pure chance of classifying the rating profiles correctly is computed by the formula (max({n₁,n₂}))/(n₁+n₂), and for groupings of three groups by the formula (max({n₁,n₂,n₃}))/(n₁+n₂+n₃)

Back to article page

ISSN: 1471-2288

Contact us

General enquiries: journalsubmissions@springernature.com