An engagement-aware predictive model to evaluate problem-solving performance from the study of adult skills' (PIAAC 2012) process data

Large-scale Assessments in Education

An IERI – International Educational Research Institute Journal

Table 3 Experiment Results 2–Task (Item)-level Average Prediction Performance

^aDV	Item	Attention-LSTM		Effort-Aware Attention-LSTM
		Performance		Performance		Engagement
		ACC	F1	ACC	F1	ACC	F1
United States	1	0.774	0.861	0.802	0.857	0.893	0.971
	2	0.861	0.848	0.764	0.830	0.804	0.879
	3	0.792	0.824	0.720	0.785	0.865	0.897
	4	0.603	0.813	0.842	0.921	0.879	0.998
	5	0.723	0.773	0.976	0.981	0.949	0.938
South Korea	1	0.844	0.936	0.842	0.934	0.869	0.892
	2	0.833	0.815	0.820	0.817	0.821	0.833
	3	0.827	0.813	0.858	0.733	0.864	0.841
	4	0.642	0.824	0.824	0.952	0.860	0.885
	5	0.53	0.752	0.758	0.992	0.869	0.952
United Kingdom	1	0.823	0.884	0.812	0.887	0.866	0.975
	2	0.860	0.839	0.837	0.860	0.824	0.896
	3	0.834	0.841	0.845	0.861	0.769	0.759
	4	0.553	0.806	0.789	0.899	0.879	0.998
	5	0.686	0.711	0.891	0.991	0.952	0.970

^aDV: Dependent Variable; The performance and engagement level is simultaneously predicted in the second model