Research Information System
Dil Eğitimi ve Araştırmaları Dergisi, vol.11, no.1, pp.369-403, 2025 (TRDizin)
Rubrics offer a solution to assessment inconsistencies frequently observed among evaluators at Turkish teaching centers, both within and across institutions. Particularly for productive language skills, rubrics can mitigate subjective evaluation by providing structured assessment frameworks. This research aimed to develop valid and reliable analytic rubrics for assessing B1-B2 level writing skills in Turkish language instruction. Using an exploratory sequential mixed methods design, the study followed systematic rubric development protocols. Validity was established through expert consultation with seven specialists using the Davis technique, while reliability was determined by having 20 evaluators score six writing samples, analyzed using Kendall's W coefficient of concordance. The study also examined whether evaluation experience (Kruskal Wallis-H test) or instructor background (foreign language vs. second language Turkish instruction; Mann Whitney U test) influenced scoring patterns when using the rubrics. Results confirmed the developed rubrics' validity and reliability. No significant differences emerged between ratings from experienced versus inexperienced instructors, or between foreign language and second language Turkish instructors, demonstrating the rubrics' effectiveness in standardizing assessment across evaluator profiles. These findings suggest that properly constructed rubrics can successfully reduce assessment inconsistencies regardless of evaluator experience or instructional background.