|Year : 2021 | Volume
| Issue : 4 | Page : 69-72
Assessment of inter-rater and intra-rater reliability of index of orthodontic treatment need index by newly trained orthodontic residents
Khyati Mahida, Ajay Kubavat, Shrish Charan Srivastava, Manish Desai, Harshit Patel, Harsh S Modi
Department of Orthodontics and Dentofacial Orthopaedics, Narsinhbhai Patel Dental College and Hospital, Visnagar, Gujarat, India
|Date of Submission||27-Feb-2021|
|Date of Decision||28-Mar-2021|
|Date of Acceptance||18-Apr-2021|
|Date of Web Publication||16-Oct-2021|
Department of Orthodontics and Dentofacial Orthopaedics, Narsinhbhai Patel Dental College and Hospital, Visnagar, Gujarat
Source of Support: None, Conflict of Interest: None
Introduction: This study aimed to test the reliability of the index of treatment need by newly trained orthodontic residents. Materials and Methods: A total of 100 patients were assessed by the panel of four consultant orthodontists using the index of orthodontic treatment need (IOTN) index, and the agreed category was set as the 'gold standard'. Four residents of the department in the initial stages of formal orthodontic training and lacking the knowledge of using the IOTN index were trained to score the treatment need according to the IOTN index. All four residents were then asked to score the 100 sets of models using both the indexes of IOTN. Results: Kappa scores for inter-rater agreement with the expert panel for the major categories (1–4) demonstrated moderate agreement (kappa: 0.59–0.5623) for all raters. The percentage agreement ranged from 84.03% to 88.10% in all cases. Intra-rater agreement for the major categories was fair to moderate (kappa: 0.53–0.80).Conclusions: The IOTN index is a reliable index to be used for determining the priority of treatment needs in orthodontic treatment. The clarity and simplicity make the index easy to learn and apply with minimal training. The simplicity of the IOTN index makes it easy to learn and apply by novices.
Keywords: Aesthetic component, dental health component, index of orthodontic treatment need
|How to cite this article:|
Mahida K, Kubavat A, Srivastava SC, Desai M, Patel H, Modi HS. Assessment of inter-rater and intra-rater reliability of index of orthodontic treatment need index by newly trained orthodontic residents. Adv Hum Biol 2021;11:69-72
|How to cite this URL:|
Mahida K, Kubavat A, Srivastava SC, Desai M, Patel H, Modi HS. Assessment of inter-rater and intra-rater reliability of index of orthodontic treatment need index by newly trained orthodontic residents. Adv Hum Biol [serial online] 2021 [cited 2021 Dec 4];11:69-72. Available from: https://www.aihbonline.com/text.asp?2021/11/4/69/328393
| Introduction|| |
All over the world, indices are being used in health sciences for a variety of purposes like classification of disease and evaluating the prognosis of treatment outcome. Indices can be of use in planning treatment and health policy for a population by determining the prevalence of a disease or malocclusion. In recent times, the planning of services, especially by the publically funded health services (UK National Health Service), has shown great interest. The index of orthodontic treatment need (IOTN) has been used in NHS primary care in England and Wales since 2006. IOTN index was developed with the prime objective of evaluating the functional need for treatment through its dental health component (DHC). The psychosocial need, on the other hand, is prioritised by its aesthetic component (AC). The psychosocial need on the other hand is prioritised by it Aesthetic component (AC) and Indices such as the handicapping labiolingual deviation index (Draker, 1960), the treatment priority index.
The handicapping malocclusion assessment record and the occlusal index were developed earlier than the IOTN, but IOTN has become quite popular. The IOTN actually is a modification of an index previously developed by the Swedish Dental Health Board. The aim of this article is to test the inter-rater and intra-rater reliability of the orthodontic residents in initial years of formal orthodontic training in scoring the malocclusion according to the treatment needs using the IOTN index.
| Materials and Methods|| |
The panel of four expert orthodontists worked in pairs to score 100 sets of study models using the IOTN index. The scores were then compared, and experts came to a common understanding of the score by sorting out the differences by discussion. Four residents of the department in the initial stages of formal orthodontic training and lacking the knowledge of using the IOTN index were trained to score the treatment need according to the IOTN index. All four residents were then asked to score the 100 sets of models using both the components of the IOTNs to test agreement with the expert panel scores. The scores were then analysed using Cohen's kappa for inter-operator agreement with the expert panel scores. For the inter-rater agreement, the scores of two raters were compared, wherein the ratings were taken at two different time intervals 2 months apart.
| Results|| |
The scores of the raters were analysed using statistical methods to determine the percentage of agreement between the raters, and the inter-rater and intra-rater reliability was determined by calculating the Cohen kappa scores. Inter-rater agreement of AC for rater 1 with rater 2 had 88.10572687224669% of agreement with kappa value k: 0.594401429422275 showing a moderate agreement between the two raters. Rater 1 with rater 3 had 85.1063829787234% of agreement, with kappa value of 0.5710560625814863 showing a moderate agreement between the two raters and rater 1 with rater 4 had 84.03361344537815% of agreement with kappa value of 0.5623729797735411 showing a moderate agreement. Inter-rater agreement for the DHC showed that rater 1 had a fair agreement with a percentage of agreement of 68.02721088435374% and Cohen's k : 0.359; rater 1 with rater 3 also had a fair agreement (67.11409395973155% Cohen's k: 0.34219299035949197), whereas rater 1 with rater 4 showed a moderate agreement (% of agreement: 70.6713780918728% Cohen's k: 0.406108568683472) [Graph 1] and [Graph 2].
Inter-rater reliability of AC of rater 1T1 (rater 1 at time T1) and rater 1T2 (rater 1 at time 2) had percentage of agreement 85.1063829787234% with Cohen's k: 0.5710560625814863 showing a moderate agreement, whereas rater 2T1 and rater 2T2 had a percentage of agreement of 80.97165991902834% with Cohen's k: 0.5335315626632378 showing a moderate agreement [Graph 3] and [Graph 4].
| Discussion|| |
Numerous studies have been conducted to assess the validity and reliability of the IOTN. The strengths of the IOTN DHC component are its validity, reliability and its ease of use.
The present study was undertaken to evaluate the inter-rater reliability and intra-rater reliability of the IOTN index. The inter-rater reliability measures the extent to which two or more raters or examiners agree. It can be evaluated by using percentage agreement and Cohen's kappa statistics. High inter-rater reliability values depict a high percentage of agreement, and lower values show a low degree of agreement between the raters.
In this study, moderate, inter-rater reliability was found with respect to the scores of the AC of the IOTN and fair-to-moderate agreement was found for DHC scores. This shows that the validity and reliability of the IOTN index are sufficiently good. The discrepancy in the finding of the resident and the expert panel can be attributed to the differences in clinical experience. One of the advantages of IOTN is that the grading is unaffected by age. This is one of the cardinal reasons for its wide acceptance in orthodontic research.
The IOTN DHC is a relatively straightforward 5-point scale, with the greatest need for treatment classified as being Group 5 and little or no need for treatment classified as Group 1. Each group has well-defined descriptors of the features of the malocclusion deemed as indicators of orthodontic need. The index is quick and easy to apply because malocclusion is scored simply on the worst feature. To identify this feature in a systematic manner, it is recommended that the assessor uses the acronym MOCDO (missing teeth, overjet, crossbites, displacement of contact points and overbite). This is why even the novice orthodontist with minimal orthodontic experience could score the malocclusion with sufficient accuracy comparable to that of the gold standard set by the panel of an experienced orthodontist. Apart from the many advantages, there are some limitations of the IOTN. For example, the AC of the index comprises only Class I and Class II division 1 incisor relationships, and there are no Class II division 2 or Class III incisor relationships. This probably could be the reason for the differences in the cores of the raters.
To evaluate the test–retest reliability, two raters re-assessed the study models, which they had assessed in the inter-rater reliability phase. The re-assessments took place 2 months after the inter-rater reliability phase to reduce the possibility of recalling the scores of the first phase.
IOTN is a reliable tool for planning the budget and improve the focus of services by ensuring greater uniformity in the assessment of orthodontic treatment needs. The index is reliable and reproducible due to its simplicity and objective nature of the application. The IOTN scores the need for orthodontic treatment according to the highest potential risk from the malocclusion. The IOTN index has been used as a useful method in allocating treatment services where resources are limited. The use of the IOTN index can hence be used to prioritise the treatment need of malocclusion. Ever since the IOTN was developed by Brook and Shaw, it has become one of the most commonly used diagnostic tools in orthodontics; it rates malocclusion on the basis of both normative and subjective treatment needs.
The IOTN fulfils all the recommendations of the World Health Organisation's guidelines for an ideal index. Quite a few studies have shown the validity of the IOTN. It has a high ranking in terms of accuracy, reproducibility and validity. It is easy to use and can be performed quickly. Many researchers have considered it a powerful tool to assess treatment needs. The findings of the present study are also in the results of these studies.,,,,,,
We evaluated inter-rater and test-retest reliability for student raters with minimal experience in scoring the malocclusion on the basis of the IOTN index. Readers should exercise caution when generalising the results of our study to other types of raters. Reliability could differ according to raters' disciplines and levels of training. Reliability in our study also could have been affected by the specific training program we gave to the students.
| Conclusions|| |
Although being used for a long in orthodontics, the application of IOTN in practice is being revisited, but the ease of applying the index still makes it one of the best choices available. In light of the findings of the present study, it has been shown that IOTN is easy to learn and apply and has got good to moderate inter-rater and intra-rater reliability.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Sharabiani MT, Aylin P, Bottle A. Systematic review of comorbidity indices for administrative data. Med Care 2012;50:1109-18.
Brook PH, Shaw WC. The development of an index of orthodontic treatment priority. Eur J Orthod 1989;11:309-20.
Holmes A, Willmot DR. The Consultant Orthodontists Group 1994 survey of the use of the Index of Orthodontic Treatment Need (IOTN). Br J Orthod 1996;23:57-9.
Draker HL. Handicapping labio-lingual deviations: A proposed index for public health purposes. Am J Orthod 1960;46:295-315.
Grainger RM. Orthodontic Treatment Priority Index. Public Health Service Publication No. 1000, Series 2, No. 25. Washington, DC: US Government Printing Office; 1967.
Linder-Aronson S. Orthodontics in the swedish public dental health system. Trans EurOrthodSoc 1974;50:233-40.
Cardoso CF, Drummond AF, Lages EM, Pretti H, Ferreira EF, Abreu MH. The Dental Aesthetic Index and dental health component of the Index of Orthodontic Treatment Need as tools in epidemiological studies. Int J Environ Res Public Health 2011;8:3277-86.
Cooper S, Mandall NA, DiBiase D, Shaw WC. The reliability of the Index of Orthodontic Treatment Need over time. J Orthod 2000;27:47-53.
de Oliveira CM. The planning, contracting and monitoring of orthodontic services, and the use of the IOTN index: A survey of consultants in dental public health in the United Kingdom. Br Dent J 2003;195:704-6.
Borzabadi-Farahani A, Borzabadi-Farahani A, Eslamipour F. Orthodontic treatment needs in an urban Iranian population, an epidemiological study of 11-14 year old children. Eur J Paediatr Dent 2009;10:69-74.
Borzabadi-Farahani A, Naretto S. An overview of selected orthodontic treatment need indices. In: Principles in Contemporary Orthodontics. London, SW7 2QJ, UNITED KINGDOM : InTech; 2011. p. 215-36.
Cardoso CF, Drummond AF, Lages EM, Pretti H, Efigênia F, Ferreira EF, et al
. The dental aesthetic index and dental health component of the Index of Orthodontic Treatment Need as tools in epidemiological studies. Int J Environ Res Public Health 2011;8:3277-86.
Profit WR, Fields HW, David MS. Contemporary Orthodontics.Elsevier; 2013.
Siddiqui TA, Shaikh A, Fida M. Agreement between orthodontist and patient perception using Index of Orthodontic Treatment Need. Saudi Dent J 2014;26:156-65.
Shivakumar K, Chandu G, Shafiulla M. Severity of malocclusion and orthodontic treatment needs among 12- to 15-year-old school children of Davangere district, Karnataka, India. Eur J Dent 2010;4:298-307.
Onyeaso CO, BeGole EA. Orthodontic treatment need in an accredited graduate orthodontic center in north america: A pilot study. J Contemp Dent Pract 2006;7:87-94.
Onyeaso CO, Sanu OO. Perception of personal dental appearance in Nigerian adolescents. Am J Orthod Dentofacial Orthop 2005;127:700-6.