• Users Online: 156
  • Home
  • Print this page
  • Email this page
Home About us Editorial board Ahead of print Current issue Search Archives Submit article Instructions Subscribe Contacts Login 


 
 Table of Contents  
ORIGINAL ARTICLE
Year : 2021  |  Volume : 11  |  Issue : 4  |  Page : 69-72

Assessment of inter-rater and intra-rater reliability of index of orthodontic treatment need index by newly trained orthodontic residents


Department of Orthodontics and Dentofacial Orthopaedics, Narsinhbhai Patel Dental College and Hospital, Visnagar, Gujarat, India

Date of Submission27-Feb-2021
Date of Decision28-Mar-2021
Date of Acceptance18-Apr-2021
Date of Web Publication16-Oct-2021

Correspondence Address:
Khyati Mahida
Department of Orthodontics and Dentofacial Orthopaedics, Narsinhbhai Patel Dental College and Hospital, Visnagar, Gujarat
India
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/aihb.aihb_27_21

Rights and Permissions
  Abstract 


Introduction: This study aimed to test the reliability of the index of treatment need by newly trained orthodontic residents. Materials and Methods: A total of 100 patients were assessed by the panel of four consultant orthodontists using the index of orthodontic treatment need (IOTN) index, and the agreed category was set as the 'gold standard'. Four residents of the department in the initial stages of formal orthodontic training and lacking the knowledge of using the IOTN index were trained to score the treatment need according to the IOTN index. All four residents were then asked to score the 100 sets of models using both the indexes of IOTN. Results: Kappa scores for inter-rater agreement with the expert panel for the major categories (1–4) demonstrated moderate agreement (kappa: 0.59–0.5623) for all raters. The percentage agreement ranged from 84.03% to 88.10% in all cases. Intra-rater agreement for the major categories was fair to moderate (kappa: 0.53–0.80).Conclusions: The IOTN index is a reliable index to be used for determining the priority of treatment needs in orthodontic treatment. The clarity and simplicity make the index easy to learn and apply with minimal training. The simplicity of the IOTN index makes it easy to learn and apply by novices.

Keywords: Aesthetic component, dental health component, index of orthodontic treatment need


How to cite this article:
Mahida K, Kubavat A, Srivastava SC, Desai M, Patel H, Modi HS. Assessment of inter-rater and intra-rater reliability of index of orthodontic treatment need index by newly trained orthodontic residents. Adv Hum Biol 2021;11:69-72

How to cite this URL:
Mahida K, Kubavat A, Srivastava SC, Desai M, Patel H, Modi HS. Assessment of inter-rater and intra-rater reliability of index of orthodontic treatment need index by newly trained orthodontic residents. Adv Hum Biol [serial online] 2021 [cited 2021 Dec 4];11:69-72. Available from: https://www.aihbonline.com/text.asp?2021/11/4/69/328393




  Introduction Top


All over the world, indices are being used in health sciences for a variety of purposes like classification of disease and evaluating the prognosis of treatment outcome.[1] Indices can be of use in planning treatment and health policy for a population by determining the prevalence of a disease or malocclusion. In recent times, the planning of services, especially by the publically funded health services (UK National Health Service), has shown great interest. The index of orthodontic treatment need (IOTN)[2] has been used in NHS primary care in England and Wales since 2006.[3] IOTN index was developed with the prime objective of evaluating the functional need for treatment through its dental health component (DHC). The psychosocial need, on the other hand, is prioritised by its aesthetic component (AC). The psychosocial need on the other hand is prioritised by it Aesthetic component (AC) and Indices such as the handicapping labiolingual deviation index (Draker, 1960)[4], the treatment priority index.[5]

The handicapping malocclusion assessment record and the occlusal index were developed earlier than the IOTN, but IOTN has become quite popular. The IOTN actually is a modification of an index previously developed by the Swedish Dental Health Board.[6] The aim of this article is to test the inter-rater and intra-rater reliability of the orthodontic residents in initial years of formal orthodontic training in scoring the malocclusion according to the treatment needs using the IOTN index.


  Materials and Methods Top


The panel of four expert orthodontists worked in pairs to score 100 sets of study models using the IOTN index. The scores were then compared, and experts came to a common understanding of the score by sorting out the differences by discussion. Four residents of the department in the initial stages of formal orthodontic training and lacking the knowledge of using the IOTN index were trained to score the treatment need according to the IOTN index. All four residents were then asked to score the 100 sets of models using both the components of the IOTNs to test agreement with the expert panel scores. The scores were then analysed using Cohen's kappa for inter-operator agreement with the expert panel scores. For the inter-rater agreement, the scores of two raters were compared, wherein the ratings were taken at two different time intervals 2 months apart.


  Results Top


The scores of the raters were analysed using statistical methods to determine the percentage of agreement between the raters, and the inter-rater and intra-rater reliability was determined by calculating the Cohen kappa scores. Inter-rater agreement of AC for rater 1 with rater 2 had 88.10572687224669% of agreement with kappa value k: 0.594401429422275 showing a moderate agreement between the two raters. Rater 1 with rater 3 had 85.1063829787234% of agreement, with kappa value of 0.5710560625814863 showing a moderate agreement between the two raters and rater 1 with rater 4 had 84.03361344537815% of agreement with kappa value of 0.5623729797735411 showing a moderate agreement. Inter-rater agreement for the DHC showed that rater 1 had a fair agreement with a percentage of agreement of 68.02721088435374% and Cohen's k : 0.359; rater 1 with rater 3 also had a fair agreement (67.11409395973155% Cohen's k: 0.34219299035949197), whereas rater 1 with rater 4 showed a moderate agreement (% of agreement: 70.6713780918728% Cohen's k: 0.406108568683472) [Graph 1] and [Graph 2].



Inter-rater reliability of AC of rater 1T1 (rater 1 at time T1) and rater 1T2 (rater 1 at time 2) had percentage of agreement 85.1063829787234% with Cohen's k: 0.5710560625814863 showing a moderate agreement, whereas rater 2T1 and rater 2T2 had a percentage of agreement of 80.97165991902834% with Cohen's k: 0.5335315626632378 showing a moderate agreement [Graph 3] and [Graph 4].




  Discussion Top


Numerous studies have been conducted to assess the validity and reliability of the IOTN. The strengths of the IOTN DHC component are its validity, reliability and its ease of use.[7]

The present study was undertaken to evaluate the inter-rater reliability and intra-rater reliability of the IOTN index. The inter-rater reliability measures the extent to which two or more raters or examiners agree. It can be evaluated by using percentage agreement and Cohen's kappa statistics. High inter-rater reliability values depict a high percentage of agreement, and lower values show a low degree of agreement between the raters.

In this study, moderate, inter-rater reliability was found with respect to the scores of the AC of the IOTN and fair-to-moderate agreement was found for DHC scores. This shows that the validity and reliability of the IOTN index are sufficiently good. The discrepancy in the finding of the resident and the expert panel can be attributed to the differences in clinical experience. One of the advantages of IOTN is that the grading is unaffected by age.[8] This is one of the cardinal reasons for its wide acceptance in orthodontic research.[9]

The IOTN DHC is a relatively straightforward 5-point scale, with the greatest need for treatment classified as being Group 5 and little or no need for treatment classified as Group 1. Each group has well-defined descriptors of the features of the malocclusion deemed as indicators of orthodontic need. The index is quick and easy to apply because malocclusion is scored simply on the worst feature. To identify this feature in a systematic manner, it is recommended that the assessor uses the acronym MOCDO (missing teeth, overjet, crossbites, displacement of contact points and overbite). This is why even the novice orthodontist with minimal orthodontic experience could score the malocclusion with sufficient accuracy comparable to that of the gold standard set by the panel of an experienced orthodontist. Apart from the many advantages, there are some limitations of the IOTN. For example, the AC of the index comprises only Class I and Class II division 1 incisor relationships, and there are no Class II division 2 or Class III incisor relationships. This probably could be the reason for the differences in the cores of the raters.

To evaluate the test–retest reliability, two raters re-assessed the study models, which they had assessed in the inter-rater reliability phase. The re-assessments took place 2 months after the inter-rater reliability phase to reduce the possibility of recalling the scores of the first phase.

IOTN is a reliable tool for planning the budget and improve the focus of services by ensuring greater uniformity in the assessment of orthodontic treatment needs. The index is reliable and reproducible due to its simplicity and objective nature of the application. The IOTN scores the need for orthodontic treatment according to the highest potential risk from the malocclusion. The IOTN index has been used as a useful method in allocating treatment services where resources are limited. The use of the IOTN index can hence be used to prioritise the treatment need of malocclusion. Ever since the IOTN was developed by Brook and Shaw, it has become one of the most commonly used diagnostic tools in orthodontics; it rates malocclusion on the basis of both normative and subjective treatment needs.[10]

The IOTN fulfils all the recommendations of the World Health Organisation's guidelines for an ideal index. Quite a few studies have shown the validity of the IOTN. It has a high ranking in terms of accuracy, reproducibility and validity. It is easy to use and can be performed quickly. Many researchers have considered it a powerful tool to assess treatment needs. The findings of the present study are also in the results of these studies.[11],[12],[13],[14],[15],[16],[17]

We evaluated inter-rater and test-retest reliability for student raters with minimal experience in scoring the malocclusion on the basis of the IOTN index. Readers should exercise caution when generalising the results of our study to other types of raters. Reliability could differ according to raters' disciplines and levels of training. Reliability in our study also could have been affected by the specific training program we gave to the students.


  Conclusions Top


Although being used for a long in orthodontics, the application of IOTN in practice is being revisited, but the ease of applying the index still makes it one of the best choices available. In light of the findings of the present study, it has been shown that IOTN is easy to learn and apply and has got good to moderate inter-rater and intra-rater reliability.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.



 
  References Top

1.
Sharabiani MT, Aylin P, Bottle A. Systematic review of comorbidity indices for administrative data. Med Care 2012;50:1109-18.  Back to cited text no. 1
    
2.
Brook PH, Shaw WC. The development of an index of orthodontic treatment priority. Eur J Orthod 1989;11:309-20.  Back to cited text no. 2
    
3.
Holmes A, Willmot DR. The Consultant Orthodontists Group 1994 survey of the use of the Index of Orthodontic Treatment Need (IOTN). Br J Orthod 1996;23:57-9.  Back to cited text no. 3
    
4.
Draker HL. Handicapping labio-lingual deviations: A proposed index for public health purposes. Am J Orthod 1960;46:295-315.  Back to cited text no. 4
    
5.
Grainger RM. Orthodontic Treatment Priority Index. Public Health Service Publication No. 1000, Series 2, No. 25. Washington, DC: US Government Printing Office; 1967.  Back to cited text no. 5
    
6.
Linder-Aronson S. Orthodontics in the swedish public dental health system. Trans EurOrthodSoc 1974;50:233-40.  Back to cited text no. 6
    
7.
Cardoso CF, Drummond AF, Lages EM, Pretti H, Ferreira EF, Abreu MH. The Dental Aesthetic Index and dental health component of the Index of Orthodontic Treatment Need as tools in epidemiological studies. Int J Environ Res Public Health 2011;8:3277-86.  Back to cited text no. 7
    
8.
Cooper S, Mandall NA, DiBiase D, Shaw WC. The reliability of the Index of Orthodontic Treatment Need over time. J Orthod 2000;27:47-53.  Back to cited text no. 8
    
9.
de Oliveira CM. The planning, contracting and monitoring of orthodontic services, and the use of the IOTN index: A survey of consultants in dental public health in the United Kingdom. Br Dent J 2003;195:704-6.  Back to cited text no. 9
    
10.
Borzabadi-Farahani A, Borzabadi-Farahani A, Eslamipour F. Orthodontic treatment needs in an urban Iranian population, an epidemiological study of 11-14 year old children. Eur J Paediatr Dent 2009;10:69-74.  Back to cited text no. 10
    
11.
Borzabadi-Farahani A, Naretto S. An overview of selected orthodontic treatment need indices. In: Principles in Contemporary Orthodontics. London, SW7 2QJ, UNITED KINGDOM : InTech; 2011. p. 215-36.  Back to cited text no. 11
    
12.
Cardoso CF, Drummond AF, Lages EM, Pretti H, Efigênia F, Ferreira EF, et al. The dental aesthetic index and dental health component of the Index of Orthodontic Treatment Need as tools in epidemiological studies. Int J Environ Res Public Health 2011;8:3277-86.  Back to cited text no. 12
    
13.
Profit WR, Fields HW, David MS. Contemporary Orthodontics.Elsevier; 2013.  Back to cited text no. 13
    
14.
Siddiqui TA, Shaikh A, Fida M. Agreement between orthodontist and patient perception using Index of Orthodontic Treatment Need. Saudi Dent J 2014;26:156-65.  Back to cited text no. 14
    
15.
Shivakumar K, Chandu G, Shafiulla M. Severity of malocclusion and orthodontic treatment needs among 12- to 15-year-old school children of Davangere district, Karnataka, India. Eur J Dent 2010;4:298-307.  Back to cited text no. 15
    
16.
Onyeaso CO, BeGole EA. Orthodontic treatment need in an accredited graduate orthodontic center in north america: A pilot study. J Contemp Dent Pract 2006;7:87-94.  Back to cited text no. 16
    
17.
Onyeaso CO, Sanu OO. Perception of personal dental appearance in Nigerian adolescents. Am J Orthod Dentofacial Orthop 2005;127:700-6.  Back to cited text no. 17
    




 

Top
 
 
  Search
 
Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
Access Statistics
Email Alert *
Add to My List *
* Registration required (free)

 
  In this article
Abstract
Introduction
Materials and Me...
Results
Discussion
Conclusions
References

 Article Access Statistics
    Viewed182    
    Printed6    
    Emailed0    
    PDF Downloaded15    
    Comments [Add]    

Recommend this journal


[TAG2]
[TAG3]
[TAG4]