Automated reading level classification model based on improved orbital pattern

Abstract

Automatic reading level for detection and classification is a challenging problem in machine learning. A multilevel feature extraction-based self-organized model may be useful to overcome this hurdle without using deep learning, which requires an ultra-large sample size. In this work, a novel speech dataset was collected from 57 primary school students by reading a fixed paragraph, and experts labeled these speeches as good, moderate, or bad. We then developed a handcrafted, self-organized learning model. We constructed a novel method using a multilevel feature extraction method, termed improved orbital pattern (IOP) and wavelet packet decomposition (WPD). The proposed IOP generates textural features from the speeches and the used wavelet bands. These extracted features are input to neighborhood components analysis (NCA) to reduce feature dimension. Then the feature set is input to the support vector machine (SVM) classifier to obtain loss values. The output of ten feature vectors of the NCA and SVM classifiers are merged to provide the final feature vector. The most significant 512 features were selected using the NCA feature selection function. These 512 features are classified via the SVM classifier with tenfold cross-validation (CV) and leave-one-subject-out (LOSO) validation strategies. The proposed IOP and WPD-based model yielded an accuracy of 92.75% with a tenfold CV and a 76.18% accuracy using LOSO validation strategies in classifying bad, intermediate, and good reading levels. Our developed model is ready to be validated with more data before its actual usage in schools to aid the teachers.

Appendix 1

The used stable reading text and translation of this text are given below.

Reading text in Turkish.

“Bir kartal denizden uzak bir dağ yolunun kenarında yuva kurdu. Kartalın orada yavruları oldu. Bir gün pençesinde kocaman bir balıkla yuvasına geldi kartal. Yuva yaptığı ağacın çevresinde çalışan insanlar vardı. Balığı gördüklerinde ağacın etrafında toplanıp bağırmaya, kartala taş atmaya başladılar. Balık sonunda kartalın pençesinden kayıp yere düştü. Adamlar balığı alıp gittiler. Kartal yuvasının bir köşesine çekilip tünedi. Yavruları ise havaya başlarını dikip yiyecek, yiyecek diye bağrışmaya başladılar. Oysa kartal çok yorulmuş denize kadar uçacak gücü kalmamıştı. Yuvasına iyice yerleşip yavrularını kanatlarının altına aldı. Onları sevdi, okşadı ve küçücük tüylerini düzeltti. Sanki “Ne olur birazcık sabredin!” diye yalvarıyordu onlara. Fakat yavrular okşandıkça seslerini daha da yükseltip bağrışmaya devam ettiler. Kartal uçtu ve daha yüksek bir dala kondu. Yavrular, anneleri uçup gidince daha da acıklı bir sesle bağrıştılar. Sonunda kartal çaresizlik içinde acı bir çığlık attı ve kanatlarını açıp ağır ağır denize doğru uçtu. Anne kartal akşam olup geç vakit yuvaya dönerken ağır ağır ve alçaktan uçmaktaydı. Yine pençelerinde kocaman bir balık vardı. Ağaca yaklaşırken çevrede başkaları var mı diye etrafı kolaçan etti bu kez. Güven içinde olduğunu hissettikten sonra kanatlarını kısıp yuvasının bir ucuna kondu. Yavru kartallar gagalarını açıp boyunlarını uzattılar. Anne kartal ise balığı parçaladı ve başladı yavrularını doyurmaya.”

The English translation of the above paragraph is given below.

"An eagle nested by the side of a mountain road far from the sea. The eagle had hatchlings there. One day, an eagle came to its nest with a huge fish in its talons. There were people working around the tree where he made his nest. When they saw the fish, they gathered around the tree and started shouting at the eagle and throwing stones. The fish eventually slipped from the eagle's talons and fell to the ground. The men took the fish and left. The eagle stepped back and perched on a corner of its nest. The hatchlings, on the other hand, raised their heads and started shouting for food, food. However, the eagle was so tired that it did not have the strength to fly into the sea. She settled in her nest and took her hatchlings under her wing. She loved them, touched them, and straightened their tiny feathers. "Please be patient!" she begged them. But as the hatchlings were attended, they kept raising their voices and shouting. The eagle flew and landed on a higher branch. The hatchlings cried out even more pathetically as their mother flew away. Finally, the eagle let out a cry of despair and spread its wings and flew slowly towards the sea. The mother eagle was flying slowly and low, as it was late in the evening, and was returning to her nest. It also had a huge fish in its paws. As she approached the tree, this time she looked around to see if anyone else was around. Feeling secure, she folded her wings and perched on one end of her nest. The baby eagles opened their beaks and stretched their necks. The mother eagle crushed the fish and began to feed her young."