LUNG CANCER CLASSIFICATION USING PYTHON AND ML

Abstract

Lung cancer is one of the most common cancers and the leading cause of cancerrelated deaths worldwide. Survival depends strongly on the stage at which the disease is detected, yet most patients are diagnosed only after the cancer has spread, because early symptoms are mild and easily confused with ordinary respiratory problems. This paper presents a lung cancer classification system implemented in Python using machine learning, which aims to support early identification of high-risk individuals and to assist in distinguishing malignant from benign findings in chest CT images. The system works with two complementary sources of data. The first is a surveybased clinical dataset recording age, gender, smoking habit, yellow fingers, anxiety, peer pressure, chronic disease, fatigue, allergy, wheezing, alcohol consumption, coughing, shortness of breath, swallowing difficulty, and chest pain. The second is a set of chest CT slices from a public imaging collection, labelled as normal, benign, or malignant. Treating the CT image as a two-dimensional signal, the system applies image processing steps familiar to electronics engineers, including noise filtering, contrast enhancement, thresholding, and morphological operations to segment the lung region and candidate nodules. From the segmented images, handcrafted features are extracted, including grey-level co-occurrence texture measures such as contrast, correlation, energy, and homogeneity, intensity statistics, and shape descriptors such as area, perimeter, and circularity. These features, together with the survey attributes in the clinical track, are used to train and compare several classifiers, namely logistic regression, k-nearest neighbours, a support vector machine, naive Bayes, a decision tree, random forest, and gradient boosting. Class imbalance is handled with synthetic oversampling applied within the training folds. The models are evaluated using accuracy, precision, recall, F1 score, specificity, and the area under the ROC curve, with particular emphasis on recall because a missed malignant case has serious consequences. Ensemble models achieved the best results in both tracks. A Streamlit-based web application allows a user to enter survey responses for a risk estimate, or to upload a CT slice and view the segmented region, the extracted features, and the predicted class with its probability. The system is intended as a decision-support and screening aid, not a replacement for radiologists or oncologists, and every positive result is accompanied by advice to consult a specialist. It demonstrates how classical image processing and machine learning in Python can be combined into a low-cost tool that could be used in teaching hospitals and screening programmes. Future work includes three-dimensional nodule analysis, deep convolutional networks, and validation on larger multi-centre datasets.