Generalizable Skin-Lesion Classification with Deep Learning

A reproducible study of seven deep learning architectures and a probability-level ensemble for dermoscopic skin-lesion classification, with internal evaluation on HAM10000 and external evaluation on ISIC 2019.

This research investigates the generalizability of deep learning systems for multiclass dermoscopic skin-lesion classification through harmonized internal evaluation on HAM10000 and external evaluation on ISIC 2019.

Seven architectures are evaluated: a conventional CNN, ResNet-50, DenseNet-121, EfficientNet-B3, ConvNeXt-Tiny, MobileNetV3-Large, and ViT-B/16. Their class-probability outputs are additionally combined using an equal-weight probability-level ensemble. The study examines classification performance, discrimination, calibration, statistical uncertainty, external generalization, and model interpretability.

Study Design

Model Architectures
  • Conventional CNN
  • ResNet-50
  • DenseNet-121
  • EfficientNet-B3
  • ConvNeXt-Tiny
  • MobileNetV3-Large
  • Vision Transformer (ViT-B/16)
Ensemble Strategy

The ensemble uses equal-weight arithmetic averaging of the harmonized class-probability vectors produced by the seven component models. Predictions are mapped to a common seven-class output order before aggregation.

Datasets & Evaluation

HAM10000

HAM10000 is used for the standardized retrospective internal evaluation. The harmonized evaluation cohort contains 2,003 dermoscopic images spanning seven diagnostic classes: AKIEC, BCC, BKL, DF, MEL, NV, and VASC.

ISIC 2019

ISIC 2019 is used for external evaluation after harmonizing its labels with the seven-class prediction space. This analysis assesses how performance changes when the models are evaluated beyond the internal HAM10000 setting.

Evaluation Framework
  • Accuracy and weighted F1 score
  • Macro and micro ROC-AUC for HAM10000
  • Macro and weighted ROC-AUC for ISIC 2019
  • Expected Calibration Error (ECE)
  • Bootstrap 95% confidence intervals
  • Paired McNemar comparisons
  • Paired bootstrap model-difference analysis
  • Per-class external performance analysis

Key Results

The results demonstrate that strong internal performance does not necessarily translate into equivalent external performance. The study therefore emphasizes comparative evaluation and generalization rather than assuming that an ensemble will uniformly outperform every individual architecture.

HAM10000

ViT-B/16 achieved the strongest individual HAM10000 point estimates:

  • Accuracy: 0.9591
  • Weighted F1: 0.9585
  • Macro ROC-AUC: 0.9959
  • Micro ROC-AUC: 0.9978
Seven-Model Ensemble

The equal-weight ensemble remained highly competitive on the harmonized HAM10000 evaluation:

  • Accuracy: 0.9386
  • Weighted F1: 0.9374
  • Macro ROC-AUC: 0.9951
  • Micro ROC-AUC: 0.9971
External Evaluation

On ISIC 2019, the ensemble achieved:

  • Accuracy: 0.6830
  • Weighted F1: 0.6446
  • Macro ROC-AUC: 0.8981
  • Weighted ROC-AUC: 0.9011

The reduction relative to internal HAM10000 performance highlights the importance of external evaluation when assessing the generalizability of medical-image classification systems.

Comparative classification performance across the evaluated architectures and ensemble.

Explainable AI

Grad-CAM is used for qualitative attribution analysis to visualize image regions associated with model predictions. The repository contains curated class-specific ensemble visualizations for all seven lesion categories, together with selected model-comparison examples.

Example Grad-CAM attribution for the melanoma class. Attribution maps indicate regions associated with the model prediction and should not be interpreted as independent clinical evidence.

Reproducibility

The public repository provides the implementation and curated reproducibility materials for the study, including:

  • training and evaluation code;
  • seven-model ensemble evaluation workflow;
  • HAM10000 data-preparation utilities;
  • machine-readable result tables;
  • ROC curves and confusion matrices;
  • calibration and statistical-comparison outputs;
  • Grad-CAM attribution figures;
  • automated tests and continuous-integration checks; and
  • installation, training, evaluation, and ensemble documentation.
GitHub Repository Zenodo Archive

Publication Status

This project forms the reproducibility framework for the manuscript “Generalizable Ensemble Deep Learning for Dermoscopic Skin-Lesion Classification: Internal Evaluation on HAM10000 and External Evaluation on ISIC 2019” , which is currently under peer review.

Manuscript status and bibliographic information will be updated following peer-review decisions and publication.