File(s) under permanent embargo
Proper comparison among methods using a confusion matrix
conference contribution
posted on 2023-05-23, 10:22 authored by Brian SalmonBrian Salmon, Kleynhans, W, Schwegmann, CP, Jan OlivierJan OlivierAn important aspect of research in the remote sensing field is to objectively compare different classifiers. This is the foundation of hundreds of research projects and in this paper we will address some raising concerns when evaluating solutions for classification of data sets with skewed class distributions. The quality of assessment is based on the problem specified by the user and the corresponding hypothesis defined. This hypothesis will determine how two or more classifiers are scored to determine which one is better for a particular application. In this paper we present two experiments that illustrate how, if unaware and misunderstood, statistical measurements can be misleading. One experiment is based on a Synthetic Aperture Radar image with a highly skewed class distribution and the second experiment is based on a Landsat image with a minor skewed distribution. From both experiments it can be seen that ill-defining the problem, can lead to false statements and the reporting of statistically invalid conclusions.
History
Publication title
2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS)Editors
IEEEPagination
3057-3060ISBN
978-1-4799-7929-5Department/School
School of EngineeringPublisher
Institute of Electrical and Electronics EngineersPlace of publication
United States of AmericaEvent title
International Geoscience and Remote Sensing Symposium 2015Event Venue
Milan, ItalyDate of Event (Start Date)
2015-07-26Date of Event (End Date)
2015-07-31Rights statement
Copyright unknownRepository Status
- Restricted