Improved Learning Through Neural Component Search

Simple item page
dc.contributor.advisorHougen, Dean
dc.contributor.authorMorgan, Brandon
dc.contributor.committeeMemberDiochnos, Dimitrios
dc.contributor.committeeMemberPan, Chongle
dc.contributor.committeeMemberNicholson, Charles
dc.date.accessioned2024-07-29T16:11:57Z
dc.date.available2024-07-29T16:11:57Z
dc.date.issued2024-08-01
dc.date.manuscript2024-07
dc.description.abstractDeep learning models contain many different hyper-parameters that need to be tuned prior to training. These hyper-parameters greatly influence the quality of the final model. Historically, most attention and research has been performed on tuning the architecture. However, with the advent of automated machine learning and the success from neural architecture search, automated methods have been successfully applied to other components of neural networks, challenging the very inspiration of the classical methodologies. In this work, automated methods, through the use of evolutionary algorithms, were applied to the learning components of a neural network: the loss function, optimizer, learning rate schedule, and output activation function of computer vision models with the goal of finding drop-in replacements for standard components. I expand upon previous research in each of these respective domains through the proposal of new search spaces, surrogate functions, genetic algorithms, and better found components. In the end, multiple loss functions, optimizers, learning rate schedules, and output activation functions, all evolved from scratch, were found to be able to outperform cross-entropy, Adam, one cycle cosine decay, and softmax on the CIFAR datasets.en_US
dc.identifier.urihttps://hdl.handle.net/11244/340526
dc.languageen_USen_US
dc.rightsAttribution-NonCommercial 4.0 International*
dc.rights.urihttp://creativecommons.org/licenses/by-nc/4.0/*
dc.subjectDeep Learningen_US
dc.subjectGenetic Algorithmsen_US
dc.subjectComputer Visionen_US
dc.subjectNeural Architecture Searchen_US
dc.thesis.degreePh.D.en_US
dc.titleImproved Learning Through Neural Component Searchen_US
ou.groupGallogly College of Engineering::School of Computer Scienceen_US

Files

Original bundle
Now showing 1 - 2 of 2
Thumbnail Image
Name:
2024_Morgan_Brandon_Dissertation.pdf
Size:
26.81 MB
Format:
Adobe Portable Document Format
Description:
Download
No Thumbnail Available
Name:
2024_Morgan_Brandon_Dissertation.zip
Size:
26.38 MB
Format:
Unknown data format
Description:
Download
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.71 KB
Format:
Item-specific license agreed upon to submission
Description:
Download

Collections

OU - Dissertations