Show simple item record

dc.contributor.advisorGrant, Christan
dc.contributor.authorGraham, Austin
dc.date.accessioned2019-08-02T16:14:53Z
dc.date.available2019-08-02T16:14:53Z
dc.date.issued2019-08
dc.identifier.urihttps://hdl.handle.net/11244/321112
dc.description.abstractComplex deep learning objectives such as object detection and saliency, semantic segmentation, sequence-to-sequence translation, and others have given rise to training processes requiring increasing amounts of time and computational resources. Human-in-the-loop solutions have addressed this problem in several ways; one such pain point is model hyperparameter search. Common methods of parameter search have high time costs and require iterative training of several models. Several algorithms have been proposed to manipulate a neural network's architecture and alleviate this cost. However, these algorithms require tuning of parameters to achieve desired performance and provide little to no intuition as to how such a change may affect overall performance. In this thesis, I present EigenRemove and WeakExpansion for removal and addition of weights providing a human-in-the-loop solution to the architecture search problem in both classical feedforward and convolutional neural network layers. EigenRemove yields results comparable to or better than the more popular Minimum Weight Selection pruning strategy, producing final test accuracies increased by 2-3% at larger compressions on the VGG16 object detection network. WeakExpand is compared with a trivial Zero Weight Expansion approach, where new connections are assigned no weight. WeakExpand is shown to produce final test accuracies in VGG16 comparable to that of Zero Weight Expansion, while providing new trainable weights rather than the dead weights produced by Zero Weight Expansion. Finally, I propose heuristics outlining how a user may use WeakExpand and EigenRemove to have a desired effect based on the current state of their network's training.en_US
dc.languageenen_US
dc.rightsAttribution 4.0 International*
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/*
dc.subjectmachine learningen_US
dc.subjectneural networksen_US
dc.subjecthuman in the loopen_US
dc.subjectsingular value decompositionen_US
dc.titleApplied High-Order Singular Value Decomposition for Weight Compression and Expansion in Deep Neural Networksen_US
dc.contributor.committeeMemberMcGovern, Amy
dc.contributor.committeeMemberHougen, Dean
dc.date.manuscript2019-07
dc.thesis.degreeMaster of Scienceen_US
ou.groupGallogly College of Engineering::School of Computer Scienceen_US


Files in this item

Thumbnail
Thumbnail
Thumbnail

This item appears in the following Collection(s)

Show simple item record


Attribution 4.0 International
Except where otherwise noted, this item's license is described as Attribution 4.0 International