How much can pruning algorithms teach us about the fundamentals of learning representations in ne... more How much can pruning algorithms teach us about the fundamentals of learning representations in neural networks? A lot, it turns out. Neural network model compression has become a topic of great interest in recent years, and many different techniques have been proposed to address this problem. In general, this is motivated by the idea that smaller models typically lead to better generalization. At the same time, the decision of what to prune and when to prune necessarily forces us to confront our assumptions about how neural networks actually learn to represent patterns in data. In this work we set out to test several long-held hypotheses about neural network learning representations and numerical approaches to pruning. To accomplish this we first reviewed the historical literature and derived a novel algorithm to prune whole neurons (as opposed to the traditional method of pruning weights) from optimally trained networks using a second-order Taylor method. We then set about testing ...
We report on our system used in the TRECVID 2015 Multimedia Event Detection (MED) task. On the ME... more We report on our system used in the TRECVID 2015 Multimedia Event Detection (MED) task. On the MED task, the CMU team submitted runs in the Semantic Query (SQ) and 10Ex settings. The proposed system is essentially the same as our MED 2014 system. 1.1 MED System Description On the MED task, the CMU team uses the MED 2014 [1] system which has enabled the system to achieve good performance in the 000Ex and 010Ex settings. Furthermore, our system is very efficient in that it can complete Event Query Generation (EQG) in 16 minutes and Event Search (ES) over 200,000 videos in less than 5 minutes on a single workstation. Please see [1] for the detailed system for the details about our 000Ex and 010Ex runs. The CMU system utilizes multiple modalities, classifiers and fusion methods to perform Multimedia Event Detection. The multiple modalities include visual, audio and text modalities. For 10Ex, two classifiers were used: linear SVM and linear regression. The fusion method used for 010Ex is...
In 2002, the African Energy Research Policy Network (AFREPREN) published an article in Energy Pol... more In 2002, the African Energy Research Policy Network (AFREPREN) published an article in Energy Policy magazine which estimated that roughly 68% of the inhabitants of Sub-Saharan Africa live in rural areas without access to grid-powered electricity. In the conventional interest of development and poverty reduction, the question of how to provide modern energy services to this enormous proportion of the population is of paramount importance. As many governments in SubSaharan Africa have proven incapable or unwilling to tackle this issue given the immense infrastructural investments required to expand electrical grid capacity, the general consensus of the development community at large has been to emphasize the dissemination of renewable energy technologies to rural areas. This focus on localized power generation has led to the implementation of a variety of developmental strategies and programs, not the least of which is the use of photovoltaic, or solar, technology to provide generall...
... initiated and data is being transferred, the update function cannot interrupt the control flo... more ... initiated and data is being transferred, the update function cannot interrupt the control flow, and ... specification separates fields with the colon character and the RTU specification is based on timing. ... any software application that uses this system is the ability to decode DTMF tones ...
How much can pruning algorithms teach us about the fundamentals of learning representations in ne... more How much can pruning algorithms teach us about the fundamentals of learning representations in neural networks? A lot, it turns out. Neural network model compression has become a topic of great interest in recent years, and many different techniques have been proposed to address this problem. In general, this is motivated by the idea that smaller models typically lead to better generalization. At the same time, the decision of what to prune and when to prune necessarily forces us to confront our assumptions about how neural networks actually learn to represent patterns in data. In this work we set out to test several long-held hypotheses about neural network learning representations and numerical approaches to pruning. To accomplish this we first reviewed the historical literature and derived a novel algorithm to prune whole neurons (as opposed to the traditional method of pruning weights) from optimally trained networks using a second-order Taylor method. We then set about testing ...
We report on our system used in the TRECVID 2015 Multimedia Event Detection (MED) task. On the ME... more We report on our system used in the TRECVID 2015 Multimedia Event Detection (MED) task. On the MED task, the CMU team submitted runs in the Semantic Query (SQ) and 10Ex settings. The proposed system is essentially the same as our MED 2014 system. 1.1 MED System Description On the MED task, the CMU team uses the MED 2014 [1] system which has enabled the system to achieve good performance in the 000Ex and 010Ex settings. Furthermore, our system is very efficient in that it can complete Event Query Generation (EQG) in 16 minutes and Event Search (ES) over 200,000 videos in less than 5 minutes on a single workstation. Please see [1] for the detailed system for the details about our 000Ex and 010Ex runs. The CMU system utilizes multiple modalities, classifiers and fusion methods to perform Multimedia Event Detection. The multiple modalities include visual, audio and text modalities. For 10Ex, two classifiers were used: linear SVM and linear regression. The fusion method used for 010Ex is...
In 2002, the African Energy Research Policy Network (AFREPREN) published an article in Energy Pol... more In 2002, the African Energy Research Policy Network (AFREPREN) published an article in Energy Policy magazine which estimated that roughly 68% of the inhabitants of Sub-Saharan Africa live in rural areas without access to grid-powered electricity. In the conventional interest of development and poverty reduction, the question of how to provide modern energy services to this enormous proportion of the population is of paramount importance. As many governments in SubSaharan Africa have proven incapable or unwilling to tackle this issue given the immense infrastructural investments required to expand electrical grid capacity, the general consensus of the development community at large has been to emphasize the dissemination of renewable energy technologies to rural areas. This focus on localized power generation has led to the implementation of a variety of developmental strategies and programs, not the least of which is the use of photovoltaic, or solar, technology to provide generall...
... initiated and data is being transferred, the update function cannot interrupt the control flo... more ... initiated and data is being transferred, the update function cannot interrupt the control flow, and ... specification separates fields with the colon character and the RTU specification is based on timing. ... any software application that uses this system is the ability to decode DTMF tones ...
Uploads
Papers by Nikolas Wolfe