short-paper

BERT4NILM: A Bidirectional Transformer Model for Non-Intrusive Load Monitoring

Authors:

Camilo Requena Witzig,

Hans-Arno JacobsenAuthors Info & Claims

NILM'20: Proceedings of the 5th International Workshop on Non-Intrusive Load Monitoring

Pages 89 - 93

https://doi.org/10.1145/3427771.3429390

Published: 18 November 2020 Publication History

Abstract

Non-intrusive load monitoring (NILM) based energy disaggregation is the decomposition of a system's energy into the consumption of its individual appliances. Previous work on deep learning NILM algorithms has shown great potential in the field of energy management and smart grids. In this paper, we propose BERT4NILM, an architecture based on bidirectional encoder representations from transformers (BERT) and an improved objective function designed specifically for NILM learning. We adapt the bidirectional transformer architecture to the field of energy disaggregation and follow the pattern of sequence-to-sequence learning. With the improved loss function and masked training, BERT4NILM outperforms state-of-the-art models across various metrics on the two publicly available datasets UK-DALE and REDD.

References

[1]

Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. Layer normalization. arXiv preprint arXiv:1607.06450 (2016).

[2]

Kunjin Chen, Yu Zhang, Qin Wang, Jun Hu, Hang Fan, and Jinliang He. 2019. Scale-and Context-Aware Convolutional Non-Intrusive Load Monitoring. IEEE Transactions on Power Systems 35, 3 (2019), 2362--2373.

[3]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. CoRR abs/1810.04805 (2018). arXiv:1810.04805

[4]

K. Ehrhardt-Martinez, K. Donnelly, and J. Laitner. 2010. Advanced metering initiatives and residential feedback programs: a meta-review for household electricity-saving opportunities. American Council for an Energy-Efficient Economy (2010).

[5]

Corinna Fischer. 2008. Feedback on household electricity consumption: A tool for saving energy. Energy Efficiency 1 (2008). https://doi.org/10.1007/s12053-008-9009-7

[6]

F. Gong, N. Han, Y. Zhou, S. Chen, D. Li, and S. Tian. 2019. A SVM Optimized by Particle Swarm Optimization Approach to Load Disaggregation in Non-Intrusive Load Monitoring in Smart Homes. In 2019 IEEE 3rd Conference on Energy Internet and Energy System Integration (EI2).

[7]

Caglar Gulcehre, Kyunghyun Cho, Razvan Pascanu, and Yoshua Bengio. 2014. Learned-norm pooling for deep feedforward and recurrent neural networks. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 530--546.

Digital Library

[8]

George William Hart. 1992. Nonintrusive appliance load monitoring. Proc. IEEE 80, 12 (1992).

[9]

Dan Hendrycks and Kevin Gimpel. 2016. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415 (2016).

[10]

Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In 2018 IEEE International Conference on Data Mining (ICDM). IEEE, 197--206.

[11]

Jack Kelly and William Knottenbelt. 2015. The UK-DALE dataset, domestic appliance-level electricity demand and whole-house demand from five UK homes. Scientific Data 2, 150007 (2015). https://doi.org/10.1038/sdata.2015.7

[12]

Jack Kelly and William J. Knottenbelt. 2015. Neural NILM: Deep Neural Networks Applied to Energy Disaggregation. CoRR abs/1507.06594 (2015). arXiv:1507.06594

[13]

Junseong Kim. 2018. BERT. https://github.com/codertimo/BERT-pytorch.

[14]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[15]

J. Z. Kolter and T. Jaakkola. 2012. Approximate inference in additive factorial HMMs with application to energy disaggregation. In AISTATS (2012).

[16]

J. Z. Kolter and Matthew J. Johnson. 2011. REDD: A Public Data Set for Energy Disaggregation Research. In In SUSTKDD.

[17]

L. Mauch and B. Yang. 2016. A novel DNN-HMM-based approach for extracting single loads from aggregate power signals. In 2016 ICASSP.

[18]

Y. Pan, K. Liu, Z. Shen, X. Cai, and Z. Jia. 2020. Sequence-To-Subsequence Learning With Conditional Gan For Power Disaggregation. In 2020 ICASSP.

[19]

H. Rafiq, H. Zhang, H. Li, and M. K. Ochani. 2018. Regularized LSTM Based Deep Learning Model: First Step towards Real-Time Non-Intrusive Load Monitoring. In 2018 IEEE International Conference on Smart Energy Grid Engineering (SEGE).

[20]

Antonio Maria Sudoso and Veronica Piccialli. 2019. Non-Intrusive Load Monitoring with an Attention-based Deep Neural Network. arXiv preprint arXiv:1912.00759 ( 2019).

[21]

Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In CIKM '19.

Digital Library

[22]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems.

[23]

Yandong Yang, Jing Zhong, Wei Li, T Aaron Gulliver, and Shufang Li. 2019. Semi-Supervised Multi-Label Deep Learning based Non-intrusive Load Monitoring in Smart Grids. IEEE Transactions on Industrial Informatics (2019).

[24]

B. Zhang, S. Zhao, Q. Shi, and R. Zhang. 2019. Low-Rate Non-Intrusive Appliance Load Monitoring Based on Graph Signal Processing. In 2019 IEEE ICSPAC.

[25]

Chaoyun Zhang, Mingjun Zhong, Zongzuo Wang, Nigel H. Goddard, and Charles A. Sutton. 2018. Sequence-to-point learning with neural networks for nonintrusive load monitoring. In AAAI.

[26]

B. Zhao, K. He, L. Stankovic, and V. Stankovic. 2018. Improving Event-Based Non-Intrusive Load Monitoring Using Graph Signal Processing. IEEE Access (2018).

Cited By

Okazawa KKaneko NZhao DNishikawa HTaniguchi ICatthoor FOnoye T(2024)Evaluation of Deep Learning-Based Non-Intrusive Thermal Load MonitoringEnergies10.3390/en1709201217:9(2012)Online publication date: 24-Apr-2024
https://doi.org/10.3390/en17092012
Laceda Taloma RComminiello DPisani PCuomo F(2024)UNet-WD: Deep Learning for Multi-Appliance Water Disaggregation2024 IFIP Networking Conference (IFIP Networking)10.23919/IFIPNetworking62109.2024.10619811(702-707)Online publication date: 3-Jun-2024
https://doi.org/10.23919/IFIPNetworking62109.2024.10619811
Wang ZZhang XWang YPeng XXu ZShu YLiu JTan RHe YChen J(2024)Hawk: An Efficient NALM System for Accurate Low-Power Appliance RecognitionProceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems10.1145/3666025.3699359(578-591)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3666025.3699359
Show More Cited By

Index Terms

BERT4NILM: A Bidirectional Transformer Model for Non-Intrusive Load Monitoring
1. Computing methodologies
  1. Artificial intelligence
2. Information systems
  1. Information systems applications

Recommendations

Assessing the Effectiveness of Supervised and Semi-supervised NILM Approaches in an Industrial Context
CIIS '23: Proceedings of the 2023 6th International Conference on Computational Intelligence and Intelligent Systems

Non-Intrusive Load Monitoring (NILM) is a technique that aims to estimate the energy consumption and operational status of individual appliances in a building by analyzing only the aggregate power usage data. This technique plays a crucial role in demand-...
Multi-scale residual network for energy disaggregation

Energy disaggregation technology is a key technology to realise real-time non-intrusive load monitoring (NILM). Current energy disaggregation methods use the same scale to extract features from the sequence, which makes part of the local features lost, ...
IMG-NILM: A Deep learning NILM approach using energy heatmaps
SAC '23: Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing

Energy disaggregation estimates appliance-by-appliance electricity consumption from a single meter that measures the whole home's electricity demand. Compared with intrusive load monitoring, NILM (Non-intrusive load monitoring) is low-cost, easy to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

NILM'20: Proceedings of the 5th International Workshop on Non-Intrusive Load Monitoring

November 2020

109 pages

ISBN:9781450381918

DOI:10.1145/3427771

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGEnergy: ACM Special Interest Group on Energy Systems and Informatics

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 November 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Conference

BuildSys '20

Sponsor:

SIGEnergy

BuildSys '20: The 7th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation

November 18, 2020

Virtual Event, Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

62
Total Citations
View Citations
622
Total Downloads

Downloads (Last 12 months)168
Downloads (Last 6 weeks)22

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Okazawa KKaneko NZhao DNishikawa HTaniguchi ICatthoor FOnoye T(2024)Evaluation of Deep Learning-Based Non-Intrusive Thermal Load MonitoringEnergies10.3390/en1709201217:9(2012)Online publication date: 24-Apr-2024
https://doi.org/10.3390/en17092012
Laceda Taloma RComminiello DPisani PCuomo F(2024)UNet-WD: Deep Learning for Multi-Appliance Water Disaggregation2024 IFIP Networking Conference (IFIP Networking)10.23919/IFIPNetworking62109.2024.10619811(702-707)Online publication date: 3-Jun-2024
https://doi.org/10.23919/IFIPNetworking62109.2024.10619811
Wang ZZhang XWang YPeng XXu ZShu YLiu JTan RHe YChen J(2024)Hawk: An Efficient NALM System for Accurate Low-Power Appliance RecognitionProceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems10.1145/3666025.3699359(578-591)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3666025.3699359
Zhu LJiang LHe XJin JZheng Y(2024)Transformative enhancement of predictive models via fourier transformer-based denoising for non-intrusive load monitoringFourth International Conference on Applied Mathematics, Modelling, and Intelligent Computing (CAMMIC 2024)10.1117/12.3035106(9)Online publication date: 21-Jul-2024
https://doi.org/10.1117/12.3035106
Wang YCheng SZhang XLi XWang SGu Y(2024)A sequence labeling solution for identifying multiple variable-length events in non-intrusive load monitoringThird International Conference on Electronic Information Engineering, Big Data, and Computer Technology (EIBDCT 2024)10.1117/12.3031456(370)Online publication date: 19-Jul-2024
https://doi.org/10.1117/12.3031456
Agarwal VArdakanian OPal S(2024)A Robust and Privacy-Aware Federated Learning Framework for Non-Intrusive Load MonitoringIEEE Transactions on Sustainable Computing10.1109/TSUSC.2024.33708379:5(766-777)Online publication date: Sep-2024
https://doi.org/10.1109/TSUSC.2024.3370837
Zhou KZhang ZLu X(2024)Non-Intrusive Load Monitoring Based on an Efficient Deep Learning Model With Local Feature ExtractionIEEE Transactions on Industrial Informatics10.1109/TII.2024.338352120:7(9497-9507)Online publication date: Jul-2024
https://doi.org/10.1109/TII.2024.3383521
Athanasoulias SSykiotis SKaselimi MDoulamis ADoulamis NIpiotis N(2024)OPT-NILM: An Iterative Prior-to-Full-Training Pruning Approach for Cost-Effective User Side Energy DisaggregationIEEE Transactions on Consumer Electronics10.1109/TCE.2023.332449370:1(4435-4446)Online publication date: Feb-2024
https://doi.org/10.1109/TCE.2023.3324493
Azzam ASanami SAghdam A(2024)Low-Frequency Load Identification Using CNN-BiLSTM Attention Mechanism2024 32nd Mediterranean Conference on Control and Automation (MED)10.1109/MED61351.2024.10566167(712-717)Online publication date: 11-Jun-2024
https://doi.org/10.1109/MED61351.2024.10566167
Xu YXu QYang Y(2024)MCIR-CNN: Multichannel Imaging-Based Nonintrusive Load Identification With Rényi Entropy Window for Improved Sensor Data ProcessingIEEE Sensors Journal10.1109/JSEN.2023.332509824:1(377-389)Online publication date: 1-Jan-2024
https://doi.org/10.1109/JSEN.2023.3325098
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents