Domestic Electrical Load Metering Data 1994-2014
South Africa, 1994 - 2014
Get Microdata
Reference ID
zaf-eskom-uct-us-delm-1994-2014-v1
Producer(s)
Eskom, Stellenbosch University, University of Cape Town
Collections
Created on
May 30, 2019
Last modified
Dec 14, 2022
Page views
6003
Downloads
1456
Identification
Survey ID Number
zaf-eskom-uct-us-delm-1994-2014-v1
Title
Domestic Electrical Load Metering Data 1994-2014
Country
| Name | Country code |
|---|---|
| South Africa | ZAF |
Study type
Sensor Monitoring Study
Abstract
This dataset contains the electricity metering data from the NRS Load Research Programme collected at 5 minute intervals. From 1994 to 2008 electricity meters were installed at households to measure the voltage and current. From 2009 to 2014 loggers were upgraded and the current, voltage, real and reactive power and power frequency of households were metered.
The NRS Load Research Programme was started in 1994 to provide inputs towards policy development and technical design guidelines for the domestic electricity distribution business in South Africa. The programme was overseen by the National Rationalised Specification (NRS) 034 Working Group at Eskom. Under this programme the Domestic Electrical Load (DEL) Study (also referred to as the Domestic Load Research Project) was designed and managed to collect electricity meter readings and conduct an annual socio-demographic survey of metered households. The resulting DEL data collection and research outputs present a collaborative, multi-party public-academic-private collaboration.
Initiated by Dr Ron Herman (Stellenbosch University) and Prof. Trevor Gaunt (University of Cape Town), the study was promoted by the NRS 034 Working Group established within Eskom for this purpose. Early funders and collaborators included the Department of Minerals and Energy Affairs (now Department of Energy), the Council for Scientific and Industrial Research, as well as Stellenbosch, eThekwini and Nelson Mandela Bay Municipalities. From 1994 to 2009 eight municipalities contributed to data collection. Eskom Research, Testing and Development became actively involved in the study in 1997. From 2001 onwards Eskom was the major data contributor and funder of the study. Prior to 1994, the National Energy Council and Development Bank of Southern Africa funded the development of the data loggers used in the study, as well as early research efforts by Dr Ron Herman and J.J. Kritzinger that influenced the study.
This study made a major contribution to the electrification of South African households and enabled the development of planning tools and applications that Eskom and municipalities to accurately forecast and right-size new power transmission and distribution infrastructure. The research outputs that emerged from the data collected in this study, such as the Hermann-Beta distribution and the Geo-based Load Forecasting Standard, informed the design of South Africa's power system and have been used in the design of power grids in other developing countries.
The NRS Load Research Programme was started in 1994 to provide inputs towards policy development and technical design guidelines for the domestic electricity distribution business in South Africa. The programme was overseen by the National Rationalised Specification (NRS) 034 Working Group at Eskom. Under this programme the Domestic Electrical Load (DEL) Study (also referred to as the Domestic Load Research Project) was designed and managed to collect electricity meter readings and conduct an annual socio-demographic survey of metered households. The resulting DEL data collection and research outputs present a collaborative, multi-party public-academic-private collaboration.
Initiated by Dr Ron Herman (Stellenbosch University) and Prof. Trevor Gaunt (University of Cape Town), the study was promoted by the NRS 034 Working Group established within Eskom for this purpose. Early funders and collaborators included the Department of Minerals and Energy Affairs (now Department of Energy), the Council for Scientific and Industrial Research, as well as Stellenbosch, eThekwini and Nelson Mandela Bay Municipalities. From 1994 to 2009 eight municipalities contributed to data collection. Eskom Research, Testing and Development became actively involved in the study in 1997. From 2001 onwards Eskom was the major data contributor and funder of the study. Prior to 1994, the National Energy Council and Development Bank of Southern Africa funded the development of the data loggers used in the study, as well as early research efforts by Dr Ron Herman and J.J. Kritzinger that influenced the study.
This study made a major contribution to the electrification of South African households and enabled the development of planning tools and applications that Eskom and municipalities to accurately forecast and right-size new power transmission and distribution infrastructure. The research outputs that emerged from the data collected in this study, such as the Hermann-Beta distribution and the Geo-based Load Forecasting Standard, informed the design of South Africa's power system and have been used in the design of power grids in other developing countries.
Kind of Data
Observation data
Unit of Analysis
Households
Version
Version Description
v1: edited data for use in DataFirst's secure research data centre https://www.datafirst.uct.ac.za/services/secure-data-services
Version Date
2019-05-15
Version Notes
Version 1
Scope
Notes
The scope of the secure South African Domestic Electrical Load Metering data includes:
CURRENT: Amperes (A) at 5 minute cadence
VOLTAGE: Volt (V) at 5 minute cadence
REAL POWER: kilo-Watt-hour (kWh) at 5 minute cadence
REACTIVE POWER: kilo-Volt-Ampere (kVA) at 5 minute cadence
FREQUENCY: Hertz (Hz) at 5 minute cadence
CURRENT: Amperes (A) at 5 minute cadence
VOLTAGE: Volt (V) at 5 minute cadence
REAL POWER: kilo-Watt-hour (kWh) at 5 minute cadence
REACTIVE POWER: kilo-Volt-Ampere (kVA) at 5 minute cadence
FREQUENCY: Hertz (Hz) at 5 minute cadence
Keywords
| Keyword |
|---|
| current |
| voltage |
| power |
| real |
| reactive |
| frequency |
| South Africa |
| rural |
| electrification |
| power quality |
| load data |
| time-of-use |
| electrical |
| household |
| consumption |
| energy |
| electricity |
| domestic |
| residential |
Coverage
Geographic Coverage
The study had national coverage.
Geographic Unit
The DELMS 1994-2014 dataset does not contain geographic information. When combined with the DELSS 1994-2014 dataset, the lowest unit of geographic aggregation is street address, with household GPS coordinates from 2000 onwards.
Universe
The metering study covers electrified households that received electricity either directly from Eskom or from their local municipality. Particular attention was devoted to rural and low income households, as well as surveying households electrified over a range of years, thus having had access to electricity from recent times to several decades.
Producers and sponsors
Primary investigators
| Name | Affiliation |
|---|---|
| Eskom | Government of South Africa |
| Stellenbosch University | |
| University of Cape Town |
Producers
| Name | Affiliation | Role |
|---|---|---|
| Council for Scientific and Industrial Research | University of Cape Town | Technical assistance |
| Marcus Dekenah Consulting | Data collection, Technical assistance | |
| Schalk Heunis | Enerweb | [Technical assistance in] sampling methodology/selection, data processing, data quality control, statistical analysis, data analysis (1995 - 2014) |
| Wiebke Toussaint | University of Cape Town | [Technical assistance in] data science, data stewardship, data archiving, data publishing (2017 - 2019) |
Funding Agency/Sponsor
| Name | Abbreviation | Role |
|---|---|---|
| Department of Minerals and Energy Affairs (now: Department of Energy) | DMEA | Funder from 1994 - 1997 |
| Eskom Research | Eskom | Funder from 1998 - 2014 |
| South African National Energy Development Initiative | SANEDI | Funder in 2019 for data archiving |
Other Identifications/Acknowledgments
| Name | Affiliation | Role |
|---|---|---|
| City of Cape Town | Government of South Africa | provided data loggers and human resources for data collection (1997 - 2006) |
| City of Joahnnesburg | Government of South Africa | provided data loggers and human resources for data collection (1996 - 2000) |
| City of Tshwane | Government of South Africa | provided data loggers and human resources for data collection (2001 - 2003, 2005 - 2009) |
| eThekwini Municipality | Government of South Africa | provided data loggers and human resources for data collection (1995, 1997 - 2002, 2005, 2006) |
| Msunduzi Municipaliy | Government of South Africa | provided data loggers and human resources for data collection (1996, 1997) |
| Mantsopa Municipality | Government of South Africa | provided data loggers and human resources for data collection (1996, 1997) |
| Nelson Mandela Bay Municipality | Government of South Africa | provided data loggers and human resources for data collection (1995, 1997 - 2006) |
| Stellenbosch Municipality | Government of South Africa | provided data loggers and human resources for data collection (1994) |
Sampling
Sampling Procedure
The sampling procedure and sample design are described in detail in the annual NRS Load Research Reports and in particular in the Load Data Collection Guides. The sample design was reviewed annually and updated from time to time as the need arose.
SAMPLE POPULATION CHARACTERISTICS
Sampling communities were selected based on the following requirements outlined in programme reports: The target community should have a high degree of electrification, should be stable and willing to co-operate with the project. There should not be many gapsi n connectivity. As first-time consumers require a period of adjustment to the use of electrical power, it was assumed that individual load patterns would be erratic for the first two years. Thus "newly electrified" communities should have had access to electricity for at least 24 months before being selected to participate in the study.
SAMPLE SIZE
70 - 100 consumers (households) were deemed a sufficient sample population for statistically significant load metering.
SAMPLE SELECTION
A random systematic method was suggested and where possible used to select households to be monitored. In general sample selection was optimised to fully utilise data loggers, meaning that loggers were installed on electrical poles that had the most connections so that all logger channels could be utilised. The approach taken at the beginning of the study was as follows:
1. List all the dwelling stand numbers from the township plans.
2. Divide the number of stands by the number of available loggers (call the resulting number sl)
3. Select a random starting point, say at stand sp.
4. Add multiples of sl to sp to give the stand numbers at which to site the loggers.
5. Check (4) to ensure that all or most of the data channels can be used at the point under consideration. If necessary move one pole forward or backward to optimise logger utilisation.
6. Repeat the process until all the loggers have been sited. Meticulous attention must now be given to identifying each monitored dwelling with its logger and channel.
SAMPLE POPULATION CHARACTERISTICS
Sampling communities were selected based on the following requirements outlined in programme reports: The target community should have a high degree of electrification, should be stable and willing to co-operate with the project. There should not be many gapsi n connectivity. As first-time consumers require a period of adjustment to the use of electrical power, it was assumed that individual load patterns would be erratic for the first two years. Thus "newly electrified" communities should have had access to electricity for at least 24 months before being selected to participate in the study.
SAMPLE SIZE
70 - 100 consumers (households) were deemed a sufficient sample population for statistically significant load metering.
SAMPLE SELECTION
A random systematic method was suggested and where possible used to select households to be monitored. In general sample selection was optimised to fully utilise data loggers, meaning that loggers were installed on electrical poles that had the most connections so that all logger channels could be utilised. The approach taken at the beginning of the study was as follows:
1. List all the dwelling stand numbers from the township plans.
2. Divide the number of stands by the number of available loggers (call the resulting number sl)
3. Select a random starting point, say at stand sp.
4. Add multiples of sl to sp to give the stand numbers at which to site the loggers.
5. Check (4) to ensure that all or most of the data channels can be used at the point under consideration. If necessary move one pole forward or backward to optimise logger utilisation.
6. Repeat the process until all the loggers have been sited. Meticulous attention must now be given to identifying each monitored dwelling with its logger and channel.
Data Collection
Dates of Data Collection
| Start | End |
|---|---|
| 1994 | 2014 |
Time periods
| Start date | End date | Cycle |
|---|---|---|
| 1994-06-17 | 1995-06-26 | G1994 |
| 1996-02-09 | 1996-09-10 | G1996 |
| 1996-09-19 | 1997-12-14 | G1996 |
| 1998-01-24 | 1998-10-28 | G1998 |
| 1999-05-17 | 1999-09-09 | G1999 |
| 2000-02-12 | 2000-10-20 | G2000 |
| 2001-03-14 | 2001-11-28 | G2001 |
| 2002-12-11 | 2002-12-23 | G2002 |
| 2003-12-01 | 2004-05-17 | G2004 |
| 2004-07-16 | 2005-02-08 | G2006 |
| 2006-02-23 | 2006-10-24 | G2005 |
| 2007-01-05 | 2007-04-23 | G2007 |
| 2008-01-31 | 2008-11-12 | G2008 |
| 2009-01-29 | 2009-11-12 | G2009 |
| 2010-01-19 | 2010-12-01 | G2010 |
| 2010-12-15 | 2011-11-29 | G2011 |
| 2012-01-01 | 2013-01-09 | G2012 |
| 2013-12-31 | 2014-01-01 | G2013 |
| 2014-08-31 | G2014 |
Data Collection Mode
Other [oth]
Data Collection Notes
SELECTON OF 5 MINUTE METERING CADENCE
Based on the evidence presented in early investigations predating the NRS Load Research Programme, the data for this study was collected at 5 minute interavls so that it would be useable for 'quality of supply' analysis (see CT Gaunt. Implications of Planning and Design Decisions in Electricity Distribution. AMEU 12th Technical Meeting. Potchefstroom (1988)).
FEEDBACK LOOPS ON LOGGERS
Data collections from every month were gathered together into a feedback report where any problems with data collection at a site were communicated to the site manager and resolved. Site referencing was done on an annual basis just prior to the winter survey collection process to capture the site 'as is' with minimal likelihood of alteration due to maintenance interventions. During site referencing process the galvanic connectivity between a household or energy customer and the corresponding data logger channels was documented and updated in the database to associate a customer load with the correct questionaire.
Based on the evidence presented in early investigations predating the NRS Load Research Programme, the data for this study was collected at 5 minute interavls so that it would be useable for 'quality of supply' analysis (see CT Gaunt. Implications of Planning and Design Decisions in Electricity Distribution. AMEU 12th Technical Meeting. Potchefstroom (1988)).
FEEDBACK LOOPS ON LOGGERS
Data collections from every month were gathered together into a feedback report where any problems with data collection at a site were communicated to the site manager and resolved. Site referencing was done on an annual basis just prior to the winter survey collection process to capture the site 'as is' with minimal likelihood of alteration due to maintenance interventions. During site referencing process the galvanic connectivity between a household or energy customer and the corresponding data logger channels was documented and updated in the database to associate a customer load with the correct questionaire.
Questionnaires
Questionnaires
NA
Data Processing
Data Editing
This dataset has been produced by extracting all electrcity metering data from the original NRS Load Research SQL database using the saveRawProfiles function from the delretrieve python package (https://github.com/wiebket/delretrieve: release v1.0). Full instructions on how to use delretrieve to extract data are in the README file contained in the package.
DATA EXTRACTION AND FILE STRUCTURE
To manage data volumes, meter readings were extracted in batches and are stored in a file hierarchy arranged by metering unit (A, Hz, kVA, kW, V) and collection year (1994 - 2015).
MISSING VALUES
No post-processing was done after data extraction and all database records, including missing values, are stored exactly as retrieved.
DATA EXTRACTION AND FILE STRUCTURE
To manage data volumes, meter readings were extracted in batches and are stored in a file hierarchy arranged by metering unit (A, Hz, kVA, kW, V) and collection year (1994 - 2015).
MISSING VALUES
No post-processing was done after data extraction and all database records, including missing values, are stored exactly as retrieved.
Data Appraisal
Data Appraisal
CALIBRATION of voltages and instruments
Prior to 2009 data loggers were built inhouse and only elementary calibration was done (insufficient for commercial standards). After 2009 all loggers were changed to commercial loggers with standard industry calibration of electricity meters.
TIME SYNCHRONISATION
Meter readings have date and time stamps. Every time data was downloaded from the logger, the meter clock was adjusted to the laptop clock, which was set before going into the field.
LOGGING ERRORS
Early logging devices had a 6 week storage capacity. When this capacity was exceeded a "data buffer full" error would occur.
Other common modes of technical failure included 'floating' data channels, readings failing to '0' load and readings failing to full scale Amps.
DATA VALIDATION MODELS
A data marking table was generated to validate profile IDs on each day against a set of data quality rules (incuded as external resoure). Based on these rules readings were marked as 'Y' (valid) or 'N' (invalid).
SAMPLING SUFFICIENCY
Sampling sufficiency was determined by calculating the standard deviation on customer behaviour at the time of annual peak demand (ie 60 or more customers were require to contribute to the annual peak demand, within an acceptable standard deviation)
Prior to 2009 data loggers were built inhouse and only elementary calibration was done (insufficient for commercial standards). After 2009 all loggers were changed to commercial loggers with standard industry calibration of electricity meters.
TIME SYNCHRONISATION
Meter readings have date and time stamps. Every time data was downloaded from the logger, the meter clock was adjusted to the laptop clock, which was set before going into the field.
LOGGING ERRORS
Early logging devices had a 6 week storage capacity. When this capacity was exceeded a "data buffer full" error would occur.
Other common modes of technical failure included 'floating' data channels, readings failing to '0' load and readings failing to full scale Amps.
DATA VALIDATION MODELS
A data marking table was generated to validate profile IDs on each day against a set of data quality rules (incuded as external resoure). Based on these rules readings were marked as 'Y' (valid) or 'N' (invalid).
SAMPLING SUFFICIENCY
Sampling sufficiency was determined by calculating the standard deviation on customer behaviour at the time of annual peak demand (ie 60 or more customers were require to contribute to the annual peak demand, within an acceptable standard deviation)
Access policy
Contacts
| Name | Affiliation | URL | |
|---|---|---|---|
| DataFirst | University of Cape Town | support@data1st.org | support.data1st.org |
Access conditions
Secure Research Data Centre access https://www.datafirst.uct.ac.za/services/secure-data-services
Citation requirements
Eskom, Stellenbosch University, University of Cape Town. Domestic Electrical Load Metering-Secure Data 1994-2014 [dataset]. version 1. Johannesburg: Eskom, Cape Town: UCT, Stellenbosch: US [producers], 2014. Cape Town: DataFirst [distributor]. DOI: https://doi.org/10.25828/p3k7-r965
Access authority
| Name | Affiliation | URL | |
|---|---|---|---|
| DataFirst | University of Cape Town | support@data1st.org | support.data1st.org |
Metadata production
DDI Document ID
DDI-ZAF-DELMS-1994-v1
Producers
| Name | Abbreviation | Affiliation | Role |
|---|---|---|---|
| Energy Research Centre | ERC | University of Cape Town | Metadata producer |
Date of Metadata Production
2022-12-14
DDI Document version
Version 2