Grid integration and smart grid implementation of emerging technologies in electric power systems through approximate dynamic programming

Xiao, Jingjie (2013): Grid integration and smart grid implementation of emerging technologies in electric power systems through approximate dynamic programming. Published in: Ph.D. Dissertation (13 August 2013)

Abstract

A key hurdle for implementing real-time pricing of electricity is a lack of con-sumers’ responses. Solutions to overcome the hurdle include the energy management system that automatically optimizes household appliance usage such as plug-in hybrid electric vehicle charging (and discharging with vehicle-to-grid) via a two-way com-munication with the grid. Real-time pricing, combined with household automation devices, has a potential to accommodate an increasing penetration of plug-in hybrid electric vehicles. In addition, the intelligent energy controller on the consumer-side can help increase the utilization rate of the intermittent renewable resource, as the demand can be managed to match the output profile of renewables, thus making the intermittent resource such as wind and solar more economically competitive in the long run.

One of the main goals of this dissertation is to present how real-time retail pricing, aided by control automation devices, can be integrated into the wholesale electricity market under various uncertainties through approximate dynamic programming. What distinguishes this study from the existing work in the literature is that whole-sale electricity prices are endogenously determined as we solve a system operator’s economic dispatch problem on an hourly basis over the entire optimization horizon. This modeling and algorithm framework will allow a feedback loop between electricity prices and electricity consumption to be fully captured. While we are interested in a near-optimal solution using approximate dynamic programming; deterministic linear programming benchmarks are use to demonstrate the quality of our solutions.The other goal of the dissertation is to use this framework to provide numerical ev-idence to the debate on whether real-time pricing is superior than the current flat rate structure in terms of both economic and environmental impacts. For this pur-pose, the modeling and algorithm framework is tested on a large-scale test case with hundreds of power plants based on data available for California, making our findings useful for policy makers, system operators and utility companies to gain a concrete understanding on the scale of the impact with real-time pricing.

• Item Type: MPRA Paper
• Language: English
• Keywords: Energy Economics, Electricity Markets, Plug in hybrid vehicles, electric vehicles, mathematical modeling
• Subjects: C - Mathematical and Quantitative Methods > C6 - Mathematical Methods ; Programming Models ; Mathematical and Simulation Modeling; Q - Agricultural and Natural Resource Economics ; Environmental and Ecological Economics > Q2 - Renewable Resources and Conservation; Q - Agricultural and Natural Resource Economics ; Environmental and Ecological Economics > Q4 - Energy; Q - Agricultural and Natural Resource Economics ; Environmental and Ecological Economics > Q4 - Energy > Q42 - Alternative Energy Sources; Q - Agricultural and Natural Resource Economics ; Environmental and Ecological Economics > Q4 - Energy > Q47 - Energy Forecasting; Q - Agricultural and Natural Resource Economics ; Environmental and Ecological Economics > Q4 - Energy > Q48 - Government Policy

References

H. Allcott, “Rethinking real-time electricity pricing,” Resource and Energy Eco-nomics, vol. 33, no. 4, pp. 820–842, 2011.

H. Allcott, “Real time pricing and electricity market design,” Working Paper, New York University, 2012.

S. Borenstein, M. Jaske, and A. Rosenfeld, “Dynamic pricing, advanced me-tering, and demand response in electricity markets,” University of California Energy Institute, 2002.

J. Bushnell, B. Hobbs, and F. Wolak, “When it comes to demand response, is FERC its own worst enemy?,” The Electricity Journal, vol. 22, no. 8, pp. 9–18, 2009.

W. Hogan, “Demand response pricing in organized wholesale markets,” ISO/RTO Council Comments on Demand Response Compensation in Organized Wholesale Energy Markets, vol. 13, 2010.

Y. Oum, S. Oren, and S. Deng, “Hedging quantity risks with standard power options in a competitive wholesale electricity market,” Naval Research Logistics, vol. 53, no. 7, pp. 697–712, 2006.

A. Rose, G. Oladosu, and S. Liao, “Business interruption impacts of a terrorist attack on the electric power system of Los Angeles: customer resilience to a total blackout,” Risk Analysis, vol. 27, no. 3, pp. 513–531, 2007.

Y. Xue, “The was from a simple contingency to system-wide disaster – Lessons from the Eastern Interconnection blackout in 2003,” Automation of Electric Power Systems, vol. 18, 2003.

S. Chen, S. Song, L. Li, and J. Shen, “Survey on smart grid technology,” Power System Technology, vol. 33, no. 8, pp. 1–7, 2009.

X. Fang, S. Misra, G. Xue, and D. Yang, “Smart grid – the new and improved power grid: A survey,” 2011.

H. Farhangi, “The path of the smart grid,” IEEE Power and Energy Magazine, vol. 8, no. 1, pp. 18–28, 2010.

J. Kassakian and R. Schmalensee, “The future of the electric grid,” An Inter-disciplinary MIT Study, 2011.

S. Massoud and B. Wollenberg, “Toward a smart grid: Power delivery for the 21st century,” IEEE Power and Energy Magazine, vol. 3, no. 5, pp. 34–41, 2005.

Federal Energy Regulatory Commission, “A national assessment of demand response potential,” 2009.

M. Albadi and E. El-Saadany, “Demand response in electricity markets: An overview,” pp. 1–5, 2007.

M. Albadi and E. El-Saadany, “A summary of demand response in electricity markets,” Electric Power Systems Research, vol. 78, no. 11, pp. 1989–1996, 2008.

P. Cappers, C. Goldman, and D. Kathan, “Demand response in US electricity markets: Empirical evidence,” Energy, vol. 35, no. 4, pp. 1526–1535, 2010.

J. Torriti, M. Hassan, and M. Leach, “Demand response experience in Europe: Policies, programs and implementation,” Energy, vol. 35, no. 4, pp. 1575–1583, 2010.

A. Faruqui, R. Hledik, and J. Palmer, “Time-varying and dynamic rate design,” Regulatory Assistance Project, 2012.

H. Aalami, M. Moghaddam, and G. Yousefi, “Demand response modeling con-sidering interruptible/curtailable loads and capacity market programs,” Applied Energy, vol. 87, no. 1, pp. 243–250, 2010.

S. Caron and G. Kesidis, “Incentive-based energy consumption scheduling algorithms for the smart grid,” First IEEE International Conference on Smart Grid Communications, pp. 391–396, 2010.

M. Parvania and M. Fotuhi-Firuzabad, “Demand response scheduling by stochastic SCUC,” IEEE Transactions on Smart Grid, vol. 1, no. 1, pp. 89– 98, 2010.

M. El-Hawary, Introduction to electrical power systems, vol. 50. John Wiley & Sons, 2008.

Federal Energy Regulatory Commission, “National action plan on demand response,” 2010.

D. Hart, “Using AMI to realize the Smart Grid,” IEEE Power and Energy Society General Meeting-Conversion and Delivery of Electrical Energy in the 21st Century, pp. 1–2, 2008.

S. Andersen, “Saving the smart grid – Hype, hysteria, and strategic planning,” Public Utilities Fortnightly, pp. 33–39, January 2011.

C. Ibars, M. Navarro, and L. Giupponi, “Distributed demand management in smart grid with a congestion game,” First IEEE International Conference on Smart Grid Communications, pp. 495–500, 2010.

A. Mohsenian-Rad, V. Wong, J. Jatskevich, R. Schober, and A. Leon-Garcia, “Autonomous demand-side management based on game-theoretic energy con-sumption scheduling for the future smart grid,” IEEE Transactions on Smart Grid, vol. 1, no. 3, pp. 320–331, 2010.

C. Stephan and J. Sullivan, “Environmental and energy implications of plug-in hybrid-electric vehicles,” Environmental Science & Technology, vol. 42, no. 4, pp. 1185–1190, 2008.

C. Giff, R. Hill, M. Gardner, and M. Hasegawa, “Gaining traction: A customer view of electric vehicle mass adoption in the US automotive market,” Deloitte Development LLC, 2010.

S. Hadley and A. Tsvetkova, “Potential impacts of plug-in hybrid electric ve-hicles on regional power generation,” The Electricity Journal, vol. 22, no. 10, pp. 56–68, 2009.

K. Clement, E. Haesen, and J. Driesen, “The impact of charging plug-in hybrid electric vehicles on the distribution grid,” 2008.

C. Roe, J. Meisel, A. Meliopoulos, F. Evangelos, and T. Overbye, “Power system level impacts of PHEVs,” IEEE International Conference on System Sciences, pp. 1–10, 2009.

K. Clement, E. Haesen, and J. Driesen, “Coordinated charging of multiple plug-in hybrid electric vehicles in residential distribution grids,” pp. 1–7, 2009.

K. Clement, E. Haesen, and J. Driesen, “The impact of charging plug-in hybrid electric vehicles on a residential distribution grid,” IEEE Transactions on Power Systems, vol. 25, no. 1, pp. 371–380, 2010.

P. Denholm and W. Short, “An evaluation of utility system impacts and benefits of optimally dispatched plug-in hybrid electric vehicles,” National Renewable Energy Laboratory, 2006.

E. Sortomme, M. Hindi, S. Mac Pherson, and S. Venkata, “Coordinated charging of plug-in hybrid electric vehicles to minimize distribution system losses,” IEEE Transactions on Smart Grid, vol. 2, no. 1, pp. 198–205, 2011.

G. Kalogridis, C. Efthymiou, S. Denic, T. Lewis, and R. Cepeda, “Privacy for smart meters: Towards undetectable appliance load signatures,” pp. 232–237, 2010.

J. Axsen, K. Kurani, R. McCarthy, and C. Yang, “Plug-in hybrid vehicle GHG impacts in California: Integrating consumer-informed recharge profiles with an electricity-dispatch model,” Energy Policy, vol. 39, no. 3, pp. 1617–1629, 2011.

S. Huang, B. Hodge, F. Taheripour, J. Pekny, G. Reklaitis, and W. Tyner, “The effects of electricity pricing on PHEV competitiveness,” Energy Policy, vol. 39, no. 3, pp. 1552–1561, 2011.

K. Parks, P. Denholm, and A. Markel, “Costs and emissions associated with plug-in hybrid electric vehicle charging in the Xcel Energy Colorado service territory,” National Renewable Energy Laboratory, 2007.

A. Conejo, J. Morales, and L. Baringo, “Real-time demand response model,” IEEE Transactions on Smart Grid, vol. 1, no. 3, pp. 236–242, 2010.

S. Han, S. Han, and K. Sezaki, “Development of an optimal vehicle-to-grid aggregator for frequency regulation,” IEEE Transactions on Smart Grid, vol. 1, no. 1, pp. 65–72, 2010.

S. Kishore and L. Snyder, “Control mechanisms for residential electricity de-mand in smart grids,” First IEEE International Conference on Smart Grid Communications, pp. 443–448, 2010.

K. Valentine, W. Temple, and K. Zhang, “Intelligent electric vehicle charging: Rethinking the valley-fill,” Journal of Power Sources, vol. 196, no. 24, pp. 10717–10726, 2011.

A. Mohsenian-Rad and A. Leon-Garcia, “Optimal residential load control with price prediction in real-time electricity pricing environments,” IEEE Transactions on Smart Grid, vol. 1, no. 2, pp. 120–133, 2010.

P. Samadi, A. Mohsenian, R. Schober, V. Wong, and J. Jatskevich, “Optimal real-time pricing algorithm based on utility maximization for smart grid,” First IEEE International Conference on Smart Grid Communications, pp. 415–420, 2010.

R. Sioshansi, “Modeling the impacts of electricity tariffs on plug-in hybrid elec-tric vehicle charging, costs, and emissions,” Operations Research, vol. 60, no. 3, pp. 506–516, 2012.

D. Livengood and R. Larson, “The energy box: Locally automated optimal control of residential electricity usage,” INFORMS Service Science, vol. 1, no. 1, pp. 1–16, 2009.

D. O’Neill, M. Levorato, A. Goldsmith, and U. Mitra, “Residential demand response using reinforcement learning,” First IEEE International Conference on Smart Grid Communications, pp. 409–414, 2010.

C. Chen, S. Kishore, and L. Snyder, “An innovative RTP-based residential power scheduling scheme for smart grids,” IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 5956–5959, 2011.

W. Kempton and J. Tomi´c, “Vehicle-to-grid power fundamentals: Calculating capacity and net revenue,” Journal of Power Sources, vol. 144, no. 1, pp. 268– 279, 2005.

W. Kempton and J. Tomi´c, “Vehicle-to-grid power implementation: From sta-bilizing the grid to supporting large-scale renewable energy,” Journal of Power Sources, vol. 144, no. 1, pp. 280–294, 2005.

A. Saber and G. Venayagamoorthy, “Intelligent unit commitment with vehicle-to-gridA cost-emission optimization,” Journal of Power Sources, vol. 195, no. 3, pp. 898–911, 2010.

R. Sioshansi and P. Denholm, “Emissions impacts and benefits of plug-in hybrid electric vehicles and vehicle-to-grid services,” Environmental science & technol-ogy, vol. 43, no. 4, pp. 1199–1204, 2009.

E. Sortomme and M. El-Sharkawi, “Optimal charging strategies for unidirectional vehicle-to-grid,” IEEE Transactions on Smart Grid, vol. 2, no. 1, pp. 131– 138, 2011.

California Public Utilities Commission, “33% renewables portfolio standard: Implementation analysis preliminary results,” Technical Analysis by E3 and Aspen Environmental Group, 2009.

M. Blanco, “The economics of wind energy,” Renewable and Sustainable Energy Reviews, vol. 13, no. 6, pp. 1372–1382, 2009.

A. Fabbri, T. Romn, J. Abbad, and V. Quezada, “Assessment of the cost associ-ated with wind generation prediction errors in a liberalized electricity market,” IEEE Transactions on Power Systems, vol. 20, no. 3, pp. 1440–1446, 2005.

B. Hodge and M. Milligan, “Wind power forecasting error distributions over multiple timescales,” pp. 1–8, 2011.

F. Bouffard and F. Galiana, “Stochastic security for operations planning with significant wind power generation,” pp. 1–11, 2008.

R. Doherty and M. O’Malley, “A new approach to quantify reserve demand in systems with significant installed wind capacity,” IEEE Transactions on Power Systems, vol. 20, no. 2, pp. 587–595, 2005.

J. Morales, A. Conejo, and J. P´erez-Ruiz, “Economic valuation of reserves in power systems with high penetration of wind power,” IEEE Transactions on Power Systems, vol. 24, no. 2, pp. 900–910, 2009.

M. Ortega-Vazquez and D. Kirschen, “Estimating the spinning reserve requirements in systems with significant wind power generation penetration,” IEEE Transactions on Power Systems, vol. 24, no. 1, pp. 114–124, 2009.

J. Wang, M. Shahidehpour, and Z. Li, “Security-constrained unit commitment with volatile wind power generation,” IEEE Transactions on Power Systems, vol. 23, no. 3, pp. 1319–1327, 2008.

J. Xiao, B. Hodge, J. Pekny, and G. Reklaitis, “Operating reserve policies with high wind power penetration,” Computers & Chemical Engineering, vol. 35, no. 9, pp. 1876–1885, 2011.

S. Borenstein, “The long-run effciency of real-time electricity pricing,” Energy Journal, vol. 26, no. 3, 2005.

C. De Jonghe, B. Hobbs, and R. Belmans, “Integrating short-term demand response into long-term investment planning,” 2011.

S. Gabriel, A. Kydes, and P. Whitman, “The National Energy Modeling System: a large-scale energy-economic equilibrium model,” Operations Research, vol. 49, no. 1, pp. 14–25, 2001.

L. Fishbone and H. Abilock, “Markal, a linear-programming model for energy systems analysis: Technical description of the bnl version,” International jour-nal of Energy research, vol. 5, no. 4, pp. 353–375, 1981.

S. Stoft, Power system economics. IEEE press Piscataway, 2002.

A. Mazer, Electric power planning for regulated and deregulated markets. Wiley, 2007.

A. Keane, M. Milligan, C. Dent, B. Hasche, C. D’Annunzio, K. Dragoon, H. Holttinen, N. Samaan, L. Soder, and M. O’Malley, “Capacity value of wind power,” IEEE Transactions on Power Systems, vol. 26, no. 2, pp. 564–572, 2011.

W. Powell, A. George, H. Sim~ao, W. Scott, A. Lamont, and J. Stewart, “SMART: A stochastic multiscale model for the analysis of energy resources, technology, and policy,” INFORMS Journal on Computing, vol. 24, no. 4, pp. 665–682, 2012.

R. Bellman, “Dynamic programming and the smoothing problem,” Management Science, vol. 3, no. 1, pp. 111–113, 1956.

W. B. Powell, Approximate dynamic programming: Solving the curses of di-mensionality. WILEY, 2nd ed., 2011.

D. Zhang and D. Adelman, “An approximate dynamic programming approach to network revenue management with customer choice,” Transportation Science, vol. 43, no. 3, pp. 381–394, 2009.

H. P. Sim~ao, J. Day, A. P. George, T. Gifford, J. Nienow, and W. Powell, “An approximate dynamic programming algorithm for large-scale fleet management: A case application,” Transportation Science, vol. 43, no. 2, pp. 178–197, 2009.

K. Kariuki and R. Allan, “Evaluation of reliability worth and value of lost load,” IEE proceedings-Generation, transmission and distribution, vol. 143, no. 2, pp. 171–180, 1996.

R. Rosenthal, “GAMS–a user’s guide,” ALS-NSCORT, 2004.

Cplex, ILOG, “11.0 Users manual,” ILOG SA, 2007.

California ISO, “California ISO Open Access Same-time Information System, http://oasis.caiso.com/.”

U.S. Environmental Protection Agency, “Emissions & Generation Re-source Integrated Database,

National Renewable Energy Laboratory, “NREL Wind Integration Datasets, http://www.nrel.gov/electricity/transmission/wind

J. Hamilton, Time series analysis, vol. 2. Cambridge University Press, 1994.

W. Hoeffding and H. Robbins, “The central limit theorem for dependent random variables,” Duke Mathematical Journal, vol. 15, no. 3, pp. 773–780, 1948.

B. Hodge, S. Huang, J. Siirola, J. Pekny, and G. V. Reklaitis, “A multi-paradigm modeling framework for energy systems simulation and analysis,” Computers & Chemical Engineering, vol. 35, no. 9, pp. 1725–1737, 2011.

B. Hodge, A. Shukla, S. Huang, G. Reklaitis, V. Venkatasubramanian, and J. Pekny, “Multi-paradigm modeling of the effects of PHEV adoption on electric utility usage levels and emissions,” Industrial & Engineering Chemistry Research, vol. 50, no. 9, pp. 5191–5203, 2011.

S. Huang, H. Safiullah, J. Xiao, B. Hodge, R. Hoffman, J. Soller, D. Jones,D. Dininger, W. Tyner, and A. Liu, “The effects of electric vehicles on residential households in the city of Indianapolis,” Energy Policy, 2012.

J. Ma and Y. Pei, “Development and application of TransCAD for urban traffc planning,” Journal of Harbin University of Civil Engineering and Architecture, 2002.

U.S. Department of Transportation Federal Highway Administration, “2009 national household travel survey user’s guide,” 2011.

R. Axelrod, The complexity of cooperation: Agent-based models of competition and collaboration. Princeton University Press, 1997.

M. Batty, Cities and complexity: understanding cities with cellular automata, agent-based models, and fractals. The MIT press, 2007.

E. Bonabeau, “Agent-based modeling: Methods and techniques for simulating human systems,” Proceedings of the National Academy of Sciences of the United States of America, vol. 99, no. Suppl 3, pp. 7280–7287, 2002.

AnyLogic, The big book of AnyLogic, http://www.anylogic.com/the-big-book-of-anylogic/.

T. Jaakkola, M. Jordan, and S. Singh, “On the convergence of stochastic it-erative dynamic programming algorithms,” Neural Computation, vol. 6, no. 6, pp. 1185–1201, 1994.

R. Sutton, “Learning to predict by the methods of temporal differences,” Ma-chine learning, vol. 3, no. 1, pp. 9–44, 1988.

C. Watkins and P. Dayan, “Q-learning,” Machine learning, vol. 8, no. 3-4, pp. 279–292, 1992.

J. Birge and F. Louveaux, Introduction to stochastic programming. Springer, 2nd ed., 2011.

Offce of Transportation and Air Quality, U.S. Environmental Protection Agency, “Average annual emissions and fuel consumption for gasoline-fueled passenger cars and light trucks,” 2008.

“EPA and NHTSA set standards to reduce greenhouse gases and improve fuel economy for model years 2017-2025 cars and light trucks,” 2012.

R. Tidball, J. Bluestein, N. Rodriguez, and S. Knoke, “Cost and performance as-sumptions for modeling electricity generation technologies,” Contract, vol. 303, pp. 275–300, 2010.

U.S. Environmental Protection Agency, “Unit conversions, emissions factors, and other reference data,” 2004.

K. Hassett, A. Mathur, and G. Metcalf, “The incidence of a US carbon tax: A lifetime and regional analysis,” National Bureau of Economic Research, 2007.

J. Xiao, B. Hodge, A. L., J. Pekny, and G. Reklaitis, “Long-term planning of wind farm siting in the electricity grid,” Proceeding for 21st European Sympo-sium on Computer Aided Process Engineering, 2011.

R. O’Neill, P. Sotkiewicz, B. Hobbs, M. Rothkopf, and J. Stewart, “Effcient market-clearing prices in markets with nonconvexities,” European Journal of Operational Research, vol. 164, no. 1, pp. 269–285, 2005.

M. Carri´on and J. Arroyo, “A computationally effcient mixed-integer linear formulation for the thermal unit commitment problem,” IEEE Transactions on Power Systems, vol. 21, no. 3, pp. 1371–1378, 2006.

P. Ruiz, C. Philbrick, E. Zak, K. Cheung, and P. Sauer, “Uncertainty management in the unit commitment problem,” IEEE Transactions on Power Systems, vol. 24, no. 2, pp. 642–651, 2009.

V. Kumar, A. Grama, A. Gupta, and G. Karypis, Introduction to parallel com-puting, vol. 110. Benjamin/Cummings Redwood City, 1994.