Please wait a minute...
Submit  |   Chinese  | 
Advanced Search
   Home  |  Online Now  |  Current Issue  |  Focus  |  Archive  |  For Authors  |  Journal Information   Open Access  
Submit  |   Chinese  | 
Engineering    2017, Vol. 3 Issue (2) : 257 -265
Research |
Optimal Bidding and Operation of a Power Plant with Solvent-Based Carbon Capture under a CO2 Allowance Market: A Solution with a Reinforcement Learning-Based Sarsa Temporal-Difference Algorithm
Ziang Li1,Zhengtao Ding1(),Meihong Wang2
1. School of Electrical and Electronic Engineering, The University of Manchester, Manchester M13 9PL, UK
2. Department of Chemical and Biological Engineering, The University of Sheffield, Sheffield S1 3JD, UK

In this paper, a reinforcement learning (RL)-based Sarsa temporal-difference (TD) algorithm is applied to search for a unified bidding and operation strategy for a coal-fired power plant with monoethanolamine (MEA)-based post-combustion carbon capture under different carbon dioxide (CO2) allowance market conditions. The objective of the decision maker for the power plant is to maximize the discounted cumulative profit during the power plant lifetime. Two constraints are considered for the objective formulation. Firstly, the tradeoff between the energy-intensive carbon capture and the electricity generation should be made under presumed fixed fuel consumption. Secondly, the CO2 allowances purchased from the CO2 allowance market should be approximately equal to the quantity of CO2 emission from power generation. Three case studies are demonstrated thereafter. In the first case, we show the convergence of the Sarsa TD algorithm and find a deterministic optimal bidding and operation strategy. In the second case, compared with the independently designed operation and bidding strategies discussed in most of the relevant literature, the Sarsa TD-based unified bidding and operation strategy with time-varying flexible market-oriented CO2 capture levels is demonstrated to help the power plant decision maker gain a higher discounted cumulative profit. In the third case, a competitor operating another power plant identical to the preceding plant is considered under the same CO2 allowance market. The competitor also has carbon capture facilities but applies a different strategy to earn profits. The discounted cumulative profits of the two power plants are then compared, thus exhibiting the competitiveness of the power plant that is using the unified bidding and operation strategy explored by the Sarsa TD algorithm.

Keywords Power plants      Post-combustion carbon capture      Chemical absorption      CO2 allowance market      Optimal decision-making      Reinforcement learning     
Corresponding Authors: Zhengtao Ding   
Just Accepted Date: 24 March 2017   Online First Date: 17 April 2017    Issue Date: 27 April 2017
E-mail this article
E-mail Alert
Articles by authors
Ziang Li
Zhengtao Ding
Meihong Wang
Cite this article:   
Ziang Li,Zhengtao Ding,Meihong Wang. Optimal Bidding and Operation of a Power Plant with Solvent-Based Carbon Capture under a CO2 Allowance Market: A Solution with a Reinforcement Learning-Based Sarsa Temporal-Difference Algorithm[J]. Engineering, 2017, 3(2): 257 -265 .
URL:     OR
1   Lawal A, Wang M, Stephenson P, Yeung H. Dynamic modelling of CO2 absorption for post combustion capture in coal-fired power plants. Fuel 2009;88(12):2455–62
doi: 10.1016/j.fuel.2008.11.009
2   Wang M, Lawal A, Stephenson P, Sidders J, Ramshaw C. Post-combustion CO2 capture with chemical absorption: A state-of-the-art review. Chem Eng Res Des 2011;89(9):1609–24
doi: 10.1016/j.cherd.2010.11.005
3   Lin YJ, Pan TH, Wong DSH, Jang SS, Chi YW, Yeh CH. Plantwide control of CO2 capture by absorption and stripping using monoethanolamine solution. Ind Eng Chem Res 2011;50(3):1338–45
doi: 10.1021/ie100771x
4   Lin YJ, Wong DSH, Jang SS, Ou JJ. Control strategies for flexible operation of power plant with CO2 capture plant. AIChE J 2012;58(9):2697–704
doi: 10.1002/aic.12789
5   Luu MT, Manaf NA, Abbas A. Dynamic modelling and control strategies for flexible operation of amine-based post-combustion CO2 capture systems. Int J Greenh Gas Control 2015;39:377–89
doi: 10.1016/j.ijggc.2015.05.007
6   Nittaya T, Douglas PL, Croiset E, Ricardez-Sandoval LA. Dynamic modelling and control of MEA absorption processes for CO2 capture from power plants. Fuel 2014;116:672–91
doi: 10.1016/j.fuel.2013.08.031
7   Sahraei MH, Ricardez-Sandoval L. Controllability and optimal scheduling of a CO2 capture plant using model predictive control. Int J Greenh Gas Control 2014;30:58–71
doi: 10.1016/j.ijggc.2014.08.017
8   Luo X, Wang M. Optimal operation of MEA-based post-combustion carbon capture for natural gas combined cycle power plants under different market conditions. Int J Greenh Gas Control 2016;48(2):312–20
doi: 10.1016/j.ijggc.2015.11.014
9   Mac Dowell N, Shah N. Identification of the cost-optimal degree of CO2 capture: An optimisation study using dynamic process models. Int J Greenh Gas Control 2013;13:44–58
doi: 10.1016/j.ijggc.2012.11.029
10   Luckow P, Stanton EA, Fields S, Biewald B, Jackson S, Fisher J, et al. 2015 carbon dioxide price forecast. Cambridge (MA): Synapse Energy Economics, Inc; 2015 Mar.
11   California Environmental Protection Agency.California cap on greenhouse gas emissions and market-based compliance mechanisms [Internet]. Eagan: Thomson Reuters; c2017 [cited 2016 Nov 5]. Available from:
12   Chen Y, Wang L. A power market model with renewable portfolio standards, green pricing and GHG emissions trading programs. In: Proceedings of the Energy 2030 Conference; 2008 Nov 17–18; Atlanta, USA.Piscataway: IEEE; 2008. p. 1–7
doi: 10.1109/energy.2008.4780995
13   Nanduri V. Application of reinforcement learning-based algorithms in CO2 allowance and electricity markets. In: Proceedings of the 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL); 2011 Apr 11–15; Paris: France.Piscataway: IEEE; 2011. p. 164–9.
14   Air Resources Board. 2016 detailed auction requirements and instructions, California cap-and-trade program and Québec cap-and-trade system joint auction of greenhouse gas allowances [Internet].[cited 2016 Oct 24]. Available from:
15   AspenTech. Rate-based model of the CO2 capture process by MEA using Aspen Plus. Burlington: Aspen Technology, Inc; 2008. 23p.
16   Dugas RE. Pilot plant study of carbon dioxide capture by aqueous monoethanolamine [dissertation]. Austin: The University of Texas at Austin; 2006.
17   Lawal A, Wang M, Stephenson P, Obi O. Demonstrating full-scale post-combustion CO2 capture for coal-fired power plants through dynamic modelling and simulation. Fuel 2012;101:115–28
doi: 10.1016/j.fuel.2010.10.056
18   Agbonghae EO, Hughes KJ, Ingham DB, Ma L, Pourkashanian M. Optimal process design of commercial-scale amine-based CO2 capture plants. Ind Eng Chem Res 2014;53(38):14815–29
doi: 10.1021/ie5023767
19   Aroonwilas A, Veawab A. Integration of CO2 capture unit using single- and blended-amines into supercritical coal-fired power plants: Implications for emission and energy management. Int J Greenh Gas Control 2007;1(2):143–50
doi: 10.1016/S1750-5836(07)00011-4
20   Oko E, Wang M. Dynamic modelling, validation and analysis of coal-fired subcritical power plant. Fuel 2014;135:292–300
doi: 10.1016/j.fuel.2014.06.055
21   US Energy Information Administration.Updated capital cost estimates for utility scale electricity generating plants. Final report.Washington DC: US Energy Information Administration; 2013 Apr.
22   Song H, Liu CC, Lawarrée J, Dahlgren RW. Optimal electricity supply bidding by Markov decision process. IEEE Trans Power Syst 2000;15(2):618–24
doi: 10.1109/59.867150
23   Sutton RS, Barto AG. Reinforcement learning: An introduction.Cambridge: MIT press; 1998.
24   Busoniu L, Babuska R, De Schutter B, Ernst D. Reinforcement learning and dynamic programming using function approximators.Boca Raton: CRC press; 2010
doi: 10.1201/9781439821091
[1] Zhuo Cheng, Lang Qin, Jonathan A. Fan, Liang-Shih Fan. New Insight into the Development of Oxygen Carrier Materials for Chemical Looping Systems[J]. Engineering, 2018, 4(3): 343 -351 .
[2] Jennifer A. Clark, Erik E. Santiso. Carbon Sequestration through CO2 Foam-Enhanced Oil Recovery: A Green Chemistry Perspective[J]. Engineering, 2018, 4(3): 336 -342 .
[3] Andrea Di Maria, Karel Van Acker. Turning Industrial Residues into Resources: An Environmental Impact Assessment of Goethite Valorization[J]. Engineering, 2018, 4(3): 421 -429 .
[4] Lance A. Davis. Falcon Heavy[J]. Engineering, 2018, 4(3): 300 .
[5] Augusta Maria Paci. A Research and Innovation Policy for Sustainable S&T: A Comment on the Essay ‘‘Exploring the Logic and Landscape of the Knowledge System”[J]. Engineering, 2018, 4(3): 306 -308 .
[6] Ning Duan. When Will Speed of Progress in Green Science and Technology Exceed that of Resource Exploitation and Pollutant Generation?[J]. Engineering, 2018, 4(3): 299 .
[7] Jian-guo Li, Kai Zhan. Intelligent Mining Technology for an Underground Metal Mine Based on Unmanned Equipment[J]. Engineering, 2018, 4(3): 381 -391 .
[8] Veena Sahajwalla. Green Processes: Transforming Waste into Valuable Resources[J]. Engineering, 2018, 4(3): 309 -310 .
[9] Junye Wang, Hualin Wang, Yi Fan. Techno-Economic Challenges of Fuel Cell Commercialization[J]. Engineering, 2018, 4(3): 352 -360 .
[10] Raymond RedCorn, Samira Fatemi, Abigail S. Engelberth. Comparing End-Use Potential for Industrial Food-Waste Sources[J]. Engineering, 2018, 4(3): 371 -380 .
[11] Ning Duan, Linhua Jiang, Fuyuan Xu, Ge Zhang. A Non-Contact Original-State Online Real-Time Monitoring Method for Complex Liquids in Industrial Processes[J]. Engineering, 2018, 4(3): 392 -397 .
[12] Keith E. Gubbins, Kai Gu, Liangliang Huang, Yun Long, J. Matthew Mansell, Erik E. Santiso, Kaihang Shi, Małgorzata Ś liwińska-Bartkowiak, Deepti Srivastava. Surface-Driven High-Pressure Processing[J]. Engineering, 2018, 4(3): 311 -320 .
[13] Steff Van Loy, Koen Binnemans, Tom Van Gerven. Mechanochemical-Assisted Leaching of Lamp Phosphors: A Green Engineering Approach for Rare-Earth Recovery[J]. Engineering, 2018, 4(3): 398 -405 .
[14] Robert S. Weber, Johnathan E. Holladay. Modularized Production of Value-Added Products and Fuels from Distributed Waste Carbon-Rich Feedstocks[J]. Engineering, 2018, 4(3): 330 -335 .
[15] Hualin Wang, Pengbo Fu, Jianping Li, Yuan Huang, Ying Zhao, Lai Jiang, Xiangchen Fang, Tao Yang, Zhaohui Huang, Cheng Huang. Separation-and-Recovery Technology for Organic Waste Liquid with a High Concentration of Inorganic Particles[J]. Engineering, 2018, 4(3): 406 -415 .
Copyright © 2015 Higher Education Press & Engineering Sciences Press, All Rights Reserved.