文档名：基于分层约束强化学习的综合能源多微网系统优化调度
摘要：构建多微网系统是消纳可再生能源、提升电网稳定性的有效方式.通过各微网的协调调度,可有效提升微网的运行效益以及可再生能源的消纳水平.现有多微网优化问题场景多元,变量众多,再加上源荷不确定性及多微网主体的数据隐私保护等问题,为模型的高效求解带来了巨大挑战.为此,该文提出了一种分层约束强化学习优化方法.首先,构建了多微网分层强化学习优化框架,上层由智能体给出各微网储能优化策略和微网间功率交互策略;下层各微网以上层策略为约束,基于自身状态信息采用数学规划法对各微网内部的分布式电源出力进行自治优化.通过分层架构,减小通信压力,保护微网内部数据隐私,充分发挥强化学习对源荷不确定性的自适应能力,大幅提升了模型求解速度,并有效兼顾了数学规划法的求解精度.此外,将拉格朗日乘子法与传统强化学习方法相结合,提出一种约束强化学习求解方法,有效地解决了传统强化学习方法难以处理的约束越限问题.最后通过算例验证了该方法的有效性和优势.
Abstract：Theoptimizationoftheintegratedenergymulti-microgridsystemisacomplextask,withnumerousvariablesandchallengesincludingdataprivacyprotectionanduncertaintiesofpowergenerationandload,posingsignificantchallengesfortheefficientimplementationoftraditionalmathematicaloptimizationmethods.Recently,manyscholarshaveturnedtheirattentiontodeepreinforcementlearning(DRL)methods,whichrelyondata-drivenprinciplesandexhibitstrongadaptabilitytouncertaintiesofpowergenerationandload.Nevertheless,thedifficultyofconvergencepersistswithincreasingsystemscale,andtraditionalDRLmethodsthathandleconstraintsbyaddingpenaltytermstotherewardfunctionmayobscuretheboundarybetweenobjectivesandconstraints,makingitdifficulttoensurethatconstraintsarefullysatisfiedandresultinginexcessivelyconservativelearningstrategiesorsuboptimalsolutions.Toaddresstheseissues,thispaperproposedahierarchicalconstraintreinforcementlearningoptimizationmethod.Firstly,thispaperproposedahierarchicalDRLoptimizationframeworkformulti-microgridsystems.Theproposedframeworkdividestheoptimizationproblemintotwolayers:anupperlayerandalowerlayer.Theupperlayerdoesnotrequireobtainingalltheoperatingstatusinformationofeachmicrogrid.Instead,itutilizesnetloadpredictioninformationandenergystoragestateinformationtoprovideenergystorageoptimizationstrategiesandpowerinteractionstrategies.Ontheotherhand,thelowerlayerenableseachmicrogridtoautonomouslyoptimizetheoutputofitsinternaldevicesbasedonitsownstatusinformationthroughmathematicalprogramming,withtheupperlayerstrategyasaconstraint.Theproposedframeworkleveragescooperationbetweentheupperandlowerlayerstoachieveoveralloptimizationofthemulti-microgridsystem.ThisframeworkfullyutilizestheadvantagesofDRLbasedondata-drivenprinciplesandeffectivelyconsidersthesolutionaccuracyofmathematicalprogramming.Basedonthishierarchicalframework,aconstraintDRLmethodisproposedthatcombinesDRLmethodswithLagrangemultipliermethods.Thismethodtransformstheconstraintoptimizationproblemintoanunconstrainedoptimizationproblem,enablingtheagenttofindtheoptimalstrategywhilestrictlysatisfyingtheconstraints.Comparedtotraditionalcentralizedoptimizationmethods,theproposedmethoddynamicallyrespondstothefluctuationsofpowergenerationandloadtomeetonlineoptimizationrequirementsandprotectsmicrogriddataprivacybynotrequiringtheaggregationofallmicrogridstatusinformation.ComparedtogeneralDRLmethods,ourapproacheffectivelysolvestheproblemofconstraintviolationandsignificantlyimprovesboththeconvergencespeedandaccuracy.Thefollowingconclusionscanbedrawnfromthecasestudies:(1)Ahierarchicaldesignapproachisproposedtosimplifytheoptimizationofmulti-microgridsystems.Theapproachdoesnotrequireinformationexchangebetweenmicrogridsandonlynecessitatesuploadingnetloadandenergystoragestateinformation.Microgridscanindependentlyandparallellysolvetheoptimizationproblembasedontheirownstatusinformation.Thisapproachcanprovideschedulingresultsinreal-timeconsistentwiththeoptimalsolutionwhenlocalstatusinformationisavailable.(2)Theproposedapproachcombinesdata-drivenprincipleswithtraditionalmethods,simplifyingthecomplexityofactionspaceandrewarddesign.IteffectivelybalancestherapidsolvingabilityofDRLandthesolutionaccuracyofmathematicalprogramming.ComparedtotraditionalDRLmethods,theproposedapproachsignificantlyimprovesbothconvergencespeedandaccuracy.(3)TheapproachcombinesDRLmethodswithLagrangemultipliermethodstotransformtheconstrainedoptimizationproblemintoanunconstrainedone.Thisensuresthattheagentcanfindtheoptimalstrategywhilestrictlysatisfyingtheconstraints.TheapproachavoidsconvergencedifficultiesandconstraintviolationissuescausedbymanuallysettingthepenaltycoefficientintraditionalDRLmethods.(4)Themodelexhibitsrobustnessandcaneffectivelyadapttothefluctuationsofpowergenerationandload,makingrapiddecisionsonpowerinteractionsofeachmicrogrid.
作者：董雷 杨子民 乔骥 陈盛 王新迎 蒲天骄 Author：DongLei YangZimin QiaoJi ChenSheng WangXinying PuTianjiao
作者单位：华北电力大学电气与电子工程学院北京102206中国电力科学研究院有限公司北京100192
刊名：电工技术学报
Journal：TransactionsofChinaElectrotechnicalSociety
年，卷(期)：2024, 39(5)
分类号：TM73
关键词：多微网系统 分层约束强化学习 不确定性 数据隐私保护
Keywords：Multi-microgrid hierarchicalconstraintreinforcementlearning uncertainty dataprivacyprotection
机标分类号：TM73TP391TV697.1
在线出版日期：2024年3月19日
基金项目：国家重点研发计划，国家自然科学基金基于分层约束强化学习的综合能源多微网系统优化调度[
期刊论文] 电工技术学报--2024, 39(5)董雷 杨子民 乔骥 陈盛 王新迎 蒲天骄构建多微网系统是消纳可再生能源、提升电网稳定性的有效方式.通过各微网的协调调度,可有效提升微网的运行效益以及可再生能源的消纳水平.现有多微网优化问题场景多元,变量众多,再加上源荷不确定性及多微网主体的数据隐私...参考文献和引证文献
参考文献
引证文献
本文读者也读过
相似文献
相关博文
基于分层约束强化学习的综合能源多微网系统优化调度 Optimal Scheduling of Integrated Energy Multi-Microgrid System Based on Hierarchical Constraint Reinforcement Learning
基于分层约束强化学习的综合能源多微网系统优化调度.pdf
- 文件大小:
- 1.73 MB
- 下载次数:
- 60
-
高速下载
|
|
