energies

Review
Application of Artificial Intelligence for EV Charging and
Discharging Scheduling and Dynamic Pricing: A Review
Qin Chen and Komla Agbenyo Folly *

Department of Electrical Engineering, University of Cape Town, Cape Town 7701, South Africa;
chnqin002@myuct.ac.za
* Correspondence: komla.folly@uct.ac.za

Abstract: The high penetration of electric vehicles (EVs) will burden the existing power delivery
infrastructure if their charging and discharging are not adequately coordinated. Dynamic pricing is a
special form of demand response that can encourage EV owners to participate in scheduling programs.
Therefore, EV charging and discharging scheduling and its dynamic pricing model are important
fields of study. Many researchers have focused on artificial intelligence-based EV charging demand
forecasting and scheduling models and suggested that artificial intelligence techniques perform better
than conventional optimization methods such as linear, exponential, and multinomial logit models.
However, only a few research studies focused on EV discharging scheduling (i.e., vehicle-to-grid,
V2G) because the concept of EV discharging electricity back to the power grid is relatively new and
evolving. Therefore, a review of existing EV charging and discharging-related studies is needed
to understand the research gaps and to make some improvements in future studies. This paper
reviews EV charging and discharging-related studies and classifies them into forecasting, scheduling,
and pricing mechanisms. The paper determines the linkage between forecasting, scheduling, and
pricing mechanism and identifies the research gaps in EV discharging scheduling and dynamic
pricing models.

Keywords: dynamic pricing; electric vehicles; neural networks; reinforcement learning; vehicle-to-grid

Citation: Chen, Q.; Folly, K.A.

1. Introduction
Application of Artificial Intelligence
for EV Charging and Discharging In recent years, the rollout of electric vehicles (EVs) has been accelerated in many
Scheduling and Dynamic Pricing: A countries to combat the energy crisis and environmental concerns such as high CO2 emis-
Review. Energies 2023, 16, 146. sions and climate change [1]. Around two million EVs were sold in the first quarter of 2022,
https://doi.org/10.3390/en16010146 an increase of 75% from the same period in 2021 [2]. The number of EVs will continue to
grow with governments’ incentives and policies around the globe. On the one hand, the
Academic Editor: Enrique
Romero-Cadaval
increasing and uncoordinated EV charging will burden the existing power grid. On the
other hand, EVs’ batteries are mobile energy storage systems that can be used to provide
Received: 24 October 2022 ancillary services for power grids, such as peak-shaving and valley-filling, voltage and
Revised: 8 December 2022 frequency regulations [3–5]. In addition, EVs’ batteries can be used as flexible load and
Accepted: 17 December 2022 supply to maximize renewable energy utilization, if EVs charging and discharging are
Published: 23 December 2022
properly coordinated to closely match renewable generation profiles [6,7].
The idea of supplying electricity from EVs’ batteries back to the power grid is known as
vehicle-to-grid (V2G). This idea was first introduced by Kempton and Letendre in 1997 [8].
Copyright: © 2022 by the authors.
Using EVs as energy storage through the V2G program can bring many environmental,
Licensee MDPI, Basel, Switzerland. economic, and social-technological benefits to program participants, such as EV owners,
This article is an open access article grid operators, government, and aggregators [9]. For example, the V2G services can reduce
distributed under the terms and the total ownership cost of EVs for EV owners by allowing them to sell stored energy back
conditions of the Creative Commons to power grids during peak hours [10]. Furthermore, the V2G program can reduce power
Attribution (CC BY) license (https:// grid congestion, emissions, and increase renewable energy utilization for grid operators by
creativecommons.org/licenses/by/ shifting EV charging to a high renewable generation period and discharging back to power
4.0/). grids during a low renewable energy generation and high load demand period [11,12].

Energies 2023, 16, 146. https://doi.org/10.3390/en16010146 https://www.mdpi.com/journal/energies

Energies 2023, 16, 146 2 of 26

However, the penetration of EVs is currently still low, and the concept of V2G is
relatively new and evolving. Many V2G projects remain in the pilot project stages of
development [13,14]. Some of the representative V2G pilot projects with different goals
and services are summarized in Table 1. A complete list of V2G pilot projects around the
world can be found in [14]. As shown in Table 1, most of the V2G pilot projects were only
initiated in recent years, and the scale of these pilot projects is small. Moreover, most of the
existing literature only focuses on the EV charging aspect. For example, the classification
of EVs’ charging techniques is discussed in [10,15,16]. Authors in [17–30] designed EV
charging strategies to minimize charging costs. In [31], Al-Ogaili et al., reviewed EVs’ charg-
ing scheduling, clustering, and forecasting techniques in the existing literature. Optimal
charging strategy for electric vehicles under dynamic pricing schemes, including Real-Time
Pricing (RTP), Time of Use (ToU), Peak Time Rebates (PTR), and Critical Peak Pricing (CPP),
has been reviewed in [32]. A few studies focused on different aspects of V2G, such as cus-
tomer acceptance of V2G [33], integration of renewable energy into transportation systems
via V2G [11], the economic feasibility of V2G for a university campus [10], coordination of
V2G with energy trade [12], and optimal energy management systems [34]. Encouraging
EV owners to participate in the V2G program without monetary incentives is difficult
because it costs energy and time to feed electricity back to power grids. Therefore, the
economic and operational aspects of V2G will become very important when the penetration
of EVs with V2G capability is high. Many EV charging scheduling approaches have been
effective in following ToU and RTP signals for peak demand reduction [35], alleviating
the impact of load fluctuation [36], and reducing EV charging costs [17–26]. However, the
effectiveness of these pricing policies in reflecting power system conditions has barely been
discussed in the literature.
Moreover, the economic and operational aspects of the V2G program, such as discharg-
ing scheduling and pricing mechanism, have only been proposed in a few papers [27,37–51].
Al-Ogaili et al. [31] suggested that artificial intelligence models perform better than proba-
bilistic models. With the availability of the dataset and increasing computational power,
artificial intelligence algorithms such as neural networks and reinforcement learning have
become very popular and effective for many applications, including forecasting and op-
timization problems. The learning ability from datasets of artificial intelligence models
makes them superior to conventional optimization models, such as linear and exponential
optimization models. Moreover, artificial intelligence models usually do not require expert
knowledge of complex systems, which might be challenging to obtain. As a result, artificial
intelligence models have been used in many EVs related studies ranging from EV battery
design and management to V2G applications [52]. A review of the different roles that artifi-
cial intelligence plays in the mass adoption of EVs can be found in [52]. Shahriar et al. [53]
reviewed machine learning-based EV charging behavior prediction and classification and
pointed out the potential of reinforcement learning for EV scheduling. In addition, many
artificial intelligence models have been implemented to solve EV charging scheduling-
related tasks such as predicting EV charging electricity price [5,20,22,54–57], EV driving pat-
terns [58], aggregated available capacity of batteries [59–62], charging load demand [63–67],
charging pattern [68], and EV charging/discharging scheduling [28,68–73]. However, the
linkage and gaps between each study have not been comprehensively discussed in the
existing literature. Thus, this review aims to explore artificial intelligence-based forecasting,
scheduling, and dynamic pricing models. In addition, the relationship between forecasting,
scheduling, and dynamic pricing is also discussed in this paper. Moreover, the research
gaps found in the existing literature are mentioned, and future research direction related to
EV charging and discharging is discussed.
The main contribution of this paper is to compare, summarize and analyze the existing
artificial intelligence-based algorithms of three critical EV charging/discharging compo-
nents: forecasting, scheduling, and dynamic pricing. This paper also points out the research
gaps in effective EV discharging scheduling and pricing policy based on the finding of the
existing literature.
Energies 2023, 16, 146 3 of 26

Table 1. Summary of some significant V2G pilot projects worldwide [14].

No. of
Project Name Country Timespan Service
Chargers
Realising Electric Vehicle to
Australia 51 2020–2022 Frequency response, reserve
Grid Services
Arbitrage, distribution services,
Parker Denmark 50 2016–2018
frequency response
Bidirektionales Arbitrage, frequency response,
Germany 50 2021–2022
Lademanagement—BDL time shifting
Fiat-Chrysler V2G Italy 600 2019–2021 Load balancing
Leaf to home Japan 4000 2012–ongoing Emergency backup, time shifting
Utrecht V2G charge hubs Netherlands 80 2018–ongoing Arbitrage
Distribution services, frequency
Share the Sun/Deeldezon Project Netherlands 80 2019–2021
response, time shifting
VGI core comp. dev. and V2G Arbitrage, frequency response,
South Korea 100 2018–2022
demo. using CC1 reserve, time shifting
Time shifting, pricing scheme
SunnYparc Switzerland 250 2022–2025
testing, reserve
Distribution services, reserve,
Electric Nation Vehicle to Grid UK 100 2020–2022
time shifting
OVO Energy V2G UK 320 2018–2021 Arbitrage
Powerloop: Domestic V2G Arbitrage, distribution services,
UK 135 2018–ongoing
Demonstrator Project emergency backup, time shifting
UK Vehicle-2-Grid (V2G) UK 100 2016–ongoing Support power grid
Distribution services, frequency
INVENT—UCSD/Nissan/Nuvve US 50 2017–2020
response, time shifting
Distribution services, frequency
SmartMAUI, Hawaii US 80 2012–2015
response, time shifting

The rest of the paper is structured as follows. Section 2 briefly discusses the EV charg-
ing/discharging control techniques, the concept of V2G, and battery degradation. Section 3
summarizes different artificial intelligence-based models for predicting EV charging-related
tasks such as electricity price and EV charging load demand. Artificial intelligence-based
EV charging/discharging scheduling models are reviewed in Section 4. Section 5 discusses
dynamic pricing and peer-to-peer energy transaction strategies. Discussion of the current
advanced technology and research gaps are presented in Section 6. Finally, conclusions and
future research directions are presented in Section 7.

2. EV Charging/Discharging and Battery Degradation

2.1. EV Charging and Discharging Techniques
There are four charging/discharging techniques, namely, uncontrolled charging-
discharging [10,15], controlled charging-discharging [10,16], smart charging [10], and indi-
rectly controlled charging [10]. Table 2 summarizes the benefits and challenges faced by
each charging/discharging technique. In addition, the schema of each charging technique
is illustrated in Figure 1. The uncontrolled charging-discharging approach allows EVs to
charge or discharge at rated power as soon as it is plugged in until the battery’s storage
level equals the maximum state of charge or unplugged [10,15]. Thus, this charging method
is inflexible for demand-side management DSM [10]. The uncontrolled charging technique
is convenient for EV owners to make charging decisions freely. However, uncontrolled EV
charging might cause a negative impact on local distribution networks, such as power loss,
demand-supply unbalance, shorter transformer lifespan, and harmonic distortion [31].
Energies 2023, 16, 146 4 of 26

Table 2. Summary of different EV charging and discharging techniques.

Techniques Benefits Challenges

• Easy implementation • Add burden to power grids
• EV owners have the freedom to make • Charging costs might be higher than
Uncontrolled
charging decisions smart charging
• Convenience for EV owners • Inflexible for demand-side management

• System operators have more freedom to • EV owners have to cede control to the
Controlled
make decisions system operators

• Balance demand and supply

• Make charging decisions based on the • Hard to encourage EV owners to
Smart real-time conditions participate in smart charging
• Maximize profits for power • Benefits for EV owners are usually unclear
system operators

• Use monetary terms to encourage EV

• Electricity pricing signals need to be
Indirectly controlled owners to participate in smart charging
accurate to be effective
• Incentive is clear

The controlled charging-discharging method, also known as unidirectional V2G, gives

system operators more freedom to decide when EVs will be charged and discharged [10,16].
However, EV owners have to cede control to the system operators or aggregators immedi-
ately after the EV is plugged in under the controlled charging-discharging strategy [74].
The smart charging technique manages EVs’ charging and discharging based on real-
time energy demand, grid requirements, and grid quality [10]. However, it is not easy to
encourage EV owners with different preferences to participate in smart charging programs
without incentives [75]. Smart charging allows EV owners to charge or discharge their
EVs at a certain time and rate to achieve predefined goals such as minimizing charging
costs or balancing demand and supply. However, smart charging strategies are usually
designed by system operators to maximize their profit. In addition, EV owners do not
clearly understand how smart charging can benefit them monetarily.
On the other hand, an indirectly controlled charging mechanism uses more straight-
forward price signals to incentivize EV owners to provide ancillary services to power grids.
Wang and Wang [3] suggested using macro-incentive policies such as variable electricity
tariffs to attract more EV owners to participate in the V2G system [3]. Dynamic pricing
schemes [76], such as Time of Use (ToU) and Real-Time Pricing (RTP), have been commonly
used as a special form of power load demand response, which encourages EV owners to
choose charging or discharging time according to financial incentives [4,10,37,42]. Dutschke
and Paetz [77] conducted two empirical studies in Germany to get the public’s opinions
about the dynamic electricity pricing determined by the Time of Use (ToU) tariff and
household load profile. The results indicate that consumers are open to dynamic pricing
but like more straightforward programs with a smaller price fluctuation range [77,78].
Latinopoulos et al. [79] investigated EV drivers’ responses to the dynamic pricing of park-
ing and charging services. The results suggest that younger individuals are more likely to
exhibit forward-looking behaviors. Therefore, indirectly controlled charging with dynamic
electricity pricing strategies will influence EV owners’ charging and discharging behaviors.
PEER REVIEW 5 of 25
Energies 2023, 16, 146 5 of 26

Figure 1. Schema of V2G development

Figure 1. Schema ofphases and corresponding
V2G development EV charging
phases and corresponding EV techniques, (a). smart
charging techniques, (a). smart
charging, (b). controlled charging,
charging, (c). smart
(b). controlled charging,
charging, (d). indirectly
(c). smart charging, (d).controlled charging.
indirectly controlled The solid
charging. The solid
black arrow indicates the power flow, the dash red arrow indicates
black arrow indicates the power flow, the dash red arrow indicates the charging control, and thethe charging control, and the blue
dash arrow indicates
blue dash arrow indicates the information flow. the information flow.

2.2. Vehicle to Grid (V2G) Concept

2.2. Vehicle to Grid (V2G) The
Concept
V2G technology allows EVs to utilize onboard batteries as an energy source for
driving and energy storage systems for power grids [8]. Therefore, utilizing EVs’ batteries
The V2G technology allows EVs to utilize onboard batteries as an energy source for
with fast charging and discharging reaction time (as fast as tens of milliseconds [16]) as
driving and energy storage systems
energy storage and for power
power grids
sources [8].technology
via V2G Therefore, canutilizing EVs’ batteries
avoid additional investment for
with fast charging and discharging
a battery reaction
storage system. time average
In addition, (as fastcars
asare
tens of milliseconds
parked 95% of the time, [16]) as to
equivalent
the total operating hours of baseload generators [8]. Therefore,
energy storage and power sources via V2G technology can avoid additional investment V2G technology can benefit
system operators by providing ancillary services and energy storage for renewable energy.
for a battery storage Itsystem. In addition, average cars are parked 95% of the time, equiv-
can also reduce EV owners’ charging costs by allowing them to sell extra stored energy
alent to the total operating hours of baseload generators [8]. Therefore, V2G technology
can benefit system operators by providing ancillary services and energy storage for re-
newable energy. It can also reduce EV owners’ charging costs by allowing them to sell
extra stored energy back to power grids. Parsons et al. [33] investigated potential EV own-
Energies 2023, 16, 146 6 of 26

back to power grids. Parsons et al. [33] investigated potential EV owners’ willingness to
pay for an EV with V2G capability and contract terms. The survey results suggest that the
V2G concept is most likely to attract more EV buyers if power aggregators provide either
upfront cash payment or pay-as-you-go basis types of contracts.
Lund and Kempton [11] modeled a power system that integrates renewable energy
into the transport and electricity sectors using V2G technology. The simulation results
indicate that adding EVs with V2G capability can enhance renewable energy utilization
(i.e., align EV charging pattern with renewable energy generation pattern) and reduce CO2
emissions. Scott et al. [10] simulated a V2G model for a university campus under various
charging scenarios. The simulation results show that using EVs’ batteries as energy storage
over ten years can reduce electricity costs by 64.7% and 9.79%, respectively, compared to
purchasing electricity from the power grid and using sole battery storage [10]. Al-Awami
and Sortomme [12] have formulated a mixed-integer stochastic linear programming model
to coordinate V2G services with energy trading. The simulation results show that using
V2G services to coordinate short-term energy trading can increase the profit of the load-
serving entity (LSE) by 2.4% and reduce emissions by 5.3% compared to the uncoordinated
one. The advantages and disadvantages of the V2G concept are summarized in Table 3.

Table 3. Summary of the advantages and disadvantages of V2G.

Advantages Disadvantages
• Avoid additional investment in a battery storage system
• Enhance renewable energy utilization, thus reducing emissions • Battery degradation concerns
• Mobile energy storage with a fast reaction time • Charging-discharging efficiency concerns
• Provide ancillary service for power grids • Additional upfront investment

• Reduce charging costs for EV owners

Although V2G technology can potentially provide many benefits to power systems
and EV owners, the implementation of V2G still faces some challenges, such as high upfront
investment [34], battery degradation [8], and charging-discharging efficiency concern [80].
In addition, the V2G concept is still relatively new and evolving. Many pilot V2G projects
remain in the development stages [13]. By 2018, only 2 out of 486 (0.41%) utilities in-
vestigated by the Smart Electric Power Alliance (SEPA) had implemented the V2G pilot
project [81]. By 2022, there were only around 100 V2G plot projects with different scales
and testing phases worldwide [14,82]. Therefore, to accelerate V2G implementation, the
charging and discharging efficiency need to be improved. On top of that, the V2G payment
procedure and contract terms need to be simplified. Batteries’ degradation rate and upfront
investment of V2G need to be quantified to allow EV owners to make informative decisions.

2.3. Battery Degradation and Charging Efficiency

The degradation of lithium-ion batteries occurs throughout their lives due to several
chemicals and mechanical processes that reduce the cyclable lithium and other active
materials [83]. Battery degradation depends on many factors, such as the charging and
discharging rates, depth of discharge (DOD), temperature, voltage, cycle number, and
storage stage of charge, which are complex to quantify [3,34,83]. The degradation of the
battery can be classified into two types: calendar aging and cycle aging. Calendar aging
occurs during storage, whereas cycle aging happens during charging and discharging.
Battery temperature and state of charge (SOC) are the key factors that influence calendar
aging, whereas cycle aging is affected by cycle number, charging rate, and DOD [84]. There-
fore, additional cycle numbers due to the V2G service will accelerate battery degradation.
It is important to quantify the degradation due to V2G service. Thus, adequate battery
monitoring systems are needed to monitor the batteries’ SOC and state of health (SOH)
during the charging and discharging process. Meng et al. [85] performed a Lithium-Ion
Energies 2023, 16, 146 7 of 26

battery monitoring and observability analysis with an extended equivalent circuit model.
As a result, the necessary observability conditions for battery capacity are clearly indicated,
which can be used to aid battery charging control design. Meng et al. [86] proposed a
Kalman filter and Gaussian process regression-based battery end-of-life (EOL) prediction
model. The simulation results show that the proposed model provides a better battery
EOL prediction than the particle filter, a popular method for battery EOL prediction. The
effectiveness of Gaussian process regression on battery SOH estimation is also shown
in [87]. Chaoui and Ibe-Ekeocha [88] proposed a recurrent neural networks(RNNs)-based
SOC and SOH estimation model which does not require battery modeling or knowledge of
batter parameters. The simulation results indicate that the recurrent neural networks-based
estimation model can make good SOC and SOH estimations based on measured voltage,
current, and ambient temperature. The measured data’s accuracy is vital for empirical
models’ performance.
More charging/discharging cycles occur in the V2G service than when there is no V2G
service; thus, battery degradation due to V2G services might be more severe than without
it [84]. Petit et al. [89] assessed the impact of V2G on two types of Lithium-ion batteries,
nickel cobalt aluminium oxides (NCA) and lithium ferro-phosphate (LFP). The simulation
results indicate that the effects of V2G on different batteries are different. For example, NCA
is more sensitive to cycle aging compared to LEP cells. In addition, high SOC can increase
battery capacity loss during storage. Pelletier et al. [90] also indicated that the calendar
aging process occurs faster for the battery stored at high SOC. The authors in [90–92] found
that battery overcharging and over-discharging degradation happen when the battery
operates outside its specified voltage range. Although battery degradation is inevitable
at the moment, it is possible to minimize the process by avoiding overcharging/over-
discharging and encouraging charging/discharging batteries at an optimal rate, under the
optimal temperature range, and storing the battery at an optimal SOC. Therefore, battery
degradation estimation and modeling need to be more accurate to support battery charging
and discharging control design.
Besides battery degradation concerns, charging and discharging efficiency is also one
of the primary concerns of the V2G program. Apostolaki-Iosifidou et al. [80] conducted
experimental measurements to determine power loss during EV charging and discharging.
The measurement results indicate that most power losses occur in the power electronics
used for AC-DC conversion [80]. Usually, the highest efficiency of the power electronics
occurs at the top region of their rated power [80]. In addition, the efficiency of the power
electronics is higher during charging than discharging [80]. This is due to the higher voltage
during charging than discharging at a given power. The higher charging voltage results in
a lower charging current, thus lowering internal resistance losses [93]. Therefore, the trade-
off between inevitable battery degradation and power loss during battery discharging and
the benefit of V2G needs to be farther investigated. More research on battery design and
management is required to minimize battery degradation during charging/discharging.

3. Artificial Intelligence-Based Forecasting Model

The relationship between forecasting, scheduling, and dynamic pricing is shown in
Figure 2. As can be seen, there are three critical components of EV charging/discharging
that are interlinked. These components are EV charging/discharging-related forecast-
ing, scheduling, and dynamic pricing. Accurate forecasting results provide insightful
information on system conditions to scheduling models, which provide optimal charg-
ing/discharging control and pricing signals which are then used to update the forecasting
model to enhance prediction accuracy. Thus, accurate forecasting results are crucial for
making optimal EV charging/discharging strategies. Commonly used artificial intelligence-
based forecasting models related to EV charging/discharging are discussed in this section.
OR PEER REVIEW 8 of 25
Energies 2023, 16, 146 8 of 26

Figure 2. Relationship between

Figure EV charging
2. Relationship andEV
between discharging
charging andscheduling, forecasting,
discharging scheduling, and dynamic
forecasting, and dy-
pricing. namic pricing.

3.1. Supervised Learning Methods

3.1. Supervised Learning Methods
EV charging-related forecasting results are generally used as inputs to the optimiza-
EV charging-related forecasting
tion models to enhanceresults are generally
EV charging scheduling used as inputs
performance. Many to different
the optimiza-
types of
machine learning methods can be used for both regression
tion models to enhance EV charging scheduling performance. Many different types of and classification problems.
Commonly used machine learning methods for forecasting tasks are linear regression (LR),
machine learning methods
decision treecan be[94],
(DT) used for both
random forest regression and vector
(RF) [95], support classification problems.
machine (SVM) [96], k-
Commonly used machine learning
nearest neighbor (KNN) methods forneural
[97], artificial forecasting
networkstasks
(ANNs)are [98],linear regression
convolutional neural
network(CNN) [99], deep convolutional neural network (DCNN)
(LR), decision tree (DT) [94], random forest (RF) [95], support vector machine (SVM) [96], [100], recurrent neural
k-nearest neighbornetwork
(KNN)(RNN) and its two gating mechanisms, gated recurrent units (GRUs) [101] and
[97], artificial neural networks (ANNs) [98], convolutional neu-
long short-term memory (LSTM) [102]. LR is suitable for linear models. Although LR
ral network(CNN)is[99],easydeep convolutional
to implement, neuralofnetwork
the performance (DCNN)
LR is largely affected[100], recurrent
by overfitting neu-
issues and
ral network (RNN)outliers
and its
[53].two gating mechanisms,
EV charging-related forecastinggated recurrent
tasks are nonlinear. units (GRUs)
Therefore, LR has[101]
barely
and long short-term been used to handle
memory (LSTM) EV[102].
charging-related problems.
LR is suitable for In contrast,
linear DT is able
models. to separate
Although LRa
complex decision into multiple straightforward decisions. However, a single DT cannot
is easy to implement, the performance of LR is largely affected by overfitting issues and
always guarantee a good prediction. Moreover, DT is often prone to overfitting issues [53].
outliers [53]. EV charging-related
RF is an ensemble of forecasting
DTs; thus, ittasks
is more are nonlinear.
accurate than DT Therefore, LR hasoverfitting
and can overcome barely
been used to handle EV charging-related problems. In contrast, DT is able to separate a
complex decision into multiple straightforward decisions. However, a single DT cannot
always guarantee a good prediction. Moreover, DT is often prone to overfitting issues [53].
RF is an ensemble of DTs; thus, it is more accurate than DT and can overcome overfitting
Energies 2023, 16, 146 9 of 26

by aggregating multiple DTs [95]. RF is proposed in [72] to forecast EV charging load. The
results of the simulations indicate the effectiveness of RF on EV charging load prediction.
Although SVM and KNN are suitable for solving regression problems, they are generally
used for classification tasks. Erol-Kantarci and Mouftah [54] used KNN to predict elec-
tricity prices to reduce PHEVs’ charging costs and CO2 emissions. The simulation results
show that the prediction-based charging scheme reduces the PHEVs’ operating costs and
CO2 emissions. SVM is used in [103] to forecast EV charging demand with a 3.69% mean
absolute percentage error (MAPE), which is lower than the 8.99% MAPE of the Monte
Carlo technique.

3.2. Gated Recurrent Units (GRUs)

Deep neural networks, such as RNN, GRU, LSTM, and CNN, are very efficient and
popular nowadays due to the availability of a large number of datasets and strong compu-
tational power. It has been proven that RNN with gating units (GRU and LSTM) performs
better than without them [104]. The Gated recurrent units (GRUs) gating mechanism of
recurrent neural networks was first introduced in [101]. GRUs is similar to LSTM, which
can also be used to solve the vanishing and exploding gradient problems. In addition,
the structure of GRU is simpler than LSTM. Moreover, GRU usually performs better than
LSTM in solving problems with a low sample size [105]. Therefore, GRU usually requires
less processing power and training time than LSTM [106]. As a result, GRU is also popular
for solving problems with long sequence samples, such as spot electricity price predic-
tion [57], load demand, and PV power production [107]. Lu et al. [57] proposed a GRU to
forecast spot electricity prices used to plan travel routes and charging schemes. The results
of simulations indicate that the route planning algorithm can effectively and accurately
provide optimal charging routes for EVs users. The prediction of the future trend of load
demand and PV power production is made by GRU in [107] to reduce the operating costs
of EV charging stations. Boulakhbar et al. [73] compared the EV charging load forecasting
performance of four popular deep learning models: ANN, GRU, LTSM, and RNN. The
simulation results indicate that GRU has the best performance, followed by RNN, LTSM,
and ANN. However, their results should only be used as guidance when selecting a model
for forecasting tasks because the generalization of these models was not tested in [73].

3.3. Long Short-Term Memory (LSTM)

Long short-term memory (LSTM) is another popular gating mechanism of RNN. It was
first introduced in [102]. GRU and LSTM can solve complex, long-sequence problems better
than recurrent neural networks (RNNs). Additionally, LSTM usually performs better for
solving high-complexity sequences than GRU [108]. Therefore, LSTM has been generally
used to solve problems with long and high-complexity sequence samples. Kriekinge et al.,
proposed an LSTM model to predict load demand [64] and one day ahead of EV charging
demand [67]. Schedulers then use the forecasted load demand to plan EVs’ charging and
discharging. Wang et al. [55], Zhang et al. [22], and Wan et al. [20] used LSTM to predict
future charging prices based on the historical charging price. LSTM networks and deep
LSTM have been applied to predict V2G capacity by Shange et al. [59] and Li et al. [60],
respectively. Nogay [61] compared the V2G capacity forecasting performance of LSTM
with the Nonlinear Autoregressive Neural Network (NAR). The simulation results indicate
that NAR performs slightly better than LSTM. However, both models can use historical
knowledge to correct learned aggregated available capacity deviations for V2G service.

3.4. Hybrid and Ensemble

Different forecasting mechanisms have various advantages and disadvantages. The
forecasting performances should be able to improve if the advantages of different mech-
anisms are combined to form hybrid models. Shipman et al. [62] proposed a hybrid of
CNN and LSTM-based models to predict the next 24 h of aggregated available SOC of
48 vehicles for V2G services based on the previous 24 h of data. The performance of the
Energies 2023, 16, 146 10 of 26

model proposed in [62] is further enhanced by continually refining the prediction model
based on the latest observed behavior [109]. Zhong and Xiong [66] proposed a load de-
mand and renewable energy output prediction model that combines CNN with a deep
belief network (DBN) to form the CNN-DBN prediction model. The forecasted results are
then used to schedule EVs’ charging to minimize the operating cost of the distribution
network. Sun et al. [68] proposed an artificial intelligence-based hybrid model that consists
of K-means clustering, KNN classification, and LSTM prediction to predict EV driver charg-
ing behavior. Zhang et al. [65] proposed a CNN-based ensemble model to forecast traffic
flow and EV load demand. The forecasted load demand can be used to make electricity
trading decisions.
Artificial intelligence-based forecasting models are usually supervised learning-based
models which use labeled data to train the models for predictions. The forecasting models
are used to predict electricity price, EV load demand, driving pattern, and availability of
state of charge, which can then provide EV charging and discharging schedules and pricing.
However, the performance of these charging scheduling models is highly dependent on
the accuracy of the prediction models. Uncertainty is an inherent property of forecasting
models. The stochastic and unpredictable EV charging, discharging, and driving behaviors
make forecasting harder. Therefore, forecasting results alone will not result in optimal
charging scheduling strategies. Other than the suggested hybrid and ensemble techniques,
online-based prediction models [110–113], that take the latest available information to
update the prediction models might reduce the forecasting uncertainty.
Moreover, probabilistic learning algorithms, such as Gaussian processes (GP) [87,114],
can provide prediction with an uncertainty interval that could also be applied to EV
charging/discharging-related studies.

4. Artificial Intelligence-Based Scheduling

Conventional numerical optimization methods, such as convex optimization and
linear programming, need to make some physically unknown assumptions [21]. The inves-
tigation results of [31] indicate that most researchers prefer artificial intelligence models
to probabilistic models because artificial intelligence is suitable for complex nonlinear
problems like forecasting EV charging demand and optimizing charging schedules. More-
over, artificial intelligence models such as neural networks and reinforcement learning
approaches can obtain nonlinear relationships from historical data and learn via interacting
with the environment, respectively. Therefore, they usually do not require expert knowl-
edge of a complex system to build a model from system developers due to the ability of
artificial intelligence to learn from the existing dataset. Furthermore, complex problems
like demand forecasting and charging scheduling optimization are stochastic, requiring
the models to be continuously updated. The online setup of artificial intelligence allows
models to learn the latest available dataset to improve model performance. The following
are some of the artificial intelligence-based scheduling algorithms.

4.1. Heuristic Algorithms

Heuristic algorithms such as Genetic Algorithm (GA) [115], Particle Swarm Optimiza-
tion (PSO) [116], Differential Evaluation (DE) [117], Artificial Bee Colony (ABC) [118], to
name only a few, are very popular for solving optimization problems due to their easy im-
plementation and fast training speed. Among all the heuristic algorithms, GA and PSO are
the most commonly used ones for EV charging and discharging scheduling problems. A GA
was employed by [119] to optimize the battery charging and discharging schedule. The
simulation results show that the proposed model can provide a demand response to adjust
the main transformer load. Farahani [120] proposed a PSO-based EV charging/discharging
scheduling to minimize the voltage unbalance factor of low-voltage distribution networks.
The simulation results indicate that the proposed model is able to reduce the voltage
unbalance factor. PSO-based EV charging scheduling model is also proposed in [121] to
reduce power loss and peak valley difference. Although GA and PSO-based optimization
Energies 2023, 16, 146 11 of 26

models mentioned in [119–121] perform well, the performance comparison of these models
is lacking in [119–121]. Dogan et al. [122] tested the performance of four different heuristic
algorithms on EV charging/discharging scheduling. The simulation results show that GA
can provide the lowest EV station operating cost and the most convenience for EV owners,
followed by PSO, DE, and ABC. Although heuristic algorithms are easy to implement and
can provide a fast solution, they do not always provide optimal solutions because they
might be trapped in local optimal and provide unstable performances [123]. In addition,
the parameters of these algorithms can largely impact their performance. Thus, the ranking
of these algorithms should only be used as guidance.

4.2. Fuzzy Logic

A fuzzy logic-based controller [124] has been implemented in [69,70] to manage and
schedule EVs’ charging/discharging intelligently. Authors in [69] used fuzzy logic-based
controllers to maintain a balance between system operators and EV owners. The simulation
results indicate that the fuzzy logic controller is able to improve the power system efficiency
and increase profit for aggregators. The control action of [70] is based on EVs’ initial and
target state of charge and available charging duration. The simulation results indicate that
the controller can consider EV owners’ and power providers’ preferences and provide
economic benefits for both EV owners and power providers.

4.3. Q-Learning and Deep Reinforcement Learning (DRL)

Many authors have applied reinforcement learning (RL) algorithms to schedule EVs’
charging and discharging. RL-based EV charging scheduling is usually present in the
Markov decision process (MDP) [22,23,125]. A finite MDP can be solved by dynamic
programming (DP), which assumes perfect knowledge of the environment and requires
high computation expense [126]. Rotering and Ilic [5] implemented a DP-based model to
optimally control PHEVs’ charging time and power rate based on the forecasted electricity
prices in deregulated electricity markets. The simulation results indicate that the model can
reduce daily electricity costs substantially. Iversen et al. [58] proposed an inhomogeneous
Markov chain model to predict the probability of EVs’ driving patterns. Then a stochastic
DP model is applied to the EVs’ driving pattern and electricity price to provide the op-
timal charging policy. The simulation results show that the optimal charging policy can
reduce daily energy costs. However, the environment of EV charging scheduling tasks
is not usually known. DP optimizes its policy based on a known environment, whereas
Q-learning, a model-free RL, optimizes its policy via interacting with its environment.
Watkins introduced Q-learning in 1989 [127], which is popular for scheduling problems
with a limited number of state and action spaces.
Dang et al. [56] proposed artificial neural networks (ANN) forecasting model to predict
the next day’s electricity price. The forecasted results are then used to train the Q-learning-
based charging scheduling model. Lee and Choi [29] proposed a Q-learning-based energy
management system that considers the ToU tariff, home appliances, and energy storage
system with charging/discharging functionality. The simulation results show that the
proposed model provides a 14% electricity bill reduction compared to MILP. The charg-
ing/discharging functionality of the energy storage systems is similar to that of EVs’ charg-
ing and discharging. Therefore, the results should be similar if the proposed model in [29]
is applied for EV charging and discharging. Mhaisen et al. [128] proposed a Q-learning
model to give EVs real-time charging and discharging schedules. The simulation results
indicate that the proposed model can achieve a cost saving of approximately 68% compared
to the uncontrolled one. However, one of the limitations of Q-learning algorithms is the
curse of dimensionality. Thus, deep reinforcement learning [23–28,30,51,55,71,125,129],
which combines deep neural networks with reinforcement learning, has been widely used
to overcome the issue of the curse of dimensionality.
Deep-Q networks (DQN) combine deep neural networks with Q-learning to make RL
more suitable for high-dimensional and complex environments [28,130]. DQN has been
Energies 2023, 16, 146 12 of 26

used in [20,21] to reduce charging costs and increase power grid reliability. Wang et al. [71]
proposed a DQN-based battery-swapping scheduling strategy to provide fast frequency
regulation services. The simulation results indicate that the proposed model can maximize
the battery swapping station operators’ revenue by autonomously scheduling the hourly
regulation capacity in real-time. Wang et al. [55] combined DQN and Dyna Q learning [126]
to allow the model to learn from both real-time experiences and simulated experiences,
which speeds up training. The proposed algorithm can reduce the long-term charging cost
while avoiding battery depletion during trips. Double-DQN was introduced in [131] to
reduce the overestimation of action values of DQN, and it is applied in [28] to control EVs’
charging/discharging based on the hourly electricity price. The simulation results indicate
that the Double-DQN algorithm reduces the correlation of the action values with the target
and provides better profit for EV owners than other state-of-the-art models such as RNN,
CDNN, and LSTM. However, the training time of DQN for problems with complex and
large state spaces is usually long due to its value iteration nature. Asynchronous Advantage
Actor Critic (A3C), which requires less training time than DQN [132], was introduced in
2016 by Google’s DeepMind [133]. Although A3C has not been applied to EV charging
and discharging-related studies in the reviewed literature, it is a good algorithm for a
complex problem with many states like EV charging and discharging and should be used
in the future.
In addition, both Q-learning and DQN are only suitable for discrete action spaces,
significantly reducing the effectiveness of the EV charging control algorithm due to the
limitation of the action space exploration. The deep deterministic policy gradient (DDPG)
is suitable for tasks with continuous state and action spaces [134]. It has been proposed
in [22–26] to maximize system operators’ financial benefit or reduce the charging cost
of EV owners. The simulation results in [24] show that the higher pricing frequency
can better reflect power system demand and supply situations and shift EV charging
load. The simulation results in [23] suggest that a DDPG-based EV charging strategy
can strictly guarantee voltage security by scheduling EV charging and discharging at
suitable time periods. Sun and Qiu [129] proposed DDPG to solve the voltage control
strategy problem to mitigate voltage fluctuation caused by stochastic EV and load demand.
Qiu et al. [30] proposed a prioritized deep deterministic policy gradient (PDDPG), a
combination of DDPG and prioritized experience replay, to optimize EV owners’ and
aggregators’ profits. PDDPG can solve the problem in multi-dimensional continuous
state and action spaces, which is preferable for practical application and requires lower
computational resources than DDPG with uniform sampling. The simulation results of [30]
show that PDDOG can provide 31%, 13%, and 5% higher profit than Q-learning, DQN,
and DDPG, respectively. Another off-policy algorithm, soft-actor-critic (SAC) [135], is used
by Yan et al. [125] to balance charging costs and drivers’ anxiety. The maximum entropy
framework of SAC makes the proposed model more sample efficient and robust [125]. Lee
and Choi [27] proposed a SAC model to provide dynamic EV charging and discharging
pricing to maximize the profits of multiple EVCSs.

5. Dynamic Pricing and Peer-to-Peer for EV Charging/Discharging

The performance of these EV charging scheduling algorithms is highly dependent
on the accuracy of the electricity pricing on reflection of the real-time power system
condition. However, the predicted electricity discussed in Section 3, and the electricity used
to schedule EV charging and discharging discussed in Section 4, are based on historical data
or assumed figures. A large amount of electricity is already traded several months or years
in advance via forwarding contracts and over-the-counter markets under the conventional
electricity market [136], which cannot reflect the actual demand and supply of electricity in
real-time [137]. Thus, dynamic pricing that can reflect real-time power system conditions is
needed to enhance the performance of EV charging and discharging scheduling.
Energies 2023, 16, 146 13 of 26

5.1. Time of Use (ToU)

Many utility companies in India, Sweden, the United Kingdom, Canada, and the
United States, etc., have used the ToU electricity tariff scheme to encourage users to shift
load demand from peak periods to off-peak periods [138]. ToU tariff provides different
charging pricing for peak, standard, and off-peak hours [54]. In the study [78], the estima-
tion results show that ToU is the most preferable pricing rule for consumers in Japan. The
impact of the ToU electricity tariff scheme on a smart grid with plug-in hybrid electric vehi-
cles (PHEVs) is investigated in [35]. The simulation results indicate that a properly designed
ToU tariff scheme can reduce the peak demand for smart grids with PHEV penetration.
Guo et al. [139] used the mixed-integer linear programming (MILP) model to determine
the impact of ToU on different charging strategies. The simulation results indicate that
smart charging can benefit from the ToU tariff compared with uncontrolled and constrained
charging. Xu et al. [36] proposed a zonal and ToU EV charging pricing model based on
demand price and power quality. The simulation results show that the proposed pricing
model can smooth the load curve and alleviate the impact of load fluctuation. Wolbertus
and Gerzon [140] investigated the impact of a time-based fee on the idle time of EVs at an
EV charging station (EVCS). The idea is to charge an additional fee for the EVs that have
completed the charging process and are still parked at the charging station. The simulation
results show that a time-based fee can reduce EVs’ idle time at an EVCS and improve the
efficiency of EVCSs.
Many EV charging strategies [17–19] are designed to adapt ToU tariffs to reduce
charging costs. Cao et al. [17] proposed an intelligent method to control EV charging loads
in response to the ToU tariff in a regulated market. The results of simulations show that
the optimized charging pattern has excellent benefits in reducing cost and flatting the load
curve. Lee et al. [18] proposed an EV charging pricing scheme to optimize the trade-off
between inexpensive ToU pricing and peak power draw. Wang et al. [19] proposed a
kernel-based estimator to estimate charging session parameters, such as stay duration and
energy consumption, to schedule EV charging according to ToU tariff and renewable energy
generation. The simulation results show that the average charging cost can be reduced
by 29.42% with the proposed scheduling framework. Most of the existing ToU pricing
strategies are determined based on the load information collected over many years [54],
except Gautam et al. [63], who used a multilayer perceptron (MLP) model to predict load
demand which is then used to determine ToU charging price in real-time. It is known that
the traditional ToU electricity tariff is not dynamically adapted to changes in operating
conditions [4]. Furthermore, authors in [35,141] suggest that the ToU tariff can economically
entice PEVs to charge during off-peak periods to reduce the load peak. However, a new,
more significant peak might be formed during the original off-peak hours due to the
simultaneous charging of EVs during the off-peak period [44].
It can be argued that many researchers have focused on designing a suitable EV
charging scheduling model that considers the ToU tariff. The scheduling models can
shift EV load demand to the off-peak period to minimize charging costs. However, ToU
tariffs are semi-dynamic because tariff, peak, and off-peak hours are determined based
on historical data, and the values are set to last for a while. Moreover, EVs’ charging and
discharging are more stochastic and challenging to predict than the load demand. Therefore,
a more dynamic pricing model that can reflect the conditions of power systems is needed to
schedule EVs’ charging more effectively, especially when the EV penetration is high.

5.2. Real-Time Pricing (RTP)

RTP with higher pricing updating frequency might overcome some of the challenges
encountered by the ToU tariff [44]. The price updating frequency is one of the main
distinguishing factors between RTP and ToU. RTP usually has high updating frequency than
ToU. It has been shown that hourly-based RTP would have saved EV owners significantly
over one without it [142]. However, the limitations of hourly-based RTP are that it cannot
reflect the instantaneous power system conditions. Instantaneous RTP that has been
Energies 2023, 16, 146 14 of 26

proposed in [44] is more effective than hourly-based RTP in that it can more closely reflect
the actual conditions of the power system and electricity market. Therefore, the price
updating frequency of RTP is a crucial factor when designing the RTP mechanism.

5.2.1. Application of RTP

RTP can benefit both EV owners and power system operators if it can accurately reflect
real-time power system conditions. On the one hand, EV owners can choose to charge
their EVs when the electricity price is low and discharge their EVs when it is high. On
the other hand, power system operators can use real-time power grid conditions, such
as load demand, electricity price, and renewable energy generation, to generate price
signals to change charging and discharging behaviors to benefit power grids. Although
RTP has been effective for many applications, only a few research applied RTP for EV
charging and discharging scheduling [27,37–51].These models have been applied to maxi-
mize the welfare of the whole charging system in [37], satisfy customers’ quality of service
and power-grid stability in [45], reduce charging waiting time in [46], maintain a stable
demand-supply curve in [39,43,44], minimize EV owners’ charging costs in [41,47,50],
and maximize EVCS operators’ revenue in [27,38,40,42,48,49,51]. However, in most stud-
ies [27,38–40,42,45,46,48,49,51], system operators determine the electricity price and control
the EV charging to optimize the power system and maximize system operators’ profit,
thus, giving the EV owners less freedom to trade stored electricity freely at their preferred
price. The EV owners’ preferences and willingness to charge are considered in [41,51].
Kim et al. [41] have proposed an improved reinforcement learning-based real-time pricing
algorithm that considers the customers’ load demand levels and the electricity cost. The
simulation results show that reinforcement-based dynamic pricing achieves higher and
long-term performance than the approach that only focuses on immediate system perfor-
mance [41]. Maestre et al. [51] have used DQN techniques to provide dynamic pricing
models to maximize revenue while considering fairness and equality.

5.2.2. RTP Classification

The majority of the existing EV charging scheduling RTP is based on either artificial
intelligence [27,38,40,41,43,48,51] or optimization algorithms [37,39,42,44–47,49]. Reinforce-
ment learning and GA are the most used to optimize EV charging pricing models. It has
been shown in [40] that artificial intelligence such as ANN, GA, and Population-Based
Incremental Learning (PBIL) can model and optimize dynamic pricing models better than
well-established demand models such as linear, exponential, and multinomial logit mod-
els. Mao et al. [38] proposed a GA-based V2G pricing model that considers the system
load condition, maximum power limit, and fair price rate to improve system operators’
profits and encourage more V2G program participants. RL-based optimization models
are used in [41,43,48,51]. For example, Lou et al. [48] proposed DP based charging pricing
optimization model, which uses the day-ahead wholesale electricity price, EV charging
demand, and renewable energy generation to calculate dynamic charging price. The
simulation results show that the DP model can increase profit by considering the entire
profile. Moghaddam et al. [43] proposed an online RL for RTP for EV charging stations
to reduce peak demand. The simulation results indicate that the online RL-based RTP
can effectively reduce peak demand and increase the profits of charging stations. In ad-
dition, an improved reinforcement learning and DQN-based real-time pricing algorithms
are proposed in [41,51], respectively, as discussed previously. Although reinforcement
learning-based RTP models discussed in this section can achieve goals such as maximizing
system operators’ profits and maintaining a stable demand-supply curve, many types of
RL, as mentioned in Section 4.3, can also be used to solve RTP optimization problems.
In addition, the dynamic pricing models in most studies [37,45–48,51] only allow EV
charging from power grids, which prevents EVs from feeding stored electricity back to
the grid. Therefore, these pricing models are not suitable for EV discharging. Moreover,
all the participants, such as EV owners and system operators, should co-decide on the
Energies 2023, 16, 146 15 of 26

charging and discharging price. Co-decided charging price can effectively improve the
EVCS operator’s income and efficiency of the EV charging system while reducing the EV
users’ charging costs [50]. A more transparent and open real-time pricing policy might
attract more EV owners to participate in the V2G program.

5.3. Peer-to-Pear (P2P)

P2P electricity trading allows sellers and buyers to co-decide on the charging and
discharging prices. Liu et al. [137] proposed a P2P EV power trading model based on
blockchain and a reverse auction mechanism to optimize social welfare and market ef-
ficiency. Blockchain technology ensures safer, more efficient, and transparent transac-
tions [137]. The reverse auction mechanism allows EV owners to adjust the bidding price
to improve the matching success rate [137]. Sather et al. [143] also proposed a linear
programming-based P2P electricity trading model for an industrial site. The results show
that P2P electricity trading can reduce electricity costs, distributed energy resource curtail-
ment rate, and grid power feed-in. Liu et al. [144] proposed a dynamic internal pricing
model based on PV energy supply and demand ratio (SDR) to operate the energy shar-
ing zone effectively. The results show that P2P trading with an internal pricing scheme
allows the PV prosumers to earn more than directly trading with the utility grid under the
feed-in tariff. The internal prices are co-decided by all the prosumers during the energy
sharing. Co-decided internal prices have both advantages and disadvantages. Conversely,
co-decided internal prices allow all the prosumers to decide the internal price. On the
other hand, the flexibility of prosumers’ load profile makes it hard to converge on the
internal price.
P2P trading enables individual EV owners to trade electricity at their preferred prices.
However, human beings such as EV owners and EVCS operators are unpredictable and
easily influenced by external factors such as prejudices, mood swings, and lack of capacity
to process large data [145]. Therefore, artificial intelligence machines are expected to be used
for decision-making in the fourth industrial revolution [145]. Using an intelligent machine
to make decisions can minimize human behavioral characteristics such as heuristics, risk
aversion, and diminishing sensitivity to losses and gains. Artificial intelligence-based pricing
policy can provide an accurate and fair electricity market for EV owners because these models
can process a large amount of real-time data and make decisions based on the data.

6. Discussion
The development of V2G can be classified into three phases [146]. The schema of V2G
development phases and corresponding EV charging/discharging techniques is shown in
Figure 1. First, the EV charging load only accounts for a small load demand proportion
of power grids in the preliminary phase. The development of V2G is currently in the first
phase. Therefore, the main charging control strategies presently used are uncontrolled
charging and controlled charging. Many V2G pilot projects are still in the planning and
testing phases. In addition, only a tiny portion of the existing commercial EVs is built
with V2G capability. The existing ToU tariff that is applied to other load demands can also
influence EV charging behaviors. However, the traditional ToU tariff will not be able to
handle the high penetration of EVs if stochastic EV charging loads are not considered in
the design of the ToU tariff.
In the second phase, the EV charging load will account for a higher proportion of
power grids’ load due to the high penetration of EVs. A large number of uncoordinated and
stochastic EV charging requests might burden power grids during peak demand periods,
which could endanger power grid stability. Therefore, aggregators must play an important
role in coordinating EVs to provide DSM services, such as enhancing renewable energy
utilization rate, peak shaving, valley filling, and alleviating congestion of distribution
networks [146]. The smart charging/discharging control strategies should be considered
in this phase. The main focus of the existing EV-related literature is on EV smart charg-
Energies 2023, 16, 146 16 of 26

ing/discharging. Table 4 summarizes the relevant literature on EV charging/discharging

forecasting, scheduling, and pricing strategies.
Most prediction models are supervised learning-based models that many researchers
apply to predict future electricity prices, EV load demand, renewable energy generation,
and the availability of battery SOC for V2G services. Optimization models then optimize
the forecasted information to provide EV charging/discharging decisions. A small forecast-
ing error might have a significant impact on decision-making. Therefore, the forecasting
models’ accuracy is crucial for optimal EV charging scheduling models. Although there are
many different types of artificial intelligence methods are very popular due to the avail-
ability of datasets and strong computational power, among all the deep learning methods,
long short-term memory (LSTM) and gate recurrent units (GRUs) are the most popular
methods for forecasting EV charging/discharging related tasks due to their ability to handle
nonlinear and long-term dependency. Although supervised learning models can provide
adequate forecasting accuracy, uncertainty is one of the inherent properties of forecasting
models. EVs’ charging and discharging patterns are stochastic and unpredictable, making
forecasting even harder. Hybrid and ensemble techniques can combine the advantages
of different forecasting methods or model configurations to enhance forecasting perfor-
mance. Other than hybrid and ensemble techniques, online-based forecasting [110–113],
and prediction confidence interval techniques [114,147] are two potential solutions to mini-
mize the impact of the inherent forecasting uncertainty. Online-based forecasting models
can use the latest available data to update the model, and prediction confidence interval
techniques allow decision-makers to plan by considering uncertainty ranges. Moreover,
individual forecasted information alone is insufficient for optimal EV charging/discharging
decisions, such as when to charge EVs and discharge power from EVs back to power grids.
For example, it is difficult to make optimal charging/discharging decisions based on the
forecasted EV charging demand alone. The charging/discharging decision should be made
based on all the important power system conditions, such as renewable generation, system
constraints, supply and demand curves, and charging/discharging dynamic pricing signals.
Therefore, optimization models are required to process the essential forecasted information
to make optimal decisions.
In recent years, reinforcement learning-based models have been commonly used to
schedule EVs’ charging/discharging. However, the conventional Q-learning requires a
lookup table which is infeasible for a problem like EV charging scheduling with many
states and actions [29,128]. Deep-Q networks (DQN), which combine deep neural networks
with Q-learning, have been implemented by many researchers to overcome the curse of
dimensionality faced by conventional Q-learning-based approaches. Therefore, DQN-
based models are more suitable for high-dimensional and complex environments [20,21,71].
Although DQN can handle high-dimensional and complex environments, the discrete
action spaces limit its action space exploration. In addition, overestimation of action values
of DQN for some applications has been encountered [131]. Double DQN can be used to
address the overestimation issue of DQN [28,131]. In addition, DDPG and SAC have been
used to provide continuous state and action spaces to further improve the performance of
DQN [22–27,125,129]. Although DQN, DDPG, and SAC can handle continuous complex
problems, they usually require a long time to train. It has been shown in [132,133] that
A3C requires less training time than DQN. Based on the literature that was reviewed,
it is found that A3C has not yet been used for EV charging/discharging-related tasks.
However, A3C, with multiple agents training in parallel, has many advantages, such as
fast convergence towards optimal policies and learning process stabilization. Therefore, it
should be considered in future studies.
Additionally, episodic memory and meta-reinforcement learning techniques have the
potential to improve training speed and meta-reinforcement learning. More information
about episodic memory and meta-reinforcement learning can be found in [148,149]. Fur-
thermore, reward functions and many hyperparameters of reinforcement learning models
need to be tuned. Adjustable reward function [150] and Bayesian Optimization [151,152]
Energies 2023, 16, 146 17 of 26

can be used for reward function and tuning hyperparameters, respectively. Moreover, RL
methods are evolving and improving regularly. Therefore, the latest advanced RL methods
should be applied to EV charging/discharging scheduling and dynamic pricing to see if
performance can be further enhanced. Different parameters of RL-based models, such as
reward functions, learning rate, exploration and exploitation trade-off, and model structures,
also need to be tested and compared to find optimal combinations to suit the tasks.
Although simulation results of most EV charging/discharging scheduling show good
performance, most of them were tested on historical or artificial data. The historical or
artificial electricity price cannot reflect real-time operation conditions such as EV charging
load demand, V2G discharging requests, and renewable energy generation. Therefore, a
suitable dynamic pricing model that accurately reflects power system operating conditions
is essential.
Finally, in the third phase, adequate technology and management will allow many EVs
to provide ancillary services to power grids via bidirectional power flow to achieve an ideal
state of smart grids [146]. By then, substantial communication resources will be required
for EV owners and power system operators to interact with each other. The event-triggered
communication mechanism can save bandwidth and signal-processing resources compared
with the periodic communication mechanism [153]. In addition, battery degradation
and charging/discharging losses should be minimized to accelerate the adoption of V2G
technology. Therefore, artificial intelligence-based battery design and management are also
important research fields. Although V2G has many benefits, it is challenging to encourage
EV owners with different driving and charging preferences to participate in the same V2G
program. Moreover, designing different V2G programs to satisfy all the participants is
not feasible. Thus, dynamic pricing can be used as a special form of demand response to
influence EV owners charging and discharging behaviors.
Dynamic pricing has been applied in many fields, such as airline ticket pricing, e-
commerce product pricing, and advertising bid pricing. The strategy is to increase the
selling price when demand is higher than supply and decrease the selling price when
supply is high than demand. More factors, such as power system constraints, renewable
energy generation, and EV charging/discharging preferences, need to be considered when
designing a dynamic pricing model for power systems with high EV penetration. Therefore,
a properly designed dynamic pricing strategy that can accurately reflect the conditions
of the power grid is needed to correctly guide EV charging and discharging to optimize
the energy use of power grids. However, there is a limited literature focusing on the
design of EV charging/discharging dynamic pricing schemes. Most researchers focus
on designing scheduling models that consider dynamic pricing to minimize charging
costs [27,38,40,42,48,49,51]. In addition, most of the existing dynamic pricing schemes
provided by power utility companies might not consider the effect of high EV penetration
and EV owners’ preferences [27,38–40,42,45,46,48,49,51]. Although the P2P electricity
transaction mechanism allows sellers and buyers to co-decide a tariff, the auction-based
mechanism does not always converge. Moreover, it is difficult for EV owners to make
pricing decisions without having the whole picture of power system conditions and the
electricity market. For example, EV owners might undervalue their stored battery power
during power system constraints and overvalue their power when renewable energy
production is high. Therefore, more research should be done on dynamic pricing schedule
design that can solve the above-mentioned issues.
Energies 2023, 16, 146 18 of 26

Table 4. Summary of the relevant literature on EV charging and discharging forecasting, scheduling, and pricing strategies.

Algorithm Applications Advantages Disadvantages Possible enhancement

• Load demand prediction. • Hybrid and ensemble techniques.

• Battery capacity • To learn complex relationships • Uncertainty is one of the inherent properties of • Online-based learning uses the
Supervised Learning availability prediction. from historical data. forecasting models. latest available data to update the
[20,22,54–57,59–68,72,73,103,107,109] • Renewable energy • Able to handle nonlinear and • Small forecasting errors might have a model frequently [110–113].
generation prediction. long-term dependency. significant impact on decision-making. • Use a confidence interval [147], i.e.,
• Electricity price prediction. a 95% confidence interval.

• Adjustable reward function during

training [150].
• Able to learn from experiences. • Many hyperparameters to tune.
• Schedule charging • Use Bayesian Optimization to
Reinforcement Learning • Maximize performance. • Require expert knowledge when designing
and discharging. optimize hyperparameters [151,152].
[5,20–30,51,55,56,58,71,125,128,129] • Able to solve complex and reward functions.
• Reduce charging cost. • Use episodic memory [148,149],
continuous problems with • Require large computation power to train
• Maximize the profits. and meta-reinforcement learning
function approximators. models for complex problems.
to make training faster and
more efficient.

• ToU tariffs are usually determined based on

• Update tariffs more frequently to
• Balance supply • Able to reflect power system historical data, which cannot reflect power
reflect the actual demand and
• and demand. conditions in real-time. systems’ actual conditions.
supply situation.
Dynamic Pricing [27,37–51] • Reduce charging waiting time. • Able to encourage EV owners • ToU might cause simultaneous charging at low
• Use forecasting and scheduling
• Increase the profit of to shift charging time or to tariff periods, which results in a new peak.
model to provide real-time pricing
system operators. discharge power back to the • RTP is mainly designed by system operators to
that reflects power system
• Reduce charging costs. grid when needed. achieve their goals without considering EV
conditions better.
owners’ preferences
Energies 2023, 16, 146 19 of 26

7. Conclusions
This paper reviews three crucial aspects of EV charging and discharging: forecasting,
scheduling, and dynamic pricing. The interconnected relationship between forecasting,
scheduling, and dynamic pricing is identified. Scheduling models’ performance mainly
depends on the accuracy of forecasting results and pricing strategies. On the other hand,
forecasting accuracy and scheduling performance largely influence the effectiveness of
dynamic pricing strategies in reflecting real-time power system conditions.
Most forecasting models mentioned in this paper are supervised, learning-based
models. Among them, LSTM and GRU are the most popular methods due to their ability to
handle nonlinear and long-term dependency. However, uncertainty is one of the inherent
properties of forecasting models. Therefore, the performance of forecasting models needs
to continue improving. Besides hybrid and ensemble techniques, using the latest available
data to update forecasting models and adding uncertainty intervals are other options to
assist decision-making.
Reinforcement learning-based optimization models that can take many variables as
state spaces have been applied by many researchers to make optimal EV charging and
discharging decisions based on the forecasted results, including charging and discharging
prices. DQN, DDPG, and SAC are some of the most popular reinforcement learning
models. Each of them has its advantages and disadvantages. DQN can overcome the curse
of dimensionality faced by conventional Q-learning. However, overestimation of action
values and long training time requirements are common issues faced by these methods.
Double-DQN and A3C can solve action value overestimation and reduce training time,
respectively. Improving reinforcement learning performance is also a key field of research.
Scheduling models cannot make effective charging/discharging decisions to optimize
power grids if the information they use to make decisions cannot accurately reflect the
power grid’s real-time conditions. Therefore, both forecasting results and dynamic pricing
signals that can reflect the real-time conditions of the power grid are important.
Many studies have explored the forecasting and scheduling aspects of EV charg-
ing. However, only a limited literature focused on EV discharging and dynamic pricing
design. Most of the existing dynamic pricing is designed by system operators without
considering EV owners’ preferences. In addition, not all the key factors related to EV charg-
ing/discharging are used to design dynamic pricing models. Dynamic pricing strategies are
very important for indirectly controlled charging/discharging. Moreover, dynamic pricing
can incentivize more EV owners to participate in V2G programs. Therefore, researchers
should pay more attention to designing dynamic pricing schemes that can reflect real-time
power system conditions and provide a balance between system operators and EV owners.
In addition to the technical aspect of EV charging/discharge scheduling and dynamic
pricing discussed in this paper, research on the social and economic aspects of EV charg-
ing/discharging and dynamic pricing is required. On the social aspect, the EV owners’
and system operators’ response to dynamic pricing needs to be surveyed and analyzed.
The opinions of all the involved parties can be used to enhance the dynamic pricing policy
design. On economic aspects, the feasibility and profitability of the dynamic pricing model
for the overall system, including individual EV owners, system operators, and power
systems, need to be investigated.

Author Contributions: Q.C. was responsible for conceptualization, formal analysis, and original
draft preparation; K.A.F. was responsible for the supervision, funding acquisition, review, and
editing of the final draft of the paper. All authors have read and agreed to the published version of
the manuscript.
Funding: This research was funded in part by the National Research Foundation (NRF) of South
Africa, grants number NRF Grant UID 118550.
Data Availability Statement: Not applicable.
Energies 2023, 16, 146 20 of 26

Conflicts of Interest: The authors declare no conflict of interest.

Abbreviations

A3C Asynchronous Advantage Actor Critic

ABC Artificial Bee Colony
AC-DC Alternating Current-Direct Current
ANN Artificial Neural Networks
CNN Convolutional Neural Network
CPP Critical Peak Pricing
DBN Deep Belief Network
DCNN Deep Convolutional Neural Network
DDPG Deep Deterministic Policy Gradient
DE Differential Evaluation
DOD Depth of Discharge
DP Dynamic Programming
DQN Deep-Q Network
DRL Deep Reinforcement Learning
DSM Demand-Side Management
DT Decision Tree
EOL End-of-Life
EV Electric Vehicle
EVCS EV Charging Station
GA Genetic Algorithm
GP Gaussian Processes
GRU Gated Recurrent Unit
KNN K-Nearest Neighbor
LFP Lithium Ferro-Phosphate
LR Linear Regression
LSTM Long Short-Term Memory
MAPE Mean Absolute Percentage Error
MDP Markov Decision Process
MILP Mixed-Integer Linear Programming
NCA Nickel Cobalt Aluminium Oxides
P2P Peer-to-Pear
PBIL Population-Based Incremental Learning
PDDPG Prioritized Deep Deterministic Policy Gradient
PHEV Plug-in Hybrid Electric Vehicle
PSO Particle Swarm Optimization
PTR Peak Time Rebates
RF Random Forest
RNN Recurrent Neural Network
RTP Real-Time Pricing
SAC Soft-Actor-Critic
SDR Supply and Demand Ratio
SOC State of Charge
SVM Support Vector Machine
ToU Time of Use
V2G Vehicle-to-Grid

References
1. Sun, X.; Li, Z.; Wang, X.; Li, C. Technology development of electric vehicles: A review. Energies 2020, 13, 90. [CrossRef]
2. IEA. Global EV Outlook 2022. IEA 2022. May 2022. Available online: https://www.iea.org/reports/global-ev-outlook-2022
(accessed on 5 October 2022).
3. Wang, Z.; Wang, S. Grid power peak shaving and valley filling using vehicle-to-grid systems. IEEE Trans. Power Deliv. 2013, 28,
1822–1829. [CrossRef]
4. Limmer, S. Dynamic pricing for electric vehicle charging—A literature review. Energies 2019, 12, 3574. [CrossRef]
Energies 2023, 16, 146 21 of 26

5. Rotering, N.; Ilic, M. Optimal charge control of plug-in hybrid electric vehicles in deregulated electricity markets. IEEE Trans.
Power Syst. 2011, 26, 1021–1029. [CrossRef]
6. Mohammad, A.; Zamora, R.; Lie, T.T. Integration of electric vehicles in the distribution network: A review of pv based electric
vehicle modelling. Energies 2020, 13, 4541. [CrossRef]
7. Kempton, W.; Tomic, J. Vehicle-to-grid power implementation: From stabilizing the grid to supporting large-scale renewable
energy. J. Power Sources 2005, 144, 280–294. [CrossRef]
8. Kempton, W.; Letendre, S.E. Electric vehicles as a new power source for electric utilities. Transp. Res. Part D Transp. Environ. 1996,
2, 157–175. [CrossRef]
9. Ravi, S.S.; Aziz, M. Utilization of electric vehicles for vehicle-to-grid services: Progress and perspectives. Energies 2022, 15, 589.
[CrossRef]
10. Scott, C.; Ahsan, M.; Albarbar, A. Machine learning based vehicle to grid strategy for improving the energy performance of public
buildings. Sustainability 2021, 13, 4003. [CrossRef]
11. Lund, H.; Kempton, W. Integration of renewable energy into the transport and electricity sectors through V2G. Energy Policy 2008,
36, 3578–3587. [CrossRef]
12. Al-Awami, A.T.; Sortomme, E. Coordinating vehicle-to-grid services with energy trading. IEEE Trans. Smart Grid 2012, 3, 453–462.
[CrossRef]
13. Sovacool, B.K.; Axsen, J.; Kempton, W. The future promise of vehicle-to-grid (V2G) integration: A sociotechnical review and
research agenda. Annu. Rev. Environ. Resour. 2017, 42, 377–406. [CrossRef]
14. V2G Hub Insights. Available online: https://www.v2g-hub.com/insights (accessed on 4 November 2022).
15. Kern, T.; Dossow, P.; von Roon, S. Integrating bidirectionally chargeable electric vehicles into the electricity markets. Energies
2020, 13, 5812. [CrossRef]
16. Sovacool, B.K.; Noel, L.; Axsen, J.; Kempton, W. The neglected social dimensions to a vehicle-to-grid (v2g) transition: A critical
and systematic review. Environ. Res. Lett. 2018, 13, 013001. [CrossRef]
17. Cao, Y.; Tang, S.; Li, C.; Zhang, P.; Tan, Y.; Zhang, Z.; Li, J. An optimized EV charging model considering TOU price and SOC
curve. IEEE Trans. Smart Grid 2012, 3, 388–393. [CrossRef]
18. Lee, Z.J.; Pang, J.Z.F.; Low, S.H. Pricing EV charging service with demand charge. Electr. Power Syst. Res. 2020, 189, 106694.
[CrossRef]
19. Wang, B.; Wang, Y.; Nazaripouya, H.; Qiu, C.; Chu, C.; Gadh, R. Predictive scheduling framework for electric vehicles with
uncertainties of user behaviors. IEEE Internet Things J. 2016, 4, 52–63. [CrossRef]
20. Wan, Z.; Li, H.; He, H.; Prokhorov, D. Model-free real-time EV charging scheduling based on deep reinforcement learning. IEEE
Trans. Smart Grid 2019, 10, 5246–5257. [CrossRef]
21. Lee, J.; Lee, E.; Kim, J. Electric vehicle charging and discharging algorithm based on reinforcement learning with data-driven
approach in dynamic pricing scheme. Energies 2020, 13, 1950. [CrossRef]
22. Zhang, F.; Yang, Q.; An, D. CDDPG: A deep-reinforcement-learning-based approach for electric vehicle charging control. IEEE
Internet Things J. 2021, 8, 3075–3087. [CrossRef]
23. Ding, T.; Zeng, Z.; Bai, J.; Qin, B.; Yang, Y.; Shahidehpour, M. Optimal electric vehicle charging strategy with markov decision
process and reinforcement learning technique. IEEE Trans. Ind. Appl. 2020, 56, 5811–5823. [CrossRef]
24. Liu, D.; Wang, W.; Wang, L.; Jia, H.; Shi, M. Deep deterministic policy gradient (DDPG) base reinforcement learning algorithm.
IEEE Access 2021, 9, 21556–21566. [CrossRef]
25. Li, S.; Hu, W.; Cao, D.; Dragičević, T.; Huang, Q.; Chen, Z.; Blaabjerg, F. Electric vehicle charging management based on deep
reinforcement learning. J. Mod. Power Syst. Clean Energy 2022, 10, 719–730. [CrossRef]
26. Wang, K.; Wang, H.; Yang, J.; Feng, J.; Li, Y.; Zhang, S.; Okoye, M.O. Electric vehicle clusters scheduling strategy considering
real-time electricity prices based on deep reinforcement learning. In Proceedings of the International Conference on New Energy
and Power Engineering (ICNEPE 2021), Sanya, China, 19–21 November 2021.
27. Lee, S.; Choi, D.H. Dynamic pricing and energy management for profit maximization in multiple smart electric vehicle charging
stations: A privacy-preserving deep reinforcement learning approach. Appl. Energy 2021, 304, 117754. [CrossRef]
28. Kiaee, F. Integration of electric vehicles in smart grid using deep reinforcement learning. In Proceedings of the 11th International
Conference on Information and Knowledge Technology (IKT), Tehran, Iran, 22–23 December 2020.
29. Lee, S.; Choi, D.H. Reinforcement learning-based energy management of smart home with rooftop solar photovoltaic system,
energy storage system, and home appliances. Sensors 2019, 19, 3937. [CrossRef]
30. Qiu, D.; Ye, Y.; Papadaskalopoulos, D.; Strbac, G. A deep reinforcement learning method for pricing electric vehicles with discrete
charging levels. IEEE Trans. Ind. Appl. 2020, 56, 5901–5912. [CrossRef]
31. Al-Ogaili, A.S.; Hashim, T.J.T.; Rahmat, N.A.; Ramasamy, A.K.; Marsadek, M.B.; Faisal, M.; Hannan, M.A. Review on scheduling,
clustering, and forecasting strategies for controlling electric vehicle charging: Challenges and recommendations. IEEE Access
2019, 7, 128353–128371. [CrossRef]
32. Amin, A.; Tareen, W.U.K.; Usman, M.; Ali, H.; Bari, I.; Horan, B.; Mekhilef, S.; Asif, M.; Ahmed, S.; Mahmood, A. A review of
optimal charging strategy for electric vehicles under dynamic pricing schemes in the distribution charging network. Sustainability
2020, 12, 10160. [CrossRef]
Energies 2023, 16, 146 22 of 26

33. Parsons, G.R.; Hidrue, M.K.; Kempton, W.; Gardner, M.P. Willingness to pay for vehicle-to-grid (V2G) electric vehicles and their
contract terms. Energy Econ. 2014, 42, 313–324. [CrossRef]
34. Shariff, S.M.; Iqbal, D.; Alam, M.S.; Ahmad, F. A state of the art review of electric vehicle to grid (V2G) technology. IOP Conf. Ser.
Mater. Sci. Eng. 2019, 561, 012103. [CrossRef]
35. Shao, S.; Zhang, T.; Pipattanasomporn, M.; Rahman, S. Impact of TOU rates on distribution load shapes in a smart grid with
PHEV penetration. In Proceedings of the IEEE PES T&D 2010, New Orleans, LA, USA, 19–22 April 2010.
36. Xu, X.; Niu, D.; Li, Y.; Sun, L. Optimal pricing strategy of electric vehicle charging station for promoting green behavior based on
time and space dimensions. J. Adv. Transp. 2020, 2020, 8890233. [CrossRef]
37. Lu, Z.; Qi, J.; Zhang, J.; He, L.; Zhao, H. modelling dynamic demand response for plug-in hybrid electric vehicles based on
real-time charging pricing. IET Gener. Transm. Distrib. 2017, 11, 228–235. [CrossRef]
38. Mao, T.; Lau, W.H.; Shum, C.; Chung, H.S.H.; Tsang, K.F.; Tse, N.C.F. A regulation policy of EV discharging price for demand
scheduling. IEEE Trans. Power Syst. 2018, 33, 1275–1288. [CrossRef]
39. Chekired, D.A.E.; Dhaou, S.; Khoukhi, L.; Mouftah, H.T. Dynamic pricing model for EV charging-discharging service based
on cloud computing scheduling. In Proceedings of the 13th International Wireless Communications and Mobile Computing
Conference, Valencia, Spain, 26–30 June 2017.
40. Shakya, S.; Kern, M.; Owusu, G.; Chin, C.M. Neural network demand models and evolutionary optimisers for dynamic pricing.
Knowl. Based Syst. 2012, 29, 44–53. [CrossRef]
41. Kim, B.G.; Zhang, Y.; van der Schaar, M.; Lee, J.W. Dynamic pricing and energy consumption scheduling with reinforcement
learning. IEEE Trans. Smart Grid 2016, 7, 2187–2198. [CrossRef]
42. Cedillo, M.H.; Sun, H.; Jiang, J.; Cao, Y. Dynamic pricing and control for EV charging stations with solar generation. Appl. Energy
2022, 326, 119920. [CrossRef]
43. Moghaddam, V.; Yazdani, A.; Wang, H.; Parlevliet, D.; Shahnia, F. An online reinforcement learning approach for dynamic pricing
of electric vehicle charging stations. IEEE Access 2020, 8, 130305–130313. [CrossRef]
44. Bitencourt, L.D.A.; Borba, B.S.M.C.; Maciel, R.S.; Fortes, M.Z.; Ferreira, V.H. Optimal EV charging and discharging control
considering dynamic pricing. In Proceedings of the 2017 IEEE Manchester PowerTech, Manchester, UK, 18–22 June 2017.
45. Ban, D.; Michailidis, G.; Devetsikiotis, M. Demand response control for PHEV charging stations by dynamic price adjustments. In
Proceedings of the 2012 IEEE PES Innovative Smart Grid Technologies (ISGT), Washington, DC, USA, 16–20 January 2012.
46. Xu, P.; Sun, X.; Wang, J.; Li, J.; Zheng, W.; Liu, H. Dynamic pricing at electric vehicle charging stations for waiting time
reduction. In Proceedings of the 4th International Conference on Communication and Information Processing, Qingdao, China,
2–4 November 2018.
47. Erdinc, O.; Paterakis, N.G.; Mendes, T.D.P.; Bakirtzis, A.G.; Catalão, J.P.S. Smart household operation considering bi-directional
EV and ESS utilization by real-time pricing-based DR. IEEE Trans. Smart Grid 2015, 6, 1281–1291. [CrossRef]
48. Luo, C.; Huang, Y.F.; Gupta, V. Dynamic pricing and energy management strategy for ev charging stations under uncertainties. In
Proceedings of the International Conference on Vehicle Technology and Intelligent Transport Systems—VEHITS, Rome, Italy,
23–24 April 2016.
49. Guo, Y.; Xiong, J.; Xu, S.; Su, W. Two-stage economic operation of microgrid-like electric vehicle parking deck. IEEE Trans. Smart
Grid 2016, 7, 1703–1712. [CrossRef]
50. Wang, B.; Hu, Y.; Xiao, Y.; Li, Y. An EV charging scheduling mechanism based on price negotiation. Future Internet 2018, 10, 40.
[CrossRef]
51. Maestre, R.; Duque, J.; Rubio, A.; Arevalo, J. Reinforcement learning for fair dynamic pricing. In Proceedings of the Intelligent
Systems Conference, London, UK, 6–7 September 2018.
52. Ahmed, M.; Zheng, Y.; Amine, A.; Fathiannasab, H.; Chen, Z. The role of artificial intelligence in the mass adoption of electric
vehicles. Joule 2021, 5, 2296–2322. [CrossRef]
53. Shahriar, S.; Al-Ali, A.R.; Osman, A.H.; Dahou, S.; Nijim, M. Machine learning approaches for EV charging behavior: A review.
IEEE Access 2020, 8, 168980–168993. [CrossRef]
54. Erol-Kantarci, M.; Mouftah, H.T. Prediction-based charging of PHEVs from the smart grid with dynamic pricing. In Proceedings
of the IEEE Local Computer Network Conference, Denver, CO, USA, 10–14 October 2010.
55. Wang, F.; Gao, J.; Li, M.; Zhao, L. Autonomous PEV charging scheduling using dyna-q reinforcement learning. IEEE Trans. Veh.
Technol. 2020, 69, 12609–12620. [CrossRef]
56. Dang, Q.; Wu, D.; Boulet, B. EV charging management with ANN-based electricity price forecasting. In Proceedings of the 2020
IEEE Transportation Electrification Conference & Expo (ITEC), Chicago, IL, USA, 23–26 June 2020.
57. Lu, Y.; Gu, J.; Xie, D.; Li, Y. Integrated route planning algorithm based on spot price and classified travel objectives for ev users.
IEEE Access 2019, 7, 122238–122250. [CrossRef]
58. Iversen, E.B.; Morales, J.M.; Madsen, H. Optimal charging of an electric vehicle using a markov decision process. Appl. Energy
2014, 123, 1–12. [CrossRef]
59. Shang, Y.; Li, Z.; Shao, Z.; Jian, L. Distributed V2G dispatching via LSTM network within cloud-edge collaboration framework. In
Proceedings of the 2021 IEEE/IAS Industrial and Commercial Power System Asia (I&CPS Asia), Chengdu, China, 18–21 July 2021.
Energies 2023, 16, 146 23 of 26

60. Li, S.; Gu, C.; Li, J.; Wang, H.; Yang, Q. Boosting grid efficiency and resiliency by releasing v2g potentiality through a novel rolling
prediction-decision framework and deep-lstm algorithm. IEEE Syst. J. 2021, 15, 2562–2570. [CrossRef]
61. Nogay, H.S. Estimating the aggregated available capacity for vehicle to grid services using deep learning and Nonlinear
Autoregressive Neural Network. Sustain. Energy Grids Netw. 2022, 29, 100590. [CrossRef]
62. Shipman, R.; Roberts, R.; Waldron, J.; Naylor, S.; Pinchin, J.; Rodrigues, L.; Gillotta, M. We got the power: Predicting available
capacity for vehicle-to-grid services using a deep recurrent neural network. Energy 2021, 221, 119813. [CrossRef]
63. Gautam, A.; Verma, A.K.; Srivastava, M. A novel algorithm for scheduling of electric vehicle using adaptive load forecasting
with vehicle-to-grid integration. In Proceedings of the 2019 8th International Conference on Power Systems (ICPS), Jaipur, India,
20–22 December 2019.
64. Kriekinge, G.V.; Cauwer, C.D.; Sapountzoglou, N.; Coosemans, T.; Messagie, M. Peak shaving and cost minimization using model
predictive control for uni- and bi-directional charging of electric vehicles. Energy Rep. 2021, 7, 8760–8771. [CrossRef]
65. Zhang, X.; Chan, K.W.; Li, H.; Wang, H.; Qiu, J.; Wang, G. Deep-Learning-Based Probabilistic Forecasting of Electric Vehicle
Charging Load With a Novel Queuing Model. IEEE Trans. Cybern. 2021, 51, 3157–3170. [CrossRef]
66. Zhong, J.; Xiong, X. An orderly EV charging scheduling method based on deep learning in cloud-edge collaborative environment.
Adv. Civ. Eng. 2021, 2021, 6690610. [CrossRef]
67. Kriekinge, G.V.; Cauwer, C.D.; Sapountzoglou, N.; Coosemans, T.; Messagie, M. Day-ahead forecast of electric vehicle charging
demand with deep neural networks. World Electr. Veh. J. 2021, 12, 178. [CrossRef]
68. Sun, D.; Ou, Q.; Yao, X.; Gao, S.; Wang, Z.; Ma, W.; Li, W. Integrated human-machine intelligence for EV charging prediction in
5G smart grid. EURASIP J. Wirel. Commun. Netw. 2020, 2020, 139. [CrossRef]
69. Wang, K.; Gu, L.; He, X.; Guo, S.; Sun, Y.; Vinel, A.; Shen, J. Distributed energy management for vehicle-to-grid networks. IEEE
Netw. 2017, 31, 22–28. [CrossRef]
70. Patil, V.; Sindhu, M.R. An intelligent control strategy for vehicle-to-grid and grid-to-vehicle energy transfer. IOP Conf. Ser. Mater.
Sci. Eng. 2019, 561, 012123. [CrossRef]
71. Wang, X.; Wang, J.; Liu, J. Vehicle to grid frequency regulation capacity optimal scheduling for battery swapping station using
deep q-network. IEEE Trans. Ind. Inform. 2021, 17, 1342–1351. [CrossRef]
72. Lu, Y.; Li, Y.; Xie, D.; Wei, E.; Bao, X.; Chen, H.; Zhong, X. The application of improved random forest algorithm on the prediction
of electric vehicle charging load. Energies 2018, 11, 3207. [CrossRef]
73. Boulakhbar, M.; Farag, M.; Benabdelaziz, K.; Kousksou, T.; Zazi, M. A deep learning approach for prediction of electrical vehicle
charging stations power demand in regulated electricity markets: The case of Morocco. Clean. Energy Syst. 2022, 3, 100039.
[CrossRef]
74. Lee, E.H.P.; Lukszo, Z.; Herder, P. Conceptualization of vehicle-to-grid contract types and their formalization in agent-based
models. Complexity 2018, 2018, 3569129.
75. Gao, Y.; Chen, Y.; Wang, C.Y.; Liu, K.J.R. Optimal contract design for ancillary services in vehicle-to-grid networks. In Pro-
ceedings of the 2012 IEEE Third International Conference on Smart Grid Communications (SmartGridComm), Tainan, Taiwan,
5–8 November 2012.
76. Wahyuda, S.B. Dynamic pricing in electricity: Research potential in Indonesia. Procedia Manuf. 2015, 4, 300–306. [CrossRef]
77. Dutschke, E.; Paetz, A.G. Dynamic electricity pricing—Which programs do consumers prefer? Energy Policy 2013, 59, 226–234.
[CrossRef]
78. Yoshida, Y.; Tanaka, K.; Managi, S. Which dynamic pricing rule is most preferred by consumers?—Application of choice
experiment. Econ. Struct. 2017, 6, 4. [CrossRef]
79. Latinopoulos, C.; Sivakumar, A.; Polak, J.W. Response of electric vehicle drivers to dynamic pricing of parking and charging
services: Risky choice in early reservations. Transp. Res. Part C 2017, 80, 175–189. [CrossRef]
80. Apostolaki-Iosifidou, E.; Codani, P.; Kempton, W. Measurement of power loss during electric vehicle charging and discharing.
Energy 2017, 127, 730–742. [CrossRef]
81. Myers, E.H.; Surampudy, M.; Saxena, A. Utilities and Electric Vehicles: Evolving to Unlock Grid Value; Smart Electric Power Alliance:
Washington, DC, USA, 2018.
82. Väre, V. The Vehicle-to-Grid Boom Is around the Corner. Virta Global. 7 December 2020. Available online: https://www.virta.
global/blog/vehicle-to-grid-boom-is-around-the-corner (accessed on 19 October 2022).
83. Barre, A.; Deguilhem, B.; Grolleau, S.; Gerard, M.; Suard, F.; Riu, D. A review on lithium-ion battery ageing mechanisms and
estimations for automotive applications. J. Power Source 2013, 241, 680–689. [CrossRef]
84. Guo, J.; Yang, J.; Lin, Z.; Serrano, C.; Cortes, A.M. Impact analysis of V2G services on EV battery degradation—A review. In
Proceedings of the 2019 IEEE Milan PowerTech, Milan, Italy, 23–27 June 2019.
85. Meng, J.; Boukhnifer, M.; Diallo, D. Lithium-Ion battery monitoring and observability analysis with extended equivalent circuit
model. In Proceedings of the 28th Mediterranean Conference on Control and Automation (MED), Saint-Raphaël, France,
15–18 September 2020.
86. Meng, J.; Yue, M.; Diallo, D. A degradation empirical-model-free battery end-of-life prediction framework based on Gaussian
process regression and Kalman filter. IEEE Trans. Transp. Electrif. 2022, 1–11. [CrossRef]
Energies 2023, 16, 146 24 of 26

87. Wang, J.; Deng, Z.; Yu, T.; Yoshida, A.; Xu, L.; Guan, G.; Abudula, A. State of health estimation based on modified Gaussian
process regression for lithium-ion batteries. J. Energy Storage 2022, 51, 104512. [CrossRef]
88. Chaoui, H.; Ibe-Ekeocha, C.C. State of charge and state of health estimation for Lithium batteries using recurrent neural networks.
IEEE Trans. Veh. Technol. 2017, 66, 8773–8783. [CrossRef]
89. Petit, M.; Prada, E.; Sauvant-Moynot, V. Development of an empirical aging model for Li-ion batteries and application to assess
the impact of Vehicle-to-Grid strategies on battery lifetime. Appl. Energy 2016, 172, 398–407. [CrossRef]
90. Pelletier, S.; Jabali, O.; Laporte, G.; Veneroni, M. Battery degradation and behaviour for electric vehicles: Review and numerical
analyses of several models. Transp. Res. Part B Methodol. 2017, 103, 158–187. [CrossRef]
91. Prochazka, P.; Cervinka, D.; Martis, J.; Cipin, R.; Vorel, P. Li-Ion battery deep discharge degradation. ECS Trans. 2016, 74, 31–36.
[CrossRef]
92. Guo, R.; Lu, L.; Ouyang, M.; Feng, X. Mechanism of the entire overdischarge process and overdischarge-induced internal short
circuit in lithium-ion batteries. Sci. Rep. 2016, 6, 30248. [CrossRef] [PubMed]
93. Krieger, E.M.; Arnold, C.B. Effects of undercharge and internal loss on the rate dependence of battery charge storage efficiency.
J. Power Source 2012, 210, 286–291. [CrossRef]
94. Quinlan, J.R. Induction of decision trees. Mach. Learn. 1986, 1, 81–106. [CrossRef]
95. Breiman, L. Random Forests. Mach. Learn. 2001, 45, 5–32. [CrossRef]
96. Cortes, C.; Vapnik, V. Support-vector networks. Machine Leaming 1995, 20, 273–297. [CrossRef]
97. Cover, T.; Hart, P.E. Nearest neighbor pattern classification. IEEE Trans. Inf. Theory 1967, 13, 21–27. [CrossRef]
98. Jain, A.K.; Mao, J.; Mohiuddin, K.M. Artificial neural networks: A tutorial. Computer 1996, 29, 31–44. [CrossRef]
99. O’Shea, K.; Nash, R. An introduction to convolutional neural networks. arXiv 2015, arXiv:1511.08458.
100. Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet classification with deep convolutional neural networks. Adv. Neural Inf.
Process. Syst. 2017, 60, 84–90. [CrossRef]
101. Cho, K.; Merrienboer, B.v.; Bahdanau, D.; Bengio, Y. On the properties of neural machine translation: Encoder-decoder approaches.
arXiv 2014, 1–9, arXiv:1409.1259.
102. Hochreiter, S.; Schmidhuber, J. Long Short-Term Memory. Neural Comput. 1997, 9, 1735–1780. [CrossRef] [PubMed]
103. Xydas, E.S.; Marmaras, C.E.; Cipcigan, L.M.; Hassan, A.S.; Jenkins, N. Forecasting electric vehicle charging demand using Support
Vector Machines. In Proceedings of the 48th International Universities’ Power Engineering Conference (UPEC), Dublin, Ireland,
2–5 September 2013.
104. Chung, J.; Gulcehre, C.; Cho, K.; Bengio, Y. Empirical evaluation of gated recurrent neural networks on sequence modelling. arXiv
2014, arXiv:1412.3555.
105. Gruber, N.; Jockisch, A. Are GRU cells more specific and LSTM cells more sensitive in motive classification of text? Front. Artif.
Intell. 2020, 3, 40. [CrossRef]
106. Mateus, B.C.; Mendes, M.; Farinha, J.T.; Assis, R.; Cardoso, A.M. Comparing LSTM and GRU models to predict the condition of a
pulp paper press. Energies 2021, 14, 6958. [CrossRef]
107. Jin, H.; Lee, S.; Nengroo, S.H.; Har, D. Development of charging/discharging scheduling algorithm for economical and energy-
efficient operation of multi-EV charging station. Appl. Sci. 2022, 12, 4786. [CrossRef]
108. Cahuantzi, R.; Chen, X.; Güttel, S. A comparison of LSTM and GRU networks for learning symbolic sequences. arXiv 2021,
arXiv:2107.02248.
109. Shipman, R.; Roberts, R.; Waldron, J.; Rimmer, C.; Rodrigues, L.; Gillotta, M. Online machine learning of available capacity for
vehicle-to-grid services during the coronavirus pandemic. Energies 2021, 14, 7176. [CrossRef]
110. Raju, M.P.; Laxmi, A.J. IOT based online load forecasting using machine learning algorithms. Procedia Comput. Sci. 2020, 171,
551–560. [CrossRef]
111. Laouafi, A.; Mordjaoui, M.; Haddad, S.; Boukelia, T.E.; Ganouche, A. Online electricity demand forecasting based on an effective
forecast combination methodology. Electr. Power Syst. Res. 2017, 148, 35–47. [CrossRef]
112. Krannichfeldt, L.V.; Wang, Y.; Hug, G. Online ensemble learning for load forecasting. IEEE Trans. Power Syst. 2021, 36, 545–548.
[CrossRef]
113. Fekri, M.N.; Patel, H.; Grolinger, K.; Sharma, V. Deep learning for load forecasting with smart meter data: Online adaptive
recurrent neural network. Appl. Energy 2021, 282, 116177. [CrossRef]
114. Zhang, Y.; Li, M.; Dong, Z.Y.; Meng, K. Probabilistic anomaly detection approach for data-driven wind turbine condition
monitoring. CSEE J. Power Energy Syst. 2019, 5, 149–158. [CrossRef]
115. Holland, J.H. Genetic algorithms. Sci. Am. 1992, 267, 66–73. [CrossRef]
116. Kennedy, J.; Eberhart, R.C. Particle swarm optimization. In Proceedings of the ICNN’95-International Conference on Neural
Networks, Perth, WA, Australia, 27 November–1 December 1995.
117. Storn, R.; Price, K. Differential evolution—A simple and efficient heuristic for global optimization over continuous spaces. J. Glob.
Optim. 1997, 11, 341–359. [CrossRef]
118. Karaboga, D.; Basturk, B. Artificial bee colony (ABC) optimization algorithm for solving constrained optimization problems. In
Proceedings of the Foundations of Fuzzy Logic and Soft Computing, Cancun, Mexico, 18–21 June 2007.
Energies 2023, 16, 146 25 of 26

119. Ke, B.R.; Lin, Y.H.; Chen, H.Z.; Fang, S.C. Battery charging and discharging scheduling with demand response for an electric bus
public transportation system. Sustain. Energy Technol. Assess. 2020, 40, 100741. [CrossRef]
120. Farahani, H.F. Improving voltage unbalance of low-voltage distribution networks using plug-in electric vehicles. J. Clean. Prod.
2017, 148, 336–346. [CrossRef]
121. Gong, L.; Cao, W.; Liu, K.; Zhao, J.; Li, X. Spatial and temporal optimization strategy for plug-in electric vehicle charging to
mitigate impacts on distribution network. Energies 2018, 11, 1373. [CrossRef]
122. Dogan, A.; Bahceci, S.; Daldaban, F.; Alci, M. Optimization of charge/discharge coordination to satisfy network requirements
using heuristic algorithms in vehicle-to-grid concept. Adv. Electr. Comput. Eng. 2018, 18, 121–130. [CrossRef]
123. Qiu, H.; Liu, Y. Novel heuristic algorithm for large-scale complex optimization. Procedia Comput. Sci. 2016, 80, 744–751. [CrossRef]
124. Zadeh, L. Fuzzy sets. Inf. Control 1965, 8, 338–353. [CrossRef]
125. Yan, L.; Chen, X.; Zhou, J.; Chen, Y.; Wen, J. Deep reinforcement learning for continuous electric vehicles charging control with
dynamic user behaviors. IEEE Trans. Smart Grid 2021, 12, 5124–5134. [CrossRef]
126. Sutton, R.S.; Barto, A.G. Reinforcement Learning: An Introduction; The MIT Press: London, UK, 2018.
127. Watkins, C.J.C.H.; Dayan, P. Q-learning. Mach. Learn. 1992, 8, 279–292. [CrossRef]
128. Mhaisen, N.; Fetais, N.; Massoud, A. Real-time scheduling for electric vehicles charging/discharging using reinforcement learning.
In Proceedings of the 2020 IEEE International Conference on Informatics, IoT, and Enabling Technologies (ICIoT), Doha, Qatar,
2–5 February 2020.
129. Sun, X.; Qiu, J. A customized voltage control strategy for electric vehicles in distribution networks with reinforcement learning
method. IEEE Trans. Ind. Inform. 2021, 17, 6852–6863. [CrossRef]
130. Mnih, V.; Kavukcuoglu, K.; Silver, D.; Graves, A.; Antonoglou, I.; Wierstra, D.; Riedmiller, M. Playing Atari with deep reinforce-
ment learning. arXiv 2013, arXiv:1312.5602.
131. Hasselt, H.v.; Guez, A.; Silver, D. Deep reinforcement learning with double Q-learning. arXiv 2015, arXiv:1509.06461. [CrossRef]
132. Zeng, P.; Liu, A.; Zhu, C.; Wang, T.; Zhang, S. Trust-based multi-agent imitation learning for green edge computing in smart cities.
IEEE Trans. Green Commun. Netw. 2022, 6, 1635–1648. [CrossRef]
133. Mnih, V.; Badia, A.P.; Mirza, M.; Graves, A.; Harley, T.; Lillicrap, T.P.; Silver, D.; Kavukcuoglu, K. Asynchronous methods for deep
reinforcement learning. arXiv 2016, arXiv:1602.01783.
134. Lillicrap, T.P.; Hunt, J.J.; Pritzel, A.; Heess, N.; Erez, T.; Tassa, Y.; Silver, D.; Wierstra, D. Continuous control with deep
reinforcement learning. In Proceedings of the International Conference on Learning Representations, San Juan, Puerto Rico,
2–4 May 2016.
135. Haarnoja, T.; Zhou, A.; Abbeel, P.; Levine, S. Soft Actor-Critic: Off-policy maximum entropy deep reinforcement learning with a
stochastic actor. In Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden, 10–15 July 2018.
136. Meeus, L.; Purchala, K.; Belmans, R. Development of the internal electricity market in Europe. Electr. J. 2005, 18, 25–35. [CrossRef]
137. Liu, H.; Zhang, Y.; Zheng, S.; Li, Y. Electric vehicle power trading mechanism based on blockchain and smart contract in V2G
network. IEEE Access 2019, 7, 160546–160558. [CrossRef]
138. Nhede, N. Smart Energy International, Synergy BV. 13 April 2019. Available online: https://www.smart-energy.com/industry-
sectors/electric-vehicles/as-energy-gets-smarter-time-of-use-tariffs-spread-globally/ (accessed on 6 October 2022).
139. Guo, Y.; Liu, X.; Yan, Y.; Zhang, N.; Wencong, S. Economic analysis of plug-in electric vehicle parking deck with dynamic pricing.
In Proceedings of the2014 IEEE PES General Meeting|Conference & Exposition, National Harbor, MD, USA, 27–31 July 2014.
140. Wolbertus, R.; Gerzon, B. Improving electric vehicle charging station efficiency through pricing. J. Adv. Transp. 2018, 2018, 4831951.
[CrossRef]
141. Geng, B.; Mills, J.K.; Sun, D. Two-stage charging strategy for plug-in electric vehicles at the residential transformer level. IEEE
Trans. Smart Grid 2013, 4, 1442–1452. [CrossRef]
142. Zethmayr, J.; Kolata, D. Charge for less: An analysis of hourly electricity pricing for electric vehicles. Would Electr. Veh. J. 2019, 10, 6.
[CrossRef]
143. Sather, G.; Granado, P.C.D.; Zaferanlouei, S. Peer-to-peer electricity trading in an industrial site: Value of buildings flexibility on
peak load reduction. Energy Build. 2021, 236, 110737. [CrossRef]
144. Liu, N.; Yu, X.; Wang, C.; Li, C.; Ma, L.; Lei, J. Energy-sharing model with price-based demand response for microgrids of
peer-to-peer prosumers. IEEE Trans. Power Syst. 2017, 32, 3569–3583. [CrossRef]
145. Marwala, T.; Hurwitz, E. Artificial Intelligence and Economic Theory: Skynet in the Market; Springer: Cham, Switzerland, 2017.
146. Shi, L.; LV, T.; Wang, Y. Vehicle-to-grid service development logic and management formulation. J. Mod. Power Syst. Clean Energy
2019, 7, 935–947. [CrossRef]
147. Hazra, A. Using the confidence interval confidently. J. Thorac. Dis. 2017, 9, 4125–4130. [CrossRef]
148. Gershman, S.J.; Daw, N.D. Reinforcement learning and episodic memory in humans and animals: An integrative framework.
Annu. Rev. Psychol. 2017, 68, 101–128. [CrossRef]
149. Botvinick, M.; Ritter, S.; Wang, J.X.; Kurth-Nelson, Z.; Blundell, C.; Hassabis, D. Reinforcement learning, fast and slow. Trends
Cogn. Sci. 2019, 23, 408–422. [CrossRef]
150. Hu, Z.; Wan, K.; Gao, X.; Zhai, Y. A dynamic adjusting reward function method for deep reinforcement learning with adjustable
parameters. Math. Probl. Eng. 2019, 2019, 7619483. [CrossRef]
Energies 2023, 16, 146 26 of 26

151. Wu, J.; Chen, X.Y.; Zhang, H.; Xiong, L.D.; Lei, H.; Deng, S.H. Hyperparameter optimization for machine learning models based
on bayesian optimization. J. Electron. Sci. Technol. 2019, 17, 26–40.
152. Feurer, M.; Hutter, F. Hyperparameter Optimization. In Automated Machine Learning; Springer: Cham, Switzerland, 2019; pp. 3–33.
153. Zhang, N.; Sun, Q.; Yang, L.; Li, Y. Event-triggered distributed hybrid control scheme for the integrated energy system. IEEE
Trans. Ind. Inform. 2022, 18, 835–846. [CrossRef]

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.