KP Forecast Models
KP Forecast Models
KP Forecast Models
net/publication/253300826
Kp forecast models
CITATIONS READS
127 354
11 authors, including:
D. G. Sibeck
NASA
589 PUBLICATIONS 16,703 CITATIONS
SEE PROFILE
All content following this page was uploaded by S. Wing on 21 October 2016.
Kp forecast models
S. Wing,1 J. R. Johnson,2 J. Jen,1 C.-I. Meng,1 D. G. Sibeck,3 K. Bechtold,1 J. Freeman,5
K. Costello,6 M. Balikhin,4 and K. Takahashi1
Received 25 March 2004; revised 21 December 2004; accepted 27 January 2005; published 9 April 2005.
[1] Magnetically active times, e.g., Kp > 5, are notoriously difficult to predict, precisely
the times when such predictions are crucial to the space weather users. Taking advantage
of the routinely available solar wind measurements at Langrangian point (L1) and nowcast
Kps, Kp forecast models based on neural networks were developed with the focus on
improving the forecast for active times. To satisfy different needs and operational
constraints, three models were developed: (1) a model that inputs nowcast Kp and solar
wind parameters and predicts Kp 1 hour ahead; (2) a model with the same input as
model 1 and predicts Kp 4 hour ahead; and (3) a model that inputs only solar wind
parameters and predicts Kp 1 hour ahead (the exact prediction lead time depends on the
solar wind speed and the location of the solar wind monitor). Extensive evaluations of
these models and other major operational Kp forecast models show that while the new
models can predict Kps more accurately for all activities, the most dramatic improvements
occur for moderate and active times. Information dynamics analysis of Kp suggests
that geospace is more dominated by internal dynamics near solar minimum than near solar
maximum, when it is more directly driven by external inputs, namely solar wind and
interplanetary magnetic field (IMF).
Citation: Wing, S., J. R. Johnson, J. Jen, C.-I. Meng, D. G. Sibeck, K. Bechtold, J. Freeman, K. Costello, M. Balikhin, and
K. Takahashi (2005), Kp forecast models, J. Geophys. Res., 110, A04203, doi:10.1029/2004JA010500.
A04203 1 of 14
A04203 WING ET AL.: Kp FORECAST MODELS A04203
Kp as an input parameter. For example, the Tsyganenko developed such a model based on an artificial neural
[1989] magnetic field model, atmospheric density models network (NN) algorithm. This model is now routinely
[e.g., Hedin, 1987], ring current – radiation belt models [e.g., operational, and its predictions can be obtained at the
Fok et al., 2001], conductivity models [e.g., Hardy et al., NOAA Web site (http://www.sec.noaa.gov/rpc/costello/
1987], and the Magnetospheric Specification Forecast Mod- index.html). More recently, Boberg et al. [2000] developed
els (MSFM) (J. W. Freeman, The magnetospheric specifi- their own NN Kp model that also inputs solar wind.
cation and forecast model, unpublished manuscript, 1995, [8] Moderate and high magnetic activities are notoriously
available at http://hydra.rice.edu/freeman/ding/www/ difficult to predict [Joselyn, 1995], precisely when predic-
msfm95/index.html) all require Kp as an input parameter. tions become more crucial for space weather users. The
Also, the recently developed Oval Variation, Assessment, previous Kp models also exhibit this typical behavior
Tracking, Intensity, and Online Nowcasting (OVATION) [e.g., Detman and Joselyn, 1999]. These models, which
model can use Kp as an input parameter to determine the were developed at the time when nowcast Kp models were
equatorial boundary of the auroral oval [Newell et al., not yet operational, are driven solely by solar wind/IMF.
2002]. For space weather, what the models such as MSFM However, it can be reasonably expected that if the model
and OVATION do, in effect, is to take Kp, which provides also inputs parameters that give the present and/or the
qualitative alerts and use it to produce more quantitative history of the state of the magnetosphere in addition to
alerts, such as magnetospheric/ionospheric particle fluxes the solar wind/IMF driver, the predictions would improve.
and electromagnetic fields, auroral oval location and fluxes, For example, Johnson and Wing [2004, 2005] show that
etc. As a result, precautionary measures could be taken to past Kps have strong linear and nonlinear correlations with
avoid or reduce catastrophic damage to power grids and future Kps.
satellites. [9] This paper presents new Kp models that have signif-
[4] As a global geomagnetic activity index, Kp has some icantly higher forecast accuracies than the previous Kp
flaws, although other indices have their own difficulties. operational models. In order to satisfy different needs and
The midlatitude locations (48 – 63 magnetic latitude) of operational constraints, three different models were devel-
the 13 midlatitude stations used to compute Kp render it oped: (1) a model that inputs nowcast Kp and solar wind
rather difficult for deducing the source(s) of its variations, parameters and predicts Kp 1 hour ahead; (2) a model with
leading to some ambiguities in its interpretations. Therefore the same input as model 1 and predicts Kp 4 hours ahead;
as space physics advances, Kp will probably be supplanted and (3) a model that inputs only solar wind parameters and
by a newer index or parameter that can indicate less predicts Kp 1 hour ahead. The 1 hour and 4 hour prediction
ambiguously the state of the magnetosphere. However, for lead times are just rough estimates for solar wind monitor at
the reasons mentioned in the previous paragraph and L1. Of course, the actual prediction lead times may vary,
continuity of the space weather operations, for now and in depending on the solar wind V and the location of the solar
the near future, Kp is likely to continue to play some roles in wind monitor.
space physics and space weather.
[5] As mentioned, the official Kp is published with a few
weeks’ delay, which is usually more than adequate for space
2. Data Set
physics research that analyzes old observations. Unfortu- [10] One shortcoming of many early models is that they
nately, this long delay makes it less useful for the space were trained on a limited data set. For example, the Costello
weather operations. For that reason, Gehred et al. [1995] NN was trained on 7 years of data from 1970, 1976, 1978,
developed a nowcast Kp algorithm, which takes real-time 1980, 1981, 1982, and 1989, but the limited availability of
data from several magnetometer stations, not necessarily the the solar wind data from IMP-8 effectively reduces the
same ones used for official Kp, and applies a similar method usable Kp data by more than 50% [e.g., see Wing et al.,
employed by official Kp to derive Kp estimates. The 1995]. The amount of historical solar wind data has multi-
resulting Kp estimates do not always exactly match the plied in the time since the Costello model was developed.
official Kps, but the advantage of this algorithm is that it Because there are dynamical variations over the course of
can produce estimated Kps in near real-time. This nowcast the solar cycle, it is necessary to build a database that can
Kp has been routinely produced by the United States Air capture such variations to the extent possible. Therefore we
Force (USAF) 55th Space Weather Squadron and made built a solar wind and Kp database that spans more than two
publicly available by NOAA through its Web site. solar cycles, 1975– 2001.
[6] More recently, Takahashi et al. [2001] developed a [11] We used the solar wind and IMF data from IMP-8
more sophisticated Kp nowcast algorithm that calculates Kp (1975– 1999), Wind (1994– 2000), and ACE (2000 – 2001).
fairly accurately. The Kp estimates from this model will The data are publicly available from NASA CDAWeb (http://
soon be made publicly available at the Johns Hopkins cdaweb.gsfc.nasa.gov/). The IMP-8 plasma data are publicly
University Applied Physics Laboratory (JHU/APL) Web available at the MIT IMP-8 Web site (ftp://space. mit.edu/pub/
site (http://sd-www.jhuapl.edu/UPOS/spaceweather.html). plasma/imp/www/imp.html).
[7] For the past few years, the Advanced Composition [12] The historical official Kp index is publicly available
Explorer (ACE) spacecraft, located upstream at Lagrangian at GFZ Postdam Web site (ftp://ftp.gfz-postdam.de/pub/
point (L1), has been reliably providing solar wind measure- home/obs/kp-ap/). Although the historical Kp database is
ments up to 30– 60 min in advance of their arrival at virtually continuous and uninterrupted from 1975 to 2001,
the near-Earth space environment. Therefore Kp forecast the solar wind and IMF data have gaps, effectively reducing
models based on solar wind input could use ACE observa- the number of Kp data points that can be used for model
tions to make short-term forecasts. In fact, Costello [1997] development.
2 of 14
A04203 WING ET AL.: Kp FORECAST MODELS A04203
3 of 14
A04203 WING ET AL.: Kp FORECAST MODELS A04203
4 of 14
A04203 WING ET AL.: Kp FORECAST MODELS A04203
5 of 14
A04203 WING ET AL.: Kp FORECAST MODELS A04203
Figure 5. Example of the APL models Kp forecasts on a randomly selected 30 day period in the test
data set. (a) – (f) IMF Bx, By, Bz, solar wind density, velocity, and dynamic pressure, respectively. (g) – (i)
official Kp (black) and forecast Kp (red [or gray in the black and white version]) for APL models 1, 2,
and 3, respectively. The solar wind and IMF have been hourly averaged and time shifted.
predicts Kp 4 hours ahead based on solar wind measure- imately 1 hour ahead, but the model predicts Kp 4 hours
ments at L1. Thus unlike APL model 1, this model predicts ahead. Figure 5h shows the model Kp forecasts over a
Kp without knowing the entire history of the solar wind 30-day interval, which was randomly selected. Figure 7a
parameter leading up to the forecast time. Specifically, a presents the statistical model performance on the test data
satellite at L1 provides solar wind parameters only approx- set (with input Kp obtained from the official Kp record).
6 of 14
A04203 WING ET AL.: Kp FORECAST MODELS A04203
7 of 14
A04203 WING ET AL.: Kp FORECAST MODELS A04203
8 of 14
A04203 WING ET AL.: Kp FORECAST MODELS A04203
Figure 9. The Costello model predicts Kp more accurately near solar maximum than near solar
minimum for Kp > 3. (a) – (e) Skill scores for the Costello NN Kp model over two solar cycles for Kp =
2 – 6, respectively. The black solid line = True Skill Statistic (TSS) and the dashed line = Gilbert score
(GS). The lighter (gray) solid lines at the bottom of Figures 9c – 9e indicate the normalized monthly
average International Sunspot Number, which exhibits trends similar to the skill scores.
correlation time for high nonlinear significance events lies sphere is dominated more by internal dynamics than it is
around 40 hours (see also Johnson and Wing [2004, 2005]). around solar maximum, when it is more directly driven by
The timescale over which the nonlinear significance is the external inputs, namely solar wind and IMF. The
appreciable is on the order of a week. These timescales are presence of the internal dynamics would complicate the
similar to timescales associated with ring current Kp predictions that rely entirely or partly on the external
relaxation (hours to days) and storm relaxation (1 week). drivers.
[36] Taken together, these nonlinearities and the skill [37] The statistical information dynamical analysis of Kp
scores suggest that around solar minimum, the magneto- shown in Figure 11 indicates that Kp highly correlates with
9 of 14
A04203 WING ET AL.: Kp FORECAST MODELS A04203
Figure 10. Skill scores over two solar cycles for a Kp model that inputs solar wind parameters and
predicts Kp 1 hour ahead (APL model 3). (a)– (e) Skill scores for APL model 3 for Kp = 2 – 6,
respectively. The black solid line = True Skill Statistic (TSS) and the dashed line = Gilbert score (GS).
The lighter (gray) solid lines at the bottom of Figures 10a– 10d plot the normalized monthly average
International Sunspot Number. In Figures 10a– 10d, while the skill scores are higher than those for the
Costello model, they still exhibit yearly variations that roughly follow the same general pattern as the
sunspot number.
the past Kps within the last few hours, linearly and non- [38] Near solar minimum, as shown in Figure 11, there
linearly (the Kp has been interpolated to 15 min resolution). are significant nonlinear correlations between Kp and past
Therefore including the nowcast Kp can improve the 1 hour Kps that may be attributed to internal dynamics. Thus in
ahead Kp forecasting. However, both the linear and nonlin- theory the nowcast and all past Kps may help improve the
ear correlations drop off sharply with increasing t, which Kp forecasts, since they contain some information that
contribute to the rapid decrease in the performances of APL pertains to the internal dynamics, including persistence.
model 2 compared with APL model 1. This information is not available in the solar wind/IMF.
10 of 14
A04203 WING ET AL.: Kp FORECAST MODELS A04203
Figure 11. Normalized linear (SL) and nonlinear (SNL) significance as a function of time delay t, for
Kp data from near (a) solar minimum 1975, (b) solar maximum 1982, (c) solar minimum 1987, and
(d) solar maximum 1992. SL is proportional to the linear correlation coefficient between Kp(T) and
Kp(T t), whereas SNL gives a measure of the nonlinear correlation. SL = solid line and SNL = dashed
line. SL drops off quickly with increasing t for both solar maxima and minima. Near solar minima, there
is a strong nonlinear response with a peak around t = 40 hours, whereas near solar maxima, SNL tapers
off quickly.
The recurrent NN has some capacity to memorize the past on the solar wind V and the location of the solar wind
Kps (for APL models 1 and 2), but it is doubtful that it can monitor. All these models are based on NNs and were
effectively capture the complex correlations or dynamics developed with 27 years of data, 1975– 2001.
beyond the last few hours (the nonlinear correlations can [40] Our extensive evaluation based on data spanning
extend beyond several hours). more than two solar cycles shows that (1) our models give
significantly more accurate predictions than previous
models, with the most dramatic improvements occurring
7. Discussion and Summary during moderate and active times, Kp > 4, and (2) Kp is
[39] In order to satisfy different needs and operational slightly more predictable near solar maximum than it is near
constraints, we developed three types of Kp forecast solar minimum. Information dynamics analysis of Kp
models, two of which require nowcast Kp as an input suggests that the magnetosphere is more externally driven
parameter. The focus of the model development was near solar maximum (or the declining phase of the solar
improving the forecasts for magnetically moderate and maximum) than near solar minimum. Around solar mini-
disturbed times, which are notoriously difficult to predict mum, the internal dynamics such as loading and unloading
[e.g., Joselyn, 1995]. The three models are (1) APL model 1 of the energy in the magnetotail, ring current decays, storm
that inputs nowcast Kp and solar wind parameters and relaxation, etc., may play a more important role in the
predicts Kp 1 hour ahead; (2) APL model 2 that has the magnetospheric dynamics and may introduce some diffi-
same input as APL model 1 and predicts Kp 4 hours ahead; culties to Kp models that rely entirely or partly on solar
and (3) APL model 3 that inputs only solar wind parameters wind parameters.
and predicts Kp 1 hour ahead. The 1 and 4 hour prediction [41] The performances of various models can be summa-
lead times are just rough estimates for a solar wind monitor rized in Figure 13, which plots the TSS scores of the Kp
at L1. The actual prediction lead times may vary, depending threshold [Detman and Joselyn, 1999]. Figure 14 summa-
11 of 14
A04203 WING ET AL.: Kp FORECAST MODELS A04203
Figure 12. The nonlinear response anticorrelates with the sunspot number in every solar cycle from the
first Kp record to present, 1932 – 2003. (a) The black line shows the maximum nonlinear significance
(SNL); (b) and (c) the black line shows the integrated SNL at >95% confidence and at >99.5%
confidence, respectively. The lighter (gray) lines at the bottom in Figures 12a– 12c show the normalized
sunspot number.
12 of 14
A04203 WING ET AL.: Kp FORECAST MODELS A04203
13 of 14
A04203 WING ET AL.: Kp FORECAST MODELS A04203
Postdam for supplying Kp records. S. Wing was supported by NSF Space Lo, J., and D. Bassu (1999), Mathematical justification of recurrent multi-
Weather grant ATM-9819705, NASA grant NAG5-10971 and DOD con- layer perceptrons with long- and short-term memories, in Proceedings of
tract N00024-03-D-6606. J. R. Johnson was supported by NSF grant ATM- International Joint Conference on Neural Networks 1999, vol. 1, pp.
0218847 and NASA grant W-19880. We thank J. Vandegriff for assisting 364 – 369, IEEE Press, Piscataway, N. J.
with the coding of the recurrent network. Mauk, B. H., and C. E. McIlwain (1974), Correlation of Kp with the sub-
[47] Arthur Richmond thanks Harry C. Koons and another reviewer for storm-injected plasma boundary, J. Geophys. Res., 79, 3193 – 3196.
their assistance in evaluating this paper. Newell, P. T., S. Wing, C. I. Meng, and V. Sigillito (1990), A neural
network based system for monitoring the aurora, Johns Hopkins APL
References Tech. Dig., 11(3, 4), 291 – 299.
Newell, P. T., S. Wing, C. I. Meng, and V. Sigillito (1991), The auroral oval
Arge, C. N., and V. J. Pizzo (2000), Improvement in the prediction of solar position, structure and intensity of precipitation from 1984 onwards: An
wind conditions using near-real time solar magnetic field updates, automated on-line data base, J. Geophys. Res., 96, 5877 – 5882.
J. Geophys. Res., 105, 10,465 – 10,479. Newell, P. T., V. A. Sergeev, G. R. Bikkuzina, and S. Wing (1998),
Balikhin, M. A., O. M. Boaghe, S. A. Billings, and H. St C. K. Alleyne Characterizing the state of the magnetosphere: Testing the ion precipita-
(2001), Terrestrial magnetosphere as a nonlinear resonator, Geophys. Res. tion maxima latitude (b2i) and the ion isotropy boundary, J. Geophys.
Lett., 28, 1123 – 1126. Res., 103, 4739 – 4745.
Bartels, J. (1949), The standardized index, Ks, and the planetary index, Kp, Newell, P. T., T. Sotirelis, J. F. Carbary, K. Liou, J. P. Skura, C.-I. Meng,
IATME Bull., 12b, 97. C. Deehr, D. Wilkinson, and F. J. Rich (2002), OVATION: Oval
Boaghe, O. M., M. A. Balikhin, S. A. Billings, and H. Alleyne (2001), Variation, Assessment, Tracking, Intensity, and Online Nowcasting,
Identification of nonlinear processes in the magnetospheric dynamics and Ann. Geophys., 20, 1039 – 1047.
forecasting of Dst index, J. Geophys. Lett., 106, 30,047 – 30,066. Papitashvili, V. O., N. E. Papitashvili, and J. H. King (2000), Solar cycle
Boberg, F., P. Wintoft, and H. Lundstedt (2000), Real time Kp prediction effects in planetary geomagnetic activity: Analysis of 36-year long OMNI
from solar wind data using neural networks, Phys. Chem. Earth, 25, dataset, Geophys. Res. Lett., 27, 2797 – 2800.
275 – 280. Rostoker, G. (1972), Geomagnetic indices, Rev. Geophys., 10, 935 – 950.
Costello, K. A. (1997), Moving the Rice MSFM into a real-time forecast Rumelhart, D. E., and J. L. McClelland (Eds.) (1987), Parallel Distributed
mode using solar wind driven forecast models, Ph.D. dissertation, Rice Processing, vol. I, MIT Press, Cambridge, Mass.
Univ., Houston, Texas. Sergeev, V. A., M. Malkov, and K. Mursula (1993), Testing the isotropic
Crooker, N. U., and K. I. Gringauz (1993), On the low correlation between boundary algorithm method to evaluate the magnetic field configuration
long-term averages of solar wind speed and geomagnetic activity after in the tail, J. Geophys. Res., 98, 7609 – 7620.
1976, J. Geophys. Res., 98, 59. Takahashi, K., B. A. Toth, and J. V. Olson (2001), An automated proce-
Detman, T., and J. A. Joselyn (1999), Real-time Kp predictions from ACE dure for near-real-time Kp estimates, J. Geophys. Res., 106, 21,017 –
real time solar wind, in Solar Wind Nine, edited by S. R. Habbal et al., 21,032.
AIP Conf. Proc., 471, 729 – 732. Tsyganenko, N. A. (1989), A magnetospheric magnetic field model with a
Fernandez, B., A. G. Parlos, and W. K. Tsai (1990), Nonlinear dynamic warped tail current sheet, Planet. Space Sci., 37, 5 – 20.
system identification using artificial neural networks (ANNs), in Pro- Vandegriff, J., K. Wagstaff, G. Ho, and J. Plauger (2005), Forecasting space
ceedings of International Joint Conference on Neural Networks 1990, weather: Predicting energetic storm particle events using neural networks,
edited by M. Caudill, pp. 133 – 141, IEEE Press, Piscataway, N. J. Adv. Space Sci., in press.
Fok, M.-C., T. E. Moore, and W. N. Spjeldvik (2001), Rapid enhancement Wang, Y.-M., and N. R. Sheely (1995), Solar implications of ULYSSES
of radiation belt electron fluxes due to substorm dipolarization of the interplanetary field measurements, Astrophys. J., 447, L143 – L146.
geomagnetic field, J. Geophys. Res., 106, 3873 – 3881. Wing, S., and P. Newell (2003), LLBL contribution to the plasma sheet
Fry, C. D., W. Sun, C. S. Deehr, M. Dryer, Z. Smith, S.-I. Akasofu, ions, in Earth’s Low-Latitude Boundary Layer, Geophys. Monogr. Ser.,
M. Tokumaru, and M. Kojima (2001), Improvements to the HAF solar vol. 133, edited by P. T. Newell and T. Onsager, pp. 273 – 282, AGU,
wind model for space weather predictions, J. Geophys. Res., 106, Washington D. C.
20,985 – 21,001. Wing, S., P. T. Newell, D. G. Sibeck, and K. B. Baker (1995), Large
Garrett, H. B., A. J. Dessler, and T. W. Hill (1974), Influence of solar wind statistical study of the entry of interplanetary magnetic field Y-component
variability on geomagnetic activity, J. Geophys. Res., 79, 4603 – 4610. into the magnetosphere, Geophys. Res. Lett., 22, 2083 – 2086.
Gehred, P. A., W. Cliffswallow, and J. D. Schroeder III (1995), A compar- Wing, S., R. A. Greenwald, C.-I. Meng, V. G. Sigillito, and L. V. Hutton
ison of USAF Ap and Kp indices to Gottingen indices, Tech. Memo. ERL (2003), Neural Networks for automated classification of ionospheric
SEL-88, NOAA, Silver Spring, Md. irregularities from HF radar backscattered signals, Radio Sci., 38(4),
Gershenfeld, N. (1999), The Nature of Mathematical Modeling, pp. 150 – 1063, doi:10.1029/2003RS002869.
153, Cambridge Univ. Press, New York. Wu, J., and H. Lundstedt (1997), Geomagnetic storm predictions from solar
Hakamada, K., and S.-I. Akasofu (1982), Simulation of three-dimensional wind data with the use of dynamic neural networks, J. Geophys. Res.,
solar wind disturbances and resulting geomagnetic storms, Space Sci. 102, 14,255 – 14,268.
Rev., 31, 3 – 70.
Hardy, D. A., M. S. Gussenhoven, R. Raistrick, and W. J. McNeil (1987),
Statistical and functional representations of the pattern of auroral energy
flux, number flux, and conductivity, J. Geophys. Res., 92, 12,275 – 12,294.
Hedin, A. E. (1987), MSIS-86 thermospheric model, J. Geophys. Res., 92,
4649. M. Balikhin, Department of Automatic Control and System Engineering,
Johnson, J. R., and S. Wing (2004), A cumulant-based analysis of nonlinear University of Sheffield, Mappin Street, Sheffield S1 3JD, UK.
magnetospheric dynamics, Publ. PPPL-3919, Princeton Plasma Physics K. Bechtold, J. Jen, C.-I. Meng, K. Takahashi, and S. Wing, Johns
Laboratory, Princeton, N. J. (Available at http://www.pppl.gov/pub_ Hopkins University Applied Physics Laboratory, 11100 Johns Hopkins
report//2004/PPPL-3919.pdf) Road, Laurel, MD 20723-6099, USA. (simon.wing@jhuapl.edu)
Johnson, J. R., and S. Wing (2005), A solar cycle dependence of nonlinear- K. Costello, NASA Lyndon B. Johnson Space Flight Center, 2101 NASA
ity in magnetospheric activity, J. Geophys. Res., doi:10.1029/ Parkway, Houston, TX 77058, USA.
2004JA010638, in press. J. Freeman, Physics and Astronomy Department, Rice University, P. O.
Joselyn, J. (1995), Geomagnetic activity forecasting: The state of the art, Box 1892, Houston, TX 77005, USA.
Rev. Geophys., 33, 383 – 401. J. R. Johnson, Princeton Plasma Physics Laboratory, Princeton
Koons, H. C., and D. J. Gorney (1991), A neural network model of the University, P. O. Box 451, MS 28, Princeton, NJ 08543-0000, USA.
relativistic electron flux at geosynchronous orbit, J. Geophys. Res., 96, D. G. Sibeck, NASA Goddard Space Flight Center, LEP Code 696, 8800
5549 – 5556. Greenbelt Road, Greenbelt, MD 20771, USA.
14 of 14