From rls@email.unc.edu Thu Aug 21 16:25:41 2003 Date: Wed, 18 Dec 2002 13:30:08 -0500 (EST) From: Richard Smith To: Doug Nychka Cc: Rick Katz Subject: Re: External Advisory Panel Report MEETING OF THE GSP ADVISORY PANEL AT NCAR, JULY 11-12 2002 Present: Richard Smith (chair), Peter Guttorp, Gerald North, Brani Vidakovic Not present: Sallie Keller-McNulty, John Rice, Dan Wilks, Andrew Solow. I. SUMMARY OF PRESENTATIONS Thomas has matured considerably since last year. He has a very effective style of presentation including excellent use of dynamic graphics. The panel were generally impressed at the methodological development that has taken place since last year: this is an important problem area with implications for the broad problem of nonlinear state space modeling. There is some concern that the work has not proceeded beyond toy problems, though one panel member comments that sometimes important dynamical features can become apparent in looking at a 40-dimensional problem that would be lost in the noise if one tried directly to extend the analysis to many thousands of dimensions. Thomas' second project has resulted in what is essentially a negative finding. When setting down a simple set of equations for describing buoy motion in the Labrador Sea, ignoring a nonlinear part of the system, he chose to estimate the coriolis parameter, even though it is a known number. The resulting estimate, far away from the true value, indicated that the nonlinear part of the dynamics are important, and lead to novel ideas in how to model stochastically the nonlinear part. Hee-Seok's speaking style has shown remarkable progress since last year. He still has evident limitations, but has worked hard to give effective presentations within those limitations. He also shows a much better grasp of the scientific issues underlying his work than he did last year. As far as actual content is concerned, the robust smoothing project seems a highly competent methodological contribution, also the only project we heard that is mathematical statistics in the traditional sense. The use of pseudo-data in conjunction with wavelets is a novel way to robustify wavelet analysis. Hee-Seok's second project also shows good command of statistical methods to examine some important issues in climatology, though the use he makes of wavelets is a little unpolished and needs more attention to some fundamental issues. Enrica's work has also progressed since last year, to the point where it no longer seems a naive attack on the cloud-resolving problem. This is one of the big unsolved problems of climate science and any progress on this by a statistician could have major impact. Her second project did not show new methodological advances but she has succeeding in implementing ideas where other researchers have not. Eric's project seemed promising for a PhD student at this stage of career, though it was noted that it is mostly simulation and not yet much theoretical development. He has clearly thought carefully about the spatial-temporal model fitted to the ozone data, though it would be desirable to relate this to existing literature on spatial-temporal ARMA modeling. The major potential objection to what he is doing is the reliance on fitting a Gaussian model on the daily data and then extrapolating that to extreme value calculations - this may be the reason to try to focus more specifically on extremal properties of the data, which the current approach does not yet do. Doug's talk outlining Sarah's project produced some disagreement among committee members. There is existing work in hydrology (Bell?) which needs to be referenced. On the other hand we are not aware of work by statisticians on this kind of approach and with the application of modern statistical tools (EM, MCMC, etc) could lead to much improvement in the estimation methodology. Also it was noted that the combination of the Gaussian process thresholding approach with the initial modeling of rainfall probabilities that Sarah did earlier might be a big improvement on going directly to thresholding. Tim's contribution, as in previous years, was impressive. A project like this relies heavily on making the most effective use of modern computing tools and Tim's expertise seems indispensible to making this happen. As regards his actual presentations, the first part was probably a bit lost in technical details for most of the panel members, but as a general critique of the wind modeling project as it has developed over a number of years, it is one of the major scientific contributions of GSP. Tim's second and main talk seemed to some panel members like using a sledgehammer to crack a nut. This is not a criticism of the work done by Brandon and Tim, but it is not clear that there is new scientific progress here. Computationally this is a daring project, and has drawn attention to the need for new theoretical work on penalty functions in the style of Donoho/Johnstone work on wavelets. Claudia's talk seemed an excellent example of where GSP could make important contributions to atmospheric science - in a case where a technique has been developed on apparently ad hoc grounds by atmospheric scientists, her contribution was to point out the connections with existing statistical methodology and then to use that connection to suggest extensions of the methodology (the random effects approach). Overall, we found the talks probably the best group of talks we have heard from a group of GSP postdocs, nothing below expectations. While we recognize Doug's hard work in coaching them to give presentations, we also think they have benefitted a lot from working with each other. II. IMPACT ON NCAR In previous years we have gained the impression that many of the senior scientists at NCAR had little appreciation of what GSP was doing and were not inclined to give it much support. The problem still exists to some extent, but we believe the situation has substantially improved. There was quite a bit of discussion about how GSP could continue to improve its position within NCAR. One idea is for GSP to sponsor tutorials and workshops aimed at presenting modern statistical ideas to NCAR scientists. These need not only be by GSP members: it could be an effective way to bring in a senior statistician from somewhere else and ask him/her to give a series of lectures. Among topics that were thought potentially suitable for such treatment: extreme values, long-range dependence, modern developments in spatial-temporal statistics both within GSP and elsewhere, cyclostationarity, multiresolution methods, functional data analysis, non-Gaussian multivariate distributions, or some other statistical topic that NCAR scientists would like to hear about. III. IMPACT ON STATISTICS In previous years we have sometimes commented on the lack of traditional methodological research by GSP postdocs. As we view it now, there is probably no need for the traditional theorem-proof style of paper, but more could still possibly be done to highlight connections between GSP work and other parts of statistics. For example, where "core" problems are suggested by GSP work, more could be done to make the statistical community aware of these problems. The problem of transferring information between Lagrangian and Eulerian coordinates, mentioned in one of the talks, was cited as an example. The group seems to be developing a good publication record, though a lot of it is in nontraditional outlets compared to much research that goes on in academic statistics departments. Conferences talking about new statistical developments (e.g. the large data sets workshop of 2000) are a good idea, and also a potential recruiting tool. The idea of having roughly one such conference every two years seems good, and should be continued. IV. FUTURE OF GSP The question of continued recruitment of postdocs was discussed at some length. Evidently the present group are very good and have worked together a lot, but most of them are now leaving and this year's recruiting has not gone so well. This is seen primarily as a reflection of the job market: statistics PhDs at the moment are able to go largely where they want, and there are also more postdoc programs that compete with GSP for this style of work (e.g. the new EPA center at Chicago, NISS/SAMSI, postdoc programs at Duke and CMU). We do not perceive that GPS is doing anything fundamentally wrong but there is a risk that reviewers of the next NSF proposal may conclude the program is past its peak. Doug has been very energetic in giving talks at conferences and universities and developing personal contacts. The panel were concerned that maybe he relies too much on personal contacts and needs to make more use of traditional recruiting tools, e.g. letters and emails to department chairs. It was also pointed out that the web site is not a good advertisement for the program - this point was made in last year's report but there is still a feeling that one has to search around a lot to find important information. There was some discussion of whether GSP should continue primarily as a postdoc program or should branch out in other directions. As we see it, postdocs remain the core of this program, and there needs to be a critical mass of postdocs (suggest 3 minimum) to keep going the kinds of interactions that are needed. Having more PhD students is an excellent idea but they are of less value in creating a GSP research profile. The program should probably still be making the effort to bringing in senior statisticians as visitors, either on sabbatical leaves as in the past, or through some mechanism such as inviting a senior statistician to give a course of lectures on some topic that would be of interest to atmospheric scientists. Suggestions for NSF renewal: Develop a "recruiting plan" for postdocs Continue emphasis on senior visitors Try to emphasize ways to transfer expertise from GSP back into core statistics research More international visitors V. FUTURE OF ADVISORY PANEL The panel believes it is appropriate to continue with a panel of the present size, rather than reducing it as has been suggested. The 2002 meeting had only four of eight members present; this is undesirable but probably due primarily to the late change of date. In reality, there will always be some panel members who cannot make it to the meeting and the current composition ensures broad representation. There is still a need for a senior atmospheric scientist on the panel; names suggested included Mike Wallace and Jean Thiebaux. The panel discussed "term limits" but saw no compelling reason to change the current system.