tag:blogger.com,1999:blog-82234548913236687322024-03-08T15:23:07.693+11:00Eco-Stats Research BlogThe life and times of the <a href="http://www.eco-stats.unsw.edu.au">UNSW Ecological Statistics ("Eco-Stats") research group.</a>David Wartonhttp://www.blogger.com/profile/05231788956034178551noreply@blogger.comBlogger78125tag:blogger.com,1999:blog-8223454891323668732.post-92054124461492616642023-05-17T17:16:00.004+10:002023-05-17T17:17:11.689+10:00Fast fitting of log-Gaussian Cox processes for presence-only data, using the scampr package<p>The first paper from Elliot Dovers's PhD thesis is <a href="https://www.tandfonline.com/doi/full/10.1080/10618600.2023.2182311" target="_blank">now out in the Journal of Computational and Graphical Statistics</a>, on fast methods of fitting log-Gaussian Cox process models (LGCPs) using <a href="https://github.com/ElliotDovers/scampr" target="_blank">an R package he wrote called scampr, available on GitHub</a>. LGCPs are a good way to analyse presence-only data in ecology, they are a type of point process model that can account for clustering in data (which usually happens in practice). If this is not accounted for your results can have false confidence, very much like what happens if you have overdispersed count data and you don't account for the overdispersion.</p><p>The problem has always been that LGCPs are hard to fit - the fastest tool at the moment is INLA, which can be a bit fiddly to use and can take a long time for the typical dataset - and Elliot's work addresses this by using a basis function approximation (a lot like what is done to fit a smoother in a GAM) and coding it using TMB, which uses C++ to do the heavy lifting which is much faster computationally than R.</p><p>Look out for a tutorial on how to use scampr coming here soon... </p>David Wartonhttp://www.blogger.com/profile/05231788956034178551noreply@blogger.com0tag:blogger.com,1999:blog-8223454891323668732.post-43638790701110337212023-05-17T17:06:00.002+10:002023-05-17T17:07:23.711+10:00Eco-Stats - Data Analysis in Ecology now available from Springer<p> it was a long time in the making (like, 5 or 6 years?) but <a href="https://link.springer.com/book/10.1007/978-3-030-88443-7" target="_blank">my book is finally out at Springer</a>!</p><p>Here is my favourite plot from it - some issues to think about when minding your P's and Q's!</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEgefMZpFrE7v0QFQRoz4APoxDG1A8SqlkIpkhjIi0q-HofJUoesLeneaFsjjEIrwPvZTAVhJDq59l-n6TMulZJAFOpaFpm4uJIvQIXeICesclkAueyJS-TD-JVoVlJzR6LURy47T63xSbeOTemXUKZ1KiM3yX1Xj-Dk4d0z5Q68wa25FSJukQ9OVYfJ" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="1249" data-original-width="1016" height="538" src="https://blogger.googleusercontent.com/img/a/AVvXsEgefMZpFrE7v0QFQRoz4APoxDG1A8SqlkIpkhjIi0q-HofJUoesLeneaFsjjEIrwPvZTAVhJDq59l-n6TMulZJAFOpaFpm4uJIvQIXeICesclkAueyJS-TD-JVoVlJzR6LURy47T63xSbeOTemXUKZ1KiM3yX1Xj-Dk4d0z5Q68wa25FSJukQ9OVYfJ=w437-h538" width="437" /></a></div><br /><p></p>David Wartonhttp://www.blogger.com/profile/05231788956034178551noreply@blogger.com0tag:blogger.com,1999:blog-8223454891323668732.post-17401109121617292312021-04-30T12:06:00.012+10:002021-04-30T14:07:17.710+10:00Equivalence testing - is there evidence of similarity?<p>This May, we are going to explore equivalence testing - a method that helps us look for evidence of similarity. We will introduce univariate equivalence testing using the R package <span style="font-family: courier;">TOSTER</span> and look at multivariate equivalence testing using the package <span style="font-family: courier;">ecopower</span>. Check out the slides and code <a href="https://equivtest.netlify.app/" target="_blank">here</a>!</p>Michelle Limhttp://www.blogger.com/profile/12369189317937459619noreply@blogger.com10tag:blogger.com,1999:blog-8223454891323668732.post-85713814891401661632021-03-11T15:59:00.000+11:002021-03-11T15:59:00.834+11:00glmmTMB: the flexible and fast package for fitting generalized linear mixed models<p>February's Eco-stats lab was about the R package glmmTMB for general linear mixed models. glmmTMB is considerably faster and more stable than lme4 while retaining the same syntax as lme4. It offers more distributions such as negative binomial or Conway-Maxwell-Poisson distribution for overdispersed counts and provides extensions of GLMMs. Check out the <a href="https://rpubs.com/mmcgillycuddy/737181" target="_blank">slides</a> to see some examples of what can be done.</p>Maeve McGillycuddyhttp://www.blogger.com/profile/07625854317286887196noreply@blogger.com0tag:blogger.com,1999:blog-8223454891323668732.post-9551601001407788512020-11-16T15:27:00.000+11:002020-11-16T15:27:53.148+11:00ecoCopula : and R package for fast ordination and graphical modelling<p>I have been a bit tardy putting up October's Eco-Stats Lab, which was on the <a href="https://cran.r-project.org/web/packages/ecoCopula/index.html">ecoCopula </a>package (now on CRAN). ecoCopula does fast model based ordination, and lets you create 'graphs' of species associations (interactions) from
co-occurrence data. See the <a href="https://cran.r-project.org/web/packages/ecoCopula/vignettes/the_basics.html">vignette </a>for a walk though. <br /></p>Gordana Popovichttp://www.blogger.com/profile/13277950763972269993noreply@blogger.com0tag:blogger.com,1999:blog-8223454891323668732.post-73917024877653155542020-11-12T15:50:00.000+11:002020-11-12T15:50:03.188+11:00plotenvelope - making sense of diagnostic plots using simulation envelopesThis month's Eco-Stats Lab is on the plotenvelope function in the ecostats package. This function can be used (in place of plot) to construct residual plots of most common fitted model objects in R, including global simulation envelopes around fitted values (or around smoothers) on the plot. <a href="https://rpubs.com/dwarton/689951" target="_blank">These slides explain what this all means</a>. <a href="https://rpubs.com/dwarton/689951" target="_blank"></a>David Wartonhttp://www.blogger.com/profile/05231788956034178551noreply@blogger.com1tag:blogger.com,1999:blog-8223454891323668732.post-69322721511703222752020-09-27T14:38:00.000+10:002020-09-27T14:38:20.835+10:00<p> <img height="236" src="https://lh5.googleusercontent.com/UZTsOXT2VdtMdpOReGJrpZSrAEa7N_qQF79o6YPZMQOrMQXLEChlCwAJDQee0dnc3mOLpnB1w_VpAqG3Eqd_Oc7tpst1JbcURLoSNpyb4cGy2KfvkgUibzitZ2Pu1qsO8D8pLeZf" style="font-family: Arial; font-size: 26pt; margin-left: 0px; margin-top: 0px; text-align: center; white-space: pre-wrap;" width="624" /></p><span id="docs-internal-guid-1818b0ee-7fff-3c0a-232d-326f6e7d104d"><p dir="ltr" style="line-height: 1.38; margin-bottom: 3pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 26pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Introduction to Docker</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">It’s become increasingly important to be able to experiment, reproduce workflows and results, share code and to be able to take advantage of large scale data analysis. </span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Docker is a lightweight virtual machine technology that provides an isolated, self-contained, versioned and shareable linux-based environment that is now the tool of choice in industry. With the growth of cloud computing companies regularly spin up clusters containing hundreds or even thousands of docker instances - and with that there’s been a concurrent growth in management and orchestration tools in the docker ecosystem.</span></p><h1 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 20pt;"><span style="font-family: Arial; font-size: 20pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 400; vertical-align: baseline; white-space: pre-wrap;">Ok - that’s great - but why should statisticians care?</span></h1><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Because this is a great way to easily scale and share your research while it’s in progress. You can package up work-in-progress R code and data </span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">without</span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> having to build R packages first.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">It also allows you to experiment and test without concerns - for example R 4.0 is now available. Given the choice between testing your work against new versions of R and upgrading your base R installation, an easy solution is to use a Docker container configured with R 4.0 and leave your host machine alone.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Here’s some motivating examples:</span></p><br /><ul style="margin-bottom: 0; margin-top: 0;"><li dir="ltr" style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Test your package against new and previous versions of R packages and environments without having to upgrade all your existing packages. This allows you to detect breaking changes in your package before release</span></p></li></ul><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><span class="Apple-tab-span" style="white-space: pre;"> </span></span></p><ul style="margin-bottom: 0; margin-top: 0;"><li dir="ltr" style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Jump-start exploratory work with pre-configured images. For example the </span><a href="https://hub.docker.com/r/rocker/geospatial" style="text-decoration-line: none;"><span style="color: #1155cc; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space: pre-wrap;">https://hub.docker.com/r/rocker/geospatial</span></a><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> image comes with the full suite of R spatial packages as well as tidyverse out of the box</span></p></li></ul><br /><ul style="margin-bottom: 0; margin-top: 0;"><li dir="ltr" style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Try out a completely new technology without polluting your computer - e.g teach yourself Juliet and Jupyter notebooks in an isolated environment</span></p></li></ul><br /><ul style="margin-bottom: 0; margin-top: 0;"><li dir="ltr" style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Scale parallel code without change - parallel code that runs on your local host can take advantage of massive cloud compute power without any extra coding</span></p></li></ul><br /><ul style="margin-bottom: 0; margin-top: 0;"><li dir="ltr" style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Ship & Share - docker promotes reproducibility in research. No matter what needs to be installed in the environment to reproduce your results, you can build a complete machine image packaging up code and data and simply share the image. This is the virtual equivalent of “here - just use my computer. It works on that…”</span></p></li></ul><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Docker doesn’t replace the value of building and sharing R packages themselves. The point is simply that you do not have to wait until all your code is packaged any longer and can experiment and collaborate freely while research is in progress.</span></p><br /><br /><h1 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 20pt; text-align: center;"><span style="font-family: Arial; font-size: 20pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 400; vertical-align: baseline; white-space: pre-wrap;"><span style="border: none; display: inline-block; height: 146px; overflow: hidden; width: 262px;"><img height="146" src="https://lh5.googleusercontent.com/9iNZn9R_bQ16x74yDJtj_6KtRe6hv9Cm0SrGiPOXS8YlYxaKo1EzWdDc5qLl4BGs8F40KOwb4fKXCXZaDsAwoCY6NVNZoch1TXUPnKnw88-4xqtVA2ZetPl7RrWIcvYXb8_PCMK3" style="margin-left: 0px; margin-top: 0px;" width="262" /></span></span></h1><h1 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 20pt;"><span style="font-family: Arial; font-size: 20pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 400; vertical-align: baseline; white-space: pre-wrap;">Basic Concepts</span></h1><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">DockerHub </span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">- (https://hub.docker.com/) this is the equivalent of GitHub but much coarser. It’s the central repository for versioned Docker images - you can pull images from public repositories for your own use or pull/push images to your own private repositories. </span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">To use Docker you need to signup a free account which automatically gives you access to your own private repositories. There are also enterprise tiers with additional benefits.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">Docker Image</span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> - this is a blueprint for the machine itself. You can choose from thousands of pre-configured images available at DockerHub, use them as is or as a baseline to customise your own. An image is like a class in programming - it’s used to create instances of the running machine - known as containers.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">An image contains all the code, packages and programming environment needed to support the work you want to do. It also has a version associated with it - called a </span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">tag</span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">Docker File</span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> - used to build an image. This is where you can specify additional packages, copy source code or add additional linux command line tools to be included in the image. You only need to work with this if you want to customise an image for your own use.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">Docker Container</span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> - this is a running instance of a machine created from an image. A container is like an object instance created from a programming class. The Docker service on your local machine will instantiate the container for you - you can then connect to it and get busy.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">General Notes</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">While Docker is a mature technology it does have some drawbacks. Docker images are generally a few gigabytes in size and so can quickly take up a lot of space on a standard laptop. </span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Consequently, pushing and pulling images to and from DockerHub can be time consuming unless you have a fast internet link. This isn’t a big problem as most of the time you’re probably only working with one container image but it’s worth keeping in mind.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Secondly, interfacing with Docker can be daunting. Bare in mind that it was designed by hardcore sys-admin types so there are options (i.e command line switches) for almost anything you can think of.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">However there are generally only a few commands to know that will cover almost all use cases which we’ll summarise next.</span></p><h1 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 20pt;"><span style="font-family: Arial; font-size: 20pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 400; vertical-align: baseline; white-space: pre-wrap;">Docker Command Cheatsheet</span></h1><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">These commands are run in a terminal window on your Windows/Mac/Linux host machine:</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space: pre-wrap;">Login to DockerHub from your host machine:</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">docker login</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">This will prompt your for the username and password you used to sign up with DockerHub. </span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space: pre-wrap;">Pull an Image to your local machine:</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">docker pull rocker/geospatial</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">This pulls the latest version of the rocker/geospatial image. You can also specify a particular version you want to use by including a tag:</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">docker pull rocker/geospatial:4.0.2</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space: pre-wrap;">List images:</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">docker images</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Displays all images you have available on your host:</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><span style="border: none; display: inline-block; height: 105px; overflow: hidden; width: 624px;"><img height="105" src="https://lh6.googleusercontent.com/6JzGngaHOx2ljsWVmKDcG1ecWUDM88rqDNJMWnr2gFV1c3IH3p1EDpMyAfEQtdzh_aLLwlwgW6M37utoktaEXotrt3s74ZiqgD7tOo4Lnq0IyUjBD3ENObDF9789hMyFIn3RyZoc" style="margin-left: 0px; margin-top: 0px;" width="624" /></span></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space: pre-wrap;">Launch a container (simple version):</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">docker run -it chenobyte/geospatial:1.4 /bin/bash</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">This creates a container using the chenobyte/geospatial:1.4 local image, connects to it and runs a bash shell inside the container. The -it switches ensure the container stays interactive. As can be seen below we are inside the container as root:</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><span style="border: none; display: inline-block; height: 79px; overflow: hidden; width: 624px;"><img height="79" src="https://lh5.googleusercontent.com/nGd2adpt68PcmezL4fn0TqCYY8M_EPy_NZaJELkYM93DA0iYJfNChgyAkAiHFBa0vGKfro16MUXUbSAZU8KTNBhpTLKEdF_CTqESszLyZHyy45farXpYYOaTauf2BSx9Z4iqaFEt" style="margin-left: 0px; margin-top: 0px;" width="624" /></span></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Once inside, you can kill or exit the container either by Ctrl+D or by typing exit. This drops you back out to the host terminal. You can also exit back to the host </span><span style="font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">without</span><span style="font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;"> </span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">killing the container by typing Ctrl+PQ. This is known as </span><span style="font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">detaching</span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space: pre-wrap;">List containers:</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">docker ps -a</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Displays all containers including their status. Note that docker automatically assigns names to each instance - although you can override this if you really want to:</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><span style="border: none; display: inline-block; height: 87px; overflow: hidden; width: 624px;"><img height="87" src="https://lh5.googleusercontent.com/uuacg18_UhpqNhREgZJUttf29BCv7oqrE3_hB1hlWji5uIufkmd9ZlC7375XeWKYRT5rYrVWwp7hdYzAQjvVaIFYnTY39-oqUoimVv14qpmMrpZMcboimDV7sElbHhessxrVpIF3" style="margin-left: 0px; margin-top: 0px;" width="624" /></span></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space: pre-wrap;">Attaching to a Running Container:</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">As seen above, each container has a container identifier. You can attach (jump back into) a running container using the container id:</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">docker attach <container_id></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><span style="border: none; display: inline-block; height: 97px; overflow: hidden; width: 624px;"><img height="97" src="https://lh4.googleusercontent.com/f8DBl-bIK6w20HRNn0rQn5QAS47v27-s20nEZ-ihXRqvOYvMoXBDMnEl36fcZSPmnqveIzVLL2ZqkhFxt2FVJSv1s1MjcrCzU40RhnuyHIhi_jnQD3jeRy9132w0O3no9oNp65uj" style="margin-left: 0px; margin-top: 0px;" width="624" /></span></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space: pre-wrap;">Commit a Running Container:</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">This is useful when you’ve launched a container, made some changes while inside it (say installed some utilities or software) and now want to save the complete container state as a new image that you can reuse later.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">The general format is:</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">docker commit <container_id> name:tag</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">As an example, if I’ve launched a container using image chenobyte/geospatial:1.5 and then make some changes I want to keep (e.g installed git) , I can detach and save the current container state as a new image. If the container id is 1234 then I can make a new version:</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">docker commit 1234 chenobyte/geospatial:1.5</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">I now have a new image version which is just like any other image and can be used to launch new containers too.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space: pre-wrap;">Stopping a Running Container:</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">You stop a running container from the host using the container id:</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">docker stop <container_id></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-style: italic; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space: pre-wrap;">Launch a container (complicated version):</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">docker run -it </span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">-p 8888:8888 \</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">-v /home/ec2-user/data:/data \</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">-e NB_UID=$(id -u) \</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">-e NB_GID=$(id -g) \</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">-e GRANT_SUDO=yes \</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">-e GEN_CERT=yes \</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">--user root \</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">chenobyte/geospatial:1.4 \</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">/bin/bash</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">This example looks daunting but is actually the most useful common case. The main issues are around making sure you have permissions inside the container to write to host directories etc. You don’t need to understand all the -e options - you can just re-use as is.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">The -p switch exposes port 8888 inside the container as port 8888 to the host. This is useful when the container is running a server (RStudio, Jupyter etc). See the aside below for more information.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">The -v option is probably the most useful. This a volume mapping which says that the directory /home/ec2-user/data on the host should be made available inside the container at /data. In this way, data files or code on the host can be accessed from within the container itself. You can have as many mappings as you want here.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">There are many </span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">many</span><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"> more available options for running containers including specifying available CPUs and RAM, container lifecycle etc. For more information check out the references in the resources section below.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;"><img aria-roledescription="Drawing" height="409" src="https://docs.google.com/drawings/u/0/d/sFxKWO_kSQcPDKFc5QlTU-g/image?w=686&h=427&rev=17&ac=1&parent=10LKQkwU2_mRzDmQlPJmmyl61dMtiIFmOwNK13HhX5Mc" style="border: none;" width="657" /></span></p><h1 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 20pt;"><span style="font-family: Arial; font-size: 20pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 400; vertical-align: baseline; white-space: pre-wrap;">Building Your Own Image</span></h1><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Docker images are built using a plain text file known as the DockerFile. There are a wide range of options available (see resources below) but the main thing for us is to see how we can use an existing base image, and specify some additional R packages to be installed.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">The first line in a Docker file specifies the base image you want to use. For example you might want to use a base image that contains a Shiny server. In that case, (once you’ve found the image on DockerHub) you specify it in the DockerFile:</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">FROM rocker/shiny:4.0.0</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Note - while you don’t have to specify the tag (version) it’s best practice to do so - if you don’t it will use the latest version - which can be updated without you knowing!</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">You can also specify code to be copied into the image, set the initial working directory, environmental variables and execute commands. In the example below we use the geo-spatial image as the base and install additional R packages.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Note that the process shown below in the RUN command is specific to these R images - these ensure that R packages and dependencies are always installed from a fixed snapshot of CRAN at a specific date.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;"><span style="border: none; display: inline-block; height: 348px; overflow: hidden; width: 624px;"><img height="348" src="https://lh5.googleusercontent.com/S09AZjVA_dkC5pcARoyUUEoYRzvCQ9LVKoRCWhHrnslo6ch82xhtkTYnK3154IFp10W5NM_dphgcpTLdSXU1jfJI290WplI-u1XTCv3_YqIw6Olk8zbeei58QxoD3Nhe15jfO1MC" style="margin-left: 0px; margin-top: 0px;" width="624" /></span></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Once you’ve defined your docker file you create the image and give it a tag. This needs to be run from the same directory as the docker file (note the trailing full stop!). Here we’re saying that the new version of the image is to be tagged as version 1.5:</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">docker build --tag chenobyte/geospatial:1.5 .</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Now your image is built you can create containers with it and push the image back to your repository in DockerHub. This is the bit that can take a while as it’s usually a multi-gigabyte upload:</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 700; vertical-align: baseline; white-space: pre-wrap;">docker push chenobyte/geospatial:1.5</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">And now you can share your work with colleagues as an entire self-contained unit.</span></p><h1 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 20pt;"><span style="font-family: Arial; font-size: 20pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 400; vertical-align: baseline; white-space: pre-wrap;">Summary</span></h1><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">This is just a quick introduction but while Docker has a steep initial learning curve, it’s one that’s worth investing the time in. You don’t need to become an expert to take advantage of massive scale compute power either.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Imagine you had a high performance 10-node cluster that you wanted to run a simulation on. You would have to make sure every node had the same R environment, packages, versions and data. By creating a Docker image it doesn’t matter if it’s a 10-node or 100-node cluster. You only need to specify a run time environment once. </span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">In our recent work we used Docker containers across a 4 x 64 CPU node cluster - this allowed us to fit 17 complete large scale spatial models in parallel while using the best allocation of available cluster resources.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">This was a great realisation of the power of Docker considering all of the model fitting code was developed on a crotchety 4-core five year old mac laptop!</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">There is far more below the surface that could be discussed about Docker but most of it is not really relevant to statisticians. </span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">The main points presented here are hopefully useful for eco stats work and general research.</span></p><br /><h1 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 20pt;"><span style="font-family: Arial; font-size: 20pt; font-variant-east-asian: normal; font-variant-numeric: normal; font-weight: 400; vertical-align: baseline; white-space: pre-wrap;">Resources</span></h1><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Docker Hub - </span><a href="https://hub.docker.com/" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space: pre-wrap;">https://hub.docker.com</span></a></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Docker File Reference - </span><a href="https://docs.docker.com/engine/reference/builder" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space: pre-wrap;">https://docs.docker.com/engine/reference/builder</span></a></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">Docker Reference - </span><a href="https://docs.docker.com/engine/reference/run" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space: pre-wrap;">https://docs.docker.com/engine/reference</span></a></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; vertical-align: baseline; white-space: pre-wrap;">RStudio and Docker - </span><a href="https://support.rstudio.com/hc/en-us/articles/360021594513-Running-RStudio-with-Docker-containers" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Arial; font-size: 11pt; font-variant-east-asian: normal; font-variant-numeric: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space: pre-wrap;">How To: Run RStudio products in Docker containers – RStudio Support</span></a></p><br /><br /><br /><br /><br /><br /></span>Sam Masonhttp://www.blogger.com/profile/11288342690295108154noreply@blogger.com1tag:blogger.com,1999:blog-8223454891323668732.post-13328202290069307402019-03-29T16:29:00.000+11:002019-04-01T11:32:24.876+11:00Paper of the year 2018The paper-of-the-year competition sees Eco-Stats members nominate their favourite article hoping to win "free coffee for a year". This year saw a relaxation of some of the previous requirements - no longer did the paper need to be about ecology or statistics; or even be published; in fact one entry was even from the end of 2017. The result was a wide field - from LEGO investors; to fire wielding sh*t hawks; to the earth getting into a bit of a (blueberry) jam. After much debate the results are in:<br /><br />The winning paper was:<br /><br /><br />Anders Samberg (2018) Blueberry Planet.<br /><br /><a href="https://arxiv.org/abs/1807.10553">arXiv:1807.10553 [physics.pop-ph]</a><br /><br />This paper was nominated by Gordana Popovic because:<br /><br /><i>It shows very unpretentiously what research involves. You have a question, you find all the research in the area, you mush the two together, you get an answer.</i><br /><br /><br />Honourable mentions go to:<br /><br /><br /><a href="https://bioone.org/journals/Journal-of-Ethnobiology/volume-37/issue-4/0278-0771-37.4.700/Intentional-Fire-Spreading-by-Firehawk-Raptors-in-Northern-Australia/10.2993/0278-0771-37.4.700.short">Bonta, M et al. (2017) Intentional Fire-Spreading by “Firehawk” Raptors in Northern Australia. Journal of Ethnobiology</a><br /><br />This paper was nominated by Ben Maslen because:<br /><br /><div>
<i>I like this paper as it outlines a very intriguing and at first glance outlandish ecological behaviour that combines empirical evidence with indigenous ecological knowledge. Birds spreading fires, who knew!</i><br /><br /><br /><a href="https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3291456">Dobrynskaya, Victoria and Kishilova, Julia (2018) LEGO - The Toy of Smart Investors.</a><br /><br />This paper was nominated by Michelle (Shi Jie) Lim because:<br /><br /><i>I like this paper because Lego toys do not belong to the luxury segment and are affordable to most retail investors. Although the returns may not be significant in reality, the study shows that people are willing to pay a premium for Lego sets. Any Lego toy owner would probably find this paper relatable.</i><br /><br /><br />The other nominations in no particular order are:<br /><br /><br /><a href="https://www.tandfonline.com/doi/full/10.1080/01621459.2017.1356319">Zhu and Bradic (2018) Linear Hypothesis Testing in Dense High-Dimensional Linear Models. Journal of the American Statistical Association.</a><br /><br />This paper was nominated by David Warton because:<br /><br /><i>...developing an original new machinery for inference and applying it to a tricky problem, that of simultaneous inference for lots of parameters.</i><br /><br /><br /><a href="https://arxiv.org/pdf/1902.01265.pdf">Miller and Sanjurjo (2018) Surprised by the Hot Hand Fallacy?A Truth in the Law of Small Numbers. Econometrica.</a><br /><br />This paper was nominated by Robert Nguyen because:<br /><br /><i>I think this is interesting because it tackles something that generally people believe (there is a hot hand in sport) but as of yet there is no evidence it exists or is there?</i><br /><br /><br /><a href="https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0200303">Fraser et al. (2018) Questionable research practicesin ecology and evolution. PLoS ONE.</a><br /><br />This paper was nominated by Mitchell Lyons because:<br /><br /><i>I like these types of papers, and there’s been a few lately, including some recent press on some editorials in Nature (Google those if you like. It helps me to gain context on why people (we, and me now) in ecology and evolution think about significance the way they do.</i><br /><br /><br /><a href="https://www.tandfonline.com/doi/full/10.1080/01621459.2018.1473776">Yixin Wang & David M. Blei (2018): Frequentist Consistency of Variational</a><br /><a href="https://www.tandfonline.com/doi/full/10.1080/01621459.2018.1473776">Bayes. Journal of the American Statistical Association.</a><br /><br />This paper was nominated by Elliot Dovers because:<br /><br /><i>I like this paper because it provides a firmer theoretical footing for a technique (more generally than has been previously for variational approximations) that has perhaps been used willy-nilly for a long time in computer science (I'm not innocent here either). I also like that the authors give time to addressing (and linking) the technique with respect to both a frequentist and Bayesian approaches. The result "bridges the gap in asymptotic theory between the frequentist variational approximation, in particular the variational frequentist estimate (VFE), and variational Bayes"... good to see authors with a less binary take in on what it means to be a statistician.</i></div>
Elliothttp://www.blogger.com/profile/10322584416646379812noreply@blogger.com0tag:blogger.com,1999:blog-8223454891323668732.post-2353675604853842402019-03-20T11:21:00.001+11:002019-03-20T11:21:29.227+11:00Template Model Builder TutorialMany of the Eco-Stats group are using Template Model Builder (TMB) - a very flexible package in R for fitting all sorts of latent variable models quickly. For R users without any C++ coding experience, getting familiar with the package might be a little daunting so we've put together a gentle introduction with some simple examples. Follow the link below and get going with TMB:<br />
<br />
<a href="https://www.dropbox.com/s/f5ng7t6pgnvcvmp/TMB_Intro.html?dl=0">TMB Introduction Tutorial</a><br />
<div>
<br />
Note: before installing TMB (by your usual means of installing an R package) compiling C++ code will require a working development environment. In Windows you can just install the latest version of Rtools - follow the install guide <a href="https://github.com/kaskr/adcomp/wiki/Windows-installation">here</a>. If installing on Mac OS or Linux - following the devtools install guide will do the trick - check it out <a href="https://www.r-project.org/nosvn/pandoc/devtools.html">here</a>.</div>
Elliothttp://www.blogger.com/profile/10322584416646379812noreply@blogger.com1tag:blogger.com,1999:blog-8223454891323668732.post-41078743513359140322018-03-08T15:57:00.000+11:002018-03-08T16:03:43.924+11:00Paper of the year 2017<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="text-align: left;">
<span style="font-family: inherit;">The competition for paper of the year 2017 was heated, with the ecostatistician proposing the winning paper scoring the coveted "free coffee for a year" prize, The nominations were diverse, all the way from pure ecology to very fancy stats. </span><span style="font-family: inherit;">After much debate, the <b>winner</b> was:</span><br />
<span style="font-family: inherit;"><br /></span>
<a href="http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0185809" target="_blank"><span style="font-family: inherit;"><b><span style="background-color: white; font-family: "arial"; font-size: 13px;">Hallmann CA, Sorg M, Jongejans E, Siepel H, Hofland N, et al. (2017) </span><span style="background-color: white; font-family: "arial"; font-size: 13px;">More than 75 percent decline over 27 years in total flying insect biomass in protected areas. PLOS ONE 12(10)</span></b></span></a><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">This paper was </span>nominated<span style="font-family: inherit;"> by<a href="https://twitter.com/wilshire_h" target="_blank"> </a><a href="https://twitter.com/wilshire_h" target="_blank"><span style="background-color: white; color: #222222; white-space: nowrap;">John</span> </a></span><span style="background-color: white;"><a href="https://twitter.com/wilshire_h" style="color: #222222; font-family: inherit; white-space: nowrap;" target="_blank">Wilshire</a><span style="color: #222222; font-family: inherit;"><span style="white-space: nowrap;">, who </span></span><span style="color: #222222;"><span style="white-space: nowrap;">summarises</span></span><span style="color: #222222; font-family: inherit;"><span style="white-space: nowrap;"> it as follows:</span></span></span><br />
<a href="http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0185809" target="_blank"><span style="font-family: inherit;"><br /></span></a>
<i><span style="background-color: white; color: #222222; font-family: inherit;">Flying insects play a very important role in ecosystems, both as pollinators and as food sources for other animals. This paper shows that their populations have massively declined over a relatively short period of time (at least in protected areas in Germany). </span><span style="background-color: white; color: #222222; font-family: inherit;">I like this paper as it presents the results of a long term study, and it is a pretty scary example of the impacts we are having on ecosystems. </span><span style="background-color: white; color: #222222; font-family: inherit;">Plus it is open access and has data and code available, and the statistical analysis is presented in a clear and easy to follow manner.</span></i><br />
<div style="background-color: white; color: #222222;">
<br /></div>
<div style="background-color: white; color: #222222;">
<span style="background-color: transparent;">Other nominees were (in no particular order):</span><br />
<span style="background-color: transparent;"><br /></span>
<span style="background-color: transparent;"></span><br />
<a name='more'></a><span style="background-color: transparent;"><br /></span></div>
<br />
<span style="font-family: inherit;"><a href="http://homepages.rpi.edu/~horneb/Drazan_Loya_Horne2017.pdf" target="_blank"><span style="background-color: white;">JF Drazan, AK Loya, BD Horne, R Eglash </span>(2017) From sports to science: Using basketball analytics to broaden the appeal of math and science among youth: <span style="background-color: white;">2017 MIT Sloan Sports Analytics Conference</span></a></span><br />
<span style="background-color: white;"><span style="font-family: inherit;"><br /></span></span>
<span style="font-family: inherit;"><span style="background-color: white;">Nominated by </span><span style="background-color: white; color: #222222; white-space: nowrap;">Robert Nguyen:</span></span><br />
<span style="font-family: inherit;"><span style="background-color: white; color: #222222; white-space: nowrap;"><br /></span></span>
<span style="background-color: white; color: #222222;"><span style="font-family: inherit;"><i>I like it because its a great way to engage people that might not consider a STEM career as a possibility. I like how it gets people involved in the data collection as a starting point for engagement into the subject and goes through to a tangible final outcome for kids. But mainly I love its pro social benefit of combining something kids love (sports) with something they can consider as a future career (STEM).</i></span></span><br />
<span style="background-color: white; color: #222222;"><span style="font-family: inherit;">---------------------------</span></span><br />
<br />
<span style="background-color: white;"><span style="font-family: inherit;"><a href="http://onlinelibrary.wiley.com/doi/10.1111/biom.12540/full" target="_blank">Chi, E. C., Allen, G. I. and Baraniuk, R. G. (2017), Convex biclustering. Biom, 73: 10–19. doi:10.1111/biom.12540</a></span></span><br />
<br />
Nominated by David Warton:<br />
<span style="font-family: inherit;"><i><br /></i></span>
<span style="background-color: white; color: #222222;"><span style="font-family: inherit;"><i>This paper proposed penalised least squares, using a relatively simple fusion penalty, for biclustering of data, to simultaneously cluster observations and variables. They showed desirable statistical and computational properties, it has a cool name (COBRA), and is a simple yet powerful idea. It would be interesting to try and extend this method to handle multivariate abundance data.</i></span></span><br />
<div class="gs_gray" style="background-color: white; border: 0px; color: #777777; margin: 0px; padding: 0px;">
<span style="color: #222222;">---------------------------</span></div>
<div class="gs_gray" style="background-color: white; border: 0px; color: #777777; margin: 0px; padding: 0px;">
<br /></div>
<div class="gs_gray" style="background-color: white; border: 0px; margin: 0px; padding: 0px;">
<span style="background-color: #fcfcfc; letter-spacing: 0.238px;"><span style="font-family: inherit;"><a href="https://link.springer.com/article/10.1007/s10182-017-0301-8" target="_blank">Illian, J.B. & Burslem, D.F.R.P. AStA Adv Stat Anal (2017) 101: 495. </a></span></span></div>
<div class="gs_gray" style="background-color: white; border: 0px; color: #777777; margin: 0px; padding: 0px;">
<span style="background-color: #fcfcfc; color: #333333; letter-spacing: 0.238px;"><br /></span></div>
<div class="gs_gray" style="background-color: white; border: 0px; margin: 0px; padding: 0px;">
<div style="color: black;">
Nomina<span style="font-family: inherit;">ted by <span style="color: #222222; white-space: nowrap;">Elliot Dovers</span>:</span></div>
<div style="color: black;">
<span style="font-family: inherit;"><i><br /></i></span></div>
<div style="color: black;">
<span style="font-family: inherit;"><i>I like it because it gives a nice and fairly concise summary of the current use of spatial point process models within ecological research. While it occasionally sounds a little like bemoaning the lack of use of these models, Illian and Burslem offer a measured thoughts on how statisticians (perhaps even outside these particular models) can engage with practitioners of the applied sciences to enable the use of statistical methodologies that have previously been labelled too difficult or complex.</i></span></div>
<div style="color: black;">
<span style="color: #222222;">---------------------------</span></div>
<div>
<span style="font-family: inherit;"><a href="https://doi.org/10.1073/pnas.1610206114" target="_blank"><br /></a></span></div>
<div>
<span style="font-family: inherit;"><a href="https://doi.org/10.1073/pnas.1610206114" target="_blank">Falster, D. S., Brännström, Å., Westoby, M., & Dieckmann, U. (2017). Multitrait successional forest dynamics enable diverse competitive coexistence. Proceedings of the National Academy of Sciences:</a></span></div>
<div style="color: black;">
<br /></div>
<div style="color: black;">
Nomina<span style="font-family: inherit;">t<span style="font-family: inherit;">ed by </span><span style="color: #222222; white-space: nowrap;">Mitchell Lyons</span>:</span></div>
<div style="color: black;">
<span style="font-family: inherit;"><i><br /></i></span></div>
<div style="color: black;">
<span style="color: #222222;"><span style="font-family: inherit;"><i>I love the way this paper captures the thought process that neutral theory seems like a hand wave that is rather unflattering, but that current niche differentiation theory doesn’t work. So by developing a more complex combination of successional niche models that include more realistic mechanisms (size-structured growth and physiological trade-offs i.e. traits) that can differentiate along 2 axes (most consider only 1), a model based on niche theory can actually predict (~for the first time) the levels of diversity we observe. If you scroll right down, there’s some pretty neat (and accessible, given I can understand it) math behind the expectation of the height distribution of plants over time, and then all the little biological things you need to get there. Impressive in novelty and in shear work putting it all together. There’s also a n R package you can have a play with.</i></span></span></div>
<div style="color: black;">
<span style="color: #222222;">---------------------------</span></div>
<div style="color: black;">
<span style="font-family: inherit;"><i><br /></i></span></div>
<div>
<span style="background-color: #fcfcfc; letter-spacing: 0.238px;"><span style="font-family: inherit;"><a href="https://link.springer.com/article/10.1007/s10182-017-0302-7" target="_blank">Patterson, T.A., Parton, A., Langrock, R. et al. AStA Adv Stat Anal (2017) 101: 399. </a></span></span></div>
<div style="color: black;">
<span style="background-color: #fcfcfc; color: #333333; font-family: "source sans pro" , "helvetica" , "arial" , sans-serif; font-size: 14px; letter-spacing: 0.238px;"><br /></span></div>
<div style="color: black;">
Nomina<span style="font-family: inherit;">t<span style="font-family: inherit;">ed by </span><span style="color: #222222; white-space: nowrap;">Gordana Popovic</span>:</span></div>
<div style="color: black;">
<span style="font-family: inherit;"><i><br /></i></span></div>
<span style="color: #222222;"><i>This paper does a nice job demystifying animal movement models, by describing the most commonly used models, and when they can, and more importantly cannot, be used. It draws some connections between state space models and hidden Markov models, which helped me understand when they can be applied. I also love that the paper manages to be simple enough to read for an ecological audience, but give enough detail for a statistical audience. </i></span><br />
<div style="color: black;">
<span style="font-family: inherit;"><i><br /></i></span></div>
</div>
</div>
</div>
Gordana Popovichttp://www.blogger.com/profile/13277950763972269993noreply@blogger.com0tag:blogger.com,1999:blog-8223454891323668732.post-79661206968368082017-04-13T17:02:00.001+10:002017-04-13T17:02:29.666+10:00Special Feature in Methods in Ecology and Evolution on Eco-Stats '15There is a Special Feature in the April 2017 issue of Methods in Ecology and Evolution reporting outcomes from the Eco-Stats '15 conference, blog post about it here<br />
<br />
<a href="https://methodsblog.wordpress.com/2017/04/12/generating-new-ideas/">https://methodsblog.wordpress.com/2017/04/12/generating-new-ideas/</a><br />
<br />David Wartonhttp://www.blogger.com/profile/05231788956034178551noreply@blogger.com0tag:blogger.com,1999:blog-8223454891323668732.post-28589979602145393722016-10-21T10:22:00.000+11:002016-10-21T13:45:30.836+11:00Three simple things you should know about interpreting linear models<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="background-color: white; color: #212121; margin: 0cm 0cm 0.0001pt; text-align: left;">
<span style="color: black;"><span style="font-family: inherit;">What do the coefficients from linear models actually mean? What does it mean to "control for" another variable. How do I interpret coefficients in the presence of other variables v.s. in a model on their own? What do coefficients mean in a generalized linear model? How do I standerdise data using models? What are offsets, and how do I use them? </span></span></div>
<div style="background-color: white; color: #212121; margin: 0cm 0cm 0.0001pt; text-align: left;">
<span style="color: black;"><span style="font-family: inherit;"><br /></span></span></div>
<div style="background-color: white; color: #212121; margin: 0cm 0cm 0.0001pt; text-align: left;">
<span style="color: black;"><span style="font-family: inherit;">Learn the answers to these and many more questions this Friday, <a href="http://environmentalcomputing.net/event/ecostats-standardise-data/?instance_id=33">October 21 at 2pm</a>. </span></span></div>
<div style="background-color: white; color: #212121; margin: 0cm 0cm 0.0001pt; text-align: left;">
<span style="color: black;"><span style="font-family: inherit;"><br /></span></span></div>
<div style="background-color: white; margin: 0cm 0cm 0.0001pt; text-align: left;">
Materials for this lab can be found <a href="http://environmentalcomputing.net/how-to-interpret-linear-models/" target="_blank">here</a>.</div>
<div style="background-color: white; color: #212121; font-family: "Times New Roman", serif; font-size: 12pt; margin: 0cm 0cm 0.0001pt; text-align: left;">
<span style="color: black; font-family: "calibri" , sans-serif; font-size: 12pt;"><br /></span></div>
<div style="background-color: white; color: #212121; font-family: "Times New Roman", serif; font-size: 12pt; margin: 0cm 0cm 0.0001pt;">
<span style="color: black; font-family: "calibri" , sans-serif; font-size: 12pt;"><br /></span></div>
</div>
Gordana Popovichttp://www.blogger.com/profile/13277950763972269993noreply@blogger.com11tag:blogger.com,1999:blog-8223454891323668732.post-16206157810080321312016-08-26T08:46:00.000+10:002016-08-26T15:52:52.941+10:00Introduction to mixed models with lme4<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="MsoPlainText">
Ever wondered what a mixed model is? Have a nested design
for your experiment and don't know how to analyze it? Confused by fixed and
random effects? What's the deal with lme4?<o:p></o:p></div>
<div class="MsoPlainText">
<br /></div>
<div class="MsoPlainText">
Come and find out the answer to these questions (and
more) in an hour. <o:p></o:p></div>
<div class="MsoPlainText">
<br /></div>
<br />
<div class="MsoPlainText">
Friday 2-3 PM in Biology 640.<o:p></o:p></div>
<div class="MsoPlainText">
<br /></div>
<div class="MsoPlainText">
Materials are available <a href="http://environmentalcomputing.net/mixed-models/" target="_blank">here</a>.</div>
</div>
Gordana Popovichttp://www.blogger.com/profile/13277950763972269993noreply@blogger.com0tag:blogger.com,1999:blog-8223454891323668732.post-34519858472845403142016-08-11T09:24:00.001+10:002016-08-11T09:24:16.301+10:00Congrats to Gordana Popovic for winning a Student Prize for her awesome talk at ISEC in Seattle. She spoke about an algorithm for covariance modelling of discrete data using copulas (which are fun).
For her efforts she got a certificate, an Amazon voucher and one of these...
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjM7zFamxysos4mtyMgXCi4YDQZt0rRutbXsH_HKp-y-h7t3fhetSo4_YUzTPPhebWBkvktPNChwuOoMhy4SD_QnM3V4JbrRkgZWJWrkCdl2PqvxgMPHpwtIQNoh_RUa2VJlzsh47cH5hk/s1600/2016-07-01+13.33.53.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjM7zFamxysos4mtyMgXCi4YDQZt0rRutbXsH_HKp-y-h7t3fhetSo4_YUzTPPhebWBkvktPNChwuOoMhy4SD_QnM3V4JbrRkgZWJWrkCdl2PqvxgMPHpwtIQNoh_RUa2VJlzsh47cH5hk/s400/2016-07-01+13.33.53.jpg" width="400" /></a></div>
David Wartonhttp://www.blogger.com/profile/05231788956034178551noreply@blogger.com0tag:blogger.com,1999:blog-8223454891323668732.post-19780963899173076502016-04-28T17:47:00.002+10:002016-04-28T17:47:27.900+10:00Where to submit your paper?Today at Eco-Stats we discussed the PLoS ONE paper "<a href="http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0115451" target="_blank">Where Should I Send It? Optimizing the Submission Decision Process</a>" which did some mathematical modelling to decide on an optimal approach to choosing the order of journals to send an ecological paper to. The main factors considered were time to acceptance (a function of time to review and acceptance rate) and impact factor of the journal. The authors wrote to the editorial boards of all - yes all - ISI-listed
journals in ecology, and another six general journals (e.g. Science,
PLoS ONE) that publish ecological papers. They got responses from 61
journals, yielding an interesting dataset available as an appendix to
their paper. I've reformatted it <a href="https://cloudstor.aarnet.edu.au/plus/index.php/s/n2b2KCifF00QGjw" target="_blank">as a comma-delimited file here</a>. <br />
<br />
The authors derived a couple of metrics (e.g. to maximise expected citations) under a host of assumptions (which made me somewhat uncomfortable, as modelling papers often do), the endpoint was metrics that could be used to evaluate different publication strategies, e.g. Science then PNAS then Ecology Letters then...<br />
<br />
Their results I found largely unsurprising - they highlighted a few target journals, of the ones they had data on, in particular Ecology Letters, Ecological Monographs and PloS One, which all scored high as compromises between impact factor and time to publication. Interestingly Science didn't come out smelling like roses, although this may be a function of the metrics they used and their implicit assumptions as much as anything else. They didn't have data on all journals, e.g. I would like to know about Nature, Trends in Ecology and Evolution or Methods in Ecology and Evolution. They expressed surprise that a pretty good strategy seemed to be submitting to journals in order of impact factor. They expected a loss of impact due to long times spent in review, I mean you end up bouncing around between journals for years and years. I think in practice that strategy would do worse than their model suggested, for most of us, because it didn't incorporate the positive correlation in outcomes from submitting the same paper to different journals (or more generally, any measure of how significant a given paper actually is).<br />
<br />
Over time I've become more of a statistician than a modeller and so I was especially interested in the data behind this work, and I learnt the most just by looking at the raw data that was tucked away in an appendix. Here are a few choice graphs which explain the main drivers behind their results.<br />
<br />
First, Impact Factor vs time in review:<br />
<br />
<br />
<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEY_iTD9AS_knQ2OhjKjw-gwuMQG_ASeSNvSMxPG3aUo2KUCMNa9nbFP5Q4VRxCdMzpUnwFvOzVrmW2FCCHDBJBl3Z18Ha6FF0IHZON_UxMyN5VUTY7HPpln35L5QcKBtpKYmRiWkEawc/s1600/IFtime.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEY_iTD9AS_knQ2OhjKjw-gwuMQG_ASeSNvSMxPG3aUo2KUCMNa9nbFP5Q4VRxCdMzpUnwFvOzVrmW2FCCHDBJBl3Z18Ha6FF0IHZON_UxMyN5VUTY7HPpln35L5QcKBtpKYmRiWkEawc/s320/IFtime.png" width="320" /></a>There is a decent negative correlation between impact factor and time in review (r=-0.5). For those of us who have submitted a few papers to journals at each end of the spectrum this won't be news. This is presumably one of the reasons why a journal has high impact - faster acceptances has a direct effect on citation metrics, and increases the incentive to submit good papers there.<br />
<br />
The Science journal is a bit of an outlier on this graph - it has the highest impact factor but a pretty average review time, more than twice as long as Ecology Letters, so if you take these numbers at face value (are they measured the same way across journals?), and if 50 days means a lot to you, there is a case for having Ecology Letters as your plan A rather than Science. Hmmm...<br />
<br />
Good journals are towards the top left, and apart from Ecology Letters and Science we also have Ecological Monographs on the shortlist because it has a slightly shorter time in review than most journals with similar impact factors. Although I wonder how large that difference is relative to sampling error (would it come out to the left of the pack next year too?)...<br />
<br />
Next graph is Impact Factor vs Acceptance rate:<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZTEbLfdKOVX-_VMk_t73yLzWmvShL_2dONh6khINWm3j_aMNf37LV-zuGshzh_929dJZp64hi0fvbSfv4AteZkcGJvgUGGlRZKbFQY9z5N0UtN6K6Odkq5j8FuIJjmQ7t0TJ-KloORDk/s1600/IFaccept.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZTEbLfdKOVX-_VMk_t73yLzWmvShL_2dONh6khINWm3j_aMNf37LV-zuGshzh_929dJZp64hi0fvbSfv4AteZkcGJvgUGGlRZKbFQY9z5N0UtN6K6Odkq5j8FuIJjmQ7t0TJ-KloORDk/s320/IFaccept.png" width="320" /></a>There is a slightly stronger negative association this time (r=-0.6). I vaguely remember a bulletin article a few years ago suggesting no relation between impact factor and acceptance rate - that article used a small sample size and made the classic mistake of assuming that no evidence of a relationship means no relationship. Well given some more data clearly there is a relationship. <br />
<br />
This time we are looking for papers towards to top-right. The journal fitting the bill is by far the biggest outlier, PLoS ONE - a journal with a different editorial policy to most that reviews largely for technical correctness rather than for novelty. It ends up with quite a high acceptance rate, and nevertheless manages a pretty high impact factor. But its impact factor was calculated across all disciplines, what is it when limited to just ecology papers?<br />
<br />
So anyway, from looking at the raw data and taking it at face value, what would be your publishing strategy? A sensible (and relatively common) strategy is to first go for a high impact journal (or two) with relatively short turnaround times, which Ecology Letters is known for, and when you get tired/discouraged by lack of success, or when just trying to squeeze a paper out quickly, PLoS ONE is a good option. This is pretty much what the paper said using fancy metrics, I guess it is reassuring to get the same sort of answer from eyeballing scatterplots of the raw data.<br />
<br />
There are a few simplifying assumptions in this discussion and in the paper itself - a key one is that all paper are treated as equal, when in fact some are more likely to be accepted than others, and some are more suited to some journals than others. There are assumptions like citations being the be-all and end-all, and the modelling in the original paper further assumed that the citations a paper will get are a function of the journal it is published in alone, and not to do with the quality of the paper that is published. But it's all good fun and there are certainly some lessons to be learnt here.<br />
<br />
David Wartonhttp://www.blogger.com/profile/05231788956034178551noreply@blogger.com12tag:blogger.com,1999:blog-8223454891323668732.post-24840403658054387262016-04-21T18:29:00.002+10:002016-04-21T18:29:57.226+10:00Structural equation modelingThe Ecostatistics group gathered today for a discussion of the paper "<a href="http://amstat.tandfonline.com/doi/abs/10.1198/016214505000001005" target="_blank">Structural equation models: a review with applications to environmental epidemiology</a>" by Brisa Sanchez et al. from the December 2005 edition of the Journal of the American Statistical Association.<br />
<br />
We were sparked to read this paper due to multiple requests from clients at the <a href="http://www.analytical.unsw.edu.au/stats-central" target="_blank">Stats Central</a> consulting lab, who were attempting to implement structural equation models (SEMs) but did not understand the methodology. An SEM is typically used as a model for multivariate responses that depend on latent variables. The methodology is used mostly in the field of psychometrics, for instance to model someone's score on several tests as reflecting some unobservable latent variable like "spatial intelligence" or "motor skills". The paper by Sanchez et al. provides a nice overview of SEMs for statisticians who don't necessarily keep up-to-date in the psychometrics literature, with two example analyses drawn from the field of environmental epidemiology.<br />
<br />
The first example is a model that explains the concentration of lead in four body tissues (umbilical cord blood, maternal whole blood, the patella, and the tibia tibia) via five environmental covariates (time living in Mexico City, age, use of ceramics for cooking [both long-term and recent], and the concentration of lead in the air). This isn't a case you'd normally think of as requiring the use of an SEM, but the model was structured hierarchically so that some of the response variables were also covariates for predicting the others. The figure below depicts the model's structure, with each arrow indicating a linear model. It was reproduced from the manuscript.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi3AS7JZBAYt2SK2wxg2YplnjfblbnK5te9cZf0bx8mQQfQFV5Hxy33KqHyZWTPIDyfG4-qwkDN5LynIp6gcViw0xak8XscNXPV2pJCyna8aGU9roYG3VyrDGAwaSUfeXbReZGBgZwTOXs/s1600/fig2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="256" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi3AS7JZBAYt2SK2wxg2YplnjfblbnK5te9cZf0bx8mQQfQFV5Hxy33KqHyZWTPIDyfG4-qwkDN5LynIp6gcViw0xak8XscNXPV2pJCyna8aGU9roYG3VyrDGAwaSUfeXbReZGBgZwTOXs/s640/fig2.png" width="640" /></a></div>
<br />
Since there are no latent variables in this model, we wondered whether the result of using an SEM to fit the data jointly will be very different from generating each regression model individually, and using its fitted values as inputs to the next level of the model.<br />
<br />
The other example is a model for performance on eleven neurobehavioral tests with mercury exposure as an unobserved covariate that affects the latent factors "motor ability" and "verbal ability". Whale consumption was used to predict the mercury exposure (this study was conducted in the remote Faeroe Islands north of Scotland), with hair and blood mercury used as surrogates that reflect the mercury exposure. The diagram of this model's structure is reproduced below. Note that the double-ended arrow between motor ability and verbal ability indicates that the two latent factors are correlated. <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhu0823364XLQWg-B9k6WRQZuxcBrGQX0XCNvnzH8Wj9db8mwk6yvKkA-rjXvDvtA9QjuoPOzb8ds0PQzBBwwZf88rzYydaloxL8igvB8kQQ9KQLKf2sShzsCIGcdwvMSx8tjlStfWnjNs/s1600/fig3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="350" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhu0823364XLQWg-B9k6WRQZuxcBrGQX0XCNvnzH8Wj9db8mwk6yvKkA-rjXvDvtA9QjuoPOzb8ds0PQzBBwwZf88rzYydaloxL8igvB8kQQ9KQLKf2sShzsCIGcdwvMSx8tjlStfWnjNs/s640/fig3.png" width="640" /></a></div>
<br />
One thing to note from the diagrams is that these models use a lot of parameters and impose a quite specific structure, both of which can be problematic. The number of parameters leads to concerns about the identifiability of SEMs, which must be carefully checked in each case, and the results may be sensitive to the structure, which must be chosen a priori, using the theory of the scientific field being studied. Model checking is discussed in the paper, but in a general way (necessarily, given that this is a review paper). <br />
<br />
Both examples use linear models and Gaussian distributions at every stage, though SEMs can be used with more flexible model equations such as GLMs. The R package <span style="font-family: "courier new" , "courier" , monospace;">sem<span style="font-family: "georgia" , "times new roman" , serif;"> <span style="font-family: "arial" , "helvetica" , sans-serif;">and the STATA package</span> <span style="font-family: "courier new" , "courier" , monospace;">gllamm</span> <span style="font-family: "arial" , "helvetica" , sans-serif;">are called out as the state of the art (circa 2005).</span></span></span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">We got to discussing when an Ecostatistician might want to use an SEM in their work, and the discussion focused around our favorite application: multivariate abundance modeling (<a href="https://youtu.be/KnPkH6d89l4" target="_blank">mvabund</a>). In fact, most kinds of regression and latent variable models seem to be specific cases of structural equation models (though that's not usually the most productive way to think). </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;"><span style="font-family: "georgia" , "times new roman" , serif;"><span style="font-family: "arial" , "helvetica" , sans-serif;">In sum, we found this paper to be a well-calibrated review article for statisticians who are looking to quickly understand the SEM methodology and see how it can be used in an example.</span></span></span><br />
<span id="goog_895697999"></span><span id="goog_895698000"></span>Unknownnoreply@blogger.com2tag:blogger.com,1999:blog-8223454891323668732.post-52767669673909379562016-02-18T18:13:00.000+11:002016-02-24T14:32:12.046+11:00Paper of the year 2015So we got together recently to discuss the papers we most enjoyed in 2015, and to vote one of them the Ecostats Paper of the Year. Top prize (nominator exempt from buying the next round of coffees or beers) went to James Thorsen's spatial factor analysis paper:<br />
<br />
<br />
<ul>
<li><a href="http://onlinelibrary.wiley.com/doi/10.1111/2041-210X.12359/full" target="_blank">Spatial factor analysis: a new tool for estimating joint species distributions and correlations in species range</a> by James Thorsen et al., Methods in Ecology and Evolution, June 2015.</li>
</ul>
<br />
Joint species distribution models fulfil a need for ecological models that describe how species distributions are simultaneously affected by habitat and communities. Thorsen et al. combined spatial models with latent factors, and applied it to the US Breeding Bird Survey and to a dataset consisting of trawler transects for rockfish along the west coast of the US.<br />
<br />
Other nominees were:<br />
<br />
<ul>
<li><a href="http://www.tandfonline.com/doi/abs/10.1080/01621459.2014.893884" target="_blank">A unifying model for capture-recapture and distance sampling surveys of wildlife populations</a> by David Borchers et al., JASA, March 2015.</li>
</ul>
<br />
Capture-recapture and distance sampling are both common approaches to estimating the size of a population under study. Borchers et al. have developed a statistical framework which unifies the two, by introducing a spectrum for the spatial precision of individual observations.<br />
<br />
<br />
<ul>
<li><a href="http://onlinelibrary.wiley.com/doi/10.1111/2041-210X.12236/full" target="_blank">Model-based approaches to unconstrained ordination</a> by Francis Hui et al., in Methods in Ecology and Evolution, April 2015.</li>
</ul>
<br />
Unconstrained ordination methods are widely used in exploratory analysis to identify similar and dissimilar habitats, communities, or study sites. By developing a model-based framework for ordination, Hui allows ordination to be used for statistical inference rather than purely descriptive uses. <br />
<br />
<ul>
<li><a href="http://onlinelibrary.wiley.com/doi/10.1890/14-0661.1/abstract" target="_blank">A guide to Bayesian model selection for ecologists</a> by Mevin Hooten and Tom Hobbs in Ecological Monographs, February 2015.</li>
</ul>
<br />
A very nice survey of Bayesian model selection methods from an ecological perspective.<br />
<br />
<br />
<ul>
<li> <a href="http://amstat.tandfonline.com/doi/abs/10.1080/01621459.2014.941466" target="_blank">A spatio-temporal point process model for ambulance demand</a> by Zhengyi Zhou et al. in JASA, 2015.</li>
</ul>
<br />
This paper was noted as an excellent example of clear exposition and an unexpected application of point process modeling. <br />
<br />
<br />
<ul>
<li><a href="http://onlinelibrary.wiley.com/doi/10.1111/biom.12299/full" target="_blank">Bayesian function-on-function regression for multilevel functional data</a> by Mark Meyer et al. in Biometrics, September 2015.</li>
</ul>
<br />
Nominated as a change of pace that is likely outside the comfort zone of statistician used to analyzing typical ecological data, Meyer et al. describe a procedure for analysis of data that consist entirely of functional objects.<br />
<br />
<br />
<ul>
<li><a href="http://www.tandfonline.com/doi/abs/10.1080/01621459.2013.823775" target="_blank">Estimation and accuracy after model selection</a> by Brad Efron in JASA, September 2014.</li>
</ul>
<br />
Efron's paper snuck in even though it was published in 2014. Model selection procedures like the LASSO are extremely popular among statisticians and ecologists alike, but the classical likelihood-based standard errors often don't apply post-model-selection. This paper proposes a general resampling-based theory of standard error estimation after model selection.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8223454891323668732.post-54791347946312252512015-09-22T15:46:00.000+10:002015-09-25T12:02:52.934+10:00Analysis of ordinal data in ecology<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="MsoNormal" style="background: white; text-align: left;">
<span style="font-family: inherit;"><span style="color: #151515;">The September Eco-Stats Lab (Friday
25th, 2pm, Bioscience level 6) will be on ordinal data analysis in ecology.</span><span style="color: #151515;"><o:p></o:p></span></span></div>
<div style="text-align: left;">
</div>
<div class="MsoNormal" style="background: white; text-align: left;">
<span style="color: #151515;"><span style="font-family: inherit;"><br /></span></span></div>
<span style="font-family: inherit;"><span style="color: #151515;">
Ordered categorical data are commonplace in ecology when quantitative
measurement of a variable of interest is not feasible or too costly. Examples
include size of individuals, body condition and relative abundances of species.</span><span style="color: #151515;">
Cumulative link models are a powerful class of models for analyzing such data
since observations are treated as categorical, the ordered nature is exploited
and the flexible regression framework allows in-depth analyses.</span><span style="color: #151515;"><span style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial;">We will use the<span class="apple-converted-space"> </span></span><span style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial;"></span>ordinal </span><span style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; color: #151515;">package in R to analyse some ecological ordinal data.</span></span><br />
<span style="font-family: inherit;"><span style="background-attachment: initial; background-clip: initial; background-image: initial; background-origin: initial; background-position: initial; background-repeat: initial; background-size: initial; color: #151515;"><br /></span></span>
The code and details can be found <a href="http://web.maths.unsw.edu.au/~loicthibaut/ordinalLab.html" target="_blank">here</a>.</div>
Gordana Popovichttp://www.blogger.com/profile/13277950763972269993noreply@blogger.com1tag:blogger.com,1999:blog-8223454891323668732.post-66452994103444710332015-09-10T09:19:00.003+10:002015-09-10T09:19:41.952+10:00Welcome to Loic and Wesley!We are very fortunate to have two new research associates joining us in Eco-Stats - Loic Thibaut and Wesley Brooks.<br />
<br />
Loic came from a PhD at James Cook with Sean Connelly, and he will be thinking about how to resample generalised linear mixed models.<br />
<br />
Wesley came here from Wisconsin, where he did a PhD with Jun Zhu, and he will be looking at the issue of spatial confounding and how it affects point process models.<br />
<br />
<br />David Wartonhttp://www.blogger.com/profile/05231788956034178551noreply@blogger.com0tag:blogger.com,1999:blog-8223454891323668732.post-31881306309039089212015-08-25T10:25:00.000+10:002015-08-28T15:30:44.646+10:00Time to event analysis (Survival analysis) in ecology<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="MsoNormal" style="background-color: white; margin-left: 0cm; margin-right: 0cm;">
<span style="font-family: inherit;"><span style="color: #151515; line-height: 18.4799995422363px;">The August Eco-Stats Lab (Friday 28th, 2pm, Bioscience level 6) will be on t</span><span style="background-color: transparent; line-height: 18.4799995422363px;"><span style="color: #151515;">ime to event analysis in ecology.</span></span></span></div>
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">When modelling the time taken for an event to happen (e.g.death) we often use time to event analysis (survival analysis) rather than regression models. A common feature of these data is that for some subjects the even did not happen at all during the study period, called (right) censoring, and survival analysis can elegantly incorporate the information from these subjects. Regression models on the other hand have no easy way to include these subjects.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Survival analysis can often be applied to ecological data, e.g.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">- Arrival of a parasite</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">- Survival times for animals/plants </span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">- Germination timing</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">- Response to stimulus</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">- How long fruit remain on plants before they are eaten</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">We will use the </span><span style="font-family: Courier New, Courier, monospace;">survival </span><span style="font-family: inherit;">package in R to analyse some ecological time to event data. You can find the code/</span>explanation<span style="font-family: inherit;"> <a href="http://web.maths.unsw.edu.au/~dwarton/Surv.html" target="_blank">here</a>, and the data </span><a href="https://www.dropbox.com/s/oximx5cwyqkl824/flowers.csv?dl=0" style="font-family: inherit;" target="_blank">here</a><span style="font-family: inherit;">.</span></div>
Gordana Popovichttp://www.blogger.com/profile/13277950763972269993noreply@blogger.com16tag:blogger.com,1999:blog-8223454891323668732.post-91140298932692500192015-08-19T14:13:00.002+10:002015-08-19T14:36:11.780+10:00Workshop - Introduction to Regression Modelling on R, December 6-7We will be running a two-day workshop at UNSW in the lead-up to the Eco-Stats conference in December. This course is aimed at ecologists who recall some intro stats and want to get up to speed on more modern methods of modelling data using R.<br />
<br />
The core idea in the course is to recognise that most statistical methods
you use can be understood under a single framework, as special cases of
(generalised) linear models - including linear regression, t-tests,
ANOVA, ANCOVA, logistic regression and chi-square tests. Learning these
methods in a systematic way, instead of as a "cookbook" of different
methods, enables a systematic approach to key steps in analysis (like
assumption checking) and extension to handle more complex situations you
might encounter in the future (random factors, multivariate analysis,
choosing between a set of competing models).<br />
<br />
The course will be taught by the UNSW Eco-Stats group (it will be led by Francis Hui and Gordana Popovic, with contributions by David Warton and others).<br />
<br />
Register via the conference website at <a href="http://www.eco-stats.unsw.edu.au/register.html">http://www.eco-stats.unsw.edu.au/register.html</a>David Wartonhttp://www.blogger.com/profile/05231788956034178551noreply@blogger.com0tag:blogger.com,1999:blog-8223454891323668732.post-70534362965342877272015-07-28T14:12:00.001+10:002015-07-31T07:48:44.703+10:00Forecasting with time series data <div class="MsoNormal" style="background-color: white; color: #151515; font-family: Calibri, sans-serif; font-size: 11pt; margin-left: 0cm; margin-right: 0cm;">
<span lang="EN-US" style="color: #111111;"><b>Day:</b> Friday 31st July 2pm</span></div>
<div class="MsoNormal" style="background-color: white; color: #151515; font-family: Calibri, sans-serif; font-size: 11pt; margin-left: 0cm; margin-right: 0cm;">
</div>
<div class="MsoNormal" style="background-color: white; color: #151515; font-family: Calibri, sans-serif; font-size: 11pt; margin-left: 0cm; margin-right: 0cm;">
<span lang="EN-US" style="color: #111111;"><b>Location:</b> Bioscience Level 6</span></div>
<div class="MsoNormal" style="background-color: white; color: #151515; font-family: Calibri, sans-serif; font-size: 11pt; margin-left: 0cm; margin-right: 0cm;">
</div>
<div class="MsoNormal" style="background-color: white; margin-left: 0cm; margin-right: 0cm;">
<div style="color: #151515; font-family: Calibri, sans-serif; font-size: 11pt;">
<span lang="EN-US" style="color: #111111;"><b>Topic:</b> Forecasting with time series data</span></div>
<div style="color: #151515; font-family: Calibri, sans-serif; font-size: 11pt;">
<span lang="EN-US" style="color: #111111;"><br /></span></div>
<div style="color: #151515; font-family: Calibri, sans-serif; font-size: 11pt;">
<span lang="EN-US" style="color: #111111;"><br /></span></div>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">###########################################################################</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">## Forecasting with Times Series Data.</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">## We will use Rob Hyndman's forecast R-package.</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">## For more details on the package and time-series forecasting in general, </span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">## see https://www.otexts.org/fpp</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">library(forecast)</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"></span></span><br />
<a name='more'></a><span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">##########################################################################</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">## Example 1.</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">## Our first example will use annual rainforest loss (ha) data collected </span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">## by satellite imagery for Tocantins, Brazilian Amazonia.</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">DF <- c(1650,730,580,440,409,333,333,797,320,273,576,216,244,189,212,</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"> 156,158,271,124,63,107,61,49,40,52)</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">time <- c(1988:2012)</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">## First, we set-up the data in time series form.</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">data1 <- ts(c(t(DF)), start = 1988, end = 2012, frequency = 1)</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">data1</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">## We then split the data into training and test. Remember we</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">## want to forecast 2010-2012 (this will be our test data).</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">data_train <- window(data1, start = 1988, end = 2010-.1, frequency = 1)</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">data_test <- window(data1, start = 2010, end = 2012, frequency = 1)</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">## And finally we will fit a simple exponential smoothing model.</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">fit1 <- ets(data_train, model = "ANN") # Note the "ANN".</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">year_forecast <- 3</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">fit1_forecast <- forecast(fit1, h = year_forecast) # Forecast.</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">plot(fit1_forecast) # Plot the forecast.</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">points(time[23:25], DF[23:25], lty = 1, col = "black", lwd = 4, pch = 0)</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">legend("topright", legend = c("actual", "forecasted"), lwd = 4, </span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"> col = c("black","blue"), lty = c(0,0), merge = TRUE, </span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"> bty = "n", pch = c(0,19), cex = 1.3);</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">## The value for alpha was estimated to be 0.7587.</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">fit1_forecast$model</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">## Other model structures.</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">## From our figure we see a strong downward trend, so we could add an </span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">## additive trend in the model.</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">fit2 <- ets(data_train, model = "AAN") # Note the "AAN".</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">fit2_forecast <- forecast(fit2, h = year_forecast)</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">par(mfrow = c(1,2))</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">plot(fit1_forecast)</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">points(time[23:25], DF[23:25], lty = 1, col = "black", lwd = 4, pch = 0)</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">plot(fit2_forecast) # Plot the "new" forecast.</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">points(time[23:25], DF[23:25], lty = 1, col = "black", lwd = 4, pch = 0)</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">legend("topright", legend = c("actual", "forecasted"), lwd = 4, </span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"> col = c("black","blue"), lty = c(0,0), merge = TRUE, </span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"> bty = "n", pch=c(0,19), cex = 1.3);</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">## Or just let the AIC choose the best model.</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">fit3 <- ets(data_train)</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">fit3_forecast <- forecast(fit3, h = year_forecast) # Forecast it.</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">par(mfrow = c(1,3))</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">plot(fit1_forecast) </span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">plot(fit2_forecast) </span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">plot(fit3_forecast) # Plot the "new" forecast.</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">points(time[23:25], DF[23:25], lty = 1, col = "black", lwd = 4, pch = 0)</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">legend("topright", legend = c("actual", "forecasted"), lwd = 4, </span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"> col = c("black","blue"), lty = c(0,0), merge = TRUE, </span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"> bty = "n", pch = c(0,19), cex = 1.3);</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">## We can also predict future forest cover loss (so no test data).</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">data_train <- window(data1, start = 1988, end = 2012-.1, frequency = 1)</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">fit4 <- ets(data_train)</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">fit4_forecast <- forecast(fit4, h = year_forecast)</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">par(mfrow = c(1,1))</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">plot(fit4_forecast)</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">##########################################################################</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">## Example 2.</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">## Time Series data with seasonality. Example/data taken from </span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">## Hyndman and Athanasopoulos (2014), see https://www.otexts.org/fpp</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">data(gas) # Australian monthly gas production.</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">par(mfrow = c(1,1))</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">plot(gas)</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">## No seasonality.</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">fit_gas1 <- ets(gas, model = "ANN")</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">year_forecast <- 3*12 # 3 years times 12 months.</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">fit_gas1_forecast <- forecast(fit_gas1, h = year_forecast)</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">plot(fit_gas1_forecast) </span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">## With seasonality.</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">fit_gas2 <- ets(gas, model = "ANA")</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">fit_gas2_forecast <- forecast(fit_gas2, h = year_forecast)</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">par(mfrow = c(1,2))</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">plot(fit_gas1_forecast) </span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">plot(fit_gas2_forecast) </span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">## Let ets choose the best model using AIC.</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">fit_gas3 <- ets(gas)</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">fit_gas3_forecast <- forecast(fit_gas3, h = year_forecast)</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">par(mfrow = c(1,3))</span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;"><br /></span></span>
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">plot(fit_gas1_forecast) </span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">plot(fit_gas2_forecast) </span></span><br />
<span style="font-size: 14.6666660308838px;"><span lang="EN-US" style="color: #111111; font-family: Calibri, sans-serif;"></span></span><br />
<span style="color: #111111; font-family: Calibri, sans-serif;"><span style="font-size: 14.6666660308838px;">plot(fit_gas3_forecast)</span></span></div>
Jakub Stoklosahttp://www.blogger.com/profile/05914888864872413389noreply@blogger.com0tag:blogger.com,1999:blog-8223454891323668732.post-1456866140439585622015-06-12T08:15:00.003+10:002016-04-13T10:26:09.290+10:00It's boral time!<div class="MsoNormal" style="font-family: Calibri, sans-serif; font-size: 11pt; margin-left: 0cm; margin-right: 0cm;">
<u><span lang="EN-US" style="color: #111111;">Details for the next R-lab are now available:</span></u></div>
<div class="MsoNormal" style="font-family: Calibri, sans-serif; font-size: 11pt; margin-left: 0cm; margin-right: 0cm;">
</div>
<div class="MsoNormal" style="font-family: Calibri, sans-serif; font-size: 11pt; margin-left: 0cm; margin-right: 0cm;">
<span lang="EN-US" style="color: #111111;"><b>Day:</b> Friday 19th June 2pm (it's been pushed a week early for conference reasons)</span></div>
<div class="MsoNormal" style="font-family: Calibri, sans-serif; font-size: 11pt; margin-left: 0cm; margin-right: 0cm;">
</div>
<div class="MsoNormal" style="font-family: Calibri, sans-serif; font-size: 11pt; margin-left: 0cm; margin-right: 0cm;">
<span lang="EN-US" style="color: #111111;"><b>Location:</b> Bioscience Level 6 </span></div>
<div class="MsoNormal" style="font-family: Calibri, sans-serif; font-size: 11pt; margin-left: 0cm; margin-right: 0cm;">
</div>
<div class="MsoNormal" style="font-family: Calibri, sans-serif; font-size: 11pt; margin-left: 0cm; margin-right: 0cm;">
<span lang="EN-US" style="color: #111111;"><b>Topic:</b> boral -- A R package for bayesian analysis of multivariate abundance data in ecology</span></div>
<div class="MsoNormal" style="font-family: Calibri, sans-serif; font-size: 11pt; margin-left: 0cm; margin-right: 0cm;">
</div>
<div class="MsoNormal" style="font-family: Calibri, sans-serif; font-size: 11pt; margin-left: 0cm; margin-right: 0cm;">
<span lang="EN-US" style="color: #111111;"><b>Links: </b></span></div>
<div class="MsoNormal" style="font-family: Calibri, sans-serif; font-size: 11pt; margin-left: 0cm; margin-right: 0cm;">
<span lang="EN-US" style="color: #111111;">1) Presentation slides <a href="https://www.dropbox.com/s/0hyxozxplxd3myw/presentationv0.pdf?dl=0">Go here</a></span></div>
<div class="MsoNormal" style="font-family: Calibri, sans-serif; font-size: 11pt; margin-left: 0cm; margin-right: 0cm;">
<span lang="EN-US" style="color: #111111;">2) R script: <a href="https://www.dropbox.com/s/ar81n581xi93rfv/script.R?dl=0">Go here</a></span></div>
<div class="MsoNormal" style="font-family: Calibri, sans-serif; font-size: 11pt; margin-left: 0cm; margin-right: 0cm;">
<span lang="EN-US" style="color: #111111;"><br /></span></div>
<div class="MsoNormal" style="font-family: Calibri, sans-serif; font-size: 11pt; margin-left: 0cm; margin-right: 0cm;">
<span lang="EN-US" style="color: #111111;"><b>Notes: </b>Please be aware that we will be using the MCMC package JAGS, as well as the R package boral and mvabund. If you are using your own laptop, then you could save some time by installing those prior to coming. Thanks!</span></div>
<div class="MsoNormal" style="font-family: Calibri, sans-serif; font-size: 11pt; margin-left: 0cm; margin-right: 0cm;">
<span lang="EN-US" style="color: #111111;"><br /></span></div>
<div class="MsoNormal" style="font-family: Calibri, sans-serif; font-size: 11pt; margin-left: 0cm; margin-right: 0cm;">
<span lang="EN-US" style="color: #111111;">FH</span></div>
FHhttp://www.blogger.com/profile/16928359879844308224noreply@blogger.com12tag:blogger.com,1999:blog-8223454891323668732.post-54983068425069830422015-05-27T10:36:00.000+10:002015-07-03T14:29:14.431+10:00Missing Data Analysis<div class="MsoNormal" style="font-family: Calibri, sans-serif; font-size: 11pt; margin-left: 0cm; margin-right: 0cm;">
<span lang="EN-US" style="color: #111111;">The May Eco-Stats Lab (Friday 29th, 2pm, Bioscience level 6) will be on the missing data analysis, using the method of Multiple Imputation.<o:p></o:p></span></div>
<div class="MsoNormal" style="font-family: Calibri, sans-serif; font-size: 11pt; margin-left: 0cm; margin-right: 0cm;">
<br /></div>
<div class="MsoNormal" style="font-family: Calibri, sans-serif; font-size: 11pt; margin-left: 0cm; margin-right: 0cm;">
One often encounters missing data in almost all types of studies. Ecological data is also commonly subject to missing data. However, most of the statistical analysis methods are designed for complete datasets. A common way to handle missing data is to remove cases with missing values in order to obtain a complete dataset, which reduces the sample size and thus the statistical power. This approach can result in biased estimates for descriptive statistics and regression coefficients as well. An alternative approach is to impute (fill-in) the missing data by plausible values multiple times, analyse each imputed dataset separately, and then combine the results together. This method is called Multiple Imputation (MI) and was proposed by Rubin (1987).<o:p></o:p></div>
<div class="MsoNormal" style="font-family: Calibri, sans-serif; font-size: 11pt; margin-left: 0cm; margin-right: 0cm;">
<br /></div>
<div class="MsoNormal" style="font-family: Calibri, sans-serif; font-size: 11pt; margin-left: 0cm; margin-right: 0cm;">
In this lab we will explore the method of MI implemented in the <i>mice</i> package (van Buuren & <span lang="EN-US">Groothuis-Oudshoorn, 2011</span>), which stands for multivariate imputation by chained equations. For more details see<o:p></o:p></div>
<a href="http://www.jstatsoft.org/v45/i03/" style="color: #954f72; font-family: Tahoma; font-size: 13px;" target="_blank">http://www.jstatsoft.org/v45/i03/</a><br />
<br />
Click <a href="http://web.maths.unsw.edu.au/~dwarton/missingDataLab.html" target="_blank">here</a>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-8223454891323668732.post-25229881114442786802015-05-06T11:47:00.001+10:002015-05-06T11:47:07.256+10:00Traits, community ecology and demented accountantsI've added <a href="https://methodsblog.wordpress.com/2015/04/29/traits-community-ecology-and-demented-accountants/" target="_blank">a post on trait modelling on the Methods blog</a>, title as above, to coincide with the April MEE Special Issue from the 2013 Eco-Stats Symposium.<br />
<br />
You'd be surprised how hard it is to find a Creative Commons image of people in suits in the field...David Wartonhttp://www.blogger.com/profile/05231788956034178551noreply@blogger.com0