1 Forecasting in R workshop

This is a free workshop to support the learning and application of forecasting using R in the NHS across the U.K. in collaboration with NHS-R community. The workshop is initially designed for 2 days, 09:00- 16:30.

We are offering five workshops in 2020 as per below:

I taught the first one in February 2020 in Nottinghamshire NHS Foundation Trust in person. Following the pandemic and a discussion with organisers, I have decided to deliver the next one virtually. This was my first experience delivering a 2-day workshop (lectures and tutorials) online.

To deliver the workshop virtually, we had to make couple of decisions and plan accordingly.

2 Pre-workshop decisions and planning

I had only four weeks to prepare for the workshop with a lot to adopt. Furthermorew, I needed to learn new tools and technologies for online teaching. I briefly discuss the decisions made and the tasks planned as following:

1. Number of sessions and time slots: We decided to deliver six 2-hour sessions, 10:00-12:00 a.m. (BST) each day starting from Monday 22^nd of June.

2. Socialising: We decided to start the MS Teams meeting at 09:30 A.M. to give half-an hour for socialising.

3. Video calls: NHS-R community advised us to use MS Teams platform for video communication. The video calls were not recorded.

4. Q&A and communication with participants: We used Slack to communicate with participants during the workshop. It helped to with Q/A and discussing any issues participants were facing. See a brief introduction to Slack here

5. Workshop website: We had to create a central hub for the workshop where participants can access anything they need at anytime. I have created a website for that purpose, which was again a new experience. It allowed participants to access syllabus, slides, lab sessions, data, reading list, etc in one place.

6. R Software: We used R to work on lab exercises. Before attending the workshop, participants are asked to install R, RStudio and all packages required in the workshop. There are always few that show up unprepared. RStudio Cloud is very helpful to avoid any software related issues. It is a cloud-based platform provided and maintained by RStudio which is accessible through a web browser and we can preinstall required packages for all participants.

7. Organisers emailed everyone with a calendar invite that included i) the MS Teams link ii) link to RStudio Cloud iii) Link to Slack iv) Link to the website and v) A Google Doc. (be sure it is editable for anyone!) for the pre-workshop task.

Apart from the initial email and calendar invite, all communications were in Slack and MS Teams.

3 Tools I had to learn

I had to learn few new tools to be able to prepare the workshop and deliver it virtually, herte is the list of tools and some online resources that I used to lean them:

1. Blogdown: I have created a personal website using Blogdown in the past which gave me confidence in creating the website for this workshop. But, I still had to learn what are the different ways to share materials and which one I have to choose. Another great resource from RStudio which helped me to get an overview of potential options and how to do it. The video is accessible : Sharing on Short Notice: How to Get Your Materials Online With R Markdown.

2. RStudio Cloud: Learning RStudio Cloud was pretty easy, I had to create a base project and install all packages required in the workshop and then share the link with participants. Then, they could get a copy of the project I created and work on it. This was a relief as I was not worried whether they have already installed the software and packages! Thank you RStudio! I watched a webinar to learn how to work with RStudio. It can be found here: Teaching R online with RStudio Cloud

3. Slack: This was my first experience using Slack, so I had to learn how to use its features! You can find plenty of good blogs written by Slack team in Slack 101.

4 Delivering the workshop

We have divided each session into three block of 60 minutes- 10 minutes break - 50 minutes, which included both lectures and labs to apply the concepts discussed in lectures using R.

4.2 Lectures

The most challenging part of the workshop was delivering live lectures(synchronous). Apart from one session that was recorded prior to the workshop and uploaded in the website, all other six sessions were delivered live. For each lecture, I talked through slides to explain the concept and explicitly asked students to take notes of some specific parts of the lecture which are important. Students could also ask questions in the Slack and the chat room of MS Teams.

An important aspect of any online teaching is the learner engagement and the success of it depends on how effectively you interact with the learner. To interact with participants during the lectures, we have used the following tools and strategies:

1. Opening: Opening is used to prepare learners for the workshop, connect them to each other and bring the learner’s attention into the workshop. This could happen in many different ways. I asked participants to i) identify a planning/decision task in their organisation ii) reflect on how forecasting can be helpful in supporting that particular task iii) determine what to forecast and iv) what they need to learn to produce that forecast. Participants were asked to share their thoughts in a Google Doc. that have been discussed in the first live session. This helped to set the learneing objectives.

Figure 4.1: Google Doc. used for a pre-workshop task and opening

2. Review, review and review again and again: Regular reviews in each session are vital for learners. I tried to ensure that key topics are reviewed at least four times throughout the workshop. The key here is two things: i) to use a variety of ways without referring to them as a review to the learners and ii) the review should be done by the learner and not by the instructor wherever possible!

• We used both Direct Pool and Mentimeter to ask multiple choice questions. Participants engaged very well with this type of engagement.

Figure 4.2: Multiple choice question with Mentimeter

• Use Padlet to review content by learner. Below you can see two examples where I asked participants to write the most three important thing they learnt in each session and I also provided key summaries of what I covered till session 4.

Figure 4.3: Using a Padlet wall to write the most three important things

Figure 4.4: Using a Padlet shelf to summarises the practical steps in forecasting task in R

• I also asked participants to create a mind map to summarise what they have learnt in the workshop and share it with others. They could use various platforms such as Draw, Mind master or Coggle.

Figure 4.5: Using mind map to review the whole content

• Moreover, I created a summary of key points after six sessions and share it with participants! this may work best after you finish half of the sessions. You can see the summary here.

• I have also provided incomplete piece of R code (with mistakes to correct) with blanks to complete for a particular task in forecasting.

Figure 4.6: R code to complete

4. Whiteboard: Uisng whiteboard is a wonderful way to communicate an idea, explain a complicated concept or describe how an algorithm works! I use Open Board in this workshop to explain time series cross validation and correlation in multiple regression and I found it very powerful. You can also use Good Notes if you are a Mac user.

Figure 4.7: Using white board to explain time series cross validation concept

5 What worked well

• Creating the website as a central hub for the workshop was a brilliant choice. Participants could access all required information and materials during the workshop. I am going to improve it over time, and it will remain accessible for participants.

• It is crucial to avoid multiple communication channels! Slacks was a great choice to that end. We used it for Q/A, announcements, sharing links, creating polls! I found it very structured and easy to use! It has many features which I have not yet used!

• Using the whiteboard was extremely helpful to convey ideas effectively and I recommend using it in online teaching whenever appropriate!

• Using Padlet, Direct Pool and Mentimeter worked very well to interact with the audience.

• RStudio Cloud helped to overcome many issues I have been facing in forecasting workshops. With RStudio Cloud I don’t need to be worried about attendees showing up without installing the software and required packages.

• I was fortunate to have get the support from Thomas Jemmett and Alysia Dyke (The Strategy Unit.) before and during the workshop. I think having a knowledgeable facilitator in the subject area would help alot, who does not necessarily need to be an expert in the field. It can help with the platform, technological environment, handling Q/A sessions.

6 What could be done better and future plans

In this section, I will briefly describe what I could have done better and what I plan to do to improve.

1. Avoiding too much content

   I had too much content to cover for the online session. I could prioritise my content for each topic and each task based on four criteria: i) How frequently the task is used/performed in real life? ii) How easy it is to learn how to do it? iii) How important is the task to the effectiveness of the job? iv) What is the current level of skills and knowledge of learners? and v) What is the allocated time for each session and total cumulative time. Following these criteria, I am going to divide my content into Must to Know which will be covered in the workshop and Good to Know which will not be covered but materials will be shared with participants. First, you need to assign scores to each criterion and calculate the total score for each task. Next, calculate allocated time and its cumulative value for the same task. Then, sort the table based on the total score(highest to lowest) and look at the cumulative time. If the cumulative time is greater than the total allocated time for the session, then put the topic in “Good to Know”, otherwise put it into “Must to Know”. You can download the template and use it.

You can tweak the criteria in the template to whatever is best for your context, e.g. Executive, Postgraduate, Undergraduate, Webinar, etc

2. Using a teaching plan for each session

   After prioritising your content and allocate your topics into “Must to know” and Good to know", you can start the planning for each session. It is extremely important to start the preperation at least 8 weeks before the start day of the teaching. I have created a template which I am going to use to prepare a detail plan for each session. The template can be downloaded here.

As I reflect more on session plan, I find it extremely useful to allocate time to each task, figuring out what you and leaners do during each task and which feature(e.g. Google Doc., Mentimeter, Padlet, etc) is the best to evaluate the learning outcome for that particular task.

3. Including more pre-recorded videos (asynchronous)

   I could include more pre-recorded videos to cover the main content of each session for participants to watch before each session. These videos should cover lecture slides but also it should include some questions to help students think about the content and reflect on it. Moreover, students should be encouraged to write down any question they have about the content. Both types of questions should be discussed/answered during the live session.

4. Including interactive R sessions

   For the lab sessions, I could improve the interaction with learners by creating interactive exercises with learnr package and/or fill in the blank with the flair package. I will consider this for the next workshop.

5. Introducing scenarios for each session

   Scenarios put learners in realistic situations and allow learners to practice the new acquired skills to address a challenge. I am going to include at least one scenario for each session in the future. Another practice could be asking participants to find a public available data set or use their own data set and then define a scenario based on that. This will help participants to practice their skills on a problem they are interested in.

6. Using group discussion and collaboration among learners

   We use MS Teams for video calls. Although it has Breakout rooms feature but we were not able to use it in order to divide learners into smaller groups to work together either to answer a question or review contents. I need to be sure the platform we use will allow the Breakout room feature for the future workshops.

7. Using an additional monitor

   During the online session, I was juggling between different screens on my laptop. Having an additional (or two) monitor could give me a dedicated space for things like Slack to see students questions easier.

7 Pros and cons based on this experience

It is a challenging task to transform a face-to-face training into a virtual format, but it can be done effectively with a good preparation and heart. Both online and face-to-face teaching have their own and pros and cons.

8 What participants said about the workshop

This section will be updated once I receive the feedback from participants.

9 Thank you

I have to say this would not have been possible without having the support from Thomas Jemmett and Alysia Dyke, thank you both for your support in planning the workshop and helping with the technical aspects during the workshop. Thomas is an expert in R and he also helped answering R related questions.

I had Ivan Svetunkov from Centre for Marketing Analytics and Forecasting, Lancaster University as a guest speaker on Exponential Smoothing model (ETS). He delivered a fantastic session, Thank you Ivan for your contribution!

Thank you NHS-R community for organising these workshops and looking forward to the upcoming ones.