Integrating Pavlovia and Shiny for Near Real-Time Automated Data Analysis and Visualisation

This blog post explains how to automatically analyse and visualise data collected from Pavlovia in near real-time using Shiny.

 

For teaching purposes or public engagement events, it can be very helpful to visualise results immediately after a participant has completed an experiment. Here, we describe a pipeline that integrates Pavlovia and Shiny to achieve this aim.

Pipeline overview

Briefly, this is how the pipeline works:

• Completing an experiment on Pavlovia triggers a webhook set up on gitlab.pavlovia.org.
• The webhook is received by a PHP script running on an Apache server.
• The PHP script triggers a git pull for the repository.
• This copies new result files to a directory that can be accessed by the Shiny server.
• When the Shiny app is launched, it reads all Pavlovia result files currently available, analyses the data and visualises them.

Here is a flowchart visualising the pipeline:

flowchart TD
    subgraph pavlovia["pavlovia.org"]
        A[Experiment completion]
    end

    subgraph gitlab["gitlab.pavlovia.org"]
        B[Repository update]
        C[Webhook]
    end

    subgraph server["server"]
        D[Webhook receiver script]
        E[git pull]
        F[Up-to-date copy of repository]
        G[Shiny app]
    end

    A -->|triggers| B
    B -->|triggers| C
    C -->|triggers| D
    D -->|triggers| E
    E -->|creates| F
    G -->|reads| F

    %% Style subgraphs
    classDef subgraphStyle fill:none,stroke:#000000,stroke-dasharray:5 5
    class pavlovia,gitlab,server subgraphStyle

    %% Style subgraph labels
    classDef subgraphLabelStyle fill:#ffffff,color:black,font-weight:bold
    class pavlovia,gitlab,server subgraphLabelStyle

PsychoPy & Pavlovia

The PsychoPy experiment can be created as usual. An optional addition is the inclusion of the URL of the Shiny app as “Completed URL” under “Edit Experiment Settings” → “Online”:

If the completion URL is added, the participant will be automatically forwarded to the Shiny app after completing the experiment.

To illustrate the processing pipeline, we have implemented simple and choice reaction time tasks in PsychoPy and have made the experiment publicly available on Pavlovia.

If you would like to have a go, click here to run the tasks on Pavlovia. The tasks should take about two minutes to complete. Please note that you need a hardware keyboard to complete the tasks.

GitLab

This section describes how to set up your webhook on gitlab.pavlovia.org.

If you’re viewing your experiment on Pavlovia, click on “View Code” to go to your repository on gitlab.pavlovia.org:

Once you’re viewing the code, go to “Settings” and “Webhooks” to set up the webhook:

Add the path to your webhook receiver script under “URL” and, for security purposes, also add a secret token:

Test the webhook using “Test” → “Push events” to make sure it works correctly. If it does, you should see this:

Having the repository for our experiment publicly available (a) avoids that the webhook receiver script has to authenticate against the GitLab server and (b) allows you (the readers of this blog post) to see how the experiment was set up.

If making the repository public is not an option for you, the script pulling the repository would need to be set up in a way that allows it to authenticate against the GitLab Server. The simplest way to achieve this would probably be to create a Personal Access Token (PAT) on GitLab and to store this securely in an environment variable on your server.

On your server

Instructions on how to set up Shiny Server can be found on Posit’s support pages.

When something changes in your experiment repository (e.g., when a participant completes the experiment and a new result file is added to the repository), the webhook will be triggered. You will need a web server that can receive the https request from the webhook.¹ As we had previously set up a public-facing Shiny Server on Linux (which is serving Shiny apps to the web using Apache), Apache was an obvious choice for us to receive the webhooks on. Therefore, below we will describe how to set up a system using Linux, Apache, and Shiny Server, all installed on the same server.

First, you will need a directory configured to allow serving of scripts via Apache. Based on the default settings, this could be under /var/www/cgi-bin/. This is where the script will go that receives the webhook. We opted to use a PHP script for this purpose, but the script could be written in any language (so long as there is support for it on your web server).

Second, you will need a separate directory that does not have permission to run scripts, but allows the system user for the web server (usually named ‘apache’ or ‘httpd’) to write to it. This directory is where the repository files will be saved when the webhook receiver script is triggered and executes a git pull. Based on the default settings, you might want to place this directory under /var/www/html/. For example, this could be under the path /var/www/html/repos/.

Dealing with reverse proxying

If, as in our case, reverse proxying has already had to be set up to redirect port 3838 to port 443 on the server, you will also need to make exceptions for these two directories so that they can be served directly out to port 443 by Apache.

This can be done via one of the Apache’s configuration files that gets loaded when its service starts. Ideally, it should be done in the configuration file in which the reverse proxy was set up in the first place. It might be httpd.conf (but it may well be in another configuration file). If the reverse proxy was not set up in httpd.conf, look for other files that end in .conf in the path /etc/httpd/conf.d/.

The configuration lines you’d be looking for will probably be something like this:

ProxyPass / http://127.0.0.1:3838/
ProxyPassReverse / http:/127.0.0.1:3838/

Just before those lines you would need to add exceptions for your scripts and repository directory. Assuming the defaults mentioned before, it might look this (the “!” tells Apache to exclude those directories from being reverse-proxied):

ProxyPass /cgi-bin !
ProxyPass /repos !
ProxyPass / http://127.0.0.1:3838/
ProxyPassReverse / http:/127.0.0.1:3838/

Save the config file and then restart the Apache service.

The script that is triggered by receiving the webhook contains the instructions for what needs to happen next on your server. When our PHP script is triggered, it first tells the system to take the token that was received as part of the request from GitLab (which is sent with the request in the X-Gitlab-Token HTTP header) and hash it.²

If the hash matches the secret token stored on the server (which needs to be stored somewhere non-public on the server, such as in an environment variable), the script concludes that the source of the request is genuine and does one of two things:

• Execute a git clone (if there is no destination directory on the server yet for the repository).
• Execute a git pull (if a clone of the repository has previously been done and there is already a destination directory present for it).

Our PHP webhook receiver script is adapted from a script by marcelosomers on GitHub. We’ve made a few minor adjustments, including adding support for secure token verification. You can click on “Show the code” below to view and copy the updated PHP script.

Show the code

#!/usr/bin/php
<?php
/**
  * This script is for easily deploying updates to Github repos to your local server. It will automatically git clone or
  * git pull in your repo directory every time an update is pushed to your $BRANCH (configured below).
  *
  * INSTRUCTIONS:
  * 1. Edit the variables below
  * 2. Upload this script to your server somewhere it can be publicly accessed
  * 3. Make sure the apache user owns this script (e.g., sudo chown www-data:www-data webhook.php)
  * 4. (optional) If the repo already exists on the server, make sure the same apache user from step 3 also owns that
  *    directory (i.e., sudo chown -R www-data:www-data)
  * 5. Go into your Github Repo > Settings > Service Hooks > WebHook URLs and add the public URL
  *    (e.g., http://example.com/webhook.php) If using a Secret Token, it can be added just under this.
  * 6. Put a copy of the secret token somewhere safe on your server that can not be read by the public but can be read by the apache user itself when this script is run. 
  *    (ie. in an environment variable.)
  *
**/

// Set Variables
$LOCAL_ROOT         = "/var/www/html/repos";
$LOCAL_REPO_NAME    = "Project_Name";
$LOCAL_REPO         = "{$LOCAL_ROOT}/{$LOCAL_REPO_NAME}";
$REMOTE_REPO        = "https://gitlab.pavlovia.org/YourUserName/Project_Name.git";
$BRANCH             = "master";

$secretToken = getenv('GITLAB_SECRET_TOKEN');
$receivedToken = $_SERVER['HTTP_X_GITLAB_TOKEN'];


// if ( $_POST['payload'] ) {
  // Only respond to POST requests from Github

if (hash_equals($secretToken, $receivedToken)) {

  if( file_exists($LOCAL_REPO) ) {

    // If there is already a repo, just run a git pull to grab the latest changes
    exec("cd {$LOCAL_REPO} && git pull");
    die("done pull " . mktime());
  } else {

    // If the repo does not exist, then clone it into the parent directory
    exec("cd {$LOCAL_ROOT} && git clone {$REMOTE_REPO}");

    die("done clone " . mktime());
  }
}
?>

When you set all of this up on your own server, please remember that error logs are your friends. If something isn’t working, use them to help give insight into where things might be going wrong. This proved invaluable for us.

If your goal in all of this is only to set up a webhook from a Pavlovia GitLab repository, and you intend to use or display the data in it by some other method than via Shiny Server, then there are much simpler ways to accomplish that which don’t require an entire Apache web server. E.g., you could use Flask.

If your server is behind a firewall and/or partially or wholly managed by a central IT department, then you will also likely need to navigate the joys of, e.g., firewall change requests and setting the aforementioned directories and permissions up more securely with SELinux.

Shiny app

You can find the choice-cost visualisation Shiny app here.

Click on this link to download the Shiny app code. [Update 22/10/2024: Fixed an issue that occurred when participants chose a number as nickname.]

Limitations

In all our tests, the new data were already available in the Shiny app when it was automatically launched after completing the experiment on Pavlovia. However, we have not yet tested what happens if many people complete the experiment simultaneously. If there turns out to be a delay in the data transfer, it might be necessary to reload the browser tab with the app to show the most recent data.

Acknowledgements

We would like to thank Jon Peirce and Alain Pitiot for their help at various stages of this project.

Citation

If our work assists you in setting up your own analysis pipeline or Shiny app, we kindly ask that you reference this blog post by including a link to it or, if you’re into APA style, by citing it as follows:

Derrfuss, J., & Lindberg, S. (2024, October 16). Integrating Pavlovia and Shiny for automated near real-time data analysis and visualisation. https://corrigo-lab.org/posts/2024-10-16-pavlovia-shiny-integration/

Footnotes

1. The machine running Shiny Server itself does not need to be public-facing, but if it were to be hosted on a different server to the one you intend to receive the webhook on, you will need to carefully consider how to manage getting Shiny Apps to read the relevant repository data from the web server.

2. More information about this can be found on the validating webhook deliveries page on GitHub.