Syncing Zotero and OneDrive

Zotero
Python
macOS

How to use Python and macOS launch agents to sync PDFs stored by Zotero to a folder on OneDrive.

Author

Jan Derrfuss

Published

April 29, 2023

Modified

February 14, 2026

 

Note

Update (Feb 2026): Recent versions of macOS have introduced strict background permissions and altered how OneDrive stores files locally. Furthermore, the built-in macOS version of rsync is outdated and struggles with modern macOS file metadata. This guide reflects the updated setup required to handle these changes.

After finally ditching the once-great Papers by Mekentosj, I needed to find a new PDF management software. I tried Zotero, Endnote, Jabref, Bookends and Paperpile1. I wasn’t entirely happy with any of these tools, but eventually decided to give Zotero (together with Zotfile for renaming the PDF files) a serious try.

One Major Issue

While I was happy with Zotero overall, I encountered one major issue: I wanted to be able to read PDFs not only on my desktop, but also on my tablet.2 However, it turned out that neither GoodReader nor PDF Expert were able to sync the 7,000+ PDFs that I have accumulated over the past 20 or so years. The main issue seemed to be that both PDF readers struggled with the fact that Zotero places each and every PDF in its own folder (these folders are given a random 8-character name by Zotero). To illustrate how Zotero stores PDF files, here are the first three subfolders in my Zotero storage folder:

Image illustrating how Zotero stores PDF files.

A Solution

I wanted to share a possible solution to this problem here in case others encounter the same issue. The basic idea is to keep Zotero’s storage folder as it is and to not sync this folder with OneDrive. Instead, I decided to have a copy of all PDFs in a separate folder that is synced with OneDrive. The file structure of the OneDrive folder is completely flat (i.e., there a no subfolders). The folder just contains the PDFs.

To sync the PDFs successfully in both directions (i.e., Zotero with OneDrive, and vice versa), information about the Zotero subfolder names needs to be maintained. I decided to do this by adding the Zotero subfolder name to the original file name. That is, in the OneDrive folder, Smith_2012_Cerebral Cortex.pdf becomes Smith_2012_Cerebral Cortex_2A9SZ8MM.pdf. This has the advantage that it also solves the potential problem of name conflicts (e.g., if there are two papers published by Smith in Cerebral Cortex in 2012 stored in Zotero).

Before we look at the code, there are two crucial prerequisites for a modern Mac:

1. Force OneDrive to download your files

Modern OneDrive uses “Files On-Demand,” meaning it stores 0-byte placeholders on your Mac instead of actual PDFs. To allow our Python scripts to read the files, you must first open Finder, right-click your /your/path/to/onedrive/zotero_articles folder, and select Always Keep on This Device. Wait for the cloud icons to turn into solid checkmarks.

2. Install modern rsync

Apple stopped updating rsync due to licensing disputes. The ancient built-in rsync does not properly handle modern APFS file systems or the extended attributes that cloud drives rely on. You should install the modern version via Homebrew. In the Terminal, run this command:

brew install rsync

1. The Python scripts

I wrote two Python scripts (one for syncing Zotero to OneDrive, and another for syncing OneDrive to Zotero). The actual syncing is done by a subprocess call to our newly installed Homebrew rsync.

Here is the Python code for syncing Zotero to OneDrive:

sync_zotero2onedrive.py
#!/usr/bin/env python3

import os
import subprocess
import filecmp

zoteroDir = "/your/path/to/Zotero/storage"
onedriveDir = "/your/path/to/onedrive/articles_zotero"
# Note: /opt/homebrew/bin/rsync is the path for Apple Silicon Macs. 
# If you are on an older Intel Mac, this will be /usr/local/bin/rsync
rsyncPath = "/opt/homebrew/bin/rsync"

# note that os.walk is recursive; it starts like this:
# root: /your/path/to/Zotero/storage
# dirs: ['2A9FDDID', '2A9SZ8MM', '2AJDS299', '2AEBFC7A', '2AJ52RIM', '2AADPG7M', etc.]
# files: []
# then it goes through the subdirectories:
# root: /your/path/to/Zotero/storage/2A9FDDID
# dirs: []
# files: ['.zotero-ft-info', 'Houtkamp_2010_Journal of Cognitive Neuroscience.pdf', '.zotero-ft-cache']
# and so on
for root, dirs, files in os.walk(zoteroDir):
    for filename in files:
        if filename.endswith(".pdf"):
            # get the current subfolder, e.g. 2A9FDDID
            dir = root.split("/")[-1]
            # remove the file extension (i.e., .pdf) and keep the rest
            filenameBase = filename.rsplit(".", 1)[0]
            # create the new file name, e.g. Houtkamp_2010_Journal of Cognitive Neuroscience_2A9FDDID.pdf
            filenameNew = filenameBase + "_" + dir + ".pdf"
            # create path + file name for Zotero storage
            zoteroFile = os.path.join(root, filename)
            # create path + file name for OneDrive
            onedriveFile = os.path.join(onedriveDir, filenameNew)
            
            # check if the file exists on OneDrive
            # if not, use rsync to sync it to OneDrive
            if not os.path.isfile(onedriveFile):
                print(f"New file found. Syncing to OneDrive: {filenameNew}", flush=True)
                subprocess.call([rsyncPath, '-au', '--min-size=10', '--delete', zoteroFile, onedriveFile])
            
            # if the file does exist
            else:
                # compare the two files
                compRes = filecmp.cmp(zoteroFile, onedriveFile, shallow=True)
                # if they are different according to filecmp, sync them
                if compRes == False:
                    print(f"File updated. Syncing to OneDrive: {filenameNew}", flush=True)
                    subprocess.call([rsyncPath, '-au', '--min-size=10', '--delete', zoteroFile, onedriveFile])

print("Zotero -> OneDrive sync check complete.", flush=True)

And the Python code for syncing OneDrive to Zotero:

sync_onedrive2zotero.py
#!/usr/bin/env python3

import os
import subprocess
import filecmp

zoteroDir = "/your/path/to/Zotero/storage"
onedriveDir = "/your/path/to/onedrive/articles_zotero"
rsyncPath = "/opt/homebrew/bin/rsync"

for root, dirs, files in os.walk(onedriveDir):
    for filename in files:
        if filename.endswith(".pdf"):
            # remove the file extension (i.e., .pdf) and keep the rest
            filenameBase = filename.rsplit(".", 1)[0]
            # split the filename into a list with two elements
            filenameList = filenameBase.rsplit("_", 1)
            # recreate the original file name
            filenameOrig = filenameList[0] + ".pdf"
            # get the Zotero directory name, e.g. 2A9FDDID
            dir = filenameList[1]
            # create path + file name for OneDrive
            onedriveFile = os.path.join(root, filename)
            # create path + file name for Zotero
            zoteroFile = os.path.join(zoteroDir, dir, filenameOrig)
            
            # files should never be created on OneDrive
            # however, it is necessary to check if the file was removed from Zotero
            # if that is the case, it should also be removed from OneDrive
            if not os.path.isfile(zoteroFile):
                print(f"Deleted in Zotero. Removing from OneDrive: {filename}", flush=True)
                os.remove(onedriveFile)
            
            # else: the file still exists in the Zotero storage folder
            else:
                compRes = filecmp.cmp(zoteroFile, onedriveFile, shallow=True)
                # if they are different according to filecmp, sync them
                if compRes == False:
                    print(f"Annotated on OneDrive. Updating Zotero: {filenameOrig}", flush=True)
                    subprocess.call([rsyncPath, '-au', '--min-size=10', '--existing', onedriveFile, zoteroFile])

print("OneDrive -> Zotero sync check complete.", flush=True)

2. The Bash Wrappers

If you are using Anaconda or a custom Python installation, macOS’s background process manager (launchd) will often crash because it runs in a sterile environment and cannot find your Python libraries. To fix this, we create two simple shell scripts that inject your Python environment path before running the Python scripts. The scripts below assume that you’re using Anaconda (adjust the paths in the scripts if you don’t).

Create wrapper_zotero2onedrive.sh:

wrapper_zotero2onedrive.sh
#!/bin/bash
export PATH="/your/path/to/anaconda3/bin:$PATH"
python3 /your/path/to/sync_zotero2onedrive.py

Create wrapper_onedrive2zotero.sh:

wrapper_onedrive2zotero.sh
#!/bin/bash
export PATH="/your/path/to/anaconda3/bin:$PATH"
python3 /your/path/to/sync_onedrive2zotero.py

Next, make both of these wrappers executable by running this in your Terminal:

chmod +x /your/path/to/wrapper_zotero2onedrive.sh
chmod +x /your/path/to/wrapper_onedrive2zotero.sh

3. Granting Full Disk Access

Because macOS sandboxes background processes strictly, you must give the bash shell permission to read and write your files.

  1. Go to System Settings > Privacy & Security > Full Disk Access.
  2. Click the + button.
  3. Press Cmd + Shift + G, type /bin/bash, and hit Return.
  4. Add it to the list and ensure the toggle is turned ON.

4. Running the scripts as launch agents

Finally, I set up two property list (.plist) files and added these to ~/Library/LaunchAgents. Note the following:

  • We tell launchd to explicitly run /bin/bash with our wrapper scripts to bypass security execution errors.
  • We stagger the StartInterval (600 seconds and 629 seconds) so the two scripts don’t accidentally run at the exact same millisecond and cause file-lock crashes.

This property list syncs Zotero to OneDrive:

com.jan.sync.zotero2onedrive.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>com.jan.sync.zotero2onedrive</string>
  <key>ProgramArguments</key>
  <array>
    <string>/bin/bash</string>
    <string>/your/path/to/wrapper_zotero2onedrive.sh</string>
  </array>
  <key>KeepAlive</key>
  <false/>
  <key>StartInterval</key>
  <integer>600</integer>
</dict>
</plist>

This property list syncs OneDrive to Zotero:

com.jan.sync.onedrive2zotero.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>com.jan.sync.onedrive2zotero</string>
  <key>ProgramArguments</key>
  <array>
    <string>/bin/bash</string>
    <string>/your/path/to/wrapper_onedrive2zotero.sh</string>
  </array>
  <key>KeepAlive</key>
  <false/>
  <key>StartInterval</key>
  <integer>629</integer>
</dict>
</plist>

To start running the launch agents, enter the following commands in your Terminal window3:

launchctl load -w ~/Library/LaunchAgents/com.jan.sync.zotero2onedrive.plist
launchctl load -w ~/Library/LaunchAgents/com.jan.sync.onedrive2zotero.plist

Troubleshooting: To check if your scripts are running successfully, use this command:

launchctl list | grep com.jan.sync

If you see a 0 in the middle column, it ran perfectly! If you see a 1, 78, or 127, a crash occurred.

Warning: The OneDrive “ListSync” Bug

If you set up this automated background sync, there is a known bug in the macOS version of Microsoft OneDrive that you need to be aware of.

Because Zotero libraries potentially contain thousands of files being constantly checked and updated, OneDrive’s background telemetry can occasionally panic. When this happens, it gets stuck in a loop and silently dumps gigabytes of error logs into a hidden ListSync folder, which can completely fill your Mac’s hard drive overnight.

You can check if you are affected by looking at the size of this folder: ~/Library/Logs/OneDrive/ListSync/Common/

How to prevent it:

Since we already built a Bash wrapper that runs every 10 minutes, the most elegant solution is to turn that wrapper into a “janitor.” We can tell it to silently empty the OneDrive log folder right before it syncs your PDFs.

Simply update your wrapper_zotero2onedrive.sh file to include this single rm -rf line:

wrapper_zotero2onedrive.sh
#!/bin/bash
export PATH="/your/path/to/anaconda3/bin:$PATH"

# Silently delete OneDrive's bloated telemetry logs to prevent hard drive filling
rm -rf ~/Library/Logs/OneDrive/ListSync/Common/* >/dev/null 2>&1

python3 /your/path/to/sync_zotero2onedrive.py

Footnotes

  1. I did not consider Mendeley as it encrypts the user’s data.↩︎

  2. I know that Zotfile has the option to sync PDFs with a tablet, but after trying it out I wasn’t really happy with this option either (see this page for more information on using Zotfile for syncing). In addition, as I wasn’t sure if I was going to continue using Zotero, I didn’t want to shell out $120/year for online storage.↩︎

  3. Basics about the usage of launchctl can be found in this answer on Stack Exchange.↩︎