Home / 2-Code / eMail-to-Obsidian-Sync

Email-to-Obsidian Sync (ETOS)

The ETOS script is designed to check emails with a specific subject, download PDF attachments to a temporary folder, convert these PDFs to text files, process the text through Fabric with Ollama, and finally save the extracted step-by-step instructions into a locally mounted Obsidian directory.

Overview

You can use ETOS to email PDF files of instructions, which will then be automatically added to your Obsidian vault. The Fabric pattern is configured to extract only step-by-step instructions from the PDF and save them to Obsidian. The script combines the power of Ollama, Fabric, attachment-downloader, and pdftotext utilities.

This guide does not cover installing a custom pattern for Fabric. Refer to the Fabric documentation for more details.

Basic Workflow

  1. Use attachment-downloader to retrieve PDFs from your email.
  2. Use pdftotext (part of poppler-utils) to convert PDFs to text.
  3. Pipe the extracted text into Fabric, and then save it in your Obsidian directory.

Prerequisites

You will need to replace certain placeholders with your values: - <EMAIL_SMTP_HOST> - SMTP server of your email host - <EMAIL_ADDRESS> - Email used for login - <EMAIL_PASSWORD> - Email login password - <EMAIL_FOLDER> - Email folder to download messages (e.g., Inbox) - <EMAIL_OUTPUT_LOCATION> - Local directory to save downloaded PDFs - <EMAIL_SUBJECT_FILTER> - Filter for email subjects (e.g., "etos") - <PDF_DOWNLOADED> - PDF file used for testing pdftotext - <REMOTE_OLLAMA_SERVER> - (Optional) Remote Ollama server address - <DOWNLOADED_ATTACHMENT> - PDF file pulled from your email - <DOWNLOADED_ATTACHMENT_CONVERTED> - Corresponding TXT file after conversion

Ensure you customize the Fabric pattern as needed. If you use a different name for your pattern, replace write_code accordingly in the script.

Installation and Setup

Step 1: Update and Upgrade Ubuntu

sudo apt update && sudo apt upgrade -y

Step 2: Install Required Packages

sudo apt install wget curl git nano ufw pip pipx poppler-utils ffmpeg cron

Step 3: Install attachment-downloader

pipx install attachment-downloader

Step 4: Ensure pipx Path

pipx ensurepath

Step 5: Copy attachment-downloader to $PATH

cp /root/.local/share/pipx/venvs/attachment-downloader/bin/attachment-downloader /usr/local/sbin/attachment-downloader

Step 6: Setup Your Email Account

Ensure your email is configured to receive files.

Step 7: Test attachment-downloader

attachment-downloader --host <EMAIL_SMTP_HOST> --username <EMAIL_ADDRESS> --password <EMAIL_PASSWORD> --imap-folder <EMAIL_FOLDER> --output <EMAIL_OUTPUT_LOCATION> --delete --subject-regex=<EMAIL_SUBJECT_FILTER> --filename-template="{{date.strftime('%Y-%m-%d-%H-%M-%S-%f')}}_{{ attachment_name }}"

Step 8: Test pdftotext

pdftotext <PDF_DOWNLOADED>

Step 9: Clone the Fabric Repository

git clone https://github.com/danielmiessler/fabric.git

Step 10: Change into Fabric Directory

cd ./fabric

Step 11: Install Fabric

pipx install .

Step 12: Ensure pipx Path Again

pipx ensurepath

Step 13: Run Fabric Setup

fabric --setup

Step 14: Reload Your Shell

source ~/.bashrc

Step 15: Test Fabric Installation

fabric --help

Step 16: Download Files from Email Server

attachment-downloader --host <EMAIL_SMTP_HOST> --username <EMAIL_ADDRESS> --password <EMAIL_PASSWORD> --imap-folder <EMAIL_FOLDER> --output <EMAIL_OUTPUT_LOCATION> --delete --subject-regex=<EMAIL_SUBJECT_FILTER> --filename-template="{{date.strftime('%Y-%m-%d-%H-%M-%S-%f')}}_{{ attachment_name }}"

Step 17: Convert PDF to Text

pdftotext '<DOWNLOADED_ATTACHMENT>.pdf'

Step 18: Test Fabric with Ollama

cat '<DOWNLOADED_ATTACHMENT_CONVERTED>.txt' | fabric --model llama3:latest --pattern write_code --remoteOllamaServer <REMOTE_OLLAMA_SERVER> | save NewTo-Do

Step 19: Set Up the SMB Mount for Obsidian

There are many tutorials available online on how to mount an SMB share. The goal is to ensure the files reach your Obsidian library via SMB, Syncthing, or another method of your choice.

Step 20: Pull the ETOS Script

mkdir /opt/etos
git clone https://github.com/tebwritescode/etos.git
cp ./etos/etos.sh /opt/etos/etos.sh

Step 21: Make the Script Executable

chmod +x /opt/etos/etos.sh

Step 22: Schedule a Cron Job to Pull Emails and Save to Obsidian

crontab -e

Add the following lines: bash 0 * * * * /usr/bin/bash /opt/etos/etos.sh 15 * * * * /usr/bin/bash /opt/etos/etos.sh 30 * * * * /usr/bin/bash /opt/etos/etos.sh 45 * * * * /usr/bin/bash /opt/etos/etos.sh

Your ETOS setup should now be complete and functioning to pull PDF instructions via email and save them in your Obsidian vault!