Since many of us are now using AI assisted workflows to handle some of the finicky details of personal science projects, I thought it would be a good idea to create a topic to share ideas and approaches. It’s open now!
Feel free to post long text if you want to share details, but use the “expand” function, as I did in the first post:
[details="Click to expand"]
Your long text goes here. It can include paragraphs, lists, code, etc.
[/details]Today I worked on a medium term goal of nudging my QS work toward text-based workflows, because this supports freeform reasoning using AI assistance better than trying to integrate various apps directly. One of the resources for my personal science projects is my collection of research papers in Zotero. Today I spent about 90 minutes setting up a structure to make them more easily “thinkable” by extracting metadata and notes on recent papers from my Zotero directories and putting them into .md files accessible both via command line/Claude and via Obsidian.
Here’s the summary of the session generated by the diary prompt.
Today’s session focused on developing a comprehensive strategy for integrating Zotero research management with Text-Based Workflow Knowledge (TBWK) principles. The user initiated the conversation with a request to examine file structures containing Zotero knowledge and extract research paper links, metadata, and personal notes into a text-based system that would align with their established TBWK approach.
The conversation began with an exploration of the user’s file system to locate Zotero data, but the actual Zotero files were not found in the accessible /Users/garywolf/developer directory. This led to a strategic discussion about extraction approaches rather than immediate implementation. The user provided a sample Zotero storage path (/Users/garywolf/Zotero/storage/A6Q274ZU/Gruber - 1995 - Toward principles for the design of ontologies used for knowledge sharing.pdf) which confirmed a standard Zotero installation structure.
Through iterative discussion, we developed a “Bridge and Enhance” strategy that maintains Zotero as the primary PDF manager and citation tool while creating a parallel text-based research analysis layer. The approach focuses on extracting research papers to markdown files within the user’s Obsidian vault, making them accessible to both Obsidian’s visual interface and command-line tools for AI assistant analysis. Key decisions included using a single, flexible template with pre-populated tags for deletion rather than creation, focusing initially on 2025 papers, and placing extracted files in a research_papers/2025_extraction/ subdirectory within the existing Obsidian vault.
The session culminated in creating a comprehensive step-by-step implementation plan that breaks the integration into five phases: setup and template creation, extraction setup, template refinement, workflow integration, and maintenance. The plan includes specific technical details, success metrics, risk mitigation strategies, and immediate next actions. The approach successfully balances the user’s desire for text-centric workflows with practical considerations around existing Zotero investments and research habits.
• Plan document: /Users/garywolf/developer/zotero_tbwk/zotero_plan.tbwk
• Directory structure: /Users/garywolf/developer/zotero_tbwk/ created
• Text shortcut template designed for macOS text replacement (;research trigger)
• Markdown note template framework for extracted research papers
• Strategic workflow architecture bridging Zotero and TBWK systems
• Initial request: “Examine the file structure that contains Zotero knowledge and suggest a way to extract some links to papers stored in Zotero and the associated notes for eventual inclusion in my TBWK approach”
• Focus refinement: “Focus on extraction in your answer” with specific requirements for paper links, metadata, and personal notes
• Strategic clarification: Discussion of three integration approaches (Bridge and Enhance, Export and Evolve, Hybrid Workflow)
• Template development: “I’d like to focus on recent papers in my 2025 file” and placement within Obsidian vault but accessible to AI assistants
• Workflow optimization: Development of text shortcut template with pre-populated tags for deletion rather than creation
• Documentation request: “Create a step-by-step plan as a .md document” and “Create a Claude Diary entry”
Zotero installation confirmed at: /Users/garywolf/Zotero/ with database likely at /Users/garywolf/Zotero/zotero.sqlite
Target extraction location: ~/ObsidianVault/research_papers/2025_extraction/
Filename convention: YYYY_FirstAuthor_KeywordFromTitle.md
Text replacement trigger: ;research expanding to structured template
File system exploration performed on: /Users/garywolf/developer
Search patterns used: “zotero”, “Zotero”, “sqlite”, “.rdf”, “bib”, “citation”, “reference”
Template structure includes: metadata, original intent capture, research context, connections, and follow-up actions
• Discovered the importance of capturing “moment of discovery” insights when downloading research papers
• Learned that deletion-based templates (pre-populated tags) are more user-friendly than creation-based templates
• Found that the “Bridge and Enhance” approach aligns perfectly with TBWK philosophy of enhancing existing tools rather than replacing them
• Understood that file placement within Obsidian vault but accessible to external tools creates optimal flexibility
• Recognized that text-centric research workflows can complement rather than compete with existing Zotero investments
Implement Phase 1: Set up macOS text replacement with ;research trigger
Create research_papers/2025_extraction/ directory structure in Obsidian vault
Test template workflow with 1-2 recent papers from Zotero
Locate and verify access to Zotero database at expected path
Begin Phase 2: Develop and test extraction script on small subset of papers
Refine template based on initial usage patterns
Establish command-line search workflows using ripgrep
Duration: Approximately 1.5 hours
Tools discussed: Zotero, Obsidian, macOS text replacement, ripgrep, Python extraction scripts
References: TBWK document principles, Zotero database structure, SQLite querying
File types created: .tbwk (markdown plan), directory structure
Integration complexity: Multi-tool workflow bridging GUI and command-line interfaces
I’ve been using MCP with Claude Desktop and the mcp-pyzotero MCP connector. The new versions of Zotero can work as an MCP server, letting you read and search your Zotero docs from within Claude Desktop. For example, I can prompt connect to my Zotero library and find all documents that refer to Apple Watch. and then return the results in BIB format.
There are similar MCP connectors to Obsidian (find all notes that relate to gut health) but I haven’t dug deeply into that yet.