Build a second brain with Obsidian as my Knowledge Management System

Current problem

As a lifelong learner of a wide range of subjects, I need robust tools to manage my acquired knowledge and act as an idea pad for content creation. My minimalist mindset also requires the number of deployed tools to be as few as possible. This post is a summary of my journey in notetaking with Obsidian. I’m going to go over why Obsidian is a great (although not perfect) tool for notetaking, project management and journaling. For these usage scenarios, I will provide tips/plugins to customize your usage.

background

current problems

several knowledge silos, unable to connect related ideas to form a story, or a big question to explore
for to-do tasks and questions: unable to set alert to check progress and follow up status
unable to port content to different tools and have complete control/ownership with privacy

As I am an avid reader, my first foray into knowledge management system is to create my own app BookTracker, combined with OneNote to keep track of my reading notes. Overtime, the slow performance of online spreadsheets, as well as inflexibility in changing the schema (add or remove new fields) to suit new types of content, make me venture out for a replacement.

In addition, my app only captures reading notes, which are siloed from my diary and learnings in other areas. I am a lifelong learner and pick up tips from various places like psychology, life hackers, neuroscientists, fellow learners, etc. There might be similar ideas scattered across different spheres, but the current setup prevents them from forming a cohesive knowledge system.

why obsidian

Why not use Roam, Notion, OneNote, Evernote?

My requirements

I need a completely offline solution that allows me to backup my notes freely (and individually) to wherever I like. This automatically rules out Notion (which is completely online).

I also want a solution that syncs across platforms, and allows me to open/edit notes in different tools and take them anywhere. This rules out OneNote which stores each notebook in its own format that cannot be opened elsewhere. As Obsidian stores each file as .md (markdown) file, I can open it in VSCode and any markdown editors to edit.

The convenience and flexibility of Obsidian win over me. Having a myriad of Community plugins - Obsidian Help (which I will cover later) are just icing on the cake. With Obsidian, I can

enter notes whenever and wherever (using both PC and phone, and files are Sync your notes across devices - Obsidian Help up automatically with cloud storage)
connect all the data, regardless of their origin by linking them inside Obsidian (can link to file or section level)
tag/categorize/x-link them for neat organization
act-on them using Tasks <- most important thing
index and search <- improve retrieval quality
track progress with bird’s eye view using Heatmap
enforce consistency in format and content using Templates
allow me to own and freely manipulate my content, with the tool taking a backstage seat

In other words, Obsidian provides an easy way to zoom out (see everything from a distance, with everything neatly tucked in their corner but connected in a sensible way), and zoom in (able to access what i want to know with the right information architecture)

progress

I had a brief stint with OneNote, Notion, Foam and ultimately settled with Obsidian in 2023. Whenever I adopt a process, I let it run for at least 6 months to stablize. The first few months is the most volatile period in which I get used to the tool, find plugins to suit my unique usage, and learn to tweak both the process and the tools. Then I let it sail peacefully for awhile and monitor if the output is producing what I want. Is it assisting me as a means and not imposing itself as an end (there are some ambitious tools that take the centerstage and the user needs to fit into its mold).

It’s been over a year since I adopt Obsidian. My recipe isn’t completely original. Just like any cook in the world, recipes probably comes from a myriad of sources, like from family passdown, an incidental restaurant you bump into, something you read from a book or blog, etc.

How I use Obsidian

At first, I just wanted to replace booktracker to keep my book notes. Later, I extend to use Obsidian for the followings

freeform writing to heal
collect web annotations
digital garden
project management
daily/weekly/monthly notes

book review

I consume a lot of paper and Kindle books. All these highlights and notes are useless without a structured way to view and edit them alongside my notes from paper books. Currently, the notes are in standalone silos, disconnected from each other and unable to form a cohesive knowledge system.

paper books:
1. digitalized excerpts by using Microsoft Lens to scan paper book excerpts.
2. use the Book Search or Douban (Chinese book) plugin to create a Book note in Obsidian
3. Apply my own Book Review template using the Templates plugin
4. Write my notes/review. For longer excerpts or writing longer notes, I will use Obsidian app in PC.
Kindle books: highlights siloed. I’ve used Clippings to save and edit my highlights. One drawback is that it only reads from the MyClippings.txt file, and doesn’t touch annotations and bookmarks, which usually mark specific paragraphs or chapters that I want to revisit. I have tried to use the kindle extract highlight plugin - not working for me

In each book note, I have a “to do” section to track what I can apply to my daily life after reading, which can be pulled into a central dashboard run by the Dataview plugin. I also use the Dataview plugin to track how many books have inspired my blogging. The ultimate goal is to convert my input (the data and information I gain from reading) into my own worldview and knowledge, and through introspection and practice, my Wisdom.

taking notes and research

I have tried the atomic annotation method by the "index card" methodology to record each idea I come across in a separate file. I found it a bit chaotic even with the Graph view. Now, I prefer to create annotations in a book review note with its own header. Then I create a separate note with a theme in mind, and link to all the related annotation in one place. This “view” is like a TOC that allows me to collect ideas around a topic, to assist in memory recall, formulating blogging ideas and acting as a databank for research.

Using tags on each note to classify their genres also help to organize annotations easily, so that I can do a quick query on similar notes with Dataview.

One thing I absolutely love about Obsidian is the ability to style links in the wiki format: [[link to something]]. With the use of Quick switcher plugin, I can easily pull a related note from a list of candidates by typing a few words. Then, whenever I update the name or location of the targeted note, the link is updated automatically. This helps to keep all my notes linked properly, especially when I am doing a lot of tidying up of the vault.

blogging

Views collected like a TOC can be used as a skeleton for a blog post. They are basically my thoughts combed from multiple sources. These feed into the BLOG IDEAS stream, stored in the Project/Blogging/Ideas folder.

VSCodeis my main tool for both coding and blogging. As each Obsidian note is a markdown (.md) file, it makes my blogging workflow so smooth as I can just port my notes to VSCode to be tidied up and published via Hugo (which is a static site generator that takes markdown files and converts them to HTML)

journaling

At first, I created a Diary app using AppSheet to keep my daily to-do, gratitude journal and recap. I also use it to capture ideas, questions and learnings on the go. This is to ensure that before it fades away from short term memory after 30 seconds. I have a tangible way to capture and recall it later.

This again suffers from the inability to cross-link and search. With good results from using Obsidian to track my book notes, I moved my Diary to Obsidian too. Here’s my workflow

use the Templates plugin to create templates for daily, weekly, monthly and annual note
enable the Daily Note core plugin, so that it shows up prominently in the ribbon and you can create a daily note quickly. You can also enable the show on startup to nudge you into journaling the first thing in the morninghe
on the 1st day of the month, I create a monthly note with a top-down plan on major areas of my life including work, personal growth, health, etc.
on Monday each week, I create a weekly note on
- reflection of the previous week and what I could improve
- pulling tasks from Monthly plan
everyday when I wake up, I open Obsidian to write my Daily notes
- it acts as a diary with gratitude, self affirmation, reflection and planning
- since I use the Task plugin, whatever is planned in the weekly note will automatically show up in my Daily note
  - I can see what is overdued and what is due today
  - I will add some impromptu tasks

project management

I used Azure Devops before to manage my blog and side projects. It is great for team collaboration. However, since these are all solo projects, the benefits are not enough to compensate for the inflexibility of editing assets in the browser, and the lack of content portability. Plus the inability to edit status and assets on mobile is a big hurdle.

Now, I use the Kanbanand Projectsplugins for light project management. It is a simple and clear solution.

With both, I can create/track work items and link each work item to existing contents. Projects turn your whole folder into a backlog. Each note in the folder becomes a work item. Each note’s properties become an attribute of this work item, showing up in separate columns of the backlog. Now I can store both the PM system and my assets in the same place. It fulfills my desire to reduce toolset for clarity and ease of maintenance.

In addition, I can combine the Tasks and Kanban plugins to manage active tasks in Kanban boards

Obsidian’s support for Mermaid diagram also makes it easy for me to sketch system diagrams quickly and embedthem in other produce design notes.

metrics dashboard

To be able to track how far we’ve go and if we are overlooking something is important in every aspect of life. The Dataview plugin is handy for this purpose.

For those with TSQL knowledge, Dataview’s query syntax is very similar. I use it to do a myriad of tracking, including

managing my bookshelf, to see what books I like most, or what genre I consume most or spend most time in
pull up a list of tasks completed in a specific period

I also use the Heatmap plugin to generate visualization to track my progress.

Organizing content

I follow the PARA method to organize everything in my Vault into these folders

inbox (daily/weekly/monthly/annual notes)
projects (things I am actively working on NOW, e.g., blogging, coding projects, classes I take, etc)
resources (web clippings of various topics I am interested in NOW)
archives (tips, howto on stuff I might need in the future, e.g., recipe, tech tips, financial docs)

Properties are useful to organize my content (in addition to a folder structure) so I can query and list them easily with the Dataview plugin. I also use the Multi Properties plugin to bulk assign properties to many notess at once. In addition, I can edit the .obsidian.json file to customize the data type of my own property.

Another handy plugin is the Quick switcher core plugin, which allows me to type just a short part of a note title and see a list of files Obsidian suggests.

One more plugin that seems promising initially but I seldom use is the Graph view, which shows you the relationship among notes visually. With it, you can examine the links between notes, to uncover what notes are referred most and if there are any orphaned ideas.

Routine maintenance

github

I create a private repository in Github to backup my vault everyday at night, manually or use the obsidian-git plugin

Here’s how to set it up

    git init
    git add .
    git commit -m "1st commit"
    git remote add origin https://github.com/username/repo.git
    git branch -M main
    git push -u origin main

Conclusion

As discussed above, Obsidian is not good with team collaboration or sharing, unlike OneNote and Notion. It is basically geared for individual usage as your private content repository. I do find it easy to maintain the 3 elements for sustaining practice, as discussed in the book Deeper Mindfulness: routine, structure and momentum.

Routine: daily, weekly note
Structure: template to mandate content sections and quality
Momentum: write once daily at least. I have been doing this since 2023/9 with a good streak! I found out that I am writing more and more. As I write more, my confidence in writing and myself increases, and I am willing to write even more. This is how good habits are formed.

I’m happy with my current knowledge management system setup. It provides organization of my materials, structure for content creation, as well as guardrail for keeping up a habit. One thing that it (or any other tool) cannot substitude is the drive to think and create. I hope to create more interesting content to share in the coming years!