Thursday, March 22, 2018

From ConnectedText to Zim?

It is always good to look for alternatives—or so it seems to me. I have been exclusively using ConnectedText for my notes since 2005. Recently, there have been rumors that ConnectedText won't be updated in the future, and some of us were considering alternatives to ConnectedText. It seems that these rumors are not correct. Connectedtext may be in "maintainance mode, but it will be updated as needed.

Still, I wanted to see what other options there are. So, I tried to determine whether and how compatible ConnectedText's files are with Zim. I had considered it before, but I decided to give it another chance. (I will never give Wikidpad another chance, as I found it highly unreliable, even if it was quite easy to change its markup to something closer to that of ConnectedText [see here for a start]).

I was able to transfer more than 11,000 ConnectedText topics into Zim. I used ConnectedText's backup files (*.cbk) and transformed them with the first AhK script I made (i.e. I gave the entries more friendly titles).[1] I did not use the second script (to remove the first line with "#CT#" as that produced files Zim had difficulties recognizing. (May have to do with UTF-8, but I don't know. I then cleaned up the text files with fnr.exe ( (Find & Replace Tool). But be careful, it's easy to make mistakes that spoil the files.

The titles of Zim's text files do not allow spaces. I used Advanced Renamer to replace the spaces by underscores (see I probably could have done all this with AhK, but was to lazy. You should also note that Zim is very fussy about what character it recognizes in titles. It does not recognize question mark, for instance. And a bracket at the beginning of a file name does not get recognized (so you have to remove brackets in topics like (Kuehn 2002) which I extensively use in ConnectedText for bibliographical entries.

The resulting files work, as Zim uses text files, not a database. But there are some problems:
  1. the load time is about 3 minutes on my system (off a SSD with 500MB per second transfer time). After that, it is reasonably fast in loading pages, etc.
  2. every time you change the underlying text files, it needs to re-index which can take up to 20 minutes
  3. many of ConnectedText's more advanced functions are not available
  4. I did not succeed in transferring pictures automatically
  5. it's impossible to print directly from Zim. You can only print to your browser
  6. I don't like that I have to use underscores in naming pages

Zim is clearly is not a good replacement for ConnectedText, but it is a replacement, if such is needed. It would do. in a pinch, and some of the plugins might provide more functionality that I can use. But for now and the foreseeable future, I will stick with ConnectedText. (See also my Reassurance.)

The transfer took about half a day to accomplish (and is still a work in progress, as I discover new things that need to be changed, like properties I had forgotten).

The other thing I recommend as a limited replacement for ConnectedText are Notetab outlines. They also work and are much faster. But they have even fewer features. I have written about them before as well (on this blog; just search for notetab). I use them as another "backup" for ConnectedText, that is, I create yearly files, like "Notes 2016.otl", "Notes 2017.otl", etc.

1. For the scripts, see here.
2. I first published this on the ConnectedText Forum, but I thought even people who don't frequent it, might have a passing interest.


Abu Musa said...

I'd like to thank you for running this blog as it has helped me to figure out how I want to organize my knowledge base and what app to use.

Since your post is about Zim, I thought to mention what solution did I choose: a candidate close to Zim - TiddlyWiki. Why TiddlyWiki?

1) Unlike ConnectedText, it is still alive: it has been in more or less continuous development for years and has an active, if perhaps small, community around it. The CT's web and social media seem to suggest not much is going on since 2015.

2) Unlike ConnectedText, it is open-source. As such, it existence does not rely on a single person/company. Even if discontinued, it is going to be usable - as long as web browsers using html exist. One can also try to tweak it on his own then. I think it is an important factor for creating something one wants to use for, say, decades.

3) Problem with links? I know you mentioned in one of the previous posts that for you TW is a no-go because its link structure for a given note (or node) breaks down if you change its name. Which is true and I don't intend to convince you it is not a problem. But perhaps some readers are in a different position and might find this info useful: I could choose TW because I actually don't change notes' names often. Moreover, an easy workaround is to create a new note with a new name and link it to the previous version (with emptied body). In this way, note A linked to B, which was recreated and renamed as C, is still linked to it (A -> B -> C).

4) Speed and syncing. The beauty of TiddlyWiki (and perhaps other wiki-based solutions) is that it is just one html file. My database is not as big as yours but I suspect you wouldn't encounter the problem with the load speed in TiddlyWiki. Big problem for me, in the beginning, was how to sync the file. But there are many solutions and one just has to spend the time finding the suitable one.

5) Last but not least, TiddlyWiki - thanks to its community - has a lot of various advanced features. I think TiddlyMap brings it close to the ConnectedText's experience. There is also very handy timeline plugin. And much more.

Of course, there is not a single solution best for everyone. Everyone's usage and needs vary. But I think TiddlyWiki is a great piece of note-taking software (if one can call a html file software) and deserves a longer mention on this blog.

MK said...

Thank you very much for your extended comment. I appreciate it.

The breaking of links when you change a name is indeed a big problem for me.

However, there may be another having to do with importing. I say may be because I don't know. How easy would it be for me to import 11,000 new entries?


Abu Musa said...

I cannot really answer as I haven't imported anything into my TW. And I don't know in what format you can export your data from CT. But I guess TW's options won't be much different from Zim. Generally, I can usually get an answer to this question by checking TW documentation AND/OR searching TiddlyWiki's Google Group.

As for import, without much looking, I see this info or this conversation.

MK said...

I actually was aware of both posts (and a few others). None answer my questions. The first one you mention says you can drag and drop. I cannot imagine that this would work (or be practical for 11,000 files. The second one is for CVS files, as far as I can see.

I tried to download TiddlyDesktop, by the way. But it failed. I'll try again today.

Abu Musa said...

The page under the first link says more things, actually the first bullet points to an import function in TW and does not concern drag&drop: "Use the import button (under the Tools tab in the sidebar) to select a local file". And if you check this in TW (under Tools tab), there is this info next to the button: "Import many types of file including text, image, TiddlyWiki or JSON".

As for TiddlyDesktop, my experience under Win 10 was that I got an error for the version I was supposed to be using (x64 as is my OS) but then I actually realised the other version (x32) works fine.

MK said...

To say use "import button (under the Tools tab in the sidebar) to select a local file is ominous because it suggests you cannot import many files at the same time. Still, I'll give it a trial