Zed pulled from the Chrome Webstore

Posted: 2015-07-23

This morning I received a notification from Google saying Zed has been pulled from the Chrome Webstore. This is significant because around 80% of Zed users use the Chrome web version, and about 41 thousand people have Zed installed this way.

Dear Developer,

Your Google Chrome item, "Zed Code Editor," with ID: pfmjnmeipppmcebplngmhfkleiinphhp did not comply with our program policies and was removed from the Google Chrome Web Store.

The Program Policies can be found here: https://developers.google.com/chrome/web-store/program_policies

I was surprised, so I checked the policies, but indeed: if you scroll down a bit you get to the "Chrome Apps Quality Guidelines" there's an item that reads:

Packaged and Hosted apps should not:

• Download or execute scripts dynamically outside a sandboxed environment such as a webview or a sandboxed iframe.

And guess what, this is exactly how Zed implements its extensibilty features. About 75% of code included when you download Zed runs inside a <webview> sandboxed environment, and now -- I'm not sure if this is a new policy or not -- this is not allowed.

I've always realized that running so much of the code in a sandbox, and exposing extra APIs to this sandbox was a bit of a grey area. However, I've been very open about how this works and why it works this way. In fact, I even presented this at Google itself (an extended version of a talk I gave there, I gave at the STX Next Tech Summit where it was recorded, if you're interested).

So, yeah. Awesome.

I see four paths forward:

1. I remove all extensibility features from Zed, severely limiting the ability of the user to extend Zed with custom scripts and packages. In practice, only allowing to run everything that comes bundled with Zed. This way Zed can go back in the Chrome Web Store.
2. I switch over to another version of extensibility that I once prototyped, which is to use Chrome Extensions to extend Zed (for a description of how this can work, see the STX Next tech summit talk I linked to before). This is technically possible and shouldn't break any rules in place today. But again I would be in a grey area, using Chrome extensions in a way they weren't intended. Will I face the same problem again in a few months?
3. Just accept that Zed won't be in the Chrome web store and tell people to clone the Github repo (or download a zip) and install Zed into Chrome that way.
4. Completely ditch the Chrome Web Store version and go node-webkit (or nw.js, or Github's thing) only. That is: switch over completely to a downloadable app. This simplifies a bunch of things (supporting both node-webkit and the Chrome App version isn't always easy), but as I said: right now, about 20% of Zed users use the downloadable version.

Either way, this pull will result in a lot of lost users, which is a big shame. Option 1 or 2, it will also result in a lot of work to be done to make this work. Effectively, option 3 is in place today already.

I have to think about what to do next. For now: you can clone the Github repo and install Zed that way, instructions are in the README.

If anybody has any creative ideas, get in touch.

Update: It looks like Google just put Zed back in the Web Store.

The Changelog: 1.1.0

Posted: 2015-02-24

This is a big one: we have yet another way to edit files in Zed: Zedd. No, that's not a typo. Zedd is the Zed daemon, a process you can run to expose a file system to Zed that offers a bit more flexibility, most importantly: the ability to run external programs (think: linters, git etc.). Yep, finally! This new ability to run external programs is now supported in Zedd projects as well as Local Folder projects in the Standalone version. In other projects, all commands that require this feature will simply be hidden.

The Zedd webpage describes how to set up Zedd on your machine (local or remote), but the short answer is:

sudo npm install -g zedd

Here's the full changelog:

• New project type: Zedd Folder (for editing of local and remote files), see http://zedapp.org/zedd for more details on how and when to use this.
• Support for running external tools in Zedd projects as well as Standalone local folder projects. This enables new commands and packages like:
  • Tools:External:Insert Command Output
  • Tools:External:Filter Selection Through Command
  • Package (included by default): git-tools which exposes commands like:
    • Git:Grep
    • Git:Add
    • Git:Commit
    • Git:Diff
    • Git:Status
    • Git:Blame
• Chrome App only: ability to just open one or a few files (rather than a whole folder)
• Zed version number now visible in project picker
• Theme split: you can now configure your "window theme" and "editor theme" separately (by TheKiteEatingTree)
• Multiple sandboxes: many tasks (such as checking, code completion) now happen in separate sandboxes (= threads) to speed up certain tasks. Sandboxes are automatically cleaned up after a timeout.
• Better handling of projects that are no longer accessible (removed from recent project list when opened and alert window)
• Modes:
  • JSX mode (React) updated to 0.12

The Changelog: 1.0.1

Posted: 2015-01-19

A belated happy new year! We start this year with a new Zed release. This one's primarily fixes to work with the lates and greatest ACE (our editor component), and other bug fixes and some new modes included.

Here's the changelog:

• Bug fixes
• Upgraded to latest ACE, with improved Vim mode and other improvements
• Window size persistence tweaks
• Internal: old symbol databases are now cleaned up (after 2 weeks)
• Auto symbol indexing fixed
• Modes:
  • XML: Fixed beautify feature
  • New Elixer mode
  • New Elm mode
  • HTML: now also opens .htm files
  • Erlang: now also used for header files (opie4624)

Zed Finally Hits 1.0

Posted: 2014-09-22

After over 1.5 years of development (the last ~6 months close to full-time) here it is: Zed 1.0. Of course version numbers don't really mean much, but I thought it was good to finally hit 1.0. I've been using it for over a year full-time and I'm pretty happy with it.

This release brings some big changes as well as many minor tweaks:

• Unified UI: the separate project picker window is gone and has been integrated into editor windows (using same theming etc.). Similarly the first run, dropbox project, github picker windows have all been integrated as well.
• New preferences GUI offering a more friendly way of changing preferences, access it via Configuration:Preferences (Command-,/Ctrl-,). Fun fact: the preferences UI is actually just a Zed package that runs a web app integrated using Zed's built in web-server.
• Window restore on relaunch: All windows are now restored after Zed quit (Zed:Quit) and restart.

• Zed now also supports vim-style cursor movement. These keyboard combinations have the advantage of not having to move your hand to your cursor keys to move around. If you're familiar with vi or vim, they are generally the same keys you always use to move around together with the Alt key. Supported are:

  • Alt-h: move cursor to the left
  • Alt-j: move cursor down
  • Alt-k: move cursor up
  • Alt-l: move cursor to the right
  • Alt-w: move cursor one word to the right
  • Alt-b: move cursor one word to the left
  • Alt-0/Alt-^: move cursor to the start of the line
  • Alt-$: move cursor to the end of the line
  • Alt-Shift-g: move cursor to end of the file

  In addition, many of these also work together with Shift to use them to select text, e.g. Alt-Shift-H selects one character to the left.

• Native window chrome (title bar -- see screenshots above) is now used in the standalone version of Zed as well as the Chrome App version on Linux.

• The "Notes" space has been removed, due to too frequent data loss for users, we recommend to just use a local folder instead. If you were using Notes before and would like to access it still, enter "syncfs:" in the project picker search box and press Enter.

There is another big change coming to Zed, but it's not a technical one. As many of you will be aware, on April 1st I made Zed my day time job. Since I've been able to iterate on Zed a lot. From the get go this was an experiment both financially, but also: would I enjoy working working on this in a 1-person team (me) -- of course there's some open source contributions, but most of the work is still done by me. While I'm not really surprised that after 6 months Zed doesn't generate enough income to pay a reasonable salary, what was I did not predict was that after working 3 years from an office far away from my co-workers I'm ready for a change. Therefore, I've decided to take a job in the city where I live (at STX Next). For the first time in my industrial career I'll share an office with co-workers.

This also means that starting October 1st, Zed will once again become a spare time project. Naturally, I'll keep using Zed for all my programming (and since STX Next is a Python shop, I'll use it a lot for Python coding) so I'm sure improvements will keep coming, but probably at a slower pace.

Back to Zed 1.0 itself -- beside the changes above, here are some more cool ones:

• Syntax highlighting added for TeX/LaTeX, Ada, Assembly, Cobol, diff files, FORTH, .gitignore, Jade, Lisp, Objective-C, Pascal, R, Scheme, SQL (including MySQL and Postges flavours), SVG, Tcl, Typescript, Vala, and a bunch more, some of which I never heard of — if Ace supports it, now we support it too (by lalomartins)
• Standalone: Windows version now uses Zed icon on desktop.
• Default editor window size increased to 1024x768
• Linux window size restore improved.
• Fixed mouse UI UX issues (dragging and dropping of text now supported) (by nightwing)
• New commands:
  • Navigate:Last Edit Point (useful to go back to last location you were editing after jumping to a symbol)
  • Window:List (lists all open Zed windows and allows you to quickly jump between them)
• Keyboard shortcuts have been refactored here and there to better support non-Mac OSes (ChromeOS, Linux, Windows) -- check the command list when a keybinding you were accustomed to no longer works.
• Fixed Coffeescript display bug (by nightwing)

The Changelog: 0.13.0

Posted: 2014-08-14

Big new features in this release:

1. Github support: Support for opening, editing to Github repositories without the need to clone them -- right from Zed (screencast of this feature in action).
2. Embedded web server: serve (static) files right from Zed (run Tools:Static Server:Start).
3. Zedrem change: current Zedrem users using last version's userKeys: user keys are now no longer stored in the Configuration project, but instead in a token store. Your user key will have changed. For details, see below.

Here's the full list of changes:

• Github support: There's now a "Open Github Repository" item in the project picker. The first time you'll click it it'll ask you to create a Github token that it'll save (outside the Configuration project). After this a repository picker will appear, but you can also paste in full Github urls immediately. After picking a repo it'll ask you to pick a branch to edit.
  • Changes made to a Github project are saved locally on your machine into an IndexedDB database only. To commit the changes to Github, use the Version Control:Commit command (also available in the "Tools" menu if you have menus enabled).
  • You can reset any changes you made locally using the Version Control:Reset command.
  • You can create a branch (that'll open in a new window) using Version Control:Create Branch.
• Zedrem: configuration changes:
  • the "zedrem" key under preferences in the Configuration project is now ignored, you can remove this key safely.
  • User keys are now stored in a token store (as are the Github tokens), to retrieve your zedrem userKey run the Zedrem:Get User Key command.
  • If you like you can generate a new user key using Zedrem:Generate New User Key, a Zed restart will be required for this change to take effect.
  • The project picker now shows the status of your connection to your selected zedrem server at the bottom of the screen.
• Embedded Web server -- serve files right from Zed (works both in Chrome App and Standalone): this is available both as sandbox API (so you can implement your own servers -- see zefhemel/zcms for an example) and with a default static file server you can run out of the box.
  • Start static file server using the Tools:Static Server:Start command, this will also automatically open your browser. A port is automatically assigned.
  • Stop static file server using Tools:Static Server:Stop. The server will also automatically stop when you close your editor window.
• Drag and drop of binary files: this is not an advertised feature at all, but you can actually drag and drop files onto a Zed editor window and it'll "upload" them to your project. This now also works correctly with binary files, such as images.
• Currently open files (open in some split) will now appear in your Goto file list (although at the very bottom), Zed will do some session switcharoo to make this work correctly.
• Modes:
  • Improved C++ symbol indexing support (by cessen)
  • Dockerfile mode (syntax highlighting only)
  • Ruby: behavior support (auto-inserts matching braces etc.)

Also: did you know this site is now hosted on S3 and generated using the ZCMS Zed Package? See the zedapp/website repository for source files.

The Changelog: 0.12.1

Posted: 2014-08-01

This release has a much improved flow for editing files remotely using zedrem. Rather than having to copy and paste URLs every time, you can now pass in a user-specific key to the zedrem command and windows will automatically open up when you use zedrem. See the Edit Remote Files page for updated instructions. Note: if you already downloaded the zedrem binary onto a server before, be sure to redownload it to get these new features.

• Zedrem: The zedrem binary now takes an optional -key flag where the user key can be passed in. This key is automatically generated (see below) and can be requested using the Configuration:Zedrem:Get User Key command. When passed in, edit windows will automatically appear when you run zedrem.
• Zedrem: the ~/.zedremrc file (that is: the .zedremrc file in your $HOME directory) has been extended to have a userKey entry in the [client] section, this is useful if often editing files on the same server and not wanting to pass in the user key every time.
• To support the above there is now a zedrem preference in the Configuration (under preference). By default a server is configured there (remote.zedapp.org over SSL) and a userKey generated. The project picker will establish a persistent websocket connection with this server to enable the behavior described above.
• Zedrem: ownership of files is now preserved.
• Modes:
  • Improved C++ symbol indexing (by cessen)
• Key bindings:
  • Mac: Alt-Delete now added as alternative to Alt-D (by 11684)
• Zed now works on the Chrome Beta channel again (Chrome 37)
• Symbols are now listed in order that they appear in files, rather than based on length.
• Mutliple cursor bug fixes: moving and copying lines up and down now works with multiple cursors, as does whitespace trimming.
• Standalone: no more crashes with the splash screen.

The Changelog: 0.12

Posted: 2014-07-17

This is a big release, both internally (the whole code base has been refactored to use ES6-promises instead of node.js-style callbacks) and externally: there's now an optional "traditional" UI mode that shows you a persistent file tree on the left and a menu bar along the top. In addition, there's now a basic UI for managing snippets. Let's look at each of those individually.

UI Layout Picker

The first time you run Zed 0.12 you'll be greeted with the following pop-up:

This seems a bit contrary to the Zed vision, right? Offering a "cluttered mode." The reason for offering it anyway is that:

1. Some people just really like to have a file tree available at all times. It gives them a sense of context.
2. Other people are completely confused by Zed's minimalist UI. They open an editor and then have no idea what to do (even though there's a little zed::start doc that tries to explain it) -- there's no file tree, no menu. Now what?

So, for those people -- you can now make Zed look like this:

The menu bar is a curated (hardcoded) collection of commands that I think most people need or would like to know about. Ideally, you quickly learn the keyboard shortcuts (listed in the menu) and then disable the menu again, but hey... maybe it doesn't take that much space.

Snippets Manager

From early on people have had problems creating their own snippets and understandably so. Therefore I developed a package (zedapp/snippets) to make this easier. With a file active in a mode you want to manage snippets for run the Snippets:List command:

This will list existing snippets, add new ones and allows you to edit, and remove them (if not built-in). There's also a Snippets:Add command to immediately add a new snippet for the current mode.

Full changelog

• First-run screen that asks you to pick a UI layout. This is fully configurable later on, via:
  • New persistentTree preference that, when enabled, shows a file tree along the left.
  • New showMenus preference that shows a menu along the top with a (curated) list of common commands (and their key bindings).
• Snippet manager (Snippet:List, Snippet:Add) shows and adds snippets to the mode of the currently active session.
• File:New command (bound to Command-N/Ctrl-N) that opens the Goto UI with the directory of the currently active file prefilled.
• ZPM and Snippet manager use special "Zed UI" Ace mode that highlights "buttons" more like buttons, so you can tell they're clickable.
• Language modes can now have a completionTriggers key with characters that should automatically trigger code completion, e.g. . for JavaScript or -> for PHP. And as a result...
• Code completion now automatically triggers after typing a completionTriggers sequence of characters (this can be switched on/off with the autoTriggerCompletion preference).
• Big Manual overhaul
• Modes:
  • Scala symbol indexing
  • PHP follow complete support
  • Clojure follow complete support
  • Gherkin mode (by lalomartins)
• Themes:
  • Base16 theme (by farnoy)

Enjoy the release!

Want to contribute to Zed's development? Consider buying.

The Changelog: 0.11.5

Posted: 2014-07-09

This release incorporates some of the findings in How Zed is Actually Used (specifically better support for popular Zed languages such as Python, Ruby and PHP). It also includes two new packages as part of the distribution: follow-complete and bracket-check. In addition, various bugs have been fixed and features tweaked.

Let's recap what follow-complete does:

Follow complete is based on a few observations:

1. A lot of languages use a a.b and a.b() (and similar: a->b and a->b(), a::b) notations for package namespacing, property access and method calling, e.g. JavaScript, Go, C++, PHP, Ruby, CoffeeScript, C#, Java, Dart, Groovy, Lua, Rust, Python, Scala etc.
2. Programmers tend to name variables of certain types the same all over the code base: e.g. arr for arrays, el for HTML DOM elements.
3. Many languages use the a.b pattern for namespacing, for instance in Python after you import os you call methods on it via os.some_method(). Even without explicit imports, document.createElement() is an example of this in JavaScript.

If this is the case, wouldn't indexing any sequence of x.y, x.y(...), x(...).y and x(...).y(...) found in a code base, result in some useful completions when requesting code completion after typing x.? In that case, code completions could include y and y().

This is exactly what follow complete does. It uses Zed's new database APIs to index sequences of identifiers with .s in between and uses that for completion. As it turns out, if your code base is big enough, this gets surprisingly useful.

Again, here's a Go example. Go happens to use . as a package separator. Without any Go-specific knowledge, let alone knowledge of Go's APIs, Zed can now give the following completion for the datastore package (which is an Google App Engine specific package that is used fairly heavily in this particular project):

As you can see, although Zed "knows" that these are all methods, it doesn't show method argument names, simply because it doesn't know them. Potentially these method names could be matched with the symbol list, perhaps to also give argument names. Nevertheless, even without that getting some completion for APIs that are used in your project is quite useful.

Follow complete is currently enabled for: JavaScript, Python, Ruby, Go, Java, C#, Groovy, Lua, Nix, Rust and Scala.

Now let's recap what bracket check does:

There are many more language for which parser written in JavaScript are not yet available. And writing a high-quality parser for most languages is a lot of work.

So would it be possible to build some syntax tooling that helps at least a little in detecting common mistakes writing code?

This is what bracket check does: it checks nesting of brackets: e.g. [{(. Is this a full-blown syntax check? No (although... almost for Lisps), but it helps.

For each language, you only have to configure two things: the bracket pairs to match and the parts to skip over (such as string literals, comments) etc. Here is it in action:

Here's the full changelog for 0.11.5:

• Modes:
  • Ruby: symbol indexing
  • Python: improved symbol indexing
  • C: symbol indexing
  • PHP: improved symbol indexing
  • Java: symbol indexing
  • C#: symbol indexing
  • GLSL syntax highlighting
  • Stylus syntax highlighting
• Two new packages made part of the default distribution:
  • follow-complete: offers quasi-intelligent completions for x.y property accesses/method calls for many languages (currently enabled for JavaScript, Python, Ruby, Go, Java, C#, Groovy, Lua, Nix, Rust and Scala)
  • bracket-check: checks matching brackes (currently enabled for Clojure, Go, Dart, Python, Ruby, Java, and C#)
• Local (Chrome) directories with more than 100 files/directories did not show up completely before, this is now fixed (by farnoy)
• Projects are automatically indexed (for symbols and code completion) the first time they are opened, you can do this manually with Tools:Index Project.
• Search project now only searches known file types and does so more sequentially, which should prevent crashes
• Key bindings:
  • Default keybinding for Find:Next on Linux/Windows/ChromeOS is now Ctrl-G, and Find:Previous is now Ctrl-Shift-G.
  • Nativate:Reload Filelist is now bound to Command-Shift-L on Mac and Ctrl-Shift-L on Windows/Linux/ChromeOS as well as F5 on both.
• Dotfiles (files starting with .) now appear in the file list, except those defined in gotoExcludes (by default .git)
• Goto matching algorithm tweaked, now also does fuzzy matching again (although with a lower score).
• File renaming fixes (by TheKiteEatingTree)

As always, if you have Zed installed, Zed should be upgraded automatically. If not, you can download Zed from its download page.

Thanks to all who contributed!

How is Zed Actually Used?

Posted: 2014-06-27

A few releases back, Zed added opt-in analytics. Upon first launch, Zed asks the user whether he's ok sending some anonymous usage data to further improve Zed. It has been about two months since this release, so it's time to share some results.

Of course, there's no way of telling how many people have not opted into the analytics, so total numbers don't say much. However, presumably percentages give some indication how Zed is used.

Zed Chrome vs. Standalone

As of version 0.11 Zed comes in two editions:

1. Zed Chrome, the Chrome App installable (and discoverable) via the Chrome Web Store, available for all platforms that Google Chrome runs on.
2. Zed Standalone, based on node-webkit, available for Mac, Linux and Windows.

At various points in time, features have been discussed that would need to introduce APIs (such as running CLI commands from sandbox code) that would only be implementable in the standalone version of Zed. Therefore, it's interesting to see how many users would be impacted if we'd start adding features that only work in the standalone version.

Based on data from the past two months, it turns out 79% of Zed usage comes from the Chrome App. Note that this is not just counting launches but all events -- meaning that "real" usage counts more towards this percentage than a single launch.

The question is: Why is the Chrome App so much popular? I can think of a few reasons:

1. The Chrome App is promoted in the Chrome Web Store which brings in a lot of new users that otherwise would probably never had heard of Zed.
2. The Chrome App has been around longer and people haven't switched.
3. The Standalone version seems to have some unexplainable power consumption issues, at least on Mac.
4. The standalone version does not have automatic configuration syncing, nor the syncing Notes project, nor Dropbox support.
5. The standalone version does not run on Chrome OS.

A few of these can be further investigated. For instance, option 4 includes Dropbox support and the Notes project. Are those heavily used features? We have some data on that. Here are the used "file system types" ranked by usage:

Translation of the names used:

• local is Chrome's Local Folder file system implementation.
• config is Chrome's syncing Configuration file system.
• node is the Standalone version's node.js-based local file system.
• manual is the Manual project,
• syncfs is the syncing Notes project (Chrome only).
• http/https is remote editing of directories using zedrem.
• nwconfig is the Standalone version of the Configuration project.
• dropbox is the Dropbox file system (Chrome only).
• textarea is the new "Edit in Zed" feature (Chrome only).

From this we can see that editing of local folders (local + node) accounts for 57% of all projects being opened. Chrome-specific features (Notes and Dropbox support) add up to about 10%, which isn't huge but still significant. Whether or not people choose the Chrome version for this reason -- they at least use it.

Language Modes

Zed has syntax highlighting support for many different languages. In addition, for some languages it has language-specific features, such as integrated syntax checking, linting and code beautification. However, does that impact usage? What languages do people use Zed for where language-support can be further improved?

Here are the top 10 most popular language mode by usage:

Zed's most advanced language mode is JavaScript, and indeed, that's topping the chart. More surprising are "text" and "markdown". Markdown mode has syntax highlighting and preview support, but could be further improved. Also when you open a new project with Zed a zed::start document shows that uses markdown mode, so that may skew the results. Interesting items on this list are PHP, Python, C, and Ruby. PHP has a syntax checker, but Python, C and Ruby don't have very good language features yet. Definitely something to work on in further releases.

Goto vs the File Tree

Zed has championed its Goto functionality. It grows out of the belief that the "type a few characters in a search field" way of navigating files is better than using a file tree. Nevertheless, Zed still features a file tree, although it's not always visible. So, what are users actually using?

Based on data collected, it appears that in Zed 87% of the time Goto is used to navigate between files. Note that this doesn't prove that Zed made the right call, just that users use Zed the way it was intended.

Popular Commands

Zed is command based, commands can be bound to keys, but also be executed by name. Naturally, most commands are executed through a key binding (e.g. you use the Up key rather than executing the Cursor:Up command manually), but commands can be executed by name too. It's interesting to see what commands are commonly executed by name -- because they're worth assigning a keyboard shortcut to.

Here's the top 10:

Three of these File:Rename, File:Delete and Navigate:Reload Filelist are file-management related commands. I use these three myself too, although I bound the latter to a key in my own configuration. Navigate:Reload Filelist is an annoying one. It's a command that ideally you'd never have to execute manually, Zed should just detect changes itself. Sadly, in practice this is hard to do without rescanning your whole file tree (potentially huge) every minute or so (and is every minute even enough?) as there's no way to "watch" a directory tree at an OS level (at least not in most file systems Zed supports, such as the HTML5 based local file system used in Chrome).

Operating Systems and Geography

What operating system do most Zed users use? Here's the list:

1. Windows: 42%
2. Mac: 16%
3. Linux 64-bit: 16%
4. Chrome OS: 15%
5. Other: 11%

Time to pay attention to all those Windows users, for sure. Also, there's a significant number of ChromeOS users, which I find intriguing. It'd be interesting to further investigate how ChromeOS users use Zed, because most likely they don't use the ability to edit local folders -- because there are none (other than the Downloads folder).

And just for fun -- here's the geographic distribution of Zed users:

This begs the question: People of Greenland -- y u no use Zed?

Actionable items

So, what do we do with this information? Here's a few action items, at least for me:

1. Test every release on Windows, since this is how the largest percentage of users will experience Zed.
2. Explore what new features can be added to the following language modes:
   • markdown/text
   • PHP
   • Python
   • Ruby
   • C
3. See what keyboard shortcuts can be assigned to reloading the file list and renaming a file.
4. Write up a blog post about effective ways of using Zed on ChromeOS.

The Changelog: 0.11.4

Posted: 2014-06-16

This release has some solid improvements for its Goto feature (whose matching algorithm was improved and now also searches symbols by default) and project-wide search (with case insensitive and regex support and support for only searching files matching a file pattern) as well as some other, more minor changes that should smoothen the experience.

Here's the full changelog for 0.11.4:

• Goto improvements:
  • Goto (Ctrl-E/Command-E) now also includes matches from the symbol index (without having to use the :@ or @ prefix)
  • New fuzzy matching algorithm:
    • Now also searches file directory names by default.
    • Character sequences now have to match exactly, spaces can be used as wildcards. E.g. "bcd" matches "abcde" but not "abccde", however "bc d" will match "abccde".
    • Scoring is now based on how close the pattern match is to the end of the path, the closer to the end, the higher the score. This favors file name matches over directory name matches, which is usually what you want.
• Project search improvements:
  • Now integrated into goto (Ctrl-Shift-F/Command-Shift-F)
  • New query syntax: searchphrase [-flags] [filepath pattern] for example: hello or hello *.js or hello -i (case insensitive search) he*llo -ie /docs/*.md (case insensitive regex search). Supported flags: -i (case insensitive matching) and -e (regex match).
• Batched undo: you no longer have to run undo for every character, certain operations (including typing and deleting) are now batched up.
• Better behavior when "closing" a split (going from 2 to 1 splits): the split left will now contain the file in the active split.
• Bug fix: Fix window positioning after restore
• Bug fix: Better recovery from broken configuration
• Bug fix: No longer show stacked "file changed" dialogs for each change
• Sandbox API:
  • zed/http API now also has POST, PUT, DELETE etc. support and returns not just content, but also headers (by akoenig)
  • zed/preview API now supports opening the preview split (by akoenig)
• Mode:
  • Clojure/ClojureScript mode: symbol indexing and better indenting

As always, if you have Zed installed, Zed should be upgraded automatically. If not, you can download Zed from its download page.

Thanks to all who contributed!

Language-Agnostic Tooling

Posted: 2014-05-22

One of many challenges I struggle with working on Zed as one-person company is what to spend your time on. Should I just fix bugs? Write documentation? Brag about Zed features in blog post? Write features that make great demos to draw people in? Should I build payment systems to increase income?

Basically I have to ask myself: how can I optimize my user-happiness-increase/hour ratio?

As you may be aware I have a background in programming language engineering. I have some experience in writing programs that parse and statically analyze programs. At Cloud9 I worked on language intelligence features. At one point I spent a month or two (and my colleague quite a while more after) building a type inference engine for JavaScript. It's pretty difficult to do, but the result nice and allows for quite accurate code completion of methods and properties (try it out). However, building such a feature for one language takes a developer months.

This raises the question: should take the same approach for Zed? Should I spent months building a super accurate code completion engine for one language? That would mean that nothing much else gets done. Is that the best use of time?

I decided that, no, it's not. Instead, I started to think about create ways to have similar features in Zed, but built with much less effort.

Thus far I came up with a few examples of low-hanging language tooling fruit, if you will. They're much more pragmatic and often less accurate than their full-blown, expensive to develop alternatives (= custom implementations for each language). Nevertheless, I found them to very helpful and to have a good pay-off/hour spent ratio.

1. Symbol Indexing

This is the most obvious one: Zed's symbol indexing. Rather than writing complicated parsers for the languages Zed supports, I chose an ad-hoc approach with simple regular expressions. This is fast to develop, fast to execute and works reasonably accurately (depending on the language and regular expression).

How it works is simple: for each language you have to write a function that takes source code and returns a list of symbols and their position in the file. This is how it worked for a long time. One JavaScript module with one indexing function per language mode. However, this could be further improved, because the pattern of these symbol indexers was always the same: define a few regular expressions, then run over the string to find matches and turn these into symbol info to return.

So recently I created a single generic Regex Indexer command, that can simply be configured for each language individually without having to write any JavaScript. For instance for Go:

commands: {
    "Tools:Index": {
        scriptUrl: "/default/command/regex_indexer.js",
        inputs: {
            text: true
        },
        regexes: [{
            regex: "type\\s*([A-Z-a-z0-9_]+)",
            symbolIndex: 1,
            type: "type"
        }, {
            regex: "func\\s*(\\([^\\)]+\\))?\\s*([a-zA-Z0-9_\\-\\$]+\\s*\\([^\\)]*\\))",
            symbolIndex: 2,
            type: "function"
        }]
    }
}

Or Clojure:

commands: {
    "Tools:Index": {
        scriptUrl: "/default/command/regex_indexer.js",
        inputs: {
            text: true
        },
        regexes: [{
            regex: "\\(defn-?\\s*([A-Z-a-z0-9_\\-]+)",
            symbolIndex: 1,
            type: "function"
        }]
    }
},

Regular expressions are still ugly beasts, but they work. And adding simple symbol indexing to any language is now often a matter of half an hour up to an hour (depending on your regex skills).

2. Bracket Checking

One nice feature of Zed is its built-in support for parsers and linters. For instance, Zed's JavaScript mode has JSHint built-in. JSON, PHP, Lua and CoffeeScript also have syntax checkers as part of their language mode, because there happen to be parsers for these languages written in JavaScript already, it's easy to integrate into Zed and showing markers for syntax errors inline saves you from one more trip to the terminal (or browser) and back.

That's all very nice, but there are many more language for which parser written in JavaScript are not yet available. And writing a high-quality parser for most languages is a lot of work.

So would it be possible to build some syntax tooling that helps at least a little in detecting common mistakes writing code?

One thing I've been experimenting with is my new bracket-check package (install via ZPM: gh:zefhemel/bracket-check). The name hints at what it does: it checks nesting of brackets: e.g. [{(. Is this a full-blown syntax check? No (although... almost for Lisps), but it helps.

Bracket-check's configuration file shows how to enable and configure it for various languages. You only have to configure two things: the bracket pairs to match and the parts to skip over (such as string literals, comments) etc. Here's the definition for Go:

go: {
    commands: {
        "Tools:Brackets:Check": {
            scriptUrl: "./check.js",
            internal: true,
            inputs: {
                text: true
            },
            brackets: ["()", "{}", "[]"],
            skip: [["\"", "\""], ["`", "`"], ["//", "\n"], ["/*", "*/"]]
        }
    },
    handlers: {
        check: ["Tools:Brackets:Check"]
    }
},

There are similar configurations for Clojure and Dart. More are trivial to add.

3. Property and Method Completion

The most interesting example (in my opinion) is my experimental follow complete package. Follow complete is based on a few observations:

1. A lot of languages use a a.b and a.b() (and similar: a->b and a->b(), a::b) notations for package namespacing, property access and method calling, e.g. JavaScript, Go, C++, PHP, Ruby, CoffeeScript, C#, Java, Dart, Groovy, Lua, Rust, Python, Scala etc.
2. Programmers tend to name variables of certain types the same all over the code base: e.g. arr for arrays, el for HTML DOM elements.
3. Many languages use the a.b pattern for namespacing, for instance in Python after you import os you call methods on it via os.some_method(). Even without explicit imports, document.createElement() is an example of this in JavaScript.

If this is the case, wouldn't indexing any sequence of x.y, x.y(...), x(...).y and x(...).y(...) found in a code base, result in some useful completions when requesting code completion after typing x.? In that case, code completions could include y and y().

This is exactly what follow complete does. It uses Zed's new database APIs to index sequences of identifiers with .s in between and uses that for completion. As it turns out, if your code base is big enough, this gets surprisingly useful.

Again, here's a Go example. Go happens to use . as a package separator. Without any Go-specific knowledge, let alone knowledge of Go's APIs, Zed can now give the following completion for the datastore package (which is an Google App Engine specific package that is used fairly heavily in this particular project):

As you can see, although Zed "knows" that these are all methods, it doesn't show method argument names, simply because it doesn't know them. Potentially these method names could be matched with the symbol list, perhaps to also give argument names. Nevertheless, even without that getting some completion for APIs that are used in your project is quite useful.

If you want to try out follow complete as it is today, install the gh:zefhemel/follow-complete package, and after installing be sure to reindex your whole project with the Tools:Index Project command.

These are just three examples of useful language tooling features that are relatively language-agnostic. I'm constantly thinking about new ways to do this sort of stuff.

Any suggestions are welcome.

About Zed and this blog post

A little while back I announced I would make working on Zed my day-time job. Zed was open source before, and I kept it that way. In addition to the openness in code I also want to be open in design decisions and other lessons learned during this endeavor. This post is one example of that. Do you want to support me in working in this open way? Consider "buying" Zed.

Zed Package Manager

Posted: 2014-05-15

A while back we added a very cool feature to Zed called ZPM, which we never advertised. As you may have guessed it stands for "Zed Package Manager" and the first implementation was done by Andrew Stephan.

While ZPM will likely be dramatically improved in the future, let me lay out how it works right now: how to install and manage packages and how you can create your own packages that extend the functionality of Zed.

Let's start with installing and managing.

Installing and Managing Packages

Installing a Zed package is straight forward, all you need is a copy of Zed (duh) and the URI of the packages you want to install (more on this URI later).

To see what packages are currently installed, run the Tools:Zpm:Installed Packages command from an editor window, this will open up a session listing all currently installed packages as well as the ability to uninstall, update, update all and install new packages. This list uses a regular Zed file as user interface, you can operate it with the keyboard (by moving your cursor to a "button" and hitting Enter) or mouse (by clicking on a "button"). Note that packages that come preinstalled with Zed can not be uninstalled (they have an uninstall option, but this will have no effect).

To install a new package hit the "Install New" button. Alternatively, you can run the Tools:Zpm:Install command (from anywhere, doesn't need to be from the installed packages view). Then, enter the package URI. A simple package (that we'll developer later in this post) you can try is gh:zefhemel/sample-zed-package After installation completes you should now have a new My Test Command command that says "Hello world" when you run it.

Awesomeness.

Zed's package ecosystem is fully decentralized at this time. There is no central repository like Sublime and Atom have. However, for now we can use the Zed wiki for this purpose.

Under the hood

Packages are essentially not much more than a way to easily install some files into your configuration project. When you look in your Configuration project under the /packages directory (the tree is the best way to explore here: Command-T/Ctrl-T), you'll see that there are already a number of packages pre-installed. At the time of this writing the following modes are distributed as Zed packages:

• JavaScript mode
• JSON mode
• JSON5 mode
• PHP mode
• CSS mode
• JSX mode

Over time, more and more built-in modes will be migrated to Zed packages, but this is a significant undertaking, so it will take a while.

If you open up the /default.json file in your Configuration project, you'll see a packages key listed:

packages: [
    "gh:zedapp/javascript-mode",
    "gh:zedapp/json-mode",
    "gh:zedapp/json5-mode",
    "gh:zedapp/php-mode",
    "gh:zedapp/css-mode",
    "gh:zedapp/jsx-mode"
]

ZPM ensures that each of the packages listed there are installed and kept up-to-date automatically. If you installed a package yourself you'll find it listed in your /user.json file.

So where do these packages come from? A package URI can either be a full HTTP link to a directory that contains a package.json file, or, preferably, a shortcut can be used. Shortcut URIs are prefixed with (currently) either gh: (for github) or bb: (for bitbucket). These expand to (in case of the gh:zedapp/javascript-mode) to https://raw.githubusercontent.com/zedapp/javascript-mode/master/. ZPM expects, by appending package.json, to find a JSON file similar to this:

{
    "name": "JavaScript mode",
    "uri": "gh:zedapp/javascript-mode",
    "version": "0.2",
    "description": "JavaScript mode for Zed",
    "files": [
        "beautify-js.js",
        "beautify.js",
        "check.js",
        "index.js",
        "jshint.js"
    ]
}

The required keys are:

• uri: This URI has to exactly match the URI people will use to install your package (if not, you can expect weird behavior).
• name: The name of the packaged used to display in the UI
• description: Description used to display in the UI
• version: Version number, when you increase this number people with the package installed will automatically updated (ZPM checks for updates every few hours)
• files: A list of relative paths of files that the package consists of (package.json and config.json are included automatically, you don't have to add them to this list).

Every package, in addition to package.json should at least have a config.json file that contains regular Zed config, for instance defining new commands, modes, themes or whatever your package offers. Here's the config for the JavaScript mode. The rest of the files are typically JavaScript files that implement the commands listed in the config file.

Developing your own packages

Preparation

The first thing you'll want to do is store your configuration project in a local directory (rather than SyncFS, which is the default in the Chrome edition of Zed). The reason is you'll want to upload or check in your package files to e.g. Github so you need direct access to the files. To do this in the Chrome version run the Configuration:Store in Local Folder command. If you're using the standalone version, your configuration is stored in ~/.config/zed/config on Linux and ~/Library/Application Support/zed/config on Mac. If you want to store your config elsewhere use the Configuration:Set Configuration Directory command. Personally I store my configuration somewhere in my Dropbox folder.

Decide where to host your package

I suggest you create a public Github or Bitbucket repository for your package. In this blog post we'll use hosting it on a Github repository called sample-zed-package hosted on my zefhemel github account as an example.

Create the package

The Configuration project has some developer-specific commands to get you started quickly. Run the Tools:Zpm:Create Package package. This will prompt you for the package URI. We'll enter gh:zefhemel/sample-zed-package (replace with whatever your repo is).

This will create two files in /packages/gh/zefhemel/sample-zed-package/ named package.json and config.json. It will open package.json by default. Update its default values to whatever you want, e.g.

{
    "name": "My First Zed Package",
    "uri": "gh:zefhemel/sample-zed-package",
    "version": "1.0",
    "description": "A useful new package",
    "files": []
}

Next, let's open up the config.json in the same directory (tip: Open up Goto with Command-E/Ctrl-E and press the spacebar to complete the path to the current directory, then pick config.json).

For our purposes we'll just define a sample command:

{
    commands: {
        "My Test Command": {
            scriptUrl: "./command.js"
        }
    }
}

Next, let's create /packages/gh/zefhemel/sample-zed-package/command.js (again using the same Goto trick):

var ui = require("zed/ui");

module.exports = function(info) {
    return ui.prompt("Hello world!");
};

As you can see a Zed command is implemented as a CommonJS-style JavaScript module. It exports a function taking a single argument (info) that, depending on your configuration contains useful information about the context in which the command was executed. Zed comes with a growing API that allow you to interact with the editor, all these APIs are available by require'ing modules under zed/*. In your configuration project look under /api/zed to see what's available. Admittedly, documentation is lacking on these APIs, so until that improves it is recommended to look at a lot of examples. For this, the Configuration project is your oister: all code for all installed packaegs, modes, all themes and many commands is all there to be inspected. If you have specific questions, please join the Zed Google Group and ask!

To test it out our newly created project we need to add our package to our packages list in /user.json:

packages: [
    "gh:zefhemel/sample-zed-package"
]

Automaticically, your configuration will reload and your new command should be available. Run My Test Command to try it out. If it's not there, try reloading your configuration explicitly using Configuration:Reload. You will have to run this command every time you change your config.json file. Another command you'll need a lot is the Sandbox:Reset command, which will make sure your JavaScript code is reloaded the next time it's invoked.

Publishing your package

Before publishing our package we have to remember one important thing: to update our package.json to include all our extra files (files other than package.json and config.json). Luckily Zed offers a convenience command for this: Switch to your package.json file and run Tools:Zpm:Update Package.json File List. This command will automatically scan your project and update your package.json to include the command.js file.

This is where the Zed-part ends. In brief what what's left to do is to turn our package directory into a git directory, commit all files to it and push it to github.

To do so, open a terminal and cd to your Zed configuration directory, then cd into our package directory:

$ cd packages/gh/zefhemel/sample-zed-package

Initiallize a git repository there, add all files and commit:

$ git init
$ git add *
$ git commit -m "Initial checkin"

Add the github repository we created earlier as a remote and push to it. In my case:

$ git remote add origin git@github.com:zefhemel/sample-zed-package.git
$ git push -u origin master

Done!

Testing your package

Before telling all our friends, all we have to do now is test our package. For this purpose I always have a separate Zed copy lying around (either I use the standalone version for development and the Chrome version for testing or the other way around, or I use a Zed chrome install in the Canary version of Chrome). Alternatively you can switch your Zed configuration to a new, clean, directory (and switch back later). The goal is to have a Zed install without your package already installed. You can also use another computer (e.g. your Chromebook) of course.

To install your package, in your "clean" Zed run the Tools:Zpm:Install command and enter your package's URI. If all works out well your command should now be available. Rather than running Tools:Zpm:Install, you can also update your packages list to include your package's URI in the Configuration by hand. The effect should be the same.

Works? Congratulations, be sure to add your package to our wiki page!

Development and debugging tips

You should treat the Configuration project like a development environment for developing Zed packages. I usually start out with adding my new modes and commands to my /user.json file, and if it works I migrate them to a package.

Logging: Every project has a zed::log file that lists notices, errors and warnings from Zed and its sandbox code. If your package or script doesn't work, always check zed::log first for pointers to what may be wrong. For debugging purposes you can use console.log etc. in your JavaScript, whose output also appears in zed::log (prefixed with [Sandbox]).

Reloading/restarting: Since Zed strictly separates all the package and extension code from the editor itself by running it in a sandbox, it is possible to edit your packages and test the result immediately without restarting Zed or reopening your editor window. What you have to remember is that in order to reload your configuration file (config.json) you can just run the Configuration:Reload command. And if you made changes to your JavaScript files, you just run the Sandbox:Reset command to have Zed reload those files to see your changes in action.

Protip: If you do a lot of Zed package development, it may be a good idea to bind the Configuration:Reload and Sandbox:Reset commands to a key combination. I have this in my /user.json:

keys: {
    "Sandbox:Reset": "Ctrl-Shift-S",
    "Configuration:Reload": "Ctrl-Alt-Shift-S"
}

Common problems:

• If your configuration keeps reloading constantly, after you add your package to the packages key in your /user.json a likely cause is that the URI listed in your package.json does not match the URI used in the package's path. That is, if your package URI is gh:abc/def your package.json has to be located in /packages/gh/abc/def/package.json, if it's not, looping behavior can occur.
• If you see 404 errors in zed::log when installing or running your package, this is likely a problem with your files list in your package.json. If the 404s occur while installing a package, make sure that you have checked in a package.json where all files listed under files exist. If it happens while testing a package after installation, make sure that the file that Zed claims it cannot find was listed under the files key in your package.json.

And that's it! Let's see what you can build!

The Changelog: 0.11.3

Posted: 2014-05-14

This release contains some big infrastructural changes (primarily the switch from the /tags ctags file to using IndexedDB (a fast LevelDB-powered in-browser database) that will enable some cool language-intelligence features that will most likely be released as separate Zed packages in the near future. In addition, icons are now used in the symbol list and auto complete to signify whether they're functions, types, snippets etc. Some indexers (e.g. the ones for JavaScript and Go) now also includes arguments to functions in the symbol list.

Here's the full changelog for 0.11.3:

• ZeDB: a simple abstraction layer on top of IndexedDB. Packages and other sandbox code can create their own data stores and indexes (via the databases key in configuration) and read, write and query that data quickly. This system is current primarily used for symbol indexing and code completion (see below).
• Complete rework of generic project indexing system, now storing symbol information in IndexedDB for faster retrieval (than the /tags file in ctags format that it used to use).
• Support for full function signatures and icons in symbol list (Command-R/Ctrl-R) and code completion for certain languages (JavaScript, Go, others still have to be updated)
• General-purpose regex-based symbol indexer allows to add basic symbol indexing to modes without writing any JavaScript (see Go and JavaScript mode for examples).
• Additional multi-cursor commands (Cursor:Multiple:Add Above Skip Current and Cursor:Multiple:Add Below Skip Current by thebishopgame)
• Standalone:
  • No longer crashes when an exception occurs
  • Do not save window position for minimized windows (by nightwing)
• Modes:
  • New matlab mode (by timothyrenner)
  • Markdown: disable strip whitespace (by robru)
  • Handlebars mode (by chenxsan)

Thanks to all who contributed!

The Changelog: 0.11.2

Posted: 2014-04-29

Before we get to the changes in this release, a little update on the funding situation. We're currently at $115/week, which is still way off from making this project sustainable. So, if you're a Zed user, consider buying!

Both the Chrome and Standalone version should be upgraded automatically to 0.11.2. If you somehow can't wait for the standalone version to self-update, feel free to download it again from the download page.

This release brings one major feature for Chrome Zed users: Edit in Zed. Edit in Zed is a new Chrome Extension that works in cooperation with the Zed Chrome App, so be sure you have both installed when testing it. What it does is add a little icon on any textarea on any website that shows up when you hover your mouse over it. When you click the icon, Zed will open allowing you to edit the content of the text area, constantly writing back the content. By default Zed launches in text mode, but you can, of course, easily switch to Markdown mode, use preview etc. This feature is fantastic when writing a lot of content on a web page (in a CMS, for instance). Try it out! Here's a little screencast that demoes the feature.

Here's the full changelog for 0.11.2:

• New "Edit in Zed" Chrome Extension for Zed Chrome version: adds a little on-hover Zed icon to textareas on any website in Chrome, when you click it, you can edit the contents of this text area using Zed. Download the extension from the Chrome Store.
• Usage data collection: upon first launch you will be asked to agree to collect anonymous usage data. This helps us to see what features (e.g. language modes) people are using and what to improve.
• ZPM fix github package support (by conradz)
• Fix: code beautification (formatting) for selection works now
• Scroll past end of editor preference (by TheKiteEatingTree)
• Modes:
  • JSX (react.js) mode now has JSHint support
  • Ruby mode now also works on Vagrantfile
• Various minor bug fixes (by robru and others)

Thanks to all who contributed!

The Changelog: 0.11.1

Posted: 2014-04-23

This is the first update to be pushed for both the Chrome and Standalone version, so we'll be finding out if auto-update works in the standalone version!

Before we get to the changes in this release, a little update on the funding situation. Last time we crossed the $100/week mark on Gittip and there is has been some growth, but not that much -- we're currently at $108/week, which is still way off from making this project sustainable. So, if you're a Zed user, consider buying!

Both the Chrome and Standalone version should be upgraded automatically. If you somehow can't wait for the standalone version to self-update, feel free to download it again from the download page.

This release saw even more growth in community contributions. I'm very excited about this -- Zed is really becoming a team effort.

Here's the changelog for 0.11.1:

• New auto save (which should be friendlier for people using build systems with file watchers). The new version always saves when switching between splits and files and when the editor loses focus. What's new is the ability to switch off timed saves (set to saveTimeout to 0 to disable).
• Commands to move splits around: (Split:Move To *)
• New splits now receive focus automatically
• Version number now appears in project picker
• Fixes to file tree (by eltuerto)
• Sandbox messages now appear in zed::log for standalone version
• Configuration now uses JSON5 format (which supports comments etc.)
• Fix to trim remote URLs when pasting (by surma)
• ZPM auto update fixes
• Scroll past end preference (scrollPastEnd) (by TheKiteEatingTree)
• Modes:
  • PHP mode now includes a parser and reports parse errors
  • Livescript mode (by maninalift)
  • Groovy mode (by calebmpeterson)
  • JSX mode
  • Improved Markdown preview styling (by ghotsla)

Thanks to all who contributed!

Dependency Injection in JavaScript Using Architect

Posted: 2014-04-11

Architect is an open source dependency-injection framework for both server and client-side JavaScript code. Wikipedia defines dependency injection as follows:

Dependency injection is a software design pattern that implements inversion of control and allows a program design to follow the dependency inversion principle.

Very impressive. The way I think of it -- at least in Architect's implementation -- is simply as dynamic imports. Whereas usually you import (or require) a specific module by listing its path, dependency injection allows you to postpone the decision which module to load until run time.

Why is this useful? Here's a few reasons:

1. Testing: it becomes easy to test modules if you can easily switch out imported modules with mock implementations.
2. The module to pick may depend on the context in which the application is run (e.g. arguments passed upon launch, or the runtime environment).

The second is the reason I recently refactored Zed to use Architect as part of the effort to offer both a Chrome App and a node-webkit based version.

What does such a refactor entail? Essentially it means turning require.js modules into Architect plugins.

File systems

Here's roughly what Zed's require.js modules looked like before:

define(function(require, exports, module) {
    var fs = require("./fs/local");
    module.exports = {
        method1: function() {
            fs.readFile("/.zedstate", function(err, content) {
                // Do something useful
            });
        }
    };
});

If you know a little bit about Zed's feature set you can spot the problem with this pseudo module: Zed supports many types of file systems and this ties our module to one in particular. Currently Zed supports:

• A RESTful HTTP file system (used for editing remote files)
• A local file system (using Chrome's APIs) for editing local folders
• A dropbox file system for editing files on Dropbox
• A file system that syncs to Google Drive (using Chrome's SyncFS APIs) used for the Notes and Configuration projects.
• A read-only static file system that reads files from the application bundle (used for the manual project)
• A node.js-based file system for accessing local folders (for node-webkit)
• A union file system that combines any of the above (used for the Configuration project)

Clearly, a require call to any particular file system module makes our module immediately dependent on a particular file system implementation. Even wrapping the require in an if-statement wouldn't scale, since there are quite a few options.

Alright, so now let's have a look at the Architect version, which is only slightly more verbose:

define(function(require, exports, module) {
    plugin.consumes = ["fs"];
    plugin.provides = ["mymodule"];
    return plugin;

    function plugin(options, imports, register) {
        var fs = imports.fs;
        var api = {
            method1: function() {
                fs.readFile("/.zedstate", function(err, content) {
                    // Do something useful
                });
            }
        };
        register(null, { mymodule: api });
    }
});

Rather than exporting the API immediately, the module now exports a single function plugin that takes three arguments:

• options: containing any options for the plugin.
• imports: containing a key with the API of each consumed service
• register: the function to be called to register the services provided by the plugin.

In addition, two properties are tacked onto the plugin function object:

• consumes: an array all services that this plugin relies on. Note these are symbolic names, not module paths.
• provides: an array of all services that this plugin implements and exposes.

Note that all the requires are gone and have been replaced by "consumed services." The plugin.consumes = line says "this plugin [module] requires a service named 'fs' to function." Any other plugin that provides the "fs" service can now be injected here, as long as the interfaces match. As you can see in Zed's codebase all file system plug-ins expose exactly the same methods and all provide the fs service.

So, how does an fs service get injected? That's Architects job, the plugin doesn't have to worry about it. In practice, the way this works is that the project to be opened is encoded in the URL passed to the editor.html page that hosts the editor. Based on the request parameters a piece of code figures out which plugin providing the fs service is to be loaded. More on composing plugins into applications later on.

Runtime-specific APIs

Switching out file system modules is just one useful application of Architect in the implementation of Zed. Another one is abstracting from APIs specific to the runtime environment.

As I mentioned, the challenge was to get Zed to run both as a Chrome App as a node-webkit app. Each of these provide certain APIs (e.g. to get access to the local file system, window management, etc.) that are different between the runtimes. The way this is solved in Zed is by wrapping these APIs in Architect plug-ins that have two implementations: one for Chrome and one for node-webkit.

A clean example of this are the window plugins for Chrome and for node-webkit (note the consistent use the .chrome.js and .nw.js filename patterns to make clear that these plugins are runtime specific). Both these plug-ins implement the same methods:

• close() to close the current window
• create(...) to create a new window
• fullScreen() to toggle full-screening the window
• maximize() to toggle maximizing a window

etc.

Any plugin that consumes the window service provided by this plug-in will now call into the runtime-specific implementation. Examples of this include the title bar and project picker.

Pretty cool right?

Putting it together

So, how do we compose multiple plug-ins into an application? For this we use a plugin list. In its most basic form:

var plugins = ["./mymodule", "./fs/local"];

architect.resolveConfig(plugins, ".", function(err, config) {
    architect.createApp(config, function(err, app) {
       // At this point we have an instantiated application
       var mymodule = app.getService("mymodule");
       mymodule.method1();
    });
});

In essence you specify a list of plug-ins to load and then create an application out of them, which will link them all up. Now, replacing ./fs/local with ./fs/sync would switch mymodule to another file system implementation, without having to change the file at all.

However, in practice, almost all file system plug-ins require extra arguments. These can be passed by not using the plain-string notation, but the object notation, e.g.:

var plugins = [
    "./mymodule",
    { packagePath: "./fs/local", dir: "/etc" }
];

The packagePath contains the require.js module that contains the plug-in (as before), and any other provided options are passed into the plug-in via the options argument to the plugin function.

And that's pretty much everything there's to it! For some examples of real world architect plugin lists and instantiations, have a look at boot.js (for the editor windows) and open.boot.js (for the project picker). Both of these build up a list of plugins that are laregely shared, but add some plugins based on the runtime environment (based on the isNodeWebkit variable).

Caveats

Something to be aware of using Architect:

Plugins cannot have circular dependencies, i.e. plugin 1 cannot depend on a service provided by plugin 2 if plugin 2 depends one one provided by plugin 1, even if this happens indirectly. If you do this you get obscure errors. The way I've been working around this is by assigning the resulting architect application to a global, and then calling getService("servicename") on that global.

For instance, I instantiate the application this way:

architect.resolveConfig(plugins, ".", function(err, config) {
    architect.createApp(config, function(err, app) {
        window.zed = app;
    });
});

Then, elsewhere in some plugin that doesn't have an explicit dependency on service A (listed under consumes) I call zed.getService("A") to get a reference to it anyway. Clearly, this is a bit ugly, but you need it sometimes.

Second: always call register() as soon as posisble. If you don't call register() at all, your application will never load -- every plugin has to call it. Also, if you wait with calling register too long, everything else is waiting for it, resulting in a poor user experience.

If you keep these two things in mind, you should be golden.

Do I Need Architect?

So, does every application need Architect? Probaby not. However, for applications of a certain size, and applications that need to be loaded in different configurations, it can be very useful. So evaluate its value for your application specifically.

Architect was developed largely by Tim Caswell, back when we both worked at Cloud9. Cloud9 these days uses it both for their node.js back-end and their client-side too. Another former Cloud9 colleague gave some talks about Architect that also give a good introduction.

About Zed and this Blog Post

Last week I announced I would make working on Zed my day-time job. Zed was open source before, and I kept it that way. In addition to the openness in code I also want to be open in design decisions and other lessons learned during this endeavor. This post is one example of that: it shows how Zed gets value out of Architect, but it's potentially useful for any JavaScript developer. Do you want to support me in working in this open way? Consider contributing on Gittip.

Unrelated post script: This post was written and edited on a Chromebook running Zed in Vim mode, using the Notes project that's automatically synced with my other computers. And it was a great experience.

The Changelog: v0.11 -- First Standalone Release

Posted: 2014-04-10

When I announced that I would start working on Zed as my day job I committed to one "feature" immediately: a standalone version -- a version you can use without having Chrome installed. Well, barely 2 weeks later, here it is! There may still be bugs here and there, but there's an auto-update feature, so hopefully we can quickly resolve bugs without too much work with re-downloading for you.

Bonus feature: Are you a Vim fan? Zed now has vim keybindings (see changelog below).

First, a little update with the funding situation. The good news is that we broke the $100/week barrier on Gittip. That's fantastic, but not nearly enough to make this project sustainable (we need roughly 10x that). So if you're a happy Zed user, consider supporting the project financially.

Let me get right to the download links. If you're using the Chrome version (as now more than 15k people are!), you should be getting 0.11 within a few hours. If you want to try out the standalone version, here are the download links. Note: these may still be propagating through our CDN -- thanks MaxCDN! If the links don't work for you yet, try again in a few hours!

• Mac version of Zed standalone 0.11
• Windows version of Zed standalone 0.11
• Linux 64-bit version of Zed standalone 0.11
• Linux 32-bit version of Zed standalone 0.11

Here's the changelog for 0.11.0:

• Standalone:
  • First release of Zed standalone! I'm sure there will still be bugs, please report them on github when you find them.
  • Open individual files or directories from the command line by calling "zed [dir]" or "zed [file]". On Mac, please install the CLI tools via the Tools:Install Mac CLI or the native Tools menu in the Mac app to use this.
• Multiple keybindings support. That's right, we now have Vim and Emacs keybindings. Use the Configuration:Preferences:KeyBinding:* commands to switch between them. (by nightwing)
• New commands:
  • File:Copy (by iElectric)
• Language modes:
  • Python now has CTags indexing (by gholstla)
  • Ability to switch off certain CSSLint options (by conradz)
• Bugs fixed:
  • Fixed a bug where when the project history is empty, no new projects are added.

Thanks to all who contributed!

The Changelog: v0.10.2

Posted: 2014-04-08

Woah, the past week has been a huge week. On Tuesday I announced that I would make working on Zed my day job which resulted in a lot of attention. The number of active users exploded, as did participation on Github and the Google Group. A couple of new people contributed code to the project too, which is fantastic. I couldn't be happier!

I'm also happy to see that many of you already made the effort to contribute to the sustainability of the project by donating weekly on Gittip. We haven't nearly reached financial sustainability of the project, but it's encouraging!

I just pushed Zed version 0.10.2 to the Chrome Store, which is a minor release in terms of features, but a big one in the amount of work that was done to work towards the promised standalone version. I suspect that this will be the last release before a version that will come in both a Chrome App and Standalone edition -- hopefully by the end of this week or early next week. Stay tuned!

Here's the changelog for 0.10.2:

• Refactoring of Zed core codebase to use Architect plug-ins, making it easier to create the Zed standalone (without Chrome dependency) version.
• New commands:
  • Help:Commands (bound to F1 by default) gives you a context-sensitive list of all available Zed commands in your current mode, documentation for each command (if available) and current keybinding (by robru)
  • Tools:Document Statistics gives some useful stats about your current document (word count, line count, character count etc.) (by robru)
• Sandbox updates:
  • Added sandbox commands to call goto and invoke arbitrary other commands (by robru)
  • Added a "click" handler for the sandbox.
• Language modes:
  • Separate SCSS mode (by wpapper)
  • Scala mode (by netzwerg)
  • Mode is now set for files without a file extension based on Unix shebang line, e.g. #!/bin/bash uses bash mode (by robru)
• Made configuration loading more robust, even if user.json contains JSON parse errors.
• Minor fixes of the whitespace trimmer (by robru)
• Indent on paste fixes (by TheKiteEatingTree)
• Some minor updates to the manual (particularly the config.md documentation)

Thanks to all who contributed!

Zed: The Next Phase

Posted: 2014-04-01

When I started the Zed project about a year ago I intended to build an editor just for me: something minimal, simple and stable. Writing something for myself seemed a natural thing to do after spending about a year and a half having a job building an IDE: once you've built your own tools, there's no going back. Also, having built this sort of thing before (although not from "scratch"), it was fairly quick to get something working.

Over the past year, Zed has been the ultimate piece of dogfood-ware for me -- my own personal playground that allowed me to iterate on lingering ideas I had on improving code editing. Since people asked why I created yet another editor, over time I distilled some principles behind the design of Zed.

The past half a year I've been using Zed full-time and I've been very happy with it. And I'm not the only one. The Google Chrome Web Store (Zed currently runs as a Chrome app) tells me there are well over 4000 "weekly users", and the reviews are generally very positive. Now I take the 4k number with a grain of salt, but clearly, this is no longer a 1-person audience editor. The number of community contributions has also increased lately and there appears to be a growing number of users that don't just like Zed as a cool "look what web tech can do" demo, but really love Zed as an editor, and see it as a modern-day Emacs or Vim: minimalist, powerful and extensible.

However, with a full-time day job at work, and full-time night job as a parent, there isn't always much time to spend on Zed. At least not as much as I'd like. And I have so many cool ideas I still want to implement.

It's time to fix that.

I'm making Zed my day job

So, I'm going to try something radical. As of today, I'm making Zed my day-time job.

I've launched a company to further develop Zed, with the goal to, over time, earn enough money on it to support me and my family, and to continue developing Zed into something great.

While an obvious option to make Zed profitable would be to make it proprietary and sell licenses, I won't be doing that. I think there's a lot of value for users in the fact that their editor is open source, in terms of hackability and longevity. Also, since I'm a company of one -- for the foreseeable future -- I think the best way to compete is if when the community helps out with enhancement and bug fixes, as it does already. So, Zed will remain open source.

Sustainable Open Source

I'm going to try making a living working on open source. I have not fully decided how I'll attempt this, so all ideas are welcome. However, my current plan is the simplest and most transparent thing possible: ask users to pay. If you use Zed and get value out of it, support me financially to support further development. Just as you would for regular commercial software, pay for Zed as well.

While possibly naive, this seems like the cleanest way to sustainability of this type of project. I don't want to do open core, or do weird in-editor ads (which people would be able to easily remove anyway). Consulting around Zed also doesn't seem to make much sense. Again, tips or suggestions, are welcome, so get in touch.

Why would you pay for an editor?

So why would you pay for an editor when you can get Vim, Emacs, Notepad++ and Gedit for free?

As I see it, the cost of software has very little to do with purchase price. If a piece of software just makes you 1% more productive compared to not using the software, and you earn, say, $40/hour and use it 40 hours a week (developers tend to use their editors close to full-time), very quickly this software is worth $70/month. Or rather, not using that piece of software costs you $70/month. Zed will be significantly cheaper than that, price wise.

Are you as productive using your "free" software as you are/will be with Zed? My challenge is to make the answer to that question a resounding "no."

As a bonus, in contrast to proprietary editors like Sublime and most Jetbrains IDEs, Zed is 100% open source, so you can read the source code, enhance it, fix it, break it and learn from it. Even if I or my company will go away, Zed won't, if somebody cares enough about it to develop it further. So it's a more lasting investment than paying for a commercial alternative.

Of course, this being open source, there's no way to enforce anybody to pay a dime. So what if people don't? That will mean that when my savings run out I will have to go back to a regular day time job. As a result, Zed development will slow down again. This tool that your rely on won't improve at the same pace anymore. Me unhappy, you unhappy. If instead lots of people contribute, possibly I can hire a second person, third, fourth. Who knows. Result: Zed will get even awesomer.

It's all in your hands.

Can't some random person fork the project and attempt same thing? Yep. Worse, they can fork the project, and create a proprietary version and start selling it. I'm just assuming people are decent and respect my and the community's work. That's what this whole open source thing is all about, or so I understand.

So, what cool features will we get?

I have loads of things I want to do, but the only one I want to commit to right now is a stand-alone version of Zed. Zed currently is only available as a Chrome packaged app. So you need to use Chrome to run it. This is fine, but not everybody likes it. It also limits OS integration somewhat.

The stand-alone version will be built on top of node-webkit and be available for Mac, Windows and Linux. Once the initial work there is done, both version should continue working from the same code base, so I should be able to support two versions (Chrome App and standalone) without much effort.

There are many more things I will finally have time for, but I'll talk about those over time. Those familiar with my background can probably guess what direction I could take.

Sign me up!

I'm still working out ways for users to support this project, but one that's available for those who want to support the project right away is my Gittip account. I'm going on a stretch here and am willing to accept Gittips from people who do not currently use Zed, but are excited about the dream and want to find out where this is going. I know, I'm a very generous that way. Also, if you or your company are interested in supporting this effort and don't want to use Gittip, get in touch. And if you haven't already, give Zed a try and report any bugs you may find.

I'm very excited, and frankly, a bit scared to take this step. But I've also been looking forward to it for a long time.

Let's make Zed even more awesome!

Watch this blog, my twitter account, or Zed's twitter account and subscribe to the Google Group for regular updates!

Discussion on Hacker News and Reddit.

Thoughts on Atom vs Zed

Posted: 2014-02-27

So, Atom happened, github's web tech-based code editor. I have an Atom invite and I played around with it a bit today and it's an interesting project -- and with the weight of Github behind it, definitely not something to ignore.

Of course, as the author of Zed I have to ask: how does Atom compare to Zed, and is there any reason for Zed to exist anymore? In this post I'll try to compare and contrast the two a little bit based on what I've learned so far. Of course, all of this is somewhat biased, but I try to abstain from criticism at the level of "WHY COFFEESCRIPT!? COFFEESCRIPT SUCKS!!1"

Web technology

Both Zed and Atom observed that web technology is a great way to build a hackable editor, and that a web tech stack doesn't need to live in a web page. In Zed's case it's a Chrome app (and available for Mac/Windows/Linux/ChromeOS) -- although spoiler alert I have a branch with a node-webkit prototype for Zed too --- in Atom's case it's a native app based on a forked version of node-webkit, currently only on Mac, but I'm sure with at least a Windows version is not far off. Zed is built using HTML, pure CSS and JavaScript. Atom is built using HTML, Less and CoffeeScript.

Atom is, due to its strong ties to node.js APIs, less portable than Zed. Zed currently runs as a Chrome app, and in prototype-form as a node-webkit "native" app. But it's easy to also make it work as a regular web app, if need be, or as a Firefox App if that ever becomes a thing. That said, of course, getting access to node.js directly allows the editor to do anything node.js can do (= basically anything) and Chrome or web apps can't.

UI/UX

Zed has the explicit goal of doing things differently regarding the concept of open files and tabs (or lack thereof), minimalism in UI, unifying file opening and creation, and remote file editing. Atom basically looks like sublime: file tree on the left, tab bar along the top, and borrows features like goto and the command panel from Sublime and Textmate too (Zed does too, but has tweaked the way goto works). Atom is more "traditional" UX-wise, which is a perfectly legitimate choice.

Feature set

I'm not going to do a feature-by-feature comparison. Let's say that Zed and Atom are relatively close in terms of feature set. Atom is probably stronger in terms of find and replace support (regex support, find and replace in project). Zed is stronger in language support (extensible code completion, inline markers for linters, symbol indexing, code formatters etc.) and remote file editing. The reason for Zed's language support focus is obvious, considering where I came from and you can expect more features in this area in the future.

Since both are extensible editors, my response to any missing feature in Zed can be "well, just fork the project and add it, or create an extension" and Github will say "well, just write an extension."

Extensibility

From the beginning the ability to extend Zed (beyond forking the code) has been high priority for Zed, however being a Chrome app limits its flexibility a bit. If you want to use Chrome APIs like getting access to the local file system, you need to run in a "safe" mode that does not allow a lot of things like eval'ing arbitrary (user) code. Therefore, in Zed, all extension code lives in a sandbox in a separate process. Zed exposes APIs to the sandbox to do certain things like using basic UI elements, getting access to files etc. Zed extension points are designed in advance and not accidental. Zed provides specific extension points to extend its built-in code completion, symbol indexing, linting etc. support. There are (currently) no APIs to make arbitrary changes to the DOM, listen to arbitrary events etc. Zed may eventually get these features, but they have to added explicitly.

However, putting extension code in a sandbox has two advantages:

1. Extensions can interfere with the main process in limited ways, it's difficult to make the main editor crash as a result of poor extension code, and because the sandbox lives in a separate process, performing CPU intensive tasks also does not slow down the editor.

2. It is easy to reload extension code without having to reload the whole editor (by simply killing the sandbox and restarting it). As a result, extensions can be developed without having to relaunch Zed, but just by resetting the sandbox.

Atom extensions as far as I can see, get full access to about everything: the DOM and all CoffeeScript/JS APIs. This provides a lot of flexibility, and as a result a more significant part of Atom is built as extensions compared to Zed. For instance, auto completion is not a built-in Atom feature, but built as an extension. The question is how extensions will interact, for instance if I want to add a JavaScript-specific auto completer. I suppose the autocomplete Atom extension would need to provide an API somehow (couldn't find how, or if this is supported). In Zed, code completion is a core feature and provides handlers to plug in to, to do this -- which how the word, ctag symbol and snippet completers are implemented.

However, since all code in Atom is loaded into the same JavaScript VM, reloading involves reopening windows to see the results during extension development -- at least, as far as I can see. And poor extension code, can completely screw up the editor.

Atom has apm, a package manager (from the looks of it, based on npm) to easily install, upgrade, remove and publish extensions. Very cool stuff. Zed has no such feature yet, although it's coming.

Licensing and cost

How Atom will be distributed once it leaves beta is not 100% clear. Most likely it will be free, but not open source with some sort payment model attached. Github is trying to find a balance between the ability to earn money on the product, while still open sourcing as much as possible. Zed is 100% open source, MIT licensed.

Find and Replace: the Zed Way

Posted: 2014-02-26

Most editors got their find & replace features before the advent of multiple cursors. In Zed, multiple cursors are core to your editing experience: they're used to quickly apply edits on multiple locations simultaneously, for instance, to emulate a traditional editor's find & replace functionality.

But before we get to the replace aspect, let's have a look at how to use use Zed's find features first.

Find in file

The Find:Find command (Command-F/Ctrl-F), brings up the goto UI prefixed with :/ which signifies (incremental) search in file. Enter you phrase and press enter to jump to the first match, then to jump to the next match (with the match still selected) use the Find:Next command (Command-G/Ctrl-K), or Find:Previous (Command-Shift-G/Ctrl-Shift-K) to jump to the previous match. To jump to other instances of the identifier under the cursor use Command-[/Ctrl-[ (previous instance) and Command-]/Ctrl-] (next instance).

Find and replace

To replace text, we use multiple cursors:

1. Select the phrase you want to replace (e.g. via search, followed by Enter to put focus back on the editor)
2. Add cursors to however many instances of that phrase as you want via Ctrl-Alt-Right (add the instance to the "right" -- after the current selection) and/or Ctrl-Alt-Left (add to the left). Or, to put cursors on all instances at once, use Find:All (Ctrl-Alt-F).
3. Start making your changes! The change will be applied at all locations.

Find in project

Zed also has basic find in project functionality (no replace in project yet) via Find:Find in Project (Command-Shift-F/Ctrl-Shift-F).