Irreal: Tabula

Back in early July, I wrote a rant disagreeing with Robert Zaremba about retiring the use of PDFs. Zaremba believes that PDFs are no longer a good fit for today’s devices and that we should stop using them. I strongly disagreed. In the comments to my post, Mike Zamansky zinged me by noting that “PDFs are where data goes to die.” His point, of course, is that it’s pretty hard to get data out of a PDF.

Now, finally, I have a rejoinder: Tabula, a tool for extracting PDF tables into CSV data. Once you’ve got it in that format, it’s easy to convert it into others such as an Org mode table. That should be especially handy for researchers who like to write their papers in Org mode.

The problem of extracting table data from a PDF turns out to be surprisingly difficult. Follow the Tabula link to read about some of the problems. Regardless, Tabula can (usually) do it and help researchers capture data from a PDF in a relatively painless and accurate way.

-1:-- Tabula (Post jcs)--L0--C0--July 24, 2017 11:53 AM

Emacs café: Indium 0.7 is out!

Indium – the JavaScript development environment for Emacs – version 0.7 was released a few days ago!

This release brings a major improvement: support for source maps (see the documentation).

Source maps make it possible to us a source JavaScript file (not built/minified) while a minified file is served. Indium supports source maps both for debugging and setting breakpoints.

Installation and update instructions can be found in the documentation. You can also check out the project on GitHub.

-1:-- Indium 0.7 is out! (Post Nicolas Petton)--L0--C0--July 24, 2017 11:40 AM

sachachua: 2017-07-24 Emacs news

Links from, /r/orgmode, /r/spacemacs, Hacker News,, YouTube, the changes to the Emacs NEWS file, and emacs-devel.

Past Emacs News round-ups

-1:-- 2017-07-24 Emacs news (Post Sacha Chua)--L0--C0--July 24, 2017 05:59 AM

Marcin Borkowski: TeX-parse-macro

In one of my earlier posts I promised to show how to prevent AUCTeX from putting braces after a command inserted with TeX-insert-macro.
-1:-- TeX-parse-macro (Post)--L0--C0--July 22, 2017 04:33 AM

Irreal: John Wiegley and Sacha Chua on use-package

A couple years ago, John Wiegley and Sacha Chua made a video about Wiegley’s use-package package. I’ve mentioned it a couple of times in passing but never written about it. Recently, I stumbled across it again and rewatched it.

There’s a lot of interesting material in the video and I decided that even though it’s old it was worth writing about. If you aren’t already using use-package, you should definitely take a look. The package really does make your configuration simpler and more logical. It can even be used to speed up Emacs’ start time: Wiegley reports that he has over 200 package and his Emacs starts in about a third of a second.

Even if, like me, you’re already using use-package, there’s still a good reason to spend a half hour watching the video. Wiegley discusses some things I didn’t know about. First is the macrostep package. If you only use it to see exactly what use-package is doing, it’s worth installing. The use-package macro confuses many people. If you watch the video and then expand an invocation of the macro in your configuration, you’ll easily understand what’s going on. It’s really easy to use and if you bind a key sequence to invoke it1, it won’t be loaded until you need it.

Another thing I learned is that you can fold the text in a buffer so that any text that starts more than n columns from the left margin isn’t shown. Oddly, this is a built in functionality (bound to Ctrl+x $) so you don’t need to get anything to use it.

Once you start using use-package, you’ll automatically pull in bind-key so you’ll have describe-personal-keybindings available. When you invoke it, you’ll get a nice list of all the keybindings you’ve defined along with what, if anything, they replaced. That can be really handy for organizing your configuration.

Finally, one of the nice things about use-package is that you can configure it to record how long it takes to load and configure the packages it loads. That can really be handy for tracking down the hot spots in your configuration load time. It all goes into the *Message* buffer so you can ignore it except when you need it.

I really like use-package and have converted my entire init.el to use it. If you aren’t already using it, you should start and the video will go a long way towards convincing you of that.

UPDATE [2017-07-21 Fri 11:58]: Added link to video.



Or you can ask use-package to delay loading it explicitly.

-1:-- John Wiegley and Sacha Chua on use-package (Post jcs)--L0--C0--July 21, 2017 03:11 PM

emacshorrors: make-temp-name

For someone not terribly experienced in writing safe programs, one can only hope that building blocks like make-temp-file are doing the right thing and cannot be subverted by a malicious third party. The general advice here is that it’s preferable to use the primitive for creating the temporary file instead of the primitive to generate its name. Now, does Emacs reuse mkstemp(3) for this? Or at least tmpnam(3)? Of course not! Where we go, we can just invent our own source of randomness:

make-temp-file looks as follows:

static const char make_temp_name_tbl[64] =

static unsigned make_temp_name_count, make_temp_name_count_initialized_p;

/* Value is a temporary file name starting with PREFIX, a string.

   The Emacs process number forms part of the result, so there is
   no danger of generating a name being used by another process.
   In addition, this function makes an attempt to choose a name
   which has no existing file.  To make this work, PREFIX should be
   an absolute file name.

   BASE64_P means add the pid as 3 characters in base64
   encoding.  In this case, 6 characters will be added to PREFIX to
   form the file name.  Otherwise, if Emacs is running on a system
   with long file names, add the pid as a decimal number.

   This function signals an error if no unique file name could be
   generated.  */

make_temp_name (Lisp_Object prefix, bool base64_p)
  Lisp_Object val, encoded_prefix;
  ptrdiff_t len;
  printmax_t pid;
  char *p, *data;
  char pidbuf[INT_BUFSIZE_BOUND (printmax_t)];
  int pidlen;

  CHECK_STRING (prefix);

  /* VAL is created by adding 6 characters to PREFIX.  The first
     three are the PID of this process, in base 64, and the second
     three are incremented if the file already exists.  This ensures
     262144 unique file names per PID per PREFIX.  */

  pid = getpid ();

  if (base64_p)
      pidbuf[0] = make_temp_name_tbl[pid & 63], pid >>= 6;
      pidbuf[1] = make_temp_name_tbl[pid & 63], pid >>= 6;
      pidbuf[2] = make_temp_name_tbl[pid & 63], pid >>= 6;
      pidlen = 3;
      pidlen = sprintf (pidbuf, "%"pMd, pid);
      pidbuf[0] = make_temp_name_tbl[pid & 63], pid >>= 6;
      pidbuf[1] = make_temp_name_tbl[pid & 63], pid >>= 6;
      pidbuf[2] = make_temp_name_tbl[pid & 63], pid >>= 6;
      pidlen = 3;

  encoded_prefix = ENCODE_FILE (prefix);
  len = SBYTES (encoded_prefix);
  val = make_uninit_string (len + 3 + pidlen);
  data = SSDATA (val);
  memcpy (data, SSDATA (encoded_prefix), len);
  p = data + len;

  memcpy (p, pidbuf, pidlen);
  p += pidlen;

  /* Here we try to minimize useless stat'ing when this function is
     invoked many times successively with the same PREFIX.  We achieve
     this by initializing count to a random value, and incrementing it

     We don't want make-temp-name to be called while dumping,
     because then make_temp_name_count_initialized_p would get set
     and then make_temp_name_count would not be set when Emacs starts.  */

  if (!make_temp_name_count_initialized_p)
      make_temp_name_count = time (NULL);
      make_temp_name_count_initialized_p = 1;

  while (1)
      unsigned num = make_temp_name_count;

      p[0] = make_temp_name_tbl[num & 63], num >>= 6;
      p[1] = make_temp_name_tbl[num & 63], num >>= 6;
      p[2] = make_temp_name_tbl[num & 63], num >>= 6;

      /* Poor man's congruential RN generator.  Replace with
         ++make_temp_name_count for debugging.  */
      make_temp_name_count += 25229;
      make_temp_name_count %= 225307;

      if (!check_existing (data))
          /* We want to return only if errno is ENOENT.  */
          if (errno == ENOENT)
            return DECODE_FILE (val);
            /* The error here is dubious, but there is little else we
               can do.  The alternatives are to return nil, which is
               as bad as (and in many cases worse than) throwing the
               error, or to ignore the error, which will likely result
               in looping through 225307 stat's, which is not only
               dog-slow, but also useless since eventually nil would
               have to be returned anyway.  */
            report_file_error ("Cannot create temporary name for prefix",
          /* not reached */

DEFUN ("make-temp-name", Fmake_temp_name, Smake_temp_name, 1, 1, 0,
       doc: /* Generate temporary file name (string) starting with PREFIX (a string).
The Emacs process number forms part of the result, so there is no
danger of generating a name being used by another Emacs process
\(so long as only a single host can access the containing directory...).

This function tries to choose a name that has no existing file.
For this to work, PREFIX should be an absolute file name.

There is a race condition between calling `make-temp-name' and creating the
file, which opens all kinds of security holes.  For that reason, you should
normally use `make-temp-file' instead.  */)
  (Lisp_Object prefix)
  return make_temp_name (prefix, 0);

The generated file name is therefore a combination of the prefix, the Emacs PID and three characters from the above table. This makes about 200.000 possible temporary files that can be generated with a given prefix in an Emacs session. This range can be traversed in a negligible amount of time to recreate the state of the RNG and accurately predict the next temporary file name.

(defun make-temp-file (prefix &optional dir-flag suffix)
  "Create a temporary file.
The returned file name (created by appending some random characters at the end
of PREFIX, and expanding against `temporary-file-directory' if necessary),
is guaranteed to point to a newly created empty file.
You can then use `write-region' to write new data into the file.

If DIR-FLAG is non-nil, create a new empty directory instead of a file.

If SUFFIX is non-nil, add that at the end of the file name."
  ;; Create temp files with strict access rights.  It's easy to
  ;; loosen them later, whereas it's impossible to close the
  ;; time-window of loose permissions otherwise.
  (with-file-modes ?\700
    (let (file)
      (while (condition-case ()
                   (setq file
                          (if (zerop (length prefix))
                            (expand-file-name prefix
                   (if suffix
                       (setq file (concat file suffix)))
                   (if dir-flag
                       (make-directory file)
                     (write-region "" nil file nil 'silent nil 'excl))
               (file-already-exists t))
        ;; the file was somehow created by someone else between
        ;; `make-temp-name' and `write-region', let's try again.

It’s interesting that the docstring of this function states that the return value “is guaranteed to point to a newly created empty file.”. If there were to exist a file for every possible combination for a prefix, this function would just fall into an infinite loop and block Emacs for no apparent reason. Both of these issues have been solved in a better way in glibc.

At least the impact of predicting the name is lessened if one uses make-temp-file instead of make-temp-name on its own. An attacker cannot create a symlink pointing to a rogue location with the predicted name as that would trigger a file-already-exists error and make the function use the next random name. All they could do is read out the file afterwards iff they have the same permission as the user Emacs runs with. A symlink attack can only be executed successfully with a careless make-temp-name user, thankfully I’ve not been able to find one worth subverting on GitHub yet.

Thanks to dale on #emacs for bringing this to my attention!

-1:-- make-temp-name (Post Vasilij Schneidermann)--L0--C0--July 18, 2017 07:14 AM

sachachua: 2017-07-17 Emacs news

Links from, /r/orgmode, /r/spacemacs, Hacker News,, YouTube, the changes to the Emacs NEWS file, and emacs-devel.

Past Emacs News round-ups

-1:-- 2017-07-17 Emacs news (Post Sacha Chua)--L0--C0--July 17, 2017 07:27 AM

Alex Schroeder: rcirc omit mode

I’m in some busy fake IRC channels. In real IRC channels, people don’t join and part like crazy. But when you join channels that are actually other networks like Discord, then there is a lot more joining and parting. And you don’t want to enable rcirc-omit-mode for each and every one of them by hand. Luckily, I found this useful snippet by Giorgos Keramidas:

(add-hook 'rcirc-mode-hook
  (lambda ()
    ;; rcirc-omit-mode always *toggles*, so we first 'disable' it
    ;; and then let the function toggle it *and* set things up.
    (setq rcirc-omit-mode nil)

Why doesn’t rcirc-omit-mode take a parameter like all decent minor modes?


-1:-- rcirc omit mode (Post)--L0--C0--July 15, 2017 10:20 PM

Marcin Borkowski: org-duration-format and other clock-related settings

As I mentioned many times, I use the clocking feature of Org-mode a lot. At first, I used org-clock-display (C-c C-x C-d) to see how much time I spent on various things. This command temporarily shows times for various subtrees in your current buffers. By default, it shows the time for the current year, but I set the org-clock-display-default-range variable to today. (Alternatively, I could train myself to use the C-u prefix – this I only learned recently.) See the docstring of org-clock-display to learn even more.
-1:-- org-duration-format and other clock-related settings (Post)--L0--C0--July 15, 2017 05:33 AM

Wilfred Hughes: These Weeks in Remacs II

It’s been six months since the last Remacs update, and many new features have landed!


We now have a Gitter chat room! Do drop by if you have any questions or wish to discuss Remacs. There’s a low traffic Remacs Subreddit too.

We’ve added @jeandudey and @birkenfeld to the GitHub collaborators, bringing us to five fine people who can approve your pull requests.

Elisp Features

We’re still tracking upstream GNU Emacs master, so new features there are landing in Remacs (1, 2).

We’ve added a lot new elisp primitive functions:

Strings: characterp, multibyte conversions (1, 2, 3), and comparisons

Vectors: type definitions, functions

Buffers: type definitions, functions

Symbols: various functions

A much requested feature, adding Rust support to find-function, has been added. This was an unusual PR as it includes some elisp changes in Remacs.

We now have documentation on our compatibility with GNU Emacs. This covers all known implementation differences, platform support differences, and describes how to detect Remacs in elisp code.


Platforms: We’ve dropped MS-DOS support. The Remacs build has been fixed on 32-bit Linux and 32-bit macOS.

The codebase has been split out:

  • remacs-lib (Rust equivalents of gnulib)
  • remacs-sys (type definitions of Emacs types and C functions)
  • remacs-macros (procedural macros supporting elisp primitive functions in Rust)
  • src (Rust implementation code of elisp)

Signal name mapping is pure Rust code.

We now run rustfmt on every PR.

If you fancy building Remacs without installing a dev toolchain (compilers, C libraries etc), there’s now a docker-compose.yml to make your life easy.


It wouldn’t be a proper lisp project without some macro magic.

After several PRs and discussions, Remacs now includes a procedural macro to simplify defining elisp functions in Rust.

For example, here’s vectorp:

/// Return t if OBJECT is a vector.
fn vectorp(object: LispObject) -> LispObject {

Leveraging Rust

Remacs now uses Rust crates for SHA-1 and SHA-2, and for base64 encoding. We’ve even replaced some C bit-counting functions with functions from the Rust stdlib.

Rust has also enabled us to mark Emacs functions with the ! type, a neat Rust feature that marks functions as not returning.

The #[repr(transparent)] Rust RFC has been approved, so we’re looking forward to using that in Remacs. In the meantime, Remacs has a LispObject type for use in Rust, and a CLisp_Object type for FFI compatibility.

Remacs also takes advantage of the user-defined allocators RFC, which has been approved too. We’re now up-to-date with the new API.


There’s still lots to do on Remacs: many small elisp functions to port, or larger projects to sink your teeth into. We also welcome incomplete pull requests: many of the PRs shown here have been built on top of initial implementations written by other contributors.

Join the fun at:

-1:-- These Weeks in Remacs II (Post Wilfred Hughes ( 15, 2017 12:00 AM

Alex Schroeder: Bitlbee, Twitter, and Bold

I use Bitlbee. It acts as an IRC server and is the gateway to other services that can be understood in terms to nicks and channels. I use it for Twitter, for example. When I connect, Bitlbee fetches a bunch of older Tweets to fill my timeline. Sadly, highlighting is somehow borked as far as Twitter is concerned. What sort of highlighting you ask? Well, IRC has a convention of allowing some control codes to make stuff bold, italic or coloured. Not everybody uses it, but I installed a plugin for my IRC client, rcirc. This plugin seems to be interfering with the display of the Twitter backlog, sadly.

This screenshot shows the transition between backlog (the bold stuff above) and the “live” messages which are highlighted correctly:

Image 1

What’s wrong? What can I do to fix this?

  1. M-: (setq rcirc-debug-flag t)
  2. start bitlbee and rcirc
  3. switch to the buffer *rcirc debug*
  4. take a look at the borked messages

For example (translating the real Control-B to ˆB for you to read):

:iamdevloper!iamdevloper@twitter PRIVMSG #twitter_kensanata :^B[^B^B^B12:26:28^B]^B ^B[^B12^B]^B "Now I am become death, the destroyer of *worlds" - the project manager who promised the client early project delivery

It seems to legit, even though those three consecutive ^B characters are a pain. Well, at least now I know!

I looked at rcirc-styles and found that the Emacs Wiki page points to a different repository. Somebody has been maintaining the code. I deleted my code and installed the package, and all is well, now. Yay!

Thanks, Aaron Miller. :)


-1:-- Bitlbee, Twitter, and Bold (Post)--L0--C0--July 14, 2017 10:06 AM

Modern Emacs: Migrating to Spacemacs Layers

Spacemacs is referred for its evil integration, space-based bindings, and community contributed layers that collect, configure, and integrate groups of packages.

For how much they add to Emacs, motivations for personal layers are largely undocumented.

I introduce layers then discuss benefits, approaches, and gotchas with layer-based configurations.

I've migrated my entire dotspacemacs/user-config into personal layers - now 6 lines vs 1,500.

See for my viewer-friendly configuration .

Introducing Layers

This section is not a replacement for

Layers are directories containing up to 5 files and possibly additional packages.

In load order:


Layer dependencies to load first.

(configuration-layer/declare-layers '(theming))


Packages added or configured by the layer.

(setq my-layer-packages
        (github-pkg :location (recipe :fetcher github
                                      :repo "github-user/repo-name"))
        (my-pkg :location local)))
  • Owned Packages: A layer owns a package if it defines layer-name/init-pkg-name. All packages not defined in dotspacemacs/additional/packages should have one and only one owner. It calls use-package. Common options are :init for before load config, :config for after, :if for loading if eg. a certain OS or executable is installed, :after for enforcing load order, and :defer t for deferred loading.

(defun display/init-pretty-outlines ()
  (use-package pretty-outlines
    :after outshine
      (add-hook 'outline-mode-hook 'pretty-outline-set-display-table)
      (add-hook 'outline-minor-mode-hook 'pretty-outline-set-display-table)
      (add-hook 'emacs-lisp-mode-hook 'pretty-outline-add-bullets))))
  • Unowned Packages: A layer that does not own a package can configure it with layer-name/pre-init-pkg-name and layer-name/post-init-pkg-name.

(defun config/pre-init-neotree ()
  (evil-global-set-key 'normal (kbd "M-p")

(defun config/post-init-neotree ()
  (setq neo-theme 'icons))
  • Local Packages: Personal packages at local/my-pkg/my-pkg.el.


Layer functions.

Package agnostic functions belong here.

(defmacro with-face (STR &rest PROPS)
  "Return STR propertized with PROPS."
  `(propertize ,STR 'face (list ,@PROPS)))

Guarding against particular packages being installed:

(when (configuration-layer/package-usedp 'some-pkg)
  (defun my-func ()))


Layer variables.

;; python/config.el
(defvar python-tab-width 4
  "Tab width value for python buffers")

;; init.el in dotspacemacs-configuration-layers
(python :variables python-tab-width 2)

Configuration defined here will be loaded before the package init functions are executed. Layer dependencies are actually loaded prior to config.el.

This can be used for eg. setting theme updates with the theming layer.

(setq theming-modifications
      `((solarized-dark (avy-background-face :foreground "#586e75")
                        (font-lock-doc-face :foreground "#2aa198"))
        (solarized-light ...)))


Package-agnostic key-bindings.

(global-set-key (kbd "M-d") 'spacemacs/delete-window)

;; Evil will be loaded
(evil-define-key '(normal visual motion) outline-minor-mode-map
  "gh" 'outline-up-heading)

Personal Layers


While any organization can be used, I recommend at most these 5 layers covering common needs.

A Macros/Base Layer

A base layer that all personal layers inherit packages, macros, and common functions from with (configuration-layer/declare-layers '(base)).

I load dash-functional and define with-dir, with-face, and other useful utilities here.


All packages and their configuration and key-bindings that don't fit into any neat grouping.

When any package's init gets large, consider a local package. I maintain my org-mode setup separately in a local org-config package.

Anything, excluding spacemacs toggles, can be setup here. For instance:

(setq config-packages '(evil ...))

(defun config/post-init-evil ()
  (setq evil-escape-key-sequence "jk")
  (setq evil-escape-unordered-key-sequence "true")
  (advice-add 'evil-ex-search-next :after 'config/scroll-to-center-advice)
  (advice-add 'evil-ex-search-previous :after 'config/scroll-to-center-advice))

I recommend this layer own all additional packages except themes, see gotchas.


Theme updates and display packages like spaceline-all-the-icons.

Due to how Spacemacs loads themes, I highly recommend declaring the theming layer a dependency for theme updates. It is much more efficient should you configure multiple themes, like light and dark versions, and as it is a layer, it will be loaded prior to config.el for proper code isolation.

I integrate and configure my local pretty packages here:

Langs (optional)

I find it useful to separate programming language configuration out from the config layer, though it is not necessary.

Personal (optional)

All personal packages that aren't display related I maintain in a single personal layer. This is only relevant if you write your own packages.

I setup my blogging and outline-jump packages here.

Your init.el

Layers must be declared in your dotspacemacs-configuration-layers to take effect.

I've organized my layers into several sections:

(defvar dotspacemacs/layers/local
  '((macros :location local)
    (config :location local)
    (display :location local)
    (langs :location local)
    (personal :location local))
  "Local layers housed in '~/.spacemacs.d/layers'.")

(defvar dotspacemacs/layers/core
  "Layers I consider core to Spacemacs")

(defvar dotspacemacs/layers/langs
  "Programming and markup language layers")

(defvar dotspacemacs/layers/extra
  "Miscellaneous layers")

(defun dotspacemacs/layers ()
  (setq-default dotspacemacs-configuration-layer-path '("~/.spacemacs.d/layers/")
                (append dotspacemacs/layers/core


Migrating was mostly painless. However when things go wrong you lose access to your setup, an annoying development cycle. I encountered several Spacemacs idiosyncrasies to be aware of when using layers to replace my user-config.

Non-obvious errors to avoid:


The naming scheme of setq layer-name-packages and defun layer-name/init-pkg-name is strict. Beware when refactoring that you adjust the layer name accordingly. Failure to do so will result in the package's configuration not being loaded or in the case of ownership, not being installed, rather than a direct error.

Spacemacs toggles

Some toggles like spacemacs/toggle-highlight-long-lines-globally-on do not belong in any layer and should be defined in your user-config. Six toggles are now all that compose my dotspacemacs/user-config.

This goes for some toggles not explicitly owned by Spacemacs - trying to setup fringe-mode failed for me even in a config/post-init-fringe block.

OS Configuration

I define is-linuxp and a few other OS utilities that conditionally setup dotspacemacs/init variables like font size. Layers load after these variables are set, so the utilities cannot be moved to a layer. Set them at the top of your init.el.

Additional Themes

Spacemacs layers load ordering causes issues for extra themes. Theme packages cannot be put in a layer. As a result, to use solarized I set:

;; ~/.spacemacs.d/init.el
(defun dotspacemacs/layers ()
  (setq-default dotspacemacs-additional-packages '(solarized-theme)
(defun dotspacemacs/init ()
  (setq-default dotspacemacs-themes '(solarized-dark solarized-light)

Spacemacs Core Layers

Without doing a deep dive into Spacemacs core, you can expect the following layers to always be loaded before all personal layers. This is how dash is always available and evil-define-key can be used in keybindings files.

Call g d or (spacemacs/jump-to-definition) in emacs lisp mode to jump to that layer's packages.el to check out its packages and configuration.


These layers follow the same rules and principles as every other layer. If you have the curiosity, these layers make Spacemacs what it is.

Functionality provided here can be made use of by any layer, assuming those packages and layers are not explicitly excluded.


Those that value organization and robustness will find Spacemacs layers to improve on other configuration management methods.

Following Spacemacs conventions leads to predictable, friendly configurations.

Once you've become familiar with its conventions, there is no overhead.

-1:-- Migrating to Spacemacs Layers (Post)--L0--C0--July 14, 2017 12:00 AM

Pragmatic Emacs: A workflow to quickly add photos to org-mode notes

I was at a conference this week and a colleague was making notes using Evernote on her laptop and taking photos of key slides on her phone which then appeared in her notes. Of course I was making my notes in org-mode but I was envious of this behaviour so decided to emulate it.

With the function below, I can take a photo on my phone and upload to google drive (I use Photo & Picture Resizer, but you could use anything you like to get the pictures onto your computer). Then with a single command in Emacs, I am prompted with a list of photos in the folder to which they are uploaded, with the most recent first. The selected image is then:

  1. Moved the same directory as my org-mode notes file
  2. Renamed based on the heading of the current section in my notes, with a numeric suffix if there is already a photo with that name
  3. Linked in the notes and then the image is displayed

Here is a demonstration:


Here is the code:

;; add image from conference phone upload                                 ;;
;; use case is taking a photo of a slide in a conference and uploading
;; it to google drive or dropbox or whatever to get it on your
;; computer. You then want to embed it in an org-mode document by
;; moving it to the same folder and renaming according to the current
;; section of the org file, avoiding name clashes

;; required libraries
(require 'dash)
(require 'swiper)
(require 's)

;; start directory
(defvar bjm/conference-image-dir (expand-file-name "/path/to/image/upload/dir"))

(defun bjm/insert-conference-image ()
  "Insert image from conference directory, rename and add link in current file.

The file is taken from a start directory set by `bjm/conference-image-dir' and moved to the current directory, renamed and embedded at the point as an org-mode link. The user is presented with a list of files in the start directory, from which to select the file to move, sorted by most recent first."
  (let (file-list target-dir file-list-sorted start-file start-file-full file-ext end-file end-file-base end-file-full file-number)
    ;; clean directories from list but keep times
    (setq file-list
          (-remove (lambda (x) (nth 1 x))
                   (directory-files-and-attributes bjm/conference-image-dir)))

    ;; get target directory
    (setq target-dir (file-name-directory (buffer-file-name)))

    ;; sort list by most recent
  (setq file-list-sorted
        (mapcar #'car
                (sort file-list
                      #'(lambda (x y) (time-less-p (nth 6 y) (nth 6 x))))))

  ;; use ivy to select start-file
  (setq start-file (ivy-read
                    (concat "Move selected file to " target-dir ":")
                    :re-builder #'ivy--regex
                    :sort nil
                    :initial-input nil))

  ;; add full path to start file and end-file
  (setq start-file-full
        (expand-file-name start-file bjm/conference-image-dir))
  ;; generate target file name from current org section
  ;; (setq file-ext (file-name-extension start-file t))

  ;; my phone app doesn't add an extension to the image so I do it
  ;; here. If you want to keep the existing extension then use the
  ;; line above
  (setq file-ext ".jpg")
  ;; get section heading and clean it up
  (setq end-file-base (s-downcase (s-dashed-words (nth 4 (org-heading-components)))))
  ;; shorten to first 40 chars to avoid long file names
  (setq end-file-base (s-left 40 end-file-base))
  ;; number to append to ensure unique name
  (setq file-number 1)
  (setq end-file (concat
                  (format "-%s" file-number)

  ;; increment number at end of name if file exists
  (while (file-exists-p end-file)
    ;; increment
    (setq file-number (+ file-number 1))
    (setq end-file (concat
                    (format "-%s" file-number)

  ;; final file name including path
  (setq end-file-full
        (expand-file-name end-file target-dir))
  ;; rename file
  (rename-file start-file-full end-file-full)
  (message "moved %s to %s" start-file-full end-file)
  ;; insert link
  (insert (org-make-link-string (format "file:%s" end-file)))
  ;; display image
  (org-display-inline-images t t)))
-1:-- A workflow to quickly add photos to org-mode notes (Post Ben Maughan)--L0--C0--July 06, 2017 02:56 PM

Modern Emacs: Fancy Outline Bullets

We improve upon the outline bullets presented in the previous post Managing code with Outlines.

My Headings Editing my config
/img/outline-bullets-fancy.png /img/outline-bullets-config.png

This solution:

  1. Allows for face application to the bullet. The default faces outline-1/2/3...
  2. only apply to the text, not the bullet.
  3. Adds spaces for higher level bullets. So not every outline-level needs to
  4. have its text start at the same column.
  5. Works for any number of outline levels.
  6. Generalized - uses outline regexes, all that is required to add the bullets
  7. to a mode is adding to its hook.

(require 'dash)
(require 'outshine)
(require 's)

(defmacro with-face (STR &rest PROPS)
  "Return STR propertized with PROPS."
  `(propertize ,STR 'face (list ,@PROPS)))

(defun set-icon-fonts (CODE-FONT-ALIST)
  "Utility to associate many unicode points with specified fonts."
    (-let (((font . codes) it))
      (--each codes
        (set-fontset-font t `(,it . ,it) font)))))

;; Requires all-the-icons fonts installed
 '(("material icons" #xe3d0 #xe3d1 #xe3d2 #xe3d4)))

(setq outline-bullets-bullet-list '("" "" "" ""))

(defun font-lock-display-updates (FONT-LOCK-ALIST)
  "Put text property for FONT-LOCK-ALIST for var-width replacements."
   nil (--map (-let (((rgx uni-point) it))
                `(,rgx (0 (progn
                             (match-beginning 1) (match-end 1)

(defun outline-bullets-rgx-at-level (LEVEL)
  "Calculate regex or outline-bullets at LEVEL."
  (concat "\\(^"
          (-> LEVEL
          "\\) "))

(defun propertize-bullet (LEVEL BULLET)
  "Add LEVEL-dependent face to BULLET."
  (with-face BULLET
             (pcase LEVEL
               (0 '(:inherit outline-1 :underline nil))
               (1 '(:inherit outline-2 :underline nil))
               (2 '(:inherit outline-3 :underline nil))
               (3 '(:inherit outline-4 :underline nil))
               (_ nil))))

(defun add-outline-font-locks ()
  "Use with `add-hook' to enable outline-bullets-bullet-list for mode."
     (outline-bullets-rgx-at-level (+ 1 it-index))
      (s-repeat it-index " ")
      (propertize-bullet it-index it)))
    (-take 8 (-cycle outline-bullets-bullet-list)))))

(add-hook 'emacs-lisp-mode-hook 'add-outline-font-locks)
(add-hook 'hy-mode-hook 'add-outline-font-locks)
(add-hook 'python-mode-hook 'add-outline-font-locks)
-1:-- Fancy Outline Bullets (Post)--L0--C0--July 03, 2017 12:00 AM

Raimon Grau: Announcing commit-msg-prefix.

When writing commit messages, it's usual that your company/organisation has some policies and rules about the format and contents in the messages.

Some use an issue number, or name, or start with keywords (or emojis) like Add:/Clean:/Remove:.. (that can be converted to emojis in emacs too). But it's clear that somehow there's value in some kind of standarization.

So I created commit-msg-prefix, which lists the previous git commit messages and lets you pick one of them, and it will insert the relevant part of the commit in your current buffer.

My use case is when I do not remember the issue name/number I'm working on, but I remember keywords of previous commits that belong to the same issue.

There are a few variables to configure, like the exact git (or other) command to fetch logs, and the regex to apply to the log to extract the relevant part to insert.

The variable "commit-msg-prefix-input-method" is one of the symbols ('completing-read 'ido-completing-read 'commit-msg-prefix-helm-read 'ivy-read).

it defaults to ido-completing-read, but the idea is that you use your favourite input method.  ivy or helm, I guess :)

-1:-- Announcing commit-msg-prefix. (Post Raimon Grau ( 02, 2017 01:45 PM

Wilfred Hughes: Synthesising Elisp Code

I’ve just released v0.3 of suggest.el, an Emacs package for discovering elisp functions. You supply an example input and output, and it makes suggestions.

v0.3 is much smarter, and almost magical in places. Let’s take a look.

What is this, anyway?

Communicating what suggest.el does is difficult. Bodil’s tweet did a much better job of communicating usage, so I’ve overhauled the README. It’s now full of examples like this:

suggest.el can also help you find tricky dash.el functions:

;; Inputs (one per line):
(list 'a 'b 'c 'd)

;; Desired output:

;; Suggestions:
(-elem-index 'c (list 'a 'b 'c 'd)) ;=> 2

This helps users get an idea of what they can do with suggest.el.

Let’s not segfault Emacs

It turns out that brute-forcing elisp primitives can expose some nasty Emacs bugs. suggest.el will no longer make your Emacs crash, and upstream have fixed the underlying issue.

Isn’t there a function for that?

Since the initial release, suggest.el can now suggest 38 additional functions. suggest.el is now able to help you find values in a vector, convert symbols to strings, and much more besides. Serendipitous discoveries are now much more likely.

Any number of arguments

If your input is a list, you might be able to call a function with apply to get your desired result. Given an input '(2 3) and an output of 5, suggest.el can now propose (apply #'+ '(2 3))!

Slightly magical

suggest.el had a major limitation: what if you need to combine a few functions to get the result you want?

I added a breadth-first search of nested function calls. Using a few judicious search heuristics, suggest.el can search for up to three nested function calls.

This proved to be a really fun feature that sometimes produces surprising results:

;; Inputs (one per line):

;; Desired output:

;; Suggestions:
(1+ (1+ (1+ 0.0))) ;=> 3.0
(float (length (number-to-string 0.0))) ;=> 3.0

Converting to a string, then taking the length, does indeed produce 3.0!

> (number-to-string 0.0)
> (length "0.0")
> (float 3)

Sorting results is still an open problem: suggest.el prefers fewer function calls with a crude notion of ‘simple’ functions.

;; Inputs (one per line):
'(a b c)

;; Desired output:

;; Suggestions:
(cl-third '(a b c)) ;=> 'c
(-last-item '(a b c)) ;=> 'c
(cadr (cdr '(a b c))) ;=> 'c
(car (last '(a b c))) ;=> 'c
(apply #'last (last '(a b c))) ;=> 'c

cl-third vs -last-item vs (car (last ...)) is largely a matter of taste. The last result is entirely silly.

I’ve also written a series of tests to verify that my search returns the intended result (in addition to any others). This is really helpful when tweaking search heuristics.

I wrapped this up in a pretty macro:

(ert-deftest suggest-possibilities ()
  ;; A particularly deep search.
  (should-suggest 0 => 3
                  (1+ (1+ (1+ _))))
  ;; Ensure we offer built-in list functions.
  (should-suggest '(a b c d) => '(c d)
                  (cdr (cdr _))))

However, the implementation of the should-suggest macro is not pretty at all. Macros are great for finding better ways of expressing ideas, but I often find they produce insight that leads to refactoring them away entirely. Time will tell if this macro is worth the maintenance.


If you like suggest.el, you’ll be blown away by Barliman. This takes the idea of synthesising much much further, and can even generate quines! See this video for a great demonstration.

suggest.el v0.3 is available on MELPA. If you discover some interesting function combinations, I’d love to hear about them!

-1:-- Synthesising Elisp Code (Post Wilfred Hughes ( 02, 2017 12:00 AM

Emacs café: Orgmode for GTD


I’ve been using Orgmode to implement the GTD methodology for the last 4 years.

Rather than explaining the GTD methodology itself or how Orgmode works, in this post I’ll detail how I use Orgmode to implement GTD.

If you don’t know Orgmode and are curious about it, you should head to its website first.

The orgmode files

I split my GTD in four separate files:

  • where I collect everything;
  • where I put all my projects;
  • All inactive tasks that I might do at some point in the future, but don’t want to see all the time;
  • I put entries in this file with a timestamp to get reminded at the right moment.

It’s important to add these files to the agenda file (more on the agenda later), like the following:

(setq org-agenda-files '("~/gtd/"

1. The GTD inbox

One of the most important aspects of GTD is the inbox. Every thought should be collected there, and processed later on.


Orgmode has a great feature that fits really well with this concept: org-capture.

Capturing a thought is one key press away: simply Press C-c c, and a capture popup will appear in Emacs. Once you’re done capturing, C-c C-c and it will get stored in the inbox.


Here’s how I set it up:

(setq org-capture-templates '(("t" "Todo [inbox]" entry
                               (file+headline "~/gtd/" "Tasks")
                               "* TODO %i%?")
                              ("T" "Tickler" entry
                               (file+headline "~/gtd/" "Tickler")
                               "* %i%? \n %U")))

The syntax of capture templates is explained here. It offers lots of customization options.

I press C-c c t to add an entry to my inbox, and C-c c T to add an entry to the tickler (more on that later).

Here’s how my inbox looks like:


My inbox is then processed and emptied daily. When processing the inbox, I refile each entry that is actionable and belongs to a project using C-c C-w, moving the entry to the appropriate place. If need be, I create a new project out of it.

I have set up the refile targets as follows:

(setq org-refile-targets '(("~/gtd/" :maxlevel . 3)
                           ("~/gtd/" :level . 1)
                           ("~/gtd/" :maxlevel . 2)))

So that C-c C-w prompts me either for a project, the tickler, or someday/maybe list.

2. The projects file

My main file is That’s where I keep all my active projects. I usually have around 30 active projects at the same time.

Each project contains actions to be performed. The first action of each project is called its “next action”, and that’s always the one I will do when working on a project. Once a task is done, I mark it as such using the DONE todo keyword.

Here’s an example project:

project example

The completion percentage you see on the screenshot is yet another neat feature of Orgmode :)


Tagging is done using C-c C-c on a headline, whether it’s a project or action. I use tags for several purposes:

  • Regular categories, like :emacs: or :writing:;
  • Tags that link to people, like :daniel:;
  • GTD contexts.

GTD contexts are just regular tags, starting with @. I make heavy use of them in custom Org Agenda commands.

My contexts tend to change over time, but I always have at least @home, @office, @travelling, @phone, @email, @errands to filter out next actions based on my current location for instance.

TODO keywords

I put a todo keyword in all project entries. I think I use fairly regular todo keywords: TODO, WAITING, DONE and CANCELLED mostly. The first two for are used for incomplete states, and the last two for completed states.

(setq org-todo-keywords '((sequence "TODO(t)" "WAITING(w)" "|" "DONE(d)" "CANCELLED(c)")))

When on a headline, press C-c C-t to set the TODO keyword.

Timestamps, scheduling & deadlines

I tend to avoid using timestamps in my projects as much as possible. The reason is simple: unless an entry is an appointment (to the dentist for instance) or has a fixed deadline (a release scheduled with a customer), I should decide what to work on depending on the current context (among other things). This also keeps my agenda clean, free of any fake or self-imposed deadline or schedule.

But scheduling sometimes makes sense. To do that, press C-c C-s on an entry, and enter the date and/or time. To add a deadline, press C-c C-d. Note that Orgmode is quite smart about how you can enter a date, if you don’t know about it, refer to the manual entry.

Filtering projects & actions

When deciding what to work on, I use either sparse trees – which makes it easy to filter the content of my GTD projects by tag, search term, etc., or I use custom agenda commands. When discovering Orgmode, most people think that its agenda is just a regular agenda. Sure, it does daily/weekly agendas, but it offers much more than that. Quoting the manual:

Org-mode’s built-in agenda commands are powerful tools for searching your notes and for gathering, sorting, filtering, and displaying your tasks.

I use custom agenda commands mostly to get an overview of actions by context or tag. Here’s an example custom agenda command that will display all actions for the @office context:

(setq org-agenda-custom-commands 
      '(("o" "At the office" tags-todo "@office"
         ((org-agenda-overriding-header "Office")))))

Following the GTD principle, what I usually want is to only show the first action to be done (or next action) for each project with the @office tag.

That can be achieved using a skipping condition:

(setq org-agenda-custom-commands 
      '(("o" "At the office" tags-todo "@office"
         ((org-agenda-overriding-header "Office")
          (org-agenda-skip-function #'my-org-agenda-skip-all-siblings-but-first)))))

(defun my-org-agenda-skip-all-siblings-but-first ()
  "Skip all but the first non-done entry."
  (let (should-skip-entry)
    (unless (org-current-is-todo)
      (setq should-skip-entry t))
      (while (and (not should-skip-entry) (org-goto-sibling t))
        (when (org-current-is-todo)
          (setq should-skip-entry t))))
    (when should-skip-entry
      (or (outline-next-heading)
          (goto-char (point-max))))))
(defun org-current-is-todo ()
  (string= "TODO" (org-get-todo-state)))

Creating custom agenda commands can be a bit tricky at first, one easy way is to customize them via M-x customize-variable RET org-agenda-custom-commands.

To select an agenda command to execute, press C-c a.

3. The “Someday/Maybe” list

Did you notice that is not part of the agenda files set in org-agenda-files?

That’s because I do not want to see any entry from this file appearing in my agenda buffers, unless I’m doing my weekly review. That’s exactly the purpose of the “Someday/Maybe” list.

This file should be reviewed once a week as part of the weekly review (which I do this on Sunday evenings).

During each weekly review, I move projects back and forth between the “active” state (in and “later” state (in

For instance, if a project has moved forward to a certain point, but I know that it will stall for a few weeks for some reason, I move it to During a later weekly review, I’ll move it back to when it will become active again.

To move projects around, I also use refiling.

4. The tickler

The tickler is one of the best concepts of GTD in my opinion.

Let’s say you will have to pay a bill in a month. You need to write it down in your GTD if you don’t want to miss the deadline. But you also don’t want to be reminded of that each and every time you browse through your GTD projects: now is not the time to pay it.

That’s where the tickler kicks in: Add an entry to your tickler file with a timestamp, and forget about it!

When time will come, the action will appear in your Org Agenda, reminding you of the bill you have to pay, and all you will have to do is moving it to your inbox. Until then, you can just focus on something else and free your mind from this task.


All reference documents are put in a references folder next to my Orgmode files. They are then linked (using org-store-link) from Dired buffers into my projects for quick access.

I also link emails (which I happen to read within Emacs) using org-store-link.


During my weekly reviews, I archive done projects using C-c C-x C-a (org-archive-subtree-default), which moved the entry at point to an archive file.

This way my GTD files remain uncluttered and I never delete any data.


This is a very wide topic, so obviously I haven’t covered everything, but I hope that explains the basics of how I use Orgmode to implement GTD.

This is really only my way of doing it. Orgmode is such a moldable tool that I don’t think that there are 2 exactly identical setups.

-1:-- Orgmode for GTD (Post Nicolas Petton)--L0--C0--June 30, 2017 05:00 PM

Pragmatic Emacs: Multiple choices in yasnippets

I’ve written before about using yasnippet as a powerful text expansion and template tool in Emacs. In its basic form you type a keyword and press TAB to expand that into a template with active fields and then use TAB to move between the fields adding your content. I recently discovered that you can programme a snippet to contain multiple choices for fields.

For example, the following snippet uses the keyword test and inserts a template with the cursor initially in the first field and prompts the user to choose from the options “high”, “average” and “below average”. Selecting your choice and then hitting TAB takes you to the next field with options of “easy” and “hard”.

# -*- mode: snippet -*-
# name: test
# key: test
# --
Importance: ${1:$$(yas-choose-value '("high" "average" "below average"))}
Difficulty: ${2:$$(yas-choose-value '("easy" "hard"))}
-1:-- Multiple choices in yasnippets (Post Ben Maughan)--L0--C0--June 25, 2017 11:34 PM

Flickr tag 'emacs': Groupie picture with @bbatsov of Clojure and Emacs fame at EuroClojure 2016

insoporzen posted a photo:

Groupie picture with @bbatsov of Clojure and Emacs fame at EuroClojure 2016

Better late than never to post my most important moment during the last EuroClojure and send a big thanks to @bbatsov for all his great work in the Clojure community.
@bbatsov: I worked as a Ruby dev for years, too. I hope you’ll be able to make the switch full time at some point, as well(;

-1:-- Groupie picture with @bbatsov of Clojure and Emacs fame at EuroClojure 2016 (Post insoporzen ( 25, 2017 01:00 PM

Matthias Pfeifer: Picking elements from the kill-ring with completing-read

The kill ring is a nice thing to have. Only the navigation is a bit to uncomfortable (I am refering to doing C-y again and again until the to desired element is found). So this is what I came up with to feel more ‚comfy‘: browse-kill-ring.elraw download(defun pre-process-kill-ring-element (element) (replace-regexp-in-string "^[[:space:]]+" "" (replace-regexp-in-string "[[:space:]]+$" "" […]
-1:-- Picking elements from the kill-ring with completing-read (Post Matthias)--L0--C0--June 21, 2017 01:52 PM

Raimon Grau: TIL: evil-buffer-regexps

So I have my evil-mode perfectly adapted to my workflow, activating or deactivating it by default depending on the major-mode, so that I don't get into normal mode on magit buffers, or sly-*, or eww.

BUT, there are some buffers which you cannot easily control, because they reuse some major mode where I usually want evil-normal, but not in those cases.

In my case, it's *sly-macroexpansion* and *sly-description* buffers. They are both "lisp-mode" buffers (for syntax highlighting purposes and such), but I want 'q' to quit them.   As they do not represent any file, I was out of luck binding modes to file extensions. (btw, here's some recent article talking bout the ways emacs chooses your major mode)

So trying to hack something using emacs hooks, I found that, in fact, evil has us covered, and provides a evil-buffer-regexps, that allows us to predefine modes on given buffer names.

(add-to-list 'evil-buffer-regexps
'("\\*sly-macroexpansion\\*" . emacs))

-1:-- TIL: evil-buffer-regexps (Post Raimon Grau ( 19, 2017 03:21 PM

Manuel Uberti: Keeping your .emacs.d clean

How often do you have a look at your .emacs.d directory? If you’re anything like me, not that often. It’s no wonder then that upon opening it I was horrified to see it cluttered with all sorts of files from all the packages I regularly use.

Jonas Bernoulli, of Magit fame, must be feeling similar concerns. Otherwise, why did he come up with a package like no-littering?

Installation is trivial, especially if you use the great use-package:

(use-package no-littering               ; Keep .emacs.d clean
  :ensure t
  (require 'recentf)
  (add-to-list 'recentf-exclude no-littering-var-directory)
  (add-to-list 'recentf-exclude no-littering-etc-directory))

No extra setup is required. Place this snippet somewhere at the beginning of your init.el file and you are set. Use Emacs for a while and then go back checking .emacs.d: nice, huh?

To be honest, I wiped out my .emacs.d directory, cloned my configuration from GitHub and started from fresh with no-littering installed. Too much, maybe, but I was eager to see how good this package is. And yes, it is good.

House-cleaning has never been so easy.

-1:-- Keeping your .emacs.d clean (Post)--L0--C0--June 17, 2017 12:00 AM

Martin R. Albrecht: Handling Email with Emacs

Like many other people, I write, receive and loath a lot of email. Writing it goes something like this:

  1. Create a new draft,
  2. figure out the right address to put into the To: field,
  3. write “Hi <first name>”,
  4. write the actual message,
  5. attach the correct file (if any),
  6. append “Cheers, Martin”.

Also, a lot of email is repetitive and boring but necessary, such as asking seminar speakers for their titles and abstracts, giving people advise on how to claim reimbursement when they visit Royal Holloway, responding to requests of people who’d like to pursue a PhD.

Here is my attempt to semi-automate some of the boring steps in Emacs.


I use mbsync for syncing my e-mail to my local hard disk as I often work offline, e.g. during my commute or while working on airplanes.1 Mbsync does not speak IMAP IDLE, aka push notifications, so I use imapnotify for this; here’s my (sanitised) imapnotify config file:

var child_process = require('child_process');

function getStdout(cmd) {
  var stdout = child_process.execSync(cmd);
  return stdout.toString().trim();
} = "";
exports.port = 993;
exports.tls = true;
exports.username = "";
exports.password = // whatever needs doing, e.g. call getStdout()
exports.onNewMail = "mbsync googlemail-minimal";
exports.onNewMailPost = "emacsclient  -e '(mu4e-update-index)'";
exports.boxes = [ "INBOX"];

I only need imapnotify in Emacs, so I use prodigy to start/stop it.

(use-package prodigy
  :ensure t
  :init (prodigy-define-tag
          :name 'email
          :ready-message "Checking Email using IMAP IDLE. Ctrl-C to shutdown.")
    :name "imapnotify"
    :command "imapnotify"
    :args (list "-c" (expand-file-name ".config/" (getenv "HOME")))
    :tags '(email)
    :kill-signal 'sigkill))

Once arrived, email is parsed by Mu which provides fast, powerful fulltext search. Finally, Mu4e provides that email client experience™ in Emacs.

On the other end, I’m not relying on Emacs’ built-in support for sending email but use opensmtpd.2 Using Emacs’ built-in functionality means that it will hang while sending email (due to lack of multithreading), especially on slow connections, which defeats the purpose of getting email out of the way quickly.


Mu shines at search, it’s fast and expressive. For example, to search for messages between 2 kilobytes and 2Mb, written in December 2009 with an attachment from Bill, search

size:2k..2m date:20091201..20093112 flag:attach from:bill

Below are some examples for how I use it. Other ideas can be to filter for your project students, by your course number etc.

 '("flag:unread NOT flag:trashed AND (flag:list OR"
   "Unread bulk messages" ?l))

 '("flag:unread NOT flag:trashed AND NOT flag:list AND (maildir:\"/royal holloway\" OR maildir:/INBOX)"
   "Unread messages addressed to me" ?i))

 '("mime:application/* AND NOT mime:application/pgp* AND (maildir:\"/royal holloway\" OR maildir:/INBOX)"
   "Messages with attachments for me." ?d) t)

   "Flagged messages" ?f) t)

 '("(maildir:\"/[Google Mail]/.Sent Mail\" OR maildir:\"/royal holloway/.sent\") AND"
   "Sent in last 7 days" ?s) t)

By default Mu’s search is REPL, i.e. you type a query, press <enter> and look at the results. Sometimes you want real-time updates as you type, e.g. to adapt your search quickly. In this case, helm-mu has you covered. Helm adds a generic search-as-you-type interface to Emacs, here’s a nice intro.

(use-package helm-mu
  :ensure t
  :config (progn
            (bind-key "S" #'helm-mu mu4e-main-mode-map)))

By the way, enabling helm-follow-mode via C-c C-f allows to preview emails as you search.

Sometimes, you might want to file an email with some project notes to be able to find it later without any effort or you might want to refer to it directly from your TODO list. I use Org-Mode for my TODOs and notes. Mu4e comes with Org-Mode support which provides links for messages and search queries. First, enable it

(use-package org-mu4e
  :config (setq org-mu4e-link-query-in-headers-mode nil))

and then add some org-capture templates to make filing and email or creating a TODO based on an email easy:

(use-package org-capture
  :bind ("<f9>" . org-capture)
  :config (setq org-capture-templates
                '(("r" "respond ro email (mu4e)"
                   entry (file+headline malb/inbox-org "Email")
                   "* REPLY to [[mailto:%:fromaddress][%:fromname]] on %a\nDEADLINE: %(org-insert-time-stamp (org-read-date nil t \"+1d\"))\n%U\n\n"
                   :immediate-finish t
                   :prepend t)

                  ("f" "file email (mu4e)"
                   entry (file+headline malb/inbox-org "Email")
                   "* %a by [[mailto:%:fromaddress][%:fromname]]\n%U\n\n%i%?\n"
                   :immediate-finish nil
                   :prepend nil))))


First, let’s make finding that email address easier. For this, I want an automatically maintained database holding at least

  1. first name,
  2. last name and
  3. email address

which is then used for autocompletion as I type. “Automatically maintained“ here means that this database should be built from our email correspondence, similar to e.g. what Gmail does. Adding email addresses and whatever else is in the From: field to some database isn’t difficult per se and many clients do it. For example, Mu4e comes with this built-in.

However, there are a few different conventions out there for how people write names in a From: field, so this needs a bit of tidying up. For example, Royal Holloway likes “Lastname, Firstname (Year)” for students; some people like to YELL their LASTNAME and then write the first name; some people misspell their own name. The code below canonicalises this.

(defun malb/canonicalise-contact-name (name)
  (let ((case-fold-search nil))
    (setq name (or name ""))
    (if (string-match-p "^[^ ]+@[^ ]+\.[^ ]" name)
        ;; drop email address
        (setq name (replace-regexp-in-string "^\\(.*\\) [^ ]+@[^ ]+\.[^ ]" "\\1" name)) 
        ;; strip quotes
        (setq name (replace-regexp-in-string "^\"\\(.*\\)\"" "\\1" name)) 
        ;; deal with YELL’d last names
        (setq name (replace-regexp-in-string "^\\(\\<[[:upper:]]+\\>\\) \\(.*\\)" "\\2 \\1" name))
        ;; Foo, Bar becomes Bar Foo
        (setq name (replace-regexp-in-string "^\\(.*\\), \\([^ ]+\\).*" "\\2 \\1" name)) 
        ;; look up names and replace from static table, TODO look this up by email
        (setq name (or (cdr (assoc name malb/mu4e-name-replacements)) name)) 

(defun malb/mu4e-contact-rewrite-function (contact)
  (let* ((name (or (plist-get contact :name) ""))
         (mail (plist-get contact :mail))
         (case-fold-search nil))
    (plist-put contact :name (malb/canonicalise-contact-name name))

(setq mu4e-contact-rewrite-function #'malb/mu4e-contact-rewrite-function)

Now that our addresses are canonicalised, I can use those to fill in a few more bits. Given an email starting with “To: John Doe <>” there is no point in typing the name “John” again when I do the customary “Hi …,”. Here, YASnippet comes in. YASnippet is a templating system for Emacs inspired by TextMate, which allows to map short sequences of characters to other sequences of characters, potentially by asking for more user input and/or calling some arbitrary Emacs Lisp function. For example, here’s the template we use to advertise the ISG seminar

# -*- mode: snippet -*-
# name: Announce ISG Research Seminar
# key: isg-announce
# --
${1:Thu}, $2 @ ${3:11:00} in ${4:HLT2}: $5

When:   $1, $2, 2016 @ $3
Where:  $4
Why:    Because… reasons!
Who:    $5 ($6)

# Title #


# Abstract #

# Bio #

Lorenzo & Martin

and here’s my “hi” template

# -*- mode: snippet -*-
# name: Say "hi"
# key: Hi
# --
Hi ${1:`(malb/yas-get-names-from-to-fields)`},



Using this snippet, typing Hi<Tab> triggers email boilerplate to be inserted, with the cursor eventually placed in the position of $0. The name used in the greeting is computed using the following function:

(defun malb/yas-get-names-from-fields (fields)
  (let (names
        (search-regexp (mapconcat (lambda (arg)
                                    (concat "^" arg ": "))
                                  fields "\\|"))
        (case-fold-search nil))
      (goto-char (point-min))
      (while (re-search-forward search-regexp nil t)
          (setq point-end-of-line (re-search-forward "$")))
        (setq name (buffer-substring-no-properties (point) point-end-of-line))
        (setq name (split-string name "[^ ]+@[^ ]+," t " ")) ;; split on email@address,
        (setq names (append names name)))
      (dolist (name names)
        (setq name (malb/canonicalise-contact-name name))
        (if (string-match "\\([^ ,]+\\)" name)
              (setq name (match-string 1 name))
              (setq name (capitalize name))
              (if ret
                  (setq ret (concat ret ", " name))
                (setq ret name)))))
      (if ret ret "there"))))

(defun malb/yas-get-names-from-to-fields ()
  (malb/yas-get-names-from-fields '("To")))

Of course, you can create much more elaborate snippets calling all kinds of functions to respond to all kinds of email. Once you created so many snippets that you’re at risk of loosing track, I recommend helm-yasnippet as a nice interactive interface for selecting the right snippet.

To simplify adding attachments — because traversing directory trees is boring — I wrote a small interface to Baloo, which is KDE’s version of OSX’s Spotlight, i.e. desktop search:

(defcustom helm-baloo-file-limit 100
  "Limit number of entries returned by baloo to this number."
  :group 'helm-baloo
  :type '(integer :tag "Limit"))

(defun baloo-search (pattern)
  (start-process "baloosearch" nil "baloosearch" (format "-l %d " helm-baloo-file-limit) pattern))

(defun helm-baloo-search ()
  (baloo-search helm-pattern))

(defun helm-baloo-transform (cs)
  (let '(helm-baloo-clean-up-regexp (rx (or
                                         (seq "[0;31m" (+ (not (any "["))) "[0;0m")
    (mapcar (function
             (lambda (c)
                (rx (seq bol (+ space))) ""
                (replace-regexp-in-string helm-baloo-clean-up-regexp "" c))))

(defvar helm-source-baloo
  (helm-build-async-source "Baloo"
    :candidates-process #'helm-baloo-search
    :candidate-transformer #'helm-baloo-transform
    :action '(("Open" . (lambda (x) (find-file x)))
              ("Attach to Email" . (lambda (x) (mml-attach-file x))))))

(defun helm-baloo ()
  (helm :sources helm-source-baloo
        :buffer "*helm baloo*"))

The line ("Attach to Email" . (lambda (x) (mml-attach-file x)) adds an option to attach any file to an email by pressing <F2>. If you prefer good ol’ locate, you can add this option to helm-locate too:

(helm-add-action-to-source "Attach to Email" #'mml-attach-file helm-source-locate)

Finally, a few more nice-to-have tweaks:

  • I write email in German and English and manually switching spell-checking dictionaries is not on; auto-dictionary-mode allows to pick the right dictionary automatically by looking for key words and their frequency like “the” or “der/die/das”.
  • Footnotes in email can be quite handy.
  • Typo-mode replace quotes and friends with their typographically correct counterparts because UTF-8 is a thing.
  • LaTeX notation like \lambda gets replaced by \lambda because, again, UTF-8 is a thing.
(add-hook 'message-mode-hook #'flyspell-mode)
(add-hook 'message-mode-hook #'typo-mode)
(add-hook 'message-mode-hook #'adict-guess-dictionary)
(add-hook 'message-mode-hook #'footnote-mode)



GMail takes care of all sorting into folders aka labels.


Debian GNU/Linux comes with exim4 by default, which isn’t easy to configure. OpenSMTPD, on the other hand, is rather straightforward.

-1:-- Handling Email with Emacs (Post martinralbrecht)--L0--C0--June 09, 2017 04:13 PM

Anything goes: Launch Emacs for Windows with Batch File using Emacs Configuration from Cygwin

I maintain a shared Emacs configuration under Linux and Cygwin. In Windows, I wanted to easily launch Emacs from the Desktop with the following goals.

  • Create a shortcut in the taskbar that launches Emacs for Windows
  • Have Emacs for Windows obey the Emacs configuration from Cygwin
  • Avoid opening extra console windows

Some prerequisites,

I created a file called c:\cygwin64\emacs.bat with the following contents.
c:\cygwin64\bin\run bash.exe -c ~/dotfiles/bin/

The file ~/dotfiles/bin/ is a shell script that sits in my Cygwin directory. It is a wrapper script that calls Emacs for Windows if I'm running Cygwin.

It is assumed that the default Emacs configuration sits in the ~/.emacs file under Cygwin.

Pin the c:\cygwin64\emacs.bat script to your task bar. See below if you do not know how to do this. Clicking on it will open Emacs for Windows without any intervening console windows.

Pinning a batch script to the Windows taskbar.
-1:-- Launch Emacs for Windows with Batch File using Emacs Configuration from Cygwin (Post tsengf ( 05, 2017 06:50 AM

Chen Bin (redguardtoo): Enhance diff-mode with Ivy

My current job requires me to review the freelancer's patches and apply them to our code branch under Perforce control. Due to my client's security policy, the freelancer can only work on isolated sandbox environment and can't access our code base directly.

I need two steps to finish the task:

  • Open the freelancer's patch in diff-mode
  • Run diff-apply-hunk to apply the hunks interactively

The problem is diff-mode always ask me to specify the file to be patched.

I read the code of diff-apply-hunk. The logic of diff-apply-hunk is simple. It tries different algorithms to guess the right file to patch. When the algorithms fail, it calls API read-file-name to ask me to provide the file path manually. If right file is found, the algorithms will work again and read-file-name will never be used for other hunks.

Here is my solution. I can find the file to patch in recent opened files because I store all of them by (setq recentf-max-saved-items 2048). I plan to use ivy-read from Ivy to locate the file at first. If this step fails , I can still fall back on original API read-file-name.

Here is the code

(defvar ffip-read-file-name-hijacked-p nil)
(defun ffip-diff-apply-hunk (&optional reverse)
  (interactive "P")
  (unless recentf-mode (recentf-mode 1))
  (setq ffip-read-file-name-hijacked-p t)
  (defadvice read-file-name (around ffip-read-file-name-hack activate)
      (let* ((args (ad-get-args 0))
             (file-name (file-name-nondirectory (nth 2 args)))
             (cands (remove nil (mapcar (lambda (s) (if (string-match-p (format "%s$" file-name) s) s))
                                        (mapcar #'substring-no-properties recentf-list))))
             (rlt (ivy-read "Recentf: " cands)))
        (if rlt (setq ad-return-value rlt) rlt ad-doit)))
  (diff-apply-hunk reverse)
  (setq ffip-read-file-name-hijacked-p nil))

Please note ffip-diff-apply-hunk can replace diff-apply-hunk.

BTW, I can edit the patch on the spot when applying hunks. Similar to the work flow of git add --patch.

The solution is added into

-1:-- Enhance diff-mode with Ivy (Post Chen Bin)--L0--C0--May 30, 2017 12:29 PM

Ben Simon: The surprisingly simple path to a foot powered emacs

I finally got around to soldering the components that power my USB foot pedal:

While it's hardly a polished build, it's currently working quite well. I've found that it's an essential accessory for navigating macOS. To be able to jump from monitor to monitor without moving my fingers from the home row is a joy.

On the Windows side of things, I'd planned to use the pedal to enhance emacs. There were two barriers to this: first, I needed a good key stroke related problem to solve. I've been using emacs for so long that even inefficient key-strokes feel right. The second challenge was figuring out how to integrate the pedal into emacs's keymaps and such. I imagined having to wade through thousands of lines of elisp to figure out an esoteric solution.

Turns out, I was way overthinking things. This article explained more than I needed to know about emacs key handling. It was this snippet that ultimately paved the way for a trivial implementation:

In order to actually bind a key you must first tell Emacs what key you intend to use. Unfortunately there’s more than one way of representing keys in Emacs: as a string, or as a vector. We won’t concern ourselves with either, as we will use a handy macro built in to Emacs called kbd.

The kbd macro translates a human-readable key into a format Emacs can understand.

I had previously coded the foot pedal to send F8 whenever it was pressed. I hesitantly popped into the *scratch* buffer and evaluated the following:

(global-set-key (kbd "<f8>") 'goto-line)

I pressed the foot pedal and emacs prompted me for a line to jump to.

OK, fair enough. But surely setting the foot pedal up to act as a sort of prefix key would require some elisp-fu. After invoking global-unset-key on F8 I tried the following:

(global-set-key (kbd "<f8> r") 'rgrep)
(global-set-key (kbd "<f8> g") 'goto-line)

I pushed the pedal down and typed the letter 'r'. And just like that, emacs asked me what I wanted to search for. I tried the same with pushing the pedal followed by g. That worked to.

Apparently kbd and global-set-key just do the right thing. So much for having to read mountains of elisp code.

I was then left with my initial challenge: how did I actually want to use the pedal to be efficient? For now, I've decided that any command I regularly type M-x, I'll bind to the pedal plus another key. As you can guess, M-x rgrep and M-x goto-line are two common examples of this. I'll definitely be on the lookout for more interesting challenges to solve, but at least it's a start.

Have any suggestions for how I can get put the foot pedal to use? Please drop me a comment below.

-1:-- The surprisingly simple path to a foot powered emacs (Post Ben Simon ( 24, 2017 06:49 PM

Bryan Murdock: Facebook Should Split In Two

Facebook has done wonders to get people creating and consuming content on the internet. However, Facebook has grown to the point where it has no competition and is no longer innovating in ways that benefit us. Facebook should split into Facebook the aggregator and Facebook the content hoster.  You could talk about a third piece that is Facebook the content provider, which is for providing things like gifs, templates, memes, emoji, games, and other stuff like that.  Because Facebook hasn't completely broken from open web standards those types of content providers already exist today.

Aggregators would be where you go to set up your friend list and see your feed.  It could look and feel like Facebook does now.  It would have an open standard protocol that content hosters would use if they wanted to be aggregated.  This could still be an add driven business, but subscription, self hosted, and DIY solutions could exist too.

Content hosters could either charge a monthly hosting fee, or they could serve up their own adds.  Self hosted and DIY solutions could also exist.

The big benefit to this would of course be the competition.  Since it's an open standard anyone could be a content host, and anyone could be an aggregator.

To make extra sure there is competition, and this could come in a phase two after the initial splitting up of Facebook, there should be open standards for exporting and importing friends, follows, likes, etc. to and from aggregators, and open standards for importing and exporting content from the hosters.

Speaking of follows and likes, there could also be aggregator aggregators (AAs). People could opt in to publicly and anonymously share their likes and follows and the AAs would consume those and report on trends that cross aggregator boundaries.  Anonymity could be much more protected this way while still giving us that interesting information about what is trending.

One tricky part of this is how do I as a content author only allow my friends to see certain posts of mine?  It would have to be with encryption. My content provider could keep public keys of my friends and only my friends (well, their aggregators) would be able to decrypt my posts using my friends' private keys.  I can see some challenges and holes in this, but it doesn't seem any worse overall than how Facebook protects privacy now. Open implementations and peer review could get us to better-than-Facebook privacy quickly.

Facebook would ideally recognize their stagnation and initiate this split themselves. We as their user base can and should help them understand the importance of this. Hopefully it doesn't have to come down to government enforcement of anti-trust laws, but that could be a useful tool to apply here as well.
-1:-- Facebook Should Split In Two (Post Bryan ( 24, 2017 10:58 AM

Chen Bin (redguardtoo): Firefox and Emacs

For me, there is NO difference between Firefox and Emacs. They provide useful APIs, nothing more.

Three years ago, I wrote Use firefox in Emacs way to demo how to convert Firefox into Emacs by Keynsail.

A year ago I published Hello Ivy-mode, bye Helm to prove how powerful Ivy-mode is by using its API ivy-read.

Keysnail has similar javascript API prompt.selector and it's as powerful as ivy-read if not more powerful.

For example, you can insert below snippet into ~/.keysnail.js and press ",hh" or "C-c C-h" to query browse history:

function searchHistory(evt, arg) {
  function timeSince(now, date) {

    var seconds = Math.floor((now - date) / 1000);

    var interval = Math.floor(seconds / 31536000);

    if (interval > 1) {
      return interval + " years";
    interval = Math.floor(seconds / 2592000);
    if (interval > 1) {
      return interval + " months";
    interval = Math.floor(seconds / 86400);
    if (interval > 1) {
      return interval + " days";
    interval = Math.floor(seconds / 3600);
    if (interval > 1) {
      return interval + " hours";
    interval = Math.floor(seconds / 60);
    if (interval > 1) {
      return interval + " minutes";
    return Math.floor(seconds) + " seconds";

  function searchWithKeyword(q) {
    var collection = (function() {
      //search option
      var options = PlacesUtils.history.getNewQueryOptions();
      options.maxResults = 4096;
      options.queryType = Ci.nsINavHistoryQueryOptions.QUERY_TYPE_HISTORY;
      //options.sortingMode = Ci.nsINavHistoryQueryOptions.SORT_BY_FRECENCY_DESCENDING;
      options.sortingMode = Ci.nsINavHistoryQueryOptions.SORT_BY_DATE_DESCENDING;
      options.includeHidden = true;

      //search query
      var query = PlacesUtils.history.getNewQuery();
      // read keyworld
      if(q && q !== '') {
        query.searchTerms  = q;

      var result = PlacesUtils.history.executeQuery(query, options);
      var root = result.root;

      var collection = [];
      var now = new Date().getTime();
      var siteNode;
      root.containerOpen = true;
      for (var i = 0; i < root.childCount; i++) {
        // siteNode => nsINavHistoryResultNode
        siteNode = root.getChild(i);
        collection.push([siteNode.icon,siteNode.title,siteNode.uri, siteNode.time/1000]);
      collection.sort(function(a, b) {
        return b[3]-a[3];
      // reformat the time
      for (i = 0; i < collection.length; i++) {
        collection[i][3] = timeSince(now, collection[i][3]) + ' ago';
      root.containerOpen = false;
      return collection;

      message    : "Search history"+ (q && q !== ''? (' @'+q +':') : ':' ),
      collection : collection,
      flags      : [ICON | IGNORE, 0, 0, 0],
      header     : ["Title", "Url", "Last visited"],
      width      : [30, 60, 10],
      callback: function (i) {
        if (i >= 0) {
          openUILinkIn(collection[i][2], "tab");
      onFinish: function() {

  prompt.finish(true);'Keyword to search history?', searchWithKeyword, null, null, null, 0, "history_search");
  // searchWithKeyword('test');

key.setViewKey([',', 'h', 'h'], searchHistory, "Search history");
key.setGlobalKey(['C-c', 'C-h'], searchHistory, "Search history");

Here is my complete .keysnail.js.

-1:-- Firefox and Emacs (Post Chen Bin)--L0--C0--May 23, 2017 12:24 PM

Matthias Pfeifer: Emacs memory consumption

The Emacs built-in command (garbage-collect) gives detailed information about the data structures that currently consume memory. It is propably not the most usefull information but I wanted to collect the data and plot it. I started with writing functions to access the list returned from (garbage-collect): (defsubst get-mem-conses (mi) (let ((data (nth 0 mi))) (/ […]
-1:-- Emacs memory consumption (Post Matthias)--L0--C0--May 17, 2017 09:57 AM

emacshorrors: The Dude Abides

While crafting another patch for cc-mode, I stumbled upon the following:

;; RMS says don't make these the default.
;; (April 2006): RMS has now approved these commands as defaults.
(unless (memq 'argumentative-bod-function c-emacs-features)
  (define-key c-mode-base-map "\e\C-a"    'c-beginning-of-defun)
  (define-key c-mode-base-map "\e\C-e"    'c-end-of-defun))

I’ve seen this kind of thing before, but it keeps puzzling me. Why would you:

  • Put a comment there about not making the following a default
  • Put a clarifying comment how these have been approved as defaults now
  • Keep both comments to trip up future readers

Any other project I’m involved in would immediately remove that commentary. Emacs is the only one where suggesting such drastic action would yield a lengthy bikeshedding discussion about the merits of pre-VCS traditions and how keeping them will toughen up whoever may join in the collaborative development effort. Yes, seriously.

-1:-- The Dude Abides (Post Vasilij Schneidermann)--L0--C0--May 15, 2017 07:59 PM