Irreal: Beginning Emacs Tutorial Video

Derek Banas has posted a nice introduction to Emacs video. It covers roughly the same material as the built-in tutorial with the addition of setting up a Clojure programming environment. Banas covers installing Emacs on both macOS and Windows. One of the commenters complained that he didn’t cover setting it up on Linux but it’s hard to imagine a Linux user not knowing how to install an application. He uses version 25.3 of Emacs so he’s working with an up-to-date Emacs.

Watching the video really brought home how much we all customize our Emacsen. Since he was using a stock install, he didn’t have things like minibuffer completion for file names and commands. It seemed painfully slow compared to how we work when we have something like Swiper/Counsel/Ivy or Ido/Smex installed.

My major complaint is that he used “window” and “frame” interchangeably. That distinction is confusing enough for n00bs without adding to the entropy in an introductory video. On the other hand, the production values for the video are excellent. It was very professionally done, including seamless switching between macOS and Windows.

If you’re an experienced Emacs user, the video probably won’t tell you anything you don’t already know but it’s an excellent resource for beginners. The video is a few seconds over 52 minutes so you’ll have to schedule time for it.

-1:-- Beginning Emacs Tutorial Video (Post jcs)--L0--C0--November 19, 2017 06:24 PM

Flickr tag 'emacs': GAMS mode: the basic screenshot.

shiro.takeda posted a photo:

GAMS mode: the basic screenshot.

Screenshots of GAMS mode for Emacs.

-1:-- GAMS mode: the basic screenshot. (Post shiro.takeda ( 19, 2017 07:25 AM

Flickr tag 'emacs': GAMS mode: the basic screenshot.

shiro.takeda posted a photo:

GAMS mode: the basic screenshot.

Screenshots of GAMS mode for Emacs.

-1:-- GAMS mode: the basic screenshot. (Post shiro.takeda ( 19, 2017 07:25 AM

Irreal: Magit Log for a File

Here’s a really nice tip from Wilfred Hughes’ Emacs Command of the Day Twitter feed:

This is just the thing if you need to examine to the commit history for a particular file. I ran it on one of my oldest files in Git (9 years) and could see all the changes in a buffer of only 161 lines instead of trying to pick them out from the thousands of lines in a full log listing.

Take a look at the documentation to see some additional capabilities of the command. You can, for instance, select a region in the current buffer and the command will show only the commits that affected those lines.

You can, of course, do the same thing interactively from the Magit menu but not as easily. If you find yourself needing to examine the commit history of a single file (semi-)regularly, it’s worthwhile remembering the magit-log-buffer-file command.

-1:-- Magit Log for a File (Post jcs)--L0--C0--November 18, 2017 07:02 PM

punchagan: Multiple remotes with nullmailer

This a reference for future-me, and possibly someone pulling off an all-nighter trying to get nullmailer to use the correct “remote”.

What is nullmailer and why use it?

Nullmailer is a simple mail transfer agent that can forward mail to a remote mail server (or a bunch of them).

I use Emacs to send email, and it can be configured to talk to a remote SMTP server to send email. But, this blocks Emacs until the email is sent and the connection closed. This is annoying, and having nullmailer installed locally basically lets Emacs delegate this job without blocking.

Why multiple remotes?

I have multiple email accounts, and I’d like to use the correct remote server for sending email based on the FROM address.

I expected nullmailer to have some configuration to be able to specify this. But, it turns out that nullmailer just forwards the email to all the configured remotes until one of them succeeds.

How do we, then, send email from the correct remote SMTP server?

Currently, I have two remotes - my personal domain ( and GMail.

Having GMail as the first remote in nullmailer’s configuration wouldn’t let me send emails from my personal domain. GMail seems to agree to send the email coming from, but overwrite the MAIL FROM address and change it to my GMail address.

So, has to be the first remote. But, this server also seemed to accept and send emails with a FROM address. This was causing emails sent from my GMail ID to go into spam, as expected.

I had to reconfigure this mail server to reject relaying mails that didn’t belong to the correct domain names – i.e., reject relaying emails which had in the FROM address.

smtpd_sender_restrictions had to modified to have reject_sender_login_reject along with other values, and the smtpd_sender_login_maps had to be set to allow only the domain. This serverfault answer explains this in more detail.

-1:-- Multiple remotes with nullmailer (Post)--L0--C0--November 17, 2017 11:20 PM

(or emacs: Save Ivy file completions to Dired


I think ivy-occur (C-c C-o) is one of the coolest features in ivy. It allows you to save your current search into a new buffer. This has many uses:

  • get a full overview of all candidates
  • many useful modal bindings (q, j, k, f) and mouse support
  • ability to manipulate candidates as text
  • save the search for later, with the option to refresh the search with g
  • go over candidates as a TODO list, using C-d to remove elements

Everything above works for any ivy-read session. But the most powerful features come into play when ivy-occur gets customized for a specific collection.

ivy-occur for grep-like functions

(ivy-set-occur 'swiper 'swiper-occur)

Thanks to this default customization, the resulting *ivy-occur swiper* buffer is in ivy-occur-grep-mode which inherits from grep-mode. Additionally, you can use ivy-wgrep-change-to-wgrep-mode C-x C-q to edit the result in-place - pressing C-x C-s will save the changes.

Similar customizations are available for counsel-git-grep, counsel-ag, counsel-rg, and counsel-grep.

ivy-occur for ivy-switch-buffer

(ivy-set-occur 'ivy-switch-buffer 'ivy-switch-buffer-occur)

This makes C-c C-o open your candidates in the powerful ibuffer, which adds additional info to your buffer list and allows you to manipulate buffers easily.

For instance, to delete all matching buffers you can do C-c C-o tD.

The source code is short enough to be included here:

(defun ivy-switch-buffer-occur ()
  "Occur function for `ivy-switch-buffer' using `ibuffer'."
  (ibuffer nil (buffer-name) (list (cons 'name ivy--old-re))))

The interface is quite simple: ivy-occur is responsible for generating a new buffer, and the occur function e.g. ivy-switch-buffer-occur is to fill that buffer with useful info, based on the current search parameters like ivy-text and ivy--old-re.

ivy-occur for counsel-find-file-like functions

This is a brand new feature that works for counsel-find-file, counsel-git, and counsel-fzf (which itself is quite new, thanks to @jojojames for contributing it).

Since these functions are used to complete file names, we obviously want ivy-occur to open a Dired buffer.

Example 1

To delete all *.elc files in the current folder do:

  • C-x C-f elc$ C-c C-o tDy.

Example 2

To copy all Org files in a Git project to some directory do:

  • M-x counsel-git org$ C-c C-o tC.

Example 3

To get a list of videos to watch do:

  • M-x counsel-fzf mp4$ C-c C-o.

I can further e.g. mark 3 files with m and use r to send these 3 files to vlc as a list. See this post for my dired setup that makes r work this way.

You can remove some files afterwards with the usual D or dx. And to redisplay the buffer use g.


I hope you like the new feature. I had a really good few hours figuring out how it should work exactly. Please consider joining my 72 patrons to give me the opportunity to work on Free Software a lot more. Happy hacking!

-1:-- Save Ivy file completions to Dired (Post)--L0--C0--November 17, 2017 11:00 PM

William Denton: Org clocktables II: Summarizing a month

In Org clocktables I: The daily structure I explained how I track my time working at an academic library, clocking in to projects that are either categorized as PPK (“professional performance and knowledge,” our term for “librarianship,”), PCS (“professional contributions and standing”, which covers research, professional development and the like) and Service. I do this by checking in and out of tasks with the magic of Org.

I’ll add a day to the example I used before, to make it more interesting. This is what the raw text looks like:

* 2017-12 December

** [2017-12-01 Fri]
CLOCK: [2017-12-01 Fri 09:30]--[2017-12-01 Fri 09:50] =>  0:20
CLOCK: [2017-12-01 Fri 13:15]--[2017-12-01 Fri 13:40] =>  0:25

*** PPK

**** Libstats stuff
CLOCK: [2017-12-01 Fri 09:50]--[2017-12-01 Fri 10:15] =>  0:25

Pull numbers on weekend desk activity for A.

**** Ebook usage
CLOCK: [2017-12-01 Fri 13:40]--[2017-12-01 Fri 16:30] =>  2:50

Wrote code to grok EZProxy logs and look up ISBNs of Scholars Portal ebooks.

*** PCS

*** Service

**** Stewards' Council meeting
CLOCK: [2017-12-01 Fri 10:15]--[2017-12-01 Fri 13:15] =>  3:00

Copious meeting notes here.

** [2017-12-04 Mon]
CLOCK: [2017-12-04 Mon 09:30]--[2017-12-04 Mon 09:50] =>  0:20
CLOCK: [2017-12-04 Mon 12:15]--[2017-12-04 Mon 13:00] =>  0:45
CLOCK: [2017-12-04 Mon 16:00]--[2017-12-04 Mon 16:15] =>  0:15

*** PPK

**** ProQuest visit
CLOCK: [2017-12-04 Mon 09:50]--[2017-12-04 Mon 12:15] =>  2:25

Notes on this here.

**** Math print journals
CLOCK: [2017-12-04 Mon 16:15]--[2017-12-04 Mon 17:15] =>  1:00

Check current subs and costs; update list of print subs to drop.

*** PCS

**** Pull together sonification notes
CLOCK: [2017-12-04 Mon 13:00]--[2017-12-04 Mon 16:00] =>  3:00

*** Service

All raw Org text looks ugly, especially all those LOGBOOK and PROPERTIES drawers. Don’t let that put you off. This is what it looks like on my screen with my customizations (see my .emacs for details):

Much nicer in Emacs. Much nicer in Emacs.

At the bottom of the month I use Org’s clock table to summarize all this.

#+BEGIN: clocktable :maxlevel 3 :scope tree :compact nil :header "#+NAME: clock_201712\n"
#+NAME: clock_201712
| Headline             | Time  |      |      |
| *Total time*           | *14:45* |      |      |
| 2017-12 December     | 14:45 |      |      |
| \_  [2017-12-01 Fri] |       | 7:00 |      |
| \_    PPK            |       |      | 3:15 |
| \_    Service        |       |      | 3:00 |
| \_  [2017-12-04 Mon] |       | 7:45 |      |
| \_    PPK            |       |      | 3:25 |
| \_    PCS            |       |      | 3:00 |

I just put in the BEGIN/END lines and then hit C-c C-c and Org creates that table. Whenever I add some more time, I can position the pointer on the BEGIN line and hit C-c C-c and it updates everything.

Now, there are lots of commands I could use to customize this, but this is pretty vanilla and it suits me. It makes it clear how much time I have down for each day and how much time I spent in each of the three pillars. It’s easy to read at a glance. I fiddled with various options but decided to stay with this.

It looks like this on my screen:

Much nicer in Emacs. Much nicer in Emacs.

That’s a start, but the data is not in a format I can use as is. The times are split across different columns, there are multiple levels of indents, there’s a heading and a summation row, etc. But! The data is in a table in Org, which means I can easily ingest it and process it in any language I choose, in the same Org file. That’s part of the power of Org: it turns raw data into structured data, which I can process with a script into a better structure, all in the same file, mixing text, data and output.

Which language, though? A real Emacs hacker would use Lisp, but that’s beyond me. I can get by in two languages: Ruby and R. I started doing this in Ruby, and got things mostly working, then realized how it should go and what the right steps were to take, and switched to R.

Here’s the plan:

  • ignore “Headline” and “Total time” and “2017-12 December” … in fact, ignore everything that doesn’t start with “\_”
  • clean up the remaining lines by removing “\_”
  • the first line will be a date stamp, with the total day’s time in the first column, so grab it
  • after that, every line will either be a PPK/PCS/Service line, in which case grab that time
  • or it will be a new date stamp, in which case capture that information and write out the previous day’s information
  • continue on through all the lines
  • until the end, at which point a day is finished but not written out, so write it out

I did this in R, using three packages to make things easier. For managing the time intervals I’m using hms, which seems like a useful tool. It needs to be a very recent version to make use of some time-parsing functions, so it needs to be installed from GitHub. Here’s the R:

library(hms) ## Right now, needs GitHub version
clean_monthly_clocktable <- function (raw_clocktable) {
  ## Clean up the table into something simple
  clock <- raw_clocktable %>% filter(grepl("\\\\_", Headline)) %>% mutate(heading = str_replace(Headline, "\\\\_ *", "")) %>% mutate(heading = str_replace(heading, "] .*", "]")) %>% rename(total = X, subtotal = X.1) %>% select(heading, total, subtotal)

  ## Set up the table we'll populate line by line
  newclock <- tribble(~date, ~ppk, ~pcs, ~service, ~total)

  ## The first line we know has a date and time, and always will
  date_old <- substr(clock[1,1], 2, 11)
  total_time_old <- clock[1,2]
  date_new <- NA
  ppk <- pcs <- service <- vacation <- total_time_new <- "0:00"

  ## Loop through all lines ...
  for (i in 2:nrow(clock)) {
    if      (clock[i,1] == "PPK")     { ppk      <- clock[i,3] }
    else if (clock[i,1] == "PCS")     { pcs      <- clock[i,3] }
    else if (clock[i,1] == "Service") { service  <- clock[i,3] }
    else {
     date_new <- substr(clock[i,1], 2, 11)
     total_time_new <- clock[i,2]
    ## When we see a new date, add the previous date's details to the table
    if (! {
     newclock <- newclock %>% add_row(date = date_old, ppk, pcs, service, total = total_time_old)
     ppk <- pcs <- service <- "0:00"
     date_old <- date_new
     date_new <- NA
     total_time_old <- total_time_new

  ## Finally, add the final date to the table, when all the rows are read.
  newclock <- newclock %>% add_row(date = date_old, ppk, pcs, service, total = total_time_old)
  newclock <- newclock %>% mutate(ppk = parse_hm(ppk), pcs = parse_hm(pcs), service = parse_hm(service), total = parse_hm(total), lost = as.hms(total - (ppk + pcs + service))) %>% mutate(date = as.Date(date))

All of that is in a SRC block like below, but I separated the two in case it makes the syntax highlighting clearer. I don’t think it does, but such is life. Imagine the above code pasted into this block:

#+BEGIN_SRC R :session :results values


Running C-c C-c on that will produce no output, but it does create an R session and set up the function. (Of course, all of this will fail if you don’t have R (and those three packages) installed.)

With that ready, now I can parse that monthly clocktable by running C-c C-c on this next source block, which reads in the raw clock table (note the var setting, which matches the #+NAME above), parses it with that function, and outputs cleaner data. I have this right below the December clock table.

#+BEGIN_SRC R :session :results values :var clock_201712=clock_201712 :colnames yes

|       date |      ppk |      pcs |  service |    total |     lost |
| 2017-12-01 | 03:15:00 | 00:00:00 | 03:00:00 | 07:00:00 | 00:45:00 |
| 2017-12-04 | 03:25:00 | 03:00:00 | 00:00:00 | 07:45:00 | 01:20:00 |

This is tidy data. It looks this this:

Again, in Emacs Again, in Emacs

That’s what I wanted. The code I wrote to generate it could be better, but it works, and that’s good enough.

Notice all of the same dates and time durations are there, but they’re organized much more nicely—and I’ve added “lost.” The “lost” count is how much time in the day was unaccounted for. This includes lunch (maybe I’ll end up classifying that differently), short breaks, ploughing through email first thing in the morning, catching up with colleagues, tidying up my desk, falling into Wikipedia, and all those other blocks of time that can’t be directly assigned to some project.

My aim is to keep track of the “lost” time and to minimize it, by a) not wasting time and b) properly classifying work. Talking to colleagues and tidying my desk is work, after all. It’s not immortally important work that people will talk about centuries from now, but it’s work. Not everything I do on the job can be classified against projects. (Not the way I think of projects—maybe lawyers and doctors and the self-employed think of them differently.)

The one technical problem with this is that when I restart Emacs I need to rerun the source block with the R function in it, to set up the R session and the function, before I can rerun the simple “update the monthly clocktable” block. However, because I don’t restart Emacs very often, that’s not a big problem.

The next stage of this is showing how I summarize the cleaned data to understand, each month, how much of my time I spent on PPK, PCS and Service. I’ll cover that in another post.

-1:-- Org clocktables II: Summarizing a month (Post William Denton)--L0--C0--November 17, 2017 04:46 AM

Timo Geusch: Emacs on the Linux Subsystem for Windows

I’ve had the Linux Subsystem for Windows enabled for quite a while during the time it was in Beta. With the release of the Fall Creators Update, I ended up redoing my setup from scratch. As usual I grabbed Emacs Read More

The post Emacs on the Linux Subsystem for Windows appeared first on The Lone C++ Coder's Blog.

-1:-- Emacs on the Linux Subsystem for Windows (Post Timo Geusch)--L0--C0--November 15, 2017 01:48 PM

Pragmatic Emacs: Even more PDF-tools tweaks

Once I started tweaking pdf-tools I couldn’t resist a few more bits of streamlining my workflow. I wanted to be able to commit an annotation by hitting RET rather than C-c C-c and then use SHIFT and RET to actually enter a newline, a bit like in many messaging apps. The problem was that the necessary commands are in pdf-annot-edit-contents-minor-mode-map which is a key map that is not available until that minor mode is loaded. This causes an error if you try to define a key in that map in the same way as I did for the keys to add annotations. This is remedied by wrapping the definitions in with-eval-after-load which waits to execute the code until the file that defines the pdf-annot mode is loaded.

The other thing I wanted to do was have the pdf file auto-save every time I commit an annotation. I did this by advising the function pdf-annot-edit-contents-commit to run save-buffer afterwards. This also had to be inside with-eval-after-load as the pdf-annot-edit-contents-commit function is not available to be advised until the file defining the pdf-annot mode is loaded. The only other issue I had was that when the advice to pdf-annot-edit-contents-commit called save-buffer, it passed its argument (the text of the annotation) to save-buffer which then failed. I hacked around this by writing a simple wrapper to save buffer that ignores any arguments. There is probably a better way to do this!

Putting it all together with my previous tweaks, my pdf-tools config looks like this:

;; wrapper for save-buffer ignoring arguments
(defun bjm/save-buffer-no-args ()
  "Save buffer ignoring arguments"

(use-package pdf-tools
 :pin manual ;;manually update
 ;; initialise
 (setq-default pdf-view-display-size 'fit-page)
 ;; automatically annotate highlights
 (setq pdf-annot-activate-created-annotations t)
 ;; use isearch instead of swiper
 (define-key pdf-view-mode-map (kbd "C-s") 'isearch-forward)
 ;; turn off cua so copy works
 (add-hook 'pdf-view-mode-hook (lambda () (cua-mode 0)))
 ;; more fine-grained zooming
 (setq pdf-view-resize-factor 1.1)
 ;; keyboard shortcuts
 (define-key pdf-view-mode-map (kbd "h") 'pdf-annot-add-highlight-markup-annotation)
 (define-key pdf-view-mode-map (kbd "t") 'pdf-annot-add-text-annotation)
 (define-key pdf-view-mode-map (kbd "D") 'pdf-annot-delete)
 ;; wait until map is available
 (with-eval-after-load "pdf-annot"
   (define-key pdf-annot-edit-contents-minor-mode-map (kbd "<return>") 'pdf-annot-edit-contents-commit)
   (define-key pdf-annot-edit-contents-minor-mode-map (kbd "<S-return>") 'newline)
   ;; save after adding comment
   (advice-add 'pdf-annot-edit-contents-commit :after 'bjm/save-buffer-no-args)))
-1:-- Even more PDF-tools tweaks (Post Ben Maughan)--L0--C0--November 14, 2017 09:44 PM

sachachua: 2017-11-13 Emacs news

Links from, /r/orgmode, /r/spacemacs, Hacker News,, YouTube, the changes to the Emacs NEWS file, and emacs-devel.

-1:-- 2017-11-13 Emacs news (Post Sacha Chua)--L0--C0--November 14, 2017 06:48 AM

Marcin Borkowski: My email workflow – an addendum

Some time ago I wrote about my email capturing workflow. After I’ve written that text, I noticed that my command does not work well when launched in message mode (as opposed to the headers view mode). I tried to debug this problem, but to no avail. Until yesterday.
-1:-- My email workflow – an addendum (Post)--L0--C0--November 12, 2017 01:07 PM

Chen Bin (redguardtoo): counsel-etags v1.3.1 is released

Counsel-etags is a generic solution for code navigation in Emacs.

It basically needs no setup. For example, run one command counsel-etags-find-tag-at-point is enough to start code navigation immediately.

The package solves all the problems using Ctags/Etags with Emacs.

Problem 1: Ctags takes a few seconds to update the tags file (the index file to lookup tags). The updating process blocks the user's further interaction. This problem is solved by the virtual updating function from counsel-etags. One line setup in ~/.emacs is enough:

(add-hook 'after-save-hook 'counsel-etags-virtual-update-tags)

Problem 2: Tag lookup may fail if the latest code is not scanned yet. This problem is solved by running counsel-etags-grep automatically if counsel-etags-find-tag-at-point fails. So users always get results.

There are also other enhancements.

Enhancement 1: Levenshtein Distance algorithm is used to place the better matching candidates at the the top. For example, a function named renderTable could be defined all around in a ReactJS project. But it's very possible the user prefers the definition in same component or same folder where she triggers code navigation.

Enhancement 2: It's inefficient to search the same tag again and again. counsel-etags-recent-tag is used to jump to previous definitions.

Enhancement 3: Ivy-mode provides filter UI for counsel-etags. Its means all the functionalities from Ivy is also available. For example, users can input "!keyword1" to exclude candidates matching "keyword1".

Enhancement 4: counsel-etags-grep uses the fastest grep program ripgrep if it's installed. Or else it falls back to standard grep.

Please check for more tips.

-1:-- counsel-etags v1.3.1 is released (Post Chen Bin)--L0--C0--November 12, 2017 09:40 AM

Benjamin Slade: More pdf-tools tricks

Following a couple of recent postings on pdf-tools, here are a few customisations I've found handy.

For scanned pdfs, 'pdf-view-auto-slice-minor-mode can be useful to turn on. You might bind it to something like s a. It auto trims the borders for each page of the pdf as it encounters them.

The following sets up a variety of colour-filter modes (good for night-time viewing, or anytime really that you don't want your eyeballs blasted with blazing white light):

;; midnite mode hook
(add-hook 'pdf-view-mode-hook (lambda ()
(pdf-view-midnight-minor-mode))) ; automatically turns on midnight-mode for pdfs

(setq pdf-view-midnight-colors '("#ff9900" . "#0a0a12" )) ; set the amber profile as default (see below)

(defun bms/pdf-no-filter ()
"View pdf without colour filter."
(pdf-view-midnight-minor-mode -1)

;; change midnite mode colours functions
(defun bms/pdf-midnite-original ()
"Set pdf-view-midnight-colors to original colours."
(setq pdf-view-midnight-colors '("#839496" . "#002b36" )) ; original values

(defun bms/pdf-midnite-amber ()
"Set pdf-view-midnight-colors to amber on dark slate blue."
(setq pdf-view-midnight-colors '("#ff9900" . "#0a0a12" )) ; amber

(defun bms/pdf-midnite-green ()
"Set pdf-view-midnight-colors to green on black."
(setq pdf-view-midnight-colors '("#00B800" . "#000000" )) ; green

(defun bms/pdf-midnite-colour-schemes ()
"Midnight mode colour schemes bound to keys"
(local-set-key (kbd "!") (quote bms/pdf-no-filter))
(local-set-key (kbd "@") (quote bms/pdf-midnite-amber))
(local-set-key (kbd "#") (quote bms/pdf-midnite-green))
(local-set-key (kbd "$") (quote bms/pdf-midnite-original))

(add-hook 'pdf-view-mode-hook 'bms/pdf-midnite-colour-schemes)

This automatically sets pdf-tools to display using the midnight mode amber filter.
You can return to the original/no-filter with "!" (i.e. S-1); set amber filter with "@" (i.e. S-2); set green filter with "#" (i.e. S-3); set the bluish original midnight mode colours with "$" (i.e. S-4). See below for screenshots of these different settings.

You also probably want to set the pdf-annot-default-markup-annotation-properties color to "#ff0000", as well as the pdf-annot-default-text-annotation-properties color to "#ff0000". This is a colour that is actually visible in the midnight-modes. If you go with the default yellow for markup it will not be easily visible in any of these colour-filter modes.
-1:-- More pdf-tools tricks (Post be_slayed ( 11, 2017 10:49 PM

Benjamin Slade: Take Elfeed everywhere: Mobile rss reading Emacs-style (for free/cheap)

After the demise of Google Reader, I switched for some time to NewsBlur. Unfortunately, its Android app never worked well for me. And integrating more of my life into Emacs is always desirable, so once I saw there was an Android interface for the fantastic Emacs RSS reader Elfeed, I made the switch.

The tricky thing with the Elfeed Android client is that it wants to connect to web interface of an instance of Elfeed running inside of Emacs. I could have done with my home computer, but that would require poking a hole through the firewall and in any case would be non-ideal when for instance I was travelling.

About a month ago I hit upon a cheap (in fact, free) solution for running a remote instance of Emacs running Elfeed that is connectable with the Android app. The VPS provider Wishosting offers an OpenVZ mini for $4/year, and if you stick a link to Wishosting on your own domain you can get it for free.

On my home desktop, work desktop, and laptop, I have Syncthing installed and I use this to keep the Elfeed database in sync between these machines.

In this blogpost I outline how to add a remote always-on instance of Elfeed running in Wishosting’s OpenVZ mini which also remains in sync with all of the other machines. (I use elfeed-org to organise my feeds, and just keep the file in ~/.elfeed/.) Just use Syncthing to keep the ~/.elfeed directory sync’ed between all of the machines (including any VPS’s).

I set up an Ubuntu 16.04 LTS VPS on Wishosting, and then installed Emacs and Syncthing, and that is what I would recommend.

  • Create your Ubuntu 16.04 LTS VPS.
  • Make a (non-root) user with superuser capability. Login with this user.
  • Install Emacs and Syncthing.
  • Configure Syncthing appropriately. You can connect remotely to the Web GUI.
  • Create and save a .emacs file in your user’s ~ with the following contents:
    ;; package setup here
    (require 'package)

    (package-initialize nil)
    (setq package-enable-at-startup nil)

    (add-to-list 'package-archives '("org" . "") t)

    (add-to-list 'package-archives
    '("melpa" . "") t)

    (add-to-list 'package-archives
    '("marmalade" .


    ;; general add packages to list
    (let ((default-directory "~/.emacs.d/elpa/"))

    ;; make sure 'use-package is installed
    (unless (package-installed-p 'use-package)
    (package-install 'use-package))

    ;;; use-package
    (require 'use-package)

    ;; Load elfeed
    (use-package elfeed
    :ensure t
    :bind (:map elfeed-search-mode-map
    ; ("A" . bjm/elfeed-show-all)
    ; ("E" . bjm/elfeed-show-emacs)
    ; ("D" . bjm/elfeed-show-daily)
    ("q" . bjm/elfeed-save-db-and-bury)))

    (require 'elfeed)

    ;; Load elfeed-org
    (use-package elfeed-org
    :ensure t
    (setq rmh-elfeed-org-files (list "~/.elfeed/"))

    ;; Laod elfeed-goodies
    (use-package elfeed-goodies
    :ensure t


    ;; Load elfeed-web
    (use-package elfeed-web
    :ensure t

    ;;; Elfeed
    (global-set-key (kbd "C-x w") 'bjm/elfeed-load-db-and-open)

    (define-key elfeed-show-mode-map (kbd "j") 'elfeed-goodies/split-show-next)
    (define-key elfeed-show-mode-map (kbd "k") 'elfeed-goodies/split-show-prev)
    (define-key elfeed-search-mode-map (kbd "j") 'next-line)
    (define-key elfeed-search-mode-map (kbd "k") 'previous-line)
    (define-key elfeed-show-mode-map (kbd "S-SPC") 'scroll-down-command)

    ;;write to disk when quiting
    (defun bjm/elfeed-save-db-and-bury ()
    "Wrapper to save the elfeed db to disk before burying buffer"

    ;;functions to support syncing .elfeed between machines
    ;;makes sure elfeed reads index from disk before launching
    (defun bjm/elfeed-load-db-and-open ()
    "Wrapper to load the elfeed db from disk before opening"

    (defun bjm/elfeed-updater ()
    "Wrapper to load the elfeed db from disk before opening"

    (run-with-timer 0 (* 30 60) 'bjm/elfeed-updater)

    (setq httpd-port NNNNN) ; replace NNNNN with a port equalling your start port + 10 (or whatever)

Note that you’ll need to configure the httpd-port appropriately as per the comment in the elisp above.
  • Then create the following systemd unit at ~/.config/systemd/user/emacs.service:
Description=Emacs: the extensible, self-documenting text editor

ExecStart=/usr/bin/emacs --daemon
ExecStop=/usr/bin/emacsclient --eval "(kill-emacs)"

  • Enable and start this unit with:
$ systemctl --user enable --now emacs
  • And then make sure it runs persistently even when you’re not connected to your VPS via ssh (this tripped me up for some time):
# loginctl enable-linger USERNAME
  • Install the Elfeed Android app on your mobile, enter the app’s Settings, and put in whatever your Wishosting ip is plus the port you chose above (NNNNN), e.g., for the Elfeed web url.
  • That should be it. Now you’ll have access to your rss feed all over the world and Syncthing will ensure that all changes will be propagated to all of your Emacs instances including your VPS. These changes would include adding and deleting rss feeds and marking of posts as ’read’.
  • Note: on my desktops/laptop I use the following setup for Elfeed:
    ;; Load elfeed
    (use-package elfeed
    :ensure t
    :bind (:map elfeed-search-mode-map
    ; ("A" . bjm/elfeed-show-all)
    ; ("E" . bjm/elfeed-show-emacs)
    ; ("D" . bjm/elfeed-show-daily)
    ("q" . bjm/elfeed-save-db-and-bury)))

    (require 'elfeed)

    ;; Load elfeed-org
    (use-package elfeed-org
    :ensure t
    (setq rmh-elfeed-org-files (list "~/.elfeed/"))

    ;; Laod elfeed-goodies
    (use-package elfeed-goodies
    :ensure t


    ;; Load elfeed-web
    (use-package elfeed-web
    :ensure t

    ;;functions to support syncing .elfeed between machines
    ;;makes sure elfeed reads index from disk before launching
    (defun bjm/elfeed-load-db-and-open ()
    "Wrapper to load the elfeed db from disk before opening"

    ;;write to disk when quiting
    (defun bjm/elfeed-save-db-and-bury ()
    "Wrapper to save the elfeed db to disk before burying buffer"

    ;;; Elfeed
    (global-set-key (kbd "C-x w") 'bjm/elfeed-load-db-and-open)

    (define-key elfeed-show-mode-map (kbd ";") 'visual-fill-column-mode)

    (define-key elfeed-show-mode-map (kbd "j") 'elfeed-goodies/split-show-next)
    (define-key elfeed-show-mode-map (kbd "k") 'elfeed-goodies/split-show-prev)
    (define-key elfeed-search-mode-map (kbd "j") 'next-line)
    (define-key elfeed-search-mode-map (kbd "k") 'previous-line)
    (define-key elfeed-show-mode-map (kbd "S-SPC") 'scroll-down-command)
    ;; probably temporary: hack for elfeed-goodies date column:
    (defun elfeed-goodies/search-header-draw ()
    "Returns the string to be used as the Elfeed header."
    (if (zerop (elfeed-db-last-update))
    (let* ((separator-left (intern (format "powerline-%s-%s"
    (car powerline-default-separator-dir))))
    (separator-right (intern (format "powerline-%s-%s"
    (cdr powerline-default-separator-dir))))
    (db-time (seconds-to-time (elfeed-db-last-update)))
    (stats (-elfeed/feed-stats))
    (search-filter (cond
    (if (>= (window-width) (* (frame-width) elfeed-goodies/wide-threshold))
    (search-header/draw-wide separator-left separator-right search-filter stats db-time)
    (search-header/draw-tight separator-left separator-right search-filter stats db-time)))))

    (defun elfeed-goodies/entry-line-draw (entry)
    "Print ENTRY to the buffer."

    (let* ((title (or (elfeed-meta entry :title) (elfeed-entry-title entry) ""))
    (date (elfeed-search-format-date (elfeed-entry-date entry)))
    (title-faces (elfeed-search--faces (elfeed-entry-tags entry)))
    (feed (elfeed-entry-feed entry))
    (when feed
    (or (elfeed-meta feed :title) (elfeed-feed-title feed))))
    (tags (mapcar #'symbol-name (elfeed-entry-tags entry)))
    (tags-str (concat "[" (mapconcat 'identity tags ",") "]"))
    (title-width (- (window-width) elfeed-goodies/feed-source-column-width
    elfeed-goodies/tag-column-width 4))
    (title-column (elfeed-format-column
    title (elfeed-clamp
    (tag-column (elfeed-format-column
    tags-str (elfeed-clamp (length tags-str)
    (feed-column (elfeed-format-column
    feed-title (elfeed-clamp elfeed-goodies/feed-source-column-width

    (if (>= (window-width) (* (frame-width) elfeed-goodies/wide-threshold))
    (insert (propertize date 'face 'elfeed-search-date-face) " ")
    (insert (propertize feed-column 'face 'elfeed-search-feed-face) " ")
    (insert (propertize tag-column 'face 'elfeed-search-tag-face) " ")
    (insert (propertize title 'face title-faces 'kbd-help title)))
    (insert (propertize title 'face title-faces 'kbd-help title)))))
[Also posted at r/emacs]
-1:-- Take Elfeed everywhere: Mobile rss reading Emacs-style (for free/cheap) (Post be_slayed ( 11, 2017 10:44 PM

Manuel Uberti: Inspecting log files with Emacs

As a software developer, I cannot avoid log files. Application servers and LAMP machines are always around the corner waiting for me, so I must accept log files and do my best.

There are two kinds of log files I work with:

  • static: the server wrote the content and never touched the file again
  • dynamic: the server keeps appending content as the monitored process continues

The log files could be on my hard disk or located on a remote machine. During my workflow, a “remote machine” can also be a Docker container.

I leave the math to you, but it should be clear that different combinations arise. Luckily, with Emacs everything goes smoothly and I can move seamlessly across any scenario.

When it comes to static files there are no surprises. If the log file is local, inspecting it with counsel-grep-or-swiper is pretty much all I need. Since I rarely need to modify a log file, I usually enable read-only-mode with C-x C-q to prevent accidental edits. For a remote static file, the same easily applies thanks to TRAMP.

Things get more interesting with dynamic logs. On local files, with auto-revert-mode I can see the changes in the buffer as soon as the underline file changes. If I want something fancier, auto-revert-tail-mode behaves like the good old tail -f. Dynamic remote logs, however, require a different approach. Activating auto-revert-mode is not enough, I also need to set auto-revert-remote-files to t for the desired behaviour.

What if years of tail -f made you totally indifferent to such niceties? You can follow Michael Albinus suggestion and use dired-do-shell-command in Dired.

Actually, you can even improve a little on his tip. Leave the ending & out, and go straight with dired-do-async-shell-command which in Dired is aptly bound to &. Refer to the manual for further details: Shell Commands in Dired.

-1:-- Inspecting log files with Emacs (Post)--L0--C0--November 11, 2017 12:00 AM

Ben Simon: Calculating Distance from a Cell Phone's Accelerometer - A failed, but fun attempt

Our gym had a nifty Bluetooth enabled jump rope that would report how many jumps you made in a session. Then the rope broke. This left me thinking: could I approximate the same thing by using the sensors on my phone?

Poking around, I learned that there's a class of app that gives you access to these sensors. In that genre, Physics Toolbox Suite appears to be a top pick. Launching the app gives me access to all sorts of data, from the gyroscope, to barometric pressure, to the GPS. It also gives me a way to record the data for a period of time, generating a simple CSV file. And if all that weren't enough, it's also ad free. To show my support, I just purchased the Pro version, even though the standard version does everything I need.

What caught my eye in the app was the Linear Accelerometer screen:

I've long since forgotten the majority of high school physics lessons, but I do know this: units matter. As the app reminded me (or perhaps taught me), acceleration is measured in meters per second, squared. What jumped out at me was the presence of meters, or distance. I now had my goal: rather than count the number of jumps in a jump-rope session, wouldn't it be far cooler to count distance? That is, how many vertical meters did I manage to conquer while jumping rope. I wasn't sure how I was going to tease this data out of the accelerometer, but I knew I was going to try.

The first stop in my journey was to head over to YouTube and learn how acceleration and distance are 'connected.' I found this Khan Academy video gave me enough background on distance, velocity and acceleration to let me derive a solution. The Physics Toolbox Suite app would give me a CSV file that provide an acceleration value every 100th of a second. Thinking through these concepts, I realized that I could at each moment derive speed and then distance, by looking at the change in time and acceleration. And because I knew that I was starting by standing still (that is, my velocity was zero), the whole system could be initialized and in theory, a final answer derived. I was quite proud of myself. Shira, for her part, rattled off the exact formula and explained to me that this super basic physics. Apparently, I could have skipped the Kahn Accademy, and just talked to her.

Now that I had a method for calculating distance via the accelerometer, I needed to try it in a controlled environment. I laid down a tape measure, and noted 4 meters. I then walked these 4 meters while capturing the accelerometer data via the Physics Toolbox app. With a few test runs captured, it was time to do the fun part: program the above solution.

Because the data was on my phone, I thought it appropriate to code the solution on my phone. I busted out tinyscheme, emacs and termux. Tinyscheme is a delightful tool to work with because it's so lean. This can also be a source of frustration, as to implement the above solution I needed a way of reading form a CSV file, and I'd rather not have to implement the string and file handling. Fortunately, I was able to borrow some code from the Programming Praxis Standard Prelude (thanks!), which meant that I didn't have to implement string-split or read-line.

The following function does the heavy lift for reading and parsing the CSV file:

;; Call handler on each row in the file.
;; Assume the first row is a header and the
;; following rows are all numeric data
(define (with-data file handler init)
  (define (wrap data)
    (lambda (index)
      (case index
 ((all) data)
 (else  (list-ref data index)))))
  (call-with-input-file file
    (lambda (port)
      (let ((header-row (read-line port)))
 (let loop ((line (read-line port))
     (accum (wrap init)))
   (cond ((eof-object? line) (accum 'all))
   (loop (read-line port)
         (wrap (handler
         (wrap (map string->number
      (string-split #\, line)))))))))))))

Ah, it's such a join programming this sort of thing in Scheme. It's like programming with Play-doh: you can invent new constructs with so little effort.

With the file parsing out of the way, it was time to write the actual code to calculate distance:

;; Use Physics 101 to calculate distance
;; based on the current acceleration, time
;; and velocity
(define (calculate-distance accum data)
  (let* ((t-now (data 0))
  (a-now (+
   (data 1)  ; ax 
   (data 2)  ; ay
   (data 3)  ; az
  (t-prev (accum 0))
  (v-prev (accum 1))
  (d-prev (accum 2))
  (t (- t-now t-prev))
  (v-now (+ v-prev (* a-now t)))
  (d-now (+ d-prev (* v-now t))))
    (list t-now v-now d-now)))

There's nothing particular sexy here. I'm taking in the time and acceleration data (via data) and using my accumulated values to calculate the current velocity and distance.

It was now time for the moment of truth. I ran the code against a sample data:

And, it was a bust. My code tells me that I covered 2.54 meters instead of the 4 meters I knew I covered.

I've been through the code a number of times, and I think the algorithm is fine. My guess is that I'm expecting too much accuracy out of my phone's sensor. Though, maybe my algorithm is fine and I just made a boneheaded mistake. You can see the full source code of the project here, and the data file is here.

Still, this exercise has been valuable. I learned about the Physics Toolbox app, which is definitely a keeper. I once again saw the amazing power of Termux, emacs, tinyscheme and how effortlessly they all come together on my cell phone. And I even learned some physics along the way.

While I think my goal of measuring how 'far' I jumped may not work. I'm still not convinced I can't use the sensors on my phone to at least count jumps. The data is there, the function to process the data file is ready, I just need to plug in right algorithm to make this all make sense.

Do you see where my assumptions on the accelerometer all fell apart? Please explain the comments!

-1:-- Calculating Distance from a Cell Phone's Accelerometer - A failed, but fun attempt (Post Ben Simon ( 09, 2017 04:21 PM

Manuel Uberti: Magit, the one and only

Nowadays it is basically impossible to land in a group of Emacs enthusiasts without someone talking rapturously about the wonders of Magit. If you listen closely, the words resemble a chant: “You wish you could do it with Git, until you experience the great Magit.”

Be warned: the following story goes along those magic lines.

Magit makes me look back at my experiences with CVS, SVN and Mercurial and laugh about my clumsiness with tools like Tortoise. Before Magit, I touched the regular Git command-line interface for simple tasks such as cloning a repository, always scared away by the large number of options at my disposal. I had to deal with Eclipse’s own Git client when I was working with Java, but please let me spare you the gruesome details.

Magit has never been so necessary and useful as it has been since I started my new job back in April. Git is our only version-control sytem, and the command-line is still the preferred tools in the office. Not for me, though. Magit has become a natural extension of pretty much all of my coding.

I regularly create branches, being it for an issue to fix or a new feature to implement. If the project is based on gitflow, we are covered in Magit thanks to magit-gitflow. Otherwise, rebasing and merging on master are just a few key bindings away.

Although impressive, it is not only a matter of speed. Magit taught me more about Git than every tutorial out in the wild. Especially when it comes to rebasing and cherry-picking, both of which are hard to get wrong now. Rewording a commit, exploring the repository history, dealing with submodules: it all makes sense through an interface that could not be more helpful.

The one thing against Magit I have for the haters is this:

(unbind-key "C-c C-w" git-commit-mode-map)

That binding is my prefix for Eyebrowse. Not much of a complaint, I know.

Magit brings order, balance and elegance to the powers of Git. It is the only version-control system I need.

-1:-- Magit, the one and only (Post)--L0--C0--November 09, 2017 12:00 AM

Pragmatic Emacs: More PDF-tools tweaks

I’ve added a couple more tweaks to my pdf-tools configuration.

First, I’ve found that CUA mode interferes with the ability to copy text from the pdf, so let’s turn it off in pdf-view-mode

;; turn off cua so copy works
(add-hook 'pdf-view-mode-hook (lambda () (cua-mode 0)))

Next, I want more fine grained zooming with + and - than the default 25%, so I’ll set it to 10%

;; more fine-grained zooming
(setq pdf-view-resize-factor 1.1)

Finally, for my most commonly used annotation tools (adding a highlight, adding a text note and deleting an annotation), I want quicker shortcuts. With the following I just hit h, t or D respectively for those tools (instead of e.g. C-c C-a h to highlight).

;; keyboard shortcuts
(define-key pdf-view-mode-map (kbd "h") 'pdf-annot-add-highlight-markup-annotation)
(define-key pdf-view-mode-map (kbd "t") 'pdf-annot-add-text-annotation)
(define-key pdf-view-mode-map (kbd "D") 'pdf-annot-delete)

Putting it all together, my current setup looks like this:

(use-package pdf-tools
 :pin manual ;; manually update
 ;; initialise
 ;; open pdfs scaled to fit page
 (setq-default pdf-view-display-size 'fit-page)
 ;; automatically annotate highlights
 (setq pdf-annot-activate-created-annotations t)
 ;; use normal isearch
 (define-key pdf-view-mode-map (kbd "C-s") 'isearch-forward)
 ;; turn off cua so copy works
 (add-hook 'pdf-view-mode-hook (lambda () (cua-mode 0)))
 ;; more fine-grained zooming
 (setq pdf-view-resize-factor 1.1)
 ;; keyboard shortcuts
 (define-key pdf-view-mode-map (kbd "h") 'pdf-annot-add-highlight-markup-annotation)
 (define-key pdf-view-mode-map (kbd "t") 'pdf-annot-add-text-annotation)
 (define-key pdf-view-mode-map (kbd "D") 'pdf-annot-delete))
-1:-- More PDF-tools tweaks (Post Ben Maughan)--L0--C0--November 08, 2017 09:22 PM

Raimon Grau: Finally publishing commit-msg-prefix.el

It's been a few months since I released commit-msg-prefix.
Just to remember a bit what was it about, I created a gif so that we all can relate again to it.

The user story is the following:

  • In your company/organisation, you have to follow some rule of prefixing all your commits with either a keyword (Add/Remove/Fix/Hotfix/Bump/Release) or an issue number, or something like that.
  • When you are about to commit that, you know what you just did (you should!), but you can't remember which particular issue number was that.
  • Then you go to jira/trello/github-issues/younameit and look for that and insert it.  
Usually, as it's been a long running issue, you have commits with that number, in your recent commit history, so an alternative to the last bullet point is to have a quick look at git log.

Well, commit-msg-prefix makes this automatically for you, so it shows the latest commits (filtered as you want), and lets you choose one (find-as-you-type provided by helm/ivy/ido), and inserts the first word of the commit.

Obviously, most of the relevant settings are configurable, so you can have support for fancier substitutions, or use mercurial logs.

The package itself is pretty simple, but it might be useful to others, and if it is, I'm happy to share it.  

I just issued the Pull Request for it to be in melpa, so more users can give it a try and I can test it a bit further. I hope it will be accepted  and will be available in melpa soon.

In the future, if it has some adoption, it might make sense to merge it into magit, but I don't want to bother magintainers when the idea is not widely tested yet.

Happy hacking!
-1:-- Finally publishing commit-msg-prefix.el (Post Raimon Grau ( 08, 2017 04:15 PM

William Denton: Org clocktables I: The daily structure


Recently I started tracking my time at work using Org’s clocking feature, and it’s working out very well. The actual act of tracking my time makes me much more focused, and I’m wasting less time and working more on important, planned, relevant things. It’s also helping me understand how much time I spend on each of the three main pillars of my work (librarianship + research and professional development + service). In order to understand all this well I wrote some code to turn the Org’s clocktable into something more usable. This is the first of two or three posts showing what I have.

Three pillars

In York University Libraries, where I work, librarians and archivists have academic status. We are not faculty (that’s the professors), but we’re very similar. We’re in the same union. We have academic freedom (important). We get “continuing appointment,” not tenure, but the process is much the same.

University professors have three pillars to their work: teaching, research and service. Service is basically work that contributes to the running of the university: serving on committees (universities have lots of committees, and they do important work, like vetting new courses and programs, allocating research funds, or deciding who gets tenure), being on academic governance bodies such as faculty councils and Senate, having a position in the union, etc. Usually there’s a 40/40/20 ratio on these three areas: people spend about 40% of their time on teaching, 40% on research and 20% on service. This fluctuates term to term and year to year—and person to person—but that’s the general rule in most North American universities, as I understand it.

Waiting for Sir Simon Rattle and the Berlin Philharmonic to enter Roy Thomson Hall, last November. Waiting for Sir Simon Rattle and the Berlin Philharmonic to enter Roy Thomson Hall, last November.

For librarians and archivists the situation can be different. Instead of teaching, let’s say we do “librarianship” as a catch-all term. (Or “archivy,” which the archivists assure me is a real word, but I still think it looks funny.) Then we also do professional development/research and service. In some places, like Laurentian, librarians have full parity with professors, and they have the 40/40/20 ratio. That is ideal. A regrettable example is Western, where librarians and archivists have to spend 75% of their time on professional work. That severely limits the contributions they can make both to the university and to librarianship and scholarship in general.

At York there is no defined ratio. For professors it’s understood to be the 40/40/20, but for librarians and archivists I think it’s understood that is not our ratio, but nothing is set out instead. (This, and that profs have a 2.5 annual course teaching load but we do not have an equivalent “librarianship load,” leads to problems.)

I have an idea of what the ratio should be, but I’m not going to say it here because this may become a bargaining issue. I didn’t know if my work matched that ratio because I don’t have exact details about how I spend my time. I’ve been doing a lot of service, but how much? How much of my time is spent on research?

This question didn’t come to me on my own. A colleague started tracking her time a couple of months ago, jotting things down each day. She said she hadn’t realized just how much damned time it takes to schedule things. I was inspired by her to start clocking my own time.

This is where I got to apply an aspect of Org I’d read about but never used. Org is amazing!

Work diary

I keep a file,, where I put notes on everything I do. I changed how I use subheadings and now I give every day this structure:

* 2017-12 December

** [2017-12-01 Fri]

*** PPK

*** PCS

*** Service

“PPK” is “professional performance and knowledge,” which is our official term for “librarianship” or “archivy.” “PCS” is “professional contribution and standing,” which is the umbrella term for research and more for faculty. Right now for us that pillar is called “professional development,” but that’s forty-year-old terminology we’re trying to change, so I use the term faculty use. (Check the T&P criteria for a full explanation.)

On my screen, because of my Emacs configuration, that looks like this:

Initial structure. Initial structure.

Clocking in

First thing in the morning, I create that structure, then under the date heading I run C-u C-u C-c C-x C-i (where C-c means Ctrl-c). Now, I realize that’s a completely ridiculous key combination to exist, but when you start using Emacs heavily, you get used to such incantations and they become second nature. C-c C-x C-i is the command org-clock-in. As the docs say, “With two C-u C-u prefixes, clock into the task at point and mark it as the default task; the default task will then always be available with letter d when selecting a clocking task.” That will make more sense in a minute.

When I run that command, Org adds a little block under the heading:

** [2017-12-01 Fri]
CLOCK: [2017-12-01 Fri 09:30]

*** PPK

*** PCS

*** Service

The clock is running, and a little timer shows up in my mode line that tells me how long I’ve been working on the current thing.

I’ll spend a while deleting email and checking some web sites, then let’s say I decide to respond to an email about reference desk statistics, because I can get it done before I have to head over to a 10:30 meeting. I make a new subheading under PPK, because this is librarianship work, and clock into it with C-c C-x C-i. The currently open task gets closed, the duration is noted, and a new clock starts.

** [2017-12-01 Fri]
CLOCK: [2017-12-01 Fri 09:30]--[2017-12-01 Fri 09:50] =>  0:20

*** PPK

**** Libstats stuff
CLOCK: [2017-12-01 Fri 09:50]

Pull numbers on weekend desk activity for A.

*** PCS

*** Service

(Remember this doesn’t look ugly the way I see it in Emacs. There’s another screenshot below.)

I work on that until 10:15, then I make a new task (under Service) and check into it (again with C-c C-x C-i). I’m going to a monthly meeting of the union’s stewards’ council, and walking to the meeting and back counts as part of the time spent. (York’s campus is pretty big.)

* [2017-12-01 Fri]
CLOCK: [2017-12-01 Fri 09:30]--[2017-12-01 Fri 09:50] =>  0:20

*** PPK

**** Libstats stuff
CLOCK: [2017-12-01 Fri 09:50]--[2017-12-01 Fri 10:15] =>  0;25

Pull numbers on weekend desk activity for A.

*** PCS

*** Service

**** Stewards' Council meeting
CLOCK: [2017-12-01 Fri 10:15]

The meeting ends at 1, and I head back to my office. Lunch was provided during the meeting (probably pizza or extremely bready sandwiches, but always union-made), so I don’t take a break for that. In my office I’m not ready to immediately settle into a task, so I hit C-u C-c C-x C-i (just the one prefix), which lets me “select the task from a list of recently clocked tasks.” This is where the d mentioned above comes in: a little list of recent tasks pops up, and I can just hit d to clock into the [2017-12-01 Fri] task.

** [2017-12-01 Fri]
CLOCK: [2017-12-01 Fri 09:30]--[2017-12-01 Fri 09:50] =>  0:20
CLOCK: [2017-12-01 Fri 13:15]

*** PPK

**** Libstats stuff
CLOCK: [2017-12-01 Fri 09:50]--[2017-12-01 Fri 10:15] =>  0:25

Pull numbers on weekend desk activity for A.

*** PCS

*** Service

**** Stewards' Council meeting
CLOCK: [2017-12-01 Fri 10:15]--[2017-12-01 Fri 13:15] =>  3:00

Copious meeting notes here.

Now I might get a cup of tea if I didn’t pick one up on the way, or check email or chat with someone about something. My time for the day is accruing, but not against any specific task. Then, let’s say it’s a focused day, and I settle in and work until 4:30 on a project about ebook usage. I clock in to that, then when I’m ready to leave I clock out of it with C-c C-x C-o.

** [2017-12-01 Fri]
CLOCK: [2017-12-01 Fri 09:30]--[2017-12-01 Fri 09:50] =>  0:20
CLOCK: [2017-12-01 Fri 13:15]--[2017-12-01 Fri 13:40] =>  0:25

*** PPK

**** Libstats stuff
CLOCK: [2017-12-01 Fri 09:50]--[2017-12-01 Fri 10:15] =>  0:25

Pull numbers on weekend desk activity for A.

**** Ebook usage
CLOCK: [2017-12-01 Fri 13:40]--[2017-12-01 Fri 16:30] =>  2:50

Wrote code to grok EZProxy logs and look up ISBNs of Scholars Portal ebooks.

*** PCS

*** Service

**** Stewards' Council meeting
CLOCK: [2017-12-01 Fri 10:15]--[2017-12-01 Fri 13:15] =>  3:00

Copious meeting notes here.

In Emacs, this looks much more appealing.

Final structure at end of day. Final structure at end of day.

That’s one day of time clocked. In my next post I’ll add another day and a clocktable, and then I’ll show the code I use to summarize it all into tidy data.


I’m doing all this for my own use, to help me be as effective and efficient and aware of my work habits as I can be. I want to spend as much of my time as I can working on the most important work. Sometimes that’s writing code, sometimes that’s doing union work, sometimes that’s chatting with a colleague about something that’s a minor thing to me but that takes an hour because it’s important to them, and sometimes that’s watching a student cry in my office and waiting for the moment when I can tell them that as stressful as things are right now it’s going to get better. (Women librarians get much, much more of this than men do, but I still get some. It’s a damned tough thing, doing a university degree.) I recommend my colleague Lisa Sloniowski’s award-winning article Affective Labor, Resistance, and the Academic Librarian (Library Trends Vol. 64, No. 4, 2016) for a serious look at all this.

-1:-- Org clocktables I: The daily structure (Post William Denton)--L0--C0--November 07, 2017 01:42 AM

Marcin Borkowski: Keyboard macros and asynchronicity

Some time ago I wanted to send a bunch of emails from mu4e. I had all the data in a csv file, so I figured out that I’ll just record a keyboard macro of myself sending the email and press f4 a few times. Well, it didn’t work.
-1:-- Keyboard macros and asynchronicity (Post)--L0--C0--November 06, 2017 06:01 PM

sachachua: 2017-11-06 Emacs news

Links from, /r/orgmode, /r/spacemacs, Hacker News,, YouTube, the changes to the Emacs NEWS file, and emacs-devel.

-1:-- 2017-11-06 Emacs news (Post Sacha Chua)--L0--C0--November 06, 2017 08:50 AM

Matthias Pfeifer: A maven plugin for Emacs

Introduction I am writing Java code with Emacs from time to time and normally miss a lot of the „sugar“ that „the other IDE I am using“ provides me with. From time to time I also stretch my fingers and try to improve my Emacs experience with some tooling. This resulted recently in a short […]
-1:-- A maven plugin for Emacs (Post Matthias)--L0--C0--November 02, 2017 08:45 PM

emacshorrors: bytecomp.el

It’s halloween, so here’s a real treat for you, the commentary in bytecomp.el! The author of that piece of code is Jamie Zawinski who did invaluable work for both GNU Emacs and XEmacs, these days he runs a night club and blogs. Here are my favorite parts of the file:

  • ";; We successfully didn't compile this file."
  • (insert "\n") ; aaah, unix.
  •             (when old-style-backquotes
                  (byte-compile-warn "!! The file uses old-style backquotes !!
    This functionality has been obsolete for more than 10 years already
    and will be removed soon.  See (elisp)Backquote in the manual."))
  • ;; Insert semicolons as ballast, so that byte-compile-fix-header
    ;; can delete them so as to keep the buffer positions
    ;; constant for the actual compiled code.
  • ;; To avoid consing up monstrously large forms at load time, we split
    ;; the output regularly.
  • ;; If things not being bound at all is ok, so must them being
    ;; obsolete.  Note that we add to the existing lists since Tramp
    ;; (ab)uses this feature.
  • ;; If foo.el declares `toto' as obsolete, it is likely that foo.el will
    ;; actually use `toto' in order for this obsolete variable to still work
    ;; correctly, so paradoxically, while byte-compiling foo.el, the presence
    ;; of a make-obsolete-variable call for `toto' is an indication that `toto'
    ;; should not trigger obsolete-warnings in foo.el.
  • ;; FIXME: we also use this hunk-handler to implement the function's dynamic
    ;; docstring feature.  We could actually implement it more elegantly in
    ;; byte-compile-lambda so it applies to all lambdas, but the problem is that
    ;; the resulting .elc format will not be recognized by make-docfile, so
    ;; either we stop using DOC for the docstrings of preloaded elc files (at the
    ;; cost of around 24KB on 32bit hosts, double on 64bit hosts) or we need to
    ;; build DOC in a more clever way (e.g. handle anonymous elements).
  • ;; Don't reload the source version of the files below
    ;; because that causes subsequent byte-compilation to
    ;; be a lot slower and need a higher max-lisp-eval-depth,
    ;; so it can cause recompilation to fail.
  • ;; To avoid "lisp nesting exceeds max-lisp-eval-depth" when bytecomp compiles
    ;; itself, compile some of its most used recursive functions (at load time).

Don’t get me wrong, I’m aware that these are all necessary and don’t indicate deeper faults in the source code. I merely find it interesting what hacks one has to come up with for byte-code compilation and found studying the file enlightening to say the least.

-1:-- bytecomp.el (Post Vasilij Schneidermann)--L0--C0--October 31, 2017 07:44 AM

Chen Bin (redguardtoo): Auto complete everything in Emacs

complete everything in Emacs :en:emacs:

As a web developer using modern front end framework like React/Angular, I spend a lot of time on web components.

A component instance is like:

  onSelectRow={ row => console.log(row) }
  onClickCell={ cell => console.log(cell) }
  <PaginationButtons />
  <TotalSum />
  <ReportButtons />

Basically a component instance is a big chunk of html tags.

I created a new package EACL (Emacs auto complete lines) which could help me input components in unbelievable speed.

The idea is simple. If I've already used one component elsewhere in the project. It's unnecessary to re-type the similar code again.

All I need to do is to input the first characters of the component and run M-x eacl-complete-tag which will grep the project and input the remaining part of component.

Here is a demo to input component ButtonToolbar:


Please note EACL is generic and can be use in any programming language.

M-x eacl-complete-statement to complete below Javascript code:

import {
} from 'react-bootstrap';

M-x eacl-complete-snippet to complete below C code:

static int v9fs_drop_inode(struct inode *inode)
    struct v9fs_session_info *v9ses;
    v9ses = v9fs_inode2v9ses(inode);
    if (v9ses->cache == CACHE_LOOSE || v9ses->cache == CACHE_FSCACHE)
        return generic_drop_inode(inode);
     * in case of non cached mode always drop the
     * the inode because we want the inode attribute
     * to always match that on the server.
    return 1;

You can also create your own commands based on API eacl-complete-multi-lines-internal.

For example, it is a piece of cake to support Lisp by creating comand my-complete-lisp:

(require 'eacl)
(defun my-complete-lisp ()
  (eacl-complete-multi-lines-internal "[^)]*)"))
-1:-- Auto complete everything in Emacs (Post Chen Bin)--L0--C0--October 30, 2017 11:13 AM

Modern Emacs: Deep diving into a major mode - Part 2 (IDE Features)

In Part 1 I addressed: syntax tables, indentation, font locking, and context-sensitive syntax.

We now build out a "Hyde" with: shell/async process integration, Eldoc support, and Autocompletion. Also "shift-K" documentation lookup for the symbol-at-point.


Building Inferior Hy

The basics

The easiest way to add a REPL to your major-mode is through setting the variable inferior-lisp-program, possibly adding setup code through inferior-lisp-load-command, then running the inferior-lisp command.

;; within the define-derived-mode hy-mode setup...
(setq-local inferior-lisp-program "hy")
(setq-local inferior-lisp-load-command "(print \"Hy there!\")")

This isn't sufficient for advanced shells. The custom is to create a function named run-hy (eg. there is run-python, run-haskell...) which starts up the shell, sets inferior-hy-mode , and switches to the shell.


It is a "Major mode for interacting with an inferior interpreter". Most-all inferior modes will derive from comint-mode. It provides many utilities for interacting with shell-like processes.

Hy mode's prompt is a right arrow.

(define-derived-mode inferior-hy-mode comint-mode "Inferior Hy"
  "Major mode for Hy inferior process."
  (setq-local indent-tabs-mode nil)

  ;; How to dispaly the process status in the mode-line
  (setq mode-line-process '(":%s"))
  ;; This disables editing and traversing the "=>" prompts
  (setq-local comint-prompt-read-only t)
  ;; Lets comint mode recognize the prompt
  (setq-local comint-prompt-regexp (rx bol "=>" space))

  ;; ... other specialized config introduced later ...

There are several comint components we will make use of:

  1. comint-last-prompt - a cons cell of begin/end markers of last prompt.
  2. comint-send-string - perform process-send-string with comint
  3. bookkeeping.
  4. comint-redirect-send-command-to-process and comint-redirect-completed - for sending strings asynchronously.
  5. comint-(pre)output-filter-functions - entry points into capturing and
  6. cleaning process output.
  7. make-comint-in-buffer - entry point into comint mode.

Managing buffers and processes

We must manage: the standard hy shell process, the internal hy process used for autocompletion and eldoc, and temporal buffers for more advanced buffer transformations of the standard hy shell process.

The configuration required of inferior-hy-mode :

;;; Configuration

(defconst hy-shell-interpreter "hy"
  "Default Hy interpreter name.")

(defvar hy-shell-interpreter-args "--spy"
  "Default arguments for Hy interpreter.")

;;; Internal

(defconst hy-shell-buffer-name "Hy"
  "Default buffer name for Hy interpreter.")

(defconst hy-shell-internal-buffer-name "Hy Internal"
  "Default buffer name for the internal Hy process.")

(defvar hy-shell-buffer nil
  "The current shell buffer for Hy.")

(defvar hy--shell-output-filter-in-progress nil
  "Whether we are waiting for output in `hy-shell-send-string-no-output'.")

(defvar hy--shell-font-lock-enable t
  "Whether the shell should font-lock the current line.")

Now the building blocks of the shell can be defined.

The implementations are rather straightforward. To keep space down, only name and docstring are provided:

(defun hy--shell-format-process-name (proc-name)
  "Format a PROC-NAME with closing astericks.")

(defun hy-shell-get-process (&optional internal)
  "Get process corr. to `hy-shell-buffer-name'/`hy-shell-internal-buffer-name'.")

(defun hy--shell-current-buffer-process ()
  "Get process associated with current buffer.")

(defun hy--shell-current-buffer-a-process? ()
  "Is `current-buffer' a live process?")

(defun hy--shell-get-or-create-buffer ()
  "Get or create `hy-shell-buffer' buffer for current hy shell process.")

(defun hy--shell-buffer? ()
  "Is `hy-shell-buffer' set and does it exist?")

(defun hy--shell-kill-buffer ()
  "Kill `hy-shell-buffer'.")

(defun hy--shell-calculate-command (&optional internal)
  "Calculate the string used to execute the inferior Hy process.")
  ;; Straightforward string formatting - see: `shell-quote-argument'

Starting up the shell

The commands above are enough to build out some basic shell support. Lets look at run-hy :

(defun run-hy (&optional cmd)
  "Run an inferior Hy process.

CMD defaults to the result of `hy--shell-calculate-command'."
  (unless (executable-find "hy")
    (message "Hy not found, activate a virtual environment with Hy."))

  (-> (or cmd (hy--shell-calculate-command))
     (hy--shell-make-comint hy-shell-buffer-name 'show)

Most of the work is delegated to hy--shell-make-comint as we also must have the internal variant:

(defun run-hy-internal ()
  "Start an inferior hy process in the background for autocompletion."
  (unless (executable-find "hy")
    (message "Hy not found, activate a virtual environment containing Hy to use
Eldoc, Anaconda, and other hy-mode features."))

  (when (and (not (hy-shell-get-process 'internal))
             (executable-find "hy"))
    (-let [hy--shell-font-lock-enable
          (-> (hy--shell-calculate-command 'internal)
             (hy--shell-make-comint hy-shell-internal-buffer-name nil 'internal)
        (message "Hy internal process successfully started")))))

This is a simple variation of run-hy that passes the internal argument through the hy shell building blocks and also sends setup code for eldoc-mode and company-mode.

Now we are ready to startup our inferior-hy-mode :

(defun hy--shell-make-comint (cmd proc-name &optional show internal)
  "Create and return comint process PROC-NAME with CMD, opt. INTERNAL and SHOW."
  (-when-let* ((proc-buffer-name
                (hy--shell-format-process-name proc-name))
                (not (comint-check-proc proc-buffer-name)))
                (split-string-and-unquote cmd))
                (apply 'make-comint-in-buffer proc-name proc-buffer-name
                       (car cmdlist) nil (cdr cmdlist)))
                (get-buffer-process buffer)))
    (with-current-buffer buffer
    (when show
      (display-buffer buffer))
    (if internal
        (set-process-query-on-exit-flag process nil)
      (setq hy-shell-buffer buffer))

All the work is once again delegated to our shell building blocks. There are several things to notice:

  1. If the process is meant to be autostarted/quited, make sure to use set-process-query-on-exit-flag to nil.
  2. The cmdlist car is "hy" and cdr is the hy interpreter arguments.
  3. Further accessing of the shell is done with the hy-shell-buffer variable.

Working with the shell

The shell is now functional, but we still don't have methods to send strings to the shell (for instance, sending the current-form or the buffer for evaluation). Nor do we have any support for asynchronously sending and extracting information from our internal process.

Like always, lets define some utilities:

(defun hy--shell-end-of-output? (string)
  "Return non-nil if STRING ends with the prompt."
  (s-matches? comint-prompt-regexp string))

(defun hy--shell-output-filter (string)
  "If STRING ends with input prompt then set filter in progress done."
  (when (hy--shell-end-of-output? string)
    (setq hy--shell-output-filter-in-progress nil))
  "\n=> ")

hy--shell-output-filter-in-progress is the critical component. Lets see how it is used:

(defun hy--shell-send-string (string &optional process internal)
  "Internal implementation of shell send string functionality."
  (let ((process (or process (hy-shell-get-process internal)))
        (hy--shell-output-filter-in-progress t))
    (comint-send-string process string)
    (while hy--shell-output-filter-in-progress
      (accept-process-output process))))

The shell process is obtained, we set it to be in progress, and send it off to comint. But how and when is the filter reset?

We come back to the comint-(pre)output-filter-functions. When we send the string via comint-send-string, part of its bookkeeping is to apply these filter functions to the output. However, the output can come in chunks, so simply accepting the process output is not sufficient. We must recognize when the last of the expected process output is retrieved and signal to stop accepting output.

The difference between the pre and standard filters is when they are applied. The pre variation is executed before the process output is inserted into the buffer.

Looking back to hy--shell-output-filter, what are the outcomes of using it as a pre or standard filter? Lets look at the exposed send strings:

(defun hy-shell-send-string-no-output (string &optional process internal)
  "Send STRING to hy PROCESS and inhibit printing output."
  (-let [comint-preoutput-filter-functions
    (hy--shell-send-string string process internal)))

(defun hy-shell-send-string (string &optional process)
  "Send STRING to hy PROCESS."
  (-let [comint-output-filter-functions
    (hy--shell-send-string string process)))

Lastly, these functions won't work for asynchronous ops like Eldoc and Autocompletion. You will see a Blocking call inhibiting process output error messaged in the minibuffer.

The asynchronous version is different. We redirect the process output to a temporary buffer and capture its output.

The key is the 100ms timeout argument passed to accept-process-output.

(defun hy--shell-send-async (string)
  "Send STRING to internal hy process asynchronously."
  (let ((output-buffer " *Comint Hy Redirect Work Buffer*")
        (proc (hy-shell-get-process 'internal)))
    (with-current-buffer (get-buffer-create output-buffer)
      (comint-redirect-send-command-to-process string output-buffer proc nil t)

      (set-buffer (process-buffer proc))
      (while (and (null comint-redirect-completed)
                  (accept-process-output proc nil 100 t)))
      (set-buffer output-buffer)

Our shell is now ready for autocompletion, eldoc, and other awesome IDE features.

I originally planned to go into font-locking the prompt input (highly non-trivial), but given the length of this post I will provide and link to it as a separate future post.


All the work for autocompletion was in setting up the asynchronous process support. Lets see how easy autocompletion becomes:

(defconst hy-company-setup-code
  "(import [hy.completer [Completer]])
(setv --HYCOMPANY (Completer))"
  "Autocompletion setup code to send to the internal process.")

(defconst hy--company-regexp
  (rx "'"
      (group (1+ (not (any ",]"))))
      (any "," "]"))
  "Regex to extra candidates from --HYCOMPANY.")

(defun hy--company-format-str (string)
  "Format STRING to send to hy for completion candidates."
  (when string
    (format "(.%s --HYCOMPANY \"%s\")"
            (cond ((s-starts-with? "#" string)  ; Tag matches broken in Hy atm
                  ((s-contains? "." string)

(defun hy--company-candidates (string)
  "Get candidates for completion of STRING."
  (-when-let* ((command (hy--company-format-str string))
               (candidates (hy--shell-send-async command))
               (matches (s-match-strings-all hy--company-regexp candidates)))
    (-select-column 1 matches)))  ; Get match-data-1 for each match

(defun company-hy (command &optional arg &rest ignored)
  (interactive (list 'interactive))
  (cl-case command
    (prefix (company-grab-symbol))
    (candidates (hy--company-candidates arg))
    (meta (-> arg hy--eldoc-get-docs hy--str-or-empty))))

Completer is a hy builtin that completes a given string and does all the work here. We simply call it's appropriate method, extract the items in the retrieved list, and hand it off to company.

company-grab-symbol gets the current symbol which is handed off as arg in the subsequent call.

The meta argument shows the eldoc output for the current selected company candidate in the minibuffer, as seen in this post's initial image.

company-hy can then be enabled either through adding to company-backends or for Spacemacs users adding:

;; Technically this should be within a hy layer, but this still works uncaptured
  :backends company-hy
  :modes hy-mode inferior-hy-mode)

Developing a major-mode that accommodates Spacemacs users will be touched on in future posts.


For those unfamiliar, in the initial image eldoc-mode provides the formatted docstring and arguments in the minibuffer for the symbol-at-point (or completion candidate).

Perhaps surprisingly, Eldoc is a lot more challenging than autocompletion.

For starters, your language won't provide Eldoc like strings (formatted argument list + first line of docstring) by default. How difficult inspecting language constructs is entirely dependent on the language. Hy in particular is difficult due to how macros are implemented and namespaced. I won't provide the hy-eldoc-setup-code here, it can be found within the source.

Next, your implementation must mirror any relevant DSLs. For lisps, Eldoc inspects the form opener. It is hydiomatic to:

(setv x "hi")
(.format "{} there" x)
(setv a-list [])
(.append a-list "friend")

We need to send str.format and a-list.append - the form opener alone is insufficient.


Eldoc is setup via the eldoc-documentation-function :

(defun hy-eldoc-documentation-function ()
  "Drives `eldoc-mode', retrieves eldoc msg string for inner-most symbol."
  (-> (hy--eldoc-get-inner-symbol)

(defun hy--mode-setup-eldoc ()
  (make-local-variable 'eldoc-documentation-function)
  (setq-local eldoc-documentation-function 'hy-eldoc-documentation-function)
  (eldoc-mode +1))

There are three core components:

  1. hy--eldoc-send for sending a formatted string and cleaning its output.
  2. hy--eldoc-get-inner-symbol getting opening form and completing the dot DSL.
  3. hy--eldoc-fontify-text for highlighting the final text string like in the image.

Lets look at fontifying first. We can't blindly apply Hy's font-locks as the docstring isn't captured in quotes. Since the text is static, we just add the faces to the string ourselves.

(defun hy--fontify-text (text regexp &rest faces)
  "Fontify portions of TEXT matching REGEXP with FACES."
  (when text
        (s-matched-positions-all regexp text)
      (-lambda ((beg . end))
        (--each faces
          (add-face-text-property beg end it nil text))))))

(defun hy--eldoc-fontify-text (text)
  "Fontify eldoc TEXT."
  (let ((kwd-rx
         (rx string-start (1+ (not (any space ":"))) ":"))
         (rx symbol-start "&" (1+ word)))
         (rx "`" (1+ (not space)) "`")))
     text kwd-rx 'font-lock-keyword-face)
     text kwargs-rx 'font-lock-type-face)
     text quoted-args-rx 'font-lock-constant-face 'bold-italic))

Next lets see the sending and formatting of the shell's raw eldoc output.

(defun hy--eldoc-send (string)
  "Send STRING for eldoc to internal process returning output."
  (-> string

The string/output formatting are implementation details specific to Hy and so won't be detailed. If we are dealing with an empty string, we return nil rather than the empty string to pass-by parent when clauses.

The meat of Eldoc is in extracting the innermost symbol of the current point

(defun hy--eldoc-get-inner-symbol ()
  "Traverse and inspect innermost sexp and return formatted string for eldoc."
    (-when-let* ((_ (hy-shell-get-process 'internal))
                 (state (syntax-ppss))
                 (start-pos (hy--sexp-inermost-char state))
                 (_ (progn (goto-char start-pos)
                           (not (hy--not-function-form-p))))
                 (function (progn (forward-char)
                                  (thing-at-point 'symbol))))

      ;; Attribute method call (eg. ".format str") needs following sexp
      (if (s-starts-with? "." function)
          (when (ignore-errors (forward-sexp) (forward-char) t)
            (pcase (char-after)
              ;; Can't send just .method to eldoc
              (?\) (setq function nil))
              (?\s (setq function nil))
              (?\C-j (setq function nil))  ; newline

              ;; Dot dsl doesn't work on literals
              (?\[ (concat "list" function))
              (?\{ (concat "dict" function))
              (?\  (concat "str" function))  ; the " is deleted in blog as breaks rainbow.js

              ;; Otherwise complete the dot dsl
              (_ (progn
                   (concat (thing-at-point 'symbol) function)))))

So Eldoc's path is to call hy--eldoc-get-inner-symbol if an internal process is active, syntax-ppss indicates we are within a form, and that the innermost form is a symbol. The completed string is sent off to the internal process we've built up, the output is chomped of quote characters and the prompt and syntax errors (eg. completing "str." while we are still typing) are ignored. The result is fontified and returned by the documentation function.

Spacemacs shift-k documentation lookup

A feature of Spacemacs is typing "K" to perform spacemacs/evil-smart-doc-lookup to get the full documentation of the symbol-at-point in a separate buffer.

Using Eldoc's documentation functions, with slightly different formatting, we already have most of shift-K implemented.

We moved most of hy-eldoc-documentation-function into hy--eldoc-get-docs which distinctly accepts an optional argument for buffer-style rather than eldoc-style formatting.

We then create a mirror of the documentation function as hy--docs-for-thing-at-point. We format the text to account for newlines (newlines from process output are escaped so we must trim one backslash from each newline).

(defun hy--docs-for-thing-at-point ()
  "Mirrors `hy-eldoc-documentation-function' formatted for a buffer, not a msg."
  (-> (thing-at-point 'symbol)
     (hy--eldoc-get-docs t)

(defun hy--format-docs-for-buffer (text)
  "Format raw hydoc TEXT for inserting into hyconda buffer."
  (when text
    (-let [kwarg-newline-regexp
           (rx ","
               (1+ (not (any "," ")")))
               (group-n 1 "\\\n")
               (1+ (not (any "," ")"))))]
      (--> text
         (s-replace "\\n" "\n" it)
         (replace-regexp-in-string kwarg-newline-regexp "newline" it nil t 1)))))

It is interesting how "K" is actually called, I'm not sure if any other function operates quite the same. "K" calls the function bound to "SPC m h h".

(spacemacs/set-leader-keys-for-major-mode 'hy-mode
  "hh" 'hy-describe-thing-at-point)

Lastly we need to create, switch-to, and insert the retrieved docs as hy-describe-thing-at-point.

(defun hy-describe-thing-at-point ()
  "Implement shift-k docs lookup for `spacemacs/evil-smart-doc-lookup'."
  (-when-let* ((text (hy--docs-for-thing-at-point))
               (doc-buffer "*Hyconda*"))
    (with-current-buffer (get-buffer-create doc-buffer)
      (switch-to-buffer-other-window doc-buffer)

      (insert text)
      (goto-char (point-min))

      (insert "------\n")
      (fill-region (point) (point-max))

      ;; Eventually make hyconda-view-minor-mode, atm this is sufficient
      (local-set-key "q" 'quit-window)
      (when (fboundp 'evil-local-set-key)
        (evil-local-set-key 'normal "q" 'quit-window)))))


There are several other features worth discussion like font-locking the shell prompt input and the send-(form/region/buffer)-to-shell that are sizable enough to warrant their own posts later. With this post, the series is caught up to the current featureset. Linting would be the next big problem to attack. I'd also like to integrate ert.

Going without Company and Eldoc has helped me appreciate the value in IDEs. Hy is quickly becoming a pleasant development experience.

My only guidance was source code. I hope this series make the problem more tractable for prospective major mode authors.

-1:-- Deep diving into a major mode - Part 2 (IDE Features) (Post)--L0--C0--October 29, 2017 12:00 AM

(or emacs: Orca - new package to improve org-capture from browser


Orca is a new Emacs package, an attempt to refactor my old org-fu into something that's much more re-usable and easier to get started with.

Orca functionality


When capturing from Firefox using org-protocol (together with this addon):

  • either I refile each time I capture, which is slow
  • or my captures pile up in one place, which is messy


  1. Define rules for where links from certain websites should be captured.
  2. Allow to capture directly into the current org-mode buffer, since it's likely related to what I'm working with on.

Part 1: Rules example

Here is my example list of configurations:

Corresponding code:

(setq orca-handler-list
         "~/Dropbox/org/wiki/" "Reddit")
         "~/Dropbox/org/wiki/" "\\* Questions")
         "~/Dropbox/org/" "\\* Articles")))

Part 2: Current buffer example

For example, I'm researching how to implement something with docker. This means that I have open, along with dozens of tabs in the browser.

I can configure to capture into the current org-mode buffer with a * Tasks heading like this:

(push '(orca-handler-current-buffer "\\* Tasks") orca-handler-list)

Since has * Tasks, I just click the capture button in Firefox and follow up with an immediate C-c C-c in Emacs. The link is already in the right position, no need for an extra refile step.

Using customize with orca

You can set up the capture rules using M-x customize-group RET orca RET.

Here's a screenshot: orca-customize

As you see, the customization is a list of an arbitrary length, with each element falling into one of three categories, each backed by an Elisp function (orca-handler-current-buffer, orca-handler-file, and orca-handler-match-url). Each function takes a different number of arguments (one, two, and three, respectively) - they are all annotated by the interface.

Here's the code that describes the expected :type to customize:

(defcustom orca-handler-list
  ;; ...
  :type '(repeat
            :tag "Current buffer"
            (const orca-handler-current-buffer)
            (string :tag "Heading"))
            :tag "URL matching regex"
            (const orca-handler-match-url)
            (string :tag "URL")
            (string :tag "File")
            (string :tag "Heading"))
            :tag "Default"
            (const orca-handler-file)
            (string :tag "File")
            (string :tag "Heading")))))

You can read more about the customization types in this manual section.


I hope you enjoy orca. I've submitted it to MELPA. Hopefully, it will be available for an easy install very soon.

Org-mode is a beautiful thing, but my previous attempts to configure it were huge config files of loosely related (i.e. the only thing in common was Org-mode) stuff spanning hundreds of lines. Orca is an improvement in this respect, since it focuses on a very narrow domain. It still tries to be flexible (just like org-capture) - you can plug in your own functions into orca-handler-list. But initially, the flexibility can be constrained into the customize-group interface, to allow for a self-documenting solution that's easy to get started with. Happy hacking!

PS. Thanks to all my patrons for advancing my Patreon campaign! As of this writing, we're almost at the 25% mark with 61 contributors.

-1:-- Orca - new package to improve org-capture from browser (Post)--L0--C0--October 27, 2017 10:00 PM

Chris Wellons: Make Flet Great Again

Do you long for the days before Emacs 24.3 when flet was dynamically scoped? Well, you probably shouldn’t since there are some very good reasons lexical scope. But, still, a dynamically scoped flet is situationally really useful, particularly in unit testing. The good news is that it’s trivial to get this original behavior back without relying on deprecated functions nor third-party packages.

But first, what is flet and what does it mean for it to be dynamically scoped? The name stands for “function let” (or something to that effect). It’s a macro to bind named functions within a local scope, just as let binds variables within some local scope. It’s provided by the now-deprecated cl package.

(require 'cl)  ; deprecated!

(defun norm (x y)
  (flet ((square (v) (* v v)))
    (sqrt (+ (square x) (square y)))))

However, a gotcha here is that square is visible not just to the body of norm but also to any function called directly or indirectly from the flet body. That’s dynamic scope.

(flet ((sqrt (v) (/ v 2)))  ; close enough
  (norm 2 2))
;; -> 4

Note: This works because sqrt hasn’t (yet?) been assigned a bytecode opcode. One weakness with flet is that, due to being dynamically scoped, it is unable to define or override functions whose calls evaporate under byte compilation. For example, addition:

(defun add-with-flet ()
  (flet ((+ (&rest _) :override))
    (+ 1 2 3)))

;; -> :override

(funcall (byte-compile #'add-with-flet))
;; -> 6

Since + has its own opcode, the function call is eliminated under byte-compilation and flet can’t do its job. This is similar these same functions being unadvisable.

cl-lib and cl-flet

The cl-lib package introduced in Emacs 24.3, replacing cl, adds a namespace prefix, cl-, to all of these Common Lisp style functions. In most cases this was the only change. One exception is cl-flet, which has different semantics: It’s lexically scoped, just like in Common Lisp. Its bindings aren’t visible outside of the cl-flet body.

(require 'cl-lib)

(cl-flet ((sqrt (v) (/ v 2)))
  (norm 2 2))
;; -> 2.8284271247461903

In most cases this is what you actually want. The old flet subtly changes the environment for all functions called directly or indirectly from its body.

Besides being cleaner and less error prone, cl-flet also doesn’t have special exceptions for functions with assigned opcodes. At macro-expansion time it walks the body, taking its action before the byte-compiler can interfere.

(defun add-with-cl-flet ()
  (cl-flet ((+ (&rest _) :override))
    (+ 1 2 3)))

;; -> :override

(funcall (byte-compile #'add-with-cl-flet))
;; -> :override

In order for it to work properly, it’s essential that functions are quoted with sharp-quotes (#') so that the macro can tell the difference between functions and symbols. Just make a general habit of sharp-quoting functions.

In unit testing, temporarily overriding functions for all of Emacs is useful, so flet still has some uses. But it’s deprecated!

Unit testing with flet

Since Emacs can do anything, suppose there is an Emacs package that makes sandwiches. In this package there’s an interactive function to set the default sandwich cheese.

(defvar default-cheese 'cheddar)

(defun set-default-cheese (type)
   (let* ((options '("cheddar" "swiss" "american"))
          (input (completing-read "Cheese: " options nil t)))
     (when input
       (list (intern input)))))
  (setf default-cheese type))

Since it’s interactive, it uses completing-read to prompt the user for input. A unit test could call this function non-interactively, but perhaps we’d also like to test the interactive path. The code inside interactive occasionally gets messy and may warrant testing. It would obviously be inconvenient to prompt the user for input during testing, and it wouldn’t work at all in batch mode (-batch).

With flet we can stub out completing-read just for the unit test:

;;; -*- lexical-binding: t; -*-

(ert-deftest test-set-default-cheese ()
  ;; protect original with dynamic binding
  (let (default-cheese)
    ;; simulate user entering "american"
    (flet ((completing-read (&rest _) "american"))
      (call-interactively #'set-default-cheese)
      (should (eq 'american default-cheese)))))

Since default-cheese was defined with defvar, it will be dynamically scoped despite let normally using lexical scope in this example. Both of the side effects of the tested function — setting a global variable and prompting the user — are captured using a combination of let and flet.

Since cl-flet is lexically scoped, it cannot serve this purpose. If flet is deprecated and cl-flet can’t do the job, what’s the right way to fix it? The answer lies in generalized variables.


What’s really happening inside flet is it’s globally binding a function name to a different function, evaluating the body, and rebinding it back to the original definition when the body completes. It macro-expands to something like this:

(let ((original (symbol-function 'completing-read)))
  (setf (symbol-function 'completing-read)
        (lambda (&rest _) "american"))
      (call-interactively #'set-default-cheese)
    (setf (symbol-function 'completing-read) original)))

The unwind-protect ensures the original function is rebound even if the body of the call were to fail. This is very much a let-like pattern, and I’m using symbol-function as a generalized variable via setf. Is there a generalized variable version of let?

Yes! It’s called cl-letf! In this case the f suffix is analogous to the f suffix in setf. That form above can be reduced to a more general form:

(cl-letf (((symbol-function 'completing-read)
           (lambda (&rest _) "american")))
  (call-interactively #'set-default-cheese))

And that’s the way to reproduce the dynamically scoped behavior of flet since Emacs 24.3. There’s nothing complicated about it.

(ert-deftest test-set-default-cheese ()
  (let (default-cheese)
    (cl-letf (((symbol-function 'completing-read)
               (lambda (&rest _) "american")))
      (call-interactively #'set-default-cheese)
      (should (eq 'american default-cheese)))))

Keep in mind that this suffers the exact same problem with bytecode-assigned functions as flet, and for exactly the same reasons. If completing-read were to ever be assigned its own opcode then cl-letf would no longer work for this particular example.

-1:-- Make Flet Great Again (Post)--L0--C0--October 27, 2017 09:02 PM

Phil Hagelberg: in which a path is charted through the coming apocalypse

I've long counted myself among the grumpy old-timers who grudgingly accept the shift towards web-based-everything and just try to make the most of it, wistfully remembering the days when I could just do everything from within Emacs. One of my core survival strategies in this web-first world has been to trick my browser into at least having the decency to pretend to be Emacs. I accomplished this in Firefox1 with the Keysnail extension. Keysnail has remarkable flexibility in how it overrides Firefox's default key bindings to match those of Emacs, and everything has been more or less great.

Unfortunately, a soon-to-be-released update to Firefox will remove the extension mechanism used by Keysnail.

laptop at Green Lake

I have felt very conflicted about this, because the old state of affairs is admittedly untenable. Firefox currently uses Gecko, a decades-old rendering engine written in C++, and like much software written in C++ it has a pretty distressing security track record. Version 57 of Firefox replaces parts of Gecko with functionality from Servo, a browser engine implemented in the Rust programming language. Most of the bugs in Gecko which have led to embarrassing security flaws are simply impossible in Servo. The fact that so much safety-critical code is still being written in C++ and similar languages is a sad state of affairs, and we should celebrate changes that mean end users will no longer bear the penalty for programmers' reluctance to move beyond the technology of the 1980s.

But on the other hand, losing the ability to shape your computing environment to your whims is awful. I lost track of how many times (when using Chromium or other keysnail-less browsers) I've wanted to throw my laptop out the window when I held down ctrl-n to scroll down and it opened seventeen new windows instead. I can't remember ever wanting to open a new browser window in the past decade; why should I be stuck with a key bound to that command and no way to disable it?

Of course, the new Firefox will still have an extension mechanism, but it's a pale shadow of the old one. Citing the flimsy2 excuse of security, key bindings like C-n are hard-coded into the browser and forbidden from being overridden.

Tumwater Falls

Things were looking bleak for me, and I contemplated whether I would switch to curl or just give up software development altogether for a career in goat-herding. I ended up finding a solution from a most unlikely place.

I had heard of EXWM a while ago, and it struck me as a quixotic curiosity. The X Window System uses a network socket for its control protocol, allowing a lot of flexibility including native forwarding of interfaces for remote programs. The developer of EXWM had taken an XML description of the specification for the network protocol and written a compiler to turn it into a library of Emacs Lisp functions which he then used to implement a window manager in pure Emacs Lisp. While I admired the chutzpah this must have taken, I assumed it was a novelty that could never be practical.

Eventually the Firefox conundrum prompted me to give it a second look due to a feature called Simulation Keys. The exwm-input-set-simulation-keys function allows you to define a translation mapping so that a certain key combination will be intercepted by EXWM when a non-Emacs program has focus, and a different set of key input events will be sent instead. It seemed too good to be true; I could let go of Keysnail and instead get the same features applied to every program I use3.

I'm happy to report that EXWM does actually function startlingly well as a window manager. The simulation keys feature is amazing and puts my Firefox-related fears at ease, and having all configuration written in a single language simplifies my setup dramatically. Every X window you launch is given an Emacs buffer, and all your normal splits and window resizing commands work great with it. With the tiling window managers I used in the past, it was so unusual for me to use something other than the "one fullscreen window per display" setup that I would often forget the key bindings for splitting and rearranging windows. EXWM even integrates "system tray" programs into the Emacs echo area, so your wifi connect tool shows up unobtrusively in the bottom right corner.

There are a handful of gotchas. Emacs Lisp lacks general-purpose concurrency features, but it does allow for concurrency when dealing with subprocesses and network communication. Most well-written Emacs Lisp will never block the main event loop, which is good because when using EXWM that means the entire window manager is stuck until the blocking operation completes. I only came across two exceptions to this rule. One of them is smtpmail-send-it, which can be replaced by the smtpmail-async library. The other is the racket-run command, which I was able to patch in about an hour to remove the blocking call4.

Other folks might run into more problems if they use other third-party libraries which don't take care to use the network functions properly. But for my use5, it's been very smooth, and I'm thrilled to have it.

[1] I used Conkeror for several years, but eventually things got to the point where browsing without Noscript became untenable, and I could never get the two to work well together.

[2] The rationale of "it's for security" would stand up to a little more scrutiny if it weren't for the fact that extensions can rebind C-t, a key which is used hundreds if not thousands of times more often than C-n.

[3] Granted gnumeric is the only program I use outside the browser and Emacs, but it's still greatly appreciated. I also use the Saka Key extension, which implements Keysnail's ability to trigger links from the keyboard even if they don't have text attached to them.

[4] I feel that the increasing "Emacs needs concurrency!" calls tend to overstate the problem. Yes, of course it would be nicer for the programmer to code using coroutines (coming in Emacs 26!) instead of callbacks, but in the end this is a convenience for the author, not for the end user.

[5] My customizations largely revolve around replacing my xbindkeys config with elisp, mapping workspace numbers to physical displays, and some eshell commands to give one eshell buffer per workspace. EXWM has XMonad-style workspaces where you can change the workspace for each display independently rather than forcing you to change them all at once like many more conventional WMs, and I'm very glad it does.

-1:-- in which a path is charted through the coming apocalypse (Post Phil Hagelberg)--L0--C0--October 23, 2017 07:04 PM

emacsninja: Design Is Hard

This isn’t about the pixel pushing kind of design, but the engineering one. Given a problematic matter, what choices do you make to create a tool that enables its user to effectively interact another object? More importantly, how do you deal with choices that are hard to rectify afterwards? While this is going to be a rant, the subject is one of my more popular Emacs packages, Shackle. I thought the 1.0.0 release of it with a new debugging facility to make troubleshooting easier is just the right moment to ponder a bit about those choices I made and why I regret some of them.

You may wonder “Wait, what is wrong with Shackle? It has over a hundred stars of GitHub, a few thousand downloads on MELPA, dozens of people using it in their init files and a handful of people recommending it to others.”. While all of this is true, it’s not all roses. I occasionally get issues from users that don’t understand it at all and I can’t really blame them. There is a fundamental mismatch going on here because all this package does is hijacking the display-buffer-alist variable to invent a similar, but not quite as powerful mechanism on top of it. It’s an inherently leaky abstraction which makes for less than ideal debugging: If it ever breaks down, you’ll have to understand both the abstraction and the underlying code it’s built upon.

This project started off with me not understanding how to use this variable at all. In hindsight, this should have been the first warning signal: If you can’t fully understand the problem, don’t expect to solve it in a satisfactory manner. There are a few glaring problems with display-buffer-alist:

  • The docstring for it is hard to parse. If a newbie asks how to customize the display of a certain buffer and is directed to that variable, I couldn’t blame them for just giving up on this altogether.
  • It isn’t clear how to display a buffer in a certain way. I’ve found only one example in the elisp manual so far and it’s more about display-buffer than display-buffer-alist.
  • Conditions may be buffer names and functions, but not major modes. This is rather annoying as it means you’ll have to write a function to check the major mode yourself. While this is far from fool-proof (the code setting up the buffer may enable the desired major mode only after displaying it), it works in many cases.
  • If your customization of display-buffer-alist contains a call to a function that errors out, the display of that buffer will fail. This is particularly annoying if you have a catch-all rule there that prevents the source debugger window from appearing, something I mostly ran into while developing Shackle. While you can use M-: (setq display-buffer-alist nil), it’s relatively annoying to do so.
  • The default behavior is rather inscrutable and mostly, but not only determined by display-buffer-fallback-action. Worse, some packages rely on the default behavior just to fail with customizations to display-buffer-alist.

Now, does Shackle do better? Well, it does in some ways while being worse in others:

  • Conditions are interpreted as buffer names (if a string) or modes (if a symbol) or a list of either. While this is convenient, the original design had the issue of making it impossible to match by regex or use a custom function, so I added a :regex modifier to the action (which is just wrong because it changes all of them to match by regex) and interpret a list starting with :custom as a function which isn’t nice either. Judging by GitHub’s search there’s about three users of this functionality, with the most prolific one being doom.
  • Shackle tries being easier to understand with regards to actions by abolishing the alist approach and instead going for a flat plist. There is no hierarchy whatsoever which turned out to be a mistake, people didn’t understand that there were keywords with mutually-exclusive behavior, keywords that modified other keywords and keywords that work universally. I’ve had feature requests where I was asked to allow to combine keywords more flexibly, to explain how the whole thing works and most surprisingly, to provide a grammar of the implemented language. The latter found its way into the README and is more confusing than helpful IMO. If you want to understand the behavior, you’re best off with heading to the source. I consider this to be the ultimate proof of failing at its design.
  • It’s way harder to shoot yourself in the foot, in case you do you can always bail out with M-x shackle-mode and revert to vanilla Emacs behavior.
  • The mere act of enabling Shackle will subtly change the default behavior of displaying buffers. The reason for this is shackle--display-buffer-popup-window which tries to do something sensible, but will never behave like the original.
  • I’ve added a feature that doesn’t display a window differently, but rather modifies the window parameter. Admittedly it makes things more convenient because you’d otherwise need a second package to achieve the same effect, but it’s the main reason for display of buffers intended to not be selected to have weird side effects.
  • Debugging Shackle not working as expected is rather tricky. In the best case you’ll need to look at the source code of a package to check whether it’s using display-buffer or a function using it internally (like pop-to-buffer, pop-to-buffer-same-window, switch-to-buffer-other-window, etc.). In the worst case you’ll need to debug the part of the package displaying such windows or Shackle itself while it tries matching conditions and applying actions. I’ve added a tracing mode to make the former easier, but the inherent leaky abstraction remains.
  • While Shackle stayed mostly the same, Emacs gained new capabilities for display-buffer-alist. There isn’t nearly as much reason for using Shackle now, other than laziness. Other people reached the same conclusion that it’s worth investing some of your time in customizing display-buffer-alist.

The bottom line is that I’m not happy with Shackle’s design, but am wise enough to keep it as is and not do any more invasive changes. My happiness (or the lack of) isn’t worth risking the happiness of its users.

-1:-- Design Is Hard (Post Vasilij Schneidermann)--L0--C0--October 20, 2017 09:34 AM