Irreal: Data Sharing in Publications

John Kitchin has a really interesting article in ACS Catalysis on Effective Data Sharing in Scientific Publishing. In it, Kitchin discusses various strategies for embedding supporting data—such as tables and processing code—in a publication's PDF or source file so that other researchers can recreate the results or use the data for further analysis.

He begins by noting that one can simply embed supporting documents directly in the PDF, \LaTeX or Word document. The problem with that method is that it's easy for the actual data and its representation in the final paper to diverge.

A better solution, Kitchin says, is to use Org mode and generate the published tables and results directly from data embedded in the manuscript source. Followers of Kitchin's blog (or even Irreal) will recognize that this is Kitchin's longtime publishing process. He and his group write their papers in Org mode along with the supporting data and code. That way, future researchers have everything they need to reproduce and extend the results.

If you're interested in using Org mode to write papers or in reproducible research, you'll enjoy Kitchin's paper.

-1:-- Data Sharing in Publications (Post jcs)--L0--C0--August 26, 2016 06:09 PM

Chen Bin (redguardtoo): Emacs as C++ IDE, easy way

I design a quick and newbie friendly solution.

It works at Linux/OSX/Cygwin (should work at Windows, but I don't develop at Windows).

Setup is minimum. You only need install GNU Global and two Emacs plugins:

Here is the step to step guide.

1 Step 1, create sample projects for experiment

Say I have two projects ~/proj1 and ~/proj2. Both projects will use third party library C++ header files from read-only directory /usr/include.

A new directory ~/obj to store the index files of third party libraries.

mkdir -p ~/{proj1,proj2,obj}

The content of ~/proj2/lib.cpp,

void proj2_hello(int a2, char* b2) {
}

The content of ~/proj1/main.cpp,

void proj1_hello(int a1, char* b1) {
}

int main(int argc, char *argv[]) {
    return 0;
}

2 Step 2, scan C++ code and setup Emacs

Run below command in shell to scan code,

cd /usr/include && MAKEOBJDIRPREFIX=~/obj gtags -O && cd ~/proj1 && gtags && cd ~/proj2 && gtags

After setting up the corresponding Emacs plugins (minimum setup copied from their website is enough), insert below code into ~/.emacs,

;; Please note `file-truename' must be used!
(setenv "GTAGSLIBPATH" (concat "/usr/include"
                               ":"
                               (file-truename "~/proj2")
                               ":"
                               (file-truename "~/proj1")))
(setenv "MAKEOBJDIRPREFIX" (file-truename "~/obj/"))
(setq company-backends '((company-dabbrev-code company-gtags)))

3 Usage

Use the Emacs plugins as usual.

But you need install latest company built on 25th August because I fixed a company issue yesterday.

Screenshot,

cpp-gtags-demo-nq8.png

4 Technical Details (Optional)

Check GNU Global manual to understand environment variables GTAGSLIBPATH and MAKEOBJDIRPREFIX.

-1:-- Emacs as C++ IDE, easy way (Post Chen Bin)--L0--C0--August 25, 2016 02:20 PM

Ben Simon: An Emacs Friendly Caps Lock Configuration on Windows

While this may be obvious, I was pretty dang pleased with myself when I managed to turn the Caps Lock key on my Windows 10 computer into an emacs friendly Hyper key. Here's what I did:

Step 1. Use AutoHotKey to trivially map the Caps Lock key to the Windows Menu key, or as AutoHotKey calls it, the AppsKey.

;; Add this to your standard AutoHotKey configuration
CapsLock::AppsKey

Step 2. Use this elisp code to capture the Menu key from within emacs and map it to the Hyper modifier:

;; http://ergoemacs.org/emacs/emacs_hyper_super_keys.html
(setq w32-pass-apps-to-system nil)
(setq w32-apps-modifier 'hyper) ; Menu/App key

Step 3. Enjoy! I can now map any key binding using the H- modifier. Here's some code I added to my PHP setup:

(defun bs-php-mode-hook ()
  (local-set-key '[backtab] 'indent-relative)
  (local-set-key (kbd "<H-left>") 'beginning-of-defun)
  (local-set-key (kbd "<H-right>") 'end-of-defun)
  (auto-complete-mode t)
  (require 'ac-php)
  (setq ac-sources  '(ac-source-php ))
  (yas-global-mode 1)
  (setq indent-tabs-mode nil)
  (setq php-template-compatibility nil)
  (setq c-basic-offset 2))

The result: when I open up a PHP file, I can jump between function definitions by holding down Caps Lock and left or right arrow.

I feel like I just won the keyboard shortcut lottery!

-1:-- An Emacs Friendly Caps Lock Configuration on Windows (Post Ben Simon (noreply@blogger.com))--L0--C0--August 25, 2016 02:11 PM

William Denton: Image display size in Org

I just discovered that it’s possible to change the size of an image as displayed in Org while leaving the actual file unchanged. This is great: I can scale it down so it’s just large enough I know what it is but it doesn’t get in my way or take up much real estate.

The variable is org-image-actual-width. C-h v org-image-actual-width shows the documentation:

org-image-actual-width is a variable defined in ‘org.el’. Its value is t

Documentation: Should we use the actual width of images when inlining them?

When set to t, always use the image width.

When set to a number, use imagemagick (when available) to set the image’s width to this value.

When set to a number in a list, try to get the width from any #+ATTR.* keyword if it matches a width specification like

#+ATTR_HTML: :width 300px

and fall back on that number if none is found.

When set to nil, try to get the width from an #+ATTR.* keyword and fall back on the original width if none is found.

This requires Emacs >= 24.1, build with imagemagick support.

(I build Emacs from source, and it has ImageMagick support, though I forget if I had to do anything to get that working. I think just installing ImageMagick is enough. Do ./configure | grep -i imagemagick to check if Emacs knows about it.)

I could set the variable in an init file:

(setq org-image-actual-width nil)

But for now I’m just using it as a file local variable, with this as the first line of the Org file:

# -*- org-image-actual-width: nil; -*-

Then I have, for example, this raw text:

#+NAME: fig:moodleviz
#+CAPTION: Screenshot from Moodleviz.
#+ATTR_ORG: :width 600
#+ATTR_LATEX: :width 5in
[[file:figures/moodleviz-laps.png]]

That image is 1520 pixels wide (wider than my personal laptop—it’s a screenshot taken on a larger screen) and it’s annoying to move by it up or down, so shrinking the displayed size is great. It looks like this scaled down to 600 pixels wide:

Image resized to be smaller. Image resized to be smaller.

ATTR_LATEX shrinks the image to a nice size when I export the document to PDF. There is no HTML version so I don’t care about resizing for that.

-1:-- Image display size in Org (Post William Denton)--L0--C0--August 25, 2016 01:25 PM

Irreal: Say What?!?

I don't have the words…

-1:-- Say What?!? (Post jcs)--L0--C0--August 25, 2016 01:13 PM

Ben Simon: Emacs + PHP - A Modern and (Far More Complete) Recipe

Recently, I had a chance to do a bit of emacs evangelism. Man, is that a soapbox I like to climb on! I hit all my favorite features from dired and dynamic abbreviation expansion to under appreciated gems like ispell. I talked about the power of having a fully programmable, self documenting editor at your fingertips. When I was done, I thought for sure I had managed to add another member to the tribe.

Then, yesterday, my possible convert came to me with a simple question: what mode do you use to edit PHP? Considering that most of the code I write is PHP, you'd think I would be ready to deliver a solid answer. Instead, I mumbled something about having a hacked bit of code I relied on, but really wouldn't recommend it for the general use. Yeah, not cool. I closed out the conversation with a promise: I'd see what the latest PHP options were and report back.

PHP is actually a fairly tricky language to build an editor for. That's because depending on the style it's written in, it can range from disciplined C like code to a gobbly gook C like code with HTML, CSS and JavaScript all mixed in. Add to that the classic emacs problem of having too many 85% solutions, and there's definitely room for frustration. Check out the emacs wiki to see what I mean. You'll find lots of promising options, but no one definitive solution.

After reading up on my choices and trying out some options, I do believe I have a new recipe for PHP + emacs success. Here goes.

Step 1: Setup Melpa

Melpa is a code repository that emacs can magically pull in packages from. Back in the day, adding packages to emacs meant downloading, untarring, and running make. Like all things emacs, the process has been both streamlined, and of course, is fully executable from within emacs. To add Melpa, you'll need to follow the instructions here. Assuming you have a modern version of emacs, this probably just means adding the following to your .emacs:

(add-to-list 'package-archives
             '("melpa" . "https://melpa.org/packages/"))

Step 2: Install the Melpa available PHP Mode

Emacs comes with a PHP mode out of the box, but it seems as though this one (also named php-mode) is more modern. I love that the README talks about PHP 7, showing just how up to date this mode is with respect to the latest language constructs.

Installing this mode, assuming Melpa is cooperating, couldn't be easier. Just run: M-x package-install and enter php-mode.

My preference for indentation is 2 spaces and no insertion of tab characters. Two other tweaks I was after was to turn off HTML support from within PHP and enable subword-mode. All these tweaks are stored in a function and attached to the php-mode-hook. This is all standard .emacs stuff:

(defun bs-php-mode-hook ()
  (setq indent-tabs-mode nil)
  (setq c-basic-offset 2)
  (setq php-template-compatibility nil)
  (subword-mode 1))

(add-hook 'php-mode-hook 'bs-php-mode-hook)

Step 3: Install Web-Mode.el

php-mode above is ideal for code centric PHP files. But what about those pesky layout files that contain HTML, CSS and JavaScript? For that, web-mode.el looks especially promising. Installation and configuration mirrors php-mode. Here's the code I use to customize it:

(defun bs-web-mode-hook ()
  (local-set-key '[backtab] 'indent-relative)
  (setq indent-tabs-mode nil)
  (setq web-mode-markup-indent-offset 2
        web-mode-css-indent-offset 2
        web-mode-code-indent-offset 2))

(add-hook 'web-mode-hook 'bs-web-mode-hook)

Web-mode.el is quite impressive and who knows, it may actually fulfill all my PHP needs. If that's the case, I may choose to stop using the php-mode. However, for now, I like the idea of being able to switch between the two modes. Which brings me to the next step...

Step 4: Add a Quick Mode Toggle

Inspired by this tip on the EmacsWiki, I went ahead and setup a quick toggle between php-mode and web-mode. Here's that code:

(defun toggle-php-flavor-mode ()
  (interactive)
  "Toggle mode between PHP & Web-Mode Helper modes"
  (cond ((string= mode-name "PHP")
         (web-mode))
        ((string= mode-name "Web")
         (php-mode))))

(global-set-key [f5] 'toggle-php-flavor-mode)

Now I'm only an F5 keystroke away from two different editor strategies.

When I have a say in the matter, I tend to be pretty particular about which files in my source tree are pure code and which are pure markup. At some point, I could codify this so that files in the snippets directory, for example, always open in web-mode, whereas files loaded from under lib start off in php-mode. Such is the joy of having a fully programmable editor at your fingertips. You make up the rules!

Step 5: Bonus - Setup aggressive auto-completion

For bonus points, I decided to play with ac-php, a library that supports auto completion of function and class names. I followed the install here, updated my .emacs file as noted below, created the empty file named .ac-php-conf.json in my project's root and then ran M-x ac-php-remake-tags-all. Once that's done, emacs now shouts completions like crazy at me:

It's slick, I'll give you that. I think I may have to see if I can turn down the volume, a bit. Here's what my .emacs now looks like to configure php-mode: now looks like:

(defun bs-php-mode-hook ()
  (auto-complete-mode t)                 ;; «
  (require 'ac-php)                      ;; «
  (setq ac-sources  '(ac-source-php ))   ;; «
  (yas-global-mode 1)                    ;; «
  (setq indent-tabs-mode nil)
  (setq php-template-compatibility nil)
  (setq c-basic-offset 2))

Bye-bye hacked PHP code. Hello modern, feature filled, super easy to install, mega powerful code.

Update: updated the web-mode hook I'm using to make sure all code on the page, not just markup code, is indented 2 steps.

-1:-- Emacs + PHP - A Modern and (Far More Complete) Recipe (Post Ben Simon (noreply@blogger.com))--L0--C0--August 25, 2016 04:33 AM

Pragmatic Emacs: A tweak to elfeed filtering

I wrote recently about my enthusiasm for the elfeed feed reader. Here is a microscopic tweak to the way elfeed search filters work to better suit my use.

By default, if I switch to a bookmarked filter to view e.g. my feeds tagged with Emacs (as discussed in the previous post), and then hit s to run a live filter, I can type something like “Xah” to dynamically narrow the list of stories to those containing that string. The only problem is I actually have to type ” Xah”, i.e. with a space before the filter text, since it is appended to the filter that is already present “+unread +emacs” in this case.

Since life is too short to type extra spaces, I wrote a simple wrapper for the elfeed filter command:

;;insert space before elfeed filter
(defun bjm/elfeed-search-live-filter-space ()
  "Insert space when running elfeed filter"
  (interactive)
  (let ((elfeed-search-filter (concat elfeed-search-filter " ")))
    (elfeed-search-live-filter)))

I add this to the elfeed keybindings when I initialise the package

(use-package elfeed
  :ensure t
  :bind (:map elfeed-search-mode-map
              ("A" . bjm/elfeed-show-all)
              ("E" . bjm/elfeed-show-emacs)
              ("D" . bjm/elfeed-show-daily)
              ("/" . bjm/elfeed-search-live-filter-space)
              ("q" . bjm/elfeed-save-db-and-bury)))

and now I can use / to filter my articles without needing the extra space.

-1:-- A tweak to elfeed filtering (Post Ben Maughan)--L0--C0--August 24, 2016 04:44 PM

Pragmatic Emacs: Search or swipe for the current word

It is often handy to search for the word at the current cursor position. By default, you can do this by starting a normal isearch with C-s and then hitting C-w to search for the current word. Keep hitting C-w to add subsequent words to the search.

If, like me, you use swiper for your searches, you can obtain the same effect using M-j after you start swiper.

This is all very nice, but both of those solutions above search for the string from the cursor position to the end of the word, so if “|” marks the cursor position in the word prag|matic, then either method above would search for matic. I made a small tweak to the relevant function in the ivy library that powers swiper so that the whole of the word is used, so in the example above M-j would search for the full pragmatic string.

Here is the code:

;; version of ivy-yank-word to yank from start of word
(defun bjm/ivy-yank-whole-word ()
  "Pull next word from buffer into search string."
  (interactive)
  (let (amend)
    (with-ivy-window
      ;;move to last word boundary
      (re-search-backward "\\b")
      (let ((pt (point))
            (le (line-end-position)))
        (forward-word 1)
        (if (> (point) le)
            (goto-char pt)
          (setq amend (buffer-substring-no-properties pt (point))))))
    (when amend
      (insert (replace-regexp-in-string "  +" " " amend)))))

;; bind it to M-j
(define-key ivy-minibuffer-map (kbd "M-j") 'bjm/ivy-yank-whole-word)

Update

An offline commenter pointed out that Xah Lee has a nice alternative implementation of this functionality using isearch.

-1:-- Search or swipe for the current word (Post Ben Maughan)--L0--C0--August 23, 2016 09:45 AM

Flickr tag 'emacs': DSCF2762_gmpd

MGdesigner posted a photo:

DSCF2762_gmpd

The secret attendee in coscup 2016

-1:-- DSCF2762_gmpd (Post MGdesigner (nobody@flickr.com))--L0--C0--August 22, 2016 09:25 AM

Flickr tag 'emacs': DSCF2758_gpmd

MGdesigner posted a photo:

DSCF2758_gpmd

The secret attendee in coscup 2016

-1:-- DSCF2758_gpmd (Post MGdesigner (nobody@flickr.com))--L0--C0--August 22, 2016 09:25 AM

sachachua: 2016-08-22 Emacs News

Links from reddit.com/r/emacs, /r/orgmode, Hacker News, planet.emacsen.org, Youtube, the changes to the Emacs NEWS file, and emacs-devel.

Past Emacs News round-ups

The post 2016-08-22 Emacs News appeared first on sacha chua :: living an awesome life.

-1:-- 2016-08-22 Emacs News (Post Sacha Chua)--L0--C0--August 22, 2016 05:36 AM

Grant Rettke: Emacs-wgrep Provides Writable Grep Buffers That Apply The Changes To The Files

Emacs-wgrep provides writable grep buffer that apply the changes to files.

Intuitive and familiar idea if you already like editable dired buffers.

-1:-- Emacs-wgrep Provides Writable Grep Buffers That Apply The Changes To The Files (Post Grant)--L0--C0--August 21, 2016 08:14 PM

Grant Rettke: It Is Time To Migrate from grep to ag

ag is fast, does what you expect, and works well with Emacs. Maybe it is time for you to switch.

-1:-- It Is Time To Migrate from grep to ag (Post Grant)--L0--C0--August 21, 2016 07:45 PM

sachachua: 2016-08-15 Emacs News

Links from reddit.com/r/emacs, /r/orgmode, Hacker News, planet.emacsen.org, Youtube, the changes to the Emacs NEWS file, and emacs-devel.

Past Emacs News round-ups

The post 2016-08-15 Emacs News appeared first on sacha chua :: living an awesome life.

-1:-- 2016-08-15 Emacs News (Post Sacha Chua)--L0--C0--August 15, 2016 03:57 PM

Chen Bin (redguardtoo): No worries when elpa is down

I use one liner shell command to clone Emacs Lisp Package Archive (ELPA):

mkdir -p ~/elpaclone && cd ~/elpaclone && curl -L https://elpa.gnu.org/packages/archive-contents | perl -pe 's/(^\(1|\n)//g' | perl -pe 's/\]\)/])\n/g' | perl -pe 's/^ *\(([a-z0-9A-Z-]*).*\[\(([0-9 ]*).*(single|tar).*/\1-\2.\3/g' | perl -pe 's/ /./g' | perl -pe 's/single/el/g' | perl -pe 's/\)//g' | xargs -I {} curl -L  -O https://elpa.gnu.org/packages/{} && curl -L -O https://elpa.gnu.org/packages/archive-contents

The https://elpa.gnu.org/packages/archive-contents contains all the information of packages. I re-organize it to make sure each line corresponds to one package. Then I use cURL to download everything.

Usage is simple.

Insert below line at the beginning of ~/.emacs when elpa.gnu.org is down:

(setq package-archives '(("elpaclone" . "~/elpaclone")))

This solution also works for MELPA.

-1:-- No worries when elpa is down (Post Chen Bin)--L0--C0--August 14, 2016 12:52 PM

Chris Wellons: An Elfeed Database Analysis

The end of the month marks Elfeed’s third birthday. Surprising to nobody, it’s also been three years of heavy, daily use by me. While I’ve used Elfeed concurrently on a number of different machines over this period, I’ve managed to keep an Elfeed database index with a lineage going all the way back to the initial development stages, before the announcement. It’s a large, organically-grown database that serves as a daily performance stress test. Hopefully this means I’m one of the first people to have trouble if an invisible threshold is ever exceeded.

I’m also the sort of person who gets excited when I come across an interesting dataset, and I have this gem sitting right in front of me. So a couple of days ago I pushed a new Elfeed function, elfeed-csv-export, which exports a database index into three CSV files. These are intended to serve as three tables in a SQL database, exposing the database to interesting relational queries and joins. Entry content (HTML, etc.) has always been considered volatile, so this is not exported. The export function isn’t interactive (yet?), so if you want to generate your own you’ll need to (require 'elfeed-csv) and evaluate it yourself.

All the source code for performing the analysis below on your own database can be found here:

The three exported tables are feeds, entries, and tags. Here are the corresponding columns (optional CSV header) for each:

url, title, canonical-url, author
id, feed, title, link, date
entry, feed, tag

And here’s the SQLite schema I’m using for these tables:

CREATE TABLE feeds (
    url TEXT PRIMARY KEY,
    title TEXT,
    canonical_url TEXT,
    author TEXT
);

CREATE TABLE entries (
    id TEXT NOT NULL,
    feed TEXT NOT NULL REFERENCES feeds (url),
    title TEXT,
    link TEXT NOT NULL,
    date REAL NOT NULL,
    PRIMARY KEY (id, feed)
);

CREATE TABLE tags (
    entry TEXT NOT NULL,
    feed TEXT NOT NULL,
    tag TEXT NOT NULL,
    FOREIGN KEY (entry, feed) REFERENCES entries (id, feed)
);

Web authors are notoriously awful at picking actually-unique entry IDs, even when using the smarter option, Atom. I still simply don’t trust that entry IDs are unique, so, as usual, I’ve qualified them by their source feed URL, hence the primary key on both columns in entries.

At this point I wish I had collected a lot more information. If I were to start fresh today, Elfeed’s database schema would not only fully match Atom’s schema, but also exceed it with additional logging:

  • When was each entry actually fetched?
  • How did each entry change since the last fetch?
  • When and for what reason did a feed fetch fail?
  • When did an entry stop appearing in a feed?
  • How long did fetching take?
  • How long did parsing take?
  • Which computer (hostname) performed the fetch?
  • What interesting HTTP headers were included?
  • Even if not kept for archival, how large was the content?

I may start tracking some of these. If I don’t, I’ll be kicking myself three years from now when I look at this again.

A look at my index

So just how big is my index? It’s 25MB uncompressed, 2.5MB compressed. I currently follow 117 feeds, but my index includes 43,821 entries from 309 feeds. These entries are marked with 53,360 tags from a set of 35 unique tags. Some of these datapoints are the result of temporarily debugging Elfeed issues and don’t represent content that I actually follow. I’m more careful these days to test in a temporary database as to avoid contamination. Some are duplicates due to feeds changing URLs over the years. Some are artifacts from old bugs. This all represents a bit of noise, but should be negligible. During my analysis I noticed some of these anomalies and took a moment to clean up obviously bogus data (weird dates, etc.), all by adjusting tags.

The first thing I wanted to know is the weekday frequency. A number of times I’ve blown entire Sundays working on Elfeed, and, as if to frustrate my testing, it’s not unusual for several hours to pass between new entries on Sundays. Is this just my perception or are Sundays really that slow?

Here’s my query. I’m using SQLite’s strftime to shift the result into my local time zone, Eastern Time. This time zone is the source, or close to the source, of a large amount of the content. This also automatically accounts for daylight savings time, which can’t be done with a simple divide and subtract.

SELECT tag,
       cast(strftime('%w', date, 'unixepoch', 'localtime') AS INT) AS day,
       count(id) AS count
FROM entries
JOIN tags ON tags.entry = entries.id AND tags.feed = entries.feed
GROUP BY tag, day;

The most frequent tag (13,666 appearances) is “youtube”, which marks every YouTube video, and I’ll use gnuplot to visualize it. The input “file” is actually a command since gnuplot is poor at filtering data itself, especially for histograms.

plot '< grep ^youtube, weekdays.csv' using 2:3 with boxes

Wow, things do quiet down dramatically on weekends! From the glass-half-full perspective, this gives me a chance to catch up when I inevitably fall behind on these videos during the week.

The same is basically true for other types of content, including “comic” (12,465 entries) and “blog” (7,505 entries).

However, “emacs” (2,404 entries) is a different story. It doesn’t slow down on the weekend, but Emacs users sure love to talk about Emacs on Mondays. In my own index, this spike largely comes from Planet Emacsen. Initially I thought maybe this was an artifact of Planet Emacsen’s date handling — i.e. perhaps it does a big fetch on Mondays and groups up the dates — but I double checked: they pass the date directly through from the original articles.

Conclusion: Emacs users love Mondays. Or maybe they hate Mondays and talk about Emacs as an escape.

I can reuse the same query to look at different time scales. When during the day do entries appear? Adjusting the time zone here becomes a lot more important.

SELECT tag,
       cast(strftime('%H', date, 'unixepoch', 'localtime') AS INT) AS hour,
       count(id) AS count
FROM entries
JOIN tags ON tags.entry = entries.id AND tags.feed = entries.feed
GROUP BY tag, hour;

Emacs bloggers tend to follow a nice Eastern Time sleeping schedule. (I wonder how Vim bloggers compare, since, as an Emacs user, I naturally assume Vim users’ schedules are as undisciplined as their bathing habits.) However, this also might be prolific the Irreal breaking the curve.

The YouTube channels I follow are a bit more erratic, but there’s still a big drop in the early morning and a spike in the early afternoon. It’s unclear if the timestamp published in the feed is the upload time or the publication time. This would make a difference in the result (e.g. overnight video uploads).

Do you suppose there’s a slow month?

SELECT tag,
       cast(strftime('%m', date, 'unixepoch', 'localtime') AS INT) AS day,
       count(id) AS count
FROM entries
JOIN tags ON tags.entry = entries.id AND tags.feed = entries.feed
GROUP BY tag, day;

December is a big drop across all tags, probably for the holidays. Both “comic” and “blog” also have an interesting drop in August. For brevity, I’ll only show one. This might be partially due my not waiting until the end of this month for this analysis, since there are only 2.5 Augusts in my 3-year dataset.

Unfortunately the timestamp is the only direct numerical quantity in the data. So far I’ve been binning data points and counting to get a second numerical quantity. Everything else is text, so I’ll need to get more creative to find other interesting relationships.

So let’s have a look a the lengths of entry titles.

SELECT tag,
       length(title) AS length,
       count(*) AS count
FROM entries
JOIN tags ON tags.entry = entries.id AND tags.feed = entries.feed
GROUP BY tag, length
ORDER BY length;

The shortest are the webcomics. I’ve complained about poor webcomic titles before, so this isn’t surprising. The spikes are from comics that follow a strict (uncreative) title format.

Emacs article titles follow a nice distribution. You can tell these are programmers because so many titles are exactly 32 characters long. Picking this number is such a natural instinct that we aren’t even aware of it. Or maybe all their database schemas have VARCHAR(32) title columns?

Blogs in general follow a nice distribution. The big spike is from the Dwarf Fortress development blog, which follows a strict date format.

The longest on average are YouTube videos. This is largely due to the kinds of videos I watch (“Let’s Play” videos), which tend to have long, predictable names.

And finally, here’s the most interesting-looking graph of them all.

SELECT ((date - 4*60*60) % (24*60*60)) / (60*60) AS day_time,
       length(title) AS length
FROM entries
JOIN tags ON tags.entry = entries.id AND tags.feed = entries.feed;

This is the title length versus time of day (not binned). Each point is one of the 53,360 posts.

set style fill transparent solid 0.25 noborder
set style circle radius 0.04
plot 'length-vs-daytime.csv' using 1:2 with circles

(This is a good one to follow through to the full size image.)

Again, all Eastern Time since I’m self-centered like that. Vertical lines are authors rounding their post dates to the hour. Horizontal lines are the length spikes from above, such as the line of entries at title length 10 in the evening (Dwarf Fortress blog). There’s a the mid-day cloud of entries of various title lengths, with the shortest title cloud around mid-morning. That’s probably when many of the webcomics come up.

Additional analysis could look further at textual content, beyond simply length, in some quantitative way (n-grams? soundex?). But mostly I really need to keep track of more data!

-1:-- An Elfeed Database Analysis (Post)--L0--C0--August 12, 2016 03:20 AM

Rubén Berenguel: Moving from Emacs to Spacemacs

A couple of days ago I attended (first time I managed in almost 6 months) the London chapter of the Emacs Church (also known as the local meetup for emacs lovers). In this event we were shown how to use emacs effectively for Clojure development (using Cider) and I saw in real life Spacemacs.

In case you don't know, Spacemacs is a "distribution" of Emacs prepared (is open source, of course) to be easy to setup, and somehow specially prepared for former Vim users to move to Emacs. For instance, on startup asks if you want to be in Emacs mode, evil mode or hybrid (Emacs mode keys in vim insert mode) by default.

As you may remember, I've been using Emacs with evil for around 3 years already, and have been pretty happy with it. I'm not so happy about the state of my .emacs file: currently it is 2652 lines long (of course around 400 or 500 of those are generated automatically by custom). Too big and unwieldy. And I'm too lazy to move all the nuts and bolts to something more lightweight and sane.

I thought that trying out Spacemacs could be the perfect excuse to clean the mess off my .emacs file, since I could carefully move piece by piece whatever I needed as I needed it (like I usually do when upgrading computers).

For a start, the beginning was a good experience. Since by default it includes most of the fancy stuff I use normally (helm being the biggest, fanciest helper I need, I only needed to tweak helm-files and switch-buffer, and add recentf) and I no longer use mu4e on a day-to-day basis, I could easily switch to it.

It's actually really easy to try Spacemacs alongside your normal emacs: just download it and from the spacemacs folder run something like
HOME=~/fromsource/spacemacs /Applications/Emacs.app/Contents/MacOS/Emacs
or the equivalent in your environment. Caveat: multi-term/ansi-term won't work as expected (best solution is to actually move to using Spacemacs as default). Aside from that I have had no other big issues, and recently moved to Spacemacs to be the main Emacs and if needed I can run my old configuration with the "HOME trick".

One thing I have not figured out how to exactly do "the Spacemacs way" but I needed no matter what is having my set of normal Emacs keybindings in evil insert and normal modes. Hybrid mode covers insert more or less nicely, but some commands I need them as they are because not only are they part of my muscle memory, but I also happen to like them and use them everywhere (OS X input fields, terminal windows). I tried to set this up in many places in the .spacemacs file (the user-config section, using the -init or -after macros of package initialisation...) And finally this made it work, so, in case you need to modify evil insert or normal key maps in Spacemacs:

(eval-after-load 'evil 
'(progn
(define-key evil-normal-state-map (kbd "SPC") 'ace-jump-mode)
(define-key evil-insert-state-map (kbd "C-a") 'move-beginning-of-line)
(define-key evil-insert-state-map (kbd "C-e") 'move-end-of-line)
(define-key evil-insert-state-map (kbd "C-k") 'kill-line)
(define-key evil-insert-state-map (kbd "C-w") 'kill-region)
(define-key evil-visual-state-map (kbd "C-a") 'move-beginning-of-line)
(define-key evil-visual-state-map (kbd "C-e") 'move-end-of-line)
(define-key evil-normal-state-map (kbd "C-a") 'move-beginning-of-line)
(define-key evil-normal-state-map (kbd "C-e") 'move-end-of-line)
(define-key evil-normal-state-map (kbd "C-k") 'kill-line)
(define-key evil-normal-state-map (kbd "C-y") 'yank)
(define-key evil-insert-state-map (kbd "C-y") 'yank)
(define-key evil-insert-state-map (kbd "C-t") 'transpose-chars)
(define-key evil-normal-state-map (kbd "C-w") 'kill-region)
(define-key evil-visual-state-map (kbd "C-w") 'kill-region)
(define-key evil-visual-state-map (kbd "SPC") 'ace-jump-mode)))
Something I found surprising though is the load time: Spacemacs does not load packages on startup and my .emacs setup did, and they roughly start up at the same speed. Somehow I expected a faster startup time.

Worth also downloading the latest emacs port for homebrew, which fixes some annoyances with powerline colours as well as being quite more up to date than Carbon Emacs.

Now the only big thing left I have is reconfiguring multiple-cursors and my definitions of more-like-this and all that to be a happy Spacemacs user.
-1:-- Moving from Emacs to Spacemacs (Post Rubén Berenguel (noreply@blogger.com))--L0--C0--August 07, 2016 08:42 PM

Marcin Borkowski: Highly sophisticated engineering

In one of my Emacs libraries, in which I send HTTP requests to a website, I have something like this: (if (string-match "\\?" request) "&" "?") ; this is hackish (the purpose is clear if you know how the URIs are structured). As the comment says, I actually considered this rather a duct-tape approach… …until I found this gem in request.el:
-1:-- Highly sophisticated engineering (Post)--L0--C0--August 01, 2016 08:03 PM

John Stevenson: YASnippets for Faster Clojure Coding

Using snippets saves you time typing common coding structures and helps you avoid silly typos too. Simply typing in a snippet name and pressing M-/ or using M-x yas-expand gives you the full text & code structure from the snippet template.

For example, if you are defining a new function in Clojure then type defn and press M-/ to expand to the full definition structure, including all parens. Then use TAB to move through the structure to complete the name, doc-string, arguments and behaviour of the function.

Lets look at the built-in snippets that come with the Clojure layer in Spacemacs (and should be the default in Emacs YASnippet package).

Its easy to wrtie your own snippets for Clojure or any other language you use with Spacemacs / Emacs

Where do Snippets fit in

There are several ways to speed up typing your Clojure code, Leiningen Templates, autocompletion, clj-refactor and YASnippets.

Snippets are a great way to create much of the common code structures you typically write as you are developing your code. The snippets I use the most are defn, let, ifl and for.

Some of the snippets are made redundant by other Clojure layer features in Spacemacs. For example, when I create a new file then the namespace is automatically added, so I rarely use the ns snippet. I also use clj-refactor which takes care of adding require statements for me, so the require snippet is rarely used.

Note: Smartparens does not work when you are completing an expanded snippet the first time. However, as soon as you tab out of the end of the snippet then smartparens works again. I havent determined if this is simply a missing config or some conflict between YASnippets & Smartparens.

Default snippets for Clojure

The snippets that are available in Spacemacs can be found in the yasnippet github repository or in the ~/.emacs.d/elpa/yasnippet-20160501.1841/snippets/clojure-mode directory on your laptop (when yasnippet package is updated then name of that directory will change).

All the current snippets for Clojure mode are in the following table, indicating which tab stops they have.

Snippet Description Tab Stops
bench benchmark an expression, using the time function body
bp depreciated: break point in swank
def def expression N/A
defm defmacro expression, with name, doc-string, arguments & body tabstops name, docstring, args, body
defn defn expression, with name, doc-string, arguments & body tabstops name, docstring, args, body
defr defrecord docstring, object, args, body
deft deftype docstring, object, args, body
doseq doseq name, body
fn fn - anonymous function name, body
for for condition, body
if if condition, body
ifl if-let - if true, bind a local name binding, body
import import java library library name
is is - clojure test assertion value, expected
let let - bind a local name to a value name, value, body
map map fn, col, col
map.lambda map with anonymous function #() fn, body
mdoc metadata docstring docstring
ns ns - expression with the current namespace inserted automatically N/A
opts destructuring a map with :keys, :or for default values, :as for entire vector :key binding, or defaults, :as binding
pr prn - print function string/value
print println - print function string/value
reduce reduce - reduce expression with an anonymous function args, body
require :require expression with library and alias library, alias
test deftest expression test description, value/expected
try try & catch expression try expression, exception name, body
use depreciated: use require instead
when when when expression, body
whenl when-let - local binding on when condition binding, body

Hint: The above table was created in Emacs markdown mode by enabling the minor mode: orgtbl-mode. Start the table with a | character followed by a word and use TAB to create and align the table as you go through. If you already have content, use the function orgtbl-create-or-convert-from-region. To enable orgtbl-mode by default when in markdown major mode, edit your ~/.spacemacs file, in the dotspacemacs/user section add the line (add-hook 'markdown-mode-hook 'turn-on-orgtbl).

Hope you have fun speeding up the writing of your Clojure code (and any other common text) in Spacemacs / Emacs.

Thank you.
@jr0cket

-1:-- YASnippets for Faster Clojure Coding (Post)--L0--C0--July 30, 2016 02:52 PM

John Stevenson: Spacemacs - Adding Custom Snippets to Yasnippet

Using yasnippet saves time by avoiding the need to write boilerplate code and minimising other commonly typed content. YASnippet contains mode-specific snippets that expand to anything from a simple text replacement to a code block structure that allows you to skip through parameters and other sections of the code block. See YASnippet in action in this Emacs Yasnippet video.

To use a specific snippet simply type the alias and press M-/. For example, in html-mode typing div and pressing M-/ expands to <div id="▮" class="▯">▯</div> and places the cursor so you can type in the id name, then TAB to the class name, finally TAB to the contents of the div.

You can also combine yasnippets with autocompletion select snippets from the autocompletion menu.

Spacemacs has lots of snippets for most of the languages and modes it supports. However, YASnippets also uses a simple template system in plain text, so its pretty easy to learn. Lets look at how to add your own snippets with Spacemacs.

In regular Emacs, yasnippets expand funciton is usually bound to TAB, but that key is used already in Spacemacs so M-/ is used instead.
If you just want text replacement you can also use Emacs Abbrev mode.

Adding your private snippets to Spacemacs

The easiest place to add your own snippet definitions is in the ~/.emacs.d/private/snippets directory. Under this directory structure you should create a folder named after the relevant mode for your snippets, eg markdown-mode. Inside this mode folder, create files whos names are based on the snippet alias you wish.

So for a work in progress snipped called wip in markdown mode I created ~/.emacs.d/private/snippets/markdown-mode/wip file.

You need to load this new snippet into Spacemacs by either restarting or using the command M-x yas-load-snippet-buffer command in the buffer of the new snippet you have just written. Ths snippet with then work within any markdown mode buffer.

Managing your snippets

Although the private snippets directory is easy to use, it is not under version control. So although its not over-riddend by Spacemacs it also means your private snippets are not backed up anywhere.

If you use the ~/.spacemacs.d/snippets/modename-mode/ directory structure for your snippets then you can version them with Git or similar versioning tools.

How to write a snippet

Typically each snippet template is contained in its own file, named after the alias of the snippet. So a snippet called wip will be in a filename wip, in a directory named after the relevant Emacs mode.

The basic structure of a snippet template is:

1
2
3
4
5
#key : the name of the snippet you type
#name : A description of the snippet (this shows in autocompletion menu too)
#contributor: John Stevenson <john@jr0cket.co.uk>
# --
Add the content you want to replace the snippet name with when it expands

The content can be anything, simple text or more usefully a code strucuture with placeholders for tab stops. You can even include Emacs lisp (elisp) code in there too.

Example: Simple text replacement

I use markdown mode for writing a lot of content, especially for technical workshops. As I am developing these workshops its useful to highlight which sections are still work in progress. Rather than type the common message I use, I’ve created a simple snippet called wip.

1
2
3
4
5
#key : wip
#name : WorkInProgress
#contributor: John Stevenson <john@jr0cket.co.uk>
# --
> **Fixme** work in progress

When you expand this snippet with M-/ then the snippet name is replaced by the content.

Example: Using tab stops

Lets look at an existing snippet called form in the html-mode. This expands into a html form, but also helps you jump from method, id, action and content.

1
2
3
4
5
6
#contributor : Jimmy Wu <frozenthrone88@gmail.com>
#name :<form method="..." id="..." action="..."></form>
# --
<form method="$1" id="$2" action="$3">
$0
</form>

This snippet is the same as the simpler example, except we have added tab stops using the $ sign and a number. When you expand this snippet, the snippet name is replaced by the content as usual but the cursor is placed at the first tab stop $1. Each time you press TAB you move to the next tab stop.

$0 is our exit point from the snippet, so pressing TAB reverts to the usual behaviour outside of YASnippet.

Creating a snippet from existing text

A really fast way of creating a new snippet is to use a finished version of what you would like the snippet to expand to. For a simple text replacement you just hightlight all the text and call helm-yas-create-snippet-on-region, save the snippet and you are done.

For a code structure with tab stops, simply hightlhght a completed code stucture, call helm-yas-create-snippet-on-region and edit the body of your snippet to replace the specific names and values with tab stop placeholders, $1 $2, $3, etc.

Example: Create a simple text replacement

When I write blogs I include a image thumbnail that gives a visual clue as to the topic of the article. Rather than type this in I created a snippet.

First I mark the text I want my new snippet to expand too, in this example: {% img img-thumbnail /images/spacemacs.png %} .

Then I call the function helm-yas-create-snippet-on-region. This prompts me for the mode for the snippet, in this case markdown-mode, then prompts for the location for the snippet file, ~/.emacs/private/snippets/markdown-mode/imgtmb-spacemacs. A new buffer is created with my snippet already filled in.

1
2
3
4
5
6
# -*- mode: snippet -*-
#name : imgtmb-spacemacs
#key : imgtmb-spacemacs
#contributor : jr0cket
# --
{% img img-thumbnail /images/spacemacs.png %}

The new snippet buffer already has the name and key values populated from the filename I gave for the snippet, imgtmb-spacemacs. The snippet body is also populated automatically from the text I had highlighted. So all I need to do is save the new snippet and try it out.

Testing your snippets

Once you have written your snippet, you can quickly test it using M-x yas-tryout-snippet. This opens a new empty buffer in the appropriate major mode and inserts the snippet so you can then test it with M-/.

If you just want to try the snippet in an existing buffer, then use M-x yas-load-snippet-buffer to load this new snippet into the correct mode. M-x yas-load-snippet-buffer does exactly the same except it kills the snippet buffer (prompting to save first if neccessary).

There are no default keybindings for these commands in Spacemacs, so you could create a binding under C-o, for example C-o C-s t to try a snippet and C-o C-s l to load a snippet.

Adding yas-snippets to autocompletion in Spacemacs

By adding the autocompletion layer in Spacemacs the YASnippets can be shown in the autocompletion menu as you type.

By default, snippets are not shown in the auto-completion popup, so set the variable auto-completion-enable-snippets-in-popup to t.

1
2
3
(setq-default dotspacemacs-configuration-layers
&apos((auto-completion :variables
auto-completion-enable-snippets-in-popup t)))

Summary

Find out more about YASnippets and autocompletion from the Github repository for Spacemacs autocompletion layer.

For more details and examples on writing your own snipplets, take a look at:

Thank you.
@jr0cket

-1:-- Spacemacs - Adding Custom Snippets to Yasnippet (Post)--L0--C0--July 30, 2016 01:51 PM

Wilfred Hughes: Example Driven Development

“Hey $COLLEAGUE, is there any function that takes this list and returns a list like this?”

Some functions are hard to Google. It’s often easier to speak about concrete examples: I have a list '(x y z), and an index 1, and I want a list without the element at that position: '(x z).

I was blown away when I saw this feature implemented in Smalltalk. I decided to build an Emacs implementation and it’s super helpful when you need it. The screenshot above shows it in action.

Lisp Wins

This project is much easier in a language with little syntax. Suppose we’re looking for arithmetic functions:

suggest.el simply has a list of functions, and tries each one in turn. The fact that + is syntactically the same as any other function is a big help here. We don’t need to do any special formatting for infix functions.

Brute-Force Performance

suggest.el brute-force searches its list of functions. It ignores anything that throws an error, or returns a value different to the output requested.

We currently have a list of 140 functions. I was expecting this to be slow, but it’s instant.

We also need to try every permutation of arguments, which is computationally expensive. We want users to discover functions like nth, even if they get arguments in the wrong order. There are a lot of permutations, but typically users only use 1-3 arguments, so it’s not a problem.

If suggest.el find a match, then it doesn’t try any other orderings. This prevents us suggesting both (+ 2 3) and (+ 3 2) – the user would probably prefer the order they’ve proposed anyway.

Eval All The Way

suggest.el evals all the expressions given, allowing users to enter literals, variables, or even function expressions. This is convenient when developing new packages and you want to use values from your package.

In the above screenshot, note how the suggested expression (-sum (list 2 3 4)) uses exactly the expression given, so it can be copy-pasted. However, we show the final value => 9, so it’s clear what the function is doing.

Exploiting The Emacs UI

To take best advantage of Emacs, the UI needs to work like a text editor. suggest-mode derives from emacs-lisp-mode, so your favourite lisp packages just work. The entire buffer is a valid lisp file and we get highlighting for free.

suggest.el adds a few extra touches on top. We use overlays to highlight the headings, which overrides the comment highlighting. Headings and output also use the special property read-only to guide users into typing in the correct places.

suggest.el also uses change hooks (pictured above) to inform the user that they need to re-run suggest-update. This shows C-c C-c (the default keybinding) by default. Just like the Emacs tutorial, suggest.el will always use the current keybindings.

Choosing Functions

suggest.el has a whitelist of functions that are safe to run. Which functions should we include? suggest.el is in a position of responsibility, as it affects the code that users write.

Functions absolutely cannot have side-effects. Calling delete-file could be catastrophic. If a function only mutated its input arguments, that would be safe, but it’s not necessarily what the user expected.

Strictly requiring pure functions means that many built-in Emacs functions aren’t offered. Wherever possible, we suggest built-in functions, and show them first.

suggest.el has to decide which function aliases to show. I’ve been convincingly persuaded that predicate functions should use -p, so suggest.el prefers those aliases. suggest.el also favours cl-lib over the popular, but deprecated cl (e.g. cl-first not first).

We also have to decide which packages to include. suggest.el includes third-party packages that I consider important (dash.el, s.el and f.el). Long term, I hope packages will add to suggest-functions using eval-after-load.

Third-party packages often have a higher proportion of pure functions. In the case of f.el, I’ve included a few impure functions that read the filesystem, as they’re safe.

It’s also tricky to pick which functions belong in the whitelist. suggest.el tries to offer functions that you’re likely to ‘stumble across’.

I found that the best functions take 1-3 arguments of simple types. You’re unlikely to find a higher-order function from a random example. We also want functions that are characterised by a single example. Single-argument predicates aren’t much use, since they need several examples to demonstrate their behaviour.

We currently have functions for lists, alists, plists, hash tables, numbers, strings, symbols and paths. This covers much of my day-to-day development, and I’ve already learnt some new functions!

Give It A Try!

suggest.el is on GitHub, on MELPA, and I would love to hear your feedback. You can tweet at me or file an issue.

-1:-- Example Driven Development (Post Wilfred Hughes (me@wilfred.me.uk))--L0--C0--July 30, 2016 12:00 AM

Wilfred Hughes: Example Driven Development

“Hey $COLLEAGUE, is there any function that takes this list and returns a list like this?”

Some functions are hard to Google. It’s often easier to speak about concrete examples: I have a list '(x y z), and an index 1, and I want a list without the element at that position: '(x z).

I was blown away when I saw this feature implemented in Smalltalk. I decided to build an Emacs implementation and it’s super helpful when you need it. The screenshot above shows it in action.

Lisp Wins

This project is much easier in a language with little syntax. Suppose we’re looking for arithmetic functions:

suggest.el simply has a list of functions, and tries each one in turn. The fact that + is syntactically the same as any other function is a big help here. We don’t need to do any special formatting for infix functions.

Brute-Force Performance

suggest.el brute-force searches its list of functions. It ignores anything that throws an error, or returns a value different to the output requested.

We currently have a list of 140 functions. I was expecting this to be slow, but it’s instant.

We also need to try every permutation of arguments, which is computationally expensive. We want users to discover functions like nth, even if they get arguments in the wrong order. There are a lot of permutations, but typically users only use 1-3 arguments, so it’s not a problem.

If suggest.el find a match, then it doesn’t try any other orderings. This prevents us suggesting both (+ 2 3) and (+ 3 2) – the user would probably prefer the order they’ve proposed anyway.

Eval All The Way

suggest.el evals all the expressions given, allowing users to enter literals, variables, or even function expressions. This is convenient when developing new packages and you want to use values from your package.

In the above screenshot, note how the suggested expression (-sum (list 2 3 4)) uses exactly the expression given, so it can be copy-pasted. However, we show the final value => 9, so it’s clear what the function is doing.

Exploiting The Emacs UI

To take best advantage of Emacs, the UI needs to work like a text editor. suggest-mode derives from emacs-lisp-mode, so your favourite lisp packages just work. The entire buffer is a valid lisp file and we get highlighting for free.

suggest.el adds a few extra touches on top. We use overlays to highlight the headings, which overrides the comment highlighting. Headings and output also use the special property read-only to guide users into typing in the correct places.

suggest.el also uses change hooks (pictured above) to inform the user that they need to re-run suggest-update. This shows C-c C-c (the default keybinding) by default. Just like the Emacs tutorial, suggest.el will always use the current keybindings.

Choosing Functions

suggest.el has a whitelist of functions that are safe to run. Which functions should we include? suggest.el is in a position of responsibility, as it affects the code that users write.

Functions absolutely cannot have side-effects. Calling delete-file could be catastrophic. If a function only mutated its input arguments, that would be safe, but it’s not necessarily what the user expected.

Strictly requiring pure functions means that many built-in Emacs functions aren’t offered. Wherever possible, we suggest built-in functions, and show them first.

suggest.el has to decide which function aliases to show. I’ve been convincingly persuaded that predicate functions should use -p, so suggest.el prefers those aliases. suggest.el also favours cl-lib over the popular, but deprecated cl (e.g. cl-first not first).

We also have to decide which packages to include. suggest.el includes third-party packages that I consider important (dash.el, s.el and f.el). Long term, I hope packages will add to suggest-functions using eval-after-load.

Third-party packages often have a higher proportion of pure functions. In the case of f.el, I’ve included a few impure functions that read the filesystem, as they’re safe.

It’s also tricky to pick which functions belong in the whitelist. suggest.el tries to offer functions that you’re likely to ‘stumble across’.

I found that the best functions take 1-3 arguments of simple types. You’re unlikely to find a higher-order function from a random example. We also want functions that are characterised by a single example. Single-argument predicates aren’t much use, since they need several examples to demonstrate their behaviour.

We currently have functions for lists, alists, plists, hash tables, numbers, strings, symbols and paths. This covers much of my day-to-day development, and I’ve already learnt some new functions!

Give It A Try!

suggest.el is on GitHub, on MELPA, and I would love to hear your feedback. You can tweet at me or file an issue.

-1:-- Example Driven Development (Post Wilfred Hughes (me@wilfred.me.uk))--L0--C0--July 29, 2016 12:00 AM

(or emacs: Swipe all the files!

If you've ever tried the swiper-all from swiper, forget everything about it. The command was super-awkward, since it had to parse all your open files before giving you a chance enter anything, resulting in dozens of seconds before the prompt.

Recently, I've had some time to examine and improve it and the result looks very promising. The new command is now async, which means there's no delay before the prompt comes up. Here's a result I got with no delay while having around 50 buffers open:

swiper-all.png

The shortcut I'm using:

(global-set-key (kbd "C-c u") 'swiper-all)

For efficiency's sake a small trade off had to be made: the line numbers are no longer displayed. This actually results in an advantage that you can select different candidates on the same line.

There are still a few things I plan to try for the new command, like adding file-less buffers, caching for incremental regexes and maybe even newlines in wild cards, but even now it seems very usable. So give it a try, enjoy and happy hacking!

-1:-- Swipe all the files! (Post)--L0--C0--July 28, 2016 10:00 PM

Marcin Borkowski: Making directories on the fly

Today, a friend asked me how to create a directory – or a hierarchy of them – while finding a file in a nonexistent directory. I found some clues on the ‘net, but the best one used the old advice mechanism. So I decided to do it myself, based on the hints found. It turned out to be simpler than I thought; it’s made even easier by the fact that Elisp’s make-directory function acts basically like mkdir -p when given non-nil second (optional) argument.
-1:-- Making directories on the fly (Post)--L0--C0--July 25, 2016 08:36 PM

Raimon Grau: slack -> jabber -> irc -> erc

Slack can be used though alternative gateways appart from the slack clients themselves.  They offer both jabber and irc gateways that map quite well to slack model.

I had previously used the irc gateway, and erc did the trick with no extra configuration at all. But things changed and now I only have the jabber gateway at my disposal.

The problem is that if you want to use a decent emacs client, you'll have to use bitlbee and end up mapping jabber to irc, so you can use erc/circe/rcirc (which are far better at IRC protocol than jabber.el is at xmpp)



To chat privately other peers it's a matter of /j name-of-the-buddy.  Joining rooms is not as direct, and first you have to create some 'mapping' rooms in your bitblee configuration, before being able to join them.

Whatever, here's how to integrate slack with bitlbee (and with erc)

On bitlbee buffer:
  •  chat add 0 room-name@conference.myserver-foo.xmpp.slack.com
 0 stands for the account number. 'conference' has  to be there. it's a plain string, you don't have to substitute it for anything. replace myserver-foo for your server name. room-name is the name of the chat in slack

  • chat list
This will show you the rooms available

  • /join room-name
Notice no # sign.
  • channel 4 del
4 is the chat you want to remove in case you messed up with something.

And that's it. Now you can slack from erc, and you can use all erc machinery (yes erc-robot, I'm looking at you).




-1:-- slack -&gt; jabber -&gt; irc -&gt; erc (Post Raimon Grau (noreply@blogger.com))--L0--C0--July 25, 2016 05:13 PM

Phil Hagelberg: in which a spacefaring text editor holds up against an onslaught

I've been meaning to write about my latest project Bussard for a while now. I describe it as a "spaceflight programming adventure", but it might be easier to think of it as "Emacs in space, but with a sci-fi novella in it", written in Lua with the LÖVE engine. There's a lot to tell about the game and how I want it to eventually be a way for newcomers to learn programming, but I want to write here about a particular part of the development I had a lot of fun with.

The game is played by interacting with your ship's onboard computer. Naturally because I wanted to give the player the freedom to customize the interface as much as possible, I've modeled it on Emacs. The game starts with your ship in orbit around a star and hoping to intercept an orbiting space station, but once you poke around a bit you realize that "flight mode" is only one of many modes you can activate in your onboard computer.

Pressing ctrl-o allows you to open any file in the in-game computer, and pressing ctrl-enter opens a Lua repl buffer that uses the same editor infrastructure but lets you write code interactively and explore your ship's computer's API. Communicating with space station and planet port computers is done over an SSH client that also lives inside the editor. But all the various modes of the editor are configured with Lua code that runs in user-space; basically that code defines keyboard controls which simply invoke functions of your ship's computer to edit text[1], open SSH connections, engage the ship's engine, etc. Every action you can take is just a Lua function call.

-- bind is for commands that only call their functions once even when held.
-- it takes the name of a mode, a key combo, and a function to run when pressed.
bind("flight", "escape", ship.ui.pause)

-- you can bind keys to existing functions or inline functions.
bind("flight", "ctrl-return", function()
        ship.editor.change_buffer("*console*")
end)

-- the mouse wheel is handled just like any other key press.
bind("flight", "wheelup", zoom_in)
bind("flight", "wheeldown", zoom_out)

-- regular tab selects next target in order of distance from the star.
bind("flight", "tab", ship.actions.next_target)

This is a fantastically flexible model for rich interaction—it can be completely rewritten on the fly, and it's seamless to experiment with new ideas you think might support a better way of doing things. No recompiling, no restarting, just flow. But another benefit of the editor API is that you can call it against in an automated context, such as a headless run[2] that does fuzz tests.

I had to make a few changes to the API for this to work nicely, but in the end I realized they made the system a lot more consistent anyway. The fuzz testing uncovered a nice set of nasty edge-case editor bugs that were not too difficult to fix, but would have taken a lot of time to uncover with manual testing.

local function fuzz(n)
   -- need to display the seed so we can replay problematic sequences
   local seed = tonumber(os.getenv("BUSSARD_FUZZ_SEED") or os.time())
   print("seeding with", seed)
   math.randomseed(seed)

   for i=1,n do
      local mode = editor.mode()
      -- smush together all the different sub-maps (ctrl, alt, ctrl-alt)
      local commands = lume.concat(vals(mode.map), vals(mode.ctrl),
                                   vals(mode.alt), vals(mode["ctrl-alt"]))
      local command = lume.randomchoice(commands)

      print("run " .. binding_for(mode, command) .. " in mode " .. mode.name)
      try(lume.fn(editor.wrap, command))

      -- sometimes we should try inserting some text too
      if(love.math.random(5) == 1) then
         try(lume.fn(editor.handle_textinput, random_text()))
      end
   end
end

Of course, this is pretty limited in the kinds of bugs it can catch—only problems that result in crashes or hangs can be identified by the fuzz tests. But it gives me confidence when I make further changes if I can throw 32768 cycles of random commands at it without seeing it break a sweat. And it's even better when every incoming patch automatically has the testing applied against it using GitLab's CI.

Stay tuned for a second beta of Bussard to be released very soon! There is still a lot more I want to do with the story line and missions, but the engine is getting more and more polished with each milestone. Feedback is very welcome, as are contributions.

Update: I found that the fuzzer above has a critical flaw: it does not inspect the current mode's parent mode to look for commands there. (For instance, the console's parent mode is edit, and the ssh mode's parent is the console.) Fixing this immediately uncovered four new bugs.


[1] Yes, I know I just set myself up for the old "Bussard is a great OS, it just lacks a decent text editor" joke. Honestly I am just waiting for someone to come along and implement a vim mode in-game; if any player thinks they can do better than the built-in editor they are welcome to try!

[2] It's a bit tricky to get LÖVE to run headless, but it can be done. Mostly it involves disabling love.graphics and love.window modules in conf.lua and being careful with the order of loading. You also have to make sure that no calls to love.graphics functions happen outside your love.draw function.

-1:-- in which a spacefaring text editor holds up against an onslaught (Post Phil Hagelberg)--L0--C0--July 20, 2016 05:48 PM

Bryan Murdock: Another SystemVerilog Streaming Example: Size Mismatch

I had a packed struct who's size was not evenly divisible by 8 (it was one bit short, in fact) and I had an array of bytes that I needed to stream into it. The extra bits in the array of bytes were not relevant, so I tried just doing this:

my_struct = {>>byte{my_array_of_bytes}};

But my simulator complained that my_array_of_bytes was bigger than the destination (my_struct). It took me longer to figure out than I'd like to admit that I just needed to do this:

bit extra_bit;
{my_struct, extra_bit} = {>>byte{my_array_of_bytes}};

That did the trick.
-1:-- Another SystemVerilog Streaming Example: Size Mismatch (Post Bryan (noreply@blogger.com))--L0--C0--July 19, 2016 04:44 PM

Ivan Kanis: Nano editor easter egg

Here is how to display the easter egg in nano. Run nano and enter some text. Press C-x, press y, enter "zzy" as the file name. I will not say what happens next as it spoils the fun.

As far as I know Emacs does not have one. Well there is M-x butterfly...

-1:-- Nano editor easter egg (Post Ivan Kanis)--L0--C0--July 18, 2016 12:00 AM

Endless Parentheses: A quick guide to directory-local (or project-specific) variables

One of the questions we get most often about CIDER is “can I configure X on a per-project basis?”. Occasionally, you find someone suggesting (or even implementing) some sophisticated configurable variable trying to account for multiple simultaneous use-cases. Fortunately that’s one effort we don’t need to make. Emacs already has that built-in in the form of directory-local variables (dir-local for short).

As the name implies, dir-local variable values (can) apply to all files inside a given directory (also applying recursively to its sub-directories), though you can also restrict them by major-mode or by subdirectory. To configure it:

  1. Invoke M-x add-dir-local-variable.
  2. This will prompt you for a major-mode, then a variable name, and finally a variable value.
  3. Finally, it will drop you in a (newly-created) file called .dir-locals.el in the current directory. Just save this file and you’re done!

By doing this, you have configured the given variable to always have the provided value anywhere inside the current directory, so make sure you’re at the root of your project when you call the command. If you use Projectile, there’s the projectile-edit-dir-locals command for doing just that.

Also worth noting:

  • If you type nil for the major-mode, it applies to all major-modes.
  • There is (of course) tab-completion in the variable-name prompt.
  • If you answer eval for the variable name, it will prompt you for a lisp expression, instead of a value. This expression will be saved and will be evaluated every time a file is visited.
  • After doing this, call revert-buffer on any previously-open files to apply the new value.

Update 21 Jul 2016

It’s worth mentioning that Joel McCracken posted a similar quick guide a few years ago. Some of the information is the same, but some is complementary, so you might want to have a look.

Comment on this.

-1:-- A quick guide to directory-local (or project-specific) variables (Post)--L0--C0--July 05, 2016 12:00 AM

Ivan Kanis: maybe

maybe

climb old man of hoy

write clippy.el

i♥emacs emacs for beginners

</example>

The first section is where I capture stuff. My weekly review is a natural process. I start at the top and type 'w' to refile. Using speed keys really help. If you don't know about it check out the documentation in the function org-speed-command-help.

Then I go through each section and see what needs to be done. I tried to be ruthless and put stuff in someday or maybe so that my list is current and feels fresh.

I then finish with things that are in someday (one day I will do it) to maybe (who knows?).

That pretty much sums up how I implement GTD with org mode.

-1:-- maybe (Post Ivan Kanis)--L0--C0--July 04, 2016 03:23 AM