Irreal: Scientific Writing with Zotero and Org Mode

Michael Behr has an interesting post on his scientific writing workflow. The post isn’t really about how he writes his papers but rather how he organizes and curates his research materials. As you’d expect, he uses an Org file to store links to research papers and take notes on them. If you’re an Org mode user you won’t have any difficulty imagining what this looks like.

What’s interesting about his workflow is that he use Zotero to capture and store copies of the papers he needs for his research. I used Zotero long ago when I was still using Vim and Zotero existed solely as a Firefox plugin. These days, it’s available as a standalone application as well so even if you’re not a Firefox user, you can still take advantage of it. One very nice aspect of Zotero is that it has Emacs integration so that you can interact with it from within Emacs. Behr’s post describes how this works and how he uses it.

I’m not sure how necessary Zotero is if you want a similar workflow. You can easily store the papers and link to them from within Org files without a separate application. You could, for example, use the org-attach command (see this video for an excellent introduction) to store and associate copies of the papers with an Org file. It appears that Zotero does make it a bit easier to retrieve papers but I’m pretty sure that a bit of Elisp could easily duplicate its capabilities. On the other hand, Zotero is already available and has nice integration so if you don’t mind the separate utility, it’s a good solution.

If you’re a student or researcher who needs to keep tract of papers, be sure to take a look at Behr’s post. His workflow is interesting and may give you some ideas.

-1:-- Scientific Writing with Zotero and Org Mode (Post jcs)--L0--C0--December 05, 2016 06:59 PM

Chen Bin (redguardtoo): Use Perforce inside Emacs

Perforce is a proprietary VCS.

1 Perforce workflow

  • p4 set to set up
  • p4 login to login
  • p4 client creates a new work space to tell server which files to check out or to ignore
  • p4 sync ... to download files
  • Files are read-only by default. You need p4 edit file to make files writable before editing
  • p4 add files to add new files. p4 revert to revert edited file to it original status and lose the local changes
  • p4 change to create a pending change. Then p4 submit -c #changelist to actually submit code to main branch. Pending change gives you a chance to tweak the change before submit
  • Or p4 submit -d"description" file to submit a single file directly

2 My solution

Perforce cygwin portable is not recommended.

I suggest using p4's windows native executable on both Cygwin and Windows.

Perforce server has one unique URL for every physical file on the disk. If I only feed p4 that URL, the operation is always successful.

BTW, I'm confident that my solution is better.

2.1 Emacs

I provides p4edit, p4revert, and p4submit to edit/revert/submit current file,

;; {{ perforce utilities
(defvar p4-file-to-url '("" "")
  "(car p4-file-to-url) is the original file prefix
(cadr p4-file-to-url) is the url prefix")

(defun p4-generate-cmd (opts)
  (format "p4 %s %s"
          opts
          (replace-regexp-in-string (car p4-file-to-url)
                                    (cadr p4-file-to-url)
                                    buffer-file-name)))
(defun p4edit ()
  "p4 edit current file."
  (interactive)
  (shell-command (p4-generate-cmd "edit"))
  (read-only-mode -1))

(defun p4submit (&optional file-opened)
  "p4 submit current file.
If FILE-OPENED, current file is still opened."
  (interactive "P")
  (let* ((msg (read-string "Say (ENTER to abort):"))
         (open-opts (if file-opened "-f leaveunchanged+reopen -r" ""))
         (full-opts (format "submit -d '%s' %s" msg open-opts)))
    ;; (message "(p4-generate-cmd full-opts)=%s" (p4-generate-cmd full-opts))
    (if (string= "" msg)
        (message "Abort submit.")
      (shell-command (p4-generate-cmd full-opts))
      (unless file-opened (read-only-mode 1))
      (message (format "%s submitted."
                       (file-name-nondirectory buffer-file-name))))))

(defun p4revert ()
  "p4 revert current file."
  (interactive)
  (shell-command (p4-generate-cmd "revert"))
  (read-only-mode 1))
;; }}

(defun prog-mode-hook-setup ()
  (when (string-match-p "DIR/PROJ1/"
                        (if buffer-file-name buffer-file-name ""))
    (setq-local p4-file-to-url '("^.*DIR/PROJ1"
                                 "//depot/development/DIR/PROJ1"))))
(add-hook 'prog-mode-hook prog-mode-hook-setup)

As a bonus tip, if you use find-file-in-project, insert below code into prog-mode-hook to view any perforce change inside Emacs,

(setq-local ffip-diff-backends
            '((ivy-read "p4 change to show:"
                        (split-string (shell-command-to-string "p4 changes //depot/development/DIR/PROJ1/...")
                                      "\n")
                        :action (lambda (i)
                                  (if (string-match "^ Change \\([0-9]*\\)" i)
                                      (shell-command-to-string (format "p4 describe -du -db %s"
                                                                       (match-string 1 i))))))
              "p4 diff -du -db //depot/development/DIR/PROJ1/..."))

2.2 Bash Shell

Other operations are finished in Bash Shell,

# {{ Perforce, I hope I will never use it
if [ "$OS_NAME" = "CYGWIN" ]; then
    function p4() {
        export PWD=`cygpath -wa .`
        /cygdrive/c/Program\ Files/Perforce/p4.exe $@
    }
fi

# p4 workflow:
#
#   # basic setup
#   p4 set P4CLIENT=clientname  # set your default client
#   p4 set P4PORT=SERVER:1666
#   p4 set P4USER=username
#   p4 client # create/edit client, client views selected files
#
#   # checkout code
#   p4 sync [-f] //depot/project-name/path/...
#   p4 edit file[s]
#   ... do some editing ...
#
#   # submit code
#   either `p4 submit -d"say hi" file` or `p4 change`
#
#   I recommend `p4 change` because you can edit files list before submit happens.
#   After `p4 change`,  `p4 submit -c changelist#` to actually submit change.
#
alias p4clr='p4 diff -sr | p4 -x - revert' # like `git reset HEAD`
alias p4blame='p4 annotate -c -db ' # could add -a see deleted lines
alias p4cr='p4 submit -f leaveunchanged+reopen -r'
alias reviewcl='ccollab addchangelist new'
alias p4pending='p4 changes -s pending' # add ... for current directory
alias p4untrack='find . -type f| p4 -x - fstat >/dev/null'
alias p4v='p4 resolve' # after `p4 sync ...`, maybe resolve
alias p4r='p4 revert' # discard changes
alias p4e='p4 edit'
alias p4s='p4 submit'
alias p4sr='p4 submit -f submitunchanged+reopen' #submit&reopen
alias p4up='p4 sync ...' # synchronize from current directory
alias p4o='p4 opened' # list opened files
alias p4c='p4 changes' # create a new pending change
alias p4chg='p4 change' # create a pending change
alias p4d='p4 diff -du -db'
alias p4ds='p4 diff -du -db | lsdiff' # diff summary, patchutils required
alias p4i='p4 integrate'
alias p4unsh='p4 unshelve -s' # Usage: p4unsh changelist#, like `git stash apply`
alias p4h='p4 changes -m 1 ...' # show the head change

function p4mypending {
    local P4USERNAME="`p4 user -o | grep '^User:\s' | sed 's/User:\s\([a-bA-B0-9]*\)/\1/g'`"
    p4 changes -s pending -u $P4USERNAME
}

function p4shelved {
    local P4USERNAME="`p4 user -o | grep '^User:\s' | sed 's/User:\s\([a-bA-B0-9]*\)/\1/g'`"
    p4 changes -s shelved -u $P4USERNAME # add ... for current directory
}

function p4cmp {
    if [ -z "$1" ]; then
        echo "Usage: p4cmp changelist-number changelist-number"
    else
        p4 diff2 -dub -q -u ...@$1 ...@$2
    fi
}

function p4dl {
    # git diff
    p4 diff -du -db $@ | vim -c "set syntax=diff" -R -
}
function p4sh(){
    # show specific change or the latest change
    if [ -z "$1" ]; then
        p4 changes | python ~/bin/percol.py | awk '{print $2}' | xargs -i p4 describe -du {} | vim -c "set syntax=diff" -R -
    else
        p4 describe -du -db $@ | vim -c "set syntax=diff" -R -
    fi
}

function p4lp {
    #like `git log -p`
    p4 changes $@ | awk '{print $2}' | xargs -i p4 describe -du {} | less -F
}

function p4mlp {
    #like `git log -p`
    p4 changes -u $P4USERNAME $@ | awk '{print $2}' | xargs -i p4 describe -du {} | less -F
}

function p4adddir(){
    if [ -z "$1" ]; then
        echo "Usage: p4adddir directory"
    else
        find $1 -type f -print | p4 -x - add
    fi
}

# p4's suggestion,http://kb.perforce.com/article/27/creating-release-notes
# @google "assing variable from bash to perl in a bash script"
function p4l(){
    # p4 log
    if [ -z "$1" ]; then
        # show the full log
        p4 changes -l ... | less
    else
        # p4log since-changelist-number
        p4 changes -l ...@$1,#head|perl -pe "if(\$_=~/$1/){ last;};"
    fi
}

function p4ml(){
    # my p4 log
    if [ -z "$1" ]; then
        # show the full log
        p4 changes -l -u $P4USERNAME ... | less
    else
        # p4log since-changelist-number
        p4 changes -l -u $P4USERNAME ...@$1,#head|perl -pe "if(\$_=~/$1/){ last;};"
    fi
}
# }}

-1:-- Use Perforce inside Emacs (Post Chen Bin)--L0--C0--December 05, 2016 11:51 AM

Pragmatic Emacs: Use org-mode tables and structures in emails and elsewhere

I love the way that org-mode allows you to add simple clean structures to your text, with lists and tables. You can get some of that functionality in other modes by using orgstruct-mode and orgtbl-mode, which are part of org-mode.

Enable these minor modes in any major mode for one-off use with M-x orgstruct++-mode or M-x orgtbl-mode and you can use the normal org-mode commands to create lists and tables. I find this especially useful in emails, so I use this code in my emacs config file to automatically enable these for message-mode

;; use org structures and tables in message mode
(add-hook 'message-mode-hook 'turn-on-orgtbl)
(add-hook 'message-mode-hook 'turn-on-orgstruct++)
-1:-- Use org-mode tables and structures in emails and elsewhere (Post Ben Maughan)--L0--C0--December 05, 2016 12:15 AM

Irreal: Zamansky 24: Org Capture (Part 2)

Mike Zamansky has another video up in his Using Emacs series. This one is a continuation of his last video on Org Mode capture. Most of the video explains how Zamansky pops up an Org capture buffer even if he’s not in Emacs. I use a similar method all the time and find it really useful. If I want to make a quick note or capture a browser link for a blog post, I just pop up the capture menu and use the appropriate template.

Zamansky’s method is better than mine because his pops up a temporary frame that is deleted when he saves the capture and leaves him back where he started. Mine switches to Emacs and displays the menu but I have to switch back to whatever I was doing after I save the capture. It’s a bit harder to pop up a capture buffer with MacOS because you have to use AppleScript. I think if I install a keyboard macro app I can duplicate Zamansky’s method so I’ll probably give that a try.

The other nice trick I learned from the video is obvious but I never thought of it. Zamansky maintains an Org file of useful browser links along with helpful commentary. He exports this to HTML and then saves the file as a browser bookmark. That way, he can click on the book mark to bring up the page of his links and then click on the appropriate link to visit the page he’s interested in. As I said, it’s obvious in retrospect but a great tip if you haven’t considered it.

-1:-- Zamansky 24: Org Capture (Part 2) (Post jcs)--L0--C0--December 04, 2016 08:09 PM

Marcin Borkowski: Making C-c C-j in AUCTeX do something more useful

In AUCTeX’s LaTeX mode, C-c C-j is by default bound to LaTeX-insert-item. By default, it just inserts an \item on a new line. I thought that it would be cool to make it insert something else in non-itemize environments. It turns out that I was late, though – the functionality is already there! It is not widely known, however; in particular, it’s not mentioned in the manual.
-1:-- Making C-c C-j in AUCTeX do something more useful (Post)--L0--C0--December 04, 2016 07:05 PM

Timo Geusch: What happened to XEmacs?

I used XEmacs quite a lot in the 2000s before I switched back to the more stable GNU Emacs. That was back then before GNU Emacs offered a stable official Windows build when XEmacs did, and at the time I was doing a lot of Windows development. Out of curiosity and for some research I … Continue reading "What happened to XEmacs?"

The post What happened to XEmacs? appeared first on The Lone C++ Coder's Blog.

-1:-- What happened to XEmacs? (Post Timo Geusch)--L0--C0--December 03, 2016 06:40 PM

Alex Schroeder: Emacs Avent of Code Day 2

First question of day 2: You start on key 5 on a keypad as shown below. With instructions to go up, down, left or right. Ignore instructions that would take you off the keypad.

1 2 3
4 5 6
7 8 9

The example input provided for the four numbers is shown below. At the end of each line you’ll end up on the key which is part of the answer. In this example the correct answer is 1985.

ULL
RRDDD
LURDL
UUUUD

Code:

(let ((pos 5)
      (code nil)
      (instructions '((U L L)
		      (R R D D D)
		      (L U R D L)
		      (U U U U D))))
  (dolist (instruction instructions)
    (dolist (move instruction)
      (case move
	((U) (when (> pos 3) (setq pos (- pos 3))))
	((D) (when (< pos 7) (setq pos (+ pos 3))))
	((L) (when (not (= (mod pos 3) 1)) (setq pos (- pos 1))))
	((R) (when (> (mod pos 3) 0) (setq pos (+ pos 1))))))
    (push pos code))
  (reverse code))

For the second star, I was too lazy to figure out how to compute the next position and decided to implement tables instead:

    1
  2 3 4
5 6 7 8 9
  A B C
    D

Given a direction, I just list all the possible positions and the results. It’s crude, but it works.

(let ((pos 5)
      (code nil)
      (instructions '((U L L)
		      (R R D D D)
		      (L U R D L)
		      (U U U U D))))
(dolist (instruction instructions)
    (dolist (move instruction)
      (setq pos (funcall move pos)))
    (push pos code))
  (reverse code))

(defun U (pos)
  (case pos
    ((3) 1)
    ((6) 2)
    ((7) 3)
    ((8) 4)
    ((A) 6)
    ((B) 7)
    ((C) 8)
    ((D) 'B)
    (t pos)))

(defun D (pos)
  (case pos
    ((1) 3)
    ((2) 6)
    ((3) 7)
    ((4) 8)
    ((6) 'A)
    ((7) 'B)
    ((8) 'C)
    ((B) 'D)
    (t pos)))

(defun L (pos)
  (case pos
    ((6) 5)
    ((3) 2)
    ((7) 6)
    ((B) 'A)
    ((4) 3)
    ((8) 7)
    ((C) 'B)
    ((9) 8)
    (t pos)))

(defun R (pos)
  (case pos
    ((5) 6)
    ((2) 3)
    ((6) 7)
    ((A) 'B)
    ((3) 4)
    ((7) 8)
    ((B) 'C)
    ((8) 9)
    (t pos)))

Tags:

-1:-- Emacs Avent of Code Day 2 (Post)--L0--C0--December 03, 2016 12:13 AM

Manuel Uberti: Format XML in Emacs

Since I deal with many SOAP web services for work, I also have to deal with lots of XML files. In the application server log they usually appear on one line, which makes the reading unnecessarily painful. Fortunately, Emacs coupled with the handy xmllint comes to the rescue.

First, I installed xmllint with: sudo apt-get install libxml2-utils

Now I could format my XML buffers with: C-x h C-u M-| xmllint -format - RET

However, this is Emacs, so there is always a more productive way to solve problems. Why should I have to input the options for xmllint or remember the necessary key bindings all the times?

(defun mu-xml-format ()
  "Format an XML buffer with `xmllint'."
  (interactive)
  (shell-command-on-region (point-min) (point-max)
                           "xmllint -format -"
                           (current-buffer)
                           t
                           "*Xmllint Error Buffer*"
                           t))

Since I only need this utility in XML buffers, I bound it for nxml-mode-map.

(use-package nxml-mode                  ; XML editing
  :mode "\\.xml\\'"
  :bind (:map nxml-mode-map
              ("C-c m f" . mu-xml-format))

-1:-- Format XML in Emacs (Post)--L0--C0--December 03, 2016 12:00 AM

William Denton: Emacs in Conforguration

On top of Ruby I now have Emacs itself set up in Conforguration. On a new bare machine I can git clone two GitHub repositories, run some scripts, and after a lot of compiling my entire Emacs setup will be ready and waiting. (Note: works on Ubuntu, and probably other Debian-based systems but I don’t know for sure.)

cd
sudo apt-get install git
git clone git@github.com:wdenton/.emacs.d.git
mkdir -p src
cd src
git clone git@github.com:wdenton/conforguration.git
cd conforguration/dotfiles
./dotfile-setup.sh
source ~/.bashrc
cd ../scripts
./initialize.sh
./emacs-install-requirements.sh
./emacs-install-from-source.sh

Then running emacs will download and install a couple of dozen packages. When that’s done you’ll see exactly what my Emacs is like. If you don’t want to go that far, skip the .emacs.d repository download, but make sure your PATH is set up to point to Emacs in the right place under /usr/local/src/.

Once Emacs is installed you can run the Conforguration scripts inside it, but it’s nice they also work outside, for those scary early minutes on a new machine when Emacs is not installed.

-1:-- Emacs in Conforguration (Post William Denton)--L0--C0--December 02, 2016 03:09 AM

Chris Wellons: A Showerthoughts Fortune File

I have created a fortune file for the all-time top 10,000 /r/Showerthoughts posts, as of October 2016. As a word of warning: Many of these entries are adult humor and may not be appropriate for your work computer. These fortunes would be categorized as “offensive” (fortune -o).

Download: showerthoughts (1.3 MB)

The copyright status of this file is subject to each of its thousands of authors. Since it’s not possible to contact many of these authors — some may not even still live — it’s obviously never going to be under an open source license (Creative Commons, etc.). Even more, some quotes are probably from comedians and such, rather than by the redditor who made the post. I distribute it only for fun.

Installation

To install this into your fortune database, first process it with strfile to create a random-access index, showerthoughts.dat, then copy them to the directory with the rest.

$ strfile showerthoughts
"showerthoughts.dat" created
There were 10000 strings
Longest string: 343 bytes
Shortest string: 39 bytes

$ cp showerthoughts* /usr/share/games/fortunes/

Alternatively, fortune can be told to use this file directly:

$ fortune showerthoughts
Not once in my life have I stepped into somebody's house and
thought, "I sure hope I get an apology for 'the mess'."
        ―AndItsDeepToo, Aug 2016

If you didn’t already know, fortune is an old unix utility that displays a random quotation from a quotation database — a digital fortune cookie. I use it as an interactive login shell greeting on my ODROID-C2 server:

if shopt -q login_shell; then
    fortune ~/.fortunes
fi

How was it made?

Fortunately I didn’t have to do something crazy like scrape reddit for weeks on end. Instead, I downloaded the pushshift.io submission archives, which is currently around 70 GB compressed. Each file contains one month’s worth of JSON data, one object per submission, one submission per line, all compressed with bzip2.

Unlike so many other datasets, especially when it’s made up of arbitrary inputs from millions of people, the format of the /r/Showerthoughts posts is surprisingly very clean and requires virtually no touching up. It’s some really fantastic data.

A nice feature of bzip2 is concatenating compressed files also concatenates the uncompressed files. Additionally, it’s easy to parallelize bzip2 compression and decompression, which gives it an edge over xz. I strongly recommend using lbzip2 to decompress this data, should you want to process it yourself.

cat RS_*.bz2 | lbunzip2 > everything.json

jq is my favorite command line tool for processing JSON (and rendering fractals). To filter all the /r/Showerthoughts posts, it’s a simple select expression. Just mind the capitalization of the subreddit’s name. The -c tells jq to keep it one per line.

cat RS_*.bz2 | \
    lbunzip2 | \
    jq -c 'select(.subreddit == "Showerthoughts")' \
    > showerthoughts.json

However, you’ll quickly find that jq is the bottleneck, parsing all that JSON. Your cores won’t be exploited by lbzip2 as they should. So I throw grep in front to dramatically decrease the workload for jq.

cat *.bz2 | \
    lbunzip2 | \
    grep -a Showerthoughts | \
    jq -c 'select(.subreddit == "Showerthoughts")'
    > showerthoughts.json

This will let some extra things through, but it’s a superset. The -a option is necessary because the data contains some null bytes. Without it, grep switches into binary mode and breaks everything. This is incredibly frustrating when you’ve already waited half an hour for results.

To further reduce the workload further down the pipeline, I take advantage of the fact that only four fields will be needed: title, score, author, and created_utc. The rest can — and should, for efficiency’s sake — be thrown away where it’s cheap to do so.

cat *.bz2 | \
    lbunzip2 | \
    grep -a Showerthoughts | \
    jq -c 'select(.subreddit == "Showerthoughts") |
               {title, score, author, created_utc}' \
    > showerthoughts.json

This gathers all 1,199,499 submissions into a 185 MB JSON file (as of this writing). Most of these submissions are terrible, so the next step is narrowing it to the small set of good submissions and putting them into the fortune database format.

It turns out reddit already has a method for finding the best submissions: a voting system. Just pick the highest scoring posts. Through experimentation I arrived at 10,000 as the magic cut-off number. After this the quality really starts to drop off. Over time this should probably be scaled up with the total number of submissions.

I did both steps at the same time using a bit of Emacs Lisp, which is particularly well-suited to the task:

This Elisp program reads one JSON object at a time and sticks each into a AVL tree sorted by score (descending), then timestamp (ascending), then title (ascending). The AVL tree is limited to 10,000 items, with the lowest items being dropped. This was a lot faster than the more obvious approach: collecting everything into a big list, sorting it, and keeping the top 10,000 items.

Formatting

The most complicated part is actually paragraph wrapping the submissions. Most are too long for a single line, and letting the terminal hard wrap them is visually unpleasing. The submissions are encoded in UTF-8, some with characters beyond simple ASCII. Proper wrapping requires not just Unicode awareness, but also some degree of Unicode rendering. The algorithm needs to recognize grapheme clusters and know the size of the rendered text. This is not so trivial! Most paragraph wrapping tools and libraries get this wrong, some counting width by bytes, others counting width by codepoints.

Emacs’ M-x fill-paragraph knows how to do all these things — only for a monospace font, which is all I needed — and I decided to leverage it when generating the fortune file. Here’s an example that paragraph-wraps a string:

(defun string-fill-paragraph (s)
  (with-temp-buffer
    (insert s)
    (fill-paragraph)
    (buffer-string)))

For the file format, items are delimited by a % on a line by itself. I put the wrapped content, followed by a quotation dash, the author, and the date. A surprising number of these submissions have date-sensitive content (“on this day X years ago”), so I found it was important to include a date.

April Fool's Day is the one day of the year when people critically
evaluate news articles before accepting them as true.
        ―kellenbrent, Apr 2015
%
Of all the bodily functions that could be contagious, thank god
it's the yawn.
        ―MKLV, Aug 2015
%

There’s the potential that a submission itself could end with a lone % and, with a bit of bad luck, it happens to wrap that onto its own line. Fortunately this hasn’t happened yet. But, now that I’ve advertised it, someone could make such a submission, popular enough for the top 10,000, with the intent to personally trip me up in a future update. I accept this, though it’s unlikely, and it would be fairly easy to work around if it happened.

The strfile program looks for the % delimiters and fills out a table of file offsets. The header of the .dat file indicates the number strings along with some other metadata. What follows is a table of 32-bit file offsets.

struct {
    uint32_t str_version;  /* version number */
    uint32_t str_numstr;   /* # of strings in the file */
    uint32_t str_longlen;  /* length of longest string */
    uint32_t str_shortlen; /* shortest string length */
    uint32_t str_flags;    /* bit field for flags */
    char str_delim;        /* delimiting character */
}

Note that the table doesn’t necessarily need to list the strings in the same order as they appear in the original file. In fact, recent versions of strfile can sort the strings by sorting the table, all without touching the original file. Though none of this important to fortune.

Now that you know how it all works, you can build your own fortune file from your own inputs!

-1:-- A Showerthoughts Fortune File (Post)--L0--C0--December 01, 2016 11:58 PM

Alex Schroeder: Emacs Advent of Code

I looked at Advent of Code and foolishly decided to solve the first riddle. Perhaps we’ll learn something about Elisp coding style? Feel free to leave comments regarding the code.

You must follow instructions: R or L means turn left or right, followed by a number of steps to take. What’s the final distance in steps? Example: “R5, L5, R5, R3 leaves you 12 blocks away.”

(let ((instructions (mapcar (lambda (s)
			      (cons (aref s 0)
				    (string-to-number (substring s 1))))
			    (split-string "R5, L5, R5, R3")))
      (direction 0)
      (x 0)
      (y 0))
  (dolist (instruction instructions)
    (destructuring-bind (turn . steps) instruction
      (setq direction (mod (funcall (if (= turn ?R) '1+ '1-) direction) 4))
      (case direction
	(0 (setq y (+ y steps)))
	(1 (setq x (+ x steps)))
	(2 (setq y (- y steps)))
	(3 (setq x (- x steps))))))
  (+ (abs x) (abs y)))

Sadly, for the second star, my solution fails… The question is now: which location was visited twice? The example provided says “if your instructions are R8, R4, R4, R8, the first location you visit twice is 4 blocks away, due East.” The code works for the example given but fails for the data I was provided with in the test.

(let ((instructions (mapcar (lambda (s)
			      (cons (aref s 0)
				    (string-to-number (substring s 1))))
			    (split-string "R8, R4, R4, R8")))
      (direction 0)
      (x 0)
      (y 0)
      (last-x 0)
      (last-y 0)
      (seen nil)
      (final-x nil)
      (final-y nil))
  (or (catch 'twice
	(dolist (instruction instructions)
	  (destructuring-bind (turn . steps) instruction
	    (setq direction (mod (funcall (if (= turn ?R) '1+ '1-) direction) 4))
	    (case direction
	      (0 (setq y (+ y steps)))
	      (1 (setq x (+ x steps)))
	      (2 (setq y (- y steps)))
	      (3 (setq x (- x steps)))))
	  (dolist (line (cdr seen))	; skip the last element
	    (destructuring-bind ((x1 . y1) (x2 . y2)) line
	      (when (or (and (= x last-x) ; last move was vertical 
			     (= y1 y2)	  ; looking at a horizontal
			     (>= x x1)  (<= x x2)
			     (or (and (<= y y1)  (>= last-y y1))
				 (and (>= y y1)  (<= last-y y1)))
			     (setq final-x x
				   final-y y1))
			(and (= y last-y) ; last move was horizontal
			     (= x1 x2)	  ; looking at a vertical
			     (>= y y1)  (<= y y2)
			     (or (and (<= x x1)  (>= last-x x1))
				 (and (>= x x1)  (<= last-x x1)))
			     (setq final-x x1
				   final-y y)))
		;; (error "Moving from %S to %S crosses the line from %S to %S in %S"
		;; 	   (cons last-x last-y) (cons x y)
		;; 	   (cons x1 y1) (cons x2 y2)
		;; 	   seen)
		(throw 'twice (+ (abs final-x) (abs final-y))))))
	  (push (list (cons last-x last-y) (cons x y)) seen)
	  (setq last-x x
		last-y y)))
      (+ (abs x) (abs y))))

Well, I decided to brute force it and remember every point I moved through because my “crossing lines” solution was obviously off by 3. Perhaps I just reported the wrong result, who knows.

(let ((instructions (mapcar (lambda (s)
			      (cons (aref s 0)
				    (string-to-number (substring s 1))))
			    (split-string "R8, R4, R4, R8")))
      (direction 0)
      (x 0)
      (y 0)
      (last-x 0)
      (last-y 0)
      (seen '((0 . 0))))
  (or (catch 'twice
	(dolist (instruction instructions)
	  (destructuring-bind (turn . steps) instruction
	    (setq direction (mod (funcall (if (= turn ?R) '1+ '1-) direction) 4))
	    (case direction
	      (0 (setq y (+ y steps)))
	      (1 (setq x (+ x steps)))
	      (2 (setq y (- y steps)))
	      (3 (setq x (- x steps)))))
	  (while (not (and (= last-x x) (= last-y y)))
	    (if (not (= last-x x))
		(setq last-x (funcall (if (< last-x x) '1+ '1-) last-x))
	      (setq last-y (funcall (if (< last-y y) '1+ '1-) last-y)))
	    (let ((last (cons last-x last-y)))
	      (if (member last seen)
		  (error "Repetition seen: %S in %S" last seen)
		(push last seen))))))
      (error "No repetitions in %S" seen)))

Tags:

-1:-- Emacs Advent of Code (Post)--L0--C0--December 01, 2016 10:19 AM

William Denton: Ruby in Conforguration

I got a new laptop at work, and while configuring it I’m updating Conforguration with some things I did. First up: scripts to install Ruby with rbenv. (I’m switching to it from RVM because RVM was getting on my nerves a bit, and rbenv is the standard at work.)

It needs a tweak so it’s not so brute force, but it works.

-1:-- Ruby in Conforguration (Post William Denton)--L0--C0--November 30, 2016 04:04 AM

emacspeak: Follow-Up: Soundscapes On The Emacspeak Audio Desktop

Follow-Up: Soundscapes On The Emacspeak Audio Desktop

1 Executive Summary

Nearly a year ago, I blogged here about Soundscapes on the emacspeak
Audio Desktop. That article ended with this following paragraph:


I implemented package soundscape to create a platform that would let me experiment
with different tools that aid in concentration. After using Soundscapes for about a week,
I have also found that it reduces some of the fatigue that results from having to listen
to synthetic text-to-speech for extended periods. The true value (if any) of this package
will be a function of how heavily I find myself using it six months from now --- as a
metric, complete success might mean that in mid-2016, I still have automatic soundscapes
turned on.


2 And Nearly A Year Later …

I have not found the need to turn off Soundscapes in Emacspeak. As
conjectured, it has definitely increased my productivity, specifically
in terms of staying focused on a given task at hand. Over the year,
I've also augmented the emacspeak Audio Desktop with support for
binaural audio — see module sox-gen — which provides a collection of
binaural themes for use during different times of the day. Binaural
themes generated by that module overlay Emacspeak Soundscapes to
provide an ideal auditory environment for use over headphones.


3 Soundscape Enhancements

Since the publication of the original article, Emacspeak Soundscapes
have been enhanced with additional sounds from
Freesound.org. Emacspeak Soundscapes have been updated to take
advantage of Boodler's limited abilities in the areas of spatial
positioning. I typically use Soundscapes with one of several virtual
ALSA devices that have been configured to apply different Ladspa
effects such as reverb or crossfeed depending on the ambient
environment where I am working — this significantly improves the
spatialization of soundscapes being played — see file
ladspa-asoundrc. Finally, the mapping of Soundscapes to various Emacs
modes has also been tuned. — see table below.



Soundscape (Mood) List Of Major Modes
BirdSongs shell term
BlopEchoes elfeed-search
Bonfire calendar diary
BuddhaLoop comint
Cavern prog
ChangingLoops special
ChangingLoopsPitches lisp-interaction
Drip message gnus-summary gnus-article gnus-group mspools vm-presentation vm mail twittering jabber-roster jabber-chat erc
LoopStew emacspeak-m-player
NoStormYet fundamental
RainForever Info help Man Custom messages-buffer
RainSounds magit vc
Still text view
SurfWaves w3 eww
TonkSpace tabulated-list
WaterFlow dired


4 Summary

As outlined in a previous article, sound on Linux provides unending
possibilities with respect to innovation, here's looking forward to
better things to come.

-1:-- Follow-Up: Soundscapes On The Emacspeak Audio Desktop (Post T. V. Raman (noreply@blogger.com))--L0--C0--November 29, 2016 11:06 PM

emacshorrors: When Data Becomes Code

If you’ve hung out on #emacs for a while, chances are you’ve been recommended ibuffer for advanced buffer management (like, killing many buffers at once). There is a horror lurking beneath its pretty interface though and it starts out with a customizable:

(defcustom ibuffer-always-compile-formats (featurep 'bytecomp)
  "If non-nil, then use the byte-compiler to optimize `ibuffer-formats'.
This will increase the redisplay speed, at the cost of loading the
elisp byte-compiler."
  :type 'boolean
  :group 'ibuffer)

Uh-oh:

(defun ibuffer-compile-make-eliding-form (strvar elide from-end-p)
  (let ((ellipsis (propertize ibuffer-eliding-string 'font-lock-face 'bold)))
    (if (or elide (with-no-warnings ibuffer-elide-long-columns))
        `(if (> strlen 5)
             ,(if from-end-p
                  ;; FIXME: this should probably also be using
                  ;; `truncate-string-to-width' (Bug#24972)
                  `(concat ,ellipsis
                           (substring ,strvar
                                      (string-width ibuffer-eliding-string)))
                `(concat
                  (truncate-string-to-width
                   ,strvar (- strlen (string-width ,ellipsis)) nil ?.)
                  ,ellipsis))
           ,strvar)
      strvar)))

(defun ibuffer-compile-make-substring-form (strvar maxvar from-end-p)
  (if from-end-p
      ;; FIXME: not sure if this case is correct (Bug#24972)
      `(truncate-string-to-width str strlen (- strlen ,maxvar) nil ?\s)
    `(truncate-string-to-width ,strvar ,maxvar nil ?\s)))

(defun ibuffer-compile-make-format-form (strvar widthform alignment)
  (let* ((left `(make-string tmp2 ?\s))
         (right `(make-string (- tmp1 tmp2) ?\s)))
    `(progn
       (setq tmp1 ,widthform
             tmp2 (/ tmp1 2))
       ,(pcase alignment
          (:right `(concat ,left ,right ,strvar))
          (:center `(concat ,left ,strvar ,right))
          (:left `(concat ,strvar ,left ,right))
          (_ (error "Invalid alignment %s" alignment))))))

(defun ibuffer-compile-format (format)
  (let ((result nil)
        ;; We use these variables to keep track of which variables
        ;; inside the generated function we need to bind, since
        ;; binding variables in Emacs takes time.
        (vars-used ()))
    (dolist (form format)
      (push
       ;; Generate a form based on a particular format entry, like
       ;; " ", mark, or (mode 16 16 :right).
       (if (stringp form)
           ;; It's a string; all we need to do is insert it.
           `(insert ,form)
         (let* ((form (ibuffer-expand-format-entry form))
                (sym (nth 0 form))
                (min (nth 1 form))
                (max (nth 2 form))
                (align (nth 3 form))
                (elide (nth 4 form)))
           (let* ((from-end-p (when (cl-minusp min)
                                (setq min (- min))
                                t))
                  (letbindings nil)
                  (outforms nil)
                  minform
                  maxform
                  min-used max-used strlen-used)
             (when (or (not (integerp min)) (>= min 0))
               ;; This is a complex case; they want it limited to a
               ;; minimum size.
               (setq min-used t)
               (setq strlen-used t)
               (setq vars-used '(str strlen tmp1 tmp2))
               ;; Generate code to limit the string to a minimum size.
               (setq minform `(progn
                                (setq str
                                      ,(ibuffer-compile-make-format-form
                                        'str
                                        `(- ,(if (integerp min)
                                                 min
                                               'min)
                                            strlen)
                                        align)))))
             (when (or (not (integerp max)) (> max 0))
               (setq max-used t)
               (cl-pushnew 'str vars-used)
               ;; Generate code to limit the string to a maximum size.
               (setq maxform `(progn
                                (setq str
                                      ,(ibuffer-compile-make-substring-form
                                        'str
                                        (if (integerp max)
                                            max
                                          'max)
                                        from-end-p))
                                (setq strlen (string-width str))
                                (setq str
                                      ,(ibuffer-compile-make-eliding-form
                                        'str elide from-end-p)))))
             ;; Now, put these forms together with the rest of the code.
             (let ((callform
                    ;; Is this an "inline" column?  This means we have
                    ;; to get the code from the
                    ;; `ibuffer-inline-columns' alist and insert it
                    ;; into our generated code.  Otherwise, we just
                    ;; generate a call to the column function.
                    (ibuffer-aif (assq sym ibuffer-inline-columns)
                        (nth 1 it)
                      `(,sym buffer mark)))
                   ;; You're not expected to understand this.  Hell, I
                   ;; don't even understand it, and I wrote it five
                   ;; minutes ago.
                   (insertgenfn
                    (if (get sym 'ibuffer-column-summarizer)
                        ;; I really, really wish Emacs Lisp had closures.
                        ;; FIXME: Elisp does have them now.
                        (lambda (arg sym)
                          `(insert
                            (let ((ret ,arg))
                              (put ',sym 'ibuffer-column-summary
                                   (cons ret (get ',sym
                                                  'ibuffer-column-summary)))
                              ret)))
                      (lambda (arg _sym)
                        `(insert ,arg))))
                   (mincompform `(< strlen ,(if (integerp min)
                                                min
                                              'min)))
                   (maxcompform `(> strlen ,(if (integerp max)
                                                max
                                              'max))))
               (if (or min-used max-used)
                   ;; The complex case, where we have to limit the
                   ;; form to a maximum or minimum size.
                   (progn
                     (when (and min-used (not (integerp min)))
                       (push `(min ,min) letbindings))
                     (when (and max-used (not (integerp max)))
                       (push `(max ,max) letbindings))
                     (push
                      (if (and min-used max-used)
                          `(if ,mincompform
                               ,minform
                             (if ,maxcompform
                                 ,maxform))
                        (if min-used
                            `(when ,mincompform
                               ,minform)
                          `(when ,maxcompform
                             ,maxform)))
                      outforms)
                     (push `(setq str ,callform
                                  ,@(when strlen-used
                                      `(strlen (string-width str))))
                           outforms)
                     (setq outforms
                           (append outforms
                                   (list (funcall insertgenfn 'str sym)))))
                 ;; The simple case; just insert the string.
                 (push (funcall insertgenfn callform sym) outforms))
               ;; Finally, return a `let' form which binds the
               ;; variables in `letbindings', and contains all the
               ;; code in `outforms'.
               `(let ,letbindings
                  ,@outforms)))))
       result))
    ;; We don't want to unconditionally load the byte-compiler.
    (funcall (if (or ibuffer-always-compile-formats
                     (featurep 'bytecomp))
                 #'byte-compile
               #'identity)
             ;; Here, we actually create a lambda form which
             ;; inserts all the generated forms for each entry
             ;; in the format string.
             `(lambda (buffer mark)
                (let ,vars-used
                  ,@(nreverse result))))))

(defun ibuffer-recompile-formats ()
  "Recompile `ibuffer-formats'."
  (interactive)
  (setq ibuffer-compiled-formats
        (mapcar #'ibuffer-compile-format ibuffer-formats))
  (when (boundp 'ibuffer-filter-format-alist)
    (setq ibuffer-compiled-filter-formats
          (mapcar (lambda (entry)
                    (cons (car entry)
                          (mapcar (lambda (formats)
                                    (mapcar #'ibuffer-compile-format formats))
                                  (cdr entry))))
                  ibuffer-filter-format-alist))))

(defun ibuffer-clear-summary-columns (format)
  (dolist (form format)
    (when (and (consp form)
               (get (car form) 'ibuffer-column-summarizer))
      (put (car form) 'ibuffer-column-summary nil))))

(defun ibuffer-check-formats ()
  (when (null ibuffer-formats)
    (error "No formats!"))
  (let ((ext-loaded (featurep 'ibuf-ext)))
    (when (or (null ibuffer-compiled-formats)
              (null ibuffer-cached-formats)
              (not (eq ibuffer-cached-formats ibuffer-formats))
              (null ibuffer-cached-eliding-string)
              (not (equal ibuffer-cached-eliding-string ibuffer-eliding-string))
              (eql 0 ibuffer-cached-elide-long-columns)
              (not (eql ibuffer-cached-elide-long-columns
                        (with-no-warnings ibuffer-elide-long-columns)))
              (and ext-loaded
                   (not (eq ibuffer-cached-filter-formats
                            ibuffer-filter-format-alist))
                   (and ibuffer-filter-format-alist
                        (null ibuffer-compiled-filter-formats))))
      (message "Formats have changed, recompiling...")
      (ibuffer-recompile-formats)
      (setq ibuffer-cached-formats ibuffer-formats
            ibuffer-cached-eliding-string ibuffer-eliding-string
            ibuffer-cached-elide-long-columns (with-no-warnings ibuffer-elide-long-columns))
      (when ext-loaded
        (setq ibuffer-cached-filter-formats ibuffer-filter-format-alist))
      (message "Formats have changed, recompiling...done"))))

Another weird one is that the extracted autoloads for ibuffer-ext.el reside in ibuffer.el, but that’s the lesser evil of the two.

Credits go to holomorph for discovering that maintenance nightmare.

-1:-- When Data Becomes Code (Post Vasilij Schneidermann)--L0--C0--November 28, 2016 09:31 AM

sachachua: 2016-11-28 Emacs News

Links from reddit.com/r/emacs, /r/orgmode, /r/spacemacs, Hacker News, planet.emacsen.org, Youtube, the changes to the Emacs NEWS file, and emacs-devel.

Past Emacs News round-ups

The post 2016-11-28 Emacs News appeared first on sacha chua :: living an awesome life.

-1:-- 2016-11-28 Emacs News (Post Sacha Chua)--L0--C0--November 28, 2016 05:24 AM

Pragmatic Emacs: Scroll other window

When you use one of Emacs’ help commands, another window will open with the documentation. You can use C-M-v and C-M-V to scroll the other window down and up respectively to read the help, without having to switch to that window.

-1:-- Scroll other window (Post Ben Maughan)--L0--C0--November 28, 2016 12:05 AM

Sebastian Wiesner: Bye Bye Emojis: Emacs Hates MacOS

Two years ago Emacs 24.4 dramatically improved support for MacOS (which was still OS X back then) with a new Core Text based font rendering engine. Among better and more consistent rendering this change—with the right fontset configuration—enabled colored emoji. Not the most important feature, but nice to have and doing no harm either.

In Emacs 25.1 it is gone. What I initially thought was a mistake in my configuration turned out to be a NEWS entry (emphasis mine):

On the OS X Cocoa (“Nextstep”) port, multicolor font (such as color emoji) display is disabled. […] This will be enabled again once it is also implemented in Emacs on free operating systems. […]

Let’s sink this in: The Emacs developers deliberately disabled a feature that was working perfectly fine for MacOS users just because it is not available for free systems1. What a daft decision.


In the FSF’s little ivory tower of free software happiness this decision surely makes sense, but as a user, as a maintainer of a popular Emacs package that tries to deliver a great user interface, I feel like I’m being given the finger. It is a clear message that no matter what we contribute2 as MacOS users we will always be second-class citizen in Emacs land3.

We are not welcome, and never will be.

  1. As if it was our fault that Linux is notoriously lagging behind on desktop.

  2. I wrote all of Flycheck on a Macbook.

  3. Even though the Emacs maintainer is a MacOS user himself, and wanted better support for MacOS when he took up that position. He explicitly disagreed with that change, though.

-1:-- Bye Bye Emojis: Emacs Hates MacOS (Post Sebastian Wiesner)--L0--C0--November 21, 2016 11:00 PM

emacsninja: Brave New World

Update: Finally figured out the layout after digging a bit more into the sources, it’s a QWERTY-UK (see devices/rpi2/uspi/include/uspios.h). Looks like I’ll have to modify the bundled USPI library to include a QWERTY-US layout before I can make any progress on keyboard remapping in Lisp…

I believe I’ve found an even greater time sink than writing Lisp interpreters for fun. Long time ago, I’ve read an encouraging blog post on the future of the LispM, not expecting to find an implementation of the ideas presented therein. Turns out I was wrong about that. Meet Interim OS!

In case you’re wondering why you should possibly care:

  • Small and readable codebase (most of the code is device drivers for the Raspberry Pi)
  • Simple to hack on
  • Plan9-style APIs
  • Minimal Lisp dialect
  • Runs on your favorite desktop OS in hosted mode, that is, safely contained to a terminal with the ability to spawn graphical windows
  • Runs on bare metal (Raspberry Pi 2)

Getting it to run in hosted mode is simple enough, so I won’t explain it here. Booting on bare metal however is a different story, so here we go:

$ git clone https://github.com/mntmn/interim
$ cp interim/docs/interim-0.1.0-rpi2.tgz ./
$ bsdtar -xf interim-0.1.0-rpi2.tgz # cry me a river
% mkdir /media/boot
% mount /dev/sdXN /media/boot
% cp release-rpi2/* /media/boot/
% rm /media/boot/cmdline.txt
% umount /media/boot
  • Plug in the SDHC card, a HDMI monitor and a USB keyboard
  • Optionally: Plug in a network cable and/or a USB mouse
  • Power up

You’ll be greeted by a “Welcome to Interim OS” and dropped into a promptless shell. If you’re unlucky, the chosen resolution may be unreadable, so feel free to retry this process a few times. The keyboard layout is hardcoded and somewhere between QWERTY-US and QWERTZ-DE, something I intend to fix soon. For basic usage instructions, type (bytes->str (load "/sd/hello.txt")) and hit the enter key. Happy hacking!

-1:-- Brave New World (Post Vasilij Schneidermann)--L0--C0--November 21, 2016 09:32 AM

sachachua: 2016-11-21 Emacs News

Links from reddit.com/r/emacs, /r/orgmode, /r/spacemacs, Hacker News, planet.emacsen.org, Youtube, the changes to the Emacs NEWS file, and emacs-devel.

Past Emacs News round-ups

The post 2016-11-21 Emacs News appeared first on sacha chua :: living an awesome life.

-1:-- 2016-11-21 Emacs News (Post Sacha Chua)--L0--C0--November 21, 2016 06:48 AM

Grant Rettke: Mystery Maltron

  • Addendum: <2016-11-21 Mon>
    • Roman provided the link for this product

mystery-maltron-1.jpg

mystery-maltron-2.jpg

mystery-maltron-3.jpg

-1:-- Mystery Maltron (Post Grant)--L0--C0--November 21, 2016 03:05 AM

emacspeak: Emacspeak 45.0 (IdealDog) Unleashed!

Emacspeak 45.0—IdealDog—Unleashed!

For Immediate Release:


San Jose, Calif., (Nov 21, 2016)


Emacspeak: Redefining Accessibility In The Era Of (Real)Intelligent Computing
–Zero cost of Ownership makes priceless software Universally affordable!


Emacspeak Inc (NASDOG: ESPK) — http://github.com/tvraman/emacspeak — announces the
immediate world-wide availability of Emacspeak 45.0 (IdealDog) — a
powerful audio desktop for leveraging today's evolving data, social
and service-oriented Internet cloud.


1 Investors Note:

With several prominent tweeters expanding coverage of
#emacspeak, NASDOG: ESPK has now been consistently trading over
the social net at levels close to that once attained by DogCom
high-fliers—and as of Nov 2016 is trading at levels close to
that achieved by once better known stocks in the tech sector.


2 What Is It?

Emacspeak is a fully functional audio desktop that provides complete
eyes-free access to all major 32 and 64 bit operating environments. By
seamlessly blending live access to all aspects of the Internet such as
Web-surfing, blogging, social computing and electronic messaging into
the audio desktop, Emacspeak enables speech access to local and remote
information with a consistent and well-integrated user interface. A
rich suite of task-oriented tools provides efficient speech-enabled
access to the evolving service-oriented social Internet cloud.


3 Major Enhancements:

  • Speech-enabled tide for typescript development. 🌊
  • Speech-enabled jade for Javascript WebApp development. ⺩
  • Improved slime support for Lisp programming. Λ
  • Support for rst-mode for editting ReST files.🖹
  • Version control info in modeline.⎔
  • Speech-enabled elisp-refs to aid in refactoring. ※
  • GPG integration including pinentry support. 🔐
  • ElScreen support for window-layout management. 🆜
  • Updated Librivox client for audio books. 🔊🕮
  • Updated sound themes. 🔉
  • Support for Emacs' visual-line-mode. 🎁
  • Speech-enabled Threes game. 🎮
  • Updated Google News support. 📰
  • Binaural audio support including several predefined binaural themes. ℗
    • Updated multilingual support for ESpeak. 󠀁
  • Script etc/bootstrap.sh for bootstrapping into Emacspeak on a well-configured Linux system. 👢
  • And a lot more than wil fit this margin. … 🗞


4 Establishing Liberty, Equality And Freedom:

Never a toy system, Emacspeak is voluntarily bundled with all
major Linux distributions. Though designed to be modular,
distributors have freely chosen to bundle the fully integrated
system without any undue pressure—a documented success for
the integrated innovation embodied by Emacspeak. As the system
evolves, both upgrades and downgrades continue to be available at
the same zero-cost to all users. The integrity of the Emacspeak
codebase is ensured by the reliable and secure Linux platform
used to develop and distribute the software.


Extensive studies have shown that thanks to these features, users
consider Emacspeak to be absolutely priceless. Thanks to this
wide-spread user demand, the present version remains priceless
as ever—it is being made available at the same zero-cost as
previous releases.


At the same time, Emacspeak continues to innovate in the area of
eyes-free social interaction and carries forward the
well-established Open Source tradition of introducing user
interface features that eventually show up in luser environments.


On this theme, when once challenged by a proponent of a crash-prone
but well-marketed mousetrap with the assertion "Emacs is a system from
the 70's", the creator of Emacspeak evinced surprise at the unusual
candor manifest in the assertion that it would take popular
idiot-proven interfaces until the year 2070 to catch up to where the
Emacspeak audio desktop is today. Industry experts welcomed this
refreshing breath of Courage Certainty and Clarity (CCC) at a time
when users are reeling from the Fear Uncertainty and Doubt (FUD)
unleashed by complex software systems backed by even more convoluted
press releases.


5 Independent Test Results:

Independent test results have proven that unlike some modern (and
not so modern) software, Emacspeak can be safely uninstalled without
adversely affecting the continued performance of the computer. These
same tests also revealed that once uninstalled, the user stopped
functioning altogether. Speaking with Aster Labrador, the creator of
Emacspeak once pointed out that these results re-emphasize the
user-centric design of Emacspeak; "It is the user –and not the
computer– that stops functioning when Emacspeak is uninstalled!".


5.1 Note from Aster,Bubbles and Tilden:

UnDoctored Videos Inc. is looking for volunteers to star in a
video demonstrating such complete user failure.


6 Obtaining Emacspeak:

Emacspeak can be downloaded from GitHub –see
https://github.com/tvraman/emacspeak you can visit Emacspeak on the
WWW at http://emacspeak.sf.net. You can subscribe to the emacspeak
mailing list — emacspeak@cs.vassar.edu — by sending mail to the
list request address emacspeak-request@cs.vassar.edu. The Emacspeak
Blog
is a good source for news about recent enhancements and how to
use them.


The latest development snapshot of Emacspeak is always available via
Git from GitHub at
Emacspeak GitHub .


7 History:

  • Emacspeak 45.0 (IdealDog) is named in recognition of Emacs'
    excellent integration with various programming language
    environments — thanks to this, Emacspeak is the IDE of choice
    for eyes-free software engineering.
  • Emacspeak 44.0 continues the steady pace of innovation on the
    audio desktop.
  • Emacspeak 43.0 brings even more end-user efficiency by leveraging the
    ability to spatially place multiple audio streams to provide timely
    auditory feedback.
  • Emacspeak 42.0 while moving to GitHub from Google Code continues to
    innovate in the areas of auditory user interfaces and efficient,
    light-weight Internet access.
  • Emacspeak 41.0 continues to improve
    on the desire to provide not just equal, but superior access —
    technology when correctly implemented can significantly enhance the
    human ability.
  • Emacspeak 40.0 goes back to Web basics by enabling
    efficient access to large amounts of readable Web content.
  • Emacspeak 39.0 continues the Emacspeak tradition of increasing the breadth of
    user tasks that are covered without introducing unnecessary
    bloatware.
  • Emacspeak 38.0 is the latest in a series of award-winning
    releases from Emacspeak Inc.
  • Emacspeak 37.0 continues the tradition of
    delivering robust software as reflected by its code-name.
  • Emacspeak 36.0 enhances the audio desktop with many new tools including full
    EPub support — hence the name EPubDog.
  • Emacspeak 35.0 is all about
    teaching a new dog old tricks — and is aptly code-named HeadDog in
    on of our new Press/Analyst contact. emacspeak-34.0 (AKA Bubbles)
    established a new beach-head with respect to rapid task completion in
    an eyes-free environment.
  • Emacspeak-33.0 AKA StarDog brings
    unparalleled cloud access to the audio desktop.
  • Emacspeak 32.0 AKA
    LuckyDog continues to innovate via open technologies for better
    access.
  • Emacspeak 31.0 AKA TweetDog — adds tweeting to the Emacspeak
    desktop.
  • Emacspeak 30.0 AKA SocialDog brings the Social Web to the
    audio desktop—you cant but be social if you speak!
  • Emacspeak 29.0—AKAAbleDog—is a testament to the resilliance and innovation
    embodied by Open Source software—it would not exist without the
    thriving Emacs community that continues to ensure that Emacs remains
    one of the premier user environments despite perhaps also being one of
    the oldest.
  • Emacspeak 28.0—AKA PuppyDog—exemplifies the rapid pace of
    development evinced by Open Source software.
  • Emacspeak 27.0—AKA
    FastDog—is the latest in a sequence of upgrades that make previous
    releases obsolete and downgrades unnecessary.
  • Emacspeak 26—AKA
    LeadDog—continues the tradition of introducing innovative access
    solutions that are unfettered by the constraints inherent in
    traditional adaptive technologies.
  • Emacspeak 25 —AKA ActiveDog
    —re-activates open, unfettered access to online
    information.
  • Emacspeak-Alive —AKA LiveDog —enlivens open, unfettered
    information access with a series of live updates that once again
    demonstrate the power and agility of open source software
    development.
  • Emacspeak 23.0 — AKA Retriever—went the extra mile in
    fetching full access.
  • Emacspeak 22.0 —AKA GuideDog —helps users
    navigate the Web more effectively than ever before.
  • Emacspeak 21.0
    —AKA PlayDog —continued the
    Emacspeak tradition of relying on enhanced
    productivity to liberate users.
  • Emacspeak-20.0 —AKA LeapDog —continues
    the long established GNU/Emacs tradition of integrated innovation to
    create a pleasurable computing environment for eyes-free
    interaction.
  • emacspeak-19.0 –AKA WorkDog– is designed to enhance
    user productivity at work and leisure.
  • Emacspeak-18.0 –code named
    GoodDog– continued the Emacspeak tradition of enhancing user
    productivity and thereby reducing total cost of
    ownership.
  • Emacspeak-17.0 –code named HappyDog– enhances user
    productivity by exploiting today's evolving WWW
    standards.
  • Emacspeak-16.0 –code named CleverDog– the follow-up to
    SmartDog– continued the tradition of working better, faster,
    smarter.
  • Emacspeak-15.0 –code named SmartDog–followed up on TopDog
    as the next in a continuing series of award-winning audio desktop
    releases from Emacspeak Inc.
  • Emacspeak-14.0 –code named TopDog–was

the first release of this millennium.

  • Emacspeak-13.0 –codenamed
    YellowLab– was the closing release of the
    20th. century.
  • Emacspeak-12.0 –code named GoldenDog– began
    leveraging the evolving semantic WWW to provide task-oriented speech
    access to Webformation.
  • Emacspeak-11.0 –code named Aster– went the
    final step in making Linux a zero-cost Internet access solution for
    blind and visually impaired users.
  • Emacspeak-10.0 –(AKA
    Emacspeak-2000) code named WonderDog– continued the tradition of
    award-winning software releases designed to make eyes-free computing a
    productive and pleasurable experience.
  • Emacspeak-9.0 –(AKA
    Emacspeak 99) code named BlackLab– continued to innovate in the areas
    of speech interaction and interactive accessibility.
  • Emacspeak-8.0 –(AKA Emacspeak-98++) code named BlackDog– was a major upgrade to
    the speech output extension to Emacs.
  • Emacspeak-95 (code named Illinois) was released as OpenSource on
    the Internet in May 1995 as the first complete speech interface
    to UNIX workstations. The subsequent release, Emacspeak-96 (code
    named Egypt) made available in May 1996 provided significant
    enhancements to the interface. Emacspeak-97 (Tennessee) went
    further in providing a true audio desktop. Emacspeak-98
    integrated Internetworking into all aspects of the audio desktop
    to provide the first fully interactive speech-enabled WebTop.

8 About Emacspeak:

Originally based at Cornell (NY) —
http://www.cs.cornell.edu/home/raman —home to Auditory User
Interfaces (AUI) on the WWW, Emacspeak is now maintained on GitHub
https://github.com/tvraman/emacspeak. The system is mirrored
world-wide by an international network of software archives and
bundled voluntarily with all major Linux distributions. On Monday,
April 12, 1999, Emacspeak became part of the Smithsonian's Permanent
Research Collection
on Information Technology at the Smithsonian's
National Museum of American History.


The Emacspeak mailing list is archived at Vassar –the home of the
Emacspeak mailing list– thanks to Greg Priest-Dorman, and provides a
valuable knowledge base for new users.


9 Press/Analyst Contact: Tilden Labrador

Going forward, Tilden acknowledges his exclusive monopoly on
setting the direction of the Emacspeak Audio Desktop, and
promises to exercise this freedom to innovate and her resulting
power responsibly (as before) in the interest of all dogs.


*About This Release:



Windows-Free (WF) is a favorite battle-cry of The League Against
Forced Fenestration (LAFF). –see
http://www.usdoj.gov/atr/cases/f3800/msjudgex.htm for details on
the ill-effects of Forced Fenestration.


CopyWrite )C( Aster, Hubbell and Tilden Labrador. All Writes Reserved.
HeadDog (DM), LiveDog (DM), GoldenDog (DM), BlackDog (DM) etc., are Registered
Dogmarks of Aster, Hubbell and Tilden Labrador. All other dogs belong to
their respective owners.

-1:-- Emacspeak 45.0 (IdealDog) Unleashed! (Post T. V. Raman (noreply@blogger.com))--L0--C0--November 20, 2016 04:43 PM

Marcin Borkowski: format-spec

Last week I was coding something Emacs-y, and felt the need of a format-like function. It would get a string with embedded “control codes”, like %t% or %h, and then output a string with these “control codes” replaced by actual content. Writing something like that is not particularly difficult, but since I wanted to be able to also escape the percent sign by writing %%, I didn’t really feel like reinventing the wheel (and doing it elegantly and efficiently might be a tad tricky anyway). I asked a question on the help-gnu-emacs mailing list, and voilà! Here is the format-spec function.
-1:-- format-spec (Post)--L0--C0--November 19, 2016 08:44 AM

emacsninja: Brave New World

I believe I’ve found an even greater time sink than writing Lisp interpreters for fun. Long time ago, I’ve read an encouraging blog post on the future of the LispM, not expecting to find an implementation of the ideas presented therein. Turns out I was wrong about that. Meet Interim OS!

In case you’re wondering why you should possibly care:

  • Small and readable codebase (most of the code is device drivers for the Raspberry Pi)
  • Simple to hack on
  • Plan9-style APIs
  • Minimal Lisp dialect
  • Runs on your favorite desktop OS in hosted mode, that is, safely contained to a terminal with the ability to spawn graphical windows
  • Runs on bare metal (Raspberry Pi 2)

Getting it to run in hosted mode is simple enough, so I won’t explain it here. Booting on bare metal however is a different story, so here we go:

$ git clone https://github.com/mntmn/interim
$ cp interim/docs/interim-0.1.0-rpi2.tgz ./
$ bsdtar -xf interim-0.1.0-rpi2.tgz # cry me a river
% mkdir /media/boot
% mount /dev/sdXN /media/boot
% cp release-rpi2/* /media/boot/
% rm /media/boot/cmdline.txt
% umount /media/boot
  • Plug in the SDHC card, a HDMI monitor and a USB keyboard
  • Optionally: Plug in a network cable and/or a USB mouse
  • Power up

You’ll be greeted by a “Welcome to Interim OS” and dropped into a promptless shell. If you’re unlucky, the chosen resolution may be unreadable, so feel free to retry this process a few times. The keyboard layout is hardcoded and somewhere between QWERTY-US and QWERTZ-DE, something I intend to fix soon. For basic usage instructions, type (bytes->str (load "/sd/hello.txt")) and hit the enter key. Happy hacking!

-1:-- Brave New World (Post Vasilij Schneidermann)--L0--C0--November 17, 2016 09:17 PM

Timo Geusch: Building Emacs 25.1 on Ubuntu 16.10

A reader of this blog kindly pointed out that my instructions for building Emacs 25.1 on Ubuntu 16.10 result in a core dump when the build process bootstraps emacs. I only tested the instructions on 16.04 so I hadn’t run into this issue yet. The core dump on 16.10 appears to be a known issue … Continue reading "Building Emacs 25.1 on Ubuntu 16.10"

The post Building Emacs 25.1 on Ubuntu 16.10 appeared first on The Lone C++ Coder's Blog.

-1:-- Building Emacs 25.1 on Ubuntu 16.10 (Post Timo Geusch)--L0--C0--November 16, 2016 01:00 PM

Sebastian Wiesner: Read and write files in Emacs Lisp

Apparently it is not so uncommon to use interactive commands like find-file or write-file to read or write files in Emacs Lisp. Small libraries do it, Org Mode is no exception, even built-in libraries use this pattern 1. Which is a good example that Emacs’ own code is not always to be followed as in fact it is not a good idea to use these commands from Emacs Lisp.

User-Visible Effects?

When you are writing internal state to disk, it is a good idea to do that without interfering with the user. However interactive commands like find-file or write-file are meant to have user-visible effects—and they do have user-visible effects even when used in non-interactive Emacs Lisp. These effects—while normally desirable—will confuse users and interrupt their workflow if they show up unexpectedly.

write-file for instance sets the major mode of the current buffer which in turn runs the major mode hook—one of the central extension points in Emacs. Writing to a file can thus end up running arbitrary code from the user’s configuration! Another example: find-file tries to set local variables for the current buffer which can prompt the user if there is an unsafe value or a risky variable—imagine if you would see an unexpected prompt about a local variable in a buffer that you did not even create yourself!

Libraries To The Rescue

The best way for non-interactive IO is the excellent f.el by Johan Andersson of Cask fame. With this library reading and writing files is easy enough with f-read-text and f-write-text respectively:

(let* ((filename (locate-user-emacs-file "foo.txt"))
       (contents (f-read-text filename 'utf-8)))
  (f-write-text (upcase contents) 'utf-8 filename))

f.el also provides f-read-bytes and f-write-bytes for reading and writing raw bytes. And while you are at it take a look at the rest of f.el—there are a couple of nice functions in there for dealing with paths and files in Emacs Lisp.

The Tedious Way

That said f.el can do no magic; it uses the same Emacs Lisp primitives that that are also available to you. If you cannot use external libraries—perhaps you are contributing to a GNU ELPA package where non-gnu dependencies are outlawed?—you can still read files safely, albeit with a little more boilerplate. Use insert-file-contents-literally which avoids almost every side effect that reading files normally has in Emacs Lisp2 to insert the raw contents of the file into a temporary buffer and then decode the raw bytes with decode-coding-region3:

(with-temp-buffer
  (insert-file-contents-literally file-name)
  (decode-coding-region (point-min) (point-max) 'utf-8 t))

The inverse direction is a little more involved. There is no “write” equivalent to insert-file-contents-literally so you need to be a little more explicitly to prevent Emacs from making guesses. Bind coding-system-for-write to force Emacs to write binary data and then insert the utf-8-encoded data into a special temporary buffer which automatically gets written to the target path at the end of the form4:

(let ((coding-system-for-write 'binary))
  (with-temp-file path
    (set-buffer-multibyte nil)
    (encode-coding-string contents utf-8 nil 'insert-into-buffer)))

Summary

Do not use find-file, write-file or any other interactive command to read or write files from Emacs Lisp, to avoid unintended side-effects like mode hooks or even prompts about local variables. Use f-read-text and f-write-text from the f.el library instead, or write your own safe alternatives as shown above.

  1. I’m linking to the GitHub mirror of the Emacs sources because the UI is noticeably faster. When I looked for the link to the specific source location I couldn’t help but loose a few minutes on the list of pull requests. One would assume that it’s a well-known fact that Emacs isn’t hosted on GitHub, and the GitHub page even says that it’s a mirror, but people seem to be so eager to contribute that they ignore that. Now, of course, the PRs are mostly trivial, but still… imagine the potential contributors lost here, imagine where Emacs could be if it was more welcoming to contributors.

  2. I think even insert-file-contents-literally still runs find-file-hook but I’m just too lazy to find out now According to a reader it does not even run find-file-hook.

  3. The arguments to decode-coding-region are self-explanatory—except the last one. Why should you have to pass t? And t literally, not following the good practice of passing named symbols for non-nil arguments? I dearly recommend to look up this parameter in the docstring: It’s a nice lesson about flawed API design and the pain of programming Emacs Lisp.

  4. with-temp-file is a very unfortunate and confusing name. It doesn’t actually refer to temporary files at all, but instead to a temporary buffer that gets written to a file. Emacs Lisp can be quite confusing at whiles.

-1:-- Read and write files in Emacs Lisp (Post Sebastian Wiesner)--L0--C0--November 15, 2016 11:00 PM

Chris Wellons: Emacs, Dynamic Modules, and Joysticks

Two months ago Emacs 25 was released and introduced a new dynamic module feature. Emacs can now load shared libraries built against Emacs’ module API, defined in emacs-module.h. What’s interesting about this API is that it doesn’t require linking against Emacs or any sort of library. Instead, at run time Emacs supplies the module’s initialization function with function pointers for the entire API.

As a demonstration, in this article I’ll build an Emacs joystick interface (Linux only) using a dynamic module. It will allow Emacs to read events from any joystick on the system. All the source code is here:

It includes a calibration interface (M-x joydemo) within Emacs:

Currently, Emacs’ emacs-module.h header is the entirety of the module documentation. It’s a bit thin and leaves ambiguities that requires some reading of the Emacs source code. Even reading the source, it’s not clear which behaviors are a reliable part of the interface. For example, if there’s a pending non-local exit, it’s safe for a function to return NULL since the return value is never inspected (Emacs 25.1), but will this always be the case? While mistakes are unforgiving (a hard crash), the API is mostly intuitive and it’s been pretty easy to feel my way around it.

Dynamic Module Types

All Emacs values — integers, floats, cons cells, vectors, strings, etc. — are represented as the polymorphic, pointer-valued type, emacs_value. Despite being a pointer, NULL is not a valid value, as convenient as that would be. The API includes functions for creating and extracting the fundamental types: integers, floats, strings. Almost all other object types can only be accessed by making Lisp function calls to regular Emacs functions from the module.

Modules also introduce a brand new Emacs object type: a user pointer. These are non-readable, opaque pointer values returned by modules, typically representing a handle to some resource, be it a memory block, database connection, or a joystick. These objects include a finalizer function pointer — which, surprisingly, is not permitted to be NULL — and their lifetime is managed by Emacs’ garbage collector.

User pointers are a somewhat dangerous feature since there’s little to stop Emacs Lisp code from misusing them. A Lisp program can take a user pointer from one module and pass it to a function in a different module. Since it’s just a pointer, there’s no way to type check it. At best, a module could maintain a table of all its live pointers, checking all user pointer arguments against the table before dereferencing. But I don’t expect this to be normal practice.

Module Initialization

After loading the module through the platform’s mechanism, the first thing Emacs does is check for the symbol plugin_is_GPL_compatible. While tacky, this is not surprising given the culture around Emacs.

Next it calls emacs_module_init(), passing it the first function pointer. From this, the module can get a Lisp environment and start doing Emacs things, such as binding module functions to Lisp symbols.

Here’s a complete “Hello, world!” example:

#include "emacs-module.h"

int plugin_is_GPL_compatible;

int
emacs_module_init(struct emacs_runtime *ert)
{
    emacs_env *env = ert->get_environment(ert);
    emacs_value message = env->intern(env, "message");
    const char hi[] = "Hello, world!";
    emacs_value string = env->make_string(env, hi, sizeof(hi) - 1);
    env->funcall(env, message, 1, &string);
    return 0;
}

In a real module, it’s common to create function objects for native functions, then fetch the fset symbol and make a Lisp call on it to bind the newly-created function object to a name. You’ll see this in action later.

Joystick API

The joystick API will closely resemble Linux’s own joystick API, making for a fairly thin wrapper. It’s so thin that Emacs almost doesn’t even need a dynamic module. This is because, on Linux, joysticks are just files under /dev/input/. Want to see the input events on the first joystick? Just read /dev/input/js0. So Plan 9.

Emacs already knows how to read files, but these virtual files are a little too special for that. The header linux/joystick.h defines a struct js_event:

struct js_event {
    uint32_t time;  /* event timestamp in milliseconds */
    int16_t value;
    uint8_t type;
    uint8_t number; /* axis/button number */
};

The idea is to read from the joystick device into this structure. The first several reads are initialization that define the axes and buttons of the joystick and their initial state. Further events are queued up for the file descriptor. This all means that the file can’t just be opened each time joystick input is needed. It has to be held open for the duration, and is typically configured non-blocking.

The Emacs package will be called joymacs and there will be three functions:

(joymacs-open N)
(joymacs-close JOYSTICK)
(joymacs-read JOYSTICK EVENT-VECTOR)

joymacs-open

The joymacs-open function will take an integer, opening the Nth joystick (/dev/input/jsN). It will create a file descriptor for the joystick device, returning it as a user pointer. Think of it as a sort of “joystick handle.” Now, it could instead return the file descriptor as an integer, but the user pointer has two significant benefits:

  1. The resource will be garbage collected. If the caller loses track of a file descriptor returned as an integer, the joystick device will be held open until Emacs shuts down, using up one of Emacs’ file descriptors. By putting it in a user pointer, the garbage collector will have the module to release the file descriptor if the user loses track of it.

  2. It should be difficult for the user to make a dangerous call. Emacs Lisp can’t create user pointers — they only come from modules — and so the module is less likely to get passed the wrong thing. In the case of joystick-close, the module will be calling close(2) on the argument. We definitely don’t want to make that system call on file descriptors owned by Emacs. Further, since user pointers are mutable, the module can ensure it doesn’t call close(2) twice.

Here’s the implementation for joymacs-open. I’ll over over each part in detail.

static emacs_value
joymacs_open(emacs_env *env, ptrdiff_t n, emacs_value *args, void *ptr)
{
    (void)ptr;
    (void)n;
    int id = env->extract_integer(env, args[0]);
    if (env->non_local_exit_check(env) != emacs_funcall_exit_return)
        return nil;
    char buf[64];
    int buflen = sprintf(buf, "/dev/input/js%d", id);
    int fd = open(buf, O_RDONLY | O_NONBLOCK);
    if (fd == -1) {
        emacs_value signal = env->intern(env, "file-error");
        emacs_value message = env->make_string(env, buf, buflen);
        env->non_local_exit_signal(env, signal, message);
        return nil;
    }
    return env->make_user_ptr(env, fin_close, (void *)(intptr_t)fd);
}

The C function name doesn’t matter to Emacs. It’s static because it doesn’t even matter if the function visible to Emacs. It will get the function pointer later as part of initialization.

This is the prototype for all functions callable by Emacs Lisp, regardless of its arity. It has four arguments:

  1. It gets an environment, env, through which to call back into Emacs.

  2. It gets n, the number of arguments. This is guaranteed to be the correct number of arguments, as specified later when creating the function object, so only variadic functions need to inspect this argument.

  3. The Lisp arguments are passed as an array of values, args. There’s no type declaration when declaring a function object, so these may be of the wrong type. I’ll go over how to deal with this.

  4. Finally, it gets an arbitrary pointer, supplied at function object creation time. This allows the module to create closures, but will usually be ignored.

The first thing the function does is extract its integer argument. This is actually an intmax_t, but I don’t think anyone has that many USB ports. An int will suffice.

    int id = env->extract_integer(env, args[0]);
    if (env->non_local_exit_check(env) != emacs_funcall_exit_return)
        return nil;

As for not underestimating fools, what if the user passed a value that isn’t an integer? Will the world come crashing down? Fortunately Emacs checks that in extract_integer and, if there’s a mismatch, sets a pending error signal in the environment. This is really great because checking types directly in the module is a real pain the ass. So, before committing to anything further, such as opening a file, I check for this signal and bail out early if necessary. In Emacs 25.1 it’s safe to return NULL since the return value will be completely ignored, but I’d rather hedge my bets.

By the way, the nil here is a global variable set in initialization. You don’t just get that for free!

The next step is opening the joystick device, read-only and non-blocking. The non-blocking is vital because the module would otherwise hang Emacs later if there are no events (well, except for the read being quickly interrupted by a POSIX signal).

    char buf[64];
    int buflen = sprintf(buf, "/dev/input/js%d", id);
    int fd = open(buf, O_RDONLY | O_NONBLOCK);

If the joystick fails to open (e.g. it doesn’t exist, or the user lacks permission), manually set an error signal for a non-local exit. I chose the file-error signal and I’m just using the filename as the signal data.

    if (fd == -1) {
        emacs_value signal = env->intern(env, "file-error");
        emacs_value message = env->make_string(env, buf, buflen);
        env->non_local_exit_signal(env, signal, message);
        return nil;
    }

Otherwise create the user pointer. No need to allocate any memory; just stuff it in the pointer itself. If the user mistakenly passes it to another module, it will sure be in for a surprise when it tries to dereference it.

    return env->make_user_ptr(env, fin_close, (void *)(intptr_t)fd);

The fin_close() function is defined as:

static void
fin_close(void *fdptr)
{
    int fd = (intptr_t)fdptr;
    if (fd != -1)
        close(fd);
}

The garbage collector will call this function when the user pointer is lost. If the user closes it early with joymacs-close, that function will set the user pointer to -1, an invalid file descriptor, so that it doesn’t get closed a second time here.

joymacs-close

Here’s joymacs-close, which is a bit simpler.

static emacs_value
joymacs_close(emacs_env *env, ptrdiff_t n, emacs_value *args, void *ptr)
{
    (void)ptr;
    (void)n;
    int fd = (intptr_t)env->get_user_ptr(env, args[0]);
    if (env->non_local_exit_check(env) != emacs_funcall_exit_return)
        return nil;
    if (fd != -1) {
        close(fd);
        env->set_user_ptr(env, args[0], (void *)(intptr_t)-1);
    }
    return nil;
}

Again, it starts by extracting its argument, relying on Emacs to do the check:

    int fd = (intptr_t)env->get_user_ptr(env, args[0]);
    if (env->non_local_exit_check(env) != emacs_funcall_exit_return)
        return nil;

If the user pointer hasn’t been closed yet, then close it and strip out the file descriptor to prevent further closes.

    if (fd != -1) {
        close(fd);
        env->set_user_ptr(env, args[0], (void *)(intptr_t)-1);
    }

joymacs-read

The joymacs-read function is doing something a little unusual for an Emacs Lisp function. It takes two arguments: the joystick handle and a 5-element vector. Instead of returning the event in some representation, it fills the vector with the event details. The are two reasons for this:

  1. The API has no function for creating vectors … though the module could get the make-symbol vector and call it to create a vector.

  2. The idiom for event pumps is for the caller to supply a buffer to the pump. This has better performance by avoiding lots of unnecessary allocations, especially since events tend to be message-like objects with a short, well-defined extent.

Here’s the full definition:

static emacs_value
joymacs_read(emacs_env *env, ptrdiff_t n, emacs_value *args, void *ptr)
{
    (void)n;
    (void)ptr;
    int fd = (intptr_t)env->get_user_ptr(env, args[0]);
    if (env->non_local_exit_check(env) != emacs_funcall_exit_return)
        return nil;
    struct js_event e;
    int r = read(fd, &e, sizeof(e));
    if (r == -1 && errno == EAGAIN) {
        /* No more events. */
        return nil;
    } else if (r == -1) {
        /* An actual read error (joystick unplugged, etc.). */
        emacs_value signal = env->intern(env, "file-error");
        const char *error = strerror(errno);
        size_t len = strlen(error);
        emacs_value message = env->make_string(env, error, len);
        env->non_local_exit_signal(env, signal, message);
        return nil;
    } else {
        /* Fill out event vector. */
        emacs_value v = args[1];
        emacs_value type = e.type & JS_EVENT_BUTTON ? button : axis;
        emacs_value value;
        if (type == button)
            value = e.value ? t : nil;
        else
            value =  env->make_float(env, e.value / (double)INT16_MAX);
        env->vec_set(env, v, 0, env->make_integer(env, e.time));
        env->vec_set(env, v, 1, type);
        env->vec_set(env, v, 2, value);
        env->vec_set(env, v, 3, env->make_integer(env, e.number));
        env->vec_set(env, v, 4, e.type & JS_EVENT_INIT ? t : nil);
        return args[1];
    }
}

As before, extract the first argument and check for a signal. Then call read(2) to get an event. If the read fails with EAGAIN, it’s not a real failure. There are just no more events, so return nil.

    struct js_event e;
    int r = read(fd, &e, sizeof(e));
    if (r == -1 && errno == EAGAIN) {
        /* No more events. */
        return nil;
    }

If the read failed with something else — perhaps the joystick was unplugged — signal an error. The strerror(3) string is used for the signal data.

    if (r == -1) {
        /* An actual read error (joystick unplugged, etc.). */
        emacs_value signal = env->intern(env, "file-error");
        const char *error = strerror(errno);
        emacs_value message = env->make_string(env, error, strlen(error));
        env->non_local_exit_signal(env, signal, message);
        return nil;
    }

Otherwise fill out the event vector. If the second argument isn’t a vector, or if it’s too short, the signal will automatically get raised by Emacs. The module can keep plowing through the vec_set() calls safely since it’s not committing to anything.

        /* Fill out event vector. */
        emacs_value v = args[1];
        emacs_value type = e.type & JS_EVENT_BUTTON ? button : axis;
        emacs_value value;
        if (type == button)
            value = e.value ? t : nil;
        else
            value =  env->make_float(env, e.value / (double)INT16_MAX);
        env->vec_set(env, v, 0, env->make_integer(env, e.time));
        env->vec_set(env, v, 1, type);
        env->vec_set(env, v, 2, value);
        env->vec_set(env, v, 3, env->make_integer(env, e.number));
        env->vec_set(env, v, 4, e.type & JS_EVENT_INIT ? t : nil);
        return args[1];

The Linux event struct has four fields and the function fills out five values of the vector. This is because the type field has a bit flag indicating initialization events. This is split out into an extra t/nil value. It also normalizes axis values and converts button values into t/nil, which makes more sense for Emacs Lisp. The event itself is returned since it’s a truthy value and it’s convenient for the caller.

The astute programmer might notice that the negative side of the axis could go just below -1.0, since INT16_MIN has one extra value over INT16_MAX (two’s complement). It doesn’t seem to be documented, but the joystick drivers I’ve seen never exactly return INT16_MIN, so this is in fact the correct way to normalize it.

Initialization

All that’s left is the initialization function. First declare some global variables to keep track of frequently-used symbols.

static emacs_value nil;
static emacs_value t;
static emacs_value button;
static emacs_value axis;

These are interned at the very beginning of initialization. The symbols :button and :axis are given global references so that the garbage collector doesn’t rip them out from under the module. It’s unclear from the API, but the make_global_ref() function returns the object being referenced. I trust that the t and nil symbols will never be garbage collected, so these don’t need global references.

    nil = env->intern(env, "nil");
    t = env->intern(env, "t");
    button = env->make_global_ref(env, env->intern(env, ":button"));
    axis = env->make_global_ref(env, env->intern(env, ":axis"));

    emacs_value fset = env->intern(env, "fset");

It also grabs fset locally since it will soon be needed.

Finally, bind the functions. The second and third arguments to make_function are the minimum and maximum number of arguments, which may look familiar. The last argument is that closure pointer I mentioned at the beginning.

    emacs_value args[2];
    args[0] = env->intern(env, "joymacs-open");
    args[1] = env->make_function(env, 1, 1, joymacs_open, doc, 0);
    env->funcall(env, fset, 2, args);

If the module is to be loaded with require like any other package, it needs to provide: (provide 'joymacs).

    emacs_value provide = env->intern(env, "provide");
    emacs_value joymacs = env->intern(env, "joymacs");
    env->funcall(env, provide, 1, &joymacs);

And that’s it!

The source repository now includes a port to Windows (XInput). If you’re on Linux or Windows, have Emacs 25 with modules enabled, and a joystick is plugged in, then make run in the repository should bring up Emacs running a joystick calibration demonstration. The module can’t poke at Emacs when events are ready, so instead there’s a timer that polls the module for events.

I’d like to someday see an Emacs Lisp game well-suited for a joystick.

-1:-- Emacs, Dynamic Modules, and Joysticks (Post)--L0--C0--November 05, 2016 04:01 AM

Wilfred Hughes: Introspecting Glue Code

Writing glue code is easy. Writing it well is harder.

Glue code is particularly common when developing editor plugins: you want to leverage the existing tools, often via a command line interface.

When bugs occur, it’s hard to know where the fault lies. Is this an issue with your glue code, or the underlying tool? We frequently face this problem with emacs-racer. Users feel powerless: they don’t know if they’ve misconfigured something, if the glue code is wrong, or if they’ve hit an genuine racer bug.

It turns out that flycheck already has a great solution to this. Flycheck can actually tell you why it couldn’t run a checker!

M-x flycheck-select-checker

Looking at flycheck, I realised this was exactly what emacs-racer needed. I built a M-x racer-debug command that tells you exactly what happened, and how to reproduce the issue outside of Emacs.

M-x racer-debug

This makes it much easier to see what racer is doing. Not only is it great for bug reports, but it helps us develop new features too. Developers can see exactly what data we receive from racer.

I’m still surprised by the versatility of introspective tools. When building functionality, it’s incredibly useful to be able to ask ‘what just happened?’. Debuggable debugging tools are a superpower.

-1:-- Introspecting Glue Code (Post Wilfred Hughes (me@wilfred.me.uk))--L0--C0--November 05, 2016 12:00 AM

Grant Rettke: Org Mode 9.0 is Released

Announcement.

Changes.

  • Interesting additions (that you may have already seen in development)
    • Linter
    • Choose latex compiler keyword
    • PlantUML convenience
-1:-- Org Mode 9.0 is Released (Post Grant)--L0--C0--November 03, 2016 09:18 AM

Manuel Uberti: Scala in Emacs with ENSIME

As an amateur Clojure developer, I am particularly fond of CIDER. It brings everything I need to program with Clojure right inside my favourite text editor.

Therefore, when I approached Scala one of the first things I did was looking for something similar to CIDER. I hate leaving Emacs for programming tasks, so having it ready for Scala was paramount. Fortunately, it did not take me long to find my way: ENSIME was right around the corner.

ENSIME supports the famous sbt build tool, and the documentation gently guides you through all the necessary steps to achieve a working setup. It is really a matter of minutes.

A nice trick I grabbed straight from Sam Halliday init.el is:

(remove-hook 'post-self-insert-hook
             'scala-indent:indent-on-parentheses)

(sp-local-pair 'scala-mode "(" nil
               :post-handlers '(("||\n[i]" "RET")))
(sp-local-pair 'scala-mode "{" nil
               :post-handlers '(("||\n[i]" "RET") ("| " "SPC")))

This code let the awesome Smartparens take care of parentheses in Scala buffers. Another handy solution comes from Sebastian Wiesner:

(defun mu-scala-pop-to-sbt (new-frame)
    "Open SBT REPL for this project, optionally in a NEW-FRAME.

Select the SBT REPL for the current project in a new window.  If
the REPL is not yet running, start it.  With prefix arg, select
the REPL in a new frame instead."
(interactive "P")
;; Start SBT when no running, taken from `sbt:command'
(when (not (comint-check-proc (sbt:buffer-name)))
  (sbt:run-sbt))
(let ((display-buffer-overriding-action
       (if new-frame '(display-buffer-pop-up-frame) nil)))
  (pop-to-buffer (sbt:buffer-name))))

(with-eval-after-load 'scala-mode
  (bind-key "C-c m s" #'mu-scala-pop-to-sbt scala-mode-map))

I just started my journey with Scala but I can safely say ENSIME could be the perfect companion.

-1:-- Scala in Emacs with ENSIME (Post)--L0--C0--November 01, 2016 12:00 AM

emacshorrors: PSA: Emacs is not a proper GTK application

Update: I’ve been pointed to an emacs-devel discussion about giving Emacs a proper GTK frontend, most comparable to the Win32 and NS frontends (which are free from X11isms). This would be a better approach than what’s been outlined below, with the Cairo code being repurposed for the drawing bits.

Daniel Colascione’s excellent write-up on bringing double-buffered rendering to Emacs has prompted me to do the same on a set of questions that can be occasionally spotted on #emacs:

  • Which GUI build of Emacs shall I choose?
  • What’s the difference between the GTK build and the other builds of Emacs on Linux?
  • Does the GTK build run on Wayland?
  • Does the GTK build run on Broadway?
  • Why does Emacs not render as nicely as <insert more modern text editor>?

If you’ve ever programmed a GUI application with a modern/popular GUI toolkit, you’ll have noticed that while it gives you loads of desirable features, it forces you to structure your application around its idea of an event loop. In other words, your application is forced to react asynchronously to user events. Games are pretty much the only major kind of graphical application that can get away with doing their own event loop, but end up doing their own GUI instead.

Now, the issue with Emacs is that it does its own event loop, pretending that the frontend is a textual or graphical terminal. It’s pretty much the graphical equivalent of a REPL and at the time it only had a text terminal frontend, this way of doing things worked out fairly well. However, by the time users demanded having pretty GTK widgets in Emacs, it became clear that more involved hacks were needed to make that happen. This is why Emacs runs the GTK event loop one iteration at a time, pushes its own user events into it (to make widgets react) and a plethora of more hacks to reconcile their rendering with everything done by X11.

In other words, Emacs is more of a X11 application plus your favorite widgets. The choice of GUI toolkit to build it with is mostly irrelevant, save an infamous bug with the GTK3 frontend that can crash the daemon. Emacs will therefore not run on a pure Wayland system or under Broadway in the browser. If anyone would want to make that happen, either the GTK frontend would need to yank out everything X11 (unlikely as it’s deeply entrenched) or to create a new frontend doing platform-agnostic drawing (the Cairo feature being a prime candidate for that).

Further reading material:

-1:-- PSA: Emacs is not a proper GTK application (Post Vasilij Schneidermann)--L0--C0--October 29, 2016 02:05 PM