Over at What keramida said…, Giorgos Keramidas poses an interesting problem. Given the following output from dstat, get rid of the B, k, and M suffixes by converting them into bytes.
Keramidas also wants to do a bit of reformatting of the table, which we’ll ignore for this post. He does the conversion with three invocations of replace-regexp but given that he titles his post Powerful Regular Expressions Combined with Lisp in Emacs, I thought it would be an interesting challenge to write some Elisp to get rid of those suffixes and convert the entries into bytes.
Obviously, for a one-off task, Keramidas’ solution is the best but if this is something that has to be done regularly a bit of Elisp is just what’s needed. My solution is a Lisp function that converts the table with a single call. Leave your solution in the comments and I’ll post mine in a few days.
Every Emacs user knows that he can split the current window
horizontally (with C-x 2) and vertically (with C-x 3) as much as
he desires to. However, some Emacs users don’t know what to do with
the extra windows they’ve created when they do not them.
To delete the selected window, type C-x 0 (delete-window).
Once a window is deleted, the space that it occupied
is given to an adjacent window (but not the minibuffer window, even if
that is the active window at the time). Deleting the window has no effect on the
buffer it used to display; the buffer continues to exist, and you can
still switch to with C-x b or any other buffer navigation command.
C-x 4 0 (kill-buffer-and-window) is a stronger (and fairly
unknown) command; it kills the current buffer and then deletes the
selected window (basically it combines C-x k and C-x 0). Obviously
it’s a good idea to use it on windows displaying buffers you’re no
longer needing.
C-x 1 (delete-other-windows) deletes all the windows, except the
selected one; the selected window expands to use the whole frame.
(This command cannot be used while the minibuffer window is active;
attempting to do so signals an error.) In the era of narrow screens I
used that command fairly often when I needed to focus on a particular
task. Now I keep my screen split in half vertically 99% of the time,
but I still use C-x 1 from time to time when I’m about to resplit my
screen in some uncommon way.
Windows displaying help buffers (generally created with commands like
C-h ...) warrant a special mention. They can be deleted with a
single keystroke - q. That would delete the help window altogether
if it was created by the help command, or restore its original
content if the window existing beforehand and was reused by the help command.
This post continues a topic that was introduced in
smarter open-line few months
back.
Often when editing code one wishes to open a line just above the
current one, which is properly indented relative to the existing code,
and position the cursor at its beginning. Such a feature is present
in most IDEs, such as IntelliJ IDEA, Eclipse and NetBeans. It’s
sometimes bound to Control+Shift+Enter. Last time I showed you how
to implement a similar function called smart-open-line, this time
will implement smart-open-line-above. Just add this snippet to your
.emacs (or .emacs.d/init.el or whatever):
12345678910
(defunsmart-open-line-above()"Insert an empty line above the current line.Position the cursor at it's beginning, according to the current mode."(interactive)(move-beginning-of-linenil)(newline-and-indent)(forward-line-1)(indent-according-to-mode))(global-set-key[(controlshiftreturn)]'smart-open-line-above)
Evaluate the code (or restart Emacs) and you’ll be able to use
M-x smart-open-line-above or Control+Shift+Enter (aka C-S-return).
Admittedly this keybinding kind of sucks, so here’s another option for
you - M-o (used by default as the prefix for some font setting
commands nobody ever uses) for smart-open-line and M-O for
smart-open-line-above.
David recently pulled in a couple of changes to make OB-SML happy with Emacs’ package manager, SML-MODE 6.4, and Org Babel. For you reproducible research junkies, this is super-cool. It is already out on Marmalade.
I found another one (I guess I still need to order that book). In using the UVM, I have some sequences randomizing other sub sequences. I really want it to work like this (simplified, non-UVM) example:
class Foo;
randint bar;
functionvoiddisplay();
$display("bar: %0d", bar);
endfunctionendclassclass Bar;
int bar;
functionvoidbody();
Foo foo = new();
bar = 3;
foo.display();
assert(foo.randomize() with {bar == this.bar;});
foo.display();
endfunctionendclassmodule top;
Bar bar;
initialbegin
bar = new();
bar.body();
endendmodule
See the problem there? Here's what prints out when you run the above:
bar: 0
bar: -1647275392
foo.bar is not constrained to be 3 like you might expect. That's because this.bar refers to bar that is a member of class Foo, not bar that's a member of class Bar. As far as I can tell, there is no way to refer to bar that is a member of Bar in the constraint. I guess Foo could have a reference back up to Bar, but that's really awkward. Has anyone else run into this? How do you deal with it?
Shane Celis has announced that Emacsy was accepted as a Google Summer of Code project. Celis describes Emacsy as an embeddable Emacs-like library for non-text applications: Emacs OS without the text editor. The idea is to bring the Emacs way of doing things to other applications. Emacsy will be built on top of Guile Scheme, which is already embeddable in C and C++ applications.
Celis’s post gives a hint of what he’s aiming at but for the real details you should see his Google SoC application. It seems like an interesting project and I’m looking forward to seeing the results. Take a look and see if you don’t agree.
Some years ago, I came across M-x animate-birthday-present (and therefore animate-string and animate-sequence) while reading through the output of M-x apropos-command RET . RET, which lists all of the interactive commands. (Well worth exploring! The Emacs Manual also lists a few unusual things under “Amusements.”) It’s one of my favourite examples of odd things you can find in Emacs, like M-x doctor and M-x tetris. I use animate-string to create the title sequences of Emacs chats like this one with Bastien Guerry.
It turns out that lots of people use the Emacs text editor for animating things. Andrea Rossetti (from Trieste, Italy) e-mailed to share this little thing he put together to simulate typing in Emacs. And, boggle of boggles, someone even taught a course on Emacs Lisp Animations.
Next: Maybe someone can make an onion-skin animation mode to go on top of artist-mode so that we can make Emacs flipbooks?
commander.el is a command
line parser for Emacs. It avoids messing with command-switch-alist
(and friends) and instead defines the schema in an elegant API.
Example schema
Here is a (silly) example where numbers can be added and subtracted.
Over the last week I have revamped my Gnus configuration. I used to
look at my e-mail once a day because I felt it was distracting. I now
want notification of interesting e-mail every minute.
I will detail how I have setup this feature with my new configuration.
To speed up Gnus you need to set the nnml backend as your primary
select method. This is critical as setting up a news server will use
the network and slow things down.
I use getmail to fetch e-mail on a maildir directory. This way I don't
have to deal with file locks of an mbox file. The getmail
configuration is:
[retriever]
type = SimplePOP3SSLRetriever
server = pop.gmail.com
username = ivan.kanis@googlemail.com
password = secret
[destination]
type = Maildir
path = ~/keep/Maildir/
[options]
delete = True
Before I used to do fancy splitting. It turned out to be a pain to
maintain. I now use plain splitting with three inboxes: interesting,
medium and boring.
The function private-is-email-in-bbdb will check that the person that
sent me an e-mail is in bbdb. If that's the case, the e-mail will
land in the "interesting" inbox.
I have gmane news as my secondary select. The function unplugs Gnus so
it won't check for news over the network. I think it would be too slow.
The ivan-var-display-message variable is a nasty, nasty hack to hide
all the messages issued by Gnus while it fetches e-mail. I have
advised the message function like so:
(defadvicemessage (around ivan-fun-message activate)
"Shut up message"
(when ivan-var-display-message
ad-do-it))
The following function shows the e-mail indicator when there are
messages in my interesting inbox (thanks to Julien Danjou):
(defunprivate-check-email ()
"Return t when there are interesting e-mails"
(when (boundp 'gnus-newsrc-alist)
(dolist (entry gnus-newsrc-alist)
(let ((group (car entry)))
(when (string= group "interesting")
(let ((unread (gnus-group-unread group)))
(when (and (numberp unread)
(> unread 0))
(return t))))))))
(setq display-time-mail-function 'private-check-email)
And now I have an indicator when there is an interesting mail to read.
Channel name shortening has been improved a lot, resulting in #emacs and #emacs-circe being shortened to #e and #e-c respectively.
/WL (who left) will now show the nicks in a single line, making the output a lot more readable.
New CTCP queries supported: SOURCE and CLIENTINFO.
New command: /CLEAR. This will delete all text in a chat buffer.
Lurker handling has been improved a lot and should get confused less often now.
Query buffers now are renamed to reflect the queried user’s nick when that users is seen to change their nick.
Non-blocking connects are disabled by default in win32, as that doesn’t work there and caused errors.
Added a new circe-message-handler-table which can be used instead of circe-receive-message-functions.
Display handling was rewritten, and circe-display-handler now allows for lists in addition to functions, replacing the old circe-format-strings special case.
The way Circe displays messages has been changed. The new list circe-message-option-functions is used to collect options. See the docstring for more information.
Recent channel user handling has been reworked to be more consistent. In the process, circe-parted-users-timeout was renamed to circe-channel-recent-users-timeout.
Duration strings won’t be empty anymore.
The active channels are now shown in different places in the mode line to make more sense.
circe-fool-face is now not bold anymore to avoid unwanted attention when combined with other faces.
And a number of bugfixes.
XEmacs is not supported anymore. Sorry. No one here uses it, so the code didn’t work anyhow.
Thanks go to Taylan Ulrich B, John Foerch, Pi and Donald Curtis for their contributions.
After reading this post on the MasteringEmacs blog I gave eshell a try and I like it. On this other post he shows how to implement completion with pcomplete that is automatically used by eshell.
Without further ado, here are completions for git, bzr, and hg. The git completion is basically his completion with some improvements. For example, it completes all git commands by parsing the output of git help --all.
;;**** Git Completion
(defun pcmpl-git-commands ()
"Return the most common git commands by parsing the git output."
(with-temp-buffer
(call-process-shell-command "git" nil (current-buffer) nil "help" "--all")
(goto-char 0)
(search-forward "available git commands in")
(let (commands)
(while (re-search-forward
"^[[:blank:]]+\\([[:word:]-.]+\\)[[:blank:]]*\\([[:word:]-.]+\\)?"
nil t)
(push (match-string 1) commands)
(when (match-string 2)
(push (match-string 2) commands)))
(sort commands #'string<))))
(defconst pcmpl-git-commands (pcmpl-git-commands)
"List of `git' commands.")
(defvar pcmpl-git-ref-list-cmd "git for-each-ref refs/ --format='%(refname)'"
"The `git' command to run to get a list of refs.")
(defun pcmpl-git-get-refs (type)
"Return a list of `git' refs filtered by TYPE."
(with-temp-buffer
(insert (shell-command-to-string pcmpl-git-ref-list-cmd))
(goto-char (point-min))
(let (refs)
(while (re-search-forward (concat "^refs/" type "/\\(.+\\)$") nil t)
(push (match-string 1) refs))
(nreverse refs))))
(defun pcmpl-git-remotes ()
"Return a list of remote repositories."
(split-string (shell-command-to-string "git remote")))
(defun pcomplete/git ()
"Completion for `git'."
;; Completion for the command argument.
(pcomplete-here* pcmpl-git-commands)
(cond
((pcomplete-match "help" 1)
(pcomplete-here* pcmpl-git-commands))
((pcomplete-match (regexp-opt '("pull" "push")) 1)
(pcomplete-here (pcmpl-git-remotes)))
;; provide branch completion for the command `checkout'.
((pcomplete-match "checkout" 1)
(pcomplete-here* (append (pcmpl-git-get-refs "heads")
(pcmpl-git-get-refs "tags"))))
(t
(while (pcomplete-here (pcomplete-entries))))))
;;**** Bzr Completion
(defun pcmpl-bzr-commands ()
"Return the most common bzr commands by parsing the bzr output."
(with-temp-buffer
(call-process-shell-command "bzr" nil (current-buffer) nil "help" "commands")
(goto-char 0)
(let (commands)
(while (re-search-forward "^\\([[:word:]-]+\\)[[:blank:]]+" nil t)
(push (match-string 1) commands))
(sort commands #'string<))))
(defconst pcmpl-bzr-commands (pcmpl-bzr-commands)
"List of `bzr' commands.")
(defun pcomplete/bzr ()
"Completion for `bzr'."
;; Completion for the command argument.
(pcomplete-here* pcmpl-bzr-commands)
(cond
((pcomplete-match "help" 1)
(pcomplete-here* pcmpl-bzr-commands))
(t
(while (pcomplete-here (pcomplete-entries))))))
;;**** Mercurial (hg) Completion
(defun pcmpl-hg-commands ()
"Return the most common hg commands by parsing the hg output."
(with-temp-buffer
(call-process-shell-command "hg" nil (current-buffer) nil "-v" "help")
(goto-char 0)
(search-forward "list of commands:")
(let (commands
(bound (save-excursion
(re-search-forward "^[[:alpha:]]")
(forward-line 0)
(point))))
(while (re-search-forward
"^[[:blank:]]\\([[:word:]]+\\(?:, [[:word:]]+\\)*\\)" bound t)
(let ((match (match-string 1)))
(if (not (string-match "," match))
(push (match-string 1) commands)
(dolist (c (split-string match ", ?"))
(push c commands)))))
(sort commands #'string<))))
(defconst pcmpl-hg-commands (pcmpl-hg-commands)
"List of `hg' commands.")
(defun pcomplete/hg ()
"Completion for `hg'."
;; Completion for the command argument.
(pcomplete-here* pcmpl-hg-commands)
(cond
((pcomplete-match "help" 1)
(pcomplete-here* pcmpl-hg-commands))
(t
(while (pcomplete-here (pcomplete-entries))))))
Yesterday, the first milestone of the Havana developement branch of
Ceilometer has been released and is now available for testing and download.
This means the first quarter of the OpenStack Havana development has
passed!
New features
Ten blueprints have been implemented as you can see on the
release page. I'm
going to talk through some of them here, that are the most interesting for
users.
Ceilometer can now
counts the scheduling attempt
of instances done by nova-scheduler. This can be useful to eventually bill
such information or for audit (implemented by me for eNovance).
People using the HBase backend can now do
requests filtering on any of the counter fields, something we call
metadata queries,
and which was missing for this backend driver. Thanks to Shengjie Min
(Dell) for the implementation.
Counters can now be
sent over UDP
instead of the Oslo RPC mechanism (AMQP based by default). This allows
counter transmission to be done in a much faster way, though less reliable.
The primary use case being not audit or billing, but the alarming features
that we are working on (implemented by me for eNovance).
The
initial alarm API
has been designed and implemented, thanks to Mehdi Abaakouk (eNovance) and
Angus Salkled (RedHat) who tackled this. We're now able to do CRUD actions
on these.
Posting of meters via the HTTP API is now possible. This is now another
conduct that can be used to publish and collector meter. Thanks to Angus
Salkled (RedHat) for implementing this.
I've been working on an somewhat experimental
notifier driver for Oslo
notification that publishes Ceilometer counters instead of the standard
notification, using the Ceilometer pipeline setup.
Sandy Walsh (Rackspace) has put in place the base needed to
store raw notifications (events),
with the final goal of bringing more functionnalities around these into
Ceilometer.
Obviously, all of this blueprint and bugfixes wouldn't be implemented or
fixed without the harden eyes of our entire team, reviewing code and
advising restlessly the developers. Thanks to them!
Bug fixes
Thirty-one bugs were fixed, though most of them might not interest you so I
won't elaborate too much on that. Go read
the list if you are
curious.
Toward Havana 2
We now have 21 blueprints targetting the
Ceilometer's second Havana milestone,
with some of them are already started. I'll try to make sure we'll get there
without too much trouble for the 18th July 2013. Stay tuned!
I am using the / and the z keys to act as if I pressed the Alt key.
Now I have the actual Alt key on my keyboard free for other uses. I
have mapped it to be the Hyper key. I then use this modifier so that
the keys i, j, k and l will do cursor motion.
First I set up my .xmodmap thus:
keycode 133 = Hyper_L
All keyboard are different. You can find your keycode with the program
xev.
Mapping the keys in Emacs, I disable the usual keys so that I get into
the habit:
I feel that the motion keys in emacs are awkward. With this hack I
will be able to use the same keys for Emacs, vim and the command line.
I am not sure it's worth it, time will tell.
-1:-- Hyper Key (Post Ivan Kanis)--L0--C0--May 31, 2013 12:00 AM
Babel is about letting many different languages work together. Programming languages live in blocks inside natural language Org-mode documents. A piece of data may pass from a table to a Python code block, then maybe move on to an R code block, and finally end up embedded as a value in the middle of a paragraph or possibly pass through a gnuplot code block and end up as a plot embedded in the document.
My current approach is to use multiple languages, build scripts, intermediate files to share data, and finally weaving it together inside of LaTeX. The babel way looks intriguing, with excellent support (via Emacs modes) for numerous languages. Very exciting.
This weekend I stopped playing with lua and finally got some time to upgrade org to the newest version.
Org 8 had lots of improvements and new features compared to 7.9.x. There were a couple of those that I wanted to try as soon as possible:
New Exporters: org-mode now uses org-element to parse org files. That's a big big improvement because that allows users to write new exporters relying on a somewhat more abstract and high level parser api than what we had before.
orgstruct and orgstruct++ got orgstruct-heading-prefix-regexp option to set allowed prefixes and be able to fold parts of non-org files as if they where
On the exporters side, I tried org-reveal, and it works great so far. an exporter to make presentations using reveal.js Probably I'll try it for real next week when when I'll be doing some talk at my workplace.
Meanwhile in orgstruct... being able to define prefixes for orgstruct-mode allows us to have foldable text files. For example, use the following line to make it work in elisp files.
With this, I reduce the js2-mode modeline lighter from "JavaScript IDE" to just "JS2".
I stole it from Bodil's
.emacs.d and macroified it a little.
The first argument is the package name, the second is the mode in
question, and the third is the new lighter for the mode.
I’ve picked up an interest in JavaScript and HTML5 this year, and have gone to a bunch of great technical talks in Boston. I brought a camera with me and recorded some of them, so you can see them too if you like. Here they are:
I think these are world-class talks. But if I hadn’t brought my little camera with me and recorded them, they would be destroyed. No-one else offered to record them, even though they were popular — the Paul Irish talk had 110 people signed up to attend, and more than the same number again waitlisted who couldn’t go because they wouldn’t fit in the room. So there were more people in Boston who didn’t get to see the talk (but wanted to) than who did, even before we start counting the rest of the world’s interest in technical talks.
I’m happy that I’m able to help disseminate knowledge from Boston, which has an abundance of incredibly smart people living here or visiting, to wherever in the world you’re reading from now. But I’m also sad, because there are far more talks that I don’t go to here, and I expect most of those aren’t being recorded.
We’re technologists, right? So this should be easy. It’s not like I went to video camera school:
The equipment I’m using (Panasonic Lumix G2 camera and Lumix 20mm f/1.7 lens) costs under USD $800. Maybe it could be cheaper; maybe a recent cellphone (HTC One or Galaxy S4?) would be adequate.
I use a $20 tripod which is half broken.
I don’t use an external audio recorder (just the camera’s microphone) so the audio is noisier than it could be.
My camera’s sensor is small so it doesn’t have great low-light performance, and it records 720p instead of 1080p.
Sometimes the refresh rate/frequency of the projector is out of sync with the camera and there are strobing colors going across the screen in the final video. I don’t think I can do anything about this on the camera’s side?
I don’t do any editing because I don’t have time; I just upload the raw video file to YouTube and use YouTube’s “crop” feature to trim the start and end, that’s it.
I’d really like to know what the right answer is here. Am I overestimating how important it is to record these, and how privileged I am to be somewhere where there’s an interesting talk happening almost every day? Is owning a device that can record HD video for around 90 mins rare, even amongst well-paid developers and designers? If the presenter just recorded a screencast of their laptop with audio from its microphone, is that good enough or is that too boring for a full-length talk?
Might part of the problem be that people don’t know how to find videos of technical talks (I don’t know how anyone would find these unless they were randomly searching YouTube) so there isn’t as much demand as there should be — is there a popular website for announcing new recordings of tech talks somewhere? Maybe I just need to write up a document that describes how to record talks with a minimum of hassle and make sure people see it? Do we need to make a way for someone to signify their interest in having an upcoming talk be recorded, so that a team of volunteer videographers could offer to help with that?
WebRTC is incredibly exciting, and is starting to see significant deployment: it’s available by default in Chrome and Firefox releases now. Most people think of WebRTC as an API for video calling, but there’s a general purpose method for directly sharing data between web browsers (even when they’re behind NAT) in there if you look harder. For example:
P does peer-to-peer mesh networking in JavaScript.
TowTruck allows you to add collaboration features (collaborative text editing, text chat, voice chat) to websites.
PeerCDN forms a network from a site’s visitors, and uses it to offload serving up static content away from the web server and on to the networked peers.
The Tor Project is interested in using WebRTC to enable volunteers with JavaScript-enabled web browsers to become on-ramps onto the Tor network for users under censorship, as part of the Flash Proxies project. The idea is that censoring organizations may block the public Tor relays directly, but they can’t easily block every random web browser who might route traffic for those relays over WebRTC, especially if each web browser’s proxy is short-lived.
All of this activity means that we might finally be close to solving — amongst other important world problems — the scourge of xkcd.com/949:
xkcd: File Transfer, used under CC-BY-NC 2.5.
I wanted to experiment with WebRTC and understand its datachannels better, and I also felt like the existing code examples I’ve seen are unsatisfying in a specific way: it’s a peer-to-peer protocol, but the first thing you do (for example, on sites like conversat.io) is have everyone go to the same web server to find each other (this is called “signaling” in WebRTC) and share connection information.
If we’re going to have a peer-to-peer protocol, can’t we use it without all visiting the same centralized website first? Could we instead make a WebRTC app that just runs out of a file:/// path on your local disk, even if it means you have to manually tell the person you’re trying to talk to how to connect to you?
It turns out that we can: I’ve created a serverless-webrtc project on GitHub that decouples the “signaling server” exchange of connection information from the WebRTC code itself. To run the app:
You’ll be asked whether you want to create or join a channel, and then you’re prompted to manually send the first party’s “WebRTC offer” to the second party (for example, over an instant message chat) and then to do the same thing with the second party’s “WebRTC answer” reply back. Once you’ve done that, the app provides text chat and file transfer between peers, all without any web server. (A STUN server is still used to find out your external IP for NAT-busting.)
There are open issues that I’d be particularly happy to receive pull requests for:
#1: The code doesn’t work on Chrome yet. Chrome is behind Firefox as far as DataChannels are concerned — Chrome doesn’t yet have support for binary transfers, or for “reliable” (TCP, not UDP) channels (Firefox does). These are both important for file transfers.
#2: Large file transfers often fail, or even hang the browser, but small transfers seem to work every time. I’m not sure whose code is at fault yet.
Thanks for reading this far! Here’s to the shared promise of actually being able to use the Internet to directly share files with each other some time soon.
Regular expressions are a powerful text transformation tool. Any UNIX geek will tell you that. It’s so deeply ingrained into our culture, that we even make jokes about it. Another thing that we also love is having a powerful extension language at hand, and Lisp is one of the most powerful extension languages around (and of course, we make jokes about that too).
Emacs, one of the most famous Lisp applications today, has for a while now the ability to combine both of these, to reach entirely new levels of usefulness. Combining regular expressions and Lisp can do really magical things.
An example that I recently used a few times is parsing & de-humanizing numbers in dstat output. The output of dstat includes numbers that are printed with a suffix, like ‘B’ for bytes, ‘k’ for kilobytes and ‘M’ for megabytes, e.g.:
So if you want to graph one of the columns, it’s useful to convert all the numbers in the same unit. Bytes would be nice in this case.
Separating all columns with ‘|’ characters is a good start, so you can use e.g. a CSV-capable graphing tool, or even simple awk scripts to extract a specific column. ‘C-x r t’ can do that in Emacs, and you end up with something like this:
The leading and trailing ‘|’ characters are there so we can later use orgtbl-mode, an awesome table editing and realignment tool of Emacs. Now to the really magical step: regular expressions and lisp working together.
What we would like to do is convert text like “408B” to just “408″, text like “1268k” to the value of (1268 * 1024), and finally text like “67M” to the value of (67 * 1024 * 1024). The first part is easy:
M-x replace-regexp RET \([0-9]+\)B RET \1 RET
This should just strip the “B” suffix from byte values.
For the kilobyte and megabyte values what we would like is to be able to evaluate an arithmetic expression that involves \1. Something like “replace \1 with the value of (expression \1)“. This is possible in Emacs by prefixing the substitution pattern with \,. This instructs Emacs to evaluate the rest of the substitution pattern as a Lisp expression, and use its string representation as the “real” substitution text.
So if we match all numeric values that are suffixed by ‘k’, we can use (string-to-number \1) to convert the matching digits to an integer, multiply by 1024 and insert the resulting value by using the following substitution pattern:
\,(* 1024 (string-to-number \1))
The full Emacs command would then become:
M-x replace-regexp RET \([0-9]+\)k RET \,(* 1024 (string-to-number \1)) RET
This, and the byte suffix removal, yield now the following text in our Emacs buffer:
Finally, to align everything in neat, pipe-separated columns, we enable M-x orgtbl-mode, and type “C-c C-c” with the pointer somewhere inside the transformed dstat output. The buffer now becomes something usable for pretty-much any graphing tool out there:
The trick of combining arbitrary Lisp expressions with regexp substitution patterns like \1, \2 … \9 is something I have found immensely useful in Emacs. Now that you know how it works, I hope you can find even more amusing use-cases for it.
Contributions and projects start off well, and energy might
wane depending on time and life factors. Even contributing to
tools used by many of the members of the community like [Leiningen] and
Clojars doesn't prevent it. What helps is direct involvement by
someone else.
This really resonated with me because it emphasizes that people are
more important than programs. For me sharing is the thing that
makes programming even worth doing in the first place. So it got
me thinking about different technologies and what kind of people
they're good for helping.
If you follow my writing it will be obvious that I enjoy working
in Emacs and Clojure. While these are among the most powerful,
flexible technologies I know of, collaborating with others on
tools for Emacs and Clojure basically limits me to working with
professional programmers, because both environments are very poor
from a beginner's perspective. If I'm working solo or on a team of
seasoned hackers, I'll definitely be most effective with
Clojure. If my primary goal is to interact with the widest group
of programmers possible, I would use Ruby as it's the most
commonly-used language I can bring myself to
use. But if I want to reach out to people who
don't already spend all day thinking about functions and data
structures, well that's another thing entirely.
This is particularly relevant for me personally as a father. I'm
taking an active role in the education of my sons, and of course I
think technical literacy must be an important part of it. But when you
look at how computers used in traditional educational settings,
you're much more likely to see computers programming children than
children programming computers. So I've been looking for ways to
foster technical skills and encourage algorithmic thinking in
engaging ways that can keep the attention of my five-year-old.
In the
book Mindstorms,
Seymour Papert describes the shift from concrete reasoning to
formal reasoning as one of the main transitions children undergo
as they learn to think like adults. One of the design goals of the
Logo system he created was to provide transitional concepts to
bridge the gap between the two.
Children interact with Logo by giving commands to an onscreen
object known as
the turtle.
While the turtle lives in the abstract world of geometry comprised
of points and lines, children are able to identify with it since
they tell it to move in ways which they can relate to—it has
a heading and position, and it turns and moves forward and
backwards just like they do. Because the turtle's movements on the
screen are isomorphic to their own physical movements, it gives
them a model to help them grasp abstract geometrical concepts
though they're only used to thinking in concrete terms. And
Piagetian
learning—ambient, natural learning which children are so
adept at doing without study—is all a matter of building
models of the world.
Most people know Logo from its original triangle-turtle-centric
incarnation, but in Mindstorms Papert describes Logo as more of an
educational philosophy than any single program, language, or
implementation. A more recent version of Logo
is Scratch, a drag-and-drop
visual programming environment from MIT's Media Lab targeting
school children. Since my older son is an early reader he's been
able to construct simple scripts (with some guidance) for the
characters within Scratch, watching them interact with each other
and even in some
cases the
outside world.
While it's been lots of fun to come up with ideas and talk
through how we'd bring them to life on the screen, one of the most
rewarding parts is watching his problem-solving abilities
develop. Papert talks about how children are often afraid to try
things for fear of failure, but Scratch teaches that debugging is
a normal part of making things work. Rather than "does it work",
the question becomes "how can we make it work?" This was
demonstrated the other day (outside the context of Scratch) when
he was putting together
some Snap Circuits:
Of course the goal is not to produce "little programmers". It's
primarily about developing the ability to think systematically,
but it extends beyond that into getting them thinking about
thinking itself. In some sense once they're in the habit of asking
the right epistemological questions, the parent or teacher almost
just needs to get out of the way and let them explore. At that
point the process of discovering a topic with someone is
much more rewarding than telling facts at them.
San Jose, Calif., (May 13, 2013)
Emacspeak: Redefining Accessibility In The Era Of Cloud Computing
–Zero cost of upgrades/downgrades makes priceless software affordable!
Emacspeak Inc (NASDOG: ESPK) --http://emacspeak.sf.net--
announces the immediate world-wide availability of Emacspeak 38.0
(FreeDog) –a powerful audio desktop for leveraging today's
evolving data, social and service-oriented Web cloud.
1.1 Investors Note:
With several prominent tweeters expanding coverage of
#emacspeak, NASDOG: ESPK has now been consistently trading over
the social net at levels close to that once attained by DogCom
high-fliers—and as of May 2013 is trading at levels close to
that achieved by once better known stocks in the tech sector.
1.2 What Is It?
Emacspeak is a fully functional audio desktop that provides
complete eyes-free access to all major 32 and 64 bit operating
environments. By seamlessly blending live access to all aspects
of the Internet such as Web-surfing, blogging, social computing
and electronic messaging into the audio desktop, Emacspeak
enables speech access to local and remote information with a
consistent and well-integrated user interface. A rich suite of
task-oriented tools provides efficient speech-enabled access to
the evolving service-oriented social Web cloud.
1.3 Major Enhancements:
Get directions and find Places via Google Maps. ⛯
Preliminary support for Eclipse integration via Eclim. ⛅
Speech-enabled GTags (Global) for code browsing. 🌐
Updated to work with advice implementation in Emacs 24.3. 🌚
Updated Web search wizards ꩜
Updated URL templates ♅
Plus many more changes too numerous to fit in this margin ∞
1.4 Establishing Liberty, Equality And Freedom:
Never a toy system, Emacspeak is voluntarily bundled with all
major Linux distributions. Though designed to be modular,
distributors have freely chosen to bundle the fully integrated
system without any undue pressure—a documented success for
the integrated innovation embodied by Emacspeak. As the system
evolves, both upgrades and downgrades continue to be available at
the same zero-cost to all users. The integrity of the Emacspeak
codebase is ensured by the reliable and secure Linux platform
used to develop and distribute the software.
Extensive studies have shown that thanks to these features, users
consider Emacspeak to be absolutely priceless. Thanks to this
wide-spread user demand, the present version remains priceless
as ever—it is being made available at the same zero-cost as
previous releases.
At the same time, Emacspeak continues to innovate in the area of
eyes-free social interaction and carries forward the
well-established Open Source tradition of introducing user
interface features that eventually show up in luser environments.
On this theme, when once challenged by a proponent of a
crash-prone but well-marketed mousetrap with the assertion
"Emacs is a system from the 70's", the creator of Emacspeak
evinced surprise at the unusual candor manifest in the assertion
that it would take popular idiot-proven interfaces until the year
2070 to catch up to where the Emacspeak audio desktop is
today. Industry experts welcomed this refreshing breath of
Courage Certainty and Clarity (CCC) at a time when users are
reeling from the Fear Uncertainty and Doubt (FUD) unleashed by
complex software systems backed by even more convoluted press
releases.
1.5 Independent Test Results:
Independent test results have proven that unlike some modern (and
not so modern) software, Emacspeak can be safely uninstalled without
adversely affecting the continued performance of the computer. These
same tests also revealed that once uninstalled, the user stopped
functioning altogether. Speaking with Aster Labrador, the creator of
Emacspeak once pointed out that these results re-emphasize the
user-centric design of Emacspeak; "It is the user –and not the
computer– that stops functioning when Emacspeak is uninstalled!".
1.5.1 Note from Aster,Bubbles and Tilden:
UnDoctored Videos Inc. is looking for volunteers to star in a
video demonstrating such complete user failure.
Emacspeak 38.0 is the latest in a series of award-winning
releases from Emacspeak Inc.
Emacspeak 37.0 continues the tradition of delivering robust
software as reflected by its code-name. Emacspeak 36.0 enhances
the audio desktop with many new tools including full EPub support
— hence the name EPubDog. Emacspeak 35.0 is all about teaching
a new dog old tricks — and is aptly code-named HeadDog in honor
of our new Press/Analyst contact. emacspeak-34.0 (AKA Bubbles)
established a new beach-head with respect to rapid task
completion in an eyes-free environment. Emacspeak-33.0 AKA
StarDog brings unparalleled cloud access to the audio
desktop. Emacspeak 32.0 AKA LuckyDog continues to innovate via
open technologies for better access. Emacspeak 31.0 AKA TweetDog
— adds tweeting to the Emacspeak desktop. Emacspeak 30.0 AKA
SocialDog brings the Social Web to the audio desktop—you cant but
be social if you speak! Emacspeak 29.0—AKAAbleDog—is a testament
to the resilliance and innovation embodied by Open Source
software—it would not exist without the thriving Emacs community
that continues to ensure that Emacs remains one of the premier
user environments despite perhaps also being one of the
oldest. Emacspeak 28.0—AKA PuppyDog—exemplifies the rapid pace of
development evinced by Open Source software. Emacspeak 27.0—AKA
FastDog—is the latest in a sequence of upgrades that make
previous releases obsolete and downgrades unnecessary. Emacspeak
26—AKA LeadDog—continues the tradition of introducing innovative
access solutions that are unfettered by the constraints inherent
in traditional adaptive technologies. Emacspeak 25 —AKA ActiveDog
—re-activates open, unfettered access to online
information. Emacspeak-Alive —AKA LiveDog —enlivens open,
unfettered information access with a series of live updates that
once again demonstrate the power and agility of open source
software development. Emacspeak 23.0 – AKA Retriever—went the
extra mile in fetching full access. Emacspeak 22.0 —AKA GuideDog
—helps users navigate the Web more effectively than ever
before. Emacspeak 21.0 —AKA PlayDog —continued the Emacspeak
tradition of relying on enhanced productivity to liberate
users. Emacspeak-20.0 —AKA LeapDog —continues the long
established GNU/Emacs tradition of integrated innovation to
create a pleasurable computing environment for eyes-free
interaction. emacspeak-19.0 –AKA WorkDog– is designed to
enhance user productivity at work and leisure. Emacspeak-18.0
–code named GoodDog– continued the Emacspeak tradition of
enhancing user productivity and thereby reducing total cost of
ownership. Emacspeak-17.0 –code named HappyDog– enhances user
productivity by exploiting today's evolving WWW
standards. Emacspeak-16.0 –code named CleverDog– the follow-up
to SmartDog– continued the tradition of working better, faster,
smarter. Emacspeak-15.0 –code named SmartDog–followed up on
TopDog as the next in a continuing a series of award-winning
audio desktop releases from Emacspeak Inc. Emacspeak-14.0 –code
named TopDog–was the first release of this
millennium. Emacspeak-13.0 –codenamed YellowLab– was the
closing release of the 20th. century. Emacspeak-12.0 –code named
GoldenDog– began leveraging the evolving semantic WWW to provide
task-oriented speech access to Webformation. Emacspeak-11.0
–code named Aster– went the final step in making Linux a
zero-cost Internet access solution for blind and visually
impaired users. Emacspeak-10.0 –(AKA Emacspeak-2000) code named
WonderDog– continued the tradition of award-winning software
releases designed to make eyes-free computing a productive and
pleasurable experience. Emacspeak-9.0 –(AKA Emacspeak 99) code
named BlackLab– continued to innovate in the areas of speech
interaction and interactive accessibility. Emacspeak-8.0 –(AKA
Emacspeak-98++) code named BlackDog– was a major upgrade to the
speech output extension to Emacs.
Emacspeak-95 (code named Illinois) was released as OpenSource on
the Internet in May 1995 as the first complete speech interface
to UNIX workstations. The subsequent release, Emacspeak-96 (code
named Egypt) made available in May 1996 provided significant
enhancements to the interface. Emacspeak-97 (Tennessee) went
further in providing a true audio desktop. Emacspeak-98
integrated Internetworking into all aspects of the audio desktop
to provide the first fully interactive speech-enabled WebTop.
About Emacspeak:
Originally based at Cornell (NY)
http://www.cs.cornell.edu/home/raman –home to Auditory User
Interfaces (AUI) on the WWW– Emacspeak is now maintained on
GoogleCode --http://code.google.com/p/emacspeak —and
Sourceforge —http://emacspeak.sf.net. The system is mirrored
world-wide by an international network of software archives and
bundled voluntarily with all major Linux distributions. On
Monday, April 12, 1999, Emacspeak became part of the
Smithsonian's Permanent Research Collection on Information
Technology at the Smithsonian's National Museum of American
History.
The Emacspeak mailing list is archived at Vassar –the home of the
Emacspeak mailing list– thanks to Greg Priest-Dorman, and provides a
valuable knowledge base for new users.
1.8 Press/Analyst Contact: Tilden Labrador
Going forward, Tilden acknowledges his exclusive monopoly on
setting the direction of the Emacspeak Audio Desktop, and
promises to exercise this freedom to innovate and her resulting
power responsibly (as before) in the interest of all dogs.
**About This Release:
Windows-Free (WF) is a favorite battle-cry of The League Against
Forced Fenestration (LAFF). –see
http://www.usdoj.gov/atr/cases/f3800/msjudgex.htm for details on
the ill-effects of Forced Fenestration.
CopyWrite )C( Aster and Hubbell Labrador. All Writes Reserved.
HeadDog (DM), LiveDog (DM), GoldenDog (DM), BlackDog (DM) etc., are Registered
Dogmarks of Aster, Hubbell and Tilden Labrador. All other dogs belong to
their respective owners.
One of my recent innocent tweet about Gerrit vs Github triggered much more
reponses and debate that I expected it to. I realize that it might be worth
explaining a bit what I meant, in a text longer than 140 characters.
I'm having a hard time now contributing to projects not using Gerrit. Github
isn't that good.
I always looked at Github from a distant eye, mainly because I always
disliked their pull-request handling, and saw no value in the social hype it
brings. Why?
One click away isn't one click effort
The pull-request system looks like an incredible easy way to contribute to
any project hosted on Github. You're a click away to send your contribution
to any software. But the problem is that any worthy contribution isn't an
effort of a single click.
Doing any proper and useful contribution to a software is never done right
the first time. There's a dance you will have to play. A slowly rhythmed
back and forth between you and the software maintainer or team. You'll have
to dance it until your contribution is correct and can be merged.
But as a software maintainer, not everybody is going to follow you on this
choregraphy, and you'll end up with pull-request you'll never get finished
unless you wrap things up yourself. So the gain in pull-requests here, isn't
really bigger than a good bug report in most cases.
This is where the social argument of Github isn't anymore. As soon as you're
talking about projects bigger than a color theme for your favorite text
editor, this feature is overrated.
Contribution rework
If you're lucky enough, your contributor will play along and follow you on
this pull-request review process. You'll make suggestions, he will listen
and will modify his pull-request to follow your advice.
At this point, there's two technics he can use to please you.
Technic #1: the Topping
Github's pull-requests invite you to send an entire branch, eclipsing the
fact that it is composed of several commits. The problem is that a lot of
one-click-away contributors do not masterize Git and/or do not make efforts
to build a logical patchset, and nothing warns them that their branch
history is wrong. So they tend to change stuff around, commit, make a
mistake, commit, fix this mistake, commit, etc. This kind of branch is
composed of the whole brain's construction process of your contributor, and
is a real pain to review. To the point I quite often give up.
A typical case: 3 commits to build a 4 lines long file.
Without Github, the old method that all software used, and that many
software still use (e.g. Linux), is to send a patch set over e-mail (or any
other medium like Gerrit). This method has one positive effect, that it
forces the contributor to acknowledge the list of commits he is going to
publish. So, if the contributor he has fixup commits in his history, they
are going to be seen as first class citizen. And nobody is going to want to
see that, neither your contributor, nor the software maintainers. Therefore,
such a system tend to push contributors to write atomic, logical and
self-contained patchset that can be more easily reviewed.
Technic #2: the History Rewriter
This is actually the good way to build a working and logical patchset using
Git. Rewriting history and amending problematic patches using the famous
git rebase --interactive trick.
The problem is that if your contributor does this and then repush the branch
composing your pull-request to Github, you will both lose the previous
review done, each time. There's no history on the different versions of the
branch that has been pushed.
In the old alternative system like e-mail, no information is lost when
reworked patches are resent, obviously. This is far better because it eases
the following of the iterative discussions that the patch triggered.
Of course, it would be possible for Github to enhance this and fix it, but
currently it doesn't handle this use case correctly..
Exercise for the doubtful readers: good luck finding all revisions of my
patch in the pull-request #157 of Hy.
A quick look at OpenStack workflow
It's not a secret for anyone that I've been contributing to OpenStack as a
daily routine for the last 18 months. The more I contribute, the more I like
the contribution workflow and process. It's already
well and longly described on the wiki,
so I'll summarize here my view and what I like about it.
Gerrit
To send a contribution to any OpenStack project, you need to pass via
Gerrit. This is way simpler than doing a pull-request on Github actually,
all you have to do is do your commit(s), and type
git review. That's it. Your
patch will be pushed to Gerrit and available for review.
Gerrit allows other developers to review your patch, add comments anywhere
on it, and score your patch up or down. You can build any rule you want for
the score needed for a patch to be merged; OpenStack requires one positive
scoring from two core developers before the patch is merged.
Until a patch is validated, it can be reworked and amended locally using
Git, and then resent using
git review again. That simple.
The historic and the different version of the patches are available, with
the whole comments. Gerrit doesn't lose any historic information on your
workflow.
Finally, you'll notice that this is actually the same kind of workflow
projects use when they work by patch sent over e-mail. Gerrit just build a
single place to regroup and keep track of patchsets, which is really handy.
It's also much easier for people to actually send patch using a command line
tool than their MUA or git send-email.
Gate testing
Testing is mandatory for any patch sent to OpenStack. Unit tests and
functionnals test are run for each version of each patch of the patchset
sent. And until your patch passes all tests, it will be impossible to
merge it.
Yes, this implies that all patches in a patchset must be working commits and
can be merged on their own, without the entire patchset going in! With such
a restricution, it's impossible to have "fixup commits" merged in your
project and pollute the history and the testability of the project.
Once your patch is validated by core developers, the system checks that
there is not any merge conflicts. If there's not, tests are re-run, since
the branch you are pushing to might have changed, and if everything's fine,
the patch is merged.
This is an uncredible force for the quality of the project. This implies
that no broken patchset can ever sneak in, and that the project pass always
all tests.
Conclusion: accessibility vs code review
In the end, I think that one of the key of any development process, which is
code review, is not well covered by Github pull-request system. It is, along
with history integrity, damaged by the goal of making contributions easier.
Choosing between these features is probably a trade-off that each project
should do carefully, considering what are its core goals and the quality of
code it want to reach.
I tend to find that OpenStack found one of the best trade-off available
using Gerrit and plugging testing automation via Jenkins on it, and I would
probably recommend it for any project taking seriously code reviews and
testing.
This week, I re-implemented my main website. The text on the old one needed to be revised heavily (in fact I removed almost all of it except a short bio and some longer articles) and I wanted to experiment with a more colorful look. I need to revise the stylesheet a bit more to give a better experience on small screens (mobile), but so far I’m pleased with the result.
The new site html is generated completely by emacs’ org-mode project publishing functionality (the old one used a custom bunch of perl scripts generating from HTML snippets and perl POD documents). This hopefully means it’ll be easier to add and revise content, and it also provides nice syntax highlighting for any code snippets I put in my articles.
The whole switchover was pretty smooth, especially once I wrote a bit of elisp to roughly prepare conversion from POD formatting to org:
(defun pod2org
(point mark)
"Rougly convert region from pod syntax to org-mode"
(interactive "r")
(replace-regexp "C<\\([^>]+\\)>" "=\\1=" nil point mark)
(replace-regexp "I<\\([^>]+\\)>" "/\\1/" nil point mark)
(replace-regexp "B<\\([^>]+\\)>" "*\\1*" nil point mark)
(replace-regexp "^=head1 \\(.*\\)" "* \\1" nil point mark)
(replace-regexp "^=head2 \\(.*\\)" "** \\1" nil point mark)
(replace-regexp "^=head3 \\(.*\\)" "*** \\1" nil point mark)
(replace-regexp "^=head4 \\(.*\\)" "**** \\1" nil point mark)
(replace-regexp "^=head5 \\(.*\\)" "***** \\1" nil point mark))
Very basic, but pretty useful.
Now, I’m considering replacing WordPress for this blog with org-mode too. That probably requires a bit more investigation. I want to keep at least the tags/categories feature and related rss feeds, and I’m not sure if that’s available for org at the moment.