Developing an emacs package

2022-06-01

emacs , packages

updated: 18.06.2022

I wrote a package long time ago, maybe 6 years or so. I switched my employer not long ago, the new company uses clockodo to track times used on a task. Clockodo serves a clean webpage and some packages for major operating systems. But the lack of emacs support is a pain point for me since I use emacs most of the time and it’s the first program running and the last one I close.

So, I decided to write a simple integration using the API clockodo provides. This blog post tries to give an overview for all the steps taken. The last time I wrote a package, I missed the overall documentation. Thus, I start from zero again.

Prerequisites

Before starting the package development I searched a bit through the latest ressources I found on that topic.

Elisp Snippets - Not related by a good source for quick snippets
eldev - A emacs-based build tool like cask but not cask
atomic objects blog post - A nice blog post related to package development with commit-mode
System Crafters blog - A short introduction into minor modes
Emacs package dev handbook - Extensive ressources about package development
Elisp cheat sheet - A cheat sheet is always nice to have

I expect basic emacs-lisp and programming knowledge.

Starting the package

Create a new git repository with git init <name> and open a new file/buffer from emacs in this repository.

An emacs package is just a single or bunch of elisp files which are loaded at runtime. For the clockodo package we only need a simple file which is easier to handle. Emacs uses a common scheme to correctly identify and handle packages:

 1;;; clockodo.el --- Integrate the clockodo timer into emacs. -*- lexical-binding: t -*-
 2
 3;; Copyright (c) ratzeputz
 4
 5;; Author: ratzeputz
 6;; Version: 0.9
 7;; Package-Require: ((example "1.0"))
 8;; Keywords: time, organization
 9;; URL: https://github.com/santifa/clockodo-el
10
11;;; Commentary:
12
13;; This package provides a minor mode to interact with the clockodo api
14;; with simple commands.
15
16;;;; Installation
17
18;;;;; MELPA
19
20;; If you installed from MELPA, you're done.
21
22;;;;; Manual
23
24;; (require 'package-name)
25
26;;;; Usage
27
28;; Run one of these commands:
29
30;; `package-name-command': Frobnicate the flange.
31
32;;;; Tips
33
34;; + You can customize settings in the `package-name' group.
35
36;;;; Credits
37
38;;; Code:
39
40;; Some code here
41
42(provide 'clockodo)
43;;; clockodo.el ends here

The top line starts with three ;;; and provides the name and a short description about the package. Notice the usage of the more modern lexical binding with lexical-binding: t.

The next section states some common informations about the author and the package it-self, which are used by the package management systems. The Version: is mandatory in this section. Note the commentary section starts with three ;;; again. Here is the place for the whole documentation and usage of the package. The code marker starts with three ;;; again and is important to show the end of the commentary section. At the end the package is closed by (provide 'clockodo) and the commentary below is also needed. I took the template from here.

Package basics

The clockodo package is only a small one which should wrap parts of the api to start and stop the clock and get some informations from clockodo. We have several predefined options for a new mode in emacs:

A minor mode which are mostly small packages for providing or enhancing emacs.
A major mode is mostly related to a programming language and provides options for for syntax highlighting and so one.
A commit mode is used for interacting with external commands like compilers which need fixed buffers.

For clockodo I choose a minor mode. Additionally, authentication and dealing with HTTP requests and responses are needed.

First, we load the dependencies which are needed for our package. All external packages (not provided by emacs itself) are mandatory within the ;; Package-Requires: section to work correctly when installed from a package management.

1(require 'cl-lib)
2(require 'auth-source)
3(require 'request)
4(require 'ts)
5(require 'org)

Then we introduce some customizations for the user and define package variables. Use a common prefix for your functions and variables. I use clockodo in my example.

 1;; Provide customizations
 2(defgroup clockodo nil
 3  "Customize the clockodo integration."
 4  :group 'tools)
 5
 6(defcustom clockodo-api-url "https://my.clockodo.com/api/"
 7  "The url to the api endpoint of clockodo."
 8  :type 'string
 9  :group 'clockodo)
10
11(defcustom clockodo-store-api-credential t
12  "Whether to store the api credentials or not."
13  :type 'boolean
14  :group 'clockodo)
15
16(defcustom clockodo-keymap-prefix "C-c C-#"
17  "The prefix for the clockodo mode key bindings."
18  :type 'string
19  :group 'clockodo)
20
21(defvar clockodo-debug nil
22  "Shows more information in the message buffer.")
23
24(defvar clockodo-user-id nil
25  "The user id needed for most requests.")
26
27(defun clockodo--key (key)
28  "Convert a key into a prefixed one.
29
30KEY The key which should be prefixed."
31  (kbd (concat clockodo-keymap-prefix " " key)))
32
33(defun clockodo--with-face (str &rest face-plist)
34  "Enclose a string with a face list.
35
36STR The string which gets a face.
37FACE-PLIST The list of faces."
38  (propertize str 'face face-plist))

This is just a small excerpt from all definitions used for the clockodo package. The last two functions are useful helpers. The clockodo--key function provides an easy way to bind keys to the package keymap. The clockodo--with-face function provides an easy way to color some part of the output using the emacs faces.

Now, the skeleton definition for a minor mode with the define-minor-mode macro:

 1;;;###autoload
 2(define-minor-mode clockodo-mode
 3  "Define the mode for interacting with clockodo."
 4  :init-value nil
 5  :lighter " clockodo"
 6  :group 'clockodo
 7  :global t
 8  :keymap
 9  (list (cons (clockodo--key "s" #'clockodo-start-clock)))
10
11  (if clockodo-mode
12      (message "clockodo mode enabled")
13    (message "clockodo mode disabled")))

The first function defines the minor mode clockodo-mode which is:

:init-value not enabled by default
:lighter as the name clockodo on the modeline
:group uses the customization group clockodo
:global the mode is a global mode
:keymap the modes keymap

Additionally, we could provide some function body to do some setup or cleanup. This macro also creates a hook which is run if the mode is turned on or off.

Authentication

Now, the skeleton is ready. To interact with the clockodo api, credentials are needed. Emacs provides the auth-source package to deal with authentication and credentials.

 1(defun clockodo--get-credentials ()
 2  "Return the stored api credentials for clockodo.
 3
 4The user is asked for credentials if none exists for the api url."
 5  (let* ((auth-source-creation-prompts
 6          '((api-user . "Clockodo user: ")
 7            (api-token . "Clockodo api token: ")))
 8         (api-creds (nth 0 (auth-source-search :max 1
 9                                               :host clockodo-api-url
10                                               :require '(:user :secret)
11                                               :create t))))
12    (when api-creds
13      (list (plist-get api-creds :user)
14            (let ((api-pass (plist-get api-creds :secret)))
15              (if (functionp api-pass)
16                  (funcall api-pass)
17                api-pass))
18            (plist-get api-creds :save-function)))))
19
20(defun clockodo--initialize (api-creds)
21  "A thin wrapper which set user variables for the next requests.
22
23API-CREDS The credentials for the clockodo api."
24  (when (or (null clockodo-user-id)
25           (null clockodo-default-service-id)
26           (null clockodo-default-customer-id))
27    (let* ((response (request-response-data
28                      (clockodo--get-user (nth 0 api-creds) (nth 1 api-creds)))))
29      (let-alist response
30        (setq clockodo-user-id .user.id
31              clockodo-default-service-id .company.default_services_id
32              clockodo-default-customer-id .company.default_customers_id)))))
33
34(defun clockodo-get-credentials ()
35  "A wrapper which takes long-time storing of credentials into account.
36
37It knocks the clockodo api to test if the credentials are valid before storing them."
38  (let ((api-credentials (clockodo--get-credentials)))
39    (when clockodo-store-api-credential
40      (when (functionp (nth 2 api-credentials))
41        (let ((return-code (request-response-status-code
42                            (clockodo--get-user
43                             (nth 0 api-credentials)
44                             (nth 1 api-credentials)))))
45          (when (eq return-code 200)
46            (funcall (nth 2 api-credentials))))))
47    (clockodo--initialize api-credentials)
48    api-credentials))

The function clockodo--get-credentials ask the user for credentials and splits them into the username, password and a save function to store the credentials over a single emacs session. The clockodo--initialize function pokes the clockodo api and stores some basic informations about the user. The clockodo-get-credentials ties them together. First it asks for the credentials either from the user or the auth store, than tests if we get valid informations from the clockodo api and if so stores the credentials for longe time use. The user can toggle this behaviour through the clockodo-store-api-credential variable.

Dealing with http requests and responses

To interact with the clockodo api I used the request package which is easy to use and well documented. First set the header for every request taken.

1(defun clockodo--create-header (user token)
2  "This create the header for requests against the clockodo api.
3
4USER The username used to authenticate the request.
5TOKEN The token used to authenticate the request."
6  `(("X-ClockodoApiUser" . ,user)
7        ("X-ClockodoApiKey" . ,token)
8        ("X-Clockodo-External-Application" . ,(concat "clockodo-el;" user))))

Afterwards, the basic requests are modelled as a function which gets the credentials and the partial URL for the requests. As an example, I modelled the get requests like this.

 1(defun clockodo--get-request(user token url-part)
 2  "This function abstracts a simple get request to the clockodo api.
 3
 4The result is a parsed json object.
 5USER The username used for the api request.
 6TOKEN The clockodo api token for the user.
 7URL-PART The full api part for the get request."
 8  (when clockodo-debug
 9    (message (concat "API-URL: " clockodo-api-url url-part)))
10  (let ((request-header (clockodo--create-header user token)))
11    (request (concat clockodo-api-url url-part)
12      :sync t
13      :headers request-header
14      :parser 'json-read
15      :error (cl-function (lambda (&rest args &key error-thrown &allow-other-keys)
16                            (message "Got error: %S" error-thrown))))))

Now, every get request is nothing more than a couple of lines.

 1(defun clockodo--get-clock (user token)
 2  "Request the current state of the clockodo clock service.
 3
 4USER The username used for the api request
 5TOKEN The clockodo api token for the user"
 6  (clockodo--get-request user token "/v2/clock"))
 7
 8(defun clockodo--get-all-services (user token)
 9  "Request the list of services defined within the company.
10
11USER The username used for the api request.
12TOKEN The clockodo api token for the user."
13  (clockodo--get-request user token "/services"))

Maybe there is a better way to deal with the repeating user and token variables (Let me know).

Special buffers

Besides using the clock, I wanted to show reports generated from the clockodo user data. For this purpose I use a buffer which is set into special mode.

 1(defun clockodo--build-report-buffer (name header body)
 2"Generate a new report buffer and insert the return of body.
 3
 4NAME The name of the report buffer.
 5HEADER A two element list with a header-line and a page heading.
 6BODY A function that fills the buffer."
 7(let* ((buffer (get-buffer-create (format "*clockodo-%s*" name)))
 8       (inhibit-read-only t))
 9  (if (get-buffer-window buffer)
10      (pop-to-buffer-same-window buffer)
11    (switch-to-buffer-other-window buffer))
12  (with-current-buffer buffer
13    (erase-buffer)
14    (special-mode)
15    (setq header-line-format (car header))
16    (insert (cdr header))
17    (funcall body))))

This function creates a new buffer prefixed with clockodo or takes an existing one. It is set into read-only mode and the first part of the header is set as header-line-format. The second part is a block of general information about the api user. The last step, a function is executed which renders the report body.

request returns the data as alist, thus the following function is handy.

1(let-alist (request-response-data response)
2  (do-something-with .entries))

This binds the alist properties as variables of the form .variable.subvariable.

Hints and Takeaways

This post should give me and others some hints when developing a new emacs package or dealing with some api. It is not a real template but more a set of hints since the documentation for some packages or emacs parts are not well-suited for beginners.

Lastly, another short list of general hints:

Use melpazoid as Github action
Use package-lint to get the documentation correctly
Use internal function, then cl-lib and if nothing helps external libs
Keep your functions small and composable
Take your time reading documentation and code
Reading code of others and other packages helps to understand certain patterns of elisp
Use when and unless instead of open if forms
Provide sensible default keybindings
Prefer functions with a toggle effect for keybindings
Use defvar and defcustom to allow easy modifications and prevent void variable assignments
Use (or nullable-var default) to set defaults

Thanks for the attentation and leave me a comment.