Developing an emacs package

2022-06-01

I wrote a package long time ago, maybe 6 years or so. Now, I switched my employer not long ago which uses clockodo to track times used on a task. Clockodo serves a clean webpage and some packages for major operating systems. But the lack of emacs support is a pain point for me since I use emacs most of the time and it’s the first program running and the last I close.

So, I decided to write a simple integration thanks to the API clockodo provides. This blog post tries to give an overview above all the steps taken. The last time I wrote a package I missed the overall documentation thus I start from zero again.

Prerequisites

Before starting the package development I searched a bit through the latest ressources I found on that topic.

I expect basic emacs-lisp and programming knowledge.

Starting the package

Create a new git repository with git init <name> and open a new file/buffer from emacs in this repository.

An emacs packages a just a single or bunch of elisp files which are loaded at runtime. For the clockodo package we only need a simple file which is easier to handle. Emacs uses a common scheme to correctly identify and handle packages:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

;;; clockodo.el --- Integrate the clockodo timer into emacs. -*- lexical-binding: t -*-

;; Copyright (c) ratzeputz

;; Author: ratzeputz
;; Version: 0.9
;; Package-Require: ((example "1.0"))
;; Keywords: time, organization
;; URL: https://github.com/santifa/clockodo-el

;;; Commentary:

;; This package provides a minor mode to interact with the clockodo api
;; with simple commands.

;;;; Installation

;;;;; MELPA

;; If you installed from MELPA, you're done.

;;;;; Manual

;; (require 'package-name)

;;;; Usage

;; Run one of these commands:

;; `package-name-command': Frobnicate the flange.

;;;; Tips

;; + You can customize settings in the `package-name' group.

;;;; Credits

;;; Code:

;; Some code here

(provide 'clockodo)
;;; clockodo.el ends here

The top line starts with three ;;; and provides the name and a short description about the package. Notice the usage of the more modern lexical binding with lexical-binding: t.

The next section sets some common informations about the author and the package it-self which are used by the package management systems. The Version: is mandatory in this section. Note the commentary section starts with three ;;; again. Here is the place for the whole documentation and usage of the package. The code marker starts with three ;;; again and is important to show the end of the commentary section. At the end the package is closed by (provide 'clockodo) and the commentary below is also needed. I toke the template from here.

Package basics

The clockodo package is only a small one which should wrap parts of the api to start and stop the clock and get some informations from clockodo. We have several predefined options for a new mode in emacs:

  • a minor mode which are mostly small packages for providing or enhancing emacs.
  • a major mode is mostly related to a programming language and provides options for for syntax highlighting and so one.
  • a commit mode is used for interacting with external commands like compilers which need fixed buffers.

For clockodo I choose a minor mode. Additionally, authentication and dealing with HTTP requests and responses are neede.

First we load the dependencies which are needed for our package. All external packages (not provided by emacs itself) are mandatory within the ;; Package-Requires: section to work correctly when installed from a package management.

1
2
3
4
5

(require 'cl-lib)
(require 'auth-source)
(require 'request)
(require 'ts)
(require 'org)

Then we introduce some customizations for the user and define package variables. Use a common prefix for your functions and variables. I use clockodo in my example.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

;; Provide customizations
(defgroup clockodo nil
  "Customize the clockodo integration."
  :group 'tools)

(defcustom clockodo-api-url "https://my.clockodo.com/api/"
  "The url to the api endpoint of clockodo."
  :type 'string
  :group 'clockodo)

(defcustom clockodo-store-api-credential t
  "Whether to store the api credentials or not."
  :type 'boolean
  :group 'clockodo)

(defcustom clockodo-keymap-prefix "C-c C-#"
  "The prefix for the clockodo mode key bindings."
  :type 'string
  :group 'clockodo)

(defvar clockodo-debug nil
  "Shows more information in the message buffer.")

(defvar clockodo-user-id nil
  "The user id needed for most requests.")

(defun clockodo--key (key)
  "Convert a key into a prefixed one.

KEY The key which should be prefixed."
  (kbd (concat clockodo-keymap-prefix " " key)))

(defun clockodo--with-face (str &rest face-plist)
  "Enclose a string with a face list.

STR The string which gets a face.
FACE-PLIST The list of faces."
  (propertize str 'face face-plist))

This is just a small excerpt from all definitions used for the clockodo package. The last two function are useful helpers. The clockodo--key function provides an easy way to bind keys to the package keymap. The clockodo--with-face function provides an easy way to color some part of output using the emacs faces.

Now, the skeleton definition for a minor mode with the define-minor-mode macro.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

;;;###autoload
(define-minor-mode clockodo-mode
  "Define the mode for interacting with clockodo."
  :init-value nil
  :lighter " clockodo"
  :group 'clockodo
  :global t
  :keymap
  (list (cons (clockodo--key "s" #'clockodo-start-clock)))

  (if clockodo-mode
      (message "clockodo mode enabled")
    (message "clockodo mode disabled")))

The first function defines the minor mode clockodo-mode which is:

  • :init-value not enabled by default
  • :lighter as the name clockodo on the modeline
  • :group uses the custimization group clockodo
  • :global the mode is a global mode
  • :keymap the modes keymap

Additionally, we could provide some function body to do some setup or cleanup. This macro also creates a hook which is run if the mode is turned on or off.

Authentication

Now, the skeleton is ready. To interact with the clockodo api credentails are needed. Emacs provides the auth-source package to deal with authentication and credentials.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

(defun clockodo--get-credentials ()
  "Return the stored api credentials for clockodo.

The user is asked for credentials if none exists for the api url."
  (let* ((auth-source-creation-prompts
          '((api-user . "Clockodo user: ")
            (api-token . "Clockodo api token: ")))
         (api-creds (nth 0 (auth-source-search :max 1
                                               :host clockodo-api-url
                                               :require '(:user :secret)
                                               :create t))))
    (when api-creds
      (list (plist-get api-creds :user)
            (let ((api-pass (plist-get api-creds :secret)))
              (if (functionp api-pass)
                  (funcall api-pass)
                api-pass))
            (plist-get api-creds :save-function)))))

(defun clockodo--initialize (api-creds)
  "A thin wrapper which set user variables for the next requests.

API-CREDS The credentials for the clockodo api."
  (when (or (null clockodo-user-id)
           (null clockodo-default-service-id)
           (null clockodo-default-customer-id))
    (let* ((response (request-response-data
                      (clockodo--get-user (nth 0 api-creds) (nth 1 api-creds)))))
      (let-alist response
        (setq clockodo-user-id .user.id
              clockodo-default-service-id .company.default_services_id
              clockodo-default-customer-id .company.default_customers_id)))))

(defun clockodo-get-credentials ()
  "A wrapper which takes long-time storing of credentials into account.

It knocks the clockodo api to test if the credentials are valid before storing them."
  (let ((api-credentials (clockodo--get-credentials)))
    (when clockodo-store-api-credential
      (when (functionp (nth 2 api-credentials))
        (let ((return-code (request-response-status-code
                            (clockodo--get-user
                             (nth 0 api-credentials)
                             (nth 1 api-credentials)))))
          (when (eq return-code 200)
            (funcall (nth 2 api-credentials))))))
    (clockodo--initialize api-credentials)
    api-credentials))

The function clockodo--get-credentials creates ask the user for credentials and splits them into the username, password and a save function to store the credentials over a single emacs session. The clockodo--initialize function pokes the clockodo api and stores some basic informations about the user. The clockodo-get-credentials ties them together. First it asks for the credentials either from the user or the auth store, than tests if we get valid informations from the clockodo api and if so stores the credential for longe time use. The user can toggle this behaviour through the clockodo-store-api-credential variable.

Dealing with http requests and responses

To interact with the clockodo api I used the request package which is easy to use and well documented. First set the header for every requests taken.

1
2
3
4
5
6
7
8

(defun clockodo--create-header (user token)
  "This create the header for requests against the clockodo api.

USER The username used to authenticate the request.
TOKEN The token used to authenticate the request."
  `(("X-ClockodoApiUser" . ,user)
        ("X-ClockodoApiKey" . ,token)
        ("X-Clockodo-External-Application" . ,(concat "clockodo-el;" user))))

Afterwards, the basic requests are modeled as a function which gets the credentials and the partial URL for the requests. As an example, I modeled the get requests like this.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

(defun clockodo--get-request(user token url-part)
  "This function abstracts a simple get request to the clockodo api.

The result is a parsed json object.
USER The username used for the api request.
TOKEN The clockodo api token for the user.
URL-PART The full api part for the get request."
  (when clockodo-debug
    (message (concat "API-URL: " clockodo-api-url url-part)))
  (let ((request-header (clockodo--create-header user token)))
    (request (concat clockodo-api-url url-part)
      :sync t
      :headers request-header
      :parser 'json-read
      :error (cl-function (lambda (&rest args &key error-thrown &allow-other-keys)
                            (message "Got error: %S" error-thrown))))))

Now, every get request is nothing more then a couple of lines.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

(defun clockodo--get-clock (user token)
  "Request the current state of the clockodo clock service.

USER The username used for the api request
TOKEN The clockodo api token for the user"
  (clockodo--get-request user token "/v2/clock"))

(defun clockodo--get-all-services (user token)
  "Request the list of services defined within the company.

USER The username used for the api request.
TOKEN The clockodo api token for the user."
  (clockodo--get-request user token "/services"))

Maybe there is a better way to deal with the repeating user and token variables (Let me know).

Special buffers

Besides using the clock, I wanted to show reports generated from the clockodo api. For this purpose I use a buffer which is set into special mode.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

(defun clockodo--build-report-buffer (name header body)
"Generate a new report buffer and insert the return of body.

NAME The name of the report buffer.
HEADER A two element list with a header-line and a page heading.
BODY A function that fills the buffer."
(let* ((buffer (get-buffer-create (format "*clockodo-%s*" name)))
       (inhibit-read-only t))
  (if (get-buffer-window buffer)
      (pop-to-buffer-same-window buffer)
    (switch-to-buffer-other-window buffer))
  (with-current-buffer buffer
    (erase-buffer)
    (special-mode)
    (setq header-line-format (car header))
    (insert (cdr header))
    (funcall body))))

This function creates a new buffer prefixed with clockodo or takes an existing one. It is set into read-only mode and the first part of the header is set as header-line-format. The second part is a block of general informations about the api user. As last part a function is executed which renders the report body.

request returns the data as alist thus the following function is handy.

1
2

(let-alist (request-response-data response)
  (do-something-with .entries))

This binds the alist properties as variables of the form .variable.subvariable.

Hints and Takeaways

This post should give me and others some hints when developing a new emacs package or dealing with some api. It is not a real template but more a set of hints since the documentation for some packages or emacs parts are not well-suited vor beginners.

Lastly, another short list of general hints:

  • Use melpazoid as Github action
  • Use package-lint to get the documentation correctly
  • Use internal function, than cl-lib and if nothing helps external libs
  • Keep your functions small and composable
  • Take your time reading documentation and code
  • Reading code of others and other packages helps to understand certain patterns of elisp
  • Use when and unless instead of open if forms
  • Provide sensible default keybindings
  • Prefer functions with a toggle effect for keybindings
  • Use defvar and defcustom to allow easy modifications and prevent void variable assignments
  • Use (or nullable-var default) to set defaults

Thanks for the attentation and leave me a comment.

Tags: #emacs #packages